Hack into the LIFE ;-)

Janus with jslint Vim plugin

2012-02-15T23:54:00.000-08:00

Nowadays, I write quite a bit of JavaScript and uses JSLint command line tool quite a lot. A co-worker recommends a nice Vim plugin that validates your JavaScript code when you are editing and when you save it.

While I was trying to install it for Janus (a Macvim clone that I uses), I ran into two issues:

1. By default, when you run rake inside the git cloned directory, the plugin gets installed into ~/.vim. For Janus, user customized plugins should go to ~/.janus, and Janus will load them automatically. For details, please check out the Customization section of the Janus documents.

So, to get around:

- create a jslint directory in ~/.janus
- edit vim plugin's Rake file, replace line 38 File.expand_path("~/.vim") with File.expand_path("~/.janus/jslint"), you get the idea.
- run rake from inside vim plugin directory and it should install into the correct Janus directory

2. The second issue is that I keep getting warning when I start trying out the plugin, something like "s:cmd" not defined. Did some poking around and it seems like ftplugin/javascript/jslint.vim is trying to find a JavaScript interpreter (line 62 to 75 for *UNIX systems) and somehow failed.

I am on Snow Leopard, which comes with an acceptable interpreter "jsc" at /System/Library/Frameworks/JavaScriptCore.framework/Resources/jsc, so not really sure why it did not work. I ended up just install Node.JS (for Mac users, highly recommend using Homebrew, just follow instructions here) and added node to my PATH. And it takes care of these errors.

Now, enjoy the jslint plugin and be a good JS developer ;-)

P.S. For people not happy with Crockford's personal styles (some styles don't make sense to me either), you can update options in ~/.jslintrc (see examples on jslint.vim site). Or simply use jshint vim plugin instead. JSHint is a more relaxed and reasonable fork of JSLint.

Sorry for going off the topic, but it is quite funny to read about why the original developer forked JSLint, especially entertaining are the comments. For example, Crockford's response to JSHint:

When asked for his "feelings on JSHint" Crockford replied "There are many stupid people in this world, and now there is a tool for them."

Lua Development Tools Koneki, IDE based on Eclipse

2012-01-28T13:04:00.000-08:00

Just got a comment from developers of Lua IDE Koneki on my previous post about setting up Eclipse-based Lua IDE. So, I gave it a try and it is quite neat, I really hope the developers can keep it moving and make it a great default IDE for Lua developers ;-) As of now, I feel it is not yet conveniently as useful as LuaEclipse I posted about a while back. But the project is under active development and it will become better every day, I am sure.

Here is a quick rundown what I have tried so far:

1. Installation: they offer both standalone and update site, I just installed through update site following their instructions. I am on Eclipse 3.7.1, btw. I also got lua and luarocks installed through Homebrew (by default they are installed in /usr/local/bin).

2. Create a new Lua project: create a new Lua project, add source files under src, similar to the typical Java projects.

3. Ready to run? I feel stuck at this step at first since there is no launch configuration that allows me to configure the local Lua environment. After running through several threads, it seems Koneki does not support launch configuration yet. It only supports "remote debug launch configuration".

But the developer also provides this workaround using "External Tools" and I got it to run and show the results in Console View. This is the sample configuration (you need to select the main.lua before run this external tool configuration). For the meaning of Eclipse variables like ${workspace_loc}, ${resource_loc}, see Eclipse external tools documentation.

4. Debugging. LDT supports remote debugging via DBGP, you can follow the LDT user guide to set it up.

Many thanks to the LDT developers for this nice IDE. I wish the launch configuration can be added so beginners like me can get started with Lua and LDT quickly. Also, I wish the configuration of debugging and remote debugging can be integrated into LDT.

Unbounded function wrappers

2012-01-23T09:38:00.000-08:00

I was reading "JavaScript Garden" (highly recommend for JS beginner/intermediate) and could not understand the concept of the "fast, unbound wrappers" for functions. See Function arguments for the example.

function Foo() {}

Foo.prototype.method = function(a, b, c) {
    console.log(this, a, b, c);
};

// Create an unbound version of "method" 
// It takes the parameters: this, arg1, arg2...argN
Foo.method = function() {

    // Result: Foo.prototype.method.call(this, arg1, arg2... argN)
    Function.call.apply(Foo.prototype.method, arguments);
};

Luckily I found this Stackoverflow post that explains the idea behind it. It took me a while to wrap my head around Function.call.apply ;-)

So, the basic idea is that we have a function defined in a class, but we want to use it without binding to specific object, maybe the concept of static method in Java? So, instead of creating an object and invoke the function on the object, we define the function as a property of the class, not on its prototype.

useful vim tips

2011-12-28T11:51:00.000-08:00

Vim is definitely a text editor you need to learn progressively because it has an interesting learning curve. If you have not read "Seven Habits of Effective Text Editing" from Bram Moolenaar (main author of Vim), please do that first. There is also an updated version of this article that can be found here. Basically, there are three steps to improve your Vim skills through daily work:

find repeated actions that are not efficient or quick enough
find a quicker way (search online, ask around, read documentation and tutorials, etc.)
practice the new tricks and make it a habbit

So, basically I am trying to take notes about some new tricks I learned over the past week. Hopefully, I will do the iterations and find even more efficient ways to do these in the future.

Quickly move around on a line.

0 (zero) moves the cursor to the beginning of a line (white spaces are considered as characters)
$ moves the cursor to the end of a line (white space are considered)
^ moves the cursor to the beginning of a line (white space ignored)
g_ moves to cursor to the end of a line (white space ignored)

For example (using + to stand for space here for illustration purpose):

++a line++
- -    - - 
0 ^    $ g_

w, W move the cursor forward to the beginning of a word, the difference between the two is W uses white space as separator. So, for me w is quite useful in code, W is useful in the textual sentences.

e, E move the cursor forward to the end of a word, same difference as in w and W.

b, B move the cursor backward to the beginning of a word, difference is again the separator.

ge, gE move the cursor backward to the end of a word.

*, # to quickly move forward and backward to the same word under the cursor. This is faster than search using / and press N and n.

o adds a new line under current line and start insert.
O adds a new line above current line and start insert.
s deletes the character under cursor and start insert.
S deletes the current line and start insert.
D deletes from cursor to the end of line.
C deletes from the cursor to the end of line and start insert.

Quickly move around in code.

% matches the parenthesis, bracket and curly brace. For example, put the cursor on a closing curly brace and % takes the cursor to the matching opening brace.

[{ and ]} to match the opening and closing curly braces. Place the cursor inside a code block, using these two to quickly move to the beginning and end of the closes curly braces pair.

gd jumps to the local variable definition when the cursor is on a variable name.
gD jumps to the global variable definition.

Use . (dot) command to repeat the last change in normal mode. See Vim Twiki "repeat last change".

You can also do N. to repeat the change N times. For example, dd command delete the current line, use . to delete another line, use 5. to delete additional 5 lines.

There is also a similar repeat trick @: for last command made on command line. For example, on the command line, we do :!ls to list files in the current directory. Then, we can do @: to repeat it again.

Copy/cut and paste text in visual mode. See Vim Twiki "copy and paste using visual selection".

put cursor to the beginning of the desired text
enter visual mode: use v to begin character-based selection, V to begin line based selection, ctrl-v to begin vertical block based selection (move the cursor will select the whole columns of text)
put cursor to the end of the desired text
use d to cut, y to copy
put the cursor to the paste position
use p to paste after the desired position, P to paste before the desired position

On the Vim twiki, there is also the tip to overwrite the same amount of text using the copied/cut text using Nvp, NVp, N^Vp depending on the selection mode (character, line or block).

Disable auto-indentation, manual indentation.

Sometimes, I had one annoying problem when I copy and paste formatted source code from another application - Vim will reformat the content and mess up with the indentations. To prevent Vim from retabbing, use :set paste. After you are done, use :set nopaste to diable paste. You can also add a key binding using set pastetoggle in your .vimrc file. See Vim Twiki "Toggle auto-indenting for code paste" for more details.

Instead, you want manual indentation. In normal mode, == indents the current line. In line-based visual mode (V or shift-v), select several lines and = indents them all.

Jump in the change list, moving back and forth in changed positions in the current file. One common task is to jump back and forth between edit positions. See Vim Twiki "list changes to the current file".

`. jumps to the last edit position
g; jumps back in the change list
g, jumps forward in the change list
:changes to show the change list

Interestingly, there is another (probably less useful) jump list that records the default last 100 positions a user makes a jump. A jump is one of search, substitute and mark. See Vim Twiki "jumping to previously visited locations".

ctrl-O to jump back in jump list
ctrl-I to jump forward in jump list
:jumps to show the jump list

Auto expansion, adding comments, etc.

One common case when coding is to add comments. So, how do you add something like these lines:

///////////////////////////
/*------------------*/
/*************** <multiple lines of text> *************/

Ni/<esc> inserts N / (slash), quite useful for N repeating characters.

Another way is to use auto expansion. For example, you can add these into your .vimrc:

:ab #b /************************************************
:ab #e ************************************************/
:ab #l /*----------------------------------------------*/

Then, in the edit mode, you can type #b<enter> to automatically insert /*******.

Text objects.

Last, I found out about Vim text objects, it is so powerful and convenient that saved me a lot of time. For details, please read this great post "Vim Text Objects: The Definitive Guide" and Vim documentation on text objects.

Note that text object is more convenient than motion when you want to delete a chunk of text, you don't need to move cursor to particular position. For example, the difference between dw and diw is that when you want to use dw to delete a word, you must move the cursor to the beginning of that word first. For diw, you can place the cursor anywhere inside that word.

For example, I want to refactor code that has a string value hard coded to use a variable:

public class Hello {

    public static void main(String[] args) {
        System.out.println("hello, world");
        showMe();
    }

    public static void showMe() {
        System.out.println("hello, world");
    }
}

// refectored
public class Hello {

    public final static String hi = "hello, world";
    public static void main(String[] args) {
        System.out.println(hi);
        showMe();
    }

    public static void showMe() {
        System.out.println(hi);
    }
}

So, I can add a variable hi="hello world", then replace the string value "hello, world" on line 4 and line 9 using text object:

move to line 4, a position before the string "hello, world", in normal mode, type ca", this will delete the string value including the quotes and change to insert mode.
type the new variable name hi
move to line 9, a position before the same string
type . to repeat the replacement
done!

One extra tip. Using text objects, you can delete/update the content inside the double quotes, but how do you delete the double quotes only? You can do this:

move cursor to one of the double quotes
use % to jump to the matching double quotes, delete it using x
use Ctrl-o or two back-ticks (``) to jump back to the first double quote, delete it using x
done!

Web application security readings

2011-12-18T01:30:00.000-08:00

Articles:

Mozilla Web Application Security (quick reading, 10 min)
Browser Security Handbook (quick reading, 30 min)
OWASP Cheat Sheets (quite comprehensive reading, several hours)
How Browsers Work

Slides:

Cross Site Scripting (XSS) Slides from Mozilla Security Learning Center (38 pages, 10 min)
Web Application Security: Hands On Slides from Michael Coates

Specifications:

CSP (Content Security Policy): a W3C draft spec that defines fine grained security policies for resource loading to mitigate the risk of injection attacks such as XSS. The policy per resource representation is defined in new header is Content-Security-Policy. Firefox and Chrome is experimenting in headers X-Content-Security-Policy and X-WebKit-CSP respectively. The spec also defines a report-only mode without browser enforcing the policy but sending violation reports to the server, which is helpful for gradually enforcing the right policy.

Books:

so many cookies

2011-11-22T00:42:00.001-08:00

I was reading upon HTTP cookies and it is actually quite confusing. Because there are many different cookie specs and browsers not necessarily support all of them.

Original Netscape Cookie Spec: http://curl.haxx.se/rfc/cookie_spec.html

Very simple and straightforward format:

Set-Cookie: NAME=VALUE; expires=DATE;
path=PATH; domain=DOMAIN_NAME; secure

RFC 2109 (based on Netscape spec with minor tweaks): http://tools.ietf.org/html/rfc2109

The syntax for the Set-Cookie response header is

   set-cookie      =       "Set-Cookie:" cookies
   cookies         =       1#cookie
   cookie          =       NAME "=" VALUE *(";" cookie-av)
   NAME            =       attr
   VALUE           =       value
   cookie-av       =       "Comment" "=" value
                   |       "Domain" "=" value
                   |       "Max-Age" "=" value
                   |       "Path" "=" value
                   |       "Secure"
                   |       "Version" "=" 1*DIGIT

RFC 2965, obsoletes RFC 2109: http://tools.ietf.org/html/rfc2965

It adds more syntax items. It is very interesting to read 9.1 section about the compatibility with existing cookie implementation. Basically, it should overwrite an existing cookie value set by Set-Cookie if domain and path both matches.

The syntax for the Set-Cookie2 response
   header is

   set-cookie      =       "Set-Cookie2:" cookies
   cookies         =       1#cookie
   cookie          =       NAME "=" VALUE *(";" set-cookie-av)
   NAME            =       attr
   VALUE           =       value
   set-cookie-av   =       "Comment" "=" value
                   |       "CommentURL" "=" <"> http_URL <">
                   |       "Discard"
                   |       "Domain" "=" value
                   |       "Max-Age" "=" value
                   |       "Path" "=" value
                   |       "Port" [ "=" <"> portlist <"> ]
                   |       "Secure"
                   |       "Version" "=" 1*DIGIT
   portlist        =       1#portnum
   portnum         =       1*DIGIT

RFC 6265, obsoletes RFC 2965: http://tools.ietf.org/html/rfc6265

It admits the same confusion that I had, and deprecates the Cookie2 and Set-Cookie2 headers introduced in RFC 2965!

"Prior to this document, there were at least three descriptions of

   cookies: the so-called "Netscape cookie specification" [Netscape],
   RFC 2109 [RFC2109], and RFC 2965 [RFC2965].  However, none of these
   documents describe how the Cookie and Set-Cookie headers are actually
   used on the Internet (see [Kri2001] for historical context).  In
   relation to previous IETF specifications of HTTP state management
   mechanisms, this document requests the following actions:

   1.  Change the status of [RFC2109] to Historic (it has already been
       obsoleted by [RFC2965]).

   2.  Change the status of [RFC2965] to Historic.

   3.  Indicate that [RFC2965] has been obsoleted by this document.

   In particular, in moving RFC 2965 to Historic and obsoleting it, this
   document deprecates the use of the Cookie2 and Set-Cookie2 header
   fields."

The actual format is getting updated as well. Notice that now both Expires and Max-Age are supported with Max-Age overriding Expires if both exist. If not, the cookie is kept by user agent till the current session ends.

 set-cookie-header = "Set-Cookie:" SP set-cookie-string
 set-cookie-string = cookie-pair *( ";" SP cookie-av )
 cookie-pair       = cookie-name "=" cookie-value
 cookie-name       = token
 cookie-value      = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
 cookie-octet      = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
                       ; US-ASCII characters excluding CTLs,
                       ; whitespace DQUOTE, comma, semicolon,
                       ; and backslash
 token             = <token, defined in [RFC2616], Section 2.2>

 cookie-av         = expires-av / max-age-av / domain-av /
                     path-av / secure-av / httponly-av /
                     extension-av
 expires-av        = "Expires=" sane-cookie-date
 sane-cookie-date  = <rfc1123-date, defined in [RFC2616], Section 3.3.1>
 max-age-av        = "Max-Age=" non-zero-digit *DIGIT
                       ; In practice, both expires-av and max-age-av
                       ; are limited to dates representable by the
                       ; user agent.
 non-zero-digit    = %x31-39
                       ; digits 1 through 9
 domain-av         = "Domain=" domain-value
 domain-value      = <subdomain>
                       ; defined in [RFC1034], Section 3.5, as
                       ; enhanced by [RFC1123], Section 2.1
 path-av           = "Path=" path-value
 path-value        = <any CHAR except CTLs or ";">
 secure-av         = "Secure"
 httponly-av       = "HttpOnly"
 extension-av      = <any CHAR except CTLs or ";">

To conclude, here is a paper providing a good overview and discussion about the cookie: "HTTP Cookie: Standards, Privacy and Politics".

SPAM: Please get a new EIN

2011-11-14T12:09:00.001-08:00

Another spam I got today, it looks fairly serious ;-), maybe the "Dear Sirs" part sold itself. Anyways, here it is for fun:

Title: Please get a new EIN
From: Clementina Evans
To the ATTENTION of: Accounting Department
Dear Sirs,
please be informed that you need to get a new EIN in connection
with the probability of fraudulent actions on your behalf. We
are sorry for the inconveniences that might be caused to you
by this measure. Please use the following link for instructions:
http://onesourceprocess.com/7wmzwt/index.html

Yours sincerely,
Clementina Evans
Office of Professional Responsibility
Internal Revenue Service

SPAM: New EIN

2011-11-08T13:49:00.000-08:00

Just found it interesting to publish some lastest spams I got each day here, hopefully this would be helpful to non-tech people. Please don't try the links inside ;-)

Title: New EIN
From: Robert Glass
Content:
To the ATTENTION of: Accounting Department
Dear Sirs,
we would like to notify you that you are required to receive
a new EIN in order to prevent fraud. We are sorry for any troubles
that might be caused to you by this measure. Please use the following
link for instructions:
http://web.knauf.es/5nse63/index.html

Yours sincerely,
Robert Glass
Office of Professional Responsibility
Internal Revenue Service

Unwanted Comcast features

2011-10-09T18:00:00.000-07:00

After multiple occurrences of AT&T dropping connections over the past months, I finally decided to switch to Comcast (yes, I have heard many terrible stories about their billing department, sales fraud, etc.). Plus, I got a really good deal from my colleague ($19 per month for the first year, $35 per month for the second year for their performance package, 15MB upload and 5MB download.

Based on advice, I did not choose to rent their modem and bought my own from Amazon: Motorola SB6121 and it worked great. There is an older model SB6120 that is a bit cheaper, but my friend had problems. Please check with the technician and confirm that those models work in your area.

Today, they came in and installed it for me. The overall process worked fine except for three complaints:

1. They kept calling for confirmation. In the past week, they called so many times for confirmation, 3 days before, yesterday, and this morning. I understand that some people might be forgetting things very quickly, but I am not that type of people. Plus, I have confirmed at the first call already and they are still calling as early as 7am in the morning! Since I did not get this morning's phone call, they left a message and said I would have to reschedule it, WTF. So, I ended up calling them again so they can send the technician in without rescheduling.

2. Some technicians are un-professional. I am quite busy with work and life, so my appointment is 8-10am, earliest I could get so I don't miss the work. For the first appointment, the technician waited outside for 1 hour before he came in (I saw his van and went out to tell him I am ready, but he kept sitting in his var for another half an hour). When he came in, I showed him where I want the setup and the pole to pull the signal from. He shrugged his shoulder and said "it will take at least 4-5 hours". I said I thought it'd be only 1-2 hours and I had no choice but to reschedule. Somehow from the look on his face, I felt that he did not want to do installation (bad mood, lazy, etc.) and I am fairly pissed. If you don't want to work, then don't come to waste your time and my time.

Today, another technician came in and he is much better. He finished the whole job in 1 hr and told me the previous technician "must be lazy".

3. The installation process forces you to install the desktop installer to activate. After the hardware is set up, I hooked up my wireless router and was directed to an activation page, which asked me to create a new Comcast user account. But the last step is to install Comcast desktop installer and there seems to be no way around according to the technician. (Update: according to my colleague, you don't need to install it, just restart the modem, shutdown router, connect router with modem, start the router. But I did not try this personally, not sure if it would work.)

The installer file is "ComcastDesktopInstaller_MMF3.pkg" in my case. I uploaded it online if you are interested. Luckily I unchecked "install browser toolbar" option during installation. People are suffering from this toolbar since it hijacks browser preferences and homepage. If you are the victim, please follow instructions on this page to get your previous setting restored.

After the network is up and running, I want to uninstall it but could not find much info. I did my best, and my conclusion is it might not do as much harm. It executed some Ruby scripts and installed some crap on the desktop, which you can delete easily.

Cleanup actions in summary:

1. Delete the two website shortcuts installed on my desktop.
2. Delete ~/.cim_install_log.
3. I am not familiar with Ruby, but the scripts seems to just create Xfinity bookmarks for Safari, Firefox and Chrome. For details in the script, see the gist below.
4. Besides those, I did not seem to find other suspicious malware or unwanted junks.

Detailed checkups for the Comcast app:

1. Did a grep for "comcast" and found a log file in home directory: ~/.cim_install_log. It contains all the output during installation process.

2. Used AppCleaner to check installed applications, does not find any malicious app installed from Comcast installer.

3. Used pkgutil command to search for anything related to comcast in package receipt database. See pkgutil manpage for details how to use it. I could not find any new applications installed from this installer pkg.

4. Used Pacifist to check the content of the pkg file. Did not see much suspicious content except for several Ruby and shell scripts to run during installation.

Don't install Firefox Toolbar

By default, these website shortcuts are install on your desktop, remove them

Content of ComcastDesktopInstaller_MMF3.pkg

Reference:

1. How to remove Comcast/Xfinity home page from Firefox (Mac)

2. Comcast installing something on my Mac without permission

3. How to uninstall Mac OSX pkg packages

Big list of things to get started with frontend development

2011-09-10T00:59:00.000-07:00

Quickly ran though "Getting Good with JavaScript", this is a very short and basic introduction to JavaScript for someone new to the language. Deep and difficult concepts are avoided or explained in an easy to understand way for the reader to explore further by him/herself.

I found these two resources in the "Further Study" section very helpful and interesting, both are from Rey Bango's blog:

1. What to read to get up to speed in JavaScript
2. The big list of JavaScript, CSS, and HTML development tools, libraries, projects and books

display Apple keys in HTML

2011-08-31T23:59:00.000-07:00

Just saw an interesting kbd tag supported on Stackoverflow and started wondering how to display Apple keys like command, shift, control, option, etc. on the Web page.

And it actually is very straightforward using HTML character entities. The concept is well explained in this Wikipedia article. There are two forms: numeric character reference and character entity reference:

numeric character reference: either in decimal form (&#dddd;) or in hexadecimal form (&#xhhhh;), e.g. "less than" character can be expressed as < or <

character entity reference: in the form of &name; the same "less than" character can be expressed as <

Now, it is very clear how to display these special Apple keys with the help of entity tables such as this unicode table. Somehow I could not find the corresponding character entity reference table for the same keys.

These are the most common Apple keys you might use:

⌘ Command: ⌘
⇧ Shift: ⇧
⌃ Control: ⌃
⌥ Option: ⌥
⎋ Esc: ⎋

If you use TextMate, there is a convenient HTML bundle feature that helps to insert these special characters in an HTML document. Create a new HTML document, then goto Bundles menu => HTML => Entities => Pick the character you want. You can also use the autocomplete, e.g. type "command" and press tab.

migrate from TextMate to Vim/MacVim

2011-08-31T10:42:00.000-07:00

Choosing text editor is like choosing your religion. One popular/natural choice on Mac is TextMate. However, since I do lots of work on the server/dev linux box directly, I had to switch back and forth from Vim. Probably it is time to migrate to use Vim on Mac as well, especially given the fact that there are nice distro made for Mac specifically: MacVim and a customized MacVim packaged with many useful plugins called Janus.

Here are two good testimony from other migrants:

Here are several useful learning resources:

Vim Commands Cheat Sheet (for good reference, there are multiple other similar cheat sheets)
vimtutor (simply open your terminal and type vimtutor)
A Byte of Vim
Vim Recipes
Essential Vim Plugins (plugins are very helpful to create your effective workflow)

Plugins shipped with Janus:

Spend 5 minutes to go through the brief introduction for all the plugins that ships with Janus. Janus also includes a handy way to allow you to create/edit ~/.janus.rake file to add more plugins and you can run rake inside ~/.vim to update these plugins.

Sample ~/.janus.rake

vim_plugin_task "zencoding", "git://github.com/mattn/zencoding-vim.git"
vim_plugin_task "minibufexpl", "git://github.com/fholgado/minibufexpl.vim.git"

NERDTree: provides a file browser, use \n to show/hide.

Command-T: provides an efficient way to search for files in current working directory. Use ⌘+T to show/hide.

snipMate: provides the same Textmate Snippet feature. As you do in TextMate, you can simply type snippet keywords and press tab to expand to code snippets and use tab to jump through various locations in the snippets. You can find all snippet definitions for various languages in ~/.vim/snippets.

ctags: TagList plugin is shipped with Janus. Use :Tlist to show/hide it. But I like Tagbar better than TagList (see below for installation instructions).

Other handy plugins I installed:

Syntastic: syntax checker, you can also go to the jslint route. This Stackoverflow post covers the setup in details.

Tagbar: displays tags of the current file in a sidebar, sorta like TagList, but more comprehensive and orders tags under correct scope/context.

doctorjs: previously known as jstags, provides a set of static analysis tools for JavaScript. Combined with Tagbar, it provides a more detailed code structure navigation than TagList. See this Stackoverflow post for instructions how to set it up. Note that you need to install NodeJS as a prerequisite.

There are two issues to get doctorjs set up properly that is not covered in that Stackoverflow post:

It seems that installing doctorjs needs sudo (sudo make install inside doctorjs git clone directory) to get around permission issue.
I also got errors about underscore module and ctags NodeJS module cannot be found. They are located at /usr/local/lib/jsctags/ after doctorjs is installed. But node cannot locate them somehow. To fix it, I installed underscore using npm (npm install underscore -g), then I updated my NODE_PATH environment variable to include /usr/local/lib/jsctags/.

file system based on a single file and loop device

2011-08-05T23:15:00.000-07:00

Worked on a task to test performance of read/write on various file system options on my RHEL5 box:

native ext3
virtual ext3 based on a zero-filled image file
virtual ext3 based on a sparse image file
virtual ext2 based on a sparse image file

The virtual file system is basically based on a big file connected as a loop device. This tutorial about setting up disk quota based on virtual file system has more details. For example, this is the basic steps to set up a virtual ext3 file system based on a sparse file:

dd if=/dev/null of=/tmp/vfs/sparse.image bs=1M seek=100

mkfs -t ext3 -q /tmp/vfs/sparse.image -F

mkdir /tmp/quota

sudo mount -t ext3 -o rw,loop /tmp/vfs/full.image /tmp/quota

The experiment is quite preliminary: basically, I copy over files in different sizes (1, 5, 10, 25, 50MB) into the mounted file system directory or the native file system directory to test write latency, then read the copied files to test read latency. The read/write latency is averaged over 10 iterations.

A couple of quick points from the experiments:

read latency is similar for all four cases.
write latency for native ext3 is around half of the virtual ext3 based either on a sparse file or a zero-filled file.
write latency for native ext3 is roughly the same as virtual ext2, maybe because ext2 gets rid of the journaling overhead.
creating a virtual file system based on a sparse file has many advantages over a zero-filled file: the creation time is way much faster 6-8 order of magnitude for the file sizes we tried, the bigger the file, the bigger the difference; the sparse file does not occupy physical disk space and grows when content is actually written to disc.
virtual ext3 has spikes for write, sometimes as big as 80x the average write latency, not exactly what caused it.

This is final the performance graph for comparison. I am using the average number here, notice that the high average latency for write on virtual ext3 is caused by the spikes mentioned above.

After removing the spikes, using virtual ext3 based on a sparse file or zero-filled file creates around 2X the latency for native ext2 or ext2 based on a sparse file, as show in the second graph:

Multi-language support for a website

2011-07-22T00:39:00.000-07:00

I am not a frontend developer, just stumbled upon this problem helping out my friend to set up a website that is supposed to serve multi-language content based on the language setting of the user. This is actually an interesting problem (maybe already solved and I did not know).

There are several different solutions that I found through after doing some study:

- Server side detection based on Accept-Language request header.

For example, apache mod_negotiation allows you to define a set of html pages in different languages (index.html.en, index.html.zh, etc.) and based on the user preferences from Accept-Language header, apache picks the right index.html language page to return.

You can also use mod_rewrite to perform the similar trick. Just set up the rewrite condition based on the value of the AL header and map to the correct language.

- Client side detection and action using JavaScript and navigator object.

In IE, navigator.userLanguage returns the operating system region and language setting, in Firefox, Opera, Safari and Chrome, navigator.language returns the browser application language setting. So, we can use them to get the user language and decide what language content to return.

For example, this code snippet (borrowed from an example on jsfiddle) defines multi-language content on the same page and use JavaScript to show or hide it based on the detected language.

The drawback of this approach is that a large part of the content is not needed and invisible. It is a waste of time to download.

Another approach is to use JavaScript to redirect to individual language content pages using window.location.href="redirected_page_url" or window.location.replace("redirected_page_url"). See Location object for more details on how to use them. The downside is this approach may not be search engine friendly (spiders may not be able to follow redirection).

One last note how to test multi-lang website. I tried in Firefox, simple change the language ranking in Preferences->Content Tab->Language Choose button and FF will start sending different Accept-Language headers based on the ranking. You can also play with the operating system language settings.

node.js testing frameworks

2011-06-13T17:56:00.000-07:00

There are so many testing frameworks for node.js, start to evaluate some most mentioned/used ones. Criteria that I am looking for:

easy to understand concept and constructs
nice support for async calls and flow control
easy integration with CI tools such as Hudson
code coverage report
mocks and stubs

Vows+APIeasy
Expresso+node-jscoverage: really like its code coverage feature
YUITest: somehow had issues writing async operations in setUp and tearDown, also had issues using flow control libraries such as async inside each test
Jasimine: very clean and easy to understand concept and constructs
Nodeunit: a detailed tutorial from the author
node-qunit+jscoverage, qunit is used for JQuery testing and has fairly good async support, here is a fairly detailed article for async testing with qunit
testosterone+gently
Sinon.JS

Among those, I like Jasimine most, it is easy to understand and has rich feature set. The only feature missing is lack of code coverage report. jasimine-reporters supports output of JUnit format unit test results. jasimine-node brings Jasimine to node.js. You can check out "Testing Backbone applications with Jasimine and Sinon" for a practical example.

dealing with multiple nesting callbacks in asynchronous code

2011-06-05T00:01:00.000-07:00

If you are annoyed that you have to write multiple nesting asynchronous callback functions when you work with Node.js, you are not alone. Getting used to the asynchronous programming style and always keep in mind everything runs in the event loop is a bit tricky at the beginning.

There are many libraries and articles that addresses this nesting callback issue. This should help you to avoid those ugly pyramid shaped code now ;-)

InfoQ: how to survive asynchronous programming in JavaScript (includes a list of libraries with comparisons and Q&A from their developers)

Stackoverflow post about how to deal with the nesting callbacks (also includes a list of libraries)

Isaacs's presentation on the topic and his library Slide (he is the author of npm)

Tim Caswell (creationix)'s series on control flow in Node (II and III can be found on the right column, other articles by the same author)

CommonJS Promises is another way trying to address this issue by defining a standard interface for interacting with the result object of asynchronous actions. Roughly speaking, a promise object is a placeholder for the value for the asynchronous action. And you can treat is as normal object, assign values to another variable, pass it around, etc. Right now, FuturesJS implements this and it is not solely for server-side JS. dojo.Deferred is another similar effort. This SitePen blog post has some easier to understand coverage about the promises.

StratifiedJS extends the JavaScript language to achieve similar goals of allowing asynchronous control flow to be expressed in a synchronous way. Very neat work indeed. This is a short overview presentation about it.

get rid of annoying ._filename working with TextMate on a mounted NFS

2011-05-13T17:44:00.000-07:00

Just had this annoying experience of working on an NFS mounted file system with TextMate. Basically, it tries to store metadata such as cursor position, etc. into metadata files. Since these mounted file system does not support extended attributes, so OSX instead writes to ._filename, which is really annoying since the directory is under version control and you see many of these metadata files showing up.

After poking around, I figured out how to turn this metadata feature off, yes, I lost a bit convenience feature, but no more cleanup of these ._filename anymore!

defaults write com.macromates.textmate OakDocumentDisableFSMetaData 1

References: TextMate Expert Preferences

Hello, Cloud Foundry running node.js

2011-04-23T23:45:00.000-07:00

Contrary to most people's view on the Amazon cloud outage incident, I believe this actually shows the power and outreach of the cloud computing. The technology needs some time to mature and become reliable, let's just be patient. It is very exciting to see even big sites like reddit, quora was affected, and even the cloud service provider Heroku was one of the victims.

There are many new players in the cloud computing field and Cloud Foundry from VMWare is definitely a very disruptive force to the current market. Currently, it supports Spring for Java apps, Rails and Sinatra for Ruby apps, JVM frameworks like Grails and Node.js! Best of all, the whole platform is open sourced.

CF provides a Micro Cloud for individual developers, for free now ;-) This is how to get a helloworld node.js app running:

1. Preparation

- Sign up for the free Micro Cloud for Cloud Foundry (http://www.cloudfoundry.com/).
- Wait a few days (mine took 9 days) and you will receive an approval email.
- Cloud Foundry uses a CLI tool called vmc for interactions with your Micro Cloud instance. Follow the instructions and use the credentials in the approval email to set up vmc (see "vmc getting started guide", really wish it is available not only in PDF format).

2. Write a helloworld node.js app

As of this writing, CF supports node.js-0.4.5 (use "vmc runtimes" to find out). The only thing that is different in the CF environment is that the system will assign a host and port to your application. CF adds several environment variables accessible through process.env.

For our helloworld, we are interested in VCAP_APP_HOST and VCAP_APP_PORT (there are also VMC_APP_HOST and VMC_APP_PORT, which seems still exist but will be deprecated).

So, you basically create a directory (I use the same appid as the directory name) and create an app.js for hello world like this:

var http = require('http'),
    port = Number(process.env.VCAP_APP_PORT || 3000),
    host = process.env.VCAP_APP_HOST || 'localhost';

http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Welcome to Cloud Foundry!\n' +
            host + ':' + port + '\n' +
            require('util').inspect(process.env, false, null));
}).listen(port, host);

console.log('Server running at http://' + host + ':' + port + '/');

This simple app just prints out "Welcome to Cloud Foundry!" and a bunch of environment variables ;-) Note that we put localhost and 3000 there so that it is easier to debug the app on a local node.js installation.

3. Deploy using vmc

The very first time you deploy the app, you run "vmc push <appid>". Note that you need to pick an id that is not being used, otherwise, you will get an error. It would be really nice for CF to provide a Web UI to check what appid is still available. You just follow the instructions and see the output. If everything goes well, you will see your app running at http://<appid>.cloudfoundry.com/.

To update your app, you need to run "vmc update <appid>" instead of "vmc push <appid>". You can also delete your app using "vmc delete <appid>".

vmc CLI provides many operations, "vmc -h" is your best friend to find and try them out.

4. Add npm module dependencies

In most cases, you will need to use dependent NPM modules such as Connect, Express, etc. With the current node.js 0.4.5 supported on CF, it is very straightforward.

- You create a package.json that defines your app/module with dependencies.
- Then you run "npm bundle", which creates a node_modules directory with all the required NPM modules.
- Finally, you do "vmc update <appid>" and you are done.

Here is a quick sample:

package.json

{
    "name": "helloworld-node",
    "version": "0.0.1",
    "dependencies":
        {
            "connect": "*"
        }
}

app.js

var connect = require('connect');

connect.createServer(function(req, res) {
    var body = 'hello, cloud foundry';
    res.setHeader('Content-Length', body.length);
    res.end(body);
}).listen(process.env.VCAP_APP_PORT || 3000);

console.log('connect server listening...');

dir structure:

app.js, package.json, node_modules

5. To learn more

I will try to update this helloworld guide and add a bit about bundling dependent npm modules. To learn more about Cloud Foundry, read the CF blog and code, since it is open sourced, you can simply install it from github.

Updated: found a recent post about using MongoDB on cloudfoundry ;-)

Free ebooks and tutorials for PHP, JavaScript and Ruby Beginners

2011-04-19T20:11:00.000-07:00

Many thanks to RReadWriteWeb for these links:

interesting node modules

2011-04-14T00:08:00.000-07:00

There are so many node modules out there to the point that sometimes you can have several different ones that appear to solve the same problem.

node.js on GitHub has a list of classified node modules, but still it is quite difficult to pick the best one (in terms of node version compatibility, easy to use, still actively developed, good performance, etc.).

It would be really cool to have some sites that actually give reviews and code samples of using these node modules. If performance comparison can be provided, that is even better.

Inspired by "6 Must Have Node.js Modules", I am listing the modules that I found quite interesting and useful to the current work:

HTTP clients:

XML to object conversion:

node-xml2js, a fairly lightweight and fast xml to object conversion library
node-expat, a very high performance SAX lib, native binding to libexpat, it has a fork with object conversion support

Image processing:

node-canvas: canvas implementation based on Cairo
node-qrcode: QR code generator

resources to get started with node.js

2011-03-20T14:40:00.000-07:00

There are quite some blogs and websites about "getting started with node.js". Here are some resources that I found most useful:

How To Node: maybe the best blog covering latest development in node.js with many core node.js developers and users.
Up and Running with Node.js: OReilly's new book, freely available online as of now
Mastering Node: another great online book
Node by Example: a series of examples to get you started with node.js
The Node Beginner Book (WIP new)

set up Linksys WRT54GL as a WiFi repeater with Tomato

2011-03-19T19:39:00.000-07:00

Since I got the new ASUS RT-N16, the old Linksys WRT54GL v1.1 router has been sitting there and collecting dust. There is only one phone line at home at one corner of the house, so I have been thinking of using the idle router as a WiFi repeater so that I can get decent WiFi coverage for each room.

Finally I found some time this weekend and studied about it. Many online tutorials cover how to do it using DD-WRT, another great customized firmware for WiFi routers. Since my RT-N16 already runs Tomato, I'd want to find a way to have Tomato on WRT54GL and work as the repeater as well.

After digging into the Web searches, finally I found Tomator author's FAQ for WDS, it is very straightforward. Basically, both both master and repeater routers need to set "Wireless Mode" into "AP and WDS". You also need to turn off dhcp and wan (turn WAN/Internet type into "disabled"). Then you configure WDS to "Link with..." each other's MAC address.

WDS stands for Wireless Distribution System, it enables interconnections between access points (AP). Wikipedia definition has more technical details. Ideally, using WDS we can chain more APs together, just make sure you don't keep any loop ;-)

The beautify of this is that it uses the same ssid, so when you move around, it switches to the router with the best signal strength automatically.

To test the signal strength, there is a nice free tool for Mac called AP Grapher, very handy indeed.

wireshark no network interface issue

2011-03-05T15:37:00.000-08:00

wireshark is one of the best tool to deep network analysis. But due to the permission issue on BPF devices on Mac OSX (by default only root can read and write BPF devices), you won't see network interfaces in wireshare on Mac ;-(

There are two ways to fix:

- run sudo chown <your_user_id> /dev/bpf* on command line before you start wireshark, you will need to do it again after reboot

- add a startup script to fix this permanently (see instructions that comes with wireshark, D&D Utilities/ChmodBPF directory in the installation package into /Library/StartupItems

. You can adjust the script for your needs as well, for details, see the documentation that comes with wireshare.

node.js hosting in Amazon EC2

2011-02-26T20:25:00.000-08:00

node.js has rapid growth and there are so many hosting services available. Ryan maintains a long list of providers on github already and new ones like NodeFu keeps coming out every day. I have personally tried two managed hosting services duostack and JSApp.US, both are very easy to set up and use. The only drawback is that they pose limitations on installed modules, etc.

Amazon is offering 1 year free trial for its cloud services, so called "AWS Free Usage Tier". I decided to give it a try and set up an EC2 instance for node.js.

The whole process is not quite user friendly (not as friendly as the shopping experience on Amazon). There are quite several choices for the image to choose. My first attempt with a SuSE 64bit image also got permission issues when I install gcc somehow. Finally, from the Web search results, I found Mike Leach's step-by-step guide "Installing node.js on Amazon EC2", which is really helpful. Jim Smith's blog post added a nginx as the front end, a nice addition since node.js is not multi-threading out of box.

Some minor differences when I followed Mike's steps:

instead of ssh to the vm using ec2-user@ec2-public-dns, I had to use ec2-user@<address you can find in AWS management console by right clicking on the instance and then clicking connect>, note that the instructions there told you to ssh as root@<address>, but it did not work for me, saying I had to use ec2-user@
instead of --without-ssl option to ./configure, I added openssl and openssl-devel
I did node and npm installation using the "node-and-npm-in-30-seconds.sh" script from this gist mentioned in the Joyeur post "Installing Node and npm"

handy tutorials

2011-02-12T23:31:00.000-08:00

Just found several handy tutorials/references that could quickly refresh your knowledge about the tools and get work done easily.

Linux Shell Scripting Tutorial, I am a big fan of nixCraft site for its quick tips on *NIX. I found this one easier to navigate and understand than BASH programming - Introduction HOW-TO or Advanced Bash-Scripting Guide
TextMate Basics Tutorial, a nicely written piece that walks you through the basics of TextMate, I especially like the organization of the shortcut keys
Git Reference, a much shorter quick guide than Git User's Manual (too long and boring)

Again, reading a lot is not as useful as coding a lot. But if you don't use it for day-to-day work, you might easily forget many tricks. So, some quick references are handy in this situation to bring you quickly up to speed.

what does "new" do in JavaScript

2011-02-05T15:05:00.000-08:00

Today, a colleague asked about exactly what is the difference between using "new" and calling the function directly. Only thing I can think of is the meaning for "this" inside the function. With "new", this means the object being created, but without "new", this means the global object, either window or something else. And I am also not clear what exactly happens behind the scene.

So, did some search and found some good readings that explains this topic fairly clearly. Lots of related answers from Stack Overflow is also quite helpful:

What is the 'new' keyword in JavaScript
Tim Caswell's "Learning JavaScript with Object Graphs" series (perfect for visual person like me, in II, he specifically mentions about how to create objects besides using new: use Object.create and object factory)

Here is a good article that tests your understanding of "new" keyword ;-)

Now, whether or not you should use "new" is another question:

Is JavaScript's 'new' keyword considered harmful?

Using "new" to instantiate a class has both pros (great performance) and cons (can cause nightmare if you forget to use new). Many people including Douglas Crockford discourages the use of new. A module pattern is a more preferable way. John Resig also has a simple solution to avoid using new directly without much performance sacrifice.

Hack ASUS RT-N16 to enable NAS, media server, offline BT/emule, etc.

2010-12-05T14:30:00.000-08:00

ASUS RT-N16 is the most affordable and versatile wireless router out there. I was previously looking for a NAS solution and then found that this router has 2 USB ports that could turn a printer into network printer and an external USB hard drive into a network storage.

The default firmware works ok so far. What I did is first configure it and then upgrade to the latest firmware version from ASUS website. UPnP and Samba file sharing worked fine out of box.

But after doing some online search, to really unleash the real power of this router, we need to install third-party firmware such as Tomato (it actually needs Tomato variants that support USB such as Tomato USB) and DD-WRT.

There are also several Tomato variants from Taiwan and mainland China: Tomato DualWAN, Tomato Pandora (supported routers) that supports web-based tools such as Chinese Web interface, BT, file system access, etc. I also found a Taiwan forum that discusses these variants, quite cool actually.

Searching on Tomato USB site gives several tutorials, but this one recommended by a colleague seems to be most straightforward and I am going to try it out now ;-)

Update:

The whole processing following the above tutorial went like a breeze. The only thing (not really an issue) that I encountered is that the restoration tool got stuck at 33% for quite a while and then gave a scary error message. But then I tested and the router is actually already running Tomato. So, simply ignore that error!

Note that you need to enable the USB option to get your external USB hard disk working. Here is an article that walks you through setting up a poor man's NAS with more detailed settings and controls. He also did some benchmark on a USB flash drive and found out that the flash drive has a better performance attached to the router than attached to the windows computer, unbelievable...

So, the next step will be to install Optware and get a lot of interesting toys (amule, transmission, unix tools, web servers, vpn, asterisk voip pbx, etc.) on it! Also heard there is another better UPnP solution that does transcoding for iOS devices (iPhone/iPad has many UPnP apps such as PlugPlayer, Apple TV supports XBMC), haha...

quiz time!

2010-12-01T13:58:00.000-08:00

If you are confident about your JavaScript knowledge, try out these quizzes, it just made my brain stop working...

"So, you think you know JavaScript?", answers explained by Zakas
"Another JavaScript Quiz" by James Padolsey
"JavaScript Quiz" from Perfection kills (exploring JavaScript by example)

Now, if you feel how twisted/unreasonable/difficult/loose JavaScript syntax is, you are not alone.

Several good readings that might help to clear things a bit:

Zakas explained a bit about the first quiz

It is interesting that a variable can be declared after it is being used because the engine will hoist the variable declarations to the top of execution context. Also interesting to see that the engine breaks "var i = 1" into declaration part "var i;" and initialization part "i = 1;", and move the declaration to the top.

Another interesting note is that function declaration overrides a variable declaration with the same name, but cannot override the variable declaration+initialization.

Understanding delete from Perfection kills

Really a comprehensive explanation about how execution context, scope, variable attributes, etc. work in JavaScript. Now, finally I understand the difference between variable declaration+initialization (cannot be deleted) and undeclared assignment (can be deleted) because of the former one has DontDelete flag and the latter one does not.

After reading this article, it is now apparent why the function declaration has strange override behavior for the variable with a same name. The variable declaration+initialization gets a DontDelete flag and thus cannot be overridden.

Named function expressions demystified

This one explained what is function declaration, function expression and named function expression and bugs handling named function expression in various JavaScript engines. One note is that for the named function expression, the name/identifier can only be accessed in its function body.

ECMA-262 3rd edition specification

The ultimate reference for underlying mechanisms of JavaScript.

what this means in JavaScript

2010-11-22T11:19:00.000-08:00

As someone new to JavaScript programming, there are many language features that are so uncomfortable. What *this* really means and referring to is one of the pains.

Finally, after reading some nice articles, I think I start to get the idea: it basically means the context/scope the function is called and can change at runtime.

Two great articles that explains this clearly:

Execute JavaScript from TextMate

2010-11-20T00:29:00.000-08:00

TextMate is the most popular text editor for frontend development (of course, people also use emacs, vim/gvim/macvim, Aptana, Eclipse, etc.). I have been using TM for PHP development and really enjoyed the convenience of ctrl+shift+R to execute the PHP CLI scripts and view the output from TM.

I know most of people develop and debug JavaScript from development tools provided by the browser, e.g. firebug, developer tools in Chrome, Safari, etc. But it will be really neat if I can just run JavaScript and view output directly from TM.

So, I searched and searched. There are many existing blog posts on this topic already. It basically consists of two steps:

1. get a JavaScript engine that executes the JavaScript code, here are some options:

- Mac/Safari has a JS engine (SquirrelFish Extreme) built in already, it is located at /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc

- Node.js: Node.js is a event-based JS platform based on the V8 JS engine, see my previous post on how to set it up

- Mozilla JS engines: installation guide to build/install SpiderMonkey, Rhino, TraceMonkey on Mac

- For other JS engines, do a Web search for installation instructions

2. define new TM commands for JavaScript in TM bundle editor and assign ctrl+shift+R key binding to the commands.

For command setup to use Node.js, see this github snippet by beastaugh, it basically creates a script that executes another script given the file path.

For command to set up other engines, you specify the path to the engine executable, execute the JavaScript and output in a popup window. You can also use another command and open up a terminal to show the results, see this post for how to do that.

Then, you are all set. This is a screen shot that allows me to pick one of the three engines to run my JavaScript from TM ;-)

to var or not to var

2010-11-19T23:01:00.000-08:00

It's a quite interesting and yet confusing question:

What is the difference between declaration of a variable with and without "var"?

After reading the question and answers on Stack Overflow, I finally start to get it.

First, it affects the scope of the variable.

The variable defined without var means the nearest scope in the case of a hierarchical structure, e.g. embedding one function inside another function. In most cases, a declaration without var in a function means global scope (in many cases this means window object). The variable defined with var is bound to the scope of that function.

Second, it affects the delete behavior.

The variable defined with var cannot be deleted. The variable defined without var (actually, this is a property, not a variable in the true sense) can be deleted. The reason behind this that variable defined with var has a flag of DontDelete. See this nicely explained Stack Overflow post for details.

Lastly, the general guide is to always use var.

var i = 'hello';
j = 'world';
print("i=" + i + ", j=" + j); // i=hello, j=world
(testFunc = function() {
 var i = 'foo';
 print("inside function: i=" + i); // inside function: i=foo
 print("inside function: j=" + j); // inside function: j=world
})();
delete i;
print("i=" + i); // i=hello, ha, i still exists!
delete j;
print("j=" + j); // ReferenceError: j is not defined, j got deleted

Ok, one last exercise before I close this post. Guess what is the output for this code snippet:

func2_var = 'var outside';
func2_var2 = 'var2 outside';
func2 = function() {
 var func2_var = 'var in func2 body';
 this.func2_var2 = 'var2 in func2 this';
}
func2.prototype.func2_var = 'var in func2 prototype';

f2 = new func2();
print("which var it is: " + f2.func2_var);
print("which var2 it is: " + f2.func2_var2);

The END.

Lint for JavaScript

2010-11-18T22:24:00.000-08:00

Lint tools are very useful to detect syntax and style errors in your code. It of course has variants for JavaScript. There are many different tool setup based on your editor. On the Mac, maybe the most popular/handy editor is TextMate.

1. jslint.com: a Web tool

This is the tool developed by Douglas Crockford. It has a simple Web interface. You copy and paste your code and click JSLint button, it is that simple. The drawback is every time you make a change, you need to re-copy-n-paste and repeat the validation.

2. TextMate bundles:

johnmuhl's javascript tools tmbudle
It provides a toolkit for working with JavaScript in Textmate: JSLint, various formatters, obfuscators, and compressors.

subtleGradient's javascript tools tmbundle
This is very similar to the previous tmbundle except that it uses JavaScript Lint instead of JSLint. Also, the timestamp seems more up-to-date. The installation is very straightforward, download the src, rename it to be a TextMate bundle file and let TextMate install it. Then, you access it via various key combinations.

3. TextMate integration with JSLint:

Stoyan has a nice post with all the detailed instructions. It is a bit complicated than using method #2.

4. Vim integration with JSLint:

Only saw some discussions on Stack Overflow, did not actually try it out.

Android on iPhone

2010-11-17T22:40:00.000-08:00

First, this does not make much sense unless you are someone who just wants to try out crazy stuff. Somehow I was in an unstable mode that day and jumped right in when I saw this article that details all the steps how to run Android 2.2.1 on iPhone.

And it turned out to be a disaster and waste of time.

The installation process all went fine. And I can boot into Android, though the boot process is a bit long (maybe because mine is iPhone 2G?)

After booting into Android, everything looked fine, except that I could not find the Market to download apps -_- !!!

I played around with existing apps. The camera is not really working and crashed several times on me. The phone dial seemed to be working fine and I can make phone calls.

Then I got bored and booted back into iOS. To my surprise, iTunes cannot recognize the iPhone anymore. This means I cannot sync and do backups, which really really sucks. The tutorial should have warned me in advance.

Another strange thing is that many settings are changed in iOS. All the name in my contacts are gone, so basically I only see the phone numbers now. Also settings like screenlock, wifi and others are also changed somehow.

I tried to restore and jailbreak again. It took me quite some time to find the right firmware and tool. Finally, everything is back to normal and data was restored.

So, I highly discourage this experiment to anyone since the software is still not stable.

JavaScript links

2010-11-09T23:15:00.000-08:00

Saw this interesting discussion about whether we should use href="javascript:void(0)" or href="#" with onClick event handler on Stack Overflow.

I think the best solution is to avoid using link element <a> in the first place and use CSS to style an element to make it "look" like a link and attach the event handler to that element. This article has more discussions and comparisons why we should not use links in this particular case.

As for why the event handler need to return false, it is because we want to avoid the default behavior of the link element, i.e. to follow the href and go to another page. This quirksmode article has it very clearly explained.

node.js setup on Mac

2010-11-08T22:04:00.000-08:00

node.js has been getting a lot tractions lately due to its high performance on V8 engine, event-based model and reusability of client side JavaScript on server side without any changes.

There are several articles discussing how to set it up on the Mac and I found this one from Florian Kubis to be the easiest to follow.

If you are using homebrew as your package management system instead of MacPorts, you should use this tutorial instead.

The tutorials also cover npm (the package manager for node.js) and express (a Web app framework built for node.js)

There is no MacPorts ports for npm and express yet, maybe someone should create them?

Several slides and quick intro about node.js:

"Evented I/O based web servers, explained using bunnies", the idea behind node.js in 10 pages
"Node.js and websockets intro", a bit more technical than the bunnies example
"Node.js: how JavaScript is changing server programming", more details in 65 pages
"Running YUI3 on node.js", integration with YUI3, moving client js to server side without any changes!

syntax highlight with gist

2010-10-31T22:21:00.000-07:00

Another much simpler way to add syntax highlight is using gist, which is a github tool to share code snippets. Each gist is basically a git repository that is versioned and forked. Syntax highlight happens to be a feature of it.

Simple head over to http://gist.github.com/, paste your code snippet there and select the language, then select "create public gist". On the next page, click on "embed" and it becomes a JavaScript snippet. Just embed it in your page and you are done!

No need to change Blogger template and no need to understand the configuration and syntax of SyntaxHighlighter ;-)

Here is a quick example:

syntax highlight with SyntaxHighlighter (for Blogger)

2010-10-31T14:30:00.000-07:00

SyntaxHighlighter is the standard way to put syntax highlight for your code snippets on a Web page. It is very straightforward to set up for Blogger (thanks to Leonid's excellent Blogger post).

Here is a quick summary of what you need to do (a bit variations for better performance):

1. Go to Design->Edit HTML, update the template with SyntaxHighlighter CSS and Javascript elements.

Splitting the CSS part and JavaScript part: put CSS that inside (not before) the <head> element. The JavaScript part goes right before the closing <body> element.

Comment out or delete AutoLoader JavaScript (the line with shAutoLoader.js) since we are not using it (I could not get it working either, although some people claimed it worked for them). The new AutoLoader basically loads the JavaScript brushes on demand instead of load from a list and use only one of them.

Limit the list of JavaScript brushes to what you need only. Leonid lists all the brush types he is using, you can adjust that for your own needs. Since these JavaScripts will be loaded each time, it is better to keep the list short to get good performance. Here is the list of all existing brush JavaScript files.

OK, the Blogger template update is done. Now, let's use it in the actual post.

2. In your new post, simply insert code inside a pair of <pre></pre> elements and specify brush type based on language like this:

<pre class="brush: _brush_lang_">your code</pre>

3. More adjustments and tips.

Use a different CSS theme. Leonid uses Midnight theme (shThemeMidnight.css), I am using the Eclipse theme. You can choose a theme that you like, here is the list of available themes.

Customize different styles for individual code block. The changes in the Blogger template applies to all the code blocks. You can also specify/override styles for individual blocks. The simplest way is use the parameters in the <pre> class attributes. You can read available parameters on SyntaxHighlighter configuration page.

For example, I can easily highlight line #1 and #12 using

<pre class="brush: javascript; highlight: [1, 12]">your_code</pre>

This is what it looks like:

var stringArray = new Array();
  stringArray[0] = "A";
  stringArray[1] = "B";
  stringArray[2] = "C";
  stringArray[3] = "D";
  stringArray[4] = "E";
  stringArray[5] = "F";
  stringArray[6] = "G";

  document.write("String value");
  for (var i = 0; i < stringArray.length; i++) {
    document.write(i + 1 + ". " + stringArray[i] + "");
  }
  document.write('
');

Host your own SyntaxHighlighter scripts. You can use the hosted CSS and Javascript like the URLs in Leonid's example, which always points to the current/latests SyntaxHighlighter code base. If you want, you can also host it on you own.

References:

Official user guide, it talks about another way that uses <script> tag instead of <pre>, <pre> has limitation of handling < character
List of brushes and how to make your own
List of Themes
List of configuration options and parameters

exploring Google App Engine

2010-10-30T14:20:00.000-07:00

Started to explore the Google App Engine, a cloud computing platform that enables developers to plug into the Google platform and services. It supports Python and Java (with some limitations) Web app frameworks.

Here are some links that I found useful to begin with:

Many books already available for this topic:

O'Reilly: Programming Google App Engine (Nov 2009), Using Google App Engine (May 2009)
Apress: Developing with Google App Engine (Feb 2009), Beginning Java Google App Engine (Dec 2009)

It is a pity that PHP is not supported so far, but people have found ways to get around it by using a pure Java implementation of the PHP called Quercus.

move posts from blogsome to Blogger

2010-10-29T23:15:00.000-07:00

I used to like free wordpress blog service provided by blogsome and occasionally update my technical notes at woshiadai.blogsome.com. But its online editor is really painful to use and way behind the Blogger editor.

Today, after tried and failed to set up the SyntaxHighlighter javascript code highlighter on blogsome. I decided to move from blogsome to Blogger.

But how to export existing posts and import into Blogger? After a bit Google search, here is what you should do (assuming your blogsome blog is foo.blogsome.com)

1. export existing blogsome posts in RSS 2.0 format in an XML file

- log into blogsome as admin
- go to manage->posts, write down the id of your latest post, say 125, this is the total number of posts you have for your blogsome blog
- go to options->reading->syndication feeds, change "show most recent" from x (e.g. default is 10) to 125, also change "for each article, show" to full text
- open a new browser window, enter http://foo.blogsome.com/feed/, you will get all existing posts in RSS 2.0 format, save the current page as feed.xml

2. convert exported blogsome feed.xml to a format that Blogger can recognize/accept

Blogger has some strange behavior, if you directly import this feed.xml, Blogger gives a very hard to understand error with the slide bar keeping going back and forward (Blogger, please fix this bug, at least provide some more meaningful explanation about this error)

"Sorry, the import failed due to a server error. The error code is bX-tjg9ds"

I also tried to export blogsome posts in atom format, which is supported by google, but no luck, still the same. Then I googled the error and someone has a solution already!

You simply go to wordpress2blogger conversion tool site, use "choose file" to upload your feed.xml and click "convert", and then save the result to feed-converted.xml

3. import feed-converted.xml to Blogger

blogger has a detailed tutorial on that already, so I don't want to repeat that. Just follow the instructions there. Unless you need to hide some posts, you should select "automatically publish all imported posts" on the import page.

That's it, you have a new life on Blogger!

References:

Hello world!

2010-10-29T22:45:00.046-07:00

Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!

Firefox: 50000000 downloads

2010-10-29T22:45:00.045-07:00

Firefox is celebrating 50 million downloads now

I have been using it for a long time now, it is quite neat and fast. You don’t have to worry about IE security holes and non-standard M$ tags anymore.

BTW, this is a test post using BlogJet. The only thing I found annoying with this desktop blog publishing tool is that it has poor support for encoding, e.g. I cannot post Simplified Chinese contents

check if a database foo exist in MySQL

2010-10-29T22:45:00.044-07:00

Problem: I want to see if a database named “foo” exist in MySQL server

This should not be a tough problem at first sight. However, I googled the topic and did not find an official answer except several quick&dirty solutions. Please let me know if there are better and clean ways to do it right.

Setup: MySQL v4.1.9 + mysql-connector-java v3.1.6 + J2SE v1.4.2_06

Solution 1: Catch the exception for “unknown database %s” (I used this one finally).

If you create a new Connection to the URL that points to a non-existent database in MySQL, an SQLException is thrown with the message “unknown database %s” where %s is the database name. The exception has error code of 1049 (int) and SQL state of 42000 (String). Check MySQL error code list for the full list of possible errors for MySQL.

So, I just check the SQLException error code and/or SQL state to make sure this exception happens so that I know a database with the given name exist or not.

        boolean isExistRepos = true;

        try{
            connection = DriverManager.getConnection(reposURL, “root”, “rootpassword”);
        }catch (SQLException se){
            while(se != null && isExistRepos){
                String logMessage = “\n\n An SQL Error Occured: “
                                  + se.getMessage() + “\n\t”
                                  + “Error Code: ” + se.getErrorCode()
                                  + “\n\t” + “SQLState: “
                                  + se.getSQLState() + “\n”;
                System.err.println(logMessage);

                //repos does not exist and the connection cannot set up
                //MySQL error list (http://dev.mysql.com/doc/mysql/en/error-handling.html)
                //#Error: 1049 SQLSTATE: 42000 (ER_BAD_DB_ERROR)
                //Message: Unknown database ‘%s’
                if((se.getErrorCode() == 1049) && (se.getSQLState().equalsIgnoreCase(”42000″)))
                    isExistRepos = false;
                se = se.getNextException();
            }
        }

Solution 2: Call mysql command line client using Runtime class.

I did not try it, but here is the general idea. MySQL comes with a command line client program called “mysql” where users can interact with the database server. You can type in “show databases” and a list of existing databases will be presented.

In Java, we can use Runtime.getRuntime().exec(“mysql -u root -p rootpassword”) to get a handle on a Process object, then we can use OutputStream and InputStream to input “show databases” and parse the output to see if the database foo exist in the list of databases.

Solution 3: Check if the directory corresponding to the database exists in MySQL data directory.

I just found that for each database, there exist a directory with the same name in MySQL data directory, e.g. C:\Program Files\MySQL\MySQL Server 4.1\data for my case. So, you can juse check if the directory foo exist in that directory to tell if the corresponding database exist or not. I am not sure about MySQL internals, so this solution is not stable and portable.

create a new database foo in MySQL using JDBC

2010-10-29T22:45:00.043-07:00

Problem: want to create a new database in MySQL using JDBC

Usually when people work with JDBC, they need a Connection object to the destination database, but since we need to create a new database, where to find an existing connection?

Setup: MySQL v4.1.9 + mysql-connector-java v3.1.6 + J2SE v1.4.2_06

Solution 1: Create a Connection to “mysql” admin database and use it to create the new database.

There are two preloaded databases when you install MySQL: mysql and test. mysql is the admin database that keeps metadata like access control information. We can just create a Connection object to mysql admin database and use it to create another new database.

Solution 2: Use Runtime.exec(command) to call mysql command line client.

Solution 3: Create a directory with the same name as the new database name in MySQL data directory, e.g., C:\Program Files\MySQL\MySQL Server 4.1\data. And MySQL server will “think” a new database is created. This method might have risk of corrupted metadata although I have tried this method before and no abnormal behavior was observed.

Represent inner classes in UML

2010-10-29T22:45:00.042-07:00

Problem: How to represent inner classes in the class diagram in UML?

I am doing TA for Software Methodology class this quarter and we use lots of UML diagrams. When grading students’ homework, I found this problem about representing inner classes in the class diagram.

After consulting the UML 1.5 spec and other resources, there are two ways.

Solution 1: (from Holub Associates: UML Reference Card)

Nesting, Inner Class.. Identifies nesting (containment) relationships in all diagrams. In a class diagram: an “inner” class whose definition is nested within the another class definition. Typically puts the inner class in the name space of the outer class.

Solution 2: Use a package symbol (from UML 1.5 spec, chapter 3 UML Notations (PDF), 3.48.2)

“Note that nested notation is not the correct way to show a class declared within another class. Such a declared class is not a structural part of the enclosing class but merely has scope within the namespace of the enclosing class, which acts like a package toward the inner class. Such a namescope containment may be shown by placing a package symbol in the upper right corner of the class symbol. A tool can allow a user to click on the package symbol to open the set of elements declared within it. The “anchor notation” (a cross in a circle on the end of a line) may also be used on a line between two class boxes to show that the class with the anchor icon declares the class on the other end of the line.”

Using Jakarta Commons CLI

2010-10-29T22:45:00.041-07:00

Several examples from Jakarta Commons Cookbook by Timothy M. O’Brien, O’Reilly, Nov 2004

Example 1: Parsing a Simple Command Line

import org.apache.commons.cli.CommandLineParser;
import org.apache.commons.cli.BasicParser;
import org.apache.commons.cli.Options;
import org.apache.commons.cli.CommandLine;

public static void main(String[] args) throws Exception {

    // Create a Parser
    CommandLineParser parser = new BasicParser( );
    Options options = new Options( );
    options.addOption(”h”, “help”, false, “Print this usage information”);
    options.addOption(”v”, “verbose”, false, “Print out VERBOSE information” );
    options.addOption(”f”, “file”, true, “File to save program output to”);

    // Parse the program arguments
    CommandLine commandLine = parser.parse( options, args );

    // Set the appropriate variables based on supplied options
    boolean verbose = false;
    String file = “”;

    if( commandLine.hasOption(’h') ) {
        System.out.println( “Help Message”)
        System.exit(0);
    }

    if( commandLine.hasOption(’v') ) {
        verbose = true;
    }

    if( commandLine.hasOption(’f') ) {
        file = commandLine.getOptionValue(’f');
    }
}

Example 2: Using OptionGroup

Example 3: Print Usage Info

import org.apache.commons.cli.CommandLineParser;
import org.apache.commons.cli.BasicParser;
import org.apache.commons.cli.Options;
import org.apache.commons.cli.OptionBuilder;
import org.apache.commons.cli.OptionGroup;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.HelpFormatter;

public class SomeApp {
    private static final String USAGE = “[-h] [-v] [-f <file> | -m <email>]”;
    private static final String HEADER =
        “SomeApp - A fancy and expensive program, Copyright 2010 Blah.”;
    private static final String FOOTER =
        “For more instructions, see our website at: http://www.blah123.org”;

    public static void main(String[] args) throws Exception {

        // Create a Parser
        CommandLineParser parser = new BasicParser( );
        Options options = new Options( );
        options.addOption(”h”, “help”, false, “Print this usage
                                                                      information”);
        options.addOption(”v”, “verbose”, false, “Print out VERBOSE
                                                                         information” );

        OptionGroup optionGroup = new OptionGroup( );
        optionGroup.addOption( OptionBuilder.hasArg(true).withArgName(”file”)
                                            .withLongOpt(”file”).create(’f') );
        optionGroup.addOption( OptionBuilder.hasArg(true).withArgName(”email”)
                                            .withLongOpt(”email”).create(’m') );
        options.addOptionGroup( optionGroup );
           // Parse the program arguments
        try {
            CommandLine commandLine = parser.parse( options, args );

            if( commandLine.hasOption(’h') ) {
                printUsage( options );
                System.exit(0);
            }

               // … do important stuff …
        } catch( Exception e ) {
            System.out.println( “You provided bad program arguments!” );
            printUsage( options );
            System.exit(1);
        }
    }

    private static void printUsage(Options options) {
        HelpFormatter helpFormatter = new HelpFormatter( );
        helpFormatter.setWidth( 80 );
        helpFormatter.printHelp( USAGE, HEADER, options, FOOTER );
    }
}

About BitKeeper not free anymore

2010-10-29T22:45:00.040-07:00

Recently, I am working on an assessment paper about various existing SCM systems. Basically, we see feature comparison matrices all the time as the marketing tool to sell SCM-X, however people general don’t explicitly state the scenarios for the feature comparisons. For example, you can claim SCM-X does commits 20% faster than SCM-Y, but how big is the commit size, change size, etc., are not clearly explained. We want to fill the void.

Back to the topic, the reading about BitKeeper starts from its feature comparison matrix with Subversion: http://www.bitkeeper.com/Comparisons.Subversion.html. Then, on Subversion site, developers jumped out and tried to debunk BitKeeper’s false claim: http://subversion.tigris.org/bitmover-svn.html. Then it turned out that BitMover, BitKeeper’s company is going to withdraw this free product because of many attempts of reverse-engineering in open-source world. Although Linus himself enjoyed a lot using BitKeeper for kernel development, it will happen soon and kerneltrap.org has a detailed coverage about that, also Linus’s original email. Apparently, Linus does not like Subversion at all (P.S. part) and Karl Fogel, on behalf of the Subversion team, reponded to Linus’ comments on Subversion.

Watching a war like this quite fresh experience for me: previously I thought all those open-source developers are shy, gentle, silient people with long hair or no hair. Well, it is my first time watching the flames between them.

Interestingly enough, there is a survey about “my favorite FOSS source control system” at the side bar of the editorial.

Total votes: 2393

Subversion: 37%

CVS: 24%

Darcs: 10%

Depends on project: 7%

GNU Arch: 6%, not listed: 6%

Bazaar-NG: 3%

Bazaar: 1%

Vesta: 0%, Codeville: 0%

Subversion experience

2010-10-29T22:45:00.039-07:00

My Experiences With Subversion by Simon Tatham

1. Introduction

When I’m not at work, I’m a free software developer. I maintain a variety of published projects, ranging from fairly major things like PuTTY to tiny little Unix utilities; and I have almost as wide a variety of unpublished projects as well, ranging from half-finished major programs to my personal .bashrc. Until November 2004, all these projects were stored in CVS, along with probably 90% of the other free software in the world.

Then I migrated to Subversion. This took a fair amount of thought and effort to do well, and shortly afterwards I was asked by a colleague if I could write something about my experiences. He was probably expecting something more like a couple of paragraphs, but I thought, hey, why not do the job right?

This article is not a rant. In general, I have found Subversion to be linearly superior to CVS and I certainly don’t regret migrating to it. The article is just an attempt to share my experiences: things to watch out for, how to get the most out of Subversion, that sort of thing.

More >>>

BitKeeper Testdrive

2010-10-29T22:45:00.038-07:00

BitKeeper test drive documentation from their website.

A discontinued tutorial for BitKeeper by Zac, it is quite interesting to see the reason for discontinuing the tutorial writing:

“(DISCONTINUED) A guide for starting into the world of Bit Keeper, a really cool source control system. This is the document I wish I had when I was getting started.. [UPDATE] Cancelled to do me finding out what the company behind the product is really like. Too bad, because it seems like it’s actually a pretty good product.”

I think sometimes people enjoy free lunches should not blame too much for not having free lunches suddenly. Everyone has family to support and needs money. But I would rather say at the very beginning clearly rather than giving the impression that I want to make money from the user base which were attracted because of the free services.

Notes for by Craig Grannell

2010-10-29T22:45:00.037-07:00

1. CSS shorthand for boxes:

margin: 10px — applies to all edges
margin: 10px, 20px — 10px applies to the top and bottom edges; 20px applies to left and right edges
margin: 10px, 20px, 30px — 10px applies to the top, 20px to the left and right, 30px to the bottom
margin: 10px, 20px, 30px, 40px — clockwise order starting from the top (top, right, bottom, and left)

2. Applying styles to a Web page:

a) use a link tag:
[link rel=”stylesheet” type=”text/css” href=”mystylesheet.css” /]

b) use import:
[style type=”text/css”]
@import url(mystylesheet.css);
[/style]

c) embed styles in HTML document:
[head]
[style type=”text/css” media=”all”]
p{
color: black;
}

navigation p{

color: blue;
font-weight: bold;
font-size: 120%;
}
[/style]
[/head]

d) inline styles:
[p style=”color: red;”]This is painted in Red.[/p]

3. DOCTYPE declarations:

a) XHTML strict:
[!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN”
“http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd”]

b) XHTML transitional: good for depreciated tags
[!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Transitional//EN”
“http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd”]

c) XHTML frameset:
[!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Frameset//EN”
“http://www.w3.org/TR/xhtml1/DTD/xhtml1-frameset.dtd”]

4. Go to top of a page:

a) [a id=”top” name=”top”][/a]
[a href=”#top”]Back to top[/a]

b) some browsers ignore empty elements, so we need to put a single
space in between; if we use XHTML Strict, we need to put elements into
a block element such as [p] or [div].
[div id=”topOfPageAnchor”]
[a id=”top” name=”top”] [/a][/div]

define CSS
div#topOfPageAnchor{
position: absolute;
top: 0;
left: 0;
height: 0;
}

c) use Javascript:
[div id=”topOfPageAnchor”]
[a id=”top” name=”top”] [/a][/div]

[a href=”#top” onclick=”javascript: scrollTo(0,0);”]Top of page[/a]

5. Attaching Javascript:

a) External
[script type=”text/javascript” src=”javascriptfile.js”][/script]

b) Internal
[script type=”text/javascript”] script content [/script]

6. Toggling div visibility with Javascript:

[div][a href=”#” title=”show text” onclick=”swap(’hiddenDiv’);return false;”]show text[/a][/div]
[div id=”hiddenDiv” style=”display: none;”][p]hello[/p][/div]

7. pseudo-class selectors:

For anchors

a{color: #3366cc;}
a:link{color: #3366cc;}
a:visited{color:#777700;}
a:active{color:#cc00ff;}
a:hover{color:#0066ff;}

Useful resources:

CSS hacks for different Web browsers
W3C Markup Validation
W3C Link Checker
W3C CSS Validation
iCapture shows your page in Safari
CSS switcher demo

Experiment with Perforce

2010-10-29T22:45:00.036-07:00

Perforce is one of the popular SCM system used by many IT companies. It uses RCS ,v file format for history archive file. The network communication model is client/server over TCP/IP. I am not sure about the detailed protocol Perforce uses.

Some quick points after reading Perforce user guide and administration guide:

1) Configuration and setup is a bit complicated. The document is not very straightforward about the setup process, meaning of basic concepts and usage of the environment variables. After reading the whole user guide, you finally get the basic ideas about various elements of Perforce and how they should be used, but not at the very beginning.

2) Too many commands and options. There are just too many commands and options to get a job done. And the worst part is that some commands have different names from CVS’s or Subversion’s commands, which are well-known to SCM users. Perforce also uses their own terms for well-understood concepts, e.g. integrate v.s. merge.

3) Perforce provides support for a wide range of platforms and offers multi-language API for 3rd-party developers. The Web client interface is really cool and it should save a lot of time for beginners.

4) Perforce reuses RCS history archive file format and maybe this is why it does not support directory versioning.

5) Internally, Perforce keeps a mapping between file extension and file type to distinguish binary and text files. Binary files are stored in whole while text files are stored in deltas.

6) Transaction is well supported in Perforce. Either all changed files get submitted in a changelist or none of them.

7) Branching is basically copy operation, and Perforce can keep track of the branching internally so that the history information is preserved.

8) In Perforce, a job records what needs to be done and a changelist stores the actual changes. They can be linked to represent the scenario of a bug report and how the bug is fixed.

Concepts in Perforce:

Depot: repository on Perforce server, it is a directory called “depot” in the perforce root directory on the server.

Workspace: isolated place where users get their job done. Users define a mapping between depot directory structure and workspace directory structure.

View: a mapping between files in depot and files in workspace.

Changelist: similar in spirit to the changeset concept. It records changes made as a transaction. There is a default changelist and users can define individual chagnelist called “numbered changelist” as well.

Job: a description of problems that should be solved, e.g. a bug report. Jobs are linked to changelists that actually make changes happen and fix the problems.

Label: similar to label in RCS, CVS and Subversion. Use label to make logic group of file, e.g. alpha release, bug fix, etc.

Branch: copy of files at a different directory. There are two approaches for branching: based on file specifications (from-files to-files) or branch specifications (name the branch mapping).

Perforce User Guide (2005.1)Perforce User Guide (2005.1)
Perforce Admin GuidePerforce Admin Guide

MDA and Software Factories

2010-10-29T22:45:00.035-07:00

I just read some recent articles and literatures about the idea of “building software at a higher level (than raw source code)” such as MDA, software workbench, software factories, etc. It seems like people are starting or have started (e.g. tools like OptimalJ from Compuware) using models as an efficient way of building software and leave the complexity issues to automation (as much as possible). Also, it appears to me that domain specific languages might appear as the major programming languages for specific domains. Actually, lots of XML documents have been serving as configuration files, preferences and options for quite some time, those in some sense are DSLs too.

Here are some interesting projects and articles from Eclipse and VS.NET 2005:

1) EMF+GEF+GMF: those are the modeling frameworks in Eclipse. EMF is quite similar in spirit to MDA proposed by OMG, GEF provides the framework to write graphical editors for editing EMF models. GMF (http://www.eclipse.org/gmf/) is a newly proposed project that aims to add automatic generation of GEF editors for EMF models, which bridges the gap between EMF and GEF.

2) Other MDA-related projects at Eclipse: GMT (http://www.eclipse.org/gmt/) is a set of research tools for generative model transformation. This one seems quite interesting because we are thinking about model transformation as well for our wizard-based design tools.MDDi (http://www.eclipse.org/proposals/eclipse-mddi/index.html) seems quite intersting as well, but it is also in its infancy.

3) MS has been pushing about software modeling in VS.NET 2005 and there are ideas and tools about “software factotires” (http://lab.msdn.microsoft.com/teamsystem/workshop/sf/default.aspx). They have a series of articles about software factories by Jack Greenfield. He also wrote a book (http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471202843.html).

A quick comparison of UML 1.4 and 2.0

2010-10-29T22:45:00.034-07:00

From UML Bible by Tom Pender

“The UML authors of both 1.4 and 2.0 endeavored to uphold the four-layer meta-model (M0–M3) architecture, an approach that supports the distribution of concepts across many levels of abstraction. The layering supports the specification of concepts for different purposes such as object modeling (UML) and data modeling (CWM), customization for different domains, and ultimately for different implementations.

In the UML 1.4 architecture:

The MOF provides the foundation for the UML

The UML defines a Foundation package as well as the Behavioral and Model Management features. Together these packages define all of the essential features needed to create the modeling elements used to build UML diagrams.

In the UML 2.0 architecture:

The new architecture defines an Infrastructure and a Superstructure

The Infrustructure redefines the highest level of the architecture used to create the MOF and all other MDA components.

The Superstructure is the UML portion of the architecture. The Superstructure derives all of its components from both the Infrastructure and the MOF.

The Superstructure is organized according to the three types of diagrams defined by UML, that is structural (Class, Object, and so on), behavioral (Sequence, Timing, State Machine, and the like), and supplemental (information flows, profiles, and templates).

Diagram changes from 1.4 to 2.0:

2.0 replaced the Collaboration diagram with a more limited Communication diagram.

2.0 added two new interaction diagrams: the Interaction Overview diagram and the Timing diagram.

2.0 added the Protocol State Machine.

2.0 added the Composite Structure diagram

2.0 isolated the Activity diagram with its own semantics separate from the State Machine.”

Requirment Management Tools

2010-10-29T22:45:00.033-07:00

From a Gartner report: Agile Requirements Definition and Management Will Benefit Application Development

Principal Tools:

IBM Rational RequisitePro

Borland CaliberRM

Serena Requirements & Traceability Management

Telelogic Doors

Less Known Tools:

Apptero – Apptero 2004

Axure Software Solutions Rapid Prototyper

Compuware Reconcile (with QACenter, DevPartner)

Goda Software Analyst Pro

iRise Application Simulator

MKS Requirements 2005 (with Integrity Manager)

Sofea Profesy

SpeeDev – SpeeDev RM

SteelTrace Catalyze

TCP Integral Requisite Analyzer

Others:

3SL Cradle

UGS Teamcenter

ViewSet Pace

Vitech Core

requirement engineering tools

2010-10-29T22:45:00.032-07:00

iRise
-iRise Studio: requirement development environment
-iRise Manager: middleware to manage requirements
-iRise Reader: client

RequisitePro

-Dynamic integration between Word and requirement database
-Integration with other IBM software development tools, for coding, testing, and mantanence
-Distributed access to requirement repository (not sure about the detailed mechanisms)
-Traceability and coverage analysis (among requirements)
-Impact of requirement changes (among requirements)
-Query capabilities based on attributes
-Requirement audit trail
-Central requirement repos
-Controlled access to repos
-Customizalbe requirement structure (define requirement types, each type has a unique set of atts)
-Requirement project templates (not sure what is diff with req types)

RequisitePro has several ways to edit requirements:
1) Use Word plugin: insert new requirements directly by selecting the plugin toolbar or the right-click menu.
2) use requirement editor
3) use requirement viewer

Because all requirements are stored in the database, searching and revision control have been taken care of easily.

Coverage anaylysis uses queries to create views that shows the relationship between requirements, requirement and
use cases, etc.

Change impact analysis: my understanding of how this works. Links are set up between a requirement and a use case,
for example. When the text of the requirement changes, the link will be marked as a suspect link.

RequisiteWeb: a web interface to the requisitePro
see demo (http://www3.software.ibm.com/ibmdl/pub/software/rational/web/demos/viewlets/reqpro/RequisiteWeb_V2002_viewlet.html)

In RequisitePro we can create four types of items:

1)Package: just like a folder to group things together
2)Document: 5 templates are provided: glossary, requirement management plan, supplementary requirement specification,
use case specification, vision
3)View: attribute matrix, traceability matrix, traceability tree (in or out)
4)Requirement: can have the attributes of type, priority, status, difficulty, stability, origin, contact, enhancement-request
defect (the last two are connected with clearquest tool)

CaliberRM from Borland

http://info.borland.com/techpubs/caliber_rm/

features and functions are quite similar to RequisitePro
It has a document factory to generate Word document from Word templates.
Import and export of project data support many different formats like Word doc, ASCII files, etc.

Requirement types and attributes can be customized.

Advanced version history and traceability, integrates with SCM tools like Borland StarTeam, Merant PVCS, Microsoft SourceSafe,
Rational ClearCase.

The tool integrtes with a line of Borland requirement engineering tools, e.g. estimate professional for workload/cost estimation,
datamart+BusinessObjects for advanced query/analysis to support decision making process,
Mercury quality center/testDirector.

Telelogic DOORS

1)DOORS:
2)DOORS XT: distributed requirement management
3)DOORS/Analyst: using UML diagrams to draw requirement models, a visual modeling environment
4)DOORSnet: enable Web access to the requirement management functions

DOORS:
=change tracking
=traceability
=scalability
=inherant small-scale test environment, integration with Mercury quality center/testDirectory for large-scale testing
=integration with other telelogic products

Serena Requirements Traceability Management(RTM)

Requirements authoring using Word, or visual models defined using Serena Composer, or Wb client

A uniqe polling feature that allows people to vote to make decisions

Built on Oracle DB

On-line ability to collaborate, in real-time with live data while keeping a history of that collaboration.
A true data repository that provides full traceability; with no ‘limitations’ or special proprietary scripting languages
User interfaces, Word and Web, that are industry standards and commonly used in every organization
The hybrid capability of viewing requirements data in both document form and object form
True baselining and versioning capability with full history of all changes
User designed forms to capture and manage any engineering information such as defects, enhancement requests and change requests, to name a few
icAdvisor for requirements correctness

Documents and Papers

Increasing Business/IT Relevance and Adaptability: Adopting Requirements Visualization
(a META Group White Paper)

META group estimates that 60%-80% of project failures can be attributed directly to poor
requirements gathering, analysis, and management.

config Tomcat for JNDI

2010-10-29T22:45:00.031-07:00

Had a hard time configuring Tomcat to use JNDI to access MySQL, here is how to do it. Basically, we need to add a datasource resource, then define a resource link in server.xml.

…

…
type="javax.sql.DataSource"
password="1234ge"
driverClassName="com.mysql.jdbc.Driver"
maxIdle="2"
maxWait="5000"
username="guozheng"
url="jdbc:mysql://localhost:3306/testdb"
maxActive="4"/>

…
name="localhost">

jdbc connections to DB

2010-10-29T22:45:00.030-07:00

MySQL(http://www.mysql.com)mm.mysql-2.0.2-bin.jar
Class.forName( “org.gjt.mm.mysql.Driver” );
cn = DriverManager.getConnection( “jdbc:mysql://MyDbComputerNameOrIP:3306/myDatabaseName”, sUsr, sPwd );

PostgreSQL(http://www.de.postgresql.org)pgjdbc2.jar
Class.forName( “org.postgresql.Driver” );
cn = DriverManager.getConnection( “jdbc:postgresql://MyDbComputerNameOrIP/myDatabaseName”, sUsr, sPwd );

Oracle(http://www.oracle.com/ip/deploy/database/oracle9i/)classes12.zip
Class.forName( “oracle.jdbc.driver.OracleDriver” );
cn = DriverManager.getConnection( “jdbc:oracle:thin:@MyDbComputerNameOrIP:1521:ORCL”, sUsr, sPwd );

Sybase(http://jtds.sourceforge.net)jconn2.jar
Class.forName( “com.sybase.jdbc2.jdbc.SybDriver” );
cn = DriverManager.getConnection( “jdbc:sybase:Tds:MyDbComputerNameOrIP:2638″, sUsr, sPwd );
// (Default-Username/Password: “dba”/”sql”)

Microsoft SQLServer(http://jtds.sourceforge.net)
Class.forName( “net.sourceforge.jtds.jdbc.Driver” );
cn = DriverManager.getConnection( “jdbc:jtds:sqlserver://MyDbComputerNameOrIP:1433/master”, sUsr, sPwd );

Microsoft SQLServer(http://www.microsoft.com)
Class.forName( “com.microsoft.jdbc.sqlserver.SQLServerDriver” );
cn = DriverManager.getConnection( “jdbc:microsoft:sqlserver://MyDbComputerNameOrIP:1433;databaseName=master”, sUsr, sPwd );

ODBC
Class.forName( “sun.jdbc.odbc.JdbcOdbcDriver” );
Connection cn = DriverManager.getConnection( “jdbc:odbc:” + sDsn, sUsr, sPwd );

8.DB2
Class.forName(”Com.ibm.db2.jdbc.net.DB2Driver”);
String url=”jdbc:db2://192.9.200.108:6789/SAMPLE”
cn = DriverManager.getConnection( url, sUsr, sPwd );

A dive into Web app frameworks

2010-10-29T22:45:00.029-07:00

Started to look for a good Web application framework to do my last piece of dissertation. I am quite a newbie in the world of Web app development and suddenly got swamped by the vast amount of existing frameworks. Most of these use MVC architecture–using MySQL/Postgresql as RDBMS, Hibernate/iBatis as the ORM layer, Spring to implement business logic and a presentation framework such as Spring MVC, Struts/Struts2(merge of Struts and WebWork2), JSF, Tapestry, etc.

I tried to take a look at Matt Raible’s appfuse project, which is a ready-to-use framework that provides customized stacks of existing frameworks at various layers. It’s quite cool, but still I need quite a lot of time to learn.

Then, I read about comparisons of many existing frameworks and found the new trend to favor much simpler and easy-to-use frameworks such as Wicket, Stripes, Click. After reading their documentation, quick start guide and sample applications. I decided to try Wicket.

Wicket is a bit different from those MVC frameworks in that it cleanly separates UI design (in html) and application implementation (in Java code). Also opposite to frameworks that use a lot of XML configuration files, Wicket does not use any XML configuration or Java annotations. From its rich set of live examples, ajax integration is quite good.

So, I set out and started to learn and use Wicket. First, I tried to follow its online example code. However, those code is quite outdated and its maven pom.xml file did not even work properly. I had to use mvn eclipse plugin (mvn eclipse:eclipse) to import code into eclipse. After many attempts to fix the pom file, I finally gave up. It appears to be a Wicket plugin for eclipse (Wicket Bench from Laughing Panda), however this plugin is quite primitive and showed many errors in my eclipse 3.2.2.

Then, luckily I found a wicket module for Netbeans 5.5 (module is the plugin for Netbeans). It has well-documented steps to develop wicket applications and everything is made so simple in Netbeans. I even did not have to worry about libraries and dependency configurations, etc. The bundled Tomcat 5.5.17 is launched automatically when you run the Web application and the browser page is brought out automatically as well. Nice features like HTTP monitor, log messages are also available to help.

I think now I need to explore more about Netbeans 5.5. It’s really a quite sharp and neat tool, in some aspects better than eclipse/MyEclipse 5.1.1 I am using now.

A note to configure Tomcat and use log4j 1.2 and commons-logging:
1. Shutdown Tomcat if it is currently running.
2. Download the Commons Logging package from the Apache web site (unless you already have it).
3. Copy the commons-logging.jar file from the distribution into your Tomcat common/lib directory.
4. Download the Log4j package from the Apache web site (unless you already have it).
5. Copy the log4j.jar file from the distribution into your Tomcat common/lib directory.
6. Create a log4j.properties file in your Tomcat common/classes directory (see next section).
7. Restart Tomcat.

Sample log4j.properties file:
#

Configures Log4j as the Tomcat system logger

#

#

Configure the logger to output info level messages into a rolling log file.

#
log4j.rootLogger=INFO, R

#

To continue using the “catalina.out” file (which grows forever),

comment out the above line and uncomment the next.

log4j.rootLogger=ERROR, A1

Configuration for standard output (”catalina.out”).

#
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
#

Print the date in ISO 8601 format

#
log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p %c - %m%n

#

Configuration for a rolling log file (”tomcat.log”).

#
log4j.appender.R=org.apache.log4j.DailyRollingFileAppender
log4j.appender.R.DatePattern=’.'yyyy-MM-dd
#

Edit the next line to point to your logs directory.

The last part of the name is the log file name.

#
log4j.appender.R.File=/usr/local/tomcat/logs/tomcat.log
log4j.appender.R.layout=org.apache.log4j.PatternLayout
#

Print the date in ISO 8601 format

#
log4j.appender.R.layout.ConversionPattern=%d [%t] %-5p %c - %m%n

#

Application logging options

log4j.logger.org.apache=DEBUG

log4j.logger.org.apache=INFO

log4j.logger.org.apache.struts=DEBUG

log4j.logger.org.apache.struts=INFO

Some Hibernate Lessons

2010-10-29T22:45:00.028-07:00

I have started using Hibernate in my project and it turns out to be more difficult for a beginner than I thought, especially when the domain model is a bit complicated. But I guess it is worthwhile to do the learning and fix the problems along the way because it will be even more problematic if I choose to brew a layer like that by myself.

So, here are some of the lessons I learned.

1) Inheritance: I chose the 3rd strategy, subclass per table because it is the most beautiful solution to my problem. The lesson I learned is that I don’t need to add an extra ID to the subclass since it will use the parent class’s ID anyways. But for the database table schema, you should have an ID for the subclass table.

2) One-to-many mapping: it is a bit similar to 1) except that now it is a foreign key reference that you don’t need to put into the many-to-one end POJO definition. For example, we have a parent p and a child c, p contains a set of c and c contains a single p, so this is a bi-directional one-to-many mapping. In the tablec, there should be a column like pid that references id column in tablep. But you don’t need to have a property called pid in the c mapping file. Otherwise, you will see duplicated mapping error. You don’t need to have an attribute p_id with getter and setter defined in c POJO either. Otherwise, you will see a null value returned from c.getPId() method.

Here are two links for examples:

1-to-many

subclass per table

Windows 2003 Standard Server

2010-10-29T22:45:00.027-07:00

Microsoft started offering many software to students for FREE at DreamSpark. I got a copy of Windows 2003 Standard Server and installed it over the weekend. It is much faster and memory efficient than my old Windows 2000 server. But it is quite annoying to configure it as a daily workstation too. Here are some tips that I want to take a note for.

1) Add a new user: Start -> Run…, type lusrmgr.msc and this opens the user and group management. Select Users, right click and select New User, then create a new user.

If you want to add admin right to the new user, right click on the newly create user. Select properties, open Member Of tab, click Add… , in the Enter the object names to select textbox, type Administrators and hit Check Names button (it will fill the full name of admin role), then click OK.

2) IE security tuning: IE’s default security profile is set to High, and this makes is very unusable because it keeps bugging you to add virtually every website to trusted zone. Here is how to set to a lower security level: open IE, go to Tools…, go to Internet Options…, select Security tab, in “Security Level for this zone” change High to Medium and confirm the change.

3) Install Kaspersky 6 desktop version: Kaspersky is for desktop computers only and there exists server version and it is quite expensive. I have successfully installed KAV 6 on Windows 2000 server before, but installing it on Windows 2003 server SP2 is much trickier.

Modify KAV msi file using ORCA MSI editor: open KAV msi file using ORCA, then do a ctrl-F search for “MsiNTProductType=1″, replace each occurrence with “MsiNTProductType>=1″ and save the msi file. Then you can install it without any problem.

After installation and adding the license file, when you restart, Windows 2003 server will show a blue screen. Press the famous F8 and select “safe mode with command line”. I tried to select GUI safe mode, but it just didn’t respond to keyboard.

On the command line. Type “edit” command and this will give you a basic text editor, create a file called kav.reg with the following content:
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\kl1] "Start"=dword:00000001

Save it and exit the editor. Then, type “regedit /s kav.reg” to import this change to the registry. Reboot the machine using “shutdown /r” command.

Finally, no more blue screen!

4) Hardware drivers: driver support is not so good on 2003 server. I could not find a driver for my RAID IDE-SATA converter card. I should have used software to backup drivers before upgrading. Here is a good one called Driver Genius.

5) Adding “show desktop” icon to the task bar: for the newly created user, there is no show desktop icon. You can either copy a file called Show Desktop.scf from C:\Documents and Settings\Administrator\Application Data\Microsoft\Internet Explorer\Quick Launch\ to C:\Documents and Settings\\Application Data\Microsoft\Internet Explorer\Quick Launch.

Or you can directly create a new Desktop.scf in C:\Documents and Settings\\Application Data\Microsoft\Internet Explorer\Quick Launch\, here is the file content:

[Shell] Command=2 IconFile=explorer.exe,3 [Taskbar] Command=ToggleDesktop

XHTML and CSS notes

2010-10-29T22:45:00.026-07:00

Stylin with CSS

XHTML Structure

DOCTYPE declarations:

Strict:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

Transitional:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">

Frameset:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN"
"http://www.w3.org/TR/html4/frameset.dtd">

XML namespace declaration:

Content type declaration:

Symbols such as <, &:

http://htmlhelp.com/reference/html40/entities/

A simple template:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns=http://www.w3.org/1999/xhtml lang="en" xml:lang="en">

<head>
<meta http-equiv="Content-type" content="text/html; charset=iso-8859-1" />

<title>Title for your Web page</title>

</head>

<body>

</body>

</html>

Four ways of CSS declaration:

embed <style></style> section in the head element
link to external CSS file: <link href="mystylesheetprint.css" media="screen" rel="stylesheet" type="text/css">, for print, just change media="print"

in-line style, just specify at each tag, e.g. <p style="font-size: 25pt; color: red;">
use @import in <style></style> section in the head element. The only downside is that IE6 might have so called FOUC (Flash of Un-styled Content) problem, meaning the content will be momentarily displayed without CSS formatting. See more at http://bluerobot.com/web/css/fouc.asp

CSS syntax-related stuff:

Contextual selector:
this limits style to parent element, e.g. p em{color:green;} makes only those <em> within <p> elements green color.
Class selector:
e.g. p.green{color:green;} <p class="green">green text</p>, note that if multiple classes exist, the last one declared in CSS definition file wins.
Id selector:
p#green{color:green;} <p id="green">green text</p>
Difference between id and class selector is that one id value is unique to one element and id is usually used in javascript as well. So, for those styles unique to an element, use id, for styles that can be shared among different elements, use class selector.
Attribute selector:
select element based on attribute existence or values, e.g. this example add a pdf icon after links to pdf files, use [href|="foo"] to specify link names that start with "foo"
a[href$=".pdf"] {
background:transparent url(images/iconpdf.gif) no-repeat scroll right center;
padding-right:18px;
}

Unit measurement:

try to use relative units: em (from the width of a character), ex (from the height of character x), or percentage.

Colors:

many different ways to specify a color: #RRGGBB, (R%, G%, B%), or use color name. There are only 16 color names in the spec: aqua, black, blue, fuchsia, gray, green, lime, maroon, navy, olive, purple, red, silver, teal, white, yellow.

Fonts:

serif, sans-serif, monospace, fantasy, cursive, e.g. p{font-family:sans-serif;}

Round corners:

Website Performance

2010-10-29T22:45:00.025-07:00

This is a very nice book by the creator of Yslow, an addon to Firebug. It lists 14 rules for high performance websites. Very useful.

DavidHerron.com also has many resources for high performance websites.

What happens when you open a URL in your Web browser

2010-10-29T22:45:00.024-07:00

Here is a nice article touching this topic.
Since it is not that long, I simply copy&past here:

Testing Page Load Speed
Posted at 2:27 PM
One of the most problematic tasks when working on a Web browser is getting an accurate measurement of how long you’re taking to load Web pages. In order to understand why this is tricky, we’ll need to understand what exactly browsers do when you ask them to load a URL.
So what happens when you go to a URL like cnn.com? Well, the first step is to start fetching the data from the network. This is typically done on a thread other than the main UI thread.
As the data for the page comes in, it is fed to an HTML tokenizer. It’s the tokenizer’s job to take the data stream and figure out what the individual tokens are, e.g., a start tag, an attribute name, an attribute value, an end tag, etc. The tokenizer then feeds the individual tokens to an HTML parser.
The parser’s job is to build up the DOM tree for a document. Some DOM elements also represent subresources like stylesheets, scripts, and images, and those loads need to be kicked off when those DOM nodes are encountered.
In addition to building up a DOM tree, modern CSS2-compliant browsers also build up separate rendering trees that represent what is actually shown on your screen when painting. It’s important to note two things about the rendering tree vs. the DOM tree.
(1) If stylesheets are still loading, it is wasteful to construct the rendering tree, since you don’t want to paint anything at all until all stylesheets have been loaded and parsed. Otherwise you’ll run into a problem called FOUC (the flash of unstyled content problem), where you show content before it’s ready.
(2) Image loads should be kicked off as soon as possible, and that means they need to happen from the DOM tree rather then the rendering tree. You don’t want to have to wait for a CSS file to load just to kick off the loads of images.
There are two options for how to deal with delayed construction of the render tree because of stylesheet loads. You can either block the parser until the stylesheets have loaded, which has the disadvantage of keeping you from parallelizing resource loads, or you can allow parsing to continue but simply prevent the construction of the render tree. Safari does the latter.
External scripts must block the parser by default (because they can document.write). An exception is when defer is specified for scripts, in which case the browser knows it can delay the execution of the script and keep parsing.
What are some of the relevant milestones in the life of a loading page as far as figuring out when you can actually reliably display content?
(1) All stylesheets have loaded.
(2) All data for the HTML page has been received.
(3) All data for the HTML page has been parsed.
(4) All subresources have loaded (the onload handler time).
Benchmarks of page load speed tend to have one critical flaw, which is that all they typically test is (4). Take, for example, the aforementioned cnn.com. Frequently cnn.com is capable of displaying virtually all of its content at about the 350ms mark, but because it can’t finish parsing until an external script that wants to load an advertisement has completed, the onload handler typically doesn’t fire until the 2-3 second mark!
A browser could clearly optimize for only overall page load speed and show nothing until 2-3 seconds have gone by, thus enabling a single layout and paint. That browser will likely load the overall page faster, but feel literally 10 times slower than the browser that showed most of the page at the 300 ms mark, but then did a little more work as the remaining content came in.
Furthermore benchmarks have to be very careful if they measure only for onload, because there’s no rule that browsers have to have done any layout or painting by the time onload fires. Sure, they have to have parsed the whole page in order to find all the subresources, and they have to have loaded all of those subresources, but they may have yet to lay out the objects in the rendering tree.
It’s also wise to wait for the onload handler to execute before laying out anyway, because the onload handler could redirect you to another page, in which case you don’t really need to lay out or paint the original page at all, or it could alter the DOM of the page (and if you’d done a layout before the onload, you’d then see the changes that the onload handler made happen in the page, such as flashy DHTML menu initialization).
Benchmarks that test only for onload are thus fundamentally flawed in two ways, since they don’t measure how quickly a page is initially displayed and they rely on an event (onload) that can fire before layout and painting have occurred, thus causing those operations to be omitted from the benchmark.
i-bench 4 suffers from this problem. i-bench 5 actually corrected the problem by setting minimal timeouts to scroll the page to the offsetTop of a counter element on the page. In order to compute offsetTop browsers must necessarily do a layout, and by setting minimal timers, all browsers paint as well. This means i-bench 5 is doing an excellent job of providing an accurate assessment of overall page load time.
Because tests like i-bench only measure overall page load time, there is a tension between performing well on these sorts of tests and real-world perception, which typically involves showing a page as soon as possible.
A naive approach might be to simply remove all delays and show the page as soon as you get the first chunk of data. However, there are drawbacks to showing a page immediately. Sure, you could try to switch to a new page immediately, but if you don’t have anything meaningful to show, you’ll end up with a "flashy" feeling, as the old page disappears and is replaced by a blank white canvas, and only later does the real page content come in. Ideally transitions between pages should be smooth, with one page not being replaced by another until you can know reliably that the new page will be reasonably far along in its life cycle.
In Safari 1.2 and in Mozilla-based browsers, the heuristic for this is quite simple. Both browsers use a time delay, and are unwilling to switch to the new page until that time threshold has been exceeded. This setting is configurable in both browsers (in the former using WebKit preferences and in the latter using about:config).
When I implemented this algorithm (called "paint suppression" in Mozilla parlance) in Mozilla I originally used a delay of 1 second, but this led to the perception that Mozilla was slow, since you frequently didnt see a page until it was completely finished. Imagine for example that a page is completely done except for images at the 50ms mark, but that because you’re a modem user or DSL user, the images aren’t finished until the 1 second mark. Despite the fact that all the readable content could have been shown at the 50ms mark, this delay of 1 second in Mozilla caused you to wait 950 more ms before showing anything at all.
One of the first things I did when working on Chimera (now Camino) was lower this delay in Gecko to 250ms. When I worked on Firefox I made the same change. Although this negatively impacts page load time, it makes the browser feel substantially faster, since the user clicks a link and sees the browser react within 250ms (which to most users is within a threshold of immediacy, i.e., it makes them feel like the browser reacted more or less instantly to their command).
Firefox and Camino still use this heuristic in their latest releases. Safari actually uses a delay of one second like older Mozilla builds used to, and so although it is typically faster than Mozilla-based browsers on overall page load, it will typically feel much slower than Firefox or Camino on network connections like cable modem/modem/DSL.
However, there is also a problem with the straight-up time heuristic. Suppose that you hit the 250ms mark but all the stylesheets haven’t loaded or you haven’t even received all the data for a page. Right now Firefox and Camino don’t care and will happily show you what they have so far anyway. This leads to the "white flash" problem, where the browser gets flashy as it shows you a blank white canvas (because it doesn’t yet know what the real background color for the page is going to be, it just fills in with white).
So what I wanted to achieve in Safari was to replicate the rapid response feel of Firefox/Camino, but to temper that rapid response when it would lead to gratuitous flashing. Here’s what I did.
(1) Create two constants, cMinimumLayoutThreshold and cTimedLayoutDelay. At the moment the settings for these constants are 250ms and 1000ms respectively.
(2) Don’t allow layouts/paints at all if the stylesheets haven’t loaded and if you’re not over the minimum layout threshold (250ms).
(3) When all data is received for the main document, immediately try to parse as much as possible. When you have consumed all the data, you will either have finished parsing or you’ll be stuck in a blocked mode waiting on an external script.
If you’ve finished parsing or if you at least have the body element ready and if all the stylesheets have loaded, immediately lay out and schedule a paint for as soon as possible, but only if you’re over the minimum threshold (250ms).
(4) If stylesheets load after all data has been received, then they should schedule a layout for as soon as possible (if you’re below the minimum layout threshold, then schedule the timer to fire at the threshold).
(5) If you haven’t received all the data for the document, then whenever a layout is scheduled, you set it to the nearest multiple of the timed layout delay time (so 1000ms, 2000ms, etc.).
(6) When the onload fires, perform a layout immediately after the onload executes.
This algorithm completely transforms the feel of Safari over DSL and modem connections. Page content usually comes screaming in at the 250ms mark, and if the page isn’t quite ready at the 250ms, it’s usually ready shortly after (at the 300-500ms mark). In the rare cases where you have nothing to display, you wait until the 1 second mark still. This algorithm makes "white flashing" quite rare (you’ll typically only see it on a very slow site that is taking a long time to give you data), and it makes Safari feel orders of magnitude faster on slower network connections.
Because Safari waits for a minimum threshold (and waits to schedule until the threshold is exceeded, benchmarks won’t be adversely affected as long as you typically beat the minimum threshold. Otherwise the overall page load speed will degrade slightly in real-world usage, but I believe that to be well-worth the decrease in the time required to show displayable content.

HTTP Notes

2010-10-29T22:45:00.023-07:00

From Best Practices for Speeding Up Your Web Site

1. Redirection

HTTP/1.1 301 Moved Permanently
Location: http://example.com/newuri
Content-Type: text/html

2. About ETag

Entity tags (ETags) are a mechanism that web servers and browsers use to determine whether the component in the browser’s cache matches the one on the origin server. (An "entity" is another word for what I’ve been calling a "component": images, scripts, stylesheets, etc.) ETags were added to provide a mechanism for validating entities that is more flexible than the last-modified date. An ETag is a string that uniquely identifies a specific version of a component. The only format constraints are that the string be quoted. The origin server specifies the component’s ETag using the ETag response header.
      HTTP/1.1 200 OK
      Last-Modified: Tue, 12 Dec 2006 03:03:59 GMT
      ETag: "10c24bc-4ab-457e1c1f"
      Content-Length: 12195
Later, if the browser has to validate a component, it uses the If-None-Match header to pass the ETag back to the origin server. If the ETags match, a 304 status code is returned reducing the response by 12195 bytes for this example.
      GET /i/yahoo.gif HTTP/1.1
      Host: us.yimg.com
      If-Modified-Since: Tue, 12 Dec 2006 03:03:59 GMT
      If-None-Match: "10c24bc-4ab-457e1c1f"
      HTTP/1.1 304 Not Modified
The problem with ETags is that they typically are constructed using attributes that make them unique to a specific server hosting a site. ETags won’t match when a browser gets the original component from one server and later tries to validate that component on a different server, a situation that is all too common on Web sites that use a cluster of servers to handle requests. By default, both Apache and IIS embed data in the ETag that dramatically reduces the odds of the validity test succeeding on web sites with multiple servers.
The ETag format for Apache 1.3 and 2.x is inode-size-timestamp. Although a given file may reside in the same directory across multiple servers, and have the same file size, permissions, timestamp, etc., its inode is different from one server to the next.
IIS 5.0 and 6.0 have a similar issue with ETags. The format for ETags on IIS is Filetimestamp:ChangeNumber. A ChangeNumber is a counter used to track configuration changes to IIS. It’s unlikely that the ChangeNumber is the same across all IIS servers behind a web site.
The end result is ETags generated by Apache and IIS for the exact same component won’t match from one server to another. If the ETags don’t match, the user doesn’t receive the small, fast 304 response that ETags were designed for; instead, they’ll get a normal 200 response along with all the data for the component. If you host your web site on just one server, this isn’t a problem. But if you have multiple servers hosting your web site, and you’re using Apache or IIS with the default ETag configuration, your users are getting slower pages, your servers have a higher load, you’re consuming greater bandwidth, and proxies aren’t caching your content efficiently. Even if your components have a far future Expires header, a conditional GET request is still made whenever the user hits Reload or Refresh.
If you’re not taking advantage of the flexible validation model that ETags provide, it’s better to just remove the ETag altogether. The Last-Modified header validates based on the component’s timestamp. And removing the ETag reduces the size of the HTTP headers in both the response and subsequent requests. This Microsoft Support article describes how to remove ETags. In Apache, this is done by simply adding the following line to your Apache configuration file:
      FileETag none

3. HTTP Status Code
Those codes can be found here

Javascript Notes

2010-10-29T22:45:00.022-07:00

Found several nice short tutorials for JS:

by Sergio Pereira

by fallenlord blog

Updated High Performance Website Tips

2010-10-29T22:45:00.021-07:00

Latest presentation from Yahoo! Exceptional Performance Group

| View | Upload your own

Regex for javascript to detect a URL

2010-10-29T22:45:00.020-07:00

var regex = /\b([\d\w\.\/\+\-\?\:]*)((ht|f)tp(s|)\:\/\/|[\d\d\d|\d\d]\.[\d\d\d|\d\d] \.|www\.|\.tv|\.ac|\.com|\.edu|\.gov|\.int|\.mil|\.net|\.org|\.biz|\.info|\.name|\.pro |\.museum|\.co)([\d\w\.\/\%\+\-\=\&\?\:\\\"\'\,\|\~\;]*)\b/gi;

Online tester for javascript regex:

http://www.regular-expressions.info/javascriptexample.html

Short tutorial for regex in javascript:

http://www.regular-expressions.info/javascript.html

Firefox 3 Beta

2010-10-29T22:45:00.019-07:00

I run three browsers on my computer with Opear the most frequently used one. Firefox sometimes consumes too much memory, either because of the memory leak problem or its cache implementation.

Firefox betas are out for quite sometime and I tried its latest 3.0pre from nightly build trunk. It is much faster and polished and I really love it. The only problem is that most plugins are not officially compatible with Firefox 3 betas right now.

There are several FF plugins that I cannot live without:

Greasemoney:

this animal allows you to write customized scripts to control the actual rendering of any page. One most useful script is Linkify ting, it automatically turns text URLs into clickable links by adding <a> tags. But the original script has bugs that it cannot handle many types of URLs. So, what you can do is to open that script and replace "var regex = blah blah" with the following:

"var regex = /\b([\d\w\.\/\+\-\?\:]*)((ht|f)tp(s|)\:\/\/|[\d\d\d|\d\d]\.[\d\d\d|\d\d]\.|www\.|\.tv|\.ac|\.com|\.edu|\.gov|\.int|\.mil|\.net|\.org|\.biz|\.info|\.name|\.pro|\.museum|\.co)([\d\w\.\/\%\+\-\=\&\?\:\\\"\’\,\|\~\;]*)\b/gi;".

There are also many scripts on UserScripts.org including many hacks on iGoogle and Gmail. There are also tutorials like Dive into Greasemonkey and books like Greasemonkey Hacks if you want to write your own scripts. It is really a lot of fun.

Firebug:

this is the ultimate toolbox for Web developers, it includes many powerful features to detect JavaScript errors, and design your XHTML and CSS files. There is another addon to Firebug called YSlow! from Yahoo extreme performance group that evaluates a Web site based on several performancing-improving guidelines.

Del.icio.us:

this is the Web-based bookmark solution. Although they have a new version that integrates with Firefox bookmark manager, I still like the classic version that adds to buttons on the browser toolbar better.

So, how to make these plugins working with FF3.0 betas? It is quite straightforward actually.

1. Manually download XPI file: Instead of click install button on these plugin website (actually, those buttons are greyed out and you cannot click at all), you go to the bottom of a plugin page where there is a link for "advanced details", expand that section and click on "complete version history", then download the xpi file manually using "save as…".

2. Open XPI file with WinRAR and edit install.rdf file: in target application element, there is an element called <em:maxVersion>, just change the value to 3.0pre. Save the install.rdf and put it back to the XPI file.

3. In FF3, use "open file…" to open the modified XPI file. Bingo! You got those old buddies back

For del.icio.us plugin, there is a bit more extra work due to the signature files. FF3 will have some error for "signing could not be verified" error. Simply open the XPI file with WinRAR and delete META-INF folder and save the XPI file. Here we go, del.icio.us buttons back too!

Java Garbage Collection

2010-10-29T22:45:00.018-07:00

A short article about different types of garbage collectors in Java

Never know that Java could have memory leak problem before since I believed that GC will do all the trick to reclaim unreachable objects. But actually it does due to "unintentional object retention". This is a good article explaining about this problem and providing suggestions to deal with it using things like WeakHashMap (finally I understand what this class is useful for).

OpenSUSE 10.3

2010-10-29T22:45:00.017-07:00

It’s been quite some time since I last used my OpenSUSE (I have a dual-boot machine and do most of my research projects using Windows tools). Now I am looking for a job and UNIX familiarity seems a precious resource. So… I switched back to OpenSUSE. I have been using SUSE since version 8. I tried Ubuntu again and again, the device driver problems bited me again and again while OpenSuSe always came to my rescure.

Here are something I did to set up the OpenSUSE 10.3 box as my daily development machine.

1) Install Sun J2SE. The default gcj is cool but I’d prefer Sun’s JDK more. Here is a reasonably good tutorial to set up J2SE: http://fedorasolved.org/browser-solutions/sun-jdk. Besides using update-alternatives to add java, I also added javac in a similar manner.

There is one thing that I found incorrect in this tutorial: if you add his java.sh into /etc/profile.d/, this messed up the PATH variable somehow and gdm even cannot start correctly. So, instead I added these to my ~/.profile:
export JAVAHOME=/opt/jdk1.6.006 export PATH=$PATH:$JAVAHOME/bin export MANPATH=$MANPATH:$JAVAHOME/man

Install Netbeans (as of writing, 6.1 is still RC2). Netbeans 6 caught up very quickly to Eclipse and it did a great job to integrate lots of useful things in nice packages. Since it also has good support for C/C++ development and Ruby, I decided to use it on OpenSuSe.

However, no matter what I did, there are always problems during installation. First, even though there is only one installation wizard running, it always says “there is another installation instance running, are you sure you want to start a new instance?”, yes, of course. Then it dies and Bug-Buddy, a GUI-based bug reporting application shows up and installation wizard simply crashes. I found someone having the same problem and his solution is to uninstall Bug-Buddy and try installation again, it worked!!!

BTW, you can choose to download and install JDK together with Netbeans. In this case, you still need to add Java plugin to Firefox and update alternatives for java and javac.

Install Aptana Studio. This is my current Web application IDE. It support ajax, PHP, RoR, AIR, and iPhone development, covering all major ajax libraries you will find today. And its community version is FREE! You can forget about Dreamweaver, Frontpage, M$ Expression Web, etc. Well, if you are into ASP.NET development, probably you still need VS.NET though.

Add several software update sources (I installed OpenSUSE using a live Gnome CD): OpenSUSE non-open source and open source, Mozilla, Gnome stable, etc. With these update sources, you can install software unavailable to the live Gnome installation CD.

Update Firefox to the latest stable version. It is still 2.0.0.x as of now. I have been using FF3pre, code named Minefield for sometime on Windows now and it simply blew me away with performance and new features like smart URL address field.

Install these plugins for FF: Greasemonkey, BetterGmail2, Firebug, Session Manager, del.icio.us.

Update pidgin, the IM application for linux, to latest stable version. Activate two plugins “historys” and “conversion colors”. The default font size is 12, too big for me and I changed it to 10. The default theme is quite ugly, especially the system tray icon. So, you can find a Pidgin theme on gnome-look made by Embrace.

Go download this Human Pidgin Theme (I guess it is made for Ubuntu based on colors) from here.

Then, login as root. Unpack the package and you will see two directories: pidgin and Style. Use pidgin to replace /usr/share/pixmaps/pidgin directory, which contains original theme. Make a backup if you want to roll back later. The Style directory is a bit confusing and it actually adds a background image. To use it, simply copy pidgin_bg.jpg and .gtkrc-2.0 to your home directory.

Now, start pidgin and you will see a more polished interface.

Install new login window themes, gnome application window themes and mouse icon themes. I like Mac themes but don’t have the money right now to buy a Macbook Pro, so I can tune my OpenSUSE at least look similar.

Application Themes: Go to http://art.gnome.org/themes/gtk2/ first using Firefox. Then, open “Appearance Preference” window (computer->control center->Appearance (under look and feel). Then, just drag a theme to your “Appearance Preference” window and it will get installed automatically. Then, choose that newly installed theme. I am using Glossy P theme and it looks like Mac style.

Login Themes: login as root first. Type “gdmsetup” on the terminal and this opens up gdm settings. In “Local” tab, you will see currently installed login themes. Choose one you like.

Or, go to http://art.gnome.org/themes/gdm_greeter/, download a new login theme and install it in gdmsetup window. It says you can drag and drop to install, but I did not succeeded somehow and choosed “Add…” to manually install the themes. I am using “Sunergos Blue GDM Theme” and I like it a lot, simple and beautiful.

Mouse themes: you can download a theme pack from gnome-looks.org and unpack it to ~/.icons. For example, mine icon pack name is Obsidian. Then, after you unpack it into .icons directory, it becomes ~/.icons/Obsidian, rename it to ~/.icons/default and logout and login to see the new effect. I am using a theme called “ShereKhanX” to match the Mac window theme.

You can also customize splash screen, but I will leave it for later.

OpenSUSE 10.3 - Part 2

2010-10-29T22:45:00.016-07:00

There is one last software that I need to use daily, an email client. I have been using Thunderbird for quite some time on Windows and I am very happy about its interface and performance.

Our school email server gets quite slow sometimes and I use Alpine text-based email client. This is the open source version of the popular Pine email client on UNIX and people ported it to various other platforms including Windows. It is lightweight and has a great performance.

So, which one to use on OpenSUSE? I am kinda lazy and copied over my .pinerc from Windows home directory to OpenSUSE home directory. The source code from Alpine website does not compile on my machine, complaining some strange problems and I directly downloaded a copy from here. It installed very nicely and with my existing .pinerc, all my settings from Windows are back into action.

However, I had a problem with “backspace” key. Instead of deleting the character before it, it deletes the character after it, which is the same behavior as delete key. This seems to be a known issue for Linux keyboard mapping and the correct “backspace” behavior is ctrl+H. I tried to set default editor to nano and avoid using pico, the default editor that comes with Alpine, but it did not work.

So, I had to choose another email client. Since Evolution is the default email client, I decided to give it a try. But very disappointedly, the performance really sucks and it took very long to render the interface as well as email content. Although it has nice features like notes, schedule, contacts, etc., I gave it up right away.

Finally, I installed and configured Thunderbird. World is in peace now. It runs smoothly and grabs content very fast. To emulate the Mac style, I also installed CrossOver X theme for Thunderbird. There are some other nice Mac-related themes such as CrossOver and Cutebird. Now, it feels that I am using a Mac now, haha!

=====================

Since I use English and Chinese at the same time, I need a Chinese input tool. On WindowsXP I use Google Pinyin and on Win2003 Server I use sogo.

SCIM is one of the language input platform available and it supports a wide variety of input languages. Someone ported FIT Chinese input tool onto linux and it is called Fitx. It has a sister WeFit, which is the most popular Chinese input software on iPhone.

To install Fitx with least hassle, go to openSUSE software search engine and search for fitx, then click “1-click install” button. When the installation is finished, log out and log in to see it in action. You can configure hot keys to activate and deactivate Fitx.

OpenSUSE extreme makeover

2010-10-29T22:45:00.015-07:00

First, SCIM (fitx) did not work out well for me. It works but even after I have completed Chinese input, the floating window still dangles around and giving me suggestions for the same word over and over again. Plus, I heard people say that some applications might crash SCIM input tool.

So, I simply uninstalled SCIM and got Fcitx, a lightweight input tool. This time, everything worked out very beautifully. The easiest way to install is from OpenSUSE online search and use “one-click install”. The homepage for Fcitx is here and people are really saying good words about it.

============= Make your OpenSUSE look like Mac ================

There are many tutorials from Google and this is one of the best I found that treats every aspects including bootup screen, GDM login window, GDM splash picture, window theme, Compiz/Fusion theme, Firefox theme, pidgin theme, music player theme, wallpapers, etc. This is an open source project called Mac4Lin, really solid job!

I also tried the docking application Avant Window Navigator, which emulates Mac dock for application launchers. Maybe I did not configure it right, it only shows icons representing active application windows, not application launchers that you can use to open applications. So, I did not install this docking application.

Other parts are perfectly fine and I am quite satisfied with the look and feel of my OpenSUSE now. Maybe someday when I save enough money, I will go get a Macbook Pro…

instanceof OR getClass()

2010-10-29T22:45:00.014-07:00

First, wanna share a nice vim/gvim theme called Wombat. Just download it and put it into $HOME/.vim/colors and setcolors wombat in your $HOME/.vimrc and .gvimrc.

I have seen several interview questions asking you to override equals() method for your class. There are two ways to check the classname. Take a Foo class for example:
public class Foo{ private int id; public boolean equals(Object o){ if (this == o) return true; if (!(o instanceof Foo)) return false; if (o == null) return false; Foo other = (Foo)o; return (this.id == other.id) ? true : false; }

Another way:
public class Foo{ private int id; public boolean equals(Object o){ if (this == o) return true; if (getClass() != o.getClass()) return false; if (o == null) return false; Foo other = (Foo)o; return (this.id == other.id) ? true : false; }



Basically, it is a question of using instanceof or getClass(). Josh Bloch in his Effective Java and his interview with Altima argues that instanceof should be used because of Liskov substitution principle, meaning that one subclass object should be able to replace occurrences of its superclass object without making changes for correctness. 


Cay Horstmann does not agree with the use of instanceof and argues that getClass() method should be used instead to enforce the contract of equals method. He presents a superclass-subclass example where reflexive contract for equals method is broken: parent.equals(child) is true, but child.equals(parent) is false if we stick to the instanceof approach.


I think with some special conditional checks on the inheritance structure, you can still use instanceof and satisfy reflexive contract. But getClass() is much simpler.

So, basically, the question is like this: if Eric is a manager, he is represented as an object e of Employee class and an object m of Manager class which extends Employee class. Do you think e equals m (because they are actually representing a same person, using instanceof approach) or do you think they are not equals (because they are of different classes, using getClass() approach)?

imagick for XAMPP on Mac Leopard

2010-10-29T22:45:00.013-07:00

There are different options to install/use PHP on Mac Leopard: use the default installation, use MacPort to install, or use XAMPP.

I like the all-in-one bundle XAMPP, it is easy to manage and includes most commonly used options. So, I use it for the most of the time. However, when I need to install imagick, a PHP wrapper for ImageMagick, I got into some troubles.

First, this package is not available via PEAR or PECL (PECL got into some errors). So, I end up manually compile imagick and copy the imagick.so into XAMPP extension directory. Here is what I did:

change PATH to use default Leopard PHP installation, i.e. put /usr/bin ahead of XAMPP bin directory in PATH setting

use MacPort to install ImageMagick (assume you have MacPort installed, Yahoo! it if you did not know how to)

go to http://pecl.php.net/package/imagick, and find out the latest release

manually build imagick and copy to XAMPP PHP extension directory (assuming using PHP5)
curl -O http://pecl.php.net/get/imagick-2.2.2RC1.tgz tar xvfz imagick-2.2.2RC1 cd imagick-2.2.2RC1 phpize ./configure --with-imagick=/opt/local make sudo make install (we are using /usr/bin/php, so the imagick.so is created in /usr/lib/php/extensions/no-debug-non-zts-20060613/) sudo cp /usr/lib/php/extensions/no-debug-non-zts-20060613/imagick.so /Applications/xampp/xamppfiles/lib/php/php5/extensions/no-debug-non-zts-20060613/imagick.so

edit /Applications/xampp/etc/php.ini, find “Dynamic Extensions” block, add
extension=imagick.so
6. testing, add this test.php file to /Applications/xampp/htdocs and try it at http://localhost/test.php, you will see this jpg turned 45 degree clockwise (get santa-claus.jpg from here)
< ?php $img = new Imagick('santa-claus.jpg'); $img->rotateImage(’#fff’, 45); header(’Content-type: image/jpeg’); echo $img; ?>

Reference:

Install PHP5, lighttpd and Imagick on Mac OSX Leopard (talks about using MacPorts to install these things)

Mac with PHP / Imagick to install

PECL::Package::imagick

Install ImageMagick on Mac

Compiling shared PECL extensions

iPhone notes

2010-10-29T22:45:00.012-07:00

Getting interested in iPhone development after I hacked a 2G iPhone recently. Just ordered Programming in Objective-C 2.0 from Amazon, Objective-C seems to be a quite difficult yet powerful language with a lot of nice OO features.

Here are two nice tutorials from Cocoa Dev Central:

Use Imagick to create images with transparent canvas

2010-10-29T22:45:00.011-07:00

update: just found an easier way to create a transparent canvas
so, you can pretty much ignore all these options listed below now

$im = new Imagick(); $im->newImage(100, 200, new ImagickPixel('transparent')); // use this predefined transparent color string $im->setImageFormat('png32'); //$im->setImageFormat('gif'); //$im->setImageFormat('jpg'); // note that jpg does not support transparency, so you see a default black background

==== ignore the rest of the post ====

According to ImageMagick Examples canvas page, there are multiple ways to create a transparent canvas.

It is a pity that Imagick PHP library lacks enough documentation, so I had to try my luck with it. For example, I want to create a transparent background, then copy an image with transparency on top of it.

< ?php try { $icon = new Imagick('play.png');



$im = new Imagick();


// put code here to create the transparent canvas, use options below


$im->compositeImage($icon, Imagick::COMPOSITE_COPY, 0, 0);

header('Content-Type: image/png');
echo $im;

} catch (Exception $ex) { echo ‘Exception: ‘ . $ex->getMessage() . “\n”; echo $ex->getTraceAsString(); } ?>

a) setImageOpacity
$im->newImage(200, 200, '#000000', 'png'); $im->setImageOpacity(0.0);

b) paintTransparentImage
$transparent = new ImagickPixel('#000000'); $im->newImage(200, 200, '#000000', 'png'); $im->paintTransparentImage($transparent, 0, 10);

c) matteFloodfillImage
$transparent = new ImagickPixel('#000000'); $im->newImage(200, 200, '#000000', 'png'); $im->setImageBorderColor($transparent); $im->matteFloodfillImage(0.0, 0.0, $transparent, 200, 200);

d) thresholdImage
$im->newImage(200, 200, '#000000', 'png'); $im->thresholdImage(-1, Imagick::CHANNEL_ALPHA);

e) fxImage
$im->newImage(200, 200, '#000000', 'png'); $im = $im->fxImage('0', Imagick::CHANNEL_ALPHA);

For ImageMagick 6.2.9 12/17/07 Q16, Imagick 2.2.2RC1
a) does not work at all
e) gives the correct result, transparent background for play.png and transparent background for the whole image
others give transparent background for play.png, but black background for the whole image

For ImageMagick 6.4.9-6 2009-03-25 Q16, Imagick 2.2.2
a) b) gives transparent background for play.png and whole image
c) gives an exception
others give white background for play.png, black background for whole image

Images (note that you may want to save the images and open it in a viewer with non-white background to see the difference, especially the last two):

Desired ouptput: transparent play button background and transparent canvas:

White play button background and black canvas

Transparent play button background and black canvas

Transparent gif with smooth edge using Imagick

2010-10-29T22:45:00.010-07:00

If you have a PNG with transparency and then you want to convert it to a GIF with transparency preserved. Since GIF has only 1-bit transparency, the edge will look jacky. However, if you know a color that will be used as solid background color or an average color for a gradient background, you can still achieve a smooth edge for the transparent gif. Here is how (assuming the original PNG is circle.png, the edge antialias color is RED):

<?php $red = new ImagickPixel(’red’); $transparent = new ImagickPixel(’transparent’); $im = new Imagick(’circle.png’); $im->setImageBorderColor($red); $im->frameImage($red, 0, 0, 0, 0); //or use $im->borderImage($red, 0.0, 0.0); $im->paintOpaqueImage($red, $transparent, 0.0); //replacing RED color with transparent color $im->setImageFormat(’gif’); header(’Content-Type: image/gif’); echo $im; ?>

cachegrind for Mac

2010-10-29T22:45:00.009-07:00

Start to learn how to do profiling and optimization for PHP code. Found Kcachegrind is handy to visualize XDebug data. There are other similar cachegrind tools:

Kcachegrind: you can get it from Macports, check out this if you don’t know how to get started with Macports. Note this might take a long time to install. In my case, I also need to install graphviz to see the call graph.

MacCallGrind: this displays the data in a table structure.

cachegrindvisualizer: this is a google code project, installed/coded using Adobe AIR, fairly cool UI. If you only need to see the call graph, you probably don’t need Kcachegrind anymore.

webgrind: a Web UI for the cachegrind data, also shows a table-like structure.

ICO image processing with Imagick

2010-10-29T22:45:00.008-07:00

Used to have an issue with Imagick when I convert ICO image into png. Somehow when I create the Imagick object from image content string, the final result is a crappy image. But when I create the Imagick object directly from a file, it works fine, with only the problem of a different size.

So, I filed a bug (http://pecl.php.net/bugs/bug.php?id=15701), but still got no resolution.

Got some time recently and did a bit research, found that the problem is with the ICO format: it may contain the same image with different sizes. So, when you convert the format directly, it will not work. It is similar in the case when you try to convert an animated GIF image into JPG or PNG, you get one of the frames instead of an animated image.

There are two solutions:

use foreach($ico as $im) loop to get one image and just convert that image

use flattenImages function before the format conversion

Note that after the format conversion, the image dimensions might be different from when you directly open the ICO image from a browser or image viewer. You probably need to resize the image manually.

The bug report has some code snippets if you want to try it out.

Screen Sharing

2010-10-29T22:45:00.007-07:00

I have been using Mac for quite a while, but did not have much time to explore many of its features. Today, I managed to find how to do easy VNC to my linux box using Screen Share application, a hidden treasure.

First, follow the instructions here to set up the Screen Share application. Somehow I could get the advanced toolbar working.

Second, you need to set up the VNC server on your linux box. In my case, Redhat Enterprise comes with VNC server installed already. All I need is to configure it properly. Follow this nicely written tutorial to start your VNC server. There are two ways in this tutorial: use the build-in remote desktop setup, or the DIY configuration. I chose the 2nd option since it is much flexible and the first one did not really worked for me.

Third, connect to your VNC server. In my case, I just typed in hostname:5091 and clicked connect in Screen Sharing, then it starts working, really cool!

I think you should be able to do the same with a Windows machine running VNC server, but I did not try that.

Subversion and SSH setup in Netbeans 6.7 RC3 on Mac OS X

2010-10-29T22:45:00.006-07:00

Netbeans has been making a lot of progress since version 6 and the latest 6.7 RC3 is even better. For those Eclipse fans, I strongly suggest that you check out Netbeans. It feels much faster and consumes less memory. The plugins are more organized and easy to manage (I think the plugin management in Eclipse is really messy).

The PHP plugin for Netbeans is out of beta and it is quite user friendly and versatile. So, I decided to switch from PDT to Netbeans for my PHP development.

Netbeans comes with Subversion support. However, it is not so smooth to set up Subversion over SSH on Mac OS X due to the ssh-askpass problem. This is a Mac OS X issue, not a Netbeans issue.

Download and install Netbeans 6.7 RC3 (as of June 25, 2009).
The subversion that comes with Mac OS X (/usr/bin/svn, 1.4.4 (r25188)) is quite outdated and it has problems working with newer versions of the SVN server, probably due to different format of the metadata. You need to use MacPorts to install a more recent subversion (if you don’t know how to install MacPorts, check out this). Just type this in your terminal: port install subversion, it might take a white. The subversion from MacPorts is installed in /opt/local/bin, so make sure you put /opt/local/bin before /usr/bin in your PATH environment variable.
Add SVNROOT and SVN_SSH environment variables to your .bashrc:
export SVNROOT=svn+ssh://your_svn_server/ export export SVN_SSH=/usr/bin/ssh
Launch Netbeans, go to preferences->Miscellaneous->Versioning->Subversion, for “path to SVN executable file”, enter /opt/local/bin
Now, it seems like everything is ready. However, when you try to do update/diff and other operations, you get error: ssh_askpass: exec(/usr/libexec/ssh-askpass): No such file or directory. I did some Yahoo! search and the problem is that Netbeans wants to communicate with SVN server over ssh without asking your password using ssh-askpass, but it is not available. This is a general ssh issue on Mac OS X.

I found two solutions: solution 1 and solution 2. I only tried solution 2 and it worked perfectly for me. Here are the simple steps:

a) download script from here (note that I had to add .jpg since blogsome only supports image upload) and save it in /usr/libexec. Change ssh-askpass script to executable:
wget wget http://woshiadai.blogsome.com/images/sshaskpass.jpg -O ssh-askpass sudo cp ssh-askpass /usr/libexec/ssh-askpass sudo chmod a+x /usr/libexec/ssh-askpass
b) add two more environment variables:
export SSH_ASKPASS=macos-askpass export DISPLAY=:0
Finally, you can use subversion actions from inside Netbeans. You will be asked for password for the first time, then it will be remembered for future activities.

Enjoy the Netbeans!

upgrading to snow leopard

2010-10-29T22:45:00.005-07:00

Finally, I decided to do the upgrade. It was not as smooth as it looks, especially that I turned on 64 bit as default. Normally, SL runs in 32 bit mode, you need to hold 6 and 4 keys during the boot process to get into 64 bit mode. But you can also change the config to run it as default, instruction is here.

Most of the applications still runs fine, here is a few exceptions that I encountered and luckily all of them have solutions so far. Here is a list of compatibility check for SL, note that it is not the default 64 bit mode.

1. menumeter: This is a small monitoring tool that I use every day, to check network, CPU, memory usage. Good thing is it displays info on the menu bar. Old installation not working anymore, just get the latest and reinstall.

2. delicioussafari: This is a plugin for Safari that saves and opens your delicious online bookmarks. To fix the issue, just get the latest and reinstall. The new version even has a new BMW-like button

3. macfuse: This is a handy tool to help mount various file systems. Here is the fix for 32 bit SL mode, however it did not work for 64 bit mode. You can use this pre-compiled dmg for 64 bit mode. It runs great for me. You can also roll your own following the instructions here.

4. TextMate: It has lots of issues with SL, especially due to the upgrade of Ruby. First, try to get the latest release possible. But still, I am having issues with ⌘R. Textmate blog has some workarounds and I have not tried them yet, hopefully they will fix the problems in the next release.

5. MacPorts: This is another thing that caused many headache until I found this nice blog that suggests a fresh install. It worked all fine. However, occasionally, some ports will still report error of unmatched architecture. This blog also covers "Upgrading to Snow Leopard Part 1 to 7", talking about python, mysql and other upgrade tips, very handy! He also talks about building 32 bit packages using arch and lipo.

6. VMWare Fusion: The old 2.x version even won’t start in 64 bit and asks you to restart SL in 32 bit. So, you need to get the latest Fusion 3.0. The upgrade is not free.

After a week of being little white mouse running 64 bit SL. I’d say maybe running 64 bit does cause more issues than benefits. But we can debate over it. Here is a good intro if you are interested.

The best part for me is that I got back 12G free space, very interesting. It seems to be a combination of using 1000 for 1024 in calculation and cut the new OS installation size.

set up PHP and Apache on Snow Leopard

2010-10-29T22:45:00.004-07:00

It took me quite some trouble to get imagick working on the snow leopard.

First, I tried to install xampp and mamp, which are two easy ways to get PHP, apache, mysql up and running. I used to have no issue with xampp, but the snow leopard has problems with locally complied imagick.so in xampp. So, I tried mamp as well, imagick simply does not work and kept getting me "cannot find Imagick class" error.

So, I had to try to use the PHP and apache that ships with snow leopard. I simply followed the instructions in this blog post and it worked out well for the most part.

The only problem I am having is with the userdir apache module. Using the userdir module, you can just drop web applications to /Users/<username>/Sites. But no matter how hard I tried, I just could not get this working and had to give up.

UPDATE: found this post saying the permission denied problem is related to the user permission of _www. Apache runs as _www under _www group that need to have access to /Users/<username>/Sites. I’ve added xr to everyone on that directory, not sure why it is still not working. If you want to give it a try you can do it.

setting up Lua on Mac

2010-10-29T22:45:00.003-07:00

Recently, fell insterested in Lua due to some work with lighttpd. So, want to share some experience setting up an IDE-ish environment (of course, VIM is always the best editor).

1. install lua and luarocks through MacPorts: http://www.macports.org/ports.php?by=name&substr=lua

2. failed attempt setting up LuaEclipse following instructions at http://luaeclipse.luaforge.net/manual.html#luadoc

Update: Finally I got LuaEclipse set up correctly, please see my latest post for instructions.

LuaEclipse is a great IDE, however developers did not seem to be updating the project often and it does not work due to some 64bit jni library issues on Snow Leopard.

I first tried to install through update manager from Aptana Studio 2.0, got this error:

/Applications/Aptana Studio 2.0/configuration/org.eclipse.osgi/bundles/224/1/.cp/libluajava-1.1.jnilib: no suitable image found. Did find: /Applications/Aptana Studio 2.0/configuration/org.eclipse.osgi/bundles/224/1/.cp/libluajava-1.1.jnilib: mach-o, but wrong architecture
Could not initialize class org.keplerproject.luajava.LuaState

Then I tried to install a 64bit Eclipse and install the LuaEclipse through update site and manually install the plugin zip, both failed

a) through the update site, it does not work, always complaining one dependent package is missing from update site

b) download the plugin and install manually, installed fine, but cannot run, got error a specific directory is missing

Conclusion: forget about LuaEclipse

3. Install a TextMate bundle: http://netcetera.org/cgi-bin/tmbundles.cgi?bundle=%4C%75%61

TextMate is the choice of text editor for Mac. This Lua bundle worked perfectly for me. The only thing is that TextMate might be installed differently, so you may need to tweak the script to the "Library/Application Support/TextMate/Bundles" in your home directory, at least I had to do that. Also note that you need to configure the PATH variable in TextMate so that the lua executable can be found by TextMate.

If you are interested in Lua, Programming in Lua (2nd) is the best book to cover the basics from the original Lua developer.

LuaEclipse setup on Snow Leopard

2010-10-29T22:45:00.002-07:00

So, the instructions on the LuaEclipse main site won’t work for the latest Eclipse and I had many errors last time. Today, I finally got LuaEclipse 1.3.1 set up and running now. Here are the steps:

install lua and luarocks from MacPorts (sudo port install lua luarocks)
install LuaProfiler using luarocks (luarocks install luaprofiler)
download and install Eclipse 3.4.2, this version works with current LuaEclipse release
install LuaEclipse 1.3.1 from the update site, note that the official LuaEclipse installation guide only shows update site for 1.0 and 1.2. You need to use this update site: http://luaeclipse.luaforge.net/preview/update-site/macosx.carbon/, for other OS, you can use the update sites found from here
configure LuaEclipse through Eclipse Preferences > Lua, add Lua Interpreter (point it to /opt/local/bin/lua) and Lua Profiler (point it to /opt/local/var/luarocks/lib/5.1/profiler.so)
all set, you can create a new Lua project and start writing Lua and debugging, profiling it…

Interestingly, LuaEclipse developers have started working on LuaEclipse 2. Here is the installation instruction and the main site on GitHub.

Creating shadows with ImageMagick

2010-10-29T22:45:00.001-07:00

1. Shadows for an image (create a shadow using Imagick::shadowImage and overlay the original image on top)

<?php

/* Read the image into the object /

$im = new Imagick( ‘totoro.png’ );

$im->setImageFormat("png");

/ Make the image a little smaller, maintain aspect ratio /

$im->thumbnailImage( 200, null );

/ Clone the current object /

$shadow = $im->clone();

/ Set image background color to black (the color of the shadow) /

$shadow->setImageBackgroundColor( new ImagickPixel( ‘black’ ) );

/ Create the shadow /

$shadow->shadowImage( 80, 3, 5, 5 );

/ Imagick::shadowImage only creates the shadow. That is why the original image is composited over it /

$shadow->compositeImage( $im, Imagick::COMPOSITE_OVER, 0, 0 );

$content = $shadow->getImageBlob();

$shadow->destroy();

$im->destroy();

/ Display the image */

header( "Content-Type: image/jpeg" );

header(’Content-Length: ‘ . strlen($content));

echo $content;

2. Shadow for text

Note that the trick for multi-line text is to insert a new line character (\n) in your string. You can also use the rotation degree to control the layout, but make sure you use the correct size text canvas image because when you queryFontMetrics, it is based on the horizontal layout.

<?php

// overlay the text

$draw = new ImagickDraw();

$draw->setFont(’Helvetica’);

$draw->setFontSize(13);

$draw->setFillColor(new ImagickPixel(’#B8B8B8′));

$text = "HELLO\nWORLD!"; // for multi-line text, insert new line char

$textcanvas = new Imagick();

$textprops = $textcanvas->queryFontMetrics($draw, $text, true); //you need to turn multiline on, the last param

$textw = intval($textprops[’textWidth’]);

$texth = intval($textprops[’textHeight’]);

$textcanvas->newImage($textw+20, $texth+20, new ImagickPixel(’transparent’)); //give 20 pixel spaces around the text

$textcanvas->setImageFormat(’png’);

$textcanvas->annotateImage($draw, 10, $texth, 0, $text);

// create shadow

$shadow = $textcanvas->clone();

$shadow->setImageBackgroundColor(new ImagickPixel(’black’));

$shadow->shadowImage(80, 3, 5, 5);

$shadow->compositeImage($textcanvas, Imagick::COMPOSITEOVER, 0, 0);

$shadoww = $shadow->getImageWidth();

$shadowh = $shadow->getImageHeight();

// overlay the shadowed text on the given background image

$bg = new Imagick(’background.png’);

$bg->setImageFormat(’png’);

$bgw = $bg->getImageWidth();

$bgh = $bg->getImageHeight();

//$bg->compositeImage($shadow, Imagick::COMPOSITEOVER, 0, ($bgh-$shadowh));

$bg->compositeImage($shadow, Imagick::COMPOSITEOVER, 0, 0);

// let’s show the image

$content = $bg->getImageBlob();

$textcanvas->destroy();

$shadow->destroy();

$bg->destroy();

header(’Content-Type: image/png’);

header(’Content-Length: ‘ . strlen($content));

echo $content;

Somehow, I still don’t know how to adjust the line spacing between multi-lines. I also found that ImagickDraw::setFontWeight did not work for me. Anyone knows how to fix those two issues?

Reference:

1. Mikko’s blog (the best place for ImageMagick samples, who can beat the IM developer himself)

failed to run code reformat in TextMate

2010-10-29T22:45:00.000-07:00

All of a sudden, I cannot format/reformat the code in Textmate. Got this error:

PHP Warning: require(/lib/beautify.php): failed to open stream: No such file or directory in /private/tmp/temptextmate.4tD2ED on line 19

Searched around the Web and it seems to be caused by php not setting $ENV property due to the variablesorder not properly set in /etc/php.ini (assuming you are using php shipped with Mac, if you are using php from Macports, do it accordingly).

The fix is to add/update the variablesorder value to something like variablesorder = "EGPCS"

References: