order of event queue in Node.JS event loop

I was reading the book "Node.JS in Practice", in Chapter 2, Technique 14, it talks about the order of scheduling I/O events, setImmediate, setTimeout/setInterval and process.nextTick. Like in the following screenshot from the book:

However, it seem that the ordering is a bit different, from my test, it is more like: process.nextTick, setTimeout/setInterval, setImmediate.

Here is the sample test code and result (Node.JS v4.1.0):

setImmediate(function() {
    console.log("from setImmediate");
});

setTimeout(function() {
    console.log("from setTimeout");
}, 10);

process.nextTick(function() {
    console.log("from process.nextTick");
});

console.log("hello");

output:

hello

from process.nextTick

from setTimeout

from setImmediate

A retouch of JavaScript and Node

It has been more than two years since I last touched JavaScript and Node. My current projects are mostly centered around Java. But it was always fun to come back and refresh my memory and knowledge about it.

It almost feels like traveling through time machine, too many things have changed over the past two years. Node was forked into io.js and ES6 has been out and getting traction. And there are new frameworks out each week. Frontend engineers must have been having a tough life to keep up, and tons of fun as well.

Here I just keep some book notes for my reading through the book "Node.js in Action". The book itself turned out to be a bit outdated, but many basics are still valid. There is also another more recent Manning book "Node.js in Practice".

Difference between exports and module.exports
exports is a reference to module.exports, so as long as you do not reassign it to something different, they are the same.

Module search sequence
search for node_modules recursively from current directory, if not found, then search for environment variable NODE_MODULE or NODE_PATH.

Module require search sequence
Search for index.js, then inside package.json to find the "main" definition.

Flow control libraries

async: https://github.com/caolan/async

nimble: https://github.com/caolan/nimble

step: https://github.com/creationix/step

seq: https://github.com/substack/node-seq

See article for a discussion about how to handle async programming with JavaScript: How to survive asynchronous programming in JavaScript

unit test frameworks

nodeunit: https://github.com/caolan/nodeunit

mocha: http://mochajs.org

vows: http://vowsjs.org

should.js: assertion lib: https://github.com/tj/should.js/

functional test frameworks

tobi: https://github.com/LearnBoost/tobi

soda: https://github.com/LearnBoost/soda

Keep app running/restart

monit: https://mmonit.com/monit/

forever: https://github.com/foreverjs/forever

upstart: http://upstart.ubuntu.com/index.html

Node Web frameworks, so many of them

Express: http://expressjs.com

Hapi from walmartlabs: http://hapijs.com

A good intro about why Hapi was created: http://hueniverse.com/2012/12/20/hapi-a-prologue/

Kraken from paypal: http://krakenjs.com

Restify: http://mcavage.me/node-restify/

Koa, next gen Node Web framework using ES6: http://koajs.com

LoopBack: http://loopback.io

Sailsjs, MVC pattern similar to RoR: http://sailsjs.org

Meteor: https://www.meteor.com

Quite in depth comparison of Express, Koa and hapi: https://www.airpair.com/node.js/posts/nodejs-framework-comparison-express-koa-hapi

hapi vs Express: http://megagalactic.io/hapi-vs-express/

Express, hapi, Sails: http://blog.pluralsight.com/node-web-frameworks

Performance comparison of Express, hapi, and Restify: https://raygun.io/blog/2015/03/node-performance-hapi-express-js-restify/

Node.js debugging with Theseus and Brackets

Brackets is an open source Web development editor from Adobe. It is built with HTML, CSS and JavaScript with features designed to make Web app development enjoyable rather than pain:

- live edit and preview: changes are reflected in the browser immediately without reload, currently only supporting Chrome)

- inline view and edit referenced resources: you can open the CSS definition for a tag without leaving the HTML code you are working on

- smart code hint: for JavaScript, it is using the Tern engine, one of the most advanced code analysis engines for JavaScript

- quick docs from webplatform.org: inline documentation for various elements such as tag, values, etc.

- extensible architecture: has a fast growing set of extensions already and you can use the same languages you write Web app to write extensions

- best of all, starting sprint 21 release, it has a built-in Node.js process, which opens up a whole new world of features for JavaScript development

So, if you are currently using TextMate, Sublime Text 2/3, Vim, you should really give Brackets a try, it is so pleasant to work with it on Web apps.

Here is a quick demo video from nettuts+ article "A Peeak At Brackets":



There is a nice extension "Theseus" which makes debugging JavaScript Web app and Node.js app so much easier. You can watch a demo for debugging JavaScript Web app first:


Follow installation guide from the project github README:

• Install Brackets
• Use Brackets Extension Manager (File->Extension Manager or the lego-like icon on the right hand toolbar) to install Theseus, Click "Install from URL", then enter the Amazon S3 zip file URL, currently https://s3.amazonaws.com/theseus-downloads/theseus-0.2.13.zip
• Install Node.js if you haven't got that yet, then use npm to install node-theseus: npm install -g node-theseus

Now, write your app.js file, then run it using node-theseus app.js, then open that file in Brackets and you should be able to see the debug info in a very nice visual way.

To see it in action, view author's screencast here.

Enjoy!

Setting up VJET as your Eclipse Node.JS IDE

First of all, after going through the setup process and found out some limitations about VJET, I am not really sure I'd recommend anyone to use it. The Eclipse JavaScript Development Tools (JSDT) as part of Web Tools Platform (WTP) is pretty much enough for JS IDE.

Here is a quick rundown of pros and cons of VJET (I only spent 1-2 hours playing with it):

pros:
- free and backed by ebay
- relatively active (last release was 3 months ago)
- code assist for Node.JS and many other common JS libraries.

cons:
- Node.JS lib was outdated, currently from node v4 I believe
- Cause launcher issues with chrome JS debugger (this is the biggest con to me)
- does not support Eclipse 4.2 Juno yet (has some plugin dependency on jetty, might not caused by VJET itself)

For those still interested to try it out, here are the quick instructions to set it up:

1. download Eclipse Indigo (VJET does not support JUNO yet), I picked JavaScript development distro

2. install VJET plugin:
- add new update site: https://www.ebayopensource.org/p2/vjet/eclipse/
- install VJET as Eclipse plugin

3. download VJET JavaScript Type Libraries in zip files, each zip is an Eclipse project
- http://www.ebayopensource.org/p2/vjet/typelib/
- you can only download NodejsTL.zip if you don't need others (it is for node v4, last modified was 4/18/2011)

4. import NodejsTL.zip as project into the workspace
- File->Import...
- Genearl->Existing Projects into Workspace
- Select archive file->Browse to NodejsTL.zip

5. create a sample helloworld VJET project add NodejsTL project to your helloworld VJET project build path
- create a new vjet project helloworld
- select helloworld project
- right-click -> Build Path -> Configure Build Path
- under Projects tab, add NodeJSTL project to the build path

6. configure external tool to run under Node.JS (assuming your have node installed already)
- configure external tool (see screenshot)
- select the JS file to run
- run external tool and you should see output in the Eclipse console

Bonus: JSHint Plugin

The JSDT plugin provides some realtime syntax validation, but there is a really nice jshint plugin. The best part is that it allows you to do per-project checks, it even allows you to switch to use jslint instead of jshint (not sure if you would want to).

Note that JSHint plugin is not enabled by default, you need to turn it on per project:
- select the project
- open project properties, select JSHint and select JS files (see screenshot)

JSLint: don't make functions within a loop

Interestingly got this JSLint warning "don't make functions within a loop" today, scratched my head a bit and realized it might be due to the fact that function inside a loop is very prone to cause errors. You are expecting to get different values for the collections you are looping through in each iteration, but it ended up all the same as the value from the last iteration. This post explains it well with a straightforward example.

Another post suggests having functions inside the loop actually cause performance degrade as well, quite interesting experiments.

Janus with jslint Vim plugin

Nowadays, I write quite a bit of JavaScript and uses JSLint command line tool quite a lot. A co-worker recommends a nice Vim plugin that validates your JavaScript code when you are editing and when you save it.

While I was trying to install it for Janus (a Macvim clone that I uses), I ran into two issues:

1. By default, when you run rake inside the git cloned directory, the plugin gets installed into ~/.vim. For Janus, user customized plugins should go to ~/.janus, and Janus will load them automatically. For details, please check out the Customization section of the Janus documents.

So, to get around:

- create a jslint directory in ~/.janus
- edit vim plugin's Rake file, replace line 38 File.expand_path("~/.vim") with File.expand_path("~/.janus/jslint"), you get the idea.
- run rake from inside vim plugin directory and it should install into the correct Janus directory

2. The second issue is that I keep getting warning when I start trying out the plugin, something like "s:cmd" not defined. Did some poking around and it seems like ftplugin/javascript/jslint.vim is trying to find a JavaScript interpreter (line 62 to 75 for *UNIX systems) and somehow failed.

I am on Snow Leopard, which comes with an acceptable interpreter "jsc" at /System/Library/Frameworks/JavaScriptCore.framework/Resources/jsc, so not really sure why it did not work. I ended up just install Node.JS (for Mac users, highly recommend using Homebrew, just follow instructions here) and added node to my PATH. And it takes care of these errors.

Now, enjoy the jslint plugin and be a good JS developer ;-)

P.S. For people not happy with Crockford's personal styles (some styles don't make sense to me either), you can update options in ~/.jslintrc (see examples on jslint.vim site). Or simply use jshint vim plugin instead. JSHint is a more relaxed and reasonable fork of JSLint.

Sorry for going off the topic, but it is quite funny to read about why the original developer forked JSLint, especially entertaining are the comments. For example, Crockford's response to JSHint:

When asked for his "feelings on JSHint" Crockford replied "There are many stupid people in this world, and now there is a tool for them."

Unbounded function wrappers

I was reading "JavaScript Garden" (highly recommend for JS beginner/intermediate) and could not understand the concept of the "fast, unbound wrappers" for functions. See Function arguments for the example.

function Foo() {}

Foo.prototype.method = function(a, b, c) {
    console.log(this, a, b, c);
};

// Create an unbound version of "method" 
// It takes the parameters: this, arg1, arg2...argN
Foo.method = function() {

    // Result: Foo.prototype.method.call(this, arg1, arg2... argN)
    Function.call.apply(Foo.prototype.method, arguments);
};

Luckily I found this Stackoverflow post that explains the idea behind it. It took me a while to wrap my head around Function.call.apply ;-)

So, the basic idea is that we have a function defined in a class, but we want to use it without binding to specific object, maybe the concept of static method in Java? So, instead of creating an object and invoke the function on the object, we define the function as a property of the class, not on its prototype.

Big list of things to get started with frontend development

Quickly ran though "Getting Good with JavaScript", this is a very short and basic introduction to JavaScript for someone new to the language. Deep and difficult concepts are avoided or explained in an easy to understand way for the reader to explore further by him/herself.

I found these two resources in the "Further Study" section very helpful and interesting, both are from Rey Bango's blog:

1. What to read to get up to speed in JavaScript
2. The big list of JavaScript, CSS, and HTML development tools, libraries, projects and books

dealing with multiple nesting callbacks in asynchronous code

If you are annoyed that you have to write multiple nesting asynchronous callback functions when you work with Node.js, you are not alone. Getting used to the asynchronous programming style and always keep in mind everything runs in the event loop is a bit tricky at the beginning.

There are many libraries and articles that addresses this nesting callback issue. This should help you to avoid those ugly pyramid shaped code now ;-)

InfoQ: how to survive asynchronous programming in JavaScript (includes a list of libraries with comparisons and Q&A from their developers)

Stackoverflow post about how to deal with the nesting callbacks (also includes a list of libraries)

Isaacs's presentation on the topic and his library Slide (he is the author of npm)

Tim Caswell (creationix)'s series on control flow in Node (II and III can be found on the right column, other articles by the same author)

CommonJS Promises is another way trying to address this issue by defining a standard interface for interacting with the result object of asynchronous actions. Roughly speaking, a promise object is a placeholder for the value for the asynchronous action. And you can treat is as normal object, assign values to another variable, pass it around, etc. Right now, FuturesJS implements this and it is not solely for server-side JS. dojo.Deferred is another similar effort. This SitePen blog post has some easier to understand coverage about the promises.

StratifiedJS extends the JavaScript language to achieve similar goals of allowing asynchronous control flow to be expressed in a synchronous way. Very neat work indeed. This is a short overview presentation about it.

Free ebooks and tutorials for PHP, JavaScript and Ruby Beginners

Many thanks to RReadWriteWeb for these links:

• 5 Free E-books and Tutorials for Learning PHP
• 7 Free JavaScript E-books and Tutorials
• 10 Free E-books on Ruby for Beginners

what does "new" do in JavaScript

Today, a colleague asked about exactly what is the difference between using "new" and calling the function directly. Only thing I can think of is the meaning for "this" inside the function. With "new", this means the object being created, but without "new", this means the global object, either window or something else. And I am also not clear what exactly happens behind the scene.

So, did some search and found some good readings that explains this topic fairly clearly. Lots of related answers from Stack Overflow is also quite helpful:

• What is the 'new' keyword in JavaScript
• Tim Caswell's "Learning JavaScript with Object Graphs" series (perfect for visual person like me, in II, he specifically mentions about how to create objects besides using new: use Object.create and object factory)

Here is a good article that tests your understanding of "new" keyword ;-)

Now, whether or not you should use "new" is another question:

• Is JavaScript's 'new' keyword considered harmful?

Using "new" to instantiate a class has both pros (great performance) and cons (can cause nightmare if you forget to use new). Many people including Douglas Crockford discourages the use of new. A module pattern is a more preferable way. John Resig also has a simple solution to avoid using new directly without much performance sacrifice.

quiz time!

If you are confident about your JavaScript knowledge, try out these quizzes, it just made my brain stop working...

• "So, you think you know JavaScript?", answers explained by Zakas
• "Another JavaScript Quiz" by James Padolsey
• "JavaScript Quiz" from Perfection kills (exploring JavaScript by example)

Now, if you feel how twisted/unreasonable/difficult/loose JavaScript syntax is, you are not alone.

Several good readings that might help to clear things a bit:

• Zakas explained a bit about the first quiz

It is interesting that a variable can be declared after it is being used because the engine will hoist the variable declarations to the top of execution context.  Also interesting to see that the engine breaks "var i = 1" into declaration part "var i;" and initialization part "i = 1;", and move the declaration to the top.

Another interesting note is that function declaration overrides a variable declaration with the same name, but cannot override the variable declaration+initialization.

• Understanding delete from Perfection kills

Really a comprehensive explanation about how execution context, scope, variable attributes, etc. work in JavaScript. Now, finally I understand the difference between variable declaration+initialization (cannot be deleted) and undeclared assignment (can be deleted) because of the former one has DontDelete flag and the latter one does not.

After reading this article, it is now apparent why the function declaration has strange override behavior for the variable with a same name. The variable declaration+initialization gets a DontDelete flag and thus cannot be overridden.

• Named function expressions demystified

This one explained what is function declaration, function expression and named function expression and bugs handling named function expression in various JavaScript engines. One note is that for the named function expression, the name/identifier can only be accessed in its function body.

• ECMA-262 3rd edition specification

The ultimate reference for underlying mechanisms of JavaScript.

what this means in JavaScript

As someone new to JavaScript programming, there are many language features that are so uncomfortable. What *this* really means and referring to is one of the pains.

Finally, after reading some nice articles, I think I start to get the idea: it basically means the context/scope the function is called and can change at runtime.

Two great articles that explains this clearly:

• The basics about this from quirksmode
• A more in-depth look at this and how to use it in real world from DWM

Execute JavaScript from TextMate

TextMate is the most popular text editor for frontend development (of course, people also use emacs, vim/gvim/macvim, Aptana, Eclipse, etc.). I have been using TM for PHP development and really enjoyed the convenience of ctrl+shift+R to execute the PHP CLI scripts and view the output from TM.

I know most of people develop and debug JavaScript from development tools provided by the browser, e.g. firebug, developer tools in Chrome, Safari, etc. But it will be really neat if I can just run JavaScript and view output directly from TM.

So, I searched and searched. There are many existing blog posts on this topic already. It basically consists of two steps:

1. get a JavaScript engine that executes the JavaScript code, here are some options:

- Mac/Safari has a JS engine (SquirrelFish Extreme) built in already, it is located at /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc

- Node.js: Node.js is a event-based JS platform based on the V8 JS engine, see my previous post on how to set it up

- Mozilla JS engines: installation guide to build/install SpiderMonkey, Rhino, TraceMonkey on Mac

- For other JS engines, do a Web search for installation instructions

2. define new TM commands for JavaScript in TM bundle editor and assign ctrl+shift+R key binding to the commands.

For command setup to use Node.js, see this github snippet by beastaugh, it basically creates a script that executes another script given the file path.

For command to set up other engines, you specify the path to the engine executable, execute the JavaScript and output in a popup window. You can also use another command and open up a terminal to show the results, see this post for how to do that.

Then, you are all set. This is a screen shot that allows me to pick one of the three engines to run my JavaScript from TM ;-)

to var or not to var

It's a quite interesting and yet confusing question:

What is the difference between declaration of a variable with and without "var"?


After reading the question and answers on Stack Overflow, I finally start to get it.


First, it affects the scope of the variable.

The variable defined without var means the nearest scope in the case of a hierarchical structure, e.g. embedding one function inside another function. In most cases, a declaration without var in a function means global scope (in many cases this means window object). The variable defined with var is bound to the scope of that function.

Second, it affects the delete behavior.

The variable defined with var cannot be deleted. The variable defined without var (actually, this is a property, not a variable in the true sense) can be deleted. The reason behind this that variable defined with var has a flag of DontDelete. See this nicely explained Stack Overflow post for details.

Lastly, the general guide is to always use var.

var i = 'hello';
j = 'world';
print("i=" + i + ", j=" + j); // i=hello, j=world
(testFunc = function() {
 var i = 'foo';
 print("inside function: i=" + i); // inside function: i=foo
 print("inside function: j=" + j); // inside function: j=world
})();
delete i;
print("i=" + i); // i=hello, ha, i still exists!
delete j;
print("j=" + j); // ReferenceError: j is not defined, j got deleted

Ok, one last exercise before I close this post. Guess what is the output for this code snippet:

func2_var = 'var outside';
func2_var2 = 'var2 outside';
func2 = function() {
 var func2_var = 'var in func2 body';
 this.func2_var2 = 'var2 in func2 this';
}
func2.prototype.func2_var = 'var in func2 prototype';

f2 = new func2();
print("which var it is: " + f2.func2_var);
print("which var2 it is: " + f2.func2_var2);

The END.

Lint for JavaScript

Lint tools are very useful to detect syntax and style errors in your code. It of course has variants for JavaScript. There are many different tool setup based on your editor. On the Mac, maybe the most popular/handy editor is TextMate.

1. jslint.com: a Web tool

This is the tool developed by Douglas Crockford. It has a simple Web interface. You copy and paste your code and click JSLint button, it is that simple. The drawback is every time you make a change, you need to re-copy-n-paste and repeat the validation.

2. TextMate bundles:

johnmuhl's javascript tools tmbudle
It provides a toolkit for working with JavaScript in Textmate: JSLint, various formatters, obfuscators, and compressors.

subtleGradient's javascript tools tmbundle
This is very similar to the previous tmbundle except that it uses JavaScript Lint instead of JSLint. Also, the timestamp seems more up-to-date. The installation is very straightforward, download the src, rename it to be a TextMate bundle file and let TextMate install it. Then, you access it via various key combinations.

3. TextMate integration with JSLint:

Stoyan has a nice post with all the detailed instructions. It is a bit complicated than using method #2.

4. Vim integration with JSLint:

Only saw some discussions on Stack Overflow, did not actually try it out.

JavaScript links

Saw this interesting discussion about whether we should use href="javascript:void(0)" or href="#" with onClick event handler on Stack Overflow.

I think the best solution is to avoid using link element <a> in the first place and use CSS to style an element to make it "look" like a link and attach the event handler to that element. This article has more discussions and comparisons why we should not use links in this particular case.

As for why the event handler need to return false, it is because we want to avoid the default behavior of the  link element, i.e. to follow the href and go to another page. This quirksmode article has it very clearly explained.

node.js setup on Mac

node.js has been getting a lot tractions lately due to its high performance on V8 engine, event-based model and reusability of client side JavaScript on server side without any changes.

There are several articles discussing how to set it up on the Mac and I found this one from Florian Kubis to be the easiest to follow.

If you are using homebrew as your package management system instead of MacPorts, you should use this tutorial instead.

The tutorials also cover npm (the package manager for node.js) and express (a Web app framework built for node.js)

There is no MacPorts ports for npm and express yet, maybe someone should create them?

Several slides and quick intro about node.js:

1. "Evented I/O based web servers, explained using bunnies", the idea behind node.js in 10 pages
2. "Node.js and websockets intro", a bit more technical than the bunnies example
3. "Node.js: how JavaScript is changing server programming", more details in 65 pages
4. "Running YUI3 on node.js", integration with YUI3, moving client js to server side without any changes!