Saturday, April 23, 2011

Hello, Cloud Foundry running node.js

Contrary to most people's view on the Amazon cloud outage incident, I believe this actually shows the power and outreach of the cloud computing. The technology needs some time to mature and become reliable, let's just be patient. It is very exciting to see even big sites like reddit, quora was affected, and even the cloud service provider Heroku was one of the victims.

There are many new players in the cloud computing field and Cloud Foundry from VMWare is definitely a very disruptive force to the current market. Currently, it supports Spring for Java apps, Rails and Sinatra for Ruby apps, JVM frameworks like Grails and Node.js! Best of all, the whole platform is open sourced.

CF provides a Micro Cloud for individual developers, for free now ;-) This is how to get a helloworld node.js app running:


1. Preparation

- Sign up for the free Micro Cloud for Cloud Foundry (http://www.cloudfoundry.com/).
- Wait a few days (mine took 9 days) and you will receive an approval email.
- Cloud Foundry uses a CLI tool called vmc for interactions with your Micro Cloud instance. Follow the instructions and use the credentials in the approval email to set up vmc (see "vmc getting started guide", really wish it is available not only in PDF format).

2. Write a helloworld node.js app

As of this writing, CF supports node.js-0.4.5 (use "vmc runtimes" to find out). The only thing that is different in the CF environment is that the system will assign a host and port to your application. CF adds several environment variables accessible through process.env.

For our helloworld, we are interested in VCAP_APP_HOST and VCAP_APP_PORT (there are also VMC_APP_HOST and VMC_APP_PORT, which seems still exist but will be deprecated).

So, you basically create a directory (I use the same appid as the directory name) and create an app.js for hello world like this:
var http = require('http'),
    port = Number(process.env.VCAP_APP_PORT || 3000),
    host = process.env.VCAP_APP_HOST || 'localhost';

http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Welcome to Cloud Foundry!\n' +
            host + ':' + port + '\n' +
            require('util').inspect(process.env, false, null));
}).listen(port, host);

console.log('Server running at http://' + host + ':' + port + '/');
This simple app just prints out "Welcome to Cloud Foundry!" and a bunch of environment variables ;-) Note that we put localhost and 3000 there so that it is easier to debug the app on a local node.js installation.

3. Deploy using vmc

The very first time you deploy the app, you run "vmc push <appid>". Note that you need to pick an id that is not being used, otherwise, you will get an error. It would be really nice for CF to provide a Web UI to check what appid is still available. You just follow the instructions and see the output. If everything goes well, you will see your app running at http://<appid>.cloudfoundry.com/.

To update your app, you need to run "vmc update <appid>" instead of "vmc push <appid>". You can also delete your app using "vmc delete <appid>".

vmc CLI provides many operations, "vmc -h" is your best friend to find and try them out.

4. Add npm module dependencies

In most cases, you will need to use dependent NPM modules such as Connect, Express, etc. With the current node.js 0.4.5 supported on CF, it is very straightforward.

- You create a package.json that defines your app/module with dependencies.
- Then you run "npm bundle", which creates a node_modules directory with all the required NPM modules.
- Finally, you do "vmc update <appid>" and you are done.

Here is a quick sample:

package.json
{
    "name": "helloworld-node",
    "version": "0.0.1",
    "dependencies":
        {
            "connect": "*"
        }
}

app.js
var connect = require('connect');

connect.createServer(function(req, res) {
    var body = 'hello, cloud foundry';
    res.setHeader('Content-Length', body.length);
    res.end(body);
}).listen(process.env.VCAP_APP_PORT || 3000);

console.log('connect server listening...');

dir structure: 
app.js, package.json, node_modules

5. To learn more

I will try to update this helloworld guide and add a bit about bundling dependent npm modules. To learn more about Cloud Foundry, read the CF blog and code, since it is open sourced, you can simply install it from github.

Updated: found a recent post about using MongoDB on cloudfoundry ;-)

Thursday, April 14, 2011

interesting node modules

There are so many node modules out there to the point that sometimes you can have several different ones that appear to solve the same problem.

node.js on GitHub has a list of classified node modules, but still it is quite difficult to pick the best one (in terms of node version compatibility, easy to use, still actively developed, good performance, etc.).

It would be really cool to have some sites that actually give reviews and code samples of using these node modules. If performance comparison can be provided, that is even better.

Inspired by "6 Must Have Node.js Modules", I am listing the modules that I found quite interesting and useful to the current work:

HTTP clients: 



XML to object conversion: 

  • node-xml2js, a fairly lightweight and fast xml to object conversion library
  • node-expat, a very high performance SAX lib, native binding to libexpat, it has a fork with object conversion support

Image processing: 

Sunday, March 20, 2011

resources to get started with node.js

There are quite some blogs and websites about "getting started with node.js". Here are some resources that I found most useful:

Saturday, March 19, 2011

set up Linksys WRT54GL as a WiFi repeater with Tomato

Since I got the new ASUS RT-N16, the old Linksys WRT54GL v1.1 router has been sitting there and collecting dust. There is only one phone line at home at one corner of the house, so I have been thinking of using the idle router as a WiFi repeater so that I can get decent WiFi coverage for each room.

Finally I found some time this weekend and studied about it. Many online tutorials cover how to do it using DD-WRT, another great customized firmware for WiFi routers. Since my RT-N16 already runs Tomato, I'd want to find a way to have Tomato on WRT54GL and work as the repeater as well.

After digging into the Web searches, finally I found Tomator author's FAQ for WDS, it is very straightforward. Basically, both both master and repeater routers need to set "Wireless Mode" into "AP and WDS". You also need to turn off dhcp and wan (turn WAN/Internet type into "disabled") on the repeater. Then you configure WDS to "Link with..." each other's MAC address (note this is the Wireless MAC address, not router's MAC address).

WDS stands for Wireless Distribution System, it enables interconnections between access points (AP). Wikipedia definition has more technical details. Ideally, using WDS we can chain more APs together, just make sure you don't keep any loop ;-)

The beautify of this is that it uses the same ssid, so when you move around, it switches to the router with the best signal strength automatically.

To test the signal strength, there is a nice free tool for Mac called AP Grapher, very handy indeed.

Saturday, March 5, 2011

wireshark no network interface issue

wireshark is one of the best tool to deep network analysis. But due to the permission issue on BPF devices on Mac OSX (by default only root can read and write BPF devices), you won't see network interfaces in wireshare on Mac ;-(

There are two ways to fix:

- run sudo chown <your_user_id> /dev/bpf* on command line before you start wireshark, you will need to do it again after reboot

- add a startup script to fix this permanently (see instructions that comes with wireshark, D&D Utilities/ChmodBPF directory in the installation package into /Library/StartupItems
. You can adjust the script for your needs as well, for details, see the documentation that comes with wireshare.

Saturday, February 26, 2011

node.js hosting in Amazon EC2

node.js has rapid growth and there are so many hosting services available. Ryan maintains a long list of providers on github already and new ones like NodeFu keeps coming out every day. I have personally tried two managed hosting services duostack and JSApp.US, both are very easy to set up and use. The only drawback is that they pose limitations on installed modules, etc.

Amazon is offering 1 year free trial for its cloud services, so called "AWS Free Usage Tier". I decided to give it a try and set up an EC2 instance for node.js.

The whole process is not quite user friendly (not as friendly as the shopping experience on Amazon). There are quite several choices for the image to choose. My first attempt with a SuSE 64bit image also got permission issues when I install gcc somehow. Finally, from the Web search results, I found Mike Leach's step-by-step guide "Installing node.js on Amazon EC2", which is really helpful. Jim Smith's blog post added a nginx as the front end, a nice addition since node.js is not multi-threading out of box.

Some minor differences when I followed Mike's steps:
  • instead of ssh to the vm using ec2-user@ec2-public-dns, I had to use ec2-user@<address you can find in AWS management console by right clicking on the instance and then clicking connect>, note that the instructions there told you to ssh as root@<address>, but it did not work for me, saying I had to use ec2-user@
  • instead of --without-ssl option to ./configure, I added openssl and openssl-devel
  • I did node and npm installation using the "node-and-npm-in-30-seconds.sh" script from this gist mentioned in the Joyeur post "Installing Node and npm"