Trying out Node.js

Friday 12/25/2009  –  Category: Uncategorized  –  1 Comment

Node.js ("evented I/O for V8 javascript") lets you use non-blocking callbacks to improve concurrency and performance of your app. I decided to give it a try by rewriting my Flickr Original webservice since the current Merb-based version didn't do too well when pounded--my load averages spiked fairly often and the Merb master process would get bloated into the hundreds of MB whenever I had a lot of requests coming in.

In the Merb version, the method calling the Flickr API blocks, causing the app to throw 5xx errors when there are too many requests for Merb to handle.

Using Node, I put the Flickr API call in a block with a callback that is triggered when the API returns a result. By doing this, the program doesn't have to wait for the API to accept another request. This results in being able to serve many more requests, as shown in the benchmark below (on my local machine - 2.16ghz intel core2 duo 2gb RAM).

This test simulates 80 concurrent connections:


$ httperf --server localhost --port 4000 --uri /view/4067830120 --num-conn 80 --rate 80
Total: connections 80 requests 80 replies 80 test-duration 11.665 s

Connection rate: 6.9 conn/s (145.8 ms/conn, <=80 concurrent connections)
Connection time [ms]: min 3912.1 avg 8397.8 max 11239.2 median 9315.5 stddev 2605.9
Connection time [ms]: connect 0.0
Connection length [replies/conn]: 1.000

Request rate: 6.9 req/s (145.8 ms/req)
Request size [B]: 75.0

Reply rate [replies/s]: min 0.6 avg 4.0 max 7.4 stddev 4.8 (2 samples)
Reply time [ms]: response 7264.0 transfer 1133.8
Reply size [B]: header 161.0 content 104792.0 footer 0.0 (total 104953.0)
Reply status: 1xx=0 2xx=1 3xx=5 4xx=0 5xx=74

CPU time [s]: user 0.47 system 9.76 (user 4.0% system 83.6% total 87.6%)
Net I/O: 703.4 KB/s (5.8*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0


$ httperf --server localhost --port 9900 --uri /view/4067830120 --num-conn 80 --rate 80
Total: connections 80 requests 80 replies 80 test-duration 1.182 s

Connection rate: 67.7 conn/s (14.8 ms/conn, <=20 concurrent connections)
Connection time [ms]: min 177.0 avg 200.0 max 257.6 median 196.5 stddev 17.0
Connection time [ms]: connect 0.1
Connection length [replies/conn]: 1.000

Request rate: 67.7 req/s (14.8 ms/req)
Request size [B]: 75.0

Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
Reply time [ms]: response 199.8 transfer 0.0
Reply size [B]: header 128.0 content 106.0 footer 0.0 (total 234.0)
Reply status: 1xx=0 2xx=80 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.14 system 1.00 (user 11.8% system 84.7% total 96.5%)
Net I/O: 20.4 KB/s (0.2*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

Node smokes Merb at 67.7 req/s vs 6.9 req/s! We can see that Merb could definitely not keep up with 80 concurrent requests, returning 5xx on 74/80 requests, while Node was able to serve almost 85% of the requests in the first second.

I've updated the Flickr Original webservice in production and so far things seem to be running smoothly.

I'm definitely keeping an eye on Node.js and thinking of different applications to take advantage of its event-driven model.

Ok, this is for the extremely lazy (or efficient, you pick). With Quicksilver or LaunchBar, you can launch just pretty much why not open URL links with it? I use Basecamp on a daily basis and it gets a bit tedious trying to find a specific project, especially if you have a ton of them.

Enter Basecamp Favorites, a simple script to slurp your Basecamp projects and create web shortcuts for them you can quickly launch with Quicksilver/LaunchBar/whatever.


- Ruby and RubyGems installed
- xml-simple gem (sudo gem install xml-simple)
- API access and https enabled on your Basecamp account

1. Setup your Basecamp connection:
@@session = Basecamp.establish_connection!('YOUR_BASECAMP_URL', 'YOUR_BASECAMP_USER', 'YOUR_BASECAMP_PASSWORD', true)

Make sure you omit the http/https from the url, the last parameter in the call specifies whether or not to use an https connection.

2. Open up a terminal, cd to the script directory, and:
ruby favorites.rb

Your shortcuts will be generated in the same folder you run the script. You can re-run the script whenever you want, and for bonus points, you can set up a cron job to periodically run the script to catch new projects that have been added.

How to send text from any application to Prowl

Tuesday 07/21/2009  –  Category: Uncategorized  –  No Comments

This post is a follow-up to Prowler, my Firefox extension that lets you send selected text to Prowl.

I thought it would be nice to be able to send text from any application to Prowl, say if a friend sent you directions via IM or you need to send text from a Word file.

This assumes you have Quicksilver installed.

1) Install the growlnotify utility by running ./ within the growlnotify directory [download]
2) Download this shell script somewhere (I put mine in my Documents folder) and chmod 777 the file. This script sends any text in your clipboard to the growlnotify command line tool.
3) In Quicksilver Triggers (⌘'), add a new hotkey command
4) Start typing "Documents/" and select "Run a shell script"

5) Assign a hotkey to the command -- I chose ⌘+Alt+Shift+P

To send text to Prowl, just copy (⌘+C) the text you want to send, then hit the hotkey combo you setup to run the shell script. When you hit the hotkey combo you should see the text in your clipboard pop up in a Growl notification which should be caught and forwarded to Prowl.

Flickr Search iPhone app

Monday 07/13/2009  –  Category: Uncategorized  –  No Comments

Download Flickr Search 1.1

I couldn't find any simple Flickr search apps in the App Store; they had too many other features and the photo viewer wasn't very good.

I wanted the user experience to be like the native Photos app (being able to swipe through results, pinch and zoom) and also have download functionality. To do this, I used Joe Hewitt's three20, a library of useful classes extracted from the extremely popular Facebook app. It has made a lot of developers' lives easier...kudos to Joe!

picture-1 picture-2 picture-3

I stumbled upon Prowl this week and I have to say it's a pretty brilliant idea and an excellent use of the new push notification system in iPhone 3.0. Since they have an open API to send whatever you want to a user (defined by their API key), I decided to create an extension based on IPS to send whatever text you select to your iPhone. This could be phone numbers, directions, recipes...whatever you want to quickly send. It functions like a copy and paste across devices, similar to what Evernote's Web Clipper does but with the added bonus of having whatever you sent pushed as a push notification.

UPDATE: Download Prowler 1.0.3
UPDATE: Download Prowler 1.0.2
Download Prowler 1.0

This assumes that you have the Prowl account and have the plugin and iPhone app installed. If you don't, get up to speed here.

1) Generate an API key under 'settings'.
2) Install the Prowler extension and restart Firefox when prompted
3) Copy and paste the API key you just generated into the Prowler Options preferences (under Tools > Prowler Options) and close the dialog
4) To send text to Prowl, select any text (up to 10000 characters), right click and click "Send Selected Text to Prowl"

That's all! Let me know how it works for you guys.

 Page 5 of 15  « First  ... « 3  4  5  6  7 » ...  Last »