Prowler 1.0.1

Saturday 12/26/2009  –  Category: Uncategorized  –  No Comments

I've updated the Prowler extension to make the API call in the extension instead of relying on a web service. Please upgrade for improved performance and reliability!

Download Prowler 1.0.1

The web service (hosted on Heroku) will still be available for those running older versions of the extension.

View the original Prowler post

Flickr Original 1.0.3

Saturday 12/26/2009  –  Category: Uncategorized  –  2 Comments

(UPDATE: Version 1.0.4 available!)

I've updated the Flickr Original extension to use XMLHttpRequest instead of depending on a web service. It should be faster now since the API logic had been moved to the browser.

Download Flickr Original 1.0.3

The web service (now running on node.js) will still be available for those running older versions of the extension.

View the initial Flickr Original post

Trying out Node.js

Friday 12/25/2009  –  Category: Uncategorized  –  1 Comment

Node.js ("evented I/O for V8 javascript") lets you use non-blocking callbacks to improve concurrency and performance of your app. I decided to give it a try by rewriting my Flickr Original webservice since the current Merb-based version didn't do too well when pounded--my load averages spiked fairly often and the Merb master process would get bloated into the hundreds of MB whenever I had a lot of requests coming in.

In the Merb version, the method calling the Flickr API blocks, causing the app to throw 5xx errors when there are too many requests for Merb to handle.

Using Node, I put the Flickr API call in a block with a callback that is triggered when the API returns a result. By doing this, the program doesn't have to wait for the API to accept another request. This results in being able to serve many more requests, as shown in the benchmark below (on my local machine - 2.16ghz intel core2 duo 2gb RAM).

This test simulates 80 concurrent connections:

Merb:

$ httperf --server localhost --port 4000 --uri /view/4067830120 --num-conn 80 --rate 80
Total: connections 80 requests 80 replies 80 test-duration 11.665 s

Connection rate: 6.9 conn/s (145.8 ms/conn, <=80 concurrent connections)
Connection time [ms]: min 3912.1 avg 8397.8 max 11239.2 median 9315.5 stddev 2605.9
Connection time [ms]: connect 0.0
Connection length [replies/conn]: 1.000

Request rate: 6.9 req/s (145.8 ms/req)
Request size [B]: 75.0

Reply rate [replies/s]: min 0.6 avg 4.0 max 7.4 stddev 4.8 (2 samples)
Reply time [ms]: response 7264.0 transfer 1133.8
Reply size [B]: header 161.0 content 104792.0 footer 0.0 (total 104953.0)
Reply status: 1xx=0 2xx=1 3xx=5 4xx=0 5xx=74

CPU time [s]: user 0.47 system 9.76 (user 4.0% system 83.6% total 87.6%)
Net I/O: 703.4 KB/s (5.8*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

Node:

$ httperf --server localhost --port 9900 --uri /view/4067830120 --num-conn 80 --rate 80
Total: connections 80 requests 80 replies 80 test-duration 1.182 s

Connection rate: 67.7 conn/s (14.8 ms/conn, <=20 concurrent connections)
Connection time [ms]: min 177.0 avg 200.0 max 257.6 median 196.5 stddev 17.0
Connection time [ms]: connect 0.1
Connection length [replies/conn]: 1.000

Request rate: 67.7 req/s (14.8 ms/req)
Request size [B]: 75.0

Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
Reply time [ms]: response 199.8 transfer 0.0
Reply size [B]: header 128.0 content 106.0 footer 0.0 (total 234.0)
Reply status: 1xx=0 2xx=80 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.14 system 1.00 (user 11.8% system 84.7% total 96.5%)
Net I/O: 20.4 KB/s (0.2*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

Node smokes Merb at 67.7 req/s vs 6.9 req/s! We can see that Merb could definitely not keep up with 80 concurrent requests, returning 5xx on 74/80 requests, while Node was able to serve almost 85% of the requests in the first second.

I've updated the Flickr Original webservice in production and so far things seem to be running smoothly.

I'm definitely keeping an eye on Node.js and thinking of different applications to take advantage of its event-driven model.

Basecamp Favorites - Easily generate web shortcuts to all your Basecamp projects

Friday 09/18/2009  –  Category: Uncategorized  –  1 Comment

Ok, this is for the extremely lazy (or efficient, you pick). With Quicksilver or LaunchBar, you can launch just pretty much everything...so why not open URL links with it? I use Basecamp on a daily basis and it gets a bit tedious trying to find a specific project, especially if you have a ton of them.

Enter Basecamp Favorites, a simple script to slurp your Basecamp projects and create web shortcuts for them you can quickly launch with Quicksilver/LaunchBar/whatever.

Download basecamp_favorites.zip

Requirements:
- Ruby and RubyGems installed
- xml-simple gem (sudo gem install xml-simple)
- API access and https enabled on your Basecamp account

Usage:
1. Setup your Basecamp connection:
@@session = Basecamp.establish_connection!('YOUR_BASECAMP_URL', 'YOUR_BASECAMP_USER', 'YOUR_BASECAMP_PASSWORD', true)

Make sure you omit the http/https from the url, the last parameter in the call specifies whether or not to use an https connection.

2. Open up a terminal, cd to the script directory, and:
ruby favorites.rb

Your shortcuts will be generated in the same folder you run the script. You can re-run the script whenever you want, and for bonus points, you can set up a cron job to periodically run the script to catch new projects that have been added.

How to send text from any application to Prowl

Tuesday 07/21/2009  –  Category: Uncategorized  –  No Comments

This post is a follow-up to Prowler, my Firefox extension that lets you send selected text to Prowl.

I thought it would be nice to be able to send text from any application to Prowl, say if a friend sent you directions via IM or you need to send text from a Word file.

This assumes you have Quicksilver installed.

Setup:
1) Install the growlnotify utility by running ./install.sh within the growlnotify directory [download]
2) Download this shell script somewhere (I put mine in my Documents folder) and chmod 777 the file. This script sends any text in your clipboard to the growlnotify command line tool.
3) In Quicksilver Triggers (⌘'), add a new hotkey command

4) Start typing "Documents/prowl.sh" and select "Run a shell script"

5) Assign a hotkey to the command -- I chose ⌘+Alt+Shift+P

Usage:
To send text to Prowl, just copy (⌘+C) the text you want to send, then hit the hotkey combo you setup to run the shell script. When you hit the hotkey combo you should see the text in your clipboard pop up in a Growl notification which should be caught and forwarded to Prowl.