A Python script using Tornado web server with the Twitter Streaming API

So today I decided to play with Tornado the open source version of a scalable, non-blocking web server which powers FriendFeed. While doing so I discovered it will probably be cool to play with the Twitter streaming API as well and write a Python script that uses both.

Here's the result of my experiment:

from tornado.httpclient import HTTPClient, HTTPRequest
import simplejson

def handle_tweet_stream(response):
    try:
        # Print the parsed twitter stream.
        print simplejson.loads(response)
    except ValueError:
        return
    print

if __name__ == '__main__':
    req = HTTPRequest(
        "http://stream.twitter.com/1/statuses/filter.json", 
        body='track=google',
        method = "POST",
        auth_username = "username",
        auth_password = "xxxxxxx",
        streaming_callback=handle_tweet_stream)

    client = HTTPClient()
    client.fetch(req)

Lessons learned from working in large scale

Before my internship with Google, I wasn't a big fan of unit tests, logging and cloud computing. Now I can say there's a degree of doubt in me when I see some new code without tests and that this code claims to do the right thing.

When you're running code in parallel on 1000's of machines in the cloud, machines whose physical location you don't know and machines you don't have shell access to, your idea of project development changes dramatically. Things like logging and unit testing become your friends.

A good blog on this topic is found on Matt Welsh's blog

A one line Python Quine

OK, Here's a one line Python program that prints its own source code:

s='s=%r;print s%%s';print s%s

This wikipedia article on Quines is a very good place to learn how these things work

One line quicksort in Python

Here's a one line quicksort in python. I came up with this idea after looking at a 2 lines implementation in Haskel

qsort = lambda seq: [] if not seq else qsort(filter(lambda n: n<=seq[0], seq[1:]))+[seq[0]]+qsort(filter(lambda n: n>seq[0], seq[1:]))