30 Nov 2011, 1:00am
Programming:
by

2 comments

Querying Reddit with Python

I’ve long been a fan of reddit: which is a social news site where users can submit news, they can also comment and vote on submissions of other users.  Reddit provides a form of content filtration though subreddits which are specialized by topic e.g. the Python programming language.

I thought it would be fun to figure out how to get the most recent items for a particular subreddit and the previous items for an item in a subreddit. Both these things turned out to be really simple using existing Python packages to query reddit and process the JSON formatted response.

"""Return list of items from a sub-reddit of reddit.com."""

from urllib2 import urlopen, HTTPError
from json import JSONDecoder

def getitems( subreddit, previd=''):
    """Return list of items from a subreddit."""
    url = 'http://www.reddit.com/r/%s.json' % subreddit
    # Get items after item with 'id' of previd.
    if previd != '':
        url = '%s?after=t3_%s' % (url, previd)
    try:
        json = urlopen( url ).read()
        data = JSONDecoder().decode( json )
        items = [ x['data'] for x in data['data']['children'] ]
    except HTTPError as ERROR:
        print '\tHTTP ERROR: Code %s for %s.' % (ERROR.code, url)
        items = []
    return items

if __name__ == "__main__":

    print 'Recent items for Python.'
    ITEMS = getitems( 'python' )
    for ITEM in ITEMS:
        print '\t%s - %s' % (ITEM['title'], ITEM['url'])

    print 'Previous items for Python.'
    OLDITEMS = getitems( 'python', ITEMS[-1]['id'] )
    for ITEM in OLDITEMS:
        print '\t%s - %s' % (ITEM['title'], ITEM['url'])

In my next post I’ll detail what I used this script for..

The Rands Test

Rands has posted his own ‘Rands Test‘ in the style of Joel Spolsky’s famous ‘Joel Test‘ for telling if your company is screwed or not.  Rand’s test is focuses on communication while Joel’s original test focused on engineering:

“There is a higher order goal at the intersection of the two questions The Rands Test intends to answer: Where am I? and What the hell is going on? While understanding the answers to these questions will give you a good idea about the communication health of your company, the higher order goal is selfish.”

7 Oct 2011, 1:00am
Miscellaneous
by

leave a comment

Steve Jobs 1955-2011

So passes a legend. It was impossible not to admire Steve Job’s ability to consistently produce products and companies that lead their marketplace. The following is Steve’s graduation speech at Stanford University.

Having just lost my wife to cancer less than three months ago, my thoughts are with his family at this challenging time.

Why visualise data?

This excellent short video from Column Five really demonstrates the difference that even basic data visualization techniques can make.

23 Sep 2011, 1:00am
Links
by

leave a comment

Friday Linkage

This weeks interesting links:

20 Sep 2011, 1:00am
Productivity
by

1 comment

On taking initiative

As an engineer it is easy to complain about things that don’t work as well as they could, especially when you first encounter the issue.

However the difference between an average engineer and a truly effective engineer is that effective engineers don’t just stop and complain about a problem, they will actively suggest and implement solutions to allow them to continue working.  This is especially apparent when it comes to repetitive manual tasks, I have witnessed so many people complain about tedious manual processes but then fail to automate the process!  So do not view a problem as a road block preventing further progress, instead view it as an unexpected opportunity to solve a problem and improve something.

A pleasant side effect of this proactive approach is that those that habitually remove problems when they encounter them are also building a reputation as a problem solver and someone who gets things done.  This reputation will dramatically increases the chances that they will be listened to when it comes to getting management buy in to solve a significant problem.