Querying Reddit with Python
I’ve long been a fan of reddit: which is a social news site where users can submit news, they can also comment and vote on submissions of other users. Reddit provides a form of content filtration though subreddits which are specialized by topic e.g. the Python programming language.
I thought it would be fun to figure out how to get the most recent items for a particular subreddit and the previous items for an item in a subreddit. Both these things turned out to be really simple using existing Python packages to query reddit and process the JSON formatted response.
"""Return list of items from a sub-reddit of reddit.com."""
from urllib2 import urlopen, HTTPError
from json import JSONDecoder
def getitems( subreddit, previd=''):
"""Return list of items from a subreddit."""
url = 'http://www.reddit.com/r/%s.json' % subreddit
# Get items after item with 'id' of previd.
if previd != '':
url = '%s?after=t3_%s' % (url, previd)
try:
json = urlopen( url ).read()
data = JSONDecoder().decode( json )
items = [ x['data'] for x in data['data']['children'] ]
except HTTPError as ERROR:
print '\tHTTP ERROR: Code %s for %s.' % (ERROR.code, url)
items = []
return items
if __name__ == "__main__":
print 'Recent items for Python.'
ITEMS = getitems( 'python' )
for ITEM in ITEMS:
print '\t%s - %s' % (ITEM['title'], ITEM['url'])
print 'Previous items for Python.'
OLDITEMS = getitems( 'python', ITEMS[-1]['id'] )
for ITEM in OLDITEMS:
print '\t%s - %s' % (ITEM['title'], ITEM['url'])
In my next post I’ll detail what I used this script for..
The Rands Test
Rands has posted his own ‘Rands Test‘ in the style of Joel Spolsky’s famous ‘Joel Test‘ for telling if your company is screwed or not. Rand’s test is focuses on communication while Joel’s original test focused on engineering:
“There is a higher order goal at the intersection of the two questions The Rands Test intends to answer: Where am I? and What the hell is going on? While understanding the answers to these questions will give you a good idea about the communication health of your company, the higher order goal is selfish.”
Steve Jobs 1955-2011
So passes a legend. It was impossible not to admire Steve Job’s ability to consistently produce products and companies that lead their marketplace. The following is Steve’s graduation speech at Stanford University.
Having just lost my wife to cancer less than three months ago, my thoughts are with his family at this challenging time.
Why visualise data?
This excellent short video from Column Five really demonstrates the difference that even basic data visualization techniques can make.
Friday Linkage
This weeks interesting links:
- Why Functional Programming Matters
Not a new paper (1984!) but interesting take on the usefulness and potential scalability of functional languages. - Learn you a Haskell for great good!
An excellent online book for learning the Haskell functional programming language. - How we Scrum
A description of the scrum process at an indie game developer and how they’ve tailored it to their needs. - Best resources for iOS programming
A current summary of good online resources for programming Apple’s iOS devices (e.g. iPhone & iPad). - Can you really get great shots with a point and shoot?
Inspirational article about getting great photographs from compact point and shoot cameras rather than large, expensive DSLRs. - The Fall Event
Apple has ran a iPod music event every year in the fall until this year, this article speculates on what this means for the future of the iPod and on Apple’s changing priorities.
On taking initiative
As an engineer it is easy to complain about things that don’t work as well as they could, especially when you first encounter the issue.
However the difference between an average engineer and a truly effective engineer is that effective engineers don’t just stop and complain about a problem, they will actively suggest and implement solutions to allow them to continue working. This is especially apparent when it comes to repetitive manual tasks, I have witnessed so many people complain about tedious manual processes but then fail to automate the process! So do not view a problem as a road block preventing further progress, instead view it as an unexpected opportunity to solve a problem and improve something.
A pleasant side effect of this proactive approach is that those that habitually remove problems when they encounter them are also building a reputation as a problem solver and someone who gets things done. This reputation will dramatically increases the chances that they will be listened to when it comes to getting management buy in to solve a significant problem.








