Graphing real data with MatPlotLib

In a previous post I covered the basics of graphing in Python with the MatPlotLib module.  In this post I am going to demostrate how to use MatPlotLib with some real world data retrieved from a web service and then processed into a format usable by MatPLotLib.

The example script performs the following steps:

1. Takes a specified stock’s ticker symbol and column to plot over time (from Open, High, Low, Close, Volume, Adj Close) as input.
2. Fetches the corresponding stock data from Yahoo! Finance and saves it into a CSV file using the urllib module.
3. Processes the data in the CSV file into a suitable format for matplotlib using the csv, datetime and matplotlib.dates modules.
4. Plots a graph of the data plotted over time using MatPlotLib and a saves a copy as PNG format image.

Note: To keep the example concise I am not performing any error handling.

"""Fetches specified stock data from Yahoo and graph it with MatPlotLib."""

from urllib import urlretrieve
from csv import DictReader
from matplotlib import pyplot
from matplotlib.dates import date2num
from datetime import datetime

def fetchstockdata( stockticker, filename ):
    """Fetch specified stock data and store it in named file."""
    url = 'http://ichart.finance.yahoo.com/table.csv?s=%s' % stockticker
    urlretrieve( url, filename )

def importstockdata( filename ):
    """Import CSV data into dict of lists, converting dates into timestamps."""
    results = {}
    for row in DictReader( open( filename,'rb' ) ):
        for col in row.keys():
            if col == 'Date':
                coldata = date2num( datetime.strptime( row[col], '%Y-%m-%d') )
            else:
                coldata = row[col]
            results.setdefault( col, [] ).append( coldata )
    return results

def plotstockdata( stockdata, stockticker, dates, col ):
    """Use MatPlotLib to graph speciifed stock data."""
    pyplot.plot_date( stockdata[dates], stockdata[col], '-', xdate=True )
    pyplot.title( '%s - %s / %s' % (stockticker, col, dates) )
    pyplot.xlabel( dates )
    pyplot.ylabel( col )
    pyplot.savefig( '%s.png' % stockticker )
    pyplot.show()

if __name__ == '__main__':
    from sys import argv
    # Use second argument as ticker and third argument as column.
    TICKER = argv[1].upper()
    COL = argv[2]
    # Grab the stock data from Yahoo!
    FILENAME = '%s.csv' % TICKER
    fetchstockdata( TICKER, FILENAME )
    # Import the data.
    DATA = importstockdata( FILENAME )
    # Plot the graph with Date as X-Axis and User selected column as Y-Axis.
    plotstockdata( DATA, TICKER, 'Date', COL )

Running this script with using the command line “python StockChart.py goog ‘Adj Close’” will produce a chart like the following.

This is a good example of why I like Python’s batteries included philosophy so much: it means I spend more of my time writing interesting bits of code as the utility functionality I need has already been implemented or is only an easy_install away.

Basic graphing with MatPlotLib

One of the Python modules that has most interested me recently is MatPlotLib which is a sophisticated graphing module which can be used to create journal grade graphs of almost anything.  The official gallery for MatPlotLib is worth checking out to get an idea of the sheer range of graph types it can be used to create.

It is simple enough to get started using MatPlotLib for example to create a line graph of x*x and save it as a PNG file requires only the following:

"""Simple demonstration of MatPlotLib plotting."""

from matplotlib import pyplot

X = range(0,100)
Y = [ i*i for i in X ]

pyplot.plot( X, Y, '-' )
pyplot.title( 'Plotting x*x' )
pyplot.xlabel( 'X Axis' )
pyplot.ylabel( 'Y Axis' )
pyplot.savefig( 'Simple.png' )
pyplot.show()

The above script will produce the following graph:

To plot data over a time period the simplest solution is to convert date/time units to timestamps using MatPlotLibs date2num function and then to plot using the plot_date method as follows:

"""Simple demonstration of MatPlotLib Date plotting."""

from matplotlib import pyplot
from matplotlib.dates import date2num
from datetime import datetime, timedelta

# Generate a series of timestamps from today to today + 100 years.
X = [date2num(datetime.today()+timedelta(days=365*x)) for x in range(0,100)]
Y = [i*i for i in range(0,100)]

pyplot.plot_date( X, Y, '-', xdate=True )
pyplot.title( 'Plotting x*x' )
pyplot.xlabel( 'X Axis' )
pyplot.ylabel( 'Y Axis' )
pyplot.savefig( 'SimpleDates.png' )
pyplot.show()

Which will generate a chart like the following:

As you can see it is fairly simple to graph data using MatPlotLib. This makes Python and MatPlotLib a compelling solution for data analysis when combined with the many available modules for dealing with common data storage formats like text (using RegEx), CSV, XML and JSON files and SQL databases.

Praise for Python

• Simplified Memory management
  I am so much more productive when I am not having to worry about pointer related errors e.g. pointer math or sweat the subtleties of memory management e.g. memory alignment while writing code.
• Less structural syntax
  After using Python for a while I really appreciate it’s use of indentation to give a program structure, as it makes python source code much more concise than C/C++.
• No compiling or linking
  It is so much easier to stay in the flow when your not waiting 5-30 minutes for compilation and linking.  I’ve recently taken to running PyLint when I miss the feedback from a compiler/linker on my program structure and to learn the coding style outlined in the Python Style Guide.
• Selective imports
  Having worked on large scale C/C++ projects for most of my career I really appreciate the ability to only import what I want from modules and the option to also rename (or alias) what I’ve imported.
• Batteries included philosophy
  The sheer scope of the library of modules included in Python means I can spend more time writing the interesting parts of my programs, as most of the time the utility functionality I need is just an import away.
• Package management
  The Python Package Index (PyPi) and Setup Tools module make installing most python modules as simple as ‘easy_install <module_name>’.
• Duck typing
  Python’s use of Duck Typing emphasizes interfaces over types which makes it so much easier to supply my own classes to standard library functions, as I only have to implement as much of the interface as is required.