Questions starting at 31:20 are pretty interesting.

The first is the assumption that you could create something with the same quality level as the existing solution in the stated number of days; most likely you will be lucky to have a functional prototype with minimal functionality.  It is highly unlikely that you will have produced something that is as well tested, optimised and documented as the existing solution.

Secondly, as developers writing software which we intend to sell to users to pay the bills, we should be focused on the core features that define our product.  It is these core features we should be pouring our energy and time into designing, building and testing.  Developing alternatives to existing technology that is not in our core feature set is a waste of valuable time and energy and will not differentiate our product from that of our competitors.

We should instead be looking to leverage as many established technologies or components as possible when developing new systems as this lets us spend most of our time where it matters most on our core differentiating features.

So here is a brief description of how to do it:

1. Install the X-Code developer tools (for GCC, make and other build tools) from your Snow Leopard installation DvD.
2. Install the X11 Window System from your Snow Leopard installation DvD.
3. Run ‘Software Update‘ from the apple menu on your Mac to get the latest X-Code updates.
4. Download the latest MacPorts installer (.dmg file extension) for Snow Leopard from here: http://distfiles.macports.org/MacPorts/.
5. Mount the installer image file (.dmg) and run the contained MacPorts installer (.pkg).
6. Once installation is complete open a terminal window from Applications->Utilities->Terminal.
7. Type ‘port‘ at the terminal then press enter to run MacPorts.  You should see output like the following if it installed correctly:
   MacPorts 1.8.1
   Entering interactive mode… (“help” for help, “quit” to quit)
   Then type ‘quit‘ then press enter to exit port’s interactive mode.
8. Type ‘sudo port selfupdate‘ and press enter to update MacPorts to the latest version.  You will be asked to enter the administrators password before continuing.  Depending on how new the version you downloaded is, MacPorts may do some upgrading.
9. Once the update is finished type ‘sudo port install py26-matplotlib‘ and press enter.  This will attempt to install the latest version of matplotlib for Python version 2.6.*.   You may be asked to enter the administrators password before continuing.  MacPorts will now download, configure, build and stage the dependencies needed to build the latest matplotlib for Python 2.6.  This took at least thirty minutes on my iMac and involved lots and lots of scrolling text output from the build process.
10. Next we need to switch our environment to use version 2.6.* of Python that MacPorts just built and installed with matplotlib.  To do this run the following two commands, note you may be asked to enter the administrators password before continuing:
    1. ‘sudo port install python_select‘ and hit enter.
    2. ‘sudo python_select python26‘ and hit enter.
11. To test this all worked type the following: ‘python -V’ and hit enter.  You should see output like ‘Python 2.6.4‘ which should match the version of python MacPorts built and installed.
12. Finally to test if matplotlib was installed correctly do the following:
    1. Type ‘python‘ and hit enter to enter the python interactive shell.
    2. Type ‘import matplotlib‘ and hit enter, this will import the matplotlib module.  There should be no output if this works.
    3. Type ‘print matplotlib.__version__‘ and hit enter. This will print the version of matplotlib that is installed, you should see output like ‘0.99.1.1‘.
    4. Type ‘exit()‘ to quit the python interactive shell.

This is a very convenient pairing as SQLite is an excellent lightweight SQL implementation that I find very useful for a variety of tasks e.g. data mining.  Or any task involving manipulating complex data sets where I’d otherwise end up resorting to using a full blown SQL server like MySQL.

Here is a simple example of using SQLite in Python using it’s built-in sqlite3 module:

import sqlite3

# craete a connection
con = sqlite3.connect('test.db')

# create a cursor
cur = con.cursor()

# create a test table
cur.execute( "CREATE TABLE testTable (myKey INT, myValue INT)" )

# insert some data
for i in range(0,10):
 cur.execute( "INSERT INTO testTable VALUES ( %d, %d )"%(i,i*i) )

# select the data
for row in cur.execute( "SELECT * FROM testTable" ):
 print row

# destroy (drop) our test table
cur.execute( "DROP TABLE testTable" )

# close the connection
con.close()

As you can see Python makes handling SQLite (a C language library) much easier, less error prone, and the resulting code much more compact than SQLite’s native C.

The awareness of what I don’t know helps keep me humble, humility makes working as part of a team easier:  as there is no pressure to have to know everything or not make mistakes.  In fact I tend to expect to make mistakes more now than when I first started programming.  Perhaps it is the years I have spent shipping games that finally proved to me that I too write software that contains bugs (the horror!).

I remember a professor at university telling me that the main difference between a professor and a first year student working on a programming task is that the student will start working immediately and also start making mistakes immediately, the professor will think for a while then start work and start making mistakes as well.  I have found this observation to have a surprising amount of truth in it,:whether it is at university, work or even sports.

Mistakes are healthy: without mistakes we would have not reason to every really think about what we are doing e.g. why didn’t that work?  By continually pushing (or stretching) ourselves to failure we discover our boundaries, once we know where are our boundaries are we can then start to work on pushing them further.  However if we always play it safe and never push ourselves (which can be scary) we will never discover our boundaries which makes improvement much harder and also makes approaching those boundaries harder due to fear (typically of loss of control).

An excellent example of this is people learning to ice skate: young children will tend to fling themselves around the rink with wild abandon falling all over the place (as failure is expected but irrelevant due to lack of social stigma), yet adult beginners typically skate in a much more conservative fashion taking less risks (as there is social stigma against falling as an adult: falling is seen as failure).  Interestingly when observing ice hockey players, it is noticeable that those with the best skating technique are typically falling more than the other the non-beginner players on the ice, as they do not fear falling.

As most of us are not engaged in high risk activities on a daily basis we can easily begin to revise how we think about failure and to learn to embrace it as a powerful tool for self improvement.

This can be especially evident amongst beginner programmers, who are learning new techniques almost constantly, but it only seems to become a dangerous anti-pattern if the individual becomes overly fixated on a particular technique.

Interestingly this anti-pattern can apply to more than just individuals, in my experience it can also apply to teams or even entire organisations.  One of the ways I have seen this manifest at a team or organisational level is with attempts to solve social problems with technology solutions.  This trait seems to be especially strong in software development organisations: perhaps because technology is their usual solution to problems.

Such social issues faced by development teams and organisations can include things like not breaking the build, lack of communication or lack of testing.  These problems really are not technology problems, they are social (people) problems, yet time and again I have witnessed technology solutions to these problems.  For example, more elaborate and sophisticated check-in systems being introduced in response to constantly broken builds, where a much simpler social solution such as peer pressure to not break the build would be much more effective.

Avoiding the Golden Hammer when programming is achievable with some self awarness and honesty: are you using your current technique for the right reasons?  However, identifying and interveening when a team or organisation is about to use the Golden Hammer is much more challenging as it can involve going against the solution favoured by the majority along with the polical implications that brings…

It is not that I have anything against Perl as a programming language: it is a fine language.  I find that Perl’s motto of `There’s more than one way to do it.’ means that it is a programming language with (you guessed it) more than one way to implement most operations.  This flexibility is a double edged sword: it allows a lot of creativity in program implementation, yet it also means that a dozen Perl coders will produce a dozen radically different scripts for the same problem.  This makes Perl quite hard on beginners or infrequent maintainers of Perl scripts: as to understand any given Perl script requires an understanding of much more of the Perl programming language than it would for most comparable programming languages.

I digress.

I’ve known for years that it is possible to write awful obscure code in low level languages like C and C++ but it really hadn’t occurred to me that a high level language like C# could be abused to the same degree.  Thinking about this some more, it is not the fault of the programming language itself but of the programmer that is using it.

So, brittle obtuse code is the result of a bad programmer not a bad programming language: well except maybe Brain Fuck (but thats kinda the point).

This may seem harmless but consider that the future use cases you are imagining are not something you fully understand yet: as most of us will admit we are pretty bad at anticipating the future.  So how can we hope to implement future functionality successfully?  The most likely outcome is that whatever you implement now not be sufficient when you actually try to use it in the future.  This will mean more time will have to be invested in the future to refactor or replace the functionality with what is actually required.

Unless you also plan to implement a full set of tests for this new functionality then you are adding extra code to your program that will not be correct and will not stay correct.  This can also lead to really weird run time behavior if the flow of program execution goes into your new ‘future proof’ functionality unexpectedly: which can be very hard to debug.  Adding functionality for future use without corresponding tests also means that as the rest of your program evolves the future functionality does not evolve with it, this means by the time you actually try to use it that it is most likely hopelessly out of sync with the rest of the code base.

The future functionality is also expanding the size of the source code of your program which leads to unnecessary code bloat.  In some compiled languages (e.g. C++) code bloat will lead to increased memory usage and decreased program performance which is highly undesirable.

Another thing to consider from a business stand point is that you are spending time and money implementing future functionality when you are not being paid for it.  This is a strong indication to me that implementing future functionality makes little business sense.  As it means spending money now paying developers to implement functionality that you can’t sell right now.

There is a handy acronym for this: it is YAGNI which stands for ‘You Aren’t Gonna Need It’.  The essence of this concept is to only implement the functionality you need right now and to implement in as simple and robust a fashion as possible.  As simple robust code (preferable with acompaning unit tests) is code that can be easily refactored in the future to meet future needs and that is cost effcient!

Possibly one of the most important practices in the effective use of exceptions is to only catch the exceptions you are expecting and that you can handle safely, however a very common mistake when starting to use exceptions is to catch (and possibly suppress) every exception. For example:

try
{
// some code that throws exceptions occasionally
}catch{}

In the above example the programmer has discovered that their initial code is occasionally throwing exceptions, so they have enclosed the offending block of code in a try/catch block.  The use of a try/catch block means that any exceptions that occur inside the try clause are caught by the catch clause, this prevents the program from crashing due to unhanded exceptions.  However, the use of an empty catch clause means that any exception that is thrown will be caught by the catch clause.  The drawback to this is that the catch clause is empty, which means that every exception caught effectively disappears, which makes debugging and diagnosing abnormal program behavior incredibly hard, as that empty catch clause is effectively an exception black hole destroying any evidence.  This form of ‘exception handling’ is worse in my opinion than no exception handling: though it may prevent the program crashing due to unhandled exceptions it makes debugging the program exceptionally hard. The following snippet is an improvement:

try
{
// some code that throws exceptions occasionally
}
catch( Exception e )
{
// some exception handling
}

The above snippet is a distinct improvement on the previous snippet as the catch clause now catches exceptions and attempts to perform some form of exception recovery or logging. This is a lot better as at least some form of corrective action is taking place.  Yet there is still a drawback: the catch clause is still catching every exception that is thrown by the try block. This means that even the abnormal exception types the programmer does not expect are being caught.  The following is a better snippet:

try
{
// some code that throws exceptions occasionally
}
catch( System.ArgumentException e )
{
// some specific exception handling
}

In this example the programmer is now only catching the exceptions they expect their code to throw, which means that unexpected and abnormal exceptions are not caught. This is an improvement, as these unexpected exceptions will now filter up the call stack to more appropriate exception handlers and eventually to the try/catch clause in the main function of the application.  This means that when something truly rare and weird happens, e.g. an underlying system breaks causing an unexpected exception, that the truly exceptional exception is not caught by the original try/catch block but filters up to higher level exception handlers which hopefully alert the user (or at least the investigating programmer) that a truly exceptional event has occurred.

try
{
// some code that throws exceptions occasionally
}
catch( System.ArgumentException e )
{
// some specific exception handling
}
catch( Exception e )
{
// generic exception handling
}

In this last example specific exceptions are caught by the first catch clause which catches only the exceptions the programmer is expecting and then a second catch clause catches all the other exceptions the programmer did not expect. This form of exception handling is good if there is no general exception handler further up the call stack to catch the unexpected exceptions e.g. in a library. In summary exception handling is a very useful tool but you need todevelop the habit of catching exceptions responsibly and to not catch exceptions you don’t intend to handle.

Assume the task that has been set is complex and contains some degree of uncertainty which means that the first few attempts will probably fail.  Which programmer would you want working on the task?  If the task is actually impossible then programmer A will be the cheapest from a business standpoint as they fail earliest followed by programmer B and finally programmer C.  Also, programmer A will have five attempts at a solution compared to programmer B’s two and a half attempts and programmer C’s single attempt.  As iteration leads to improved quality (Boyd’s Law) this means that programmer A is also likely to hit on a more superior solution than programmer B or programmer C in the same time period.

The interesting thing here is that in my experience new programmers tend to work like programmer C: they will attempt a single solution working diligently to overcome any hurdles they encounter, no matter how severe the hurdle.  This sounds harmless but remember that few things are impossible in programming, so the more a programmer hammers away at a problem the more likely they are to fudge a solution and that solution (in my experience) is not often of a high quality.  The counter point to this is that more productive programmers will rapidly discard attempts early in each attempt when it becomes clear that the solution though possible is not actually desirable e.g. the dog is hungry and solutions include: A) feed the dog B) kill the dog.

This can also be observed by monitoring how long a programmer will remain stuck for before asking for assistance.  The most productive programmers will get help within minutes of becoming completly stuck.  However the least productive (and often least experienced) programmers will often struggle with the problem for days before actually asking for assistance and this delay is expensive in terms of money, deadlines and product quality.