Embracing chaos

Last week there was a very public service outage in Amazon Web Services (AWS) cloud infrastructure that took down a large number of web sites and services.  This post at the High Scalability blog is a very comprehensive list of articles and posts covering the AWS outage from all possible angles.

So many great articles have been written on the Amazon Outage. Some aim at being helpful, some chastise developers for being so stupid, some chastise Amazon for being so incompetent, some talk about the pain they and their companies have experienced, and some even predict the downfall of the cloud. Still others say we have seen a sea change in future of the cloud, a prediction that’s hard to disagree with, though the shape of the change remains…cloudy.

Particularly interesting is Netflix’s Chaos Monkey (see point 3) and how it has been credited with helping Netflix stay online during the outage.

One of the first systems our engineers built in AWS is called the Chaos Monkey. The Chaos Monkey’s job is to randomly kill instances and services within our architecture. If we aren’t constantly testing our ability to succeed despite failure, then it isn’t likely to work when it matters most – in the event of an unexpected outage.

Deliberately destabilising a live system to improve its robustness is counter intuitive but it seems to have paid off for NetFlix.

Why Scrum fails…

My biggest issue with Scrum is not the methodology itself which I believe is very effective for software engineering, it is how Scrum is typically adopted by development teams. In my experience most teams that are new to SCRUM seem incapable of trying to use and understand Scrum without first modifying the Scrum process to ‘suit their unique development culture’.

These modifications while being well meant usually distort the Scrum process sufficiently to doom the process e.g.

  • The scrum master is also the product owner.
  • Assigning each programmer to multiple scrum sub-teams.
  • Weak scrum masters that are unable to control a stand up meeting.
  • Scrum teams of 20+ programmers.
  • Stand up meetings with chairs…
  • Stand up meetings taking 30-60 minutes due to meta-discussions.
  • Not reviewing task estimate accuracy.
  • Not tracking all work being performed e.g. bug fixing, training, holidays etc.
  • Not engaging stake holders.
  • Not having daily stand ups.

I think Scrum is a very useful methodology when its applied correctly as it constantly captures, measures and responds to changes in the project. However Scrum seems to have a fairly unique problem that its very hard to stop teams tampering with the methodology before they really understand the implications of their changes.

Friday Linkage

This weeks intersting links:

Installing Python, MatPlotLib & iPython on Snow Leopard

As I have detailed in a previous post the installation of MatPlotLib on Mac OS was a fairly involved process involving the using of Mac Ports to compile and build a complete Python stack.  Thankfully it would seem things have become much simpler on Mac OS X 10.6.7 if you are installing Python 2.7.1, MatPlotLib 1.0.1 and iPython 0.10.1.  Note: currently only the 32 bit version of Python will work consistently with MatPlotLib and iPython.

  1. First Python 2.7.1:
    1. Download the prebuilt ‘Python 2.7.1 Mac OS X 32-bit i386/PPC Installer’ DMG from python.org.
    2. Mount the DMG image and run the contained installer.
    3. Verify it worked by opening a terminal and running the command ‘python -V’ which should return ‘Python 2.7.1′.
  2. Next MatPlotLib 1.0.1:
    1. Download the prebuilt ‘matplotlib-1.0.1-python.org-32bit-py2.7-macosx10.3′ DMG from MatPlotLib’s SourceForge page.
    2. Mout the DMG image and  run the contained installer.
    3. Verify this worked by opening a terminal, running python and then ‘import matplotlib’ followed by ‘print matplotlib.__version__’ which should return ‘1.0.1’.
  3. Finally iPython 0.10.1:
    1. Download the iPython source ‘ipython-0.10.1.zip’ from the iPython download directory.
    2. Extract the zip file.
    3. Open a terminal window and CD into the newly extracted directory ‘ipython-0.10.1′.
    4. Run the command ‘sudo python setup.py install’ and enter your password when prompted.
    5. Verify this by running iPython with MatPlotLib via ‘ipython -pylab’ and then ‘x = randn(10000)’ followed by ‘hist(x, 100)’ and a chart window like the following image should pop up.

April Fools: Google’s GMail Motion

It seems like motion control is a theme this year for April Fool’s pranks, this time with Google getting in on the act with a motion control interface for their GMail email application.

However due to the open nature of the Microsoft Kinect sensor for Xbox360 some students at the University of Southern California’s Institute for Creative Technologies actually went off and built a working prototype of Google’s fictional Gmail Motion application after seeing Google’s April Fools video.

April Fools: Starcraft returns to console for Xbox360!

I think Blizzard’s April Fools gag (Starcraft II on Microsoft’s Xbox 360 with Kinect sensor) is the best I’ve seen so far this year: amusing concept, decent production values for the video and humourous (to fans of the game at least).

Reddit has a good round up of this year’s April Fools offerings by the games industry you can find here.