Embracing chaos
Last week there was a very public service outage in Amazon Web Services (AWS) cloud infrastructure that took down a large number of web sites and services. This post at the High Scalability blog is a very comprehensive list of articles and posts covering the AWS outage from all possible angles.
So many great articles have been written on the Amazon Outage. Some aim at being helpful, some chastise developers for being so stupid, some chastise Amazon for being so incompetent, some talk about the pain they and their companies have experienced, and some even predict the downfall of the cloud. Still others say we have seen a sea change in future of the cloud, a prediction that’s hard to disagree with, though the shape of the change remains…cloudy.
Particularly interesting is Netflix’s Chaos Monkey (see point 3) and how it has been credited with helping Netflix stay online during the outage.
One of the first systems our engineers built in AWS is called the Chaos Monkey. The Chaos Monkey’s job is to randomly kill instances and services within our architecture. If we aren’t constantly testing our ability to succeed despite failure, then it isn’t likely to work when it matters most – in the event of an unexpected outage.
Deliberately destabilising a live system to improve its robustness is counter intuitive but it seems to have paid off for NetFlix.
Why Scrum fails…
My biggest issue with Scrum is not the methodology itself which I believe is very effective for software engineering, it is how Scrum is typically adopted by development teams. In my experience most teams that are new to SCRUM seem incapable of trying to use and understand Scrum without first modifying the Scrum process to ‘suit their unique development culture’.
These modifications while being well meant usually distort the Scrum process sufficiently to doom the process e.g.
- The scrum master is also the product owner.
- Assigning each programmer to multiple scrum sub-teams.
- Weak scrum masters that are unable to control a stand up meeting.
- Scrum teams of 20+ programmers.
- Stand up meetings with chairs…
- Stand up meetings taking 30-60 minutes due to meta-discussions.
- Not reviewing task estimate accuracy.
- Not tracking all work being performed e.g. bug fixing, training, holidays etc.
- Not engaging stake holders.
- Not having daily stand ups.
I think Scrum is a very useful methodology when its applied correctly as it constantly captures, measures and responds to changes in the project. However Scrum seems to have a fairly unique problem that its very hard to stop teams tampering with the methodology before they really understand the implications of their changes.
Friday Linkage
This weeks intersting links:
- Lawyers: You’re Being Played By Twitter.
An interesting post about how social media sites like Twitter are exploiting positive feedback loops and gamification to motivate users. - How HTML5 will kill the native app.
Does the trend for business to replace native smart device (e.g. Android/iOS) applications with HTML5 web applications mean the days of the native application are numbered? - Why Can’t Developers Estimate Time?
This post discusses the issues observed with Developer time estimates and proposes an alternative solution: teaching junior engineers the meaning of ‘done’. - A Hard Thing is Done by Figuring Out How to Start.
Rands discussing various strategies for getting started on a project: the hard part being figuring out where to start. - Four Reasons We Choose Amazon’s Cloud as Our Computing Platform.
Explaining the reasoning behind NetFlix’s adoption of the Amazon Cloud as its software platform.
Installing Python, MatPlotLib & iPython on Snow Leopard
As I have detailed in a previous post the installation of MatPlotLib on Mac OS was a fairly involved process involving the using of Mac Ports to compile and build a complete Python stack. Thankfully it would seem things have become much simpler on Mac OS X 10.6.7 if you are installing Python 2.7.1, MatPlotLib 1.0.1 and iPython 0.10.1. Note: currently only the 32 bit version of Python will work consistently with MatPlotLib and iPython.
- First Python 2.7.1:
- Download the prebuilt ‘Python 2.7.1 Mac OS X 32-bit i386/PPC Installer’ DMG from python.org.
- Mount the DMG image and run the contained installer.
- Verify it worked by opening a terminal and running the command ‘python -V’ which should return ‘Python 2.7.1′.
- Next MatPlotLib 1.0.1:
- Download the prebuilt ‘matplotlib-1.0.1-python.org-32bit-py2.7-macosx10.3′ DMG from MatPlotLib’s SourceForge page.
- Mout the DMG image and run the contained installer.
- Verify this worked by opening a terminal, running python and then ‘import matplotlib’ followed by ‘print matplotlib.__version__’ which should return ’1.0.1′.
- Finally iPython 0.10.1:
- Download the iPython source ‘ipython-0.10.1.zip’ from the iPython download directory.
- Extract the zip file.
- Open a terminal window and CD into the newly extracted directory ‘ipython-0.10.1′.
- Run the command ‘sudo python setup.py install’ and enter your password when prompted.
- Verify this by running iPython with MatPlotLib via ‘ipython -pylab’ and then ‘x = randn(10000)’ followed by ‘hist(x, 100)’ and a chart window like the following image should pop up.
April Fools: Google’s GMail Motion
It seems like motion control is a theme this year for April Fool’s pranks, this time with Google getting in on the act with a motion control interface for their GMail email application.
However due to the open nature of the Microsoft Kinect sensor for Xbox360 some students at the University of Southern California’s Institute for Creative Technologies actually went off and built a working prototype of Google’s fictional Gmail Motion application after seeing Google’s April Fools video.
April Fools: Starcraft returns to console for Xbox360!
I think Blizzard’s April Fools gag (Starcraft II on Microsoft’s Xbox 360 with Kinect sensor) is the best I’ve seen so far this year: amusing concept, decent production values for the video and humourous (to fans of the game at least).
Reddit has a good round up of this year’s April Fools offerings by the games industry you can find here.
