Processing Perforce command output with python.
The Perforce command line client supports returning Python marshalled dictionary objects as command output via the ‘-G’ option. This functionality can be used to easily export data from Perforce to various formats to allow further analysis. The following script exports the submitted change lists returned by the p4 changes command to an CSV file which can easily be loaded into Microsoft Excel for analysis. The script also converts the returned timestamp into human readable date and time columns.
"""Perforce changes dumped into CSV format."""
import marshal, subprocess
def Run( cmd ):
"""Run supplied perforce command and return output as a list of dictionaries."""
results = []
cmd = "p4 -G %s" % cmd
# Run the Perforce command using -G option to get results as Python marshalled dictionary objects.
process = subprocess.Popen( cmd, stdout=subprocess.PIPE, shell=True )
# Harvest stdout output until end of file exception is thrown.
try:
while 1:
output = marshal.load( process.stdout )
results.append( output )
except EOFError:
pass
finally:
process.stdout.close()
return results
if __name__=="__main__":
import csv, datetime
# Perforce command to run, fetch the latest submitted changelists.
cmd = "changes -L"
# Get results.
results = Run( cmd )
print "'p4 -G %s' - %d" % (cmd,len(results))
# Convert timestamp into human readable form.
for result in results:
result['timestamp'] = result['time']
result['date'] = datetime.datetime.fromtimestamp(int(result['time'])).strftime("%d/%m/%Y")
result['time'] = datetime.datetime.fromtimestamp(int(result['time'])).strftime("%H:%M:%S")
# Write results to file.
f = open( 'p4.csv', 'wb' )
w = csv.DictWriter( f, results[0].keys() )
w.writeheader()
for result in results:
w.writerow( result )
f.close()
This script could also be used to export the results of most Perforce commands to CSV simply by changing the cmd string (line 25). It could also be adapted to export into a different format e.g. JSON, XML or even into a database like SQLite.
Using Perforce Counters to control syncing
Perforce has a handy system of built in variables you can use which are visible to all users for a particular server, these variables are called Counters. Counters are very handy way to distribute Perforce specific information e.g. the last change list that successfully passed automated testing.
Its relatively easy to write a batch file to sync to the change list specified in a counter. Although it isn’t pretty due to the limitations of batch scripting: first pipe the output from the p4 counter command into a temporary file, use the contents of the file to set a batch variable, then delete the temporary file (to keep things tidy) and finally call P4 sync with your Perforce path and the new batch variable.
echo off REM Get change list specified in counter. p4 counter <COUNTERNAME> > CL.txt set /p CL= < CL.txt del CL.txt REM Sync Perforce to change list specified in counter. p4 sync <P4PATH>@%CL%
I usually specify the port, user and client when I am using Perforce from a script to make sure the correct server/user/client is used as I login to many different Perforce servers so its not guaranteed that the correct server is setup in the command line environment. To specify the port, username and client spec to use insert the following after ‘p4′ and before the sync/counter command.
P4 -p <PORT> -u <USERNAME> -c <CLIENT> <CMD> <CMD ARGS>
I use a script like this to avoid syncing to known broken change lists, which is a big time saver
Running WordPress on Mac OS X with XAMPP
- Download XAMPP for Mac OS X from http://www.apachefriends.org/en/xampp.html.
- Open the xampp dmg file and copy xampp into the applications folder.
- Run the xampp control app and start apache & mysql.
- Using your browser, go to http://localhost/xampp/splash.php
- Select your language then click phpMyAdmin.
- Create a database called ‘wordpress’ using ‘utf8_unicode_ci’ collation.
- Download WordPress from http://wordpress.org/download/.
- Unzip WordPress into /Applications/XAMPP/htdocs/.
- Copy the following details into your wp-config file, and save it as wp-config.php These are the exact details you need for Xampp to work because the default user in phpmyadmin is called ‘root’ and there is no password:
- DB_NAME is ‘wordpress’.
- DB_USER is ‘root’.
- DB_PASSWORD is ”.
- DB_HOST is ‘localhost’.
- In your browser, go to http://localhost/wordpress/wp-admin/install.php to install WordPress.
- Once WordPress installation is complete go to http://localhost/wordpress/ to verify the installation.
- To get automatic updates of WordPress and installation/updates of plugins and widgets working do the following:
- Make a note of your username e.g. ‘Daniel’.
- Open /Applications/XAMPP/etc/httpd.conf as root/admin e.g. “sudo open -e ‘/Applications/XAMPP/etc/httpd.conf’” from terminal.
- Find the following lines:
User nobody
Group admin - Change the lines to the following:
User <Your Username>
Group staf f - Save the file.
- Restart XAMPP and visit http://localhost/wordpress/ you should now have a fully functional WordPress installation you can experiment on without effecting your live blog.
The next step is to mimic the configuration of the live blog on the test blog by:
- Importing the posts and comments from the live blog.
- Installing the same theme, widgets and plugins.
- Configuring the theme, widgets and plugins.
You can now start hacking away on your test blog without worrying about breaking your live site!
WordPress filters for Google Analytics
If you use Google’s excellant Analytics service to track traffic to your WordPress blog you may have noticed that days were you have been doing a lot of administrative work or writing on your blog that you had large traffic spikes. This is due to Google Analytics recording all your administrative and writing related visits to your site and this can be quite misleading if your trying to improve your traffic.
Fortunately there is a very simple way to configure Google Analytics to prevent administrative pages and post preview pages being recorded as genuine site traffic.
- Login into Google Analytics.
- Click the ‘Analytics Settings’ option in the top left:
- Click the ‘Edit’ option for your blog’s site:
- Click the ‘Edit’ option for the ‘Main Website Profile Information’ section:
- Enter ‘preview=true|wp-admin’ into the ‘Exclude URL Query Parameters:’ textbox and click ‘Save changes’:
Google Analytics will now ignore all visits to WordPress’es administrator pages (which are in the ‘wp-admin’ subfolder on a typical install) and also ignore any page previews (which have ‘preview=true’ in the URL). You can now administrate your site and write posts without worrying about your actions skewing you site’s traffic data!
The Evils of Global State and Singletons
In this Google Clean Code talk, Miško Hevery presents the evils of global state, how this relates to Singletons, testing and what to do about it.
Questions starting at 31:20 are pretty interesting.
But I can write that in…
I have lost count of the times I have heard a developer exclaim ‘but I could write that in X days’ when discussing adopting an existing piece of technology developed somewhere else. This is usually given as a reason not to adopt an existing external solution but to instead write a custom solution with some minor improvement or feature. There are two big flaws in this line of reasoning.
The first is the assumption that you could create something with the same quality level as the existing solution in the stated number of days; most likely you will be lucky to have a functional prototype with minimal functionality. It is highly unlikely that you will have produced something that is as well tested, optimised and documented as the existing solution.
Secondly, as developers writing software which we intend to sell to users to pay the bills, we should be focused on the core features that define our product. It is these core features we should be pouring our energy and time into designing, building and testing. Developing alternatives to existing technology that is not in our core feature set is a waste of valuable time and energy and will not differentiate our product from that of our competitors.
We should instead be looking to leverage as many established technologies or components as possible when developing new systems as this lets us spend most of our time where it matters most on our core differentiating features.
