Builders apprentices often get sent on pointless endeavours, such as going for “a long stand”, “tartan paint” or a “sky hook”.

I think I saw the programming equivalent today – proving that a GUID is not unique.

The poor programmer does get a bit much stick, but he has just tried to write a loop that will take 10790283070806014188970 years to compute. He then asks if using multiple threads will help solve his problem…

I’ve recently been learning Clojure, a form of Lisp that runs on the JVM.  There are already a lot of resources about Clojure on the internet, but they vary in depth and usefulness. Therefore, I’ve compiled a bit of an overview of the various resources, in the hope that it helps the new Clojure (and Lisp) programmer get started:

• Why you might want to consider learning/using Clojure is concisely covered by Rich Hickey here.
• One of the first decisions you will face is which editor to use. Unfortunately, this isn’t necessarily an easy decision. Clojure and Lisp programmers typically use a REPL to interact with their programs as they are writing them. You want your editor to have some level of integration with a Clojure REPL. This Getting Started resource covers most of the common choices. It’s probably worth saying that Emacs offers one of the most mature and powerful environments and is a good choice if you are already comfortable with the editor. If you’re on Windows, Clojure Box offers a quick way to get started with Clojure and Emacs. Personally, I’ve tried  the Eclipse plug-in counter-clockwise, but had some problems with it and I currently use Vim set-up according to this guide at write-quit. (I see counter-clockwise has had a new release since I tested it, so it may well be worth revisiting).
• Once you have your environment set up, you will probably want some sort of quick start guide. One of the most commonly referenced tutorials is R. Mark Volkmann’s at ociweb. This is certainly a very valuable and authoritative resource, but I found it rather hard-going for a first contact. The Intro to Clojure video series was highly rated by this stackoverflow question, but I can’t say I want to use videos to learn a language. Satish Talim suggested Clojure Notes, which looks like a good introduction and may be the best place to start.
• Regarding books, Stuart Halloway has ported some of the Practical Common Lisp book (original available free here) to Clojure. This leads me onto his Programming Clojure book, which is the best resource I have found so far (but not free). There is also a wikibook available, but it seems incomplete.
• Planet Clojure is your best reference for new and interesting Clojure posts (they describe themselves as a “meta-blog” of various Clojure hackers and contributors).

Hope this helps someone!

Permiere Pro has lately proven itself to be the singular worst piece of software I have ever had the misfortune to use. It has cost us far more time than it has saved, certainly several times its RRP in wasted hours. Normally I’m a software developer, but a project I was working on required me to do some basic video editing and I was shocked by the poor quality of the software being used.

In case you don’t know, Premiere Pro is “professional” video editing software, in a similar category to Avid Xpress and Apple’s Final Cut Pro. And to be fair, Premiere certainly seems to be fully featured. It’s just that it doesn’t work. It randomly crashes. Constantly. And when it’s not randomly crashing, it’s repeatedly crashing at the same point.

The first thing that irritated me is that I’m running it on a split new laptop with the maximum amount of memory (which is admittedly constrained to < 4 GB as I’m using 32-bit Windows XP) and yet it gives me messages such as “Adobe is running low on system memory. Please save your project and proceed with caution.”. Quickly you realise that it’s best to give Adobe full run of the machine and close any other programs which use memory (like Web browsers).

The next thing that perplexed me was that every time I close Premiere, I get a message telling me that Premiere unexpectedly quit and offering to send Adobe the crash report. No, Adobe, the quit wasn’t unexpected — I normally exited the application — but apparently you shouldn’t do that in Adobe land. This started to bug me, so I filled in the crash report and attempted to send it. Except I can’t, because my e-mail address isn’t valid according to Adobe¹.

Quite often, Premiere doesn’t want to import certain media. Sometimes it tells you this, other times it imports the footage but only displays a blank screen or inverted colours. This is irritating, but can normally be worked around by using free software such as ffmpeg or Virtual Dub to re-encode the footage. (Yes, these free programs seem to handle various video formats better than Adobe’s £750 payware. Adobe even suggest using VirtualDub themselves (see Solution 6)).

However, these issues pale into insignificance once you face the beast that goes by the name of “Export Media”. This launches what is sort of an associated application known as “Adobe Media Encoder”, which renders your project into your final move file (e.g. .mov, .avi or .mpg). Sometimes this works flawlessly.  Sometimes it will stop at a seemingly random point in the encoding and the logfile will contain such revelations as:

...
 - Encoding Time: 00:01:07

2009-10-28 14:18:47 : Encoding Failed

-----------------------------------------

Error compiling movie.

 Unknown error.

-----------------------------------------

Hmm, “Unknown Error”. Doesn’t really help, does it? Restarting the export will result in it stopping at exactly the same point. This forces you to come up with random hypotheses as to what caused it crash, testing them in order until you either fix the problem or give up. Sometimes re-encoding the original video footage can fix things, which would suggest the video is corrupted (although it will play fine in Premiere and other video software).

I had thought “Unknown Error” was the least useful error message I’d seen. I was wrong. Sometimes the encoding fails part-way, yet Media Encoder reports “Encoding completed successfully”. I don’t want to take personal offence from a bit of software, but it stopped with the progress bar halfway, created half a video file then declared it was happily done. Surely that’s simply rude?

Of course, you can attempt to find and follow the official Adobe advice. The first “note” here tells you to restart your computer “to refresh its memory” (read: not only is our software full of bugs, its also full of memory leaks). Solution 1 is to turn various effects off (read: lots of our effects have bugs). Solution 2 is to change the encoding (read: our codecs have bugs). The third solution annoys me even more: “Quit any applications that may be interfering with the render process”. Why? Surely Adobe should be able to share memory and resources with other processes? Solution 4 is “Use a different codec for your export” (read: our export function is full of bugs). Solution 5 is “Disable Auto Save” (read: our basic functionality breaks our other functions). I tried all of these; no dice.

In the interest of of ending on a positive, helpful note, I have the following suggestions for how Adobe can make things better:

• Replace “Unkown Error” wherever it occurs in the codebase. Handle exceptions properly – you’re meant to be a professional company and this is an very sloppy coding practice.
• Test the software. I’m using a split new machine from a vary large vendor. Why on earth doesn’t it work? There must be huge flaws in the testing process at Adobe.
• Remove support for features that don’t work. A particular encoding doesn’t work a given percentage of the time? Either fix it, or don’t pretend to support it.
• Look at your own advice. Most of these “solutions” indicate deficiencies in the underlying software.
• Provide some sort of cuddly toy with each purchase of Premiere Pro. Although this won’t make the software any better, it will reduce stress by giving users a representation of Adobe that can be tortured during periods of stress brought on by attempting to use the software.

1. I think this must be because my work address contains too many dots for Adobe. I’ve redacted my address as this post does not represent the views of my employer, just me. But trust me, it’s valid. [↩]

GridVoices have published my article “Cloud for Academia“, which takes a brief look at how various Cloud offerings could be used for HPC applications.

UPDATE 4/11/2009: This article has now been syndicated (if that’s the right word!) on HPCWire.

Package managers – your yums, portages, apts etc – must be one of the best features of linux. It used to be pretty impossible to keep an installation up-to-date, now it’s trivial most of the time.

I actually wonder if this is Linux’s killer feature – whilst your Windows installation slowly rots until you are prepared to face the pain of a major OS upgrade, your Linux installation has been incrementally improving itself. I reckon that Gnome overtook XP sometime in the last few releases in terms of features and (in some places) usability.

Sure, Windows will probably retake the lead with Windows 7, but how long will it be until the next major version of windows? XP is a staggering 8 years old, will it take as long until most people are prepared to update again? In that case, Microsoft may find they are in serious trouble by then, facing Linux distributions with with improved kernels, file systems and desktops, especially with distributions like Ubuntu pushing usability and user experience.

David Weinberger writes on “Joho the Blog” that “transparency is the new objectivity“. In the post, he explains how journalists have traditionally strived to appear objective, but today’s bloggers typically go down a different path and aim for transparency. That is to say, a journalist will not openly reveal their biases (which sometimes grow into hidden agendas) but many bloggers will happily wear their allegiances on their sleeve.

It’s common knowledge that media outlets typically favour certain strains of politics over others, for example Fox News is famously Republican and were arguably responsible for perpetuating falsehoods about the Iraq war. In the UK, the Daily Mail leans heavily towards the right and publishes very questionable stuff about immigration, whilst the Guardian leans towards the left (and at least attempts to confine its political rants to the editorials).

However, none of these outlets openly say this. The Guardian claims to live by the words “A newspaper’s primary office is the gathering of news. At the peril of its soul it must see that the supply is not tainted.” I couldn’t find an equivalent statement on the Daily Mail’s website, but AND, their parent company (how much did that address cost?!), claims “Our mission is to be the most trusted and relevant focal point in every community we serve, ensuring the best outcomes for people when making the important decisions in their lives.” (Although this arguably makes no claims to objectivity assuming it doesn’t affect their trust or relevance).

In contrast, the blogger Michelle Malkin openly declares her conserative allegiances and there’s no doubting LiberalOasis’s politics with the tag-line “where the Left is right and the Right is wrong”. For a British example, see Iain Dale, who actually stood as a conservative candidate and whose banner quotes are mildly amusing and sometimes reveal more about the person being quoted:

“Political intelligence in every sense” – Roland White, The Sunday Times
“Much wittier than your average Tory” – Lance Price, Former Labour Communications Director.
“I read it every day” – Adam Boulton, Sky News

The thing is, none of this is exactly new. Hunter S. Thompson pioneered Gonzo journalism which Wikipedia currently defines as “a style of journalism which is written subjectively, often including the reporter as part of the story”. Sound familiar? Throughout Thompson’s writing, he was often part of the story, as much making the news as writing about it (he didn’t go to the library to write his book on the Hell’s Angels – he spent a year living with them). Hunter managed to write a pretty definitive account of the 1972 presidential race, which reads with a refreshing honesty despite containing completely fictional elements (such as alleging Ed Muskie was addicted to the drug Abogaine and his advisers had flown in a Brazilian doctor to treat him, but how the hell did anyone not realise he was extracting the proverbial urine there?). My point being that Hunter forsook objectivity, yet still managed to deliver in authoritativeness through his “transparency” – there was never any doubt where Hunter’s allegiances lay. According to Wikipedia, Thompson said in an interview “Objective journalism is one of the main reasons American politics has been allowed to be so corrupt for so long. You can’t be objective about Nixon.” Of course, I’m not saying all bloggers can or should write with the vitriol of Thompson, but they should aim to be open about to which side they lean and how they came to be in possession of the facts – this way people know to check the opposing argument for themselves.

Weinberger is right – only with transparency can we ever hope to arrive somewhere close to the ever unobtainable “objective truth”.

Update:

A commenter on Hacker News pointed out that blogs are normally more analagous to editorials than news stories, and editorials are usually clear in their political bias.

I would still argue that for a newspaper to claim objectivity they should attempt much greater transparency. Some papers now allow comments on stories, which is good move in this direction (but who stops them deleting comments they disagree with?).

The GridVoices blog on Gridipedia has just moved to a WordPress installation. Back in April, I wrote this article for them, which looks at the similarities and differences between Grid and Cloud computing.

Normally when I’m programming in Java I’ll use Eclipse. However, in some cases Eclipse can be a bit heavyweight and I’ll fallback on Vim. The last time I did this I started to miss the ability to quickly and easily run unit tests on a per class basis. For this reason, I added the following function to my .vimrc which runs the appropriate test class for the current Java class.

function RunTest()

    let cla = matchstr(expand("%:p"), '^.*[/\\]src[/\\]\(test\|java\)[/\\]\zs.*')
    "still need to replace /s with .s
    let class = "java org.junit.runner.JUnitCore " .  strpart(substitute(cla, "/", "\.", "g"), 0, strlen(cla) -5)

    if match(class, "Test") == -1
        let class = class . "Test"
    endif

    echo class
    echo system(class)
endfunction

map <F6> <Esc>:echo RunTest()<CR>

The code assumes that your test directory structure mimics your source directory structure and all the names of your test classes are the same as the source classes with “Test” appended e.g. if you are editing “src/mypackage/MyClass.java”, it will attempt to run “test/mypackage/MyClassTest”. (If you are currently editing a test class, it should run that class).

I also mapped F6 as a shortcut to the function.

I really don’t know Vim script TK at all, so I’m sure the code could be a lot cleaner. Still, it was a big help for me, and I now find it quicker to run the correct test in Vim than Eclipse.

I’m the author of the diffxml tool for comparing XML documents. In this post I’d like to explain why you might want to use diffxml to compare XML documents rather than traditional text tools such as the UNIX diff command.

There are two things that diffxml understands that diff doesn’t; the syntax of XML documents (e.g. <br/> is equivalent to <br></br>) and the hierarchical structure they represent. (more…)

Tergiversations. It’s a real word: tergiversate means ‘to change repeatedly one’s attitude or opinions with respect to a subject or cause.’

Richard Gabriel, “Patterns of Software”.

I nearly called this blog “Tergiversations”, after an essay by Richard Gabriel in his book “Patterns of Software“. It’s a very important book to me. I found the book in a bargain bin in the academic bookstore on my university campus at the end of my first year. At this time I was considering if Computer Science was the correct choice of course for me, or if I should do something completely different, perhaps in Arts & Humanities.

Gabriel’s book affirmed my course choice. In it, I saw someone who clearly loved his subject matter and was capable of deep insight into it. There was no doubt that Computer Science contained serious challenges and was worthy of anyone’s attention. More than this, I saw there was a very subjective side to the field, and that people’s imaginations – even their force of character – were of vital importance (of course, now I realise that this is probably true of most scientific subjects, but it was a revelation to me then).

The book itself is a series of essays, its title coming from an essay exploring how the ideas in Christopher Alexander’s books “The Timeless Way of Building” and “A Pattern Language” apply to software. It also contains essays on writing, programming languages and some more personal stuff. All are worth reading.

I wanted to call the blog “Tergiversations” to show that I was willing to listen to others and change my viewpoints accordingly. For me, one of the most striking differences between science and other disciplines is a willingness to concede, to accept that someone else’s theory is correct and yours isn’t. Of course, this largely occurs because many things in science can be objectively proven; you can’t argue about the existence of a black swan. At least in science, objective progress can be made, in artistic or political endeavour any progress tends to be subjective. I could argue all day against the BNP, but I couldn’t prove beyond doubt that their beliefs are misguided and immoral. However, I can easily prove that my Java code runs twice as fast as yours. (Interestingly, I recently read an article where these two worlds collided, and I’m just glad I don’t have to deal with Andy Schlafly)

In the end, I didn’t name the blog “Tergiversations”; it has the negative connotations of being evasive and desertion, whereas all I wanted to indicate was a willingness to change one’s mind and to try new methods (I’m sure Richard is well aware of this; I failed to remember the precise definition and the context he used it in until I re-read the essay).

To sum up; many thanks to Richard Gabriel for writing “Patterns of Software”, and welcome to “Feeding the Bit-Bucket”¹.

1. Yes, I know this captures none of the meaning I talked about. However, it does capture the likely end product of this blog, and some of the eternal aspect of writing. [↩]