In my Inbox this morning

We provide the ointments you require at really reasonably priced.

Hmmm. Seems like the spammers are moving into haemoroid treatments.

The easter egg was the random text added to defeat the spam filters. This comprised a snippet of Hans Christian Anderson and two nibbles from Jane Austen's "Persuasion". I'm particularly taken with the phrase "all histories are against you" which from now on I shall regularly drop into arguments.

Programming: is it an art?

Ming Chow writes in his O'Reilly blog that a computer program is no different than a novel. Hmmm. Obviously both books and programs comprise words ¹ whose organisation is governed by syntax; both are written. But as activities writing a novel and writing a program are completely different.

For a start novelists are writing a story to please themselves. Programmers, on the other hand, are writing a set of instructions for a machine to meet their users' requirements. Novelists work on their own. Programmers usually work as part of a team. This means their work has to conform to common standards and have a consistent presentation. The only writers who have to work like that are a bunch of Parisian surrealist poets playing the Exquistite Corpse game.

I would particularly take issue with this statement:

Source code should be read like a novel, and great programs should be planned so like any great novel.

I don't think source code should read like a novel. Even back in the days when I was writing COBOL the code was modular. In the modern world of object-oriented programs, loose coupling, configuration files and code re-use any resemblence to linear narrative has gone right out the window.

Furthermore, few novelists do plan their novels in any great detail. Rather they start off with an initial idea, invent a character or two and see what happens next. It's not unusual to hear a novelist say in an interview that they were surprised by what one of their characters said or did. Most have only the vaguest idea of how the plot will resolve itself until they get to the end of the book. Which, on reflection, does sound a lot like eXtreme Programming. That doesn't do lots of planning up front either.

As someone who studied history at university rather than a scientific discipline I'm alert to the possibilities of regarding programming as an art. Although there are some similarities some similarities with novel writing I would regard film-making as a much more analogous activity. There's teamwork, the studio system, the need to meet a budget, open source as a Woody Allen/Russ Meyer independent movie maker, etc. Indeed, watching Lost In La Mancha, a documentary about Terry Gilliam's failed attempt to film Don Quixote was just like watching a film about a computer project crashing. But writing software is primarily an engineering activity. I find I get much more insight from pondering the metaphorical implications of structural engineering, classical architecture and town planning. These all feel like activities that have a greater bearing on programming than reading a biography of Charles Dickens.

---

^{1. Except for languages like BF of course. (back)}

OTN Forums: here we go again!

Oracle have updated the forum software underlying the Technet forums. Which means at the moment they are are frequently unavailable and currently wholly unreliable. This is the third time I have been on the receiving end of an upgrade to Oracle's forum software and every time it's been shambolic.

It's easy to mock Oracle's incompetence here, and I will. As someone who uses the OTN forums quite frequently (I am, after all, an ACE) I am on the receiving end of crashes, lost posts and general frustration. So mockery is my recompense.

I know JDeveloper promises "one click deployment" to iAS10g servers but we are supposed to test our code before clicking that button. The fact that the web site is falling apart at the moment - dodging formatting, missing pages, hideous Java error stacks and grindingly slow performance - is inexcusable. Whatever else I may think about Java and J2EE it is a language blessed with a plethora of cool testing tools. Still, at least when we do eventuially get to post stuff we have graphical emoticons instead of boring text ones :)

There is a more serious point: the forums are a public demonstration of Oracle's abilities in the web environment, and this is where Oracle want to make a big splash. So they ought to regard the forums as an advertisement for their abilities in this sphere. After all, if Oracle can't make an iAS10g site highly available and resilient what chance is there for the rest of us?

They certainly ought to make sure the site is stable before publicizing "an improved user experience overall" on the OTN homepage.

JDeveloper: Like Aretha Sang...

My mate Duncan Mills recently complained of the lack of respect shown towards him by some Java heads on account of his choice of IDE. Now Duncan is a good bloke and I'm sure he would still maintain that JDeveloper is the best Java programming environment even if he didn't work for Oracle Corporation. But scepticism is understandable: JDeveloper was always going to be a hard sell to the Java community.

Firstly, JDeveloper is not open-source, it's proprietary. Java heads tend to make a fetish of platform-independence, and so suspect anything that threatens to lock them into a single supplier. Of course, Java itself is proprietary: the IPR belongs to Sun, they just give the JDK away to all and sundry. To paraphrase the Free_Software_Foundation, Java is free as in free beer not free speech. But at least IDEs like Eclipse, by being open, allow everybody to get involved.

Secondly, JDeveloper is belong to Oracle. Famously, Oracle make an RDBMS. Java heads, by and large, loathe RDBMS. They may tolerate open source quasi-databases like MySQL but they utterly detest commercial big iron databases like Oracle. Partly this is because relational database design is orthogonal to object-oriented modelling. But also the concept of platform independence means that the persistence layer of any system can supposedly be a database or a flat file, so why sweat the details. Either way there is a general reluctance amongst Java programmers (and other OO practioners) to tangle with RDBMS. In its extreme this leads to half-baked ideas like Prevalyer or hand-rolled persistence packages. More common is the tendency to use dont-worry-about-the-database tools like Hibernate. In such a climate few people are going to look favourably upon a tool developed by an RDBMS vendor for the purpose of making it easier to build Java front-ends to database systems.

Like it or not, people will regard JDeveloper as a Java version of Forms, that is a tool for building front-ends to Oracle databases. The fact that it is genuinely platform independent, standards compliant, etc. will not sway them. I know people who switched from using JDeveloper to using Eclipse, simply because they thought using the de facto industry standard would look better on their CVs. As they are now ex-colleagues of mine I guess they bet the right way.

Of course, Duncan is correct to say people ought to be more respectful of other people's opinions but it ain't going to happen anytime soon. Probably the two largest demographic groups on the net (and certainly amongst the l33t h4x0rz who care deeply about IDEs) are

• potty-mouthed teenagers; and
  
• social inadequates who get on better with machines than with people.

These people pro-actively scorn everybody else (to avoided being scorned first). Expecting respect from them is like expecting cabbies and bicycle couriers to obey the Highway Code. Given that the web is a tool that provides an instantaneous conduit for insults, with a virtual guarantee of no meaningful comeback, the real wonder is that there civilised discourse at all.

Anti-symbols in public signage

Whilst travelling on holiday recently I noticed that airports and aeroplanes have added a new icon alongside the signs for Gents' and Ladies' toilets. It's an upside-down feeding bottle, indicating the location of a breastfeeding room..

I think this is actually an anti-symbol. Why? Simply because if you've got your baby's milk in a bottle you don't need to hide away in a room when you're feeding her. Of course, the icon has to be euphemistic: a literal representation would offend exactly the same people who are offended by the sight of mothers nursing in public.

Oracle: Logging in PL/SQL

Logging is one of those little utility functions that we shouldn't be without but generally we never get around to implementing. Logging is not just for errors - important though that is - but for dumping trace messages, diagnostic information and of course debugging. Such instrumentation is crucial to the discovery of when, where and why a program went wrong. Furthermore, properly instrumented code can be a useful tool for assessing the virtues of a prototype program and for catching buggy or poorly performing code before it gets released into production.

Yet most developers still seem to be using DBMS_OUTPUT.PUT_LINE() for these tasks. DBMS_OUTPUT.PUT_LINE() has one virtue - ease of implementation - but it has several drawbacks:

• It requires an interactive session like SQL*Plus to view;
  
• It produces no permanent output (without a spool command);
  
• It may contaminate an application that needs to output information to the actual user;
  
• We only get the output once the PL/SQL program has completed;
  
• It doesn't work with inside a FUNCTION;
  
• There's that pesky 1000000 character buffer limit.
  

So I think we can agree that DBMS_OUTPUT.PUT_LINE() is not suitable for anything other than the most cursory of debugging tasks. Yet, many people have nothing else in their toolkit.

What must be done?

What are the characteristics of a good logging solution?

Persistence

The log must be permanent. At the minimum this means logging to a file or a table, but it is nice to have the option of directing output to other destinations such as syslog.

Structure

The log must be more than a free text field. At the very least we need to be able to search on Who, What and When.

Message levels

We need to be able to distinguish different types of message. The obvious ones are ERROR, INFO and DEBUG, but it is useful to be able to define custom levels.

Filtering

We need to be able to enable the logging by message levels. Typically we would only want to run at DEBUG level in a development environment. In production we would normally log only ERROR and certain types of INFO messages. However, we must be able to change this behaviour easily, so if we want to run production with DEBUG on, we can. Nice to haves include the ability to direct output for different message levels to different destinations and the ability to log at different levels for different subsystems.

Robustness

It is crucial that the logging doesn't fail. If it does fail it must at least report it failure. Furthermore it mustn't raise an exception and cause the program it's logging to fail.

Acceptable impact

We want all this and a small performance hit. Inevitably logging will consume CPU cycles, but provided this is kept to a reasonable level the price is worth the asking, as the benefits of logging are plentiful.

Interface

The public face of the logging solution ought to conceal the workings of the program from the programs that use it. Of course, this is just standard good practice, but it's particularly crucial with something like logging as it is used throughout our application. We want to be able to tweak the logging code without having to rewrite every single application.

Enter stage left LOG4PLSQL

The project I am currently working on consists largely of a fully automated data warehouse: the user interaction is limited to the data mart on the front. The rest of it is long running batch jobs. Logging is crucial: indeed, there is no other feedback for the success, failure or progress of a job. Although I had some scraps of code for logging errors I lacked a full Monty logging implementation that I needed here. So I turned to LOG4PLSQL, an open source package. The LOG4PLSQL developer Guillaume Moulard was inspired by the open source java tool log4j, which is as close to an industry standard as you can get. So much so that even though Sun has introduced its own logging package in JDK1.4 I suspect most projects still use log4j.

Let's assess LOG4PLSQL under those headings...

Persistence: 8.5/10

LOG4PLSQL consists on a table with a PL/SQL package acting as an API. It also provides hooks to write out to the Alert Log or to throw a .trc file. Furthermore it has the ability to integrate with log4j, which adds further options (of which more later). The package uses DBMS_PIPE to connect to a log4j session running in the background. The key aspect of the PLOG API is the CTX PL/SQL record type, which stores metadata for the logging of the message.

Structure: 7.5/10

The TLOG table has columns for user, datetime and context as well as a message text. The datetime is an Oracle DATE column, but it also uses the V$TIMER view to get hundredths of a second. It would be neater if it used the TIMESTAMP data type but the package is meant to be runnable on Oracle 8 installations. I would also like to see TLOG store the SID.

Message levels: 7/10

The package defines five default message FATAL, ERROR, WARN, INFO and DEBUG, plus ALL and NONE as the outer bounds logging. These are actually constants, mapping to numeric values. Despite being "inspired by" log4j LOG4PLSQL has a different scale from that tool. It runs from 0=NONE to 70=ALL whereas log4j runs from 10000=ALL to 50000=NONE; this diverge has a key impact when it come to the integration with log4j later on.

Filtering: 7/10

LOG4PLSQL has two mechanisms for assessing message levels and deciding what to do with them. Firstly, a PLOG.CTX can be associated with each message: this allows us to individually specify the treatment of the message - write to alert log, store in table, send to DBMS_OUTPUT, etc. If no CTX record is specified the package uses a default record (and we can set the values for that default). Additionally LOG4PLSQL allows us to set a default logging level: messages of a message type lower than that level are discarded. The problem is that the logging level is hard-coded in a package specification. This is very efficient from a performance point of view but it makes it awkward to alter the level. Indeed, in a production environment most DBAs would be profoundly reluctant to edit a package spec at all, and certainly not without a change request.

We could also filter log messages by using log4j. However, because the LOG4PLSQL message levels do not map to the log4j levels we cannot use log4j filtering methods to direct different levels of messages to different appenders.

Robustness: 9/10

No complaints on this score with regards to normal running. The only problem has been with the installation scripts. The packages are annotated with PLDOC comments so that HTML documentation can be generated using the Javadoc tool. Unfortunately the annotations are not correct, which leads to error messages. So Moutard has provided two empty scripts param.sql and return.sql which suppress the error messages but throw errors of their own because some versions of SQL*Plus don't like empty files. The correct solution is to ensure that every line of a PLDOC comment starts with an asterisk.

When using log4j we have to be certain that the background instance is running. Otherwise the DBMS_PIPE gets clogged up and no further logging happens until the pipe is flushed.

Acceptable impact: 6.5/10

Writing records to a database is a fairly expensive process; logging TRACE and DEBUG messages to the TLOG table can have a severe impact on the performance of PL/SQL code. Also, the checking of the logging level is fairly cumbersome and is an overhead in its own right.

Interface: 7/10

The PLOG package exposes the workings of LOG4PLSQL to a degree unsuitable for production use. However, I think this is fair enough: the facade ought to be application specific.

Conclusion

So, to sum up, LOG4PLSQL offers a fine starting point as a logging package if you don't have one of your own. It has a solid set of functionality and does pretty much everything you would want. There are a couple of niggling things, but as it is open source, we are free to tweak the program to suit ourselves.

Having said all that I am now working on my own logging package (a man's got to have a hobby). LOG4PLSQL ultimately is engineered (and in some respects over-engineered) to suit Guillaume Moutarde's needs. My needs are somewhat different: particularly I need better performance out of logging to a table, I need to be able to change the logging level dynamically. I want certain types of log message written to V$SESSION_LONGOPS. I would like to be able to have different logging behaviours for discrete subsystems. I need a better fit with the industry standard tools: log4j and syslog.

Finally, having read Michael Feather's thoughts on stunted frameworks, what I want is a simple core logging package that is easy to extend for specific applications. My current application consists of a core of (amended) LOG4PLSQL with three packages over it; it's a horrible gnarly mess of application specific stuff (how subsystems are handled) and general stuff (writing out to syslog). Fortunately the public interface is quite clean, so (theoretically) it will be a cinch to swap in my new logging package - except I'll probably have left the project by the time I get it working properly....

Rebecca Horn: Bodylandscapes

Sunday afternoon at the Hayward Gallery to see its new exhibition Rebecca Horn - Bodylandscapes.

To be honest, we probably wouldn't have gone to this show except for a particular confluence of events: my mother offering to baby-sit, there being nothing worth seeing at the Clapham Picture House on Sunday and my wife having an annual membership to the Hayward Gallery. So off to the German avant-garde we went! The temptation to pursue the body metaphor and describe this exhibition of multimedia installations, performance pieces and drawings as "a bunch of arse" is childish and unfair but alas irresistible.

Actually some of the pieces are rather intriguing. The room of weird costumes - a unicorn headpiece, body-sized wing-like fan - are great. The blurb talks about the influence of surrealism, alchemy, esoterica but these seem like a Martian's attempts at fetishistic bondage gear. And why not?

Alas, it's mainly downhill from there. The installations are mainly mechanical pieces (not all of them working). There is a contrast between the solidity of the actual mechanics - gears, cranks, brakes, linkages, armatures - and the vacuous portentousness of the titles. Under no circumstances should you attempt to read the poetry on the walls. Maybe it is better in German.

One piece I quite like comprises a set of armatures hung with sheets of music. A crank lifts and raises the pages like wings. It's called Floating Souls (you see what I mean about the titles). Some of the music sheets are splashed with red or black ink and a raven's feather hangs from one of the armatures. Both of these (along with eggs, books and golden pointers) are what the Guide calls "recurring motifs", the technical term for shortage of ideas. It struck me as a rather teenage attitude: the repetitious drawing of symbols on your homework was supposed to imply hermeneutic wisdom, when all you were really doing was skulking in your bedroom. Also like teenagers, Horn often uses music (ethereal synthesised drones and overtone singing) to make her seem deep.

The worst offences are the drawings. Now that I have a four-year old I am more sceptical about the artistic merit of such scrawls and splashes. The major difference between the drawings of my son and Rebecca Horn is that hers are centred whereas Fred tends to start at the left-hand of the page and work rightwards. But I could take one of his drawings, give it a label like The Paradise Bird Pierces My Heart or The Burning Gardens Of Smyrna and you might think, 'Hey! I can almost see what the artist meant!" Art is whatever someone will buy.

Funnily enough, although I didn't like most of the work I find I have quite a lot to say about it. It certainly generated more conversation than popping over to the Joshua Reynolds exhibition at the Tate would have done. Makes you think, mmmm?

Robinson In Cyberspace

It's ironic that my attempt to purchase Robinson In Space on DVD (along with London) should turn into a commentary on modern consumerism. The BFI released this DVD on the same day that Optimum re-issued Last Year In Marienbad, and it so happened that I had a £5 off voucher from HMV that had to be used by end of May. Synchronicity! So off to Oxford Street I went. Hmmm, they had Last Year In Marienbad, and they had a space in the racks for London/Robinson In Space, but no actual copies of the DVD. Oh well, obviously not in stock yet.

I had a look on the HMV web site. Now it had Robinson In Space but not Last Year In Marienbad. Besides I couldn't use my voucher there anyway. I popped into the Oxford Street HMV again at the end of the week (look, I genuinely just happened to be passing), but it was still not in the racks. I noticed that HMV was flaunting an award from some music industry gong fest recognising it as Music and DVD Retailer of the Year, despite its apparent inability to offer two DVDs in the same retail space at the same time.

At the weekend I noticed in the Guardian's Guide booklet an advert for Robinson In Space issued by the BFI but with sign saying Available at HMV. So, off to HMV again on Tuesday, the last day of May, last chance to use my voucher, to discover that that's "available at HMV" as in "not available at HMV". Now I could have spent some time railing at the staff about futility of it all - I mean, what's the point in advertising something if you're not actually going to stock it?

But instead, I just did what I should have done first off. I went shopping in cyberspace. Amazon had both DVDs at a fiver off apiece, and my order qualified for free shipping. So it cost me less money than buying it from HMV would have done. Sure I've got to wait for it to be delivered, but HMV has already made me wait nine days with no guarantee of it being in their store even a week later.

So what does this say about the state of modern commerce? Well, obviously, Amazon is going to win. You cannot beat better price and more or less an infinite number of items in stock. I do feel sad about the decline of the small independent book and record shops, but Waterstones, Borders, Virgin, HMV etc. are going to kill off those anyway. So if Amazon can really stick it to them that’s about as good a result as we're going to get. Of course, Amazon may get bloated and complacent like HMV but the web makes things different. They start to take their customers for granted in the full knowledge that another book supplier is only a click away.

Robinson In Space

Possibly I am the only person in the country to be excited by the news that the BFI is issuing Patrick Keiller's "Robinson In Space" on DVD. Partly this is just because I thought it never would make it out. After all, it is a rather niche film, consisting as it does of a series of static, peopleless shorts of modern Britain with the narrative drive coming solely from a commentary that touches on Daniel Defoe, spying, shopping, "The Anatomy Of Melancholy" and the annual tonnage of goods passing through Tilbury Docks, among other things. It just goes to prove how long that long tail really is.

The film is made more precious in my mind by the circumstances of the only time I saw it. It was showing in that fleapit du jour, the Prince Charles, off Leicester Square. The cinema publicity had managed to mangle the schedule for the week, and the precis of "Robinson In Space" had been replaced by that of another film (probably this one). Consequently, a large chunk of the audience were expecting a film in which radio astronomer Charlie Sheen makes contact with an extra-terrestrial intelligence. What they got instead was:

[EXT. WEST BROM. DAY.]
Base of a clock tower.
(V.O.) When the bus finally arrived, Robinson had disappeared to a sexual encounter with a stranger he had contacted through the Internet while we were waiting at the bus-stop.

The astonishing thing was how long some people waited before they realised Charlie Sheen wasn't going to turn up and left. Those who remained got a fascinating meditation on Britain's history and modern decline as a commercial and trading power.

Of course, the film is a lefty's view of the impact of Thatcherite Conservatism so it will interesting to see how it holds up after eight years of, well, Blairite Conservatism. Really this is a film that ought to be remade every five years (instead of tedious seventies TV retreads) but I can understand why Keiller would not want to keep poring over the fortnightly edition of Port Statistics. The trouble is, nobody else wants to either. The last time I pondered the annual tonnage of cargo passing through Tilbury docks was when I was doing my geography O-Level. At the time I couldn't have been less interested. Now I appreciate that all history (and most if not all human activity) is economics. But that still doesn't make the prospect of those HMSO statistical reports anymore appetising. So watch "Robinson In Space" and be thankful to Mr Keiller: he read those dreary publications so we don't have to.