Oracle Licencing IV: The Carnage Continues!

The Register in its delightfully sarcastic way breaks the news of the latest refinement to the Oracle licencing model. Basically,

Sun's UltraSPARC T1 chip will be multiplied by a factor of .25...the dual-core x86 chips from Intel and AMD will be multiplied by a factor of .50, whilst "all other" multi-core servers - mostly the Unix crowd - will follow the .75 rule.

Contrary to the their normal high journalistic standards El Reg don't provide a source for this story but the press release is on the Oracle Corporate site.

This is not quite the revolutionary new pricing strategy Larry hinted at in his OOW2K5 keynote. Instead it further muddies the waters when it comes to costing a server installation without helping those customers for whom none of the current pricing models are suitable. Since you ask, yes, my current client is one such case. They have lots of database applications but almost no Oracle installations. Looking at the roll-out of my current project, the Oracle licensing costs are a substantial part of the problem. The per user model doesn't work with web-based systems. The per employee model doesn't work with a workforce of forty thousand most of whom will use the system maybe once a week. Which only leaves the complex, expensive and - yes - unfair per processor model.

Upgrade your server and pay more to Oracle for the privilege? Hmm, it's a toughie.

Manageability: an Oracle fan gets illogical

Thanks to Doug Burns for pointing out this interesting article on the comparatitive manageability of MS SQL Server and Oracle by Buck Woody (crazy name, crazy guy!).

I was particularly struck by his assertion that SQL Server is easier to manage because it requires fewer steps to achieve any given task. Despite touting this as a scientific assessment Buck Woody fails to provide even the most basic information, such as which versions he's comparing. His "proof" of this statement amounts to an invitation to install MSSQL and see for ourselves.

On the other hand, there was a very interesting presentation at OOW2K5 called "Which Database Is Easier to Manage: Technical Case Study Comparing Oracle, SQL Server and IBM DB2" by Kevin Canady and Aaron Werman of Edison Group, Inc (not Oracle employees) who asserted the opposite. They presented a set of findings based precisely on counting the number of steps reuired to do common database tasks using the vendor's GUI management tool. Oracle have made a lot of progress in manageability in 10g and the Edison Group assessment is Oracle 10g is considerably easier to manage than MSSQL. They have published these findings as head-to-head slapdowns(Oracle 10g vs Microsoft and Oracle 10g vs DB2) but the three-way comparison was dead instructive. In some areas MSSQL is less manageable than DB2. Of course, Canady and Werman were comparing production versions, which meant MSSQL 2000; an old, old product but whose fault is that?

In the Q&A slot I questioned whether counting steps in the vendor's GUI is the appropriate metric for assessing manageability. Particularly for repetitive tasks a GUI is a lot less productive than even SQL Worksheet; besides, many experienced DBAs would have scripts to undertake common tasks. The presenters took the point, but it's the old case of measuring what is what measurable. We can count the number of steps it takes to achieve something in a wizard. It's a lot harder to compare how easy it is to achieve that same thing by the quickest possible means, because that might vary from DBA to DBA: my PL/SQL is quicker than my Python scripting but not as quick as your Perl scripting.

Aargh! HTMLDB is giving me JDeveloper flashbacks!

When I first started working with JDeveloper 3.0 in 2000 - we were not so much early as lonely adopters of BC4J (at least on this side of the pond) - the documentation was in a parlous state. We could get a first cut something working using the wizards, but as soon as we needed to go off piste we were lost. The in-built help text was rubbish; virtually nothing at all on BC4J, which had been introduced into 3.0 and had no link with any existing Oracle technology. All OTN offered were some tutorials and, later on, some How Tos; inevitably these never quite covered the areas that were troubling us the most.

Having spent a couple of days wrassling with HTMLDB I find I'm in the same predicament. Sure I can build straightforward Create/Edit/Delete pages quite nicely but now I want to build an integrated application and ... it's not easy. The document consists of tutorials and How Tos, which - you know what's coming - don't quite show me how to do what I want to do. The problem is the documentation doesn't seem to be written by people with a lot of experience in writing form-type applications. The two day tutorial doesn't even mention Master-Detail forms, which I would have thought is a fairly basic; even spreadsheets frequently have some header info.

Of course, the HTMLDB forum seems to have a lot more answered posts than I recall the JDeveloper forum got back in the days. OTN has quite a few sample applications, so perhaps I just need to install all of them until I find an example. And there's also blogs. So no doubt the information I need is out there, somewhere. It's just finding it that's going to be the issue. At least I hope so. I've been trying to build an LOV that has its query restricted by an existing value on the page; the resources on the interweb have offered me at least three different implementations , none of which worked the way I wanted.

I'm just stressing out here probably because I'm trying to be too clever too soon. I obviously need to stop being an active user and spend two days going through the tutorials, building applications I don't need, in order to get a handle on the basics. But with a heavy heart I'm afraid I have to tell my project manager to start cranking out the Excel again. It's not going to be as easy to implement his spreadsheet in HTMLDB as I thought. Blast.

Neology Corner: Reshoring

I am introducing a new coinage into the wild: reshoring. This is what happens when a project that has been offshored turns out to have been wrongshored (heh); when everything's gone pearshaped and the project is brought back to be sorted out. Sample usage: "Have you heard about Project FFIENNES? They're reshoring it from Bangalore."

Was my coining of this phrase inspired by any specific project I have come across? You might think that, I couldn't possibly comment.

UKOUG Development Engineering SIG

Yesterday was my first real SIG on my own. My co-chair Guy Mortenson couldn't make it and the agenda was wholly organised by me. So I await the feedback with trepidation. The good news is that Guy and I were re-elected unopposed. Not that the UKOUG is a banana republic, but as Chair I had to preside over the election process...

I think the agenda was quite balanced: we had two items on new technology (HTMLDB and JDeveloper, two items on old technology (Forms) and I filled Guy's clowning role by delivering a short skit called "The worst week of my professional career".

The core audience for the DE SIG remains Forms developers; there's some people using Java but no-one using PHP or .NET and no interest in .NET. As an organiser I find the problem ith Forms is finding something new to say. Pretty much the only topics that are in the slightest bit fresh are highly technical investigations into the plumbing of Oracle application server. Kavitha Prakash from Oracle Support confirmed this to me afterwards; she said that pretty much all of the calls Support get are about app server configuration, deployment and debugging; there are almost no calls on coding problems. Gavin Leith of Sopra Newell and Budge gave an interesting talk on JDAPI, a Java tool for programatically tweaking Forms programs. This would have been useful to know about two years back when I was migrating a Forms client/server project to 9i web Forms, but to be frank I hope never to touch Forms again.

The talk that seemed to generate the most interest was given by David Richard of on HTMLDB. This was actually part 2 of a presentation that he started to give at the previous SIG in June. This time he actually managed to demonstrate his case study application and show some of the wiring under the hood. The application was a tactical solution for the NHS. This both proved the complexity of apps that we can build with HTMLDB and (I think) hinted at the limitations of the tool: A4C was a fantastic project to knock up in five weeks but it would be a nightmare to maintain. The problems of configuration management and code visibility in a metadata repository would get too pressing. Still, it inspired me. Last Friday my project manager showed me his latest spreadsheet for estimating and I told him we should be doing it soem other way; as I type this I am installing HTMLDB so that I can do it better.

Lastly Duncan Mills demonstrated his favourite new features in the new JDeveloper 10.3. I have to say the Java Server Faces implementation looks very good. My sole reservation is that it is so huge. Even Duncan had to look at his crib sheets to wrangle some piece of syntax. He's been living with this for most of the year: if he doesn't know it all, what hope is there for the rest of us? This is a serious point. Just to build a pop-up LOV required choosing a widget from a list of many dozens of options. Can there possibly be enough time to master these tools before the Java caravan moves on and there's a whole new set of APIs to learn?

How To Be A Good Guru

In her engaging UKOUG presentation on being a newbie Lisa Dobson devoted a large chunk of time to Being A Good Newbie. That is, how to post questions on Oracle-related lists in the manner most likely to elicit a helpful response. We might summarise this advice as don't poke the tigers.

Lisa didn't have the OTN Forums on her list of, er, lists but we get a lot of newbies posting questions there. Partly it's just location, location, location: the Oracle site is the obvious first port of call when you need help¹. Also, it is not a forum where the really big beasts - the Oak Table chaps - visit and the level of discussion hardly ever descends to hex dumps of block headers and other such esoterica. Hence newbies are perhaps less likely to feel embarassed about posting simple questions. This is obviously a good thing but it does impose a burden on us soi disant experts who volunteer to answer their questions. So, in homage to Eric Raymond's seminal guide to list etiquette for newbies I am modestly proposing some advice to would-be responders. And, yes, I know I still regularly break these guidelines.

How to Answer Questions the Smart Way

#1. Don't answer questions to which you don't know the answer

Obvious really, but it's very easy to think you know something about the database that is no longer true, or maybe never was. For instance it has been a long time since anybody asserted that explicit cursors performed better than implicit ones, but people still proferred this advice long after it had ceased to be true. Be sure that if you do post something factually incorrect your peers will gleefully expose your bloomer to the wider world. It is better to post the right solution second than be first with a wrong one.

It is perfectly okay to research an answer. Indeed, one of the benefits of answering questions on the forums is that we discover stuff we didn't know. Questions coming from out of leftfield can tell teach us interesting (and sometimes even useful) things about how the database works.

#2. Explain yourself

Eric Raymond advises newbies to approach technical gurus in a supplicatory fashion. They are supposed to treat lists as resources of last resort, to kowtow before the collective wisdom and then present a detailed description of their problem, including complete specifics of environment and configuration, plus all the things they've already tried and all the manuals, whitepapers, etc they've already read, before humbly beseeching us for the merest crumb of assistance. Of course the ungrateful wretches never do this but we should still respond graciously.

It's not enough to dash off the correct answer. Consider whether its correctness will be obvious to the questioner. If not, try to explain why it's the correct answer. That way there's less chance that the questioner will be posting an almost identical question next week. Whenever possible include a link to the relevant part of the manual. Linking to a specfic heading in a chapter is better than just linking to the Table of Contents, but the manual is not always tagged the way we would like. Similarly, if you need more information, explain what else you need to know and why you need to know it. If you want (say) an explain plan and you suspect they don't know what that is, link to the manual.

#3. Give as little assistance as necessary

The most effective form of learning is discovering things for ourselves. Teaching someone how to diagnose their own code (use SQL> SHOW ERROR, look up the error, check the syntax in the online documentation) is better than just rewriting their code for them. Not least because in rewriting the code we are likely to break some business rule we do not understand.

#4. Show your workings

When answering a SQL related post it is always best to include a worked through example. Take a tip out of Tom Kyte's practice: use cut'n'paste from SQL*Plus to demonstrate that what you have asserted is in fact the case. This is a corollary of #1.

If, for reasons of time or environment (for instance you do not access to a database) it is acceptable to post untested code, provided:

• you indicate it is such;
  
• you think an untested code sample is more helpful than no code sample at all;
  
• you think it is probably correct. Heh.
  

Nothing exposes a poseur faster than a piece of code that doesn't even compile, so make sure you cover yourself.

#5. Use humour judiciously

Many people using the forums do not have English as their first language. Humour does not always travel well between cultures. Some questioners will not be expecting humour in a work context². Furthermore, even amongst English speakers, humour can be difficult to spot without the non-verbal signifiers that accompany barroom banter. In particular irony is a tough one to pull off. Still, humour is a Good Thing and can liven up some dry reads so by all means be witty (if you can - lame jokes are, well, lame). It is helpful to use emoticons to signify that your preceding line was intended humourously, even if they are a debasement of the high standards of English literature. Jane Austen never posts on Oracle lists anyway :)

Oh, and sarcasm is right out (see #6).

#6. If you can't say something nice don't say anything at all

Just like your mother told you.

There are no stupid questions only stupid answers. If someone has posted a question you don't think is worthy of an answer then don't bother answering. Remember: you're a volunteer, there is no compulsion, you are choosing to answer the question. So, the person is not wasting your time by asking a stupid question, you are wasting your own time by typing a stupid (angry, sarcastic or belittling) answer. By all means explain what is wrong with the question, show how it could have been phrased better or request more details. If some newbie has posted "my code doesn't work", ask them to describe the observed behaviour, to explain the difference from expected behaviour and to paste in the error message and callstack (if appropriate).

If somebody seems to be particularly unreasonable - posting an URGENT!!! question on Saturday and then whinging in follow-ups that nobody has responded - feel free to explain to them that you are a volunteer, this is a free service and there is no SLA. In fact, if they want an answer within a set timeframe they should stop being so darned cheap and spring for a support contract. Also, if I suspect the poster is a student looking for me to do their homework I always ask for a course credit (Not being American I have no idea what this means).

#7. Avoid jargon, baffling acronyms and idiolects

Fnord. This is just another way of demonstrating your superiority over the OP. Of course, like wit, you can use such stuff if you think your audience will get it. Alternatively, embed a link to the Jargon Dictionary, although this rather undermines the point of using abbreviations. LOL.

#8. Never never never just respond with RTFM. Not ever.

Telling some newbie "RTFM" is an act of pure arrogance. It just feeds the respondent's ego without helping that questioner learn anything, except maybe not to ask for help in the forum again. Barbara Boehmer taught me this one. Keep in mind that the Oracle FM is huge; there are dozens of books and the search engine fronting the online version ain't that great. At the very least, post "RTFM" and a link to a relevant part of TFM. That way at least they will know where to go the next time they need to find out something.

STFW is at least as bad and arguably worse. Googling for information about Oracle problems takes a lot of skill, not just in filtering queries but in knowing which results can be trusted. There are plenty of snakeoil merchants out there. (You may think you know who I am talking about but there are others too).

#9. Meditate on eternity

Other people will read your posts in the future, because they will come up in search results. Obviously if these people are using the OTN Forum search engine they'll probably be after an answer to some completely different question but Google also indexes the OTN Forums. So try to make your answers suitable for a wider audience. Besides, as Jakob Nielsen observed in another context, your future boss may be reading.

#10. Keep your newbie mind

We were all newbies once. We are all still newbies in some dimension of the database, it's just too big for one mind to know everything. Even Tom Kyte occasionally farms out questions to Sean Dillon, Cameron O'Rourke et alia. So the next time you find yourself about to type a withering riposte to some "dumb" question just remember: one day, in some forum or other, you will ask a dumb question and an arrogant, big-brained geek is going to squash you like a fly.

---

^{1. Unfortunately this means people do assume their questions will be answered by Oracle employees and therefore there is some SLA we have to meet. In fact most OTN forums have little or no active Oracle participation (the HTMLDB, Developer and JDeveloper forums being honourable exceptions).}

2. Here is my favourite example (the words are approximate, this is from memory). The questioner ask a generically "stupid" question, "How can I improve the performance of my database?". A wag responded (using the UBB [code] markup so it looked authentic):

ALTER SYSTEM SET GO_FASTER='TRUE'
/

The original posted replied, "I tried this command but it failed due to illegal option". Which is funny but also rather cruel.

Another rainy day in this twenty-first century metropolis

Yesterday I decided to do a bit of work at home, as I wanted to do some data modelling (documentation, documenation, doncha just luv it?) and, for some tedious reason, I have Visio Professional on my home laptop but only Standard on my work machine.

First I decided it was finally time I activated my licence. Not having internet access at home meant having to do it over the phone. I can see now why the web method is recommended; typing in seven sets of six digits twice, once on the phone keypad the second on the computer with the added fun of pressing the phone hash key after every set is not a lot of fun. Particularly when you have a four-year-old boy registering their displeasure at you being on the phone by playing a toy synthesizer in the most unmusical fashion possible. Because I'm a masochist I registered my Office licence at the same time. Of course, MS don't seem to think people will register two products so I have to hang-up and redial.

Then I discovered that Visio doesn't support cut'n'paste of multiple lines into the Entity Definition column sheet (unless there's some trick I need to discover). After a while the repetition got to me so I decided I would be better off trying reverse engineering. Only I didn't have the database schema on my home laptop (having cleared 9i to install XE), so it's into the office after all.

On the way I decide to go to the parcel office to collect my latest consignment from Amazon. After all, I have Tooting Bec tube station at the end of my road and it's only one stop. And they are work-related books. Except that the Nortyhern Line is undergoing emergency engineering so there are no trains going south. Because it's be raining the traffic is blocked solid so I decide to walk instead of waiting for a bus. This is a sound decision: in the fifteen minutes it takes me to walk to the parcel office I overtake about half-a-dozen buses; not one overtakes me back. I casually note that most of the traffic consists of single people sitting in cars, leavened with the occasional van.

At this rate I'm starting to think that the post office will have lost my parcel but no, they have it and it contains the correct books. Hurrah! Now all I have to do is get into work, which is slow but at least I get there.

On the way home in the evening the service is suspended because of a security alert aty Balham. Naturally London's superb bus service is available to take up the slack. As if. Very long wait, very crowded buses, very slow journey.

When I get home I install the schemas into my XE database. At least that works. I fire up Visio, click into reverse engineering mode and it shows me all the objects in my XE database. But when I tried to actually import a table it fails with fatal error. No clues but I'm guessing some kind of driver issue; I'll give it another go this evening.

But wasn't the twenty-first century supposed to be different? Where is my helicopter for commuting to the office? Where is my holiday on the moon? Where is the technology that works properly, first time, all the time?

Oracle Express Edition: Good things come to those who wait

I decided to give XE one last try on Windows before turning to VM Ware. So I uninstalled XE using the MSI program. This did a stand-up job of tidying up everthing which was a relief as it's not an experience I've always had with de-installing Oracle before.

Then I set about tweaking my configuration. About the only problem on the known issues list that I didn't have was an account name with spaces. Here's what I did do:

(1) Control Panel > System > Advanced > Environment

• Remove ActiveState Perl from the PATH variable
  
• Point TEMP and TMP variables at C:\temp instead of the default
  
• Remove the ORACLE_HOME variable (not sure I have this, I think it might break an Ant script)

(2) Stop the Oracle 9i services
(3) Control Panel > Region and Language, change preference to English (US)
(4) Reboot

Then I re-ran the installer and it worked! I now have a nice 10gXE on my work machine. Just in time for the weekend, my wife will be pleased. And my 9i installation isn' t broken either so I'm a happy bunny once more.

Oracle Express Edition: Security Patching Policy

Earlier this month Pete Finnegan wondered whether we will get security patches for XE. Mark Townsend has now posted on the OTN XE Forum that Oracle intends to release fully patched versions of XE. Users will just install the new XE software over their existing install. This approach is deemed to be "easier than patching".

Given the target demographics of XE this is probably true but it will be interesting to see how this works in practice. This will presumably create additional overhead for ISVs who wish to customise the XE download (for instance by replacing the default seed DB).

Still, at least it looks as though we will avoid Pete's nightmare version of thousands on unpatched Oracle databases taking over the web.

Wikipedia: the dumbness of crowds

A posting on the XE forum observes that "By far the best known Wiki is Wikipedia". Almost certainly true but how very depressing.

The problem with the Wikipedia is that entries are of highly variable reliability and quantity. Due to the sort of people who contibute to Wikipedia, the entries on (say) Star Wars or the Klingon language are broader, deeper, more detailed and accurate than (say) the entries on relational database theory. In practice most of the entries on computing and related disciplines are reasonably reliable, because there are anough web users with relevant knowledge to correct obvious errors in their own fields (if they can be bothered).

Even so, unless you already know a fair bit about the topic it can be hard to determine whether the entry has been written by a leading expert or some passing nimrod. And when it gets to things like Latvian mythology, who knows? Is this entry on Tanis Diena, the sacred pig holiday a spoof? How would you find out, except by going to some authoritative (but less exciting) source such as the Encyclopedia Britannica in your local library? Even the Wikipedia founder says that too many entries in the Wikipedia "are nearly unreadable crap".

I am reminded of the chapter in "Surely you're joking, Mr Feynman" when Richard Feynman was reviewing physics textbooks for schools.

The man who replaced me on the commission said, "That book [that I thought was bad] was approved by sixty-five engineers at the Such-and-such Aircraft Company." I didn't doubt that the company had some pretty good engineers, but to take sixty-five engineers is to take a wide range of ability - and to necessarily include some pretty poor guys...It would have been far better for the company to decide who their better engineers were, and have them look at the book. I couldn't claim I was smarter than sixty-five other guys - but the average of sixty-five other guys, certainly!

Of course, there are some very good uses for the wiki technology. A prime example is the Extreme Programming Roadmap. This site is hosted by Ward Cunningham, who invented the Wiki concept as well as being one of the founders of XP along with Kent Beck. This wiki works because it is a site for sharing and exploring ideas. It is a conversation, an exchange of opinions, not a source of facts. It is precisely not an encyclopedia of Extreme Programming (eXPedia? or has somebody already got that?).

Wikipedia is predicated on the assumption that knowledge works like some kind of pachinko machine: the channel where the most balls go must be the truth. But actually all you end up with is a lot of balls.

Oracle Express Edition: Third Strike

Mike Townsend on the XE forum said that the ORA-12557 error indicated an Oracle Home variable pointing at the wrong set of DLLs. After some bootless tweaking of registry settings (actually wuite a lot of booting was involved, I think fruitless is the word I'm after) I found that I had an ORACLE_HOME in my environment variables.

So I set that to the XE directory. And lo! the ORA-12557 error went away and the database build progressed a bit further. Not far enough to actually build the XE database you understand, but at least it was connecting to the instance. Unfortunately changing that variable to point to XE home breaks my 9i installation very badly. As I need 9i for my work right now and I don't need XE I think that just about wraps up my XE adventures for the time being.

In an unrelated item I note that the Register has a report on a survey finding that software problems cause people to swear, drink and throw things. Well I never! The things these surveys reveal!

Oracle Express Edition: two strikes, the pitcher steps up to the plate...

The world is divided into two groups of people: those for whom installing Oracle Express edition has been a piece of cake and those for whom it's been a flipping nightmare. Unfortunately I find myself in the latter group.

The first problem was the install spinning when it was trying to create the XE services. CPU meter up to 100% for as long as you like. This turned out to be an artefact of having NLS_LANG set to UK English. All that was required was settting to American_America.WE8ISO8859P1. Ooops! We might have thought that offshoring so much development and support work would have alerted Oracle to globalisation but apparently not.

Having got past that the second problem rears its ugly head. I now have an instance but no data files. Each time the scripts tried to connect to the database the failed with ORA-12557: TNS:protocol adapter not loadable. This is a rara avis; the only note on Metalink relates to Grid controllers, which doesn't seem to fit the case here, and Google likewise draws a blank. Let's hope Mike Townsend comes up with something.

Before you Linuxen start smirking this is not particularly a Windows problem: I was able to install XE on my home laptop first time. My home machine is lower spec but same operating system so it's something about the specific configuration of my work machine that's giving me grief. I am only persevering with this because if I ever need XE, I will need it on my work machine.

Although I must admit I am starting to get very tired with the process: it requires several manual steps - editing the registry, renaming files, two reboots - to clear down XE prior to re-installing. However, I have just discovered that if I re-run the MSI against an untouched install it asks if I want to uninstall XE. I wish I known this earlier: the Installation guide talks about using Add/Remove Programs in the Control Panel but that option only appears once the installation has passed the Services point. Of course, the MSI uninstall option may also only be triggered if the prior install got to that point. I'm afraid I lack the strength to de- and re-install just to find out.

So what have I learnt so far? Not a lot. I've spent hours, literally hours, trying to install XE on my work machine with no success. I certainly haven't had time to build an app on my home machine. I do know two things. One is that the MSDE team is not yet quaking in their boots. The other is that I don't think this blog is likely to appear in the 10XE User Experiences any time soon.

UKOUG Annual Conference: A retrospective

Walking along the canals of Birmingham on Sunday I was struck by how not like San Francisco it was. Not just the colour of the sky but the whole attitude of the place. The canals of Birmingham have been reclaimed from their industrial past and re-branded as a tourist attraction. What this actually means is a lot of canal side bars with aspirational names like Panama, Ipanema and Santa Fe (not particularly famous for it's canals). One ristorante has an Venetian gondola moored outside. Meanwhile dead leaves float in the canal and people scurry past in their windcheaters and overcoats. Still, you can't get a decent pint of IPA in St Mark's Square so it cuts both ways.

Another difference between Open World and UKOUG, as Mark Rittman has also observed, is that us UKOUG committee members were there to do a job, so blogging in real time was difficult. Here are my personal highlights from the annual conference.

Best Presentation: Developing your career through system disasters by Martin Widlake

An irresistible title and a first class presentation. It wasn't just cynical laughing at the dumb things people do, it also gave tips on how to turn those dumb things to your professional advantage. Martin was full of useful insights, advice and aphorisms. "Disaster tolerant software isn't". Compressed time scales "move us into our stretch zones and develop our resilience skills". Interestingly enough the presentation turned, briefly, into a lecture on the value of certain RAD/Agile practices, specifically, developing systems in discrete chunks that take no longer than three months to deliver. As Martin said:

You don't understand your users. That's okay because they don't understand you either.

This is the situation he calls the Knowledge Curtain.

Most Brain Stretching Presentation: Null values: Nothing to worry about by Lex de Haan

It wasn't the SQL that fazed me it was the calculus. I'm a historian, get me out of here! Lex delivered a through exploration of how NULL works in SQL and the relationship between the empty set and NULL. The key fact is that a NULL in a arithmetic expression returns NULL whereas group functions ignore NULL. He gave us some good hints for writing queries to handle NULL without getting the wrong results. Definitely a presentation to download and work through the examples.

Most Interesting Factoid: How To Handle Missing Information Without Using Nulls by Hugh Darwen

I always thought that the reason the relational theory crowd didn't like NULL was because they objected to the absence of meaning. Hugh said that if SQL had implemented NULL=NULL that would have been okay. Well, okay-ish. The absence of meaning would still be a problem but it's the additional work necessary to handle NULL that really rankles (I supect Fabian Pascal may take a different stance). Anyway, Hugh started his presentation with the observation that everybody in the audience had a vested interested in the badness of SQL and ended it with an exhortation to us to pester Oracle for better SQL. Guilty as charged, but I'm afraid I'm more likely to pester Oracle for a more complete Type implementation than I am to ask for changes to the SQL standard. Although I do think being able to SELECT * EXCEPT comm FROM emp; would be nice to have.

Worst Start To A Presentation: Performance from a Different Perspective by Mogens Nørgaard

There's never a good time to hear the skirl of the bagpipes but 9.00am is really bad. (My father was Scottish so I'm allowed to say this.) At last year's conference Mogens presented without shoes. This year he presented without trousers¹. It might be a good idea to skip next year's presentation ;)

Most Depressing Fact: Does ADF live up to the hype? by Paul Jeynes

In 2000 when I was working on a project using BC4J with JDeveloper3.0 we had real problems getting any kind of assistance from Oracle Support or finding relevant How To documentation. Of course, in those days Java was still supported by the Server Tech guys because Support thought it was only used in the database. Five years on, ADF is here, Java is primarily used in the J2EE web environment and the quality of support does not seem to have improved much.

Most Intriguing Business Move: The Launch of Oracle Express Edition by Tom Kyte

As Tom observed at the start of his presentation this has already been widely blogged even before the launch but he still managed to generate a buzz. Obviously giving away a free database was (with hindsight) almost inevitable in the current database market. People do like free as a price. I think the interesting thing about Oracle XE is its potential as a MySQL killer. With Oracle XE you are going to get a reliable database with stored procedures, triggers and relational integrity. Furthermore, you've got an easy migration path if you exceed the (generous) limits or need certain enterprise features. Probably the only people who aren't going to choose XE are zealots in the Open Source and the Microsoft communities. Who'd have thought they'd end up on the same side? So, taken with the purchase of InnoDB, I think Oracle really has MySQL AB by the short and curlies.

The only thing Oracle can do to muck this up is not issue patches, at least for show stopping bugs and security holes. There's no point in encouraging thousands of new people to join the Oracle community if means exposing them to Oracle-focused worms. Whilst stealing Microsoft's shtick on helping developers and learners Oracle do not also want to give themselves Redmond's reputation on security issues.

Best Meal: The Oracle Blogger's Dinner by Mark Rittman

Actually this is not a difficult call as catering for over two thousand people over short periods of time rarely generates fine cuisine. Still, thanks to Mark for organising it and thanks to James Haslam of the UKOUG for sponsoring it. It was nice to be able to put faces to some of the blogs I read. I think next time we should wear badges with the name of our blogs. By the way, what is it with Chinese restaurants? The set menus are always ridiculously over-specified. The third course had far too many dishes. As the sole troublesome veggie I ended up with three dishes all to myself plus rice, on top of the previous two courses. Thank goodness I had been unable to fill myself up on prawn crackers beforehand.

---

^{1. To be fair Mogens did wear a kilt.}

Agile Databases and other paradoxes

So Niall Litchfield’s aside about Sandra Momali’s taste for Agile development has kicked off a bit of a debate. I thought I’d throw in my tuppenceworth, as I find XP a very seductive idea, even though I haven’t had much opportunity to use more than a couple of its techniques. I think it is easy for discussion about Agile development to descend into straightforward bashing of Java heads and developer-vs-DBA scrapping. Which can be fun for the participants but are not particularly enlightening for spectators. Besides there’s more to Agile than Java. In fact, Bruce Tate (author of Better, Faster, Lighter Java) seems ready to dump it in favour of Ruby because Java is insufficiently agile, EJB3.0 notwithstanding.

I hope we can all agree that the problems that Agile seeks to address are valid ones. Applications are all too frequently delivered which are bug ridden, late, not what the customer wanted, not what the customer now needs, and sometimes all of the above simultaneously. Let’s disregard considerations of whether there are other approaches that can also solve these problems and take the Agile approach as a given. The question that needs to be answered is, "Can Agile techniques sensibly and profitably be applied to database development?"¹

From the Agile side of the fence the answer is mostly a resounding raspberry. Agile practioners tend to regard regard databases as persistence repositories of last resort. They have a number of strategies to avoid writing SQL. Not just by deferral, using Mock Objects in the development stage, but by making deliberate architectural decisions to not use databases. These range from serialization to files through use of in-memory object stores like Prevayler to hands-off Object-Relational mapping tools like Hibernate. Last year I was talking to a UKOUG member who no longer attended the SIG I help run. He explained that his project was currently being migrated to a new version using Hibernate, Spring, Velocity, etc. I said, that sounds exciting, could someone do a presentation at the SIG? Back came the answer: No. Being Java developers they literally could not see the point in talking to a database user group. The whole point about their approach to application development was to avoid thinking about the database at all.

Some of this resistance may be down to sheer pigheadedness, but (at the risk of seeming to patronise Ron Jeffries) many of these people are actually very bright and aware people. They just don’t think databases can be Agile. Why is this? Let’s look at some of the Agile practices that are made harder by working with an RDBMS.

Incremental Design

Agile starts with rather sparsely-defined requirements and uses techniques such as Test Driven Design to flesh them out. This is not building a prototype, this is working, production quality code. By building real code the Agile practioner can show his user a working application and get meaningful feedback that allows him to refine the requirements. Hence together they build the system that the user actually wants. The anti-pattern for this is Big Design Up Front. Here we spend ages drawing a mesh of boxes called an Entity Relationship Diagram. We show it to a bemused user who nods guardedly. We turn it into a data model but the user still isn’t as excited as we are. Finally the database gets built and then the front end gets built on it and whereupon the user wails, "But that’s not what I meant at all!" The question is, does the fault lie in the database or the process? Are databases inherently BDUF?

Obviously, we can just build one table at a time. But is that enough? Probably not. We still need to normalise our database². Consequently, any attempt to build our database incrementally means constantly refactoring of our existing tables, to eliminate duplication. Can we do this? Yes we can! Of course it requires that we hide all our tables from public gaze behind an API - PL/SQL packages, views or types, which some people will just hate. The larger question is Tom Kyte's point about the importance of designing for performance right from the start. Does that imply BDUF? Yes, at least some. The other problem is that refactoring an object is a lot simpler than splitting one table filled with twenty million rows into two tables. Our database design can be reasonably plastic in development but once we go into production inertia kicks in. I think here is the killer for Agile practices: database refactoring is hard. Scott Ambler seems to be ploughing a lonely furrow in this area but his thoughts on The Process of Database Refactoring make interesting reading.

User stories

Agile practices tend to re-inforce each other and that’s the case with User Stories and Incremental Design. User stories describe how the customer’s business works. These stories will not mention data storage, except by implication. Customers don’t specify a database, they specify an application, that is, a front end. Customers cannot accept or validate databases. Even if we gave them TOAD and they wrote SQL queries that would not prove the database met any of their requirements, because their requirements are specified in terms of the front end. And without continuous user feedback we cannot be sure that we are coding in the right direction.

Kent Graziano has been working on building a datawarehouse using Agile techniques. He has presented on Agile Methods and DataWarehousing several times this year. When I saw him talk at Open World he said he had got some grief from Agilistas because the ETL section of datawarehouses doesn’t have users. So there cannot be any user stories. But as Kent points out, just because the business user (the customer) doesn’t know or care about the ETL process doesn’t mean that ETL has no users. On Kent's project they put the BI report writer in the user role. Which is a nimble, indeed agile, extension of the practice which doesn’t, as far as I can see, break any tenet.

Common Ownership of Code

When programming an agile practioner drives a path through the code base. If she finds an existing part of the application that blocks her implementation she can change, extend or fix that code. She can do this safely because the existing code has unit tests and the existing application has integration tests. More importantly, she can do this new work because the existing code base is in Java and she knows Java. But if this piece in the code base was in PL/SQL she would be lost: she doesn’t know PL/SQL. Even worse, suppose her change requires a new table? Then she would have to go to the DBA to find out about schemas, tablespaces, etc. This is too slow for the Agile way of working.

Teams of specialists are not Agile. We know this is true. How often have we been trying to progress a project only to be told, “We can’t do that, Gary’s on leave today.” To be fair this does not just apply to databases. I think Agile practices militates against heterogeneous programming environments and would be equally suspicious of (say) extensive use of shell programming. So, unless pretty much the whole team is competent in PL/SQL, using stored procedures just does not fit in an Agile project.

What this boils down to is the matter of where we should put the business logic. Unfortunately this frequently descends into a religious war. Either you believe it’s obvious that business rules belong in the database or it’s obvious to you that they belong in the middle tier. So let’s put it another way: in a client/server application where does the business logic belong? I think a large part of the middle tier proponents would then grudgingly accept the merits of the database as the repository of business logic. In most sites today we probably have heterogenous applications written in a variety of styles, techniques and languages, including many client/server applications. Our best hope of consolidating business rules is to use the database. But, this is an emotional debate and not one I expect either side to cede.

Object Oriented Programming

The alert amongst you will have noticed how often the term object appears in that list of persistence techniques: this is key. Agile programming languages are object-oriented languages. I was recently embroiled in a debate on the Test Driven Design list when I was told this surprising fact. Surprisng because I thought for several years I had been doing Test Driven Design by using Steven Feuerstein’s utPLSQL to build database applications. Apparently there’s more to Test First than merely writing our tests first.

Obviously an RDBMS is not object-oriented. A non-OO language like PL/SQL lacks certain key features supposedly necessary to the full panoply of Agile practices - abstraction, interfaces, reflection. Does this matter? I remain unconvinced. XP originated in the Smalltalk community and pretty much everybody you come across on Agile/Test Driven arenas seems to be an OO programmer. However, this is a self-fulfilling prophecy. Come out as a relational database developer and you get labelled as a handyman with only a hammer in your toolbox.

Use of Tools

Agile practioners make full use of utilities to improve their productivity - smart IDEs, code completion, continuous integration, automated unit testing, test coverage estimnators, refactoring browsers, etc. Compared to this panoply of riches PL/SQL looks very Spartan. There is the aforementioned utPLSQL but not much else. Of course, this is a vicious circle. PL/SQL lacks the tools to support Agile practices so people who want to be Agile don’t use PL/SQL so nobody builds the tools for PL/SQL.

Database development tools tend to focus on browsing the data dictionary and running SQL statements. There is very little around that genuinely supports the developer in building better PL/SQL or helps the DBA build better databases. This is a situation that is only going to get worse, as Oracle Designer slowly fades into the sunset. Paul Dorsey’s BRIM project (AKA The Tool) may be one to watch. Interestingly there is very little open source stuff for PL/SQL developers, compared to the wealth of freely downloadable utilities for Java heads and Pythonistas. I think this illustrates the lack of people who are competent in both an application (i.e. a UI building) language and PL/SQL.

Still, the Agile Manifesto says we have come to value individuals and interactions over processes and tools, so that’s all right then.

Instaneous Feedback

There is an old rule of thumb from the HCI arena that states that a user will regard as “too slow” any task that takes longer than ten seconds; if it takes longer than a minute they will start on another task and come back to the first task later. In Test Driven circles, running a suite of unit tests is such a task. The JUnit list regularly features complaints about how long it takes to run unit tests that require population of a database. This interferes with flow. Hence the popularity of Mock Objects. Hence the drive to isolate the database elements to as small a subset of the application as possible.

I think the database has to cop to this one. If your sole experience of database population is writing tedious XML files for use in DbUnit then you will naturally have a jaundiced view of RDBMS systems. But even people who know how to write SQL must admit that deleting and inserting records takes longer than instantiating some objects (once the JVM has been warmed up!). That’s the price we happily pay for rigour.

Pair Programming

Well, database people are notoriously misanthropic. Especially DBAs. We can still talk to each other.

Conclusion

The Agile argument is databases are slow to build and inherently hard to understand. This is true. But it’s true because Agile is primarily a methodology targeted at building user-facing applications. Its adherents tend to work from the user interface downwards. Of course it’s crucial that we build an application that meets the user’s needs. But the user’s needs extend beyond the functional requirements of the application screens.

That doesn’t mean we cannot apply Agile techniques to database development projects. Certainly I regard Test First to be an invaluable technique in building stored procedures; I’m always nevervous when I’m not coding to a utPLSQL test case. My experience of working on DSDM projects tells me that incremental development - in particular placing working code in front of users early and often - is a very good idea. Kent’s work proves that Agile techniques can be profitably applied even to a juggernaut like a datawarehouse. The pain comes when OO-fixated bean-mongers clash with constraint obsessed databasers. Perhaps if they drank less coffee they wouldn’t be so hyped up all the time.

---

<sup>1. By database development I mean building data structures (i.e. tables) as well as programs (PL/SQL).

2. I learnt yesterday that Codd’s Twelve Rules have been deprecated because "Codd formulated them as a quick and dirty way to counter all the nonsense that was floating at that early time, they are not orthogonal or systematic, and our understanding of RM has progressed considerably since then."</sup>

ToDo Driven Development

Lucas Jellema has just posted a technique for incorporating TODO tags in PL/SQL codes. This was a spooky piece of synchronicity because I had just stumbled across a "sexy new development methodology" called ToDo Driven Development on the SecretGeek blog. Leon Bambrick's implementation is for .Net and I was going to port it PL/SQL but Lucas has saved me the effort. Mind you, ToDo Driven Development has an extension which tracks HACKs as well as TODOs and that's obviously worth having. So I may enhance Lucas's code this evening.

Blogging about a blog about blogging

Jakob "Usability" Nielsen has just published an Alertbox on blogging. It's depressing to discover that this blog fails at least five of his strictures: No Author Photo, The Calendar is the Only Navigation, Irregular Publishing Frequency, Mixing Topics, Having a Domain Name Owned by a Weblog Service. The jury is still out on Forgetting That You Write for Your Future Boss and Nondescript Posting Titles. I'm not sure whether Classic Hits are Buried applies, as I don't know if I have any classic hits; but if I have 'em, they're buried. Sob.

Some of these are easier to fix than others. But I guess I really Must Try Harder.

Politics: Life During Wartime

Tony Blair describes the arguments in favour of his new anti-terrorist legislation as, "compelling". This is because if you disagree you will be arrested and put into a cell until you realise your fears about human rights abuses are groundless. Which brings me to David Mery. I gather this has been slashdotted but I only just heard about it.

Mery is a computer geek who has an unhealthy obsession with Bill Gates's Basic compiler for the Altair. On 28th July this year he was arrested for being in possession of a French accent and a thick jacket in a London tube station. In 1979 who would have thought that, twenty-five years on, policing practice would be based on a Not The Nine O'Clock News sketch?

Eventually Mery was released and his case has been marked No Further Action. But three months on he has still not had his confiscated computers returned. And, of course, being arrested for suspected terrorist activity will pose Mery an interesting hurdle at US Immigration and Border Control, if he ever decides to try to visit the States. Still I suppose he should count himself lucky not to have been shot eight times in the back of the head.

As someone who lives in south London and who uses the tube almost every day of his life, of course I back the police in the struggle against the Jihadists. But I too carry my laptop in a backpack. Does that mean I could be arrested because I meet some apparently arbitrary criteria? Ultimately, the Jihadists are trying to destroy a free society and replace it with one in which the individual's thoughts and actions are constrained by the pronouncements of official interpreters of divinely-given law. But, if we do this to ourselves, it won't matter that the name of the God is different, the Jihadists will still have won.

Prioritisation

In his latest article Joel Spolsky describes a workshop to prioritise features for the next version of his FogBugz software package. Like a lot of Joel's approaches it's both radical and plain commonsense. Incidentally it's strange that in our high tech industry all the best ideas seem to hinge on creative use of index cards.

Oracle: a suitable career for a young person?

A while back someone posted a pertinent question on the OTN DB forum, Is it Worth to be DBA?. The poster was in their last year of college, studying computing. They were wanted advice on the merits of various different technical jobs: sysadmin, networks, DBA. Alas they never replied my response (how did they want to measure worth: salary, intrinsic job interest, long term career prospects, fringe benefits?) and the thread withered.

Still, the question remains: how should we advise a young person wondering whether to become an Oracle DBA? Obviously being an Oracle practioner has done well by me and many of the readers of this blog. It's been my prime focus of work for thirteen years, and there's still some juice in it yet. For most of that time I have been a developer, designer and latterly architect but my time spent doing DBA work has undoubtedly informed my knowledge of Oracle. It is good for everybody who builds Oracle systems to understand the database from the perspective of a DBA, because in production the application is only going to run as well as the database permits.

Whilst I was pondering this question someone else pops up, asking how they get job as a DBA. I do find this desire to become a DBA slightly puzzling. The work, particularly in production, is generally dull except when it gets exciting, and that's when you remember that "May you live in interesting times!" is a Chinese curse. Being a DBA requires you to know the purpose of 257 parameters, 266 V$ views, 415 DBA% views, let's not forget the the X$ views... There's dry stuff like sizing redo logs, taking backups, applying patches, running installation scripts written by developers who haven't tested them, etc, etc. What precisely is the allure?

Partly it's the mystique of being in the know. The DBA is the gatekeeper, the person with the authority to run your code in production, the one who can tell you why your code is running like a three-legged dog in a vat of cold treacle, the person who laughs when you ask them to make the USERS tablespace bigger because you keep getting this ORA-1555 error. Knowing the SYSDBA password is the database equivalent of the sysadmin's got root? T-shirt. In fact, the DBAs and the sysadmins tend to inhabit the same room (and indeed are quite often the same people) but it is easier for a SQL developer to move into being a DBA than it is to become a *nix administrator.

Also, being a DBA is a glamorous job. It's not really of course, but most of the luminaries of the Oracle technical world are (or seem capable of being) DBAs: Tom Kyte, Jonathon Lewis, Connor MacDonald, Cary Milsap, Mogens Noegaard, the list goes on. But how many App Server gurus can you name? If you're developer, who is there to look up to? Steven Feuerstein and, er, that's it. (I know if you're into Java you could cite Steve Muench and Duncan Mills, but Java with Oracle is a distraction.)

So everybody wants to be a DBA because it's all about the data(base). Does that mean a person leaving university should try to become an Oracle DBA? Almost certainly not. There's a reeason why most DBAs are gnarled, bearded men: we need experience as well as knowledge before we are ready to become a DBA. It's just not a role for a neophyte. Besides, with the increasing commodisation of databases the role of the DBA is going to decrease in importance and sustained interest for the majority of people. Not to mention considerations of whether Oracle is a legacy technology.

If long term career prospects are the main concern I think a fresh graduate with a technical bent should get into networks: database, platforms, operating systems will come and go but everything about computing in the foreseeable future will still require networks. And with the rise of wireless the amount of time spent crawling under desks should be greatly reduced. If salary is their main driver then a graduate should seek to get into management, which mainly requires shedding all their technical knowledge as soon as possible.

However, we all spend the majority of the day, most days of our life at work so it is important to do something that continually engages us. There's no doubt that most Oracle roles are capable of offering stimulating work alongside the humdrum. How much time we spend doing the latter over the former is a measure of our talent. Good people understand the need to do the tedious stuff in order to get the interesting work right, so do it well and quickly; the very best know how to spin the dross into gold.

But for a job that's interesting day after day, that's full of surprise, delight and moments of wonder, that's always worthwhile despite all the poop it entails? Well, there's only parenthood. Nothing else comes close. Still, maybe not the thing to do right after graduation (unless gowns and mortarboards make you hot).

MS Office: Get your re-org boots!

Jakob Neilsen's latest Alertbox draws our attention to the iminent redesign of the MS Office user interface. Not the least of the woes ahead will be the learning of a new acronym: WYGIWYS, What You Get Is What You See. Practice it now, it's a tough one to get right.

Apparently Office 2003 is so full of functionality that many users cannot locate many of the features. Who'd have thunk it? Anyway, to solve this problem Microsoft are rejigging the various menus, toolbars and dropdowns in favour of coherently organised Command Tabs. This sounds mildly useful. More interesting is the WYGIWYS bit, whereby we format the document by choosing our desired result from a gallery of examples. As somebody is frequently driven to apoplectic fury by my inability to switch off some particular embedded Word style object I think any improvement to Word format styles is to be welcomed. Of course want I really want is a codes pane like WordPerfect had but I can't see that coming soon.

What does it all mean? Well there's some learning coming up for those of us who are bound to the corporately-supplied toolset. There's possibly some big yokks for the Open Office and Mac sparts. But just suppose Microsoft pulls it off. A new UI not swiped off Apple/Xerox that delivers real productivity gains to ordinary users: wouldn't that be cool?

Oracle Metalink: Don't shoot the pianist

Laurent Schneider recently posted on his bad Metalink experiences. Let me show you my scars...

My first stand-out bad experience was an iTAR on generating checkboxes in JavaServerPages back in the days of JDeveloper 3.0. After two weeks of hassling I got told there was a palette with a Checkbox icon in the JDeveloper IDE. This much I knew already: it was for Swing programs. I tried to explain about the difference between Java for client/server and web but the iTAR got closed anyway. This was when the Java language support was still done by the database group, who really didn't understand building client-facing applications at all. One of our Java brains figured it out eventually.

The second bad experience was a year or so later. We were having some difficulties with BC4J which dragged on without resolution. The support guy, exasperated, asked why we were using BC4J as "Nobody was using, even in Oracle". I posted that to an ODTUG list, to stir up a response. Oh boy. Steve Muench fired an enraged e-mail back to ODTUG. I had the Support manager on the phone. I expect the poor second-line drone caught it in the shorts (which I do feel bad about. Sorry, whoever you were). Did we get our problem resolved? Again, eventually. Again, this was Java. Oracle really took a long time to sort out its support for Java. But that's how I discovered the OTN forums, so it wasn't all terrible.

The third bad experience, was some kind of database thing, I forget what. The first-liner resolved it to his satisfaction and closed the iTAR. I called back to explain that he had mis-read what I had typed in the iTAR and that his "solution" solved a completely different problem. He said, 'You'll have to open another TAR.' Now that's a lot of typing to do all over again. It took seven words, 'I want to speak to your manager', to get the TAR re-opened, and resolved pretty quickly thereafter.

I've been using Metalink and Support for over ten years now and I think three memorably bad experiences in that time is pretty good going. Most iTARs get solved, and get solved within a reasonable time. Of course, not working in production means that I can take a more relaxed attitude towards rsolution time.

The real problem with Metalink is the interface. When I first started using Support it was all done with phone calls. Which was great: I almost always got someone who was knowledgeable, so most TARs were actually resolved on the phone or else they turned out to be bugs, (As I was using Designer 1.2 on Windows for Workgroups 3.1.1 to generate Forms 4.5 there were a lot of bugs). What that meant was I tended to phone up Supoort to get answers to questions of the "How do I do ...?" variety. Now that iTARs have to be raised through a web interface everything takes much longer. There just isn't the immediacy of a phone call. In fact the iTAR process is so slow it's generally a lot quicker to use the Metalink search engine or Google. As a consequence I never contact support with a "How do I do ... ?" question. They're smart, these Oracle guys.

Of course, the Metalink repository is not without it's problems. Why is it so hard to filter out anything to do with RdB or Apps of any flavour? Why can't I explicitly search for (or exclude) stuff by document type (release installation note, error report, How To bulletin)? Why is the weighting of documents so poor? It used to be that the first result when search for an error number was the OERR document. Now it's quite frequently number 79/100. The answer is quite simple: poor metadata. The Metalink repository needs much better tagging. Of course there must be so many documents in the database you can see why nobody want's to undertake the task.

But then, but then. Only last week I started to raise a TAR because I had failed to find anything helpful in the repository. having typed it all in the TAR workflow suggested six documents that might solve my issue, and the second one did. Result! So the information is there, it's just sometimes so hard to find it.

It's easy to knock Metalink, but who would really want to be without it?

OOW2K5: The wrap up

It's oh so nice to go travellin'
But it's so much nicer, yes it's so much nicer to come home

Sinatra sang that. Going to OpenWorld this year was great but it meant I missed my son starting reception class (kindergarten). Fortunately I did get back in time to collect Fred at the end of his first full day: his smile when he saw me is obviously the thing that I will treasure most from the week.

Anyway, Oracle Open World 2005. Now that I am back at work what have I been telling people about the trip?

Magnitude

Thirty-five thousand attendees. That's a small town. It's partly a sign of Oracle's success as a product but also an artifact of the major acquisitions over the last twelve months. There were large contingents of JD Edwards and PeopleSoft customers attending OOW for the first time, obviously trying to get a handle on the nature of this Oracle beast. It will be interesting to see if they attend OOW2K6 in such large numbers. At times the sheer number of attendees was imposing. Yet the conference organisation coped. There was enough food, water, coffee for everybody. Most of the time the queues were not excessive (except at Starbucks in the morning) and usually people got into to their session of first choice. The conference scaled well. It must run on Grid. Speaking of which...

Oracle's Commitment to the Grid Architecture

Probably the single most impressive fact I learned this year was the nature of Oracle's own enterprise set-up: a multi-node cluster of 288 CPUs with a BI layer running against the OLTP layer on the same instance. (By contrast, when 10g was launched at OpenWorld two years ago the reference site for the grid architecture was EA's The Sims Online with eighteen CPUs.) But it's more than just resilience. Grid architecture is becoming a key enabling technology for Oracle. Benefits of implementing the grid cropped up in sessions I attended on security, data warehouse architecture and efficient data loading. Oracle have not quite cracked the manageability angle, there is still an administrative overhead to running a grid. But once it becomes possible for us to treat a grid as if it were a single box then any barrier to take-up should disappear.

Oracle's Commitment to the Open Standards

Project Fusion was the key message of the conference: turning the entire technology stack, from database to portal, into a single suite of integrated, hot pluggable components. The integrated bit is obvious. Of course all the products from a single vendor ought to play nice with each other. The interesting bit is their commitment to open standards, such that we can swap any part of the suite for a similarly compliant product from A N Other vendor. So Oracle is going to certify its E-Business Suite against JBoss and WebSphere. They are even contemplating certification against IBM's DB2 database, although this will depend largely on the attitude of the PeopleSoft and JD Edwards customers towards database flavour. Also openness requires there to be other vendors with standards compliant software, otherwise it's just another form of vendor lock-in.

Utility Computing Revisited

Oracle have gone a bit quiet on this front but Sun seems to be going for it in a big way. It's not going to be long before get a site to host your business app will be a simple as filling in a form of six checkboxes and uploading an RPM.

Data Vault

Currently this product is in Beta but it was the most interesting announcement of the week. Essentially it answers the question, "How can I stop my DBA getting access to my customers' credit card numbers?" With Data Vault we will no longer hust have to trust the integrity of our DBAs. It is a set of packages and practices that separates user management from security policy from system administration. So for instance we will be able to enforce different levels of access in local and remote databases for the same user account created with the same system privileges. We could prevent the DBA from querying any table with sensitive business data despite her possession of the SELECT ANY TABLE privilege. We will be able to prevent anybody issuing DDL commands during business hours. Data Vault will feature a robust audit function impervious to tampering by even a DBA. Of course, it will not be bulletproof, but working around Data Vault should require collusion between at least two and probably three people, which is (we hope) much less likely than a rogue individual.

Steven Feuerstein

Can you believe it's ten years since O'Reilly first published PL/SQL Programming? Me neither. Now that the book weighs in at 1198 pages maybe it's time to same goodbye to the ants on the cover in favour of something of a more suitable scale, such as a (stunned) water buffalo.

Participation

It was nice to meet up with some other ACEs, to put faces and accents (hi Laurent!) to the names. I have a pretty good Namedrop Quotient at the moment.

Oracle seem to have improved the community side of things since two years ago. I attended a couple of get-togethers with a good mix of Oracle and non-Oracle people, and I am sure there were other similar events to which I didn't get invited. The hot technologies in the modern world are those were people can contibute rather than be passive consumers. People feel differently about Java or Python than they do about VB. Participation is the key to a thriving community. By opening up OTN to non-Oracle writers and acknowledging the contribution of non-employees Oracle is moving in the right direction. Although I think it'll be a long time before we're downloading the new version of PL/SQL from SourceForge. Which brings me to...

Badge of the Conference

A freebie from Apress with the slogan, "Life would be so much easier if we just had the source code." Which is very true when you think about it.

OOW2K5: Running out of steam

Boy these are long days. I seem to be getting more jetlagged not less.

Anyway, did you get to see Larry? At 1:00pm the queue literally stretched round the block. (I am not obsessed with queues, honest.) Anyway I decided not to join the end of the queue and get sunstroke so I headed up to floor 3 in Moscone West to get backache instead. I do think it was the right decision: there was no durm und strang before Larry came on stage and to be frank remarkably little once he did appear. Of course it didn't help that poor TV reception made him sound like Ken Godwin out of the Wheeltappers And Shunters' Social Club. He seemed mortal rather than a colossus bestriding the database world. Flourishing his glasses was a nice touch.

There were no fresh announcements. Pretty much everything Ellison said had already been announced, mainly by Charles Philips on Monday. There was only one joke, a dig at Microsoft. The most interesting thing is how long the Q&A section went on for, overrunning by so much that the sessions after the scheduled break were delayed. Larry obviously relaxed once he'd got all the messages out of the way and could just be spontaneous. Unfortunately I missed most of the questions as my back was giving me gyp. But apparently the ridicule in the media about the multi-core licencing model has struck home, and a new licencing approach is on its way. That is, yet another licencing model is on its way. Is YALM a word? It ought to be.

That's it for me for 2005. This particular thirsty bear is off to Steven Feuerstein's party. Cheers.

OOW2K5: Breakfast In America

At the bloggers dinner last night Mark Rittman was commenting on the breakfasts provided in the conference centre. They are called "continental breakfasts" but really they are just cakes and coffee. Mark was saying that the resultant sugar rush makes a good hangover cure (being an abstemious sort of fellow I couldn't possibly comment) but, if supplemented by Coca Cola and Skittles, it also enlives even the dullest of presentations. Well, the organisers must have known the Oracle Bloggers would be in need of a tonic this morning because they've laid on a spread of Krispy Kreme doughnuts, that deep-fried, sugar-coated heart attack in toroidal form. Yummy. Can't stop, must dash!

OOW2K5: Queueing Theory

The large number of attendees is really starting to bite. Queueing, not Total Integration or Project Fusion, is the dominant theme of the conference. Whenever two or three delegates gather together they will discuss the queue for a particular session or lunch or the toilet. The queue to get into Tom Kyte's presentation yesterday morning was so long that even Mark Rittman, ACE of the Year and Oracle Goldster, was unable to blag his way in. The English have a reputation for queueing, but these days it's undeserved. The queues here are really orderly, self-policed single files. The normal English bus stop "queue" is just a bundle waiting for the arrival of a bus to trigger it.

There have been rumours that the Conference might move to somewhere bigger, like Las Vegas, but this seems unlikely to me. Moving events of this magnitude makes turning an oil tanker look like spinning on a penny. In 1995 I got into conversation with a couple of events organisers in a Covent Garden bar. Naturally the talk turned to our plans for the millenium. That was the near future to them: they already knew what they would be doing in 2005. Living ten years in the future must be a very odd way to be, a bit like Philip K Dick's "The World That Jones Made". Another idea that has been mooted by a couple of people now is that the conference could be split into or even three separate events. But it seems to me that if the key message is Total Integration of the entire stack from storage technonolgy through applications to the portal then dis-integrating the conference is sending the wrong message. But let's face it, queues are a nice problem to have. They are a sign of success. It would be a lot worse if the conference was just lonely tumbleweeds blowing the Moscone Centre.

The queue for Scott McNealy's keynote started more than forty-five minutes before his keynote was scheduled to start; the keynote, and that was only supposed to last forty-five minutes. Lucky I brought a good book.

So was Scott worth the rock star treatment? Hmmm, not really. As keynotes by CEOs go he is entertaining. But he telegraphs his punches: if the Powerpoint throws up a picture of Larry you know a joke about expensive suits is coming. And the announcements are a bit underwhelming too. Scott talked about, "Let's have an iPod moment!" But Sunfire X64 4+ is not an iPod moment. If hardware floats your boat then okay. But Sun launching a new server is hardly as radical a change of direction as Apple moving into consumer electronics. Even if that server can run Windows. Still, because of the commitment to "planet-sensitive computing" I think we can say it is a Mini iPod in pastel green moment ;)

The most interesting part of Scott's keynote was the emphasis on zero cost of exit. Give away your software to the Open Source community, keep to open standards and compete on your ability to support your own servers. This is an interesting business model, the apparently exact opposite of vendor lock-in. Of course, being able to leave Sun easily is only half the problem for a CIO. If the competitors are all have lots of proprietary, closed products then where else do you go to avoid being locked-in? Smart.

I went to Dr Paul Dorsey's presentation on "Data Modelling with UML" genuinely expecting a How-To. Silly of me, I really ought to read the abstracts. What we got was a broadside against Oracle for abandoning Designer in favour of JDeveloper. Dr Dorsey is smart, articulate and boy is he ticked off. Paul is the first to state that JDeveloper is a first-class tool for developers. His name remains on the JDeveloper 10g Handbook after all. It's just not a tool for database architects. His frustration is evident and understandable. Apart from anything else, the market for JDeveloper is a small one. The tool is competing against Eclipse. A whole bunch of Java coders will never look at JDeveloper simply because it has the Oracle brand. A whole bunch more won't look at it because they think RDBMS is a dumb idea. And JDeveloper panders to this mindset by allowing Java developers to treat tables as just somewhere to persist objects. Paul was fair, he blasted Rational Rose for exactly the same things.

The key thing is that the same class model must generate the Java objects, the relational tables and the mapping between them. Just as Dr Dorsey started to talk about the only tool around that can do this, the BRIM product he has been refining for years, the battery in his laptop went. Was this sabotage by a black bag special ops team from the JDeveloper group? Who can say? I'm sure Steve Muench has an alibi.

In the evening I went to the Oracle Bloggers dinner organised by Mark Rittman. I'm sure everybody else will blog about it so I won't bother to say much. I was a bit fazed by the barman asking which vodka I wanted in my vodka martini. Er, you know, vodka. Tom queried my choice of a vodka martini, gesturing with his own glass and saying, "It's all about the gin". To which I should have replied (but didn't), "No Tom, it's all about the data".

By the way, am I the only person wondering why the Women In Technology International stand is being manned by a man?

OOW2K5: The numbers game

In his keynote yesterday Charles Philips said there were 35 000 attendees at the conference this year and it certainly feels like that. Particularly at the crossroads when we're interchanging between Moscone West and Moscone South; it's not quite Shibuya but it's not far off.

Yesterday I finally got to see Steven Feuerstein (Whom God Preserve) in action. It's 10 years since O'Reilly published the first edition of PL/SQL Programming, which made me feel very old as I bought that version. In fact I was the first kid on my block to have it. Afterwards I decided the time had come to upgrade at the Conference bookshop; I indulged the fan in me and asked Steven to autograph it. Today I court hubris by appearing at the OTN "Meet the Experts" session with Steven and Bryn Llewellyn (PL/SQL product manager). I think we ought to call it, You don't have to be bald to code PL/SQL but it helps. If you're reading this before 10:30am Tuesday 20, come along to the OTN Lounge in Moscone West.

I went to see Kent Graziano's presentation because the title seemed like a Zen paradox: Agile Methods and Data Warehousing. The Agile crowd (Scott Ambler notwithstanding) seem to have a strong antipathy towards databases of any sort so it's not surprising that Kent has had some stick from them over his interpretation of Agile practices. However I think his position is correct. As I have said elsewhere a lot of XP methods are very sensible and can deliver results. So what is wrong with refactoring the definitions to fit different sorts of projects. Agile practioners stress the importance of customer input but they only regard business people as customers. But what is wrong with saying that a BI Report programmer is a customer for an ETL process? Anyway, Kent's presentation has certainly given me some hope that it is possible to apply Agile methods even to apparently glacier-like projects such as datawarehouses.

Juan Loaiza's talk on the future of IT and databases turned out to be a discussion of Oracle's strategy for the next few years. The key messages are Total Integration, Agile Resilience and Unlimited, Unbreakable Platform. Not surprisingly the key technology is the Grid. Oracle have really pressed ahead with this: having grid-enabled the app server and the database they are now working on extending the grid to the storage layer. Fortunately for the SAN manufacturers none of the other database vendors seem to be following Oracle down the database grid road. Oracle's own E-Business suite is run on a single multi-node cluster with 288 CPUs. Of course they need all that crunch because they're running their BI and analytical programs in the same database as the OLTP applications. Now that's eating your own dogfood.

In the evening I tried to enhance my geek factor by attending the Linux Installfest. However, I'm afraid I bailed out when the kernel guys started and went in search of wine. Sorry, Todd. I resisted Laurent Schneider's urging to take the DBA Hot Seat test. This turned out to be a wise decision. I haven't really been a DBA since 8i and as most of the questions seemed to be about 10g features (especially Flashback and ASM) I would have ended up with only a promo T-Shirt, of which I have enough already.

Mind you, walking back to the hotel I could have done with a slogan T-Shirt. The sidewalks were pretty empty apart from the people sleeping on them. The slogan? "I'm not a criminal, I'm walking because I'm English".

OOW2K5: "Information Matters"

The warmup to Charles Philip's entrance was some lovely Flash animation with a modish soundtrack of (primarily British) pop rock. U2's "Beautiful Day" consolidated its position as the acme of corporate cool. First New Labour, now Oracle. Tranc techno remains the manadatory choice for soundtracking the videos.

The animation included some interesting phrases - "Information Matters", "Adaptable Process", "Information Driven". My personal favourite is "Execute With Insight"; h I can this see replacing "Terminate with extreme prejudice" amongst a certain sort of geek. Well, me at least. They also included a lovely quote from Ihop Corp endorsing the Data Hub: "we know how to sell more pancakes to our customers", which I think is a message we can all get behind.

Charles Philip's keynote was focused on Project Fusion and Oracle Fusion Architecture. It will be interesting to see whether this amounts to more than a marketing exercise in rebranding but it definitely sounds like Oracle is committed to building a complete technology stack that is both a fully integrated set of modules and a fully open framework in which any individual component can be substituted for another vendor's product. This will be an impressive one if they manage to pull it off.

I'm particularly intrigued to know how Oracle intends to implement all its apps as web services without the XML wrappers killing the performance of the suites. I know Microsoft's approach will be to use proprietary compilation of SOA calls to get the necessary performance. It can do so because it owns the OS and there are enough wholly Microsoft shops for this to be a viable strategy. Oracle can't do the same because they need to remain cross-platform, they don't own any OS and they have this tricky commitment to open standards to live up to. In the meantime, be the first on your block to describe yourself as "Hot-Pluggable!"

OOW2K5: Having an ACE time!

It's good to be back in San Francisco attending Oracle Open World 2005, as an ACE. Last time was in 2003 in the days when the ACE program was the OTN Community Award.

This time I'm staying in the Ramada Downtown, a motel so obscure that the cab driver had never heard of it. On the Ramada website it is described as "quaint". Finally I know what Americans mean by the word "quaint": it means just like in the movies. Americans call a sleepy English village quaint because it looks like something out of a Harry Potter film. My hotel is quaint because it looks like the set of an Elmore Leonard adaptation.

It doesn't have room service, just a leaflet for the local pizza delivery place. This is called Extreme Pizza, which is probably the only XP I think we can all agree on. Actually the Ramada is not at all bad. It's just it's so far out the lamp-posts have DB2 banners. Still on a clear day you can see the Marriott Intercontinental (where I got to stay in 2003).

I spent Sunday not being as jet-lagged as I was expecting. In the evening there was a dinner for the ACEs and the Oracle Magazine Editor's Choice Award winners. I got invited twice, as an ACE and a member of the magazine Editorial Board, but only had the one dinner. I did ask Kent Graziano if he was John Spencer, which was (I think) my only bloomer of the evening. Tom Kyte was on my table; at one point he used the word fricking, and he meant it. Tom and Ken Jacobs had a classic contest, matching error messages to the numbers. Tom on the role of the database in modern computing: applications come and go, only the data remains". I also learnt that Ken Jacobs is called Dr DBA after a presentation he did which was based on a television show; it turnes out to be Frasier and not Dr. Ruth (which was my guess).

I also met Mark Rittman, who is the ACE of the Year. He was remarkably coherent for a bad who spent the aftewrnoon in the pub. When I last met him it was in Birmingham, England; he had only just won the OTN Community Award and admired my OTN Expert fleece. I think he's been lobbying OTN to get to SF ever since.

BCS SPA Meeting

I read Kent Beck's Extreme Programming Explained in 2000, and it really fired me up. Project manager after project manager looked aghast as I raved, "The software is the design". Colleagues looked shifty when I mentione Pair Programming (I wear a beard, so I knew it wasn't my aftershave). Consequently I've never actually done any XP. However, I am Test Infected: I think Test First coding is the single most useful programming technique in my arsenal.

So it was mainly out of academic interest that I attended this month's BCS SPA session to hear Rachel Davies present an overview of the second edition of "eXtreme Programming Explained". There's been some re-arrangement of the furniture - the XP principles are now explained in a mind-map rather than that confused network diagram, some of the practices have been re-named - but there doesn't seem to be that much that's new.

The only new practice is something called the Informative Workspace, which is basically a visual plan of the project's progress, perhaps implemented as Stories on index cards pinned to the wall. Anybody who is interested can immediately see what the project has achieved and what is yet to be done simply by walking into the project room and looking at the wall. This is another example of the use of low tech in our high tech industry; like using the Hipster PDA instead of a Palm Pilot, the Informative Workspace works precisely because it isn't a computer-oriented approach. After all, whose first action on receiving the latest version of their project's plan in MS Project isn't to print it? Even though it comes out in an eye-wateringly small font (despite printing on A3).

There are some refinements. Simple Design and Refactoring have been amalagmated into a single practice, Incremental Design. Metaphor has been dropped, because hardly anybody used it. Some of the wackier names and concepts (Velocity, the Planning Game) have been renamed to make them more generally acceptable.

In fact, the main thrust seems to be towards making XP less, well, less extreme. The practices are graded into Core and Corollary sets. The practices are optional, in that we can now be doing XP even if we are not using all ten core practices. It's a bit like when SSADM version 4 came out: we can now tailor the method to the needs of our current project. Although I suspect we can't be XP if we're not doing Test First Coding, Pair Programming and Incremental Design.

There is a greater emphasis on continuous validation of the built software. We should always be ready to ship. So, not only are we supposed to test all the time but we are exhorted to do continuous integration all the time. Consequently the automated build and automated integration tests need to run in less than ten minutes. Daily builds have been replaced by Daily Deployments. That's right, into production with new code every day! To me this suggests that Kent Beck hasn't had much experience of dealing with FM-ed shops, where the change control process generally takes four-to-six weeks. In fact XP is still aimed at very small projects. Apparently there is a chapter on scaling XP to large projects but it basically consists of splitting one large project into many small projects, which doesn't seem like a satisfactory solution.

Down the pub afterwards someone remarked that lots of XP is simply obvious good practice and the rest is very dubious. Of course everybody thinks that about XP; if only we could get everybody to agree on which fifty percent of the practices are common sense and which are pants we'd be laughing. My colleague Paul O'Connell reckoned that XP is the product of an industry which has too many programmers and not enough engineers. So I held up my hands as an historian and we had an interesting discussion about the impact of feudalism on Japanese society.

By the way, given that XP is so keen on Pair Programming how come Kent Beck is the sole author of the book?

Book review: The Best Software Writing: Volume One, edited by Joel Spolsky

The first thing that grabbed me about Best Software Writing is the table of contents. Don't you just want to know "How Many Microsoft Employees Does It Take To Change A Lightbulb?" Or "What To Do When You're Screwed?" Or indeed "Starbucks Does Not Use Two-Phase Commit" Immediately it's obvious this is not a dry-as-dust tome on programming. Indeed, with the exception of Bruce Eckel's "Strong Typing vs Strong Testing" and Why The Lucky Stiff's canter through Ruby there's barely any programming at all.

What we get instead is writing about writing software, about the processes involved. For instance, Eric Lippert's punchline to the lightbulb joke is actually an amusing piece on change control. There are essays on coding. Ken Arnold argues cogently that it does not really matter which set of coding standards your project uses providing it uses something and everybody adheres to it. I agree. For a long time I have felt that the most useful tool Quest Software sells is not TOAD but PL/Formatter: it obviates all that useless time spent debating, codifying and enforcing layout standards. Apart from anything else coding standards tend to define that which can be defined - capitalisation, indentation, line breaks - when the most important things are: does the program implement the required functionality? Does it do so correctly, reliably and efficiently?

There are pieces about UI design and system design. Gregor Hohpe muses on the way Starbucks sells coffee as a metaphor for exploring messaging architecture. There are essays on project management, hiring developers, remunerating developers, outsourcing developers. There's even stuff about Sales! There are several TLA-splattered mediations on the future of the Web and the current state of the technology used to build it.

As someone who has wrestled with the issues thrown up by building a rich client interface out of JSPs I found some interesting insights in John Gruber's "The Location Field Is The New Command Line". Users want something that works, reliably, rather than something that looks pretty or comes with lots of gee-whizz features. Leon Bambrick makes the same point visually by imagining Google with MS Explorer's search interface. The corollary of this is the importance of methodologies like RAD (DSDM, XP, Agile, whatever). In order to demonstrate that our software works we must put working software in front of users on a regular basis. Otherwise we end up with requirements, specifications and acceptance tests predicated on aesthetics and the quantity of functions included.

The mix of technology is revealing. This is Spolsky's personal choice so the articles are skewed towards Microsoft's products and their technology base. There is C, C++ and VBA. There is a bit of Python and Ruby but almost no Java. There's lots of stuff about front-ends and middle tiers but nothing about databases. Certainly nothing about Oracle. Does this mean there's no good writing about databases or just that Spolsky doesn't read it? Actually, probably the former: most database writing is abstruse lectures about relational theory, uninformed witterings on normalisation or polemical rants about the inferiority of relational databases compared to object-oriented programming. Not many laughs there.

I have been calling these pieces as "essays" but they are an assemblage of various verbal constructs. There are proper essays, true. But there are also blog entries, transcriptions of conference speeches, even cartoon strips. What they have in common is that all are available on the internet. Even the Introduction is published on Joel On Software. What this means is we could click over to Google and by entering the table of contents into the location field assemble a copy of the book very cheaply (free if we print it at work, not that we would ever use our employer's paper for personal printing). So why buy the book?

Well, for a start it's got a nice colophon. And an index. Spolsky adds some insightful introductions and comments to each essay. The book is a nice size for reading on the tube or in the bath, neither of which is yet pratical with a laptop. But primarily, purchasing the book is a way of acknowledging the contribution the authors have made to the software writing community.