What... is your favourite colour?

Yesterday somebody called Pradeep posted a question in the OTN PL/SQL forum asking for careers advice. He is a VB programmer working in a Microsoft code factory (in India I would guess). He wanted to know what the Oracle job market was like, because he wanted thinking of training as an "oracle developer".

The reaction from some of the forum regulars was dismissive. Here is a sample:

"There are so many career/job opportunities in IT that to ask on a forum like this for career direction advice is just ... Well, just not a good idea.

But put yourself in Pradeep's position. (Note the following is an extrapolation: I don't really know his circumstances). A combination of the commoditization of IT and outsourcing has produced software sweatshops, where you, the developer equivalent of a sharecropper, churn out code chunks to be assembled into systems elsewhere. It's repetitive, stultifying and without much room for personal growth. Even if you do hear of a better job you know you are surrounded by dozens of other sharecroppers who stand just as much chance of getting it as you.

So Pradeep has thought to himself, Oracle is a famous company but he doesn't know anybody who works as an Oracle developer. That means he is going to face much less competition for any Oracle job which might arise. You might find this optimistic but it is thinking outside of the box. He then shows further initiative by posting a question about Oracle careers on the OTN forums. Because he thinks that the people who answer questions there will be helpful - which they are - and they will have the relevant knowledge - they all have careers in Oracle development.

What he hadn't anticipated was that quite so many people regard the PL/SQL forums as a suitable place for rehashing Monty Python jokes and discussing antiquated programming languages but not for offering careers advice. It's a funny old world.

Openness: CMG's unsecret sauce

The CMG Wake last Friday was a lot of fun. Estimates have placed attendance at around the 700 mark, which made the pub extremely crowded and rather noisy.

Whilst queuing at the bar I got talking to someone who left the company in 1987. He put his finger on the unique thing about CMG. He now lectures in Business Studies and cites CMG to his students as an example of openness in business. In CMG everything was open: the personal files hung in open cabinets in the office. You could read anyone's file - from their job application form to their latest staff appraisal. Yes, including salary.

That's the bit which gets people. When Brian Behlendorf talked about introducing open source principles in general working practices at OOW2K6 one of the questions afterwards was whether such openness, if taken to its logical extreme, would result in everybody knowing how much everybody else earns.

Well, why not? Partly it's just embarrassment - many people would rather discuss their medical conditions or their bedroom fantasies than their pay-packet. But the main objection seems to be "I wouldn't want my colleagues to know how much I earn". The objectors are presuming that they earn more than their co-workers. I think some of those people would be very interested to discover that all their colleagues get paid more than them. And they'd want to know why.

The accepted wisdom in CMG was that open salaries promoted fairness: the management couldn't play favourites because anybody could ask them "How come Joe Soap gets paid 10K more than me?" and have to provide an answer. In practice there were probably all sorts of anomalies - especially in pay rates across different divisions - but it felt fair. I know people who work in law firms and finances houses where discussing your salary with co-workers is a disciplinary offence. Most companies aren't that fierce but very few companies are as open as CMG was.

I don't think openness was the only thing which made CMG special, but it was one of the reasons why so many people feel affection for the company, even if they did stop working for it over twenty years ago. Although I'm sure the promise of a free bar helped too.

Update: 06-MAR-2008

Currently Dilbert has an amusing take on payroll secrecy. I think this link will eventually break, so get it whilst it's hot :D

Farewell to CMG

Thirteen years ago tomorrow I joined a consultancy company called CMG. When I went for the interview I'd never heard of it but the interview process impressed me. Fortunately things worked out well and I've been with the company ever since. In 2003 we merged with another consultancy, Logica, to become LogicaCMG. Until Wednesday, when the name was reverted back to Logica.

This makes sense in many ways. More people had heard of Logica than had heard of CMG, although the separate companies had been of equivalent size. And it was a bit of a mouthful - even I had taken to calling us plain Logica. But it is a sad moment. CMG was a special sort of company; people define themselves by the fact that they worked for CMG, even if they left the company years ago. And that's why several hundred ex-CMGers - because we are all ex-CMGers now - are descending on a pub in London to mark the passing of the name in the traditional CMG style. Cheers!

Data modelling and other dying arts

Martin Widlake sent me an e-mail after last months UKOUG Unix SIG:

"At my presentation at the UKOUG Unix SIG yesterday I suggested that formal design was almost dead, replaced with organic design and asked if anyone still used ERDs. No one did. Not one.
This kind of bothered me. Does it mean that just ERDs are dead? Or that a room full of DBAs is a room full of people who do not do systems design (I am just as shocked by that if it is true)? Or maybe formal design is a dead concept."

There's two points here. The first, the utter lack of DBAs who do data modelling tasks, doesn't surprise me in the slightest. This is the nature of the modern DBA's job. Production DBAs look after live systems: they don't design them. Increasingly people are becoming DBAs straight out of college. These guys have never worked as developers and probably never will. The older geezers, who followed the more traditional route of starting out as programmers and progressing into the DBA role, probably haven't worked on development projects in years.

Also the IT landscape has changed. Even ten years ago many organisations had one or at most a handful of databases. It was possible for a DBA to be responsible for a single database; knowing its purpose and its value to their organisation was part of the job description. These days it is not uncommon to find DBAs working in teams looking after dozens even hundreds of databases. Furthermore the production DBA may well work for a different company (i.e. an outsourcer), possibly in a different continent from the users. Their relationship with the databases they administer is mediated through SLAs and ITIL compliant procedures. So they have little incentive and even less time to appreciate the databases under their care. Indeed, given the prevalence of Sarbanes-Oxley and similar pressures, production DBAs will be increasingly encouraged to remain in ignorance. A production DBA is somebody who knows the metadata of everything and the business purpose of nothing.

Of course, there are DBAs who do work on development projects. They are often combine the role with that of being a developer, especially on smaller projects. They often get called database engineers rather than DBAs. And production DBAs tend to regard database engineers as being developers not "proper" DBAs. I have been a database engineer on sites where I wasn't allowed the SYSTEM or SYS passwords for my project's development database. I would bet that everybody who goes to the Unix SIG is a production DBA.

The second question is whether anybody uses entity relationship diagrams, or more broadly, whether anybody still does logical data modelling. I can't answer this one from personal experience. I've been on a data warehouse project for four years now: I only deal in existing schemas. Even when I have done design it has been for ETL infrastructure and similar, so I have leapt straight to physical tables. As I started out with SSADM I do feel a bit guilty about this. Although I must say I haven't exactly missed drawing Entity Life History diagrams.

Anecdotally, there does seem to be a general decline in the practice of data modelling. There were hardly any presentations on modelling at the last UKOUG conference or at Open World 2007. The Modelling and Design is one of the smaller UKOUG SIGs. The ODTUG Designer listserver has flurries of activity but since Oracle announced the death of Designer it has - understandably - experienced a major drop in traffic. There are occasional questions about data modelling in the OTN forums, but these are frequently from students rather than practitioners. It is depressing to consider that the most commonly referenced data model seems to be the fundamentally flawed Entity-Attribute-Value. My last piece of circumstantial evidence is that the Oracle blogosphere rarely features posts about data modelling. The only blog I know which regularly discusses data modelling is The Database Programmer and even Ken Downs only talks about tables.

Of course people are doing system design. There's lots of design about but I would guess that it all happens in UML. So the majority of logical data modelling these days produces class models rather than ERDs. The physical database design stage is much more likely to consist of ORM than mapping entities to tables. Now that's not the sort of party you invite a DBA to, because you just know they're going to glower in the corner, drinking heavily and muttering to themselves. So the mappings and the database design will be done by middle-tier developers. Our communal prejudices tell us this is unlikely to produce a correct and peformant database design, not least because projects which use such an approach tend to make a fetish of database agnosticism and platform independence. So in the long run we might see a resurgence in data modelling, as part of the tool set for rescuing poorly performing class models.

As a tangent, Dominic Delmolino observed in a recent blog that

"many of the people I’ve been interviewing seem to be taken aback by a few simple SQL questions, telling me that DBA’s (sic) don’t do SQL."

Again, why is this surprising? SQL knowledge is going the way of data modelling for production DBAs. There is a whole raft of GUI administration tools - Quest Spotlight, BMC Patrol, Embarcadero, OEM, etc - whose sole purpose is to allow DBAs to monitor and manage large numbers of databases without using the command line and without knowing SQL. Again this is inevitable given the landscape I described above. Old skool DBAs - the ones who started out managing a single database - will have accreted a personal library of SQL scripts, shell scripts and utilities which do all these things. But people starting out now will probably find themselves operating in shops with dozens of databases and no time to roll their own tools. If they are lucky there will be an old lag to pass on some skills and some scripts; more likely there will be a shrink-wrapped GUI tool. Besides, remember that Oracle Enterprise Manager was introduced in Oracle7: it is perfectly feasible for somebody to describe themselves as an experienced DBA who has never administered a database in any other way.

One of those days

My main task for today was to pick up some zipped files from a server on the client site and bring it back to the office to load onto our development server. Simple enough task.

Except that the network connection was a bit brittle. WinSCP kept failing at 99% complete. The same files were available on the QA server. Alas that box was down: Support hadn't noticed until I mentioned it. So it was back to the first server. I found that if I copied a single zip at a time I could at least keep track of the failures and re-copy when necessary.

Eventually I had all the files I needed. It was then a matter of burning them to CD. Inevitably the desktop I was using didn't have a CD burner but fortunately there was another a PC in the office which had a burner and could see my networked TrueCrypt folder (it was potentially sensitive stuff).

Back at my office I discovered that my access to the shared development network was locked. This happens from time to time because, well, just because. It mattered today because our two sysadmins had already left for the weekend (to Wales and France respectively) so there was no chance of me getting my account unlocked before Monday. So one of my co-workers had to transfer the files to the network. I couldn't actually do anything with the data but at least I would be able to erase the CDs.

While the transfer was happening I went to get a coffee. The coffee machine was out of coffee. Grrrr.

I recount these woes not because they are necessarily typical of my working day (some days I really get lots done) but simply to illustrate a larger point. There's a recent article on the Artima site discussing the impact of languages and frameworks on programmer productivity. The sad fact is only a relatively small part of a developer's day is actually spent coding. There are meeting to attend, cranky networks to wrestle and tea bags to be dunked because the coffee machine's on the blink.

Meeting William Gibson

Not many novelists bring out the fanboy in me, but Gibson is one of them. Neuromancer was, literally, the book that changed my life. It's why I decided to become a computer programmer.

In the summer of 1986 I was a disillusioned History student about to become a disillusioned History graduate. I had finished my finals, I had no job lined up and I had no idea what I wanted to do with my life. It was in this state of aimlessness that - inspired by an article by William Leith in the NME - I purchased Neuromancer. It was the first pure SF novel I had read in years and it was completely gripping. I can still recall the adrenal rush of the last few chapters, racing through them just to discover how the story ended.

I had read SF as an adolescent but discovering JG Ballard had derailed me: nothing else in the genre could quite match the intensity of his prose. At university I had read mainly non-fiction: there was so much reality to discover, it seemed pointless to read to stuff people had made up. When I was studying European heresy my tutor John Critchley advised me to read Umberto Eco's The Name of the Rose (I was chuffed to explain to him that the blind librarian Jorge of Burgos was a reference to Jorge Luis Borges, because that joke had passed him by). I also read Mishima because I did courses on both medieval and modern Japan, and he seemed appropriate. For light relief I read Italo Calvino's short stories and the Blandings novels of P.G.Wodehouse.

I realise that list probably makes me sound like an insufferable prick. But then I probably was an insufferable prick at university, so that's fair.

I'm not going to do a review here. I presume everybody who reads blogs has read Neuromancer. If that's not the case stop reading this right now and go read it. Like most SF it has dated but not as badly as some. Few SF authors have changed the world. My quasi-namesake Arthur C Clarke did, with his invention of the geosynchronous satellite, but largely his vision has not been realised. No lunar explorers have disinterred mysterious black slabs because there are no bases on the moon. Heck, nobody's even been to the moon for over thirty years. Whereas it's a measure of the impact Gibson's writing has had on the world that he has stopped writing SF and moved into regular fiction without any meaningful change of tone or subject matter. (I guess Bill Gates, Steve Jobs, Michael Dell, Larry Ellison and others have also had something to do with it).

Gibson's insight was to take the glamour which traditionally attached to spaceships and astronauts and transfer it to computers and programmers, who were - it's fair to say - not glamourous. At Exeter the Comp Sci students were pimply-faced youths in brown and beige, as boring as accountants. In fact, more boring: at least there were some girls amongst the accountancy students. Neuromancer changed that image by making computers cool, sexy and edgy. Which is why it has had such a profound effect on modern life. All over the world, pimply-faced youths in brown and beige set about immanentising a world in which computers - and by extension the people who worked with them - were cool sexy and edgy.

Neuromancer was Gibson's first novel. In 1986 it was a bit frustrating to discover that there wasn't anything else to read by him. I just had to wait until Count Zero came out. On the other hand it does mean that every few years there is a new novel to look forward to.

So today I joined the line of people to buy a signed copy of Gibson's new novel, Spook Country, at Forbidden Planet. I told him I was grateful to him for writing Neuromancer because it had directed me into a career in software development. His face crinkled with recognition. "Oh you're one of those guys," he said. Yep, I'm one of those guys.

Neology Corner: Reshoring

I am introducing a new coinage into the wild: reshoring. This is what happens when a project that has been offshored turns out to have been wrongshored (heh); when everything's gone pearshaped and the project is brought back to be sorted out. Sample usage: "Have you heard about Project FFIENNES? They're reshoring it from Bangalore."

Was my coining of this phrase inspired by any specific project I have come across? You might think that, I couldn't possibly comment.