OOW2K7: Like, er, Day Two Man

Tuesday at OpenWorld 2007 for me opened with Tom Kyte's keynote. This was something different from Tom. Instead of talking SQL he talked about innovative ideas and the creative thinking process. His key point was that we have to recognise the danger of educated incapacity: too much knowledge can blind us to the existence of certain solutions. The answer is not to know nothing but to use our knowledge wisely. In particular we need to distinguish between what we know that really is true and what we think we know is true but which is in fact assumption or prejudice on our part.

One of the examples of creative thinking he used was an example of tuning from Craig Shallamar. The problem was in a stock control application. The workers would run a report which generated a picklist; the picklist was produced on the printer next to the terminal. They took the picklist and went to the warehouse. The problem was the report took three minutes to run. Craig spent a lot of time tuning the query and shaved about ten percent off the runtime. But ten percent of three minutes still means a long time for the workers to be waiting for the picklist to be printed. Then he noticed that the printer was on a network. So he had it moved to the warehouse. The workers ran the report and crossed over to the warehouse; by the time they had arrived the picklist had been printed. That's fast!

Tom finished up with a lament about the state of developer-DBA relations. Too often DBAs seemed to spend their time trying to prevent developers from doing things whilst the developers spend their days inventing ways to circumvent the DBA. This is a terrible waste of creativity.

Ray Roccaforte opened his presentation on Business Intelligence and Data Warehousing in 11g with a similar philosophical perspective. Indeed, a cosmic question. Given Moore's Law and given the pervasiveness of computers in all aspects of our lives why isn't the quality of our lives doubling every eighteen months? More mundanely, why isn't the productivity of our IT departments doubling every eighteen months?

His answer is that data volumes are growing even faster than Moore's Law. Yahoo had 100TB of data in 2005; two years on it has 250TB. And as disk speeds are lagging behind computing is actually slowing down. Oracle's strategy to deal with this:

• to scale to any size;
• put analytics in the database;
• put ELT in the database.

Incidentally that last point is not a typo: the preferred term is now Extract Load Transform because most of the transformation is done in the database.

Ray showed us a screen filled with 11g New Features for BI and DW - far more than he could cover in the session, so he just ran through some key points. There are further enhancements to partitioning, including the much sought after INTERVAL option and also the REF partition, which equi-partitions parent and child tables on the basis of foreign key values. There is the much reported SQL Resultset caching. There are enhancements to automatic statistics gathering, which will monitor the SQL statements run and gather stats on correlations between frequently queried sets of columns.

The big announcement is probably the changes to the OLAP cube. BI is dominated by aggregation queries. The classical solution is to build Materialized Views, but it is often imposible to build enough MVs, let alone manage and refresh huge numbers of them. In 11g the optimizer will treat an OLAP cube as a Materialized View, and will use query rewrite to access data in the cube, just like with MVs now. This means there is only one object to manage and the refresh window is considerably reduced. The OLAP cube can be interrogated using straight SQL statements, so advanced analytic queries can be added to any application.

Finally Oracle Warehouse Builder is included in the database at no extra cost, although one of the big draws, the Data Quality pack, will be a licenceable option.

Then it was off to the Oracle Magazine lunch. I got invited because I'm on the editorial advisory board. I felt guilty about it because I hadn't been asked to do anything in the last year but Tom Haunert was very magnanimous: it was his fault for not asking the board to do things. I was at the same table as Dr Paul Dorsey, which could have been embarrassing but I guess he doesn't read my blog. Also there was Lonneke Dikmans, the SOA Developer of the Year. Fortunately she and Paul have reached a good-tempered rapprochement on the merits or otherwise of SOA. I had an awkward moment with Tom Clark, CIO of PCA because I was out of practice at convesing with guys in suits. However, he turned out to be a really nice guy, and possibly the most unlikely fellow vegetarian I've met in a long time. I also chattered with Patrick Wolf and Jon Waldron.

Alas I missed the XML presentation but I did get to hear Donald Feinberg, VP and Distinguished Analyst at Gartner on the future challenges of OLTP and Data Warehousing for DBMS tools. This was the first time I had attended a Gartner presentation and it was really interesting. He specifically avoided mentioning any vendors because Gartner has to be neutral and this was an Oracle conference. But much of what he said chimed with other presentations I've attended: management of large data volumes, the importance of data quality, the physics of disk speeds, the importance of analytics. In fact I'm going to have to write up this one separately, once I've had some more time to think about it.

I rounded off the day by chilling out in the OTN Lounge on the third floor of Moscone West. I think the crew have finally found the right formula for the lounge as there's always lots of people there. Steven Feuerstein was doing a Meet The Experts session and I lurked at the back. I even managed to answer a question which Steven couldn't. This only happens one every other year so I claim bragging rights. Somebody asked what was the best way to pivot a table. Steven he wasn't sure but he would probably write some PL/SQL. So I pointed out that 11g had a PIVOT function. He laughed and said he didn't do SQL. Of course not, he's cured his addiction.

Then it was down off to the Thirsty Bear for the Bloggers' Meetup. As well as us old lags there were lots of new faces, which is an indication of just how the Oracle blogging community has grown over the last twelve months. Drink was taken and I think everybody had a good time. Thanks to Mark Rittman and OTN's Vikki Lira for organising it.