Thursday, November 10, 2005

Wikipedia: the dumbness of crowds

A posting on the XE forum observes that "By far the best known Wiki is Wikipedia". Almost certainly true but how very depressing.

The problem with the Wikipedia is that entries are of highly variable reliability and quantity. Due to the sort of people who contibute to Wikipedia, the entries on (say) Star Wars or the Klingon language are broader, deeper, more detailed and accurate than (say) the entries on relational database theory. In practice most of the entries on computing and related disciplines are reasonably reliable, because there are anough web users with relevant knowledge to correct obvious errors in their own fields (if they can be bothered).

Even so, unless you already know a fair bit about the topic it can be hard to determine whether the entry has been written by a leading expert or some passing nimrod. And when it gets to things like Latvian mythology, who knows? Is this entry on Tanis Diena, the sacred pig holiday a spoof? How would you find out, except by going to some authoritative (but less exciting) source such as the Encyclopedia Britannica in your local library? Even the Wikipedia founder says that too many entries in the Wikipedia "are nearly unreadable crap".

I am reminded of the chapter in "Surely you're joking, Mr Feynman" when Richard Feynman was reviewing physics textbooks for schools.
The man who replaced me on the commission said, "That book [that I thought was bad] was approved by sixty-five engineers at the Such-and-such Aircraft Company." I didn't doubt that the company had some pretty good engineers, but to take sixty-five engineers is to take a wide range of ability - and to necessarily include some pretty poor guys...It would have been far better for the company to decide who their better engineers were, and have them look at the book. I couldn't claim I was smarter than sixty-five other guys - but the average of sixty-five other guys, certainly!

Of course, there are some very good uses for the wiki technology. A prime example is the Extreme Programming Roadmap. This site is hosted by Ward Cunningham, who invented the Wiki concept as well as being one of the founders of XP along with Kent Beck. This wiki works because it is a site for sharing and exploring ideas. It is a conversation, an exchange of opinions, not a source of facts. It is precisely not an encyclopedia of Extreme Programming (eXPedia? or has somebody already got that?).

Wikipedia is predicated on the assumption that knowledge works like some kind of pachinko machine: the channel where the most balls go must be the truth. But actually all you end up with is a lot of balls.


Anonymous said...

Odd that you use a Feynman quote to ridicule Wikipedia, as I find their articles on Particle Physics to be well written and informative.

APC said...

This is not an original thought, but the way to evaluate a reference is by the poverty of its worst entries not the excellence of its best ones.

I do not dispute that there are some very useful, accurate and well-written entries in Wikipedia is not in question; I often link to its articles in my postings. The problem is: unless we know the topic thoroughly how are we to sort the wheat from the chaff?

SydOracle said...

Part of it is what you intend to use the information for. As a bit of background (such as wireless networking, extreme programming) I've found it a good starting place.
If I need authorative information, it probably needs to come from a specialist source anyway. I did correct an inaccuracy in its PL/SQL entry about dynamic SQL, but realistically anyone wanting to do anything with dynamic SQL isn't looking it up in Wikipedia.

My worry about linking to articles there is that you can't be sure that, at the time someone clicks on that link, the entry may have been updated with total crud.

Anonymous said...

We do this everyday. Every time we submit a search to Metalink or Google we do this. There are contextual clues that you can combine with your existing knowledge to evaluate the 'trust' value of any piece of information. Hell, for that mater ANY input into our brains goes through this process.

Perhaps a better question is: "Why would people turn to Wikipedia as a trusted source based on the lack of content in certain articles?" As with any other analysis of sources of information, it is done with a particular person's experience and other contributors to their 'Web of Trust'.

If you only evaluate your sources of trust based on their least trust-worthy information then you will never have a strong sense of trust in any source of information. This possibly isn't a bad thing, but I don't see very many people that can reach that level of enlightenment.

Noons said...

"Wikipedia is predicated on the assumption that knowledge works like some kind of pachinko machine: the channel where the most balls go must be the truth. But actually all you end up with is a lot of balls."

This is so true...

Anonymous said...