Stephen Wolfram for $200, Alex


by Tim Moran

Stephen Wolfram watches IBM's Watson supercomputer play Jeopardy! and says his software could do better.

You might know something about Wolfram: In addition to being a big-time thinker in the areas of particle physics, cosmology, cellular automata, computational complexity theory, and computer algebra, he's the creator of the computational software program, Mathematica. His latest endeavor is Wolfram|Alpha, an "answer engine" developed by his company Wolfram Research and deemed "the greatest computer innovation of 2009" by Popular Science.

Here, Wolfram discusses the relationship between Watson and his Alpha project.

The simple version is that he sees IBM's computer as a descendant -- a very sophisticated one -- of "information retrieval" systems, early versions of which go back fifty years or so. Writes Wolfram: "Wolfram|Alpha is . . . something much more radical, based on a quite different paradigm. The key point is that [it] is not dealing with documents, or anything derived from them. Instead, it is dealing directly with raw, precise, computable knowledge. And what's inside it is not statistical representations of text, but actual representations of knowledge."

The less-simple version is worth reading at his blog. What I found particularly interesting was research he did into how well today's search engines would do at Jeopardy! -- or, at least, how well they would be able to point to places on the Web on which could be found answers to Jeopardy! questions.

Just for fun, he says, he and his team fed some 200,000 Jeopardy! clues into the various extant search engines to see what documents were matched. Here are the results:

Search in Jeopardy.JPG In other words, Google got 69 percent correct in terms of the "answer" appearing on the first page; Ask came in right there with 68 percent, while Bing hit on 63 percent. The average human contestant gets 60 percent of answers correct, although champion Ken Jennings set a record with 79 percent correct answers.

Wolfram: "[A] plain old search engine gets surprisingly far. Of course, the approach here isn't really solving the complete Jeopardy! problem: it's only giving pages on which the answer should appear, not giving specific actual answers." Watson is better than those search engines, but Wolfram says using

Wolfram|Alpha would raise its game. "When it comes to actually answering many kinds of questions, one needs the kind of ability that Wolfram|Alpha has to compute things."

Wolfram himself might someday be the question to a Jeopardy! answer. Maybe Watson will get him, for the win.