Tuesday, February 15, 2011

Man vs. Machine

I must admit that I've gotten quite caught up in the IBM Watson Jeopardy challenge. It makes me proud to work for such an amazing company. That said, and not to take anything away from Watson or the researchers that built it, I can't help thinking that the match-up isn't 100% fair...here are a couple reasons:

1. The machine is simply faster on the trigger.
I think Jennings (and probably Bruce Rutter too, funny how everyone knows Jennings but at least so far Rutter has twice his score) knew a lot of those answers...its just that Watson could consistently beat them to a buzzer (though the guys did win on a couple that Watson clearly knew the answer to as well). I've actually read that buzzing too early creates a short time penalty, so that makes it even tougher on the humans.
What would be really interesting to me is if they would compensate for human reaction time, for instance, by accepting all buzzers within the first 300-400ms or so and then randomly select one of the buzzers that made that "instantaneous" benchmark.

2. Two humans, one computer.
Assuming there are a certain class of questions that are particularly difficult for the computer and easier for humans (perhaps a large assumption?), it isn't totally fair that there are two human contestants that end up competing with each other for these scraps. Maybe it would be better if the competition was just head-to-head? It kind of reminds me of a 3rd party candidate taking more votes from one of the mainstream candidates than the other. Does that make Rutter the Teddy Roosevelt to Ken Jennings' William Howard Taft?(surprised I couldn't find a better example). Anyway, don't blame me, I voted for Kodos.

PS. After Watson finishes these guys off tomorrow (at this point I think that losing overall would be a pretty huge choke job), wouldn't it be interesting to see Watson compete against other machines...like some kinda of corporate jeopardy tournament? Like, maybe if Jeopardy would run a special machine-contestant day in a year or so?