Thursday, March 17, 2011

Man vs Machine: Watson Aftermath

  If your were one of the twelve million viewers last month watching IBM's supercomputer Watson competing on Jeopardy! against champions Brad Rutter and Ken Jennings, you saw a rather dominating performance from Watson. Watson finished the first game at $35734 and completed the three day, two game championship with a total of $77147 with Ken Jennings finishing second at $24000 and Brad Rutter with $21600. IBM claimed the million dollar prize for the championship, donating the money to charities World Vision and World Community Grid. Jennings took home $300000, donating half to VillageReach, and Rutter claimed the $200000 prize, donating half to the Lancaster County Community Foundation.
Ken Jennings finishing off Final Jeopardy with a bit of humor
 
  Watson lead a commanding performance through most of the games. Watson would only buzz in when it had a confidence level above a certain threshold but its buzz was lightning fast. The only times Jennings or Rutter were able to chime in were either when Watson had low confidence in its answer or when Rutter or Jennings anticipated the buzz and beat Watson to the punch. Watson also played very logically, if a bit bizarrely to human strategies. It would hop around the board searching for the Daily Doubles and when it found them it would bet very precise amounts based off of the current state of the game, wagering $6435 on an earlier Daily Double and $17973 on the Final Jeopardy on the third day.
  Not all of Watson's performance was perfect. Watson couldn't "hear" what the other contestants buzzed in with, if Ken or Brad rang in first and answered incorrectly, Watson would sometimes buzz in and give the same answer. At the end of the first game, I'm sure much to the chagrin of its creators, Watson's answer of "What is Toronto?????" to the Final Jeopardy category "U.S. Cities" was a bit off the mark. The presence of the five question marks displayed a very uncertain answer from Watson. The IBM creators said they didn't give much weight of choosing answers to the category title because Jeopardy! writers would often use jokes or puns in the titles. Unfortunately for Watson's human competitors, it wagered only $947 of its $36681 current first day total, not giving an ounce of respite for the challengers. For those interested, here is the answer that stumped Watson:
"This U.S. city's largest airport is named for a famous World War II hero, its second largest for a famous World War II battle."
  Overall, I was incredibly impressed by IBM's showing on Jeopardy! Watson showed a depth and understand of the human language that could have a great number of implications. Being able to parse a question and delve through terabytes of information and return an answer in a few seconds is a critical step towards an era of intelligent, helpful computers. IBM wanted to show off what they could do, and show they did.

1 comment:

  1. I think you are a journalist in disguise! This is a great post, worthy of publication. Well done!

    Of course, I would love to have you comment on the "overlord" bit. Are people's worries about Watson's performance justified?

    ReplyDelete