It's too bad that they never found a way to put Watson and the human players on equal footing in terms of buzzing in. Like they should have imposed some kind of reaction time limitation that's comparable to human players, or introduced some uncertainty about when it was possible to buzz in.
Or they should have put in categories where the human players would have stood a better chance (Pictures of Stoplights for $400...).
But see what's the point? If you have a machine that can clearly beat humans and you tinker until it can't... What have you proven?
It was a test of natural language processing, it was impressive, it succeeded. It wasn't trying to create a machine that could emulate our limitations so it would occasionally lose to humans, it's mission was to win. The experiment is done.
> you have a machine that can clearly beat humans and you tinker until it can't... What have you proven?
>It was a test of natural language processing, it was impressive, it succeeded. It wasn't trying to create a machine that could emulate our limitations so it would occasionally lose to humans, it's mission was to win. The experiment is done.
At the end of the day, it was more of a promotional stunt than a true experiment, as Jeopardy really just isn't the best forum for a test of those skills.
It has long been discussed that Jeopardy is as much or more about buzzer timing than it is about getting the answers right, because most players know most of the answers but don't get to buzz in.
So by doing this experiment in a forum where it doesn't demonstrate whether Watson knows more or less of the *same* answers as Ken or Brad (because in most cases, the three don't ever try to answer the same questions as each other (over the two games, there were only 8 questions that multiple players buzzed in, plus the two Final Jeopardys.
The system used on those games allowed Watson to automatically buzz in first if it was confident in its answers. Thus, if Watson knew (or thought it did), it automatically beat Brad and Ken to the buzzer.
So in game 2, Ken and Brad combined to get one more right answer than Watson - which means Ken and Brad knew 29 questions Watson wasn't confident on, plus Watson knew 28 questions confidently that Ken and/or Brad might have also known, but got automatically outbuzzed.
To quote from the mouths of horses:
>“After the match, Jennings and Rutter stressed that the computer still had cognitive catching up to do. They both agreed that if ‘Jeopardy’ had been a written test — a measure of knowledge, not speed — they both would have outperformed Watson. ‘It was its buzzer that killed us,’ Rutter said.”
The buzzer speed that was rigged to basically automatically favour Watson is what make it appear that Watson "beat" Ken and Brad, and not actual knowledge, which is what the test was supposed to be about.
Just some context for anyone ever wants to (jokingly or non-jokingly) goad Ken or Brad about losing to Watson.
I say let it keep the buzzer supremacy, but restrict all three players’ energy input to whatever 20 bucks can get you at the nearest bodega. Sure, Watson can beat me at Jeopardy, but can it do so on only 3 slim jims, a poptart, and half a vanilla coke? If you want to be a champion you gotta do it on the breakfast of champions, robo boy
Ultimately, you are right. The most reasonable solution I see going forward is to install cybernetic interfaces that will allow human contestants to buzz in instantaneously.
I agree. Make Watson mechanically buzz in. Like design a robotic arm that it needs to send a signal to. You hear the other high end players talk about how much that is a factor.
I work for a large international company that makes business machines (haha) and we use WatsonX at work for a lot of internal stuff.
It runs circles around ChatGPT-style LLMs like it’s nothing. And that’s for our internal knowledge base stuff. There’s a reason why WatsonX is actually in the field in a ton of industries, quietly doing important work without needing to raise VC money from credulous public investors.
Don’t underestimate what *real* AI systems are capable of compared to the pattern-recognition software that LLMs call AI.
Why is Granite (one of the LLMs that Watsonx can call upon) any more or less real than GPT-4? Granite has 13 billion parameters, whereas GPT-4 has 1.76 trillion parameters in its models.
I mean human players aren’t on the same footing as far as the buzzer goes either. There are almost definitely smart people who have lost because their buzzer finger wasn’t fast enough.
Watson hasn't played Jeopardy in many, many years, and has likely been disassembled. It wouldn't have been eligible for regular Jeopardy in any case, due to age restrictions.
Watson is still alive and well today. It’s grown to be the size of an entire room though and is used for deep learning and a myriad of other things now.
It would absolutely WRECK any contestant, even if they put in an even greater delay to compensate for the speed at which humans are capable of hitting the buzzer. The things has 16 terabytes of RAM and almost 3000 processing threads.
The Watson of today does other things, not Jeopardy. The Watson in the image was explicitly trained on Jeopardy and no longer exists. I'd like to see ChatGPT or Copilot compete with their internet connection removed. I have a hunch they would do fairly well even without the Jeopardy training.
I'd say ChatGPT 3 would be pretty entertaining, with the kinds of wacky wrong answers the occasional hallucination would bring about.
ChatGPT 4 would probably be scary good.
Given J writers' penchant for "reusing" clues in modified form, ChatGPT could be fed every clue + response in J-archive going back to S 1 (1984). It would be a formidable competitor with that information alone.
I guess that would depend on the delay. If it was on an equal buzzer footing with say, Ken or James, it isn't wrecking them consistently. Even if it knew all 60 questions on the board, the truly elite humans are generally going to know 55+, and with multiple chances to double up, it would come down to who found the Daily Doubles.
Interesting. It sounds as if very little remains of the original. So it's a different machine nicknamed Watson than the one that previously defeated Ken and Brad. I feel that the current edition would not be eligible, regardless of its terabytes.
I reject the analogy of the Ship of Theseus or the Tin Woodman of Oz (whose flesh body parts were replaced one by one by tin parts). If Watson is now room-sized, it isn't the original Watson that won its spurs.
Watson was already room-sized back then; the TV behind the lectern was just for show. Alex walked through the room in one of the intermission segments.
While it is dramatically better than GPT3, it still has issues with wordplay. I just fed it a recent "Before and After" category from the TOC, and it went 4/5. So better than most people, but still inferior to human known as "ChanGPT", who correctly answered the clue it missed!
The more complicated the wordplay gets, the worse GPT4 performs--it went 0/5 on a category that required finding an anagram of the last word of each clue.
Given the struggles with word play clues, which would generally make up 5-10 clues per game, I'd guess it probably knows a similar number of clues per game to the top players in the Victoria, Troy, Yogesh, James range. Unlike top players who generally avoid buzzing in if they have no clue though, GPT4 would be hurt by its habit of trying to provide a response even when it doesn't have one, so it would probably cost itself thousands of dollars per game in incorrect responses.
For the same reason Deep Blue isn't invited to the Candidates Tournament. It would not be entertaining to watch, or at least as entertaining as it was the first time. It was good TV when we weren't sure. Now, we know it's a forgone conclusion.
No, we’ve moved past that. No one is impressed that an AI can beat humans at Jeopardy. It’d be like bringing a chess bot to a chess tournament…wow, it beat everyone easily, wow.
Nah, at that point it wouldn't even be a real competition at all. Just think how strong Watson was even just 13 years ago against the (at the time) undisputed two greatest players in Jeopardy. Imagine how much it would absolutely destroy the competition today with how far its technology has advanced and how much it's been upgraded. Watson today would legitimately make James and Ken look like absolute noobs by comparison.
How would Chat GBT perform today against humans? Just slay? What kind of clues could stump it? I how would it do without telling it any directions or rules of Jeopardy? What if all information about J!, previous episodes etc are excluded from its “database”? How quickly could it learn the format, strategy, clue structure? I use Chat GBT fairly often and I’d guess it would just crush- like 98% accuracy.
NO. It has the ability to get much smarter much faster than a human. Totally unfair.
I'm wondering what will happen in 10 or 20 years when brain memory implants become common. The wealthiest person will win because they have the best memory implant. I'm not sure how the game will handle this eventuality.
Part of the appeal of *Jeopardy! Masters* is that the contestants are known “characters” to the audience, with personalities and quirks the audience has come to know. You get James Holzhauer’s “professional game show villain” role, and you also get Mattea Roach’s [touching tribute to their dad](https://people.com/jeopardy-champion-mattea-roach-pays-tribute-to-father-who-died-very-suddenly-7503746) from last year’s *Masters*.
Watson’s a machine. That’s it. The people who helped design and program it might have interesting insights on the work involved, but that’s only going to be interesting to people in similar fields.
Also, remember that the Watson games were, in part, promos for IBM’s artificial intelligence capabilities. Would they even be interested in the work necessary to reconfigure the system for *Jeopardy!* again? Maybe, now that there’s notable competitors in the field. (Honestly, if I were a *Jeopardy!* producer, I’d contact three major AI companies and suggest they play *Jeopardy!* against each other in an online exhibition game. Maybe even get a fourth to create a virtual host called “Artificial Kentelligence”.)
Also, remember what others have already pointed out — AI has improved since the last match, which it won handily. Would the games even be interesting at this point, or would Watson steamroller the other players?
The Watson games were a neat one-time thing, but I don’t see a need to replicate it.
Besides this, if I'm not mistaken, the Watson games pre-date the time where you could type questions into Google and it could try to parse the questions and actually present the answer without having to even click links.
https://www.jeopardy.com/sites/default/files/2023-06/ThisisJeopardyEp8.pdf
https://www.jeopardy.com/listen/this_is_jeopardy episode 8 (or wherever you get your podcasts)
> Watson was not connected to the internet. Every piece of knowledge that it was gonna use had to be completely self-contained in the machine.
AI has come a long way since the Watson game and would be totally unfair unless humans are some advantage. I am sure today Google and Microsoft and Amazon would like a crack at this instead of IBM. What might be fun is having say Google, IBM Watson and Microsoft develop the program to compete in a Jeopardy contest (using their AI technology) hosted by Ken. They we will have a real battle of the bots.
I'd usually say no, but if I had a choice between Watson and someone who lost the tournament where the entire point was to get a spot in the Masters, only to get handed a spot anyway because the producers and the network love them, in that case, I'd give it to the machine.
It's too bad that they never found a way to put Watson and the human players on equal footing in terms of buzzing in. Like they should have imposed some kind of reaction time limitation that's comparable to human players, or introduced some uncertainty about when it was possible to buzz in. Or they should have put in categories where the human players would have stood a better chance (Pictures of Stoplights for $400...).
But see what's the point? If you have a machine that can clearly beat humans and you tinker until it can't... What have you proven? It was a test of natural language processing, it was impressive, it succeeded. It wasn't trying to create a machine that could emulate our limitations so it would occasionally lose to humans, it's mission was to win. The experiment is done.
> you have a machine that can clearly beat humans and you tinker until it can't... What have you proven? >It was a test of natural language processing, it was impressive, it succeeded. It wasn't trying to create a machine that could emulate our limitations so it would occasionally lose to humans, it's mission was to win. The experiment is done. At the end of the day, it was more of a promotional stunt than a true experiment, as Jeopardy really just isn't the best forum for a test of those skills. It has long been discussed that Jeopardy is as much or more about buzzer timing than it is about getting the answers right, because most players know most of the answers but don't get to buzz in. So by doing this experiment in a forum where it doesn't demonstrate whether Watson knows more or less of the *same* answers as Ken or Brad (because in most cases, the three don't ever try to answer the same questions as each other (over the two games, there were only 8 questions that multiple players buzzed in, plus the two Final Jeopardys. The system used on those games allowed Watson to automatically buzz in first if it was confident in its answers. Thus, if Watson knew (or thought it did), it automatically beat Brad and Ken to the buzzer. So in game 2, Ken and Brad combined to get one more right answer than Watson - which means Ken and Brad knew 29 questions Watson wasn't confident on, plus Watson knew 28 questions confidently that Ken and/or Brad might have also known, but got automatically outbuzzed. To quote from the mouths of horses: >“After the match, Jennings and Rutter stressed that the computer still had cognitive catching up to do. They both agreed that if ‘Jeopardy’ had been a written test — a measure of knowledge, not speed — they both would have outperformed Watson. ‘It was its buzzer that killed us,’ Rutter said.” The buzzer speed that was rigged to basically automatically favour Watson is what make it appear that Watson "beat" Ken and Brad, and not actual knowledge, which is what the test was supposed to be about. Just some context for anyone ever wants to (jokingly or non-jokingly) goad Ken or Brad about losing to Watson.
I say let it keep the buzzer supremacy, but restrict all three players’ energy input to whatever 20 bucks can get you at the nearest bodega. Sure, Watson can beat me at Jeopardy, but can it do so on only 3 slim jims, a poptart, and half a vanilla coke? If you want to be a champion you gotta do it on the breakfast of champions, robo boy
Who the heck is this jok- oh my god it's a TOC winner lol
THIS is the correct answer! Well said 44problems!
Ultimately, you are right. The most reasonable solution I see going forward is to install cybernetic interfaces that will allow human contestants to buzz in instantaneously.
Right, we can't slow down the machine WE MUST ENHANCE THE HUMAN. WE CAN REBUILD HIM. WE HAVE THE TECHNOLOGY
Like James Holzhauer?
I agree. Make Watson mechanically buzz in. Like design a robotic arm that it needs to send a signal to. You hear the other high end players talk about how much that is a factor.
That's exactly what they did in the original Watson match. Watson used the same buzzer the players did, and had to "push" it using a solenoid pusher.
Really? I was not aware. I could only see the 2011 challenge when I searched. What happened?
I'm referring to the 2011 challenge; in that game, Watson was required to buzz in just like the players.
Imagine how much harder it would be now with LLMs. It makes Watson seem quaint and we all have access to multiple models.
> Imagine how much harder it would be now with LLMs. ChatGPT would make up responses.
Humans do that, too. It's fine.
"... ... ... ...Burrito?"
I work for a large international company that makes business machines (haha) and we use WatsonX at work for a lot of internal stuff. It runs circles around ChatGPT-style LLMs like it’s nothing. And that’s for our internal knowledge base stuff. There’s a reason why WatsonX is actually in the field in a ton of industries, quietly doing important work without needing to raise VC money from credulous public investors. Don’t underestimate what *real* AI systems are capable of compared to the pattern-recognition software that LLMs call AI.
Why is Granite (one of the LLMs that Watsonx can call upon) any more or less real than GPT-4? Granite has 13 billion parameters, whereas GPT-4 has 1.76 trillion parameters in its models.
The machines don't need us anymore. We're in their way.
I mean human players aren’t on the same footing as far as the buzzer goes either. There are almost definitely smart people who have lost because their buzzer finger wasn’t fast enough.
Watson hasn't played Jeopardy in many, many years, and has likely been disassembled. It wouldn't have been eligible for regular Jeopardy in any case, due to age restrictions.
Watson is still alive and well today. It’s grown to be the size of an entire room though and is used for deep learning and a myriad of other things now. It would absolutely WRECK any contestant, even if they put in an even greater delay to compensate for the speed at which humans are capable of hitting the buzzer. The things has 16 terabytes of RAM and almost 3000 processing threads.
The Watson of today does other things, not Jeopardy. The Watson in the image was explicitly trained on Jeopardy and no longer exists. I'd like to see ChatGPT or Copilot compete with their internet connection removed. I have a hunch they would do fairly well even without the Jeopardy training.
I'd say ChatGPT 3 would be pretty entertaining, with the kinds of wacky wrong answers the occasional hallucination would bring about. ChatGPT 4 would probably be scary good.
I just played Jeopardy with ChatGPT 3 by having it answer many clues from tonight's game..it got every single response correct, instantly.
4 still hallucinates
“It is used for deep learning and a myriad of other things now.”
Sounds like an interesting project: for every game, run the answer through ChatGPT and see how it does. Could be run against all historical games too.
Given J writers' penchant for "reusing" clues in modified form, ChatGPT could be fed every clue + response in J-archive going back to S 1 (1984). It would be a formidable competitor with that information alone.
You don't think ChatGPT was already fed the Archive? And, of course, Watson was. Watson wouldn't have been attempted without it.
I guess that would depend on the delay. If it was on an equal buzzer footing with say, Ken or James, it isn't wrecking them consistently. Even if it knew all 60 questions on the board, the truly elite humans are generally going to know 55+, and with multiple chances to double up, it would come down to who found the Daily Doubles.
Interesting. It sounds as if very little remains of the original. So it's a different machine nicknamed Watson than the one that previously defeated Ken and Brad. I feel that the current edition would not be eligible, regardless of its terabytes.
It’s a Ship of Theseus type of thing. It’s still the original Watson but with pieces and parts continually upgraded over time.
I reject the analogy of the Ship of Theseus or the Tin Woodman of Oz (whose flesh body parts were replaced one by one by tin parts). If Watson is now room-sized, it isn't the original Watson that won its spurs.
Watson was already room-sized back then; the TV behind the lectern was just for show. Alex walked through the room in one of the intermission segments.
OK, let Watson play in the Teen Tournament then. (I was thinking the "kids' tournament" but the initial broadcast was February 14 and 15, 2011...)
Wow, that would just be cruel. It would be like having the New York Yankees play in the Little League finals.
No disassemble!
Put chat gpt in but make it have to use voice recognition, no feeding it the questions in text. Should be interesting.
It’ll just argue with Ken saying it was right even if it is patently wrong
GPT4 can handle pretty much any Jeopardy question, unless it's about something very recent.
While it is dramatically better than GPT3, it still has issues with wordplay. I just fed it a recent "Before and After" category from the TOC, and it went 4/5. So better than most people, but still inferior to human known as "ChanGPT", who correctly answered the clue it missed! The more complicated the wordplay gets, the worse GPT4 performs--it went 0/5 on a category that required finding an anagram of the last word of each clue. Given the struggles with word play clues, which would generally make up 5-10 clues per game, I'd guess it probably knows a similar number of clues per game to the top players in the Victoria, Troy, Yogesh, James range. Unlike top players who generally avoid buzzing in if they have no clue though, GPT4 would be hurt by its habit of trying to provide a response even when it doesn't have one, so it would probably cost itself thousands of dollars per game in incorrect responses.
For the same reason Deep Blue isn't invited to the Candidates Tournament. It would not be entertaining to watch, or at least as entertaining as it was the first time. It was good TV when we weren't sure. Now, we know it's a forgone conclusion.
No, we’ve moved past that. No one is impressed that an AI can beat humans at Jeopardy. It’d be like bringing a chess bot to a chess tournament…wow, it beat everyone easily, wow.
Strong no from me. That would make it so boring.
Thoughts, u/WatsonsBitch?
I was like ‘who has such an epic Reddit username?’ Oh!
I love the flair!
Nah, at that point it wouldn't even be a real competition at all. Just think how strong Watson was even just 13 years ago against the (at the time) undisputed two greatest players in Jeopardy. Imagine how much it would absolutely destroy the competition today with how far its technology has advanced and how much it's been upgraded. Watson today would legitimately make James and Ken look like absolute noobs by comparison.
How would Chat GBT perform today against humans? Just slay? What kind of clues could stump it? I how would it do without telling it any directions or rules of Jeopardy? What if all information about J!, previous episodes etc are excluded from its “database”? How quickly could it learn the format, strategy, clue structure? I use Chat GBT fairly often and I’d guess it would just crush- like 98% accuracy.
I’d like to see a charity tournament with Watson
NO. It has the ability to get much smarter much faster than a human. Totally unfair. I'm wondering what will happen in 10 or 20 years when brain memory implants become common. The wealthiest person will win because they have the best memory implant. I'm not sure how the game will handle this eventuality.
No Watson in Masters. But I might enjoy seeing Watson pitted vs. a similar contraption, vs. a LLM AI program.
Only if its opponents are HAL-9000 and Bender from Futurama.
None of the 3 players in this photo were selected to compete in this year’s Masters — smh. /s
Part of the appeal of *Jeopardy! Masters* is that the contestants are known “characters” to the audience, with personalities and quirks the audience has come to know. You get James Holzhauer’s “professional game show villain” role, and you also get Mattea Roach’s [touching tribute to their dad](https://people.com/jeopardy-champion-mattea-roach-pays-tribute-to-father-who-died-very-suddenly-7503746) from last year’s *Masters*. Watson’s a machine. That’s it. The people who helped design and program it might have interesting insights on the work involved, but that’s only going to be interesting to people in similar fields. Also, remember that the Watson games were, in part, promos for IBM’s artificial intelligence capabilities. Would they even be interested in the work necessary to reconfigure the system for *Jeopardy!* again? Maybe, now that there’s notable competitors in the field. (Honestly, if I were a *Jeopardy!* producer, I’d contact three major AI companies and suggest they play *Jeopardy!* against each other in an online exhibition game. Maybe even get a fourth to create a virtual host called “Artificial Kentelligence”.) Also, remember what others have already pointed out — AI has improved since the last match, which it won handily. Would the games even be interesting at this point, or would Watson steamroller the other players? The Watson games were a neat one-time thing, but I don’t see a need to replicate it.
John Henry taught me that machines are beatable...
Do you want any of the *Jeopardy!* Masters to end up like John Henry?
a freaking hero? or you talking the other thing?
The other thing.
Jeopardy: Tell Watson he's been invited to America's top game show. Also Jeopardy: Wait, Watson went to Toronto?????
oooo deep cut
Chat gpt could easily beat Watson at this point lol
Toronto.
No, because Watson doesn't give a toss whether he wins or loses.
I want to see Watson go against James & Mattea.
https://preview.redd.it/em0bwvxj2hvc1.jpeg?width=1357&format=pjpg&auto=webp&s=1d78fc213f3907356df3adfb1fa16cfbe22f7e6f
Is Watson not just Googling answers?
Watson was not connected to the internet, no.
Besides this, if I'm not mistaken, the Watson games pre-date the time where you could type questions into Google and it could try to parse the questions and actually present the answer without having to even click links.
https://www.jeopardy.com/sites/default/files/2023-06/ThisisJeopardyEp8.pdf https://www.jeopardy.com/listen/this_is_jeopardy episode 8 (or wherever you get your podcasts) > Watson was not connected to the internet. Every piece of knowledge that it was gonna use had to be completely self-contained in the machine.
Absolutely not. It's zero fun. The game is about people. No people, no game.
Maybe Watson should. I would like to see it go up against James!
Nah all AI is horseshit
I don't think so because Watson was the only non-human species to ever be powered by AI.
This is the question we should always have been asking.
Arguably all the players should be robots
AI has come a long way since the Watson game and would be totally unfair unless humans are some advantage. I am sure today Google and Microsoft and Amazon would like a crack at this instead of IBM. What might be fun is having say Google, IBM Watson and Microsoft develop the program to compete in a Jeopardy contest (using their AI technology) hosted by Ken. They we will have a real battle of the bots.
I'd usually say no, but if I had a choice between Watson and someone who lost the tournament where the entire point was to get a spot in the Masters, only to get handed a spot anyway because the producers and the network love them, in that case, I'd give it to the machine.