Put your math hats on for a minute, and let’s check out what this beef from mid-October was about. It’s an ideal instance of what’s fallacious with AI proper now.
Bubeck was excited that GPT-5 appeared to have by some means solved quite a few puzzles referred to as Erdős issues.
Paul Erdős, some of the prolific mathematicians of the twentieth century, left behind a whole bunch of puzzles when he died. To assist hold observe of which of them have been solved, Thomas Bloom, a mathematician on the College of Manchester, UK, arrange erdosproblems.com, which lists greater than 1,100 issues and notes that round 430 of them include options.
When Bubeck celebrated GPT-5’s breakthrough, Bloom was fast to name him out. “This can be a dramatic misrepresentation,” he wrote on X. Bloom defined that an issue isn’t essentially unsolved if this web site doesn’t listing an answer. That merely means Bloom wasn’t conscious of 1. There are thousands and thousands of arithmetic papers on the market, and no person has learn all of them. However GPT-5 in all probability has.
It turned out that as an alternative of developing with new options to 10 unsolved issues, GPT-5 had scoured the web for 10 present options that Bloom hadn’t seen earlier than. Oops!
There are two takeaways right here. One is that breathless claims about massive breakthroughs shouldn’t be made through social media: Much less knee jerk and extra intestine examine.
The second is that GPT-5’s capacity to search out references to earlier work that Bloom wasn’t conscious of can be superb. The hype overshadowed one thing that ought to have been fairly cool in itself.
Mathematicians are very all in favour of utilizing LLMs to trawl via huge numbers of present outcomes, François Charton, a analysis scientist who research the applying of LLMs to arithmetic on the AI startup Axiom Math, instructed me once I talked to him about this Erdős gotcha.
