p3x.deAdvanced AI Suffers ‘Complete Accuracy Collapse’ in Face of Complex Problems, Study Finds
ThefuzzyFurryComradewww.theguardian.com/technology/2025/jun/09/appl…
16 Comments
Comments from other communities
Yes. I thought this was rather obvious to anyone that ever tried to use llms for complex tasks. It's not there yet, LLMs by themselves most likely will never be.
Rhaedas“Anybody who thinks LLMs are a direct route to the sort [of] AGI that could fundamentally transform society for the good is kidding themselves.”
A quote in an article that still uses the generic "AI" to refer to LLM models, thus losing any credibility. Probably was written by an LLM - sorry, AI, since that's what it means now. AI is popular jargon now to mean anything that seems like it's thinking, only serious people use AGI/ASI and even they often slip up and say AI sometimes. Tainted word.
I do think LLMs are/will be part of the tools needed for AGI, but alone, no, they aren't processing what they're being asked, so of course on anything more complex than their training they can go astray.
it hallucinates with easy problems as well.
I’ll just copy / pasta my comment from a different AI article from earlier today:
It’s about time that we call the hype machine what it is. Ed Zitron has been calling this out for more than a year in his newsletter and on his podcast. These charlatans pretend we’re on the edge of thinking machines. Bullshit. They are statistical word generators. Can they be made to be useful beyond that? It appears so[0], but useful other things are not available to be mass-adopted so far. Curing cancer certainly doesn’t appear to be near.
Totally. The internet is a fad, too.
There's two kinds of boosters of the LLMbecile grift: the grifters and the patsies.
Which one of the two are you?
Oh my lord, you ding dong. Try thinking for yourself once in awhile.
Ah. The patsy. Got it.
You e got nothing but baby brained insults.
You are a little fool. Good luck.
Just like NFTs.
NFT's are useless crap. AI has lots of uses.
What I see is lots of folks who are angry with cut-throat, late stage Capitalism that exploits AI to hoarde wealth and fuck over the rest of us, not AI itself, which is just a tool.
But you all are so quick to circle jerk AI hate w/o taking a moment to step back & see the root problem.
It's goddamn embarrassing.
LLM based chatbots have a lot of well documented shortfalls, and more generally, what is being promised is not what is being observed. If you also consider that none of the big players in the AI space are offering their product at a price where they could ever hope to break even, it's reasonable to assume that a market correction is incoming.
"Study finds"
How about "Anyone who knows anything about anything and actually tried to get an LLM to do anything with any level of complexity"?
No, advanced AI is more accurate than humans at identifying the things it's specialized in and it's an invaluable tool in many domains. LLMs, on the other hand, are not made to be accurate in anything in particular, and anyone expecting them to be just doesn't understand how they work.
Well, then just call it what it is, linear regression.
It is fully possible, quite likely even, for models to both be "more accurate than humans" on average while at the same time suffer occasional "accuracy collapses".
humans burn out, too
Not the same. At all.