Apple did the research; LLMs cannot do formal reasoning. Results change by as much as 10% if something as basic as the names change.
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and
Discussion
nostr:npub1f8etj9rx9wyulznv9kevf8wl0nlkr2s8cv5qkufzjh7cm3nh6t3q0yv2mv The "huge" problem is if users believe that AI does do formal reasoning, and maybe that's down to the people promoting it as a substitute for a human brain.
It can be a useful tool only as long as everyone understands its limitations.
#AI #LLMs
No surprise at all.
But isn't the new openai (strawberry) trying to get around this with very slow iterative results?