I don't mean "tokens alone"
Discussion
Appreciate the clarification attempts. But to be fair, this all started with a confident claim that “more tokens” would eventually get us there — not loops, not memory, not scaffolding — just “tokens,” full stop. That’s not a strawman; it’s quoted:
“It’s very likely that given ‘more tokens’ in the abstract sense, current AI would eventually settle on the correct answer.”
— Posted July 22, 2025 · 12:27 PM
And in case that was too subtle, a few days earlier:
“Use more tokens, kids.”
— ynniv · 4d ago
This was in direct reply to:
“You’re not scaling reasoning, you’re scaling cache size.”
— Itamar Peretz · July 20, 2025 · 08:37 AM
If your view has since changed to “I don’t mean tokens alone” (July 24, 2025 · 1:10 PM), that’s totally fair — we all evolve our thinking. But that’s not what was argued initially. And if we’re now rewriting the premise retroactively, let’s just acknowledge that clearly.
So here’s the fulcrum:
Do you still believe that scaling token count alone (in the abstract) leads current LLMs to the correct answer, regardless of architectural constraints like stateless inference, lack of global memory, or control flow?
• If yes, then respectfully, that contradicts how transformers actually work. You’re scaling width, not depth.
• If no, then we’re in agreement — and the original claim unravels on its own.
In either case, worth remembering: you can’t scale humans. And that’s still what fills the reasoning gaps in these loops.
I don't believe, and never have, that scaling context size alone will accomplish anything. I do believe, and always have, that people give up too early. I'm not sure why you're fixated on "winning" this argument – it's not an argument per se, and there are better things to do right now
I’m not fixated on “winning,” and certainly not looking to drag this out. But if we’re walking back, let’s be honest about what’s being walked.
“Use more tokens, kids.”
— ynniv · 4d ago
“It’s very likely that given ‘more tokens’ in the abstract sense, current AI would eventually settle on the correct answer.”
— July 22, 2025 · 12:27 PM
“I don’t mean ‘tokens alone.’”
— July 24, 2025 · 1:10 PM
“I don’t believe, and never have, that scaling context size alone will accomplish anything.”
— July 24, 2025 · 7:53 PM
If the position was never “tokens alone,” I don’t know what to do with these earlier posts.
So I’ll ask one last time, gently:
Was “more tokens = eventual convergence” a rhetorical device, or a belief you now revise?
We probably both agree that scaling context is not equivalent to scaling reasoning and that transformers aren’t recursive, stateful, or inherently compositional.
I was only pointing out that. If we’re aligned now, we can close the loop.
That’s a great blog post — I actually like it.
But let’s not mistake narrative for argument. I’m not disputing that experimentation, iteration, and persistence can lead to real progress. In fact, I’d argue that’s precisely why it’s worth being clear on what is being tried.
My only point is that your original phrasing clearly emphasized tokens:
“Use more tokens, kids.”
“Given enough tokens… current AI would eventually settle on the correct answer.”
Then later, you clarified:
“I don’t mean ‘tokens alone’.”
If that was always your intent — that architectural context (loops, agents, structure) matters more than just throwing tokens — I think we’re in violent agreement.
But let’s not retroactively apply that nuance to the initial bold claim unless that was the design all along.
Persistence is valuable, yes. But clarity helps the rest of us persist in the right direction.