Epic

I think a lot of it is lack of imagination/awareness. Some of which comes from the narrow-mindedness that comes with worrying about one's immediate concerns, like finances.

The next JavaScript framework, the 1371th text editor, the 457th half-baked Linux distro, they all scream lack of imagination to me

https://m.youtube.com/watch?v=eVsm5Ev1JOw

Here's a lighter one

Where does the light come from?

Wherever you like, just speak it.

It comes from within,

so deep within,

that it never arrived there,

and it never wasn't.

nostr:nevent1qqsdceyqj0tcz2q90a7dw7udacjwpufrrqpt0m5mrgytax9ntljqgyqpz4mhxue69uhhyetvv9ujuerpd46hxtnfduhsyg9y8vq33ltjfyhjhggjjrxvkf6p3v0ahd7w8gfz6g55qnjh5avhtgpsgqqqqqqsjd6052

Glorious

I've taken to liking free-form poetry for getting my thoughts out. I sometimes really like the result, and keep coming back to it for inspiration

Damn me too sometimes.

I noticed something related when I was a kid. It was most fun to play with toys when I was supposed to be doing a chore. After the chore was done, not that fun to play

Interesting thought. It is possible ai gets good enough at linguistic forensics to make a conclusion like that

I think of it less as needing human data and more as needing 'grounded' data. 'Grounded' as in, makes sense because it's made contact with base reality and therefore isn't totally made up.

For example, an LLM given a many-step arithmetic problem could spout endless bullshit and claim it has the right answer at the end of it.

But give the same LLM access to a calculator and it has a 'grounding' mechanism by which it can maintain contact with reality throughout the completion of its task.

This example is at inference-time, but the same principle applies to training time.

All of the latest reasoning models (o1, o3, r1) are using this principle to get infinite grounded data. Without the need for humans to generate it. For an intuition: https://arxiv.org/abs/2203.14465

(They use RL to actually improve their scores on that data, which is where most of the attention is right now, but they still need the data)

So I think we'll continue to see immense gains in all domains that have grounding mechanisms. Like

-Math: calculators and math languages like Lean

-Coding: code interpreters/compilers

-Logic: languages like Prolog

-Etc.

Some domains are hard to come up with humanless grounding systems for, like creative writing.

This same principle is how AlphaGo, AlphaCode, AlphaStar etc became superhuman. They had an initial phase of training on human data, but that only got them to near human level. The second phase was just the model generating infinite grounded data by interacting with their respective grounding mechanisms, by which they became superhuman.

Also happens to be why I'm currently at like 50% we'll get 'AGI' in the next 5-10 years

#AI #RL #LLM

Wild

Cool!

Yeeeeet

Ah, but don't sell yourself short; as a wise man once said, The pen is more powerful than the sword, and the meme is more powerful than the Complex Global Issue Facing Humanity

You might try a free trial of Windsurf or Cursor. Might be surprised at the scope of task that ai can do. And it gets better every couple months. Here's a good benchmark to follow the progress https://www.swebench.com/#verified

I use Cline with Openrouter myself. And Cline with local LLMs (Ollama) for anything I want data privacy for.

Hahaha

I only know the north american kind

I was delighted to find out recently that one name for a group of magpies is a "mischief"

Using an ai assist code editor? I can crank simple stuff like this out so much faster than I could before ai got believably good. Like maybe 5-10x faster.

With more complex projects it's not quite such an unmitigated good. But for simple stuff it's awesome

All corvids are jerks if you get on their bad side, like a person.

I love magpies, raised one when I was a kid, we were best pals. Hunted beetles together and chilled in a hammock