Doesn't seem to be true, in fact while many of the major companies claimed you couldn't run it without a super computer, after only a couple of weeks after META's weights leaked, the open source community had Llama running on basic consumer hardware. Even got it running (very slowly) on a raspberry pi.

In addition it seems to becoming apparent that just cramming in waaayyy more params isn't a good way to "learn," instead curated, micro step learnings from a huge open source community that get recombined and grow naturally is far faster and can produce equivalent or even better results with just a few Billion params on cheap hardware than the big guys can do with 100s of B of params and a super computer.

I think we are mistakingly thinking of this new tech like previous giant, centralized platforms, but it's beginning to look very much like the positive feedback of network effects may not apply here. Trying to centralized and lock down your weights may be a death sentence in the not-too-distant future.

I'm specifically starting a podcast because I don't think we have the right perspective on this, and I believe our attempts to control & centralize AI will actually produce the *perfectly opposite* outcome as we are hoping for, as basically all rushed, arrogant (no disrespect, but the idea that we are going to "control" this is our inner statist speaking) decisions and knee jerk reactions tend to be. I don't think the above course of action is being proposed out of knowledge, but because we are afraid. Decisions like that historically tend to produce poor results.

TL;DR - I think AI is too powerful and important to let government "control" it or "pause" what is happening. I think that will spell disaster... plus they have virtually zero chance of succeeding anyway. IMHO

Reply to this note

Please Login to reply.

Discussion

To clarify the "it doesn't have the positive feedback of network effects" comment - I mean this in the sense that currently users get "trapped" in the network owned by a single entity, like the control Facebook has over social communities, and Amazon has over retail. You cant take your friends or connections with you without starting from literally zero and rebuilding your entire social graph, while hoping everyone follows you... which they wont. So Facebook gets to control what we get to see and say.

This problem with AI is less apparent. There is a benefit to using user data to train, and thus more users means more training data. But the ease of combining and building on top of open source weights where all contributions are cumulative and shared... I don't think this plays out like people are thinking.

I love your attitude and perspective towards the state, institutions and government.

The institution is a psychopath.

Never trust, always always always verify, and if you can't, assume a deceptive, manipulative and/or coercive culture with malicious intent.