Nostr Web Client

This is super interesting, would love to know more

It’s a lot of hacking about at the moment and reading, as it’s quite an ambitious project for me.

It’s very reliant on the GPT-4 model at the moment, but I’m hoping to substitute with more powerful LLM’s asap.

One major aspect for @jack to consider is that GitHub repo monopoly is the bedrock for GPT-4’s monopoly on AI coding ability.

To have an open source LLM that can surpass GPT4 code abilities we really require a large open source code repo that can be easily structured as training data.

Without that, OpenAI’s LLMs are going to be eclipsed on chat yes, but will be stubbornly superior for coding.

Of course we now have a massive code generation engine in GPT4, so it’s likely they sow their own downfall, we just need to direct the output of that engine into a suitable open source location with inherent training data schema.

Everything else is open source.

I think its possible to have a USB with less than 80kb on it, that can iPXE boot ipxe.org from the web into a RAM only linux instance slax.org

From there it can download self assembling code, and begin downloading and installing db libraries and asking for API keys. Then map environment, complete the stand up process and then it will ask for a mission.

It’s ethereal for security reasons because it has full permissions and can write whatever it sees fit.

It will break down a prompt into token limit workscopes develop the scope and break it down again until it as able to code functionality.

The splitting up of the prompt into workscope chunks and the further splitting followed by the tracking and reassembly of the full system is the most difficult part of this. But if human orgs can do this, so can LLM’s.

Reply to this note

Please Login to reply.

Discussion

average_bitcoiner 2y ago

And I’m over here just trying to LoRA one model onto a codebase so I can ask it questions while I develop.

You’re sounding way more advanced.

symbsrcool 2y ago

It sounds like you’ve been able to get autogpt like systems to do cool things… I haven’t gotten them to do anything cool yet.

Or is what you’re suggesting not related to GPT agents?

symbsrcool 2y ago

Oops meant to reply all, #[2] see above

symbsrcool 2y ago

Wow I can’t use Nostr apparently, Nevermind I confused myself lol

⚡️🌱🌙 2y ago

I think it’s wrong to think of GPT as an entity, this anthropomorphic stuff is unhelpful and confuses people.

People think of AI models as robots or bots, but these are just the UX.

It’s an API.

What I am doing is using the API to scan and map a development environment and to then inject bootstrapping tools into that environment and to then self assemble something completely new and novel inside that environment.

I thing it’s vastly more powerful than an agent, because it’s knows its CPU temp and it knows what I/O peripherals are attached.

It can utilise its full environment for running the code it creates.

symbsrcool 2y ago

I gotcha 🤙

In the AI literature they use the term ‘agent’ to describe something that takes actions in an environment. Autogpt like systems can execute terminal commands, call external APIs, etc, so it’s fair to say they are agents in the AI academic sense.

Have you seen autogpt (https://github.com/Significant-Gravitas/Auto-GPT ) and related systems? If you haven’t, they basically make a call to a GPT API that breaks down an initial task/goal (from the user) and stores each step in a database, then recursively calls gpt apis to attempt to solve each sub task and evaluate if it was solved. If it wasn’t solved, it further creates more tasks, stores in the database, and repeats. I tried some sample tasks with OpenAI gpt3.5 and it was pretty terrible at doing anything. Tried to clone git repos that didn’t exist, etc. I’m waiting on an API Key to try GPT4.

I’d be super interested in how your planning to handle the sequence of actions and the memory of the system. Would it be a hardcoded sequence of steps to do everything you want or would you have a system/agent that can do planning?

⚡️🌱🌙 2y ago

Ah right, then I guess this is an agent.

I have used AutoGPT, but I found it really tricky to use it for anything. It’s little more than a proof of concept.

I have API keys for GPT4 and a bunch of other OpenAI models and am on waiting list for GPT4-32k.

My “agent” downloads, installs and initialises a Neo4j db then scans the environment fairly extensively and stores that info in db. There are 15 scans for various things ranging from hardware to firewall settings.

It scans and records in the db.

But it isn’t intended to execute real world tasks as an agent.

It is intended that the agent self assembles a bespoke web app / program and it is the web app / program that performs the task. The self assembler maxes out its environment constraints without colliding with limits.

My token limit is 8,192 and it’s important to assess the token cost of a task.

The trick is to assess outputs for fidelity, where 1.000 fidelity is full functional. At the outset the fidelity is 0.000 and as you break tasks down into workpacks and complete them the fidelity slowly rises until it either flattens out or hits 1.000 sometimes it has to junk some stuff and start over and the fidelity will take a hit.

You engage it via a web UI, which it creates itself, and you can use the Neo4j visuals to see the code blocks and code interfaces it builds in real time, and there’s a fidelity line chart.

You can pause and reinstruct, and it the process has a few preset hold points too, to check things are aligned.

It has full sudo permissions, which makes it rather dangerous. It can fail hard.

But the idea is, I can plug a USB into any PC and boot a brand new clean instance and invoke it to self assemble some very powerful bespoke tool and then just wipe it.

It’s a very different kind of computing.

symbsrcool 2y ago

Man thank you for taking the time to explain.

Does the role that GPT plays is to write a custom piece of software and do other things from the human user input from the web browser, after the os and base infrastructure is built up? Ultimately I’m wondering if you really even need GPT?

Side note - just saw this and though you might be interested:

https://arxiv.org/abs/2305.10601

⚡️🌱🌙 2y ago

The token limit is the thing that really constrains GPT4. It can’t handle a very complex thing in a lot of detail.

I think of it like this, GPT4 can…

Can handle 100% of scope at 0.100 fidelity

Can handle 20% of scope at 0.250 fidelity

Can handle 4% of scope at 0.500 fidelity

Can handle 1% of scope at 0.750 fidelity

Can handle 0.1% of scope at 0.950 fidelity

Figures just example, but each mission is different and non linear.

Complexity = f(scope, fidelity)

Token limit is akin to a complexity limit.

You have to create a structure that looks at top level scope and breaks it down into a number of next level down workpacks (of smaller scope). I do this by trying to keep the complexity about the same, but reducing the scope and trying to advance the fidelity.

If I ever get all this working, I’ll probably publish it.

The QA process is quite different.

symbsrcool 2y ago

I went back to your original post and found this paragraph:

“It means you can plug a USB into any computer turn it off & on again, and it will boot from the internet and load an OS without touching the hard disk. You can then instruct it via chatbox to scan the encrypted, build an assembly db and go on to build any kind of application and it will break the workflow down into subtasks of suitable token size and begin to execute the assembly, writing blocks of code, creating files and updating the assembly db. It will pause at various points to engage with the user for confirmation / feedback.”

If I’m understanding you completely it sounds like you’re saying that you need the flexibility of something like GPT4 to:

1. Handle many different hardware environments because you never know which type of computer you’re plugging the USB into

2. Build a custom piece of software from natural language instruction

⚡️🌱🌙 2y ago

It’s number 2. really.

When you ipxe boot from usb, it will ask for a URL of the ISO this URL can be on the internet.

So the USB really have almost nothing on it. Just an ipxe bootloader.

You can then load the OS on the USB flash without touching the PC’s disk. You parasitically use the PC’s RAM, CPU, etc.

Once this boot is up and running you can then install the self-assembler, which is a process that uses GPT4 (until open source catches up), to build a bespoke program for whatever task you are doing.

The self assembler scans its environment and builds a db to describe it. It can scan whatever it thinks it has to and store it.

If you want to store some persistent data through reboots you can put it on the web somewhere.

It’s a very different way to do computing.

I think this is closer to how fully autonomous machines will work. Put some hardware together and then inject some self assembling code.

Thing figures out what it is, then what it has to do, then how to do it.