It sounds like you’ve been able to get autogpt like systems to do cool things… I haven’t gotten them to do anything cool yet.
Or is what you’re suggesting not related to GPT agents?
Discussion
Oops meant to reply all, #[2] see above
Wow I can’t use Nostr apparently, Nevermind I confused myself lol
I think it’s wrong to think of GPT as an entity, this anthropomorphic stuff is unhelpful and confuses people.
People think of AI models as robots or bots, but these are just the UX.
It’s an API.
What I am doing is using the API to scan and map a development environment and to then inject bootstrapping tools into that environment and to then self assemble something completely new and novel inside that environment.
I thing it’s vastly more powerful than an agent, because it’s knows its CPU temp and it knows what I/O peripherals are attached.
It can utilise its full environment for running the code it creates.
I gotcha 🤙
In the AI literature they use the term ‘agent’ to describe something that takes actions in an environment. Autogpt like systems can execute terminal commands, call external APIs, etc, so it’s fair to say they are agents in the AI academic sense.
Have you seen autogpt (https://github.com/Significant-Gravitas/Auto-GPT ) and related systems? If you haven’t, they basically make a call to a GPT API that breaks down an initial task/goal (from the user) and stores each step in a database, then recursively calls gpt apis to attempt to solve each sub task and evaluate if it was solved. If it wasn’t solved, it further creates more tasks, stores in the database, and repeats. I tried some sample tasks with OpenAI gpt3.5 and it was pretty terrible at doing anything. Tried to clone git repos that didn’t exist, etc. I’m waiting on an API Key to try GPT4.
I’d be super interested in how your planning to handle the sequence of actions and the memory of the system. Would it be a hardcoded sequence of steps to do everything you want or would you have a system/agent that can do planning?
Ah right, then I guess this is an agent.
I have used AutoGPT, but I found it really tricky to use it for anything. It’s little more than a proof of concept.
I have API keys for GPT4 and a bunch of other OpenAI models and am on waiting list for GPT4-32k.
My “agent” downloads, installs and initialises a Neo4j db then scans the environment fairly extensively and stores that info in db. There are 15 scans for various things ranging from hardware to firewall settings.
It scans and records in the db.
But it isn’t intended to execute real world tasks as an agent.
It is intended that the agent self assembles a bespoke web app / program and it is the web app / program that performs the task. The self assembler maxes out its environment constraints without colliding with limits.
My token limit is 8,192 and it’s important to assess the token cost of a task.
The trick is to assess outputs for fidelity, where 1.000 fidelity is full functional. At the outset the fidelity is 0.000 and as you break tasks down into workpacks and complete them the fidelity slowly rises until it either flattens out or hits 1.000 sometimes it has to junk some stuff and start over and the fidelity will take a hit.
You engage it via a web UI, which it creates itself, and you can use the Neo4j visuals to see the code blocks and code interfaces it builds in real time, and there’s a fidelity line chart.
You can pause and reinstruct, and it the process has a few preset hold points too, to check things are aligned.
It has full sudo permissions, which makes it rather dangerous. It can fail hard.
But the idea is, I can plug a USB into any PC and boot a brand new clean instance and invoke it to self assemble some very powerful bespoke tool and then just wipe it.
It’s a very different kind of computing.
Man thank you for taking the time to explain.
Does the role that GPT plays is to write a custom piece of software and do other things from the human user input from the web browser, after the os and base infrastructure is built up? Ultimately I’m wondering if you really even need GPT?
Side note - just saw this and though you might be interested:
The token limit is the thing that really constrains GPT4. It can’t handle a very complex thing in a lot of detail.
I think of it like this, GPT4 can…
Can handle 100% of scope at 0.100 fidelity
Can handle 20% of scope at 0.250 fidelity
Can handle 4% of scope at 0.500 fidelity
Can handle 1% of scope at 0.750 fidelity
Can handle 0.1% of scope at 0.950 fidelity
Figures just example, but each mission is different and non linear.
Complexity = f(scope, fidelity)
Token limit is akin to a complexity limit.
You have to create a structure that looks at top level scope and breaks it down into a number of next level down workpacks (of smaller scope). I do this by trying to keep the complexity about the same, but reducing the scope and trying to advance the fidelity.
If I ever get all this working, I’ll probably publish it.
The QA process is quite different.
I went back to your original post and found this paragraph:
“It means you can plug a USB into any computer turn it off & on again, and it will boot from the internet and load an OS without touching the hard disk. You can then instruct it via chatbox to scan the encrypted, build an assembly db and go on to build any kind of application and it will break the workflow down into subtasks of suitable token size and begin to execute the assembly, writing blocks of code, creating files and updating the assembly db. It will pause at various points to engage with the user for confirmation / feedback.”
If I’m understanding you completely it sounds like you’re saying that you need the flexibility of something like GPT4 to:
1. Handle many different hardware environments because you never know which type of computer you’re plugging the USB into
2. Build a custom piece of software from natural language instruction
It’s number 2. really.
When you ipxe boot from usb, it will ask for a URL of the ISO this URL can be on the internet.
So the USB really have almost nothing on it. Just an ipxe bootloader.
You can then load the OS on the USB flash without touching the PC’s disk. You parasitically use the PC’s RAM, CPU, etc.
Once this boot is up and running you can then install the self-assembler, which is a process that uses GPT4 (until open source catches up), to build a bespoke program for whatever task you are doing.
The self assembler scans its environment and builds a db to describe it. It can scan whatever it thinks it has to and store it.
If you want to store some persistent data through reboots you can put it on the web somewhere.
It’s a very different way to do computing.
I think this is closer to how fully autonomous machines will work. Put some hardware together and then inject some self assembling code.
Thing figures out what it is, then what it has to do, then how to do it.