Indeed, there's something spooky about it. Still just a complicated calculator, but its a step closer to something more.
Here's an idea. Neurons store some biological version of bits, then connect and relay their bits of memories and some kind of emergent structure forms consciousness on top of that. So simulate a mass of neurons, either virtually or directly on silicon as physical chips, then have a different computer run an LLM but have its outputs be the inputs for the simulated mass of neurons, and then have whatever the neural mass outputs be the new input to the LLM for its next run iteration.
Maybe sentience could emerge from that? Or something closer, and then you'd adjust guardrails and stops with code at the point of exchange.
Idk, best I can do is abstractly imagine a flow. Y'all coders will do the real work, :p