Global Feed Post Login
Replying to Avatar vnprc

Currently, LLMs don't handle long conversations well. They operate on a sliding window of the most recent inputs. More advanced LLMs have some memory storage but it's not really clear when or how to trigger memory updates.

If you try to have a long conversation it will get stuck in a loop because it forgets the stuff you already resolved and starts introducing regressions.

A good strategy is to limit your conversation length. This forces you to chunk up your work into bite size pieces, which is always a good idea. For me, I like to set a goal for each conversation. Maybe I need to decide the best architectural change to accomplish some development goal, or the best library to use, or implement one small feature in isolation.

When I complete a task, if the next task builds on the last (it usually does) I ask the LLM to summarize our convo and I use the summary to kick off the next discussion. Getting a lot of mileage out of this technique.

Avatar
Sasker 9mo ago

This sounds clever. We need far more sophisticated dynamic ways of managing context data. Both context of current conversation, other convos and source files. You would even want the same system to manage functions next to context.

Reply to this note

Please Login to reply.

Discussion

No replies yet.