Replying to Avatar someone

It is technologically possible to fine tune models to each user's taste but it is costly. Nowadays decent performance comes with 70B models and those would require 140GB GPU. Like $17k computers per person. Not very feasible. And it takes lots of time to train them.

Currently, "system messages" i.e. the first message in a session that describes the assistant (the role of the LLM in the conversation) is more suitable to achieve this. Person A can say "You are the woke assistant that will recommend me the best artists in the world" and Person B can say "Please dont include the most popular artists because they are mainstream and I hate mainstream". Then the LLM will act accordingly.

It is also possible to "instruct" in every message. Like "pretend that you are a great recommendation engine for Jazz and give me some recommendations considering I like these artists".

Using RLHF techniques models and humans will have a symbiotic relationship imo in the future. I mean the concscious humans need to think about raising and training the best AI and dont leave the playground to evil actors..
Avatar
pam 1y ago

this is very useful to know. thank you for sharing!

Reply to this note

Please Login to reply.

Discussion

Avatar
someone 1y ago

But yeah all the big corps are fully aware something big is happening with LLMs and they are moving. Some smaller models will get into devices for sure. I think they will be used for inference only in the beginning (not for training). I.e. Each user will have their models to generate from, not to customize/train because it is slow to train.