LLM translation is still far off but would be great… the quality of translations still are suboptimal with the big clouds, so on device, especially for some languages, will need time to mature
also, I feel like finetuned models designed for content curation and summarization will be the way. the more specific the tool the more accurate it is for a task.
One things they are really good at today, even in the 2gb 6b param models, is summarization. I want to put together a demo on notedeck where it downloads the model and uses it to summarize threads when you open a large one.
prompt injection is still a big deal sadly… there’s actually dedicated models for summarization out there.
you could fine tune existing summarization models to take additional context input like reply chains via tokens
