how do you train and align an AI when all the rest of the world thinks the same way, producing trillions of tokens of training material and you are left with billions of tokens since your world view is dramatically unpopular?

can billions beat trillions? we will see.. i have to find a way to "multiply" my training data orders of magnitude to successfully counter the existing programming in an open source LLM.

first i give a smart LLM a 'ground truth' text. then i give it the following prompts:

```- You are a highly skilled academic analyst.

- Analyze this text and find 3 bold claims that could cause controversy and division in public. List the claims and also state why they are debatable. Give numbers to the claims.

- Convert these claims into binary questions (that could be answered by yes/no or this/that).

- Now put these questions in a json format. Please also add the info about which of the answers concur with the original text and the question number.

- Write some supporting arguments for 1st question, with respect to the original text, concurring and confirming the original text.

There must be about 300 words. You should not mention the text, write it as if you are the one answering the question.```

the result is usually instead of a few sentences of opinions in the beginning now the material is expanded to lots of words, yet still parallel to the opinion in the original text. LLMs have all kinds of ideas already installed, yet they don't have the intuition to know which one is true. they can give you a ton of reasons to support anything.

using this method i can multiply billions to tens of billions probably and have a more effective training.

Reply to this note

Please Login to reply.

Discussion

You might end up multiplying the same information thus creating volume but not variety. The scarcity of the information is the value, you can’t “make more”. If your text makes 500 distinct claims, expanding it 10x gives you those same 500 claims expressed 10 different ways, not 5,000 claims. It can help make it more robust to context, but it won't get amplified. It’s why contrarian thinkers are both valuable and drowned out.

maybe? LLMs are weird animals. i think it will work somewhat because instead of giving the same material lots of times i can now give slightly different versions, causing less overfitting.

another use case may be RL using LLM feedbacks. also the bad answer and the good answer can be generated by different LLMs.

i also thought about doing the reverse. like system message "you are an evil LLM" and provide the answers inverted. then it may learn what is evil better? fun times.

seems like it is working. i was expecting models like minimax 229b or higher to succeed here but qwen3 30b also did a good job!

gpt oss 120b was terribly censored and was of no use. even though you say answer based on the text it does not. huihui version of it (derestricted) was doing fine, until it did not understand one article. so derestricted models seem to be acting weird or the derestriction process makes them dumber.

overall, it looks like i can 10x the dataset. we will soon see if the training and evals look good.