Global Feed Post Login
Replying to Avatar SuiGenerisJohn

Im not an expert but the LLM analyzes the text broken down into little tokens like words or parts of words or letters and then the llms are trained on statistical relationships between the tokens in the data it analyzes. This gives tokens a kind of preference in relation to other tokens (preference is just a word I used not a technical term) so when you use the llm you aren't seeing humanities knowledge youre seeing a statistical representation of humanities mode of expressing its knowledge.

Avatar
SuiGenerisJohn 1y ago

This is probably a piss poor way to explain it cause Im not a computer engineer or statistician but just a way of explaining what I meant.

Reply to this note

Please Login to reply.

Discussion

No replies yet.