Do I recall correctly that you said you always read the last paragraph of the book before starting it?

yeah 😔

AI needs to learn how to communicate its error margin. It’s ok to leave room for doubt, it will actually gain more trust by doing that.

could it be they didnt feed the AI the actual book, only its 'reviews'

good movie

That’s an inherit flaw in how LLMs work. I noticed this effect very early on. This is why I think it’s ridiculous to use LLMs to summarize. It doesn’t fundamentally understand what it “read” and it doesn’t know how to tell you how it came up with the response, because it’s simply a statistical guess.

The less focused space of ideas you're letting it draw from, the more you're working with stochastic intuition. The knowledge focused worlflow deal with chunking informaition to place text into the context window, which helps a lot

The model’s performance on summarization is going to depend most of all on how popular the book is. Remember, in this context you’re relying on the model’s memorization of the book, which only really works if there’s a lot of stuff written about it. This also means the book can’t be new.

TLDR: Just use a long context model like Gemini and put the whole book into your prompt when asking it to summarize instead of trusting the LLM to remember.

Wow. I really shouldn’t use AI for information.

You have learn that AI have "temperature" and "hallucination" like humans have in some ways with some defective memory.

https://mitsloanedtech.mit.edu/ai/basics/glossary/

Finally AI is growing with the human default too, but i am sure it will do better soon.

If you ask for the same summary to a group of human reader of your book you will not have the same sentences.

And some will probably omit, or add things in the text that are not initially in the book.

And this would be amplified if there is more time between the reading moment and the summarize, because of memory weakness.

The very interesting point about AI now, is the self questioning it generate about us.

Are we really so clever or are we adapting ourself to everything our brain is saving from every information it process. Just like AI do in his training step.

I had, in a previous life some interest in AI languages. Scientists tried to clone the physical human behavior (neurons and synapses). It just failed because human brain was much more complicated than 0 or 1 state.

But nowadays the approach is different, we don't try to copy how the brain works physically, but just his behavior.

In a way we try to optimize the result, with a different source.

And i think it is a better approach because it is more adapted to the computers design.

That's why i think AI will, for sure, find a way to code a better, optimized and adapted way than we do today.

because we just code a way that is comprehensive for us, which is certainly not the most adapted to computers hardware.

Sorry for this long text, but your note was inspiring.

Thank you for your share.

#thinkstr

LLMs are pathological liars. In such humans a thing is broken that isn’t even invented for LLMs yet.

This would be humbling for that cocky Anthropic CEO to read

This is standard in my experience. People are far too trusting of AI answers. The worst thing is that it will give made up answers with complete confidence where a human would just admit they don't know. Yet another indicator that there is no real intelligences here just glorified auto-complete.

Craig Wright got absolutely rekt submitting AI generated documents, because he never bothered to check if the case law citations were real or not.

That movie was so much better to see 10 years after it came out than it was at the time.

One could ask the LLM to rate its own confidence in the provided answer as to get a better picture of the uncertainty

LLMs are currently a bunch of bullshiters!

I once asked chatgpt to summarize study results on longevity vs bowel movement frequency and that went down the toilet pretty quick. AI was getting pretty freakin' confused on 3 shits a day versus one shit every 3 days. Hiiiuuuuge Difference! And chatgpt was literally like oh yeah you're right my bad.

💯 Had the exact same convo, several times 🤷‍♂️

Wasting energy to cover their narrative that AI is the one needing energy, it's all about ₿itcoin....they just can't admit it yet

Burn After Reading is a very underrated comedy. John Malkovich’s character one of the best.

I had a similar experience with an "artificial intelligence" (maybe "simulated" intelligence would be more apropriate). I asked it a question a found, that the answer was incorrect. I corrected it, and it stated "my bad, sorry". And I repeated it again, with the same results.

For me, this was an important lesson, as not to trust the answers I get, as long as I don't verify them myself. Huh, don't trust, verify! Who would have thunk...

Yep, same.

🤦‍♂️😓 There is still a long way to go to be able to say that the AI is complete. On the one hand I feel to say thank goodness 🤷‍♂️😉

AI can't say I don't know. On the contrary, when answers always seems to be 100% sure.

Could you mention the AI you used please?

I had that exact same experience asking some AI model about some Dune story plots. I know that book pretty well. The AI answer was pure BS, pure hallucination. But I don't remember which model I used at that time.

But I see that it's all very much model-dependent, and prompt-dependent.

ChatGPT told me I was smart

nostr:nevent1qqsxejqlcsgu4ekw0mjuxzmjfqqetxvvdauadsn0p9u4evp6zqstyrgpzemhxue69uhhyetvv9ujuurjd9kkzmpwdejhgq3qa2cww4kn9wqte4ry70vyfwqyqvpswksna27rtxd8vty6c74era8sxpqqqqqqzkxweln

Not surprising based on my experience. I've seen it claim certain stock tickers don't exist so I then provided it with a link to it and it claimed that link was empty. This was with perplexity but they all do it. LLMs aren't real intelligence.

Yep now AI is lying … very human. It is “paid “ to fill in the gaps in a narrative like a human does so the narrative makes sense. Truth is what people believe it to be.