LLMs feel like they should be good at "find the top 10 products in this space and make me a comparison chart"
Multiple models from multiple companies all puke on this seemingly simple task. Missing top brand products and the specs are a total mess of inaccuracy.
I've tried this task multiple times for different product spaces and it is always a let down.
I just tried this for zero turn mowers and it didn't include a single mower available at Lowes, Home Depot, or Tractor Supply.
LLMs actually feel like they're getting worse at providing reliable answers rather than better.
I agree with this.
I was asking about plants for my garden on 2 different days. First time it highly recommended a plant that stood out to me. A couple of days later it makes a list and leaves it out. Asked why, highly invasive in my area.
Annoyance after annoyance like that is really ruining my faith. I'm not sure if devs are too stupid to spot these issues or if coding is the only thing LLMs are any good at. Every time I ask it to fill in details of something I already sort of know it gets large parts that I know wrong.
They definitely don't seem like very good knowledge databases about most things. Just useful for writing and coding.
The fact that none of them provide any citations is insane and goes agasint everything we learned in school when it comes to researching topics.
Worse than that, ask for citations and get fake citations unless you are very careful how you ask.
