😅
#AI #GenerativeAI #AIAgents #Anthropic: "Anthropic says that Claude outperforms other AI agents on several key benchmarks including SWE-bench, which measures an agent's software development skills and OSWorld, which gauges an agent's capacity to use a computer operating system. The claims have yet to be independently verified. Anthropic says Claude performs tasks in OSWorld correctly 14.9 percent of the time. This is well below humans, who generally score around 75 percent, but considerably higher than the current best agents, including OpenAI’s GPT-4, which succeed roughly 7.7 percent of the time.
Anthropic claims that several companies are already testing the agentic version of Claude. This includes Canva, which is using it to automate design and editing tasks and Replit, which uses the model for coding chores. Other early users include The Browser Company, Asana and Notion.
Ofir Press, a postdoctoral researcher at Princeton University who helped develop SWE-bench, says that agentic AI tends to lack the ability to plan far ahead and often struggle to recover from errors. “In order to show them to be useful we must obtain strong performance on tough and realistic benchmarks,” he says, like reliably planning a wide range of trips for a user and booking all the necessary tickets."