Plenty of people have opinions [on wether there is a road towards LLMs that are "right"], but so far, we don’t know, and for the time being, ‘error rates’ (if that’s even the right way to think about this) are not a gap that will get closed with a bit more engineering, the way the iPhone got copy/paste or dialup was replaced by broadband: as far as we know, they are a fundamental property of the technology.
Great essay on how to think about LLMs and how to best adopt them. This quote points to something fundamental: Many of the challenges when implementing LLMs can not be solved by better engineering, since the probabilistic behavior is part of what an LLM is.
For the current hype on agentic AI, Evan’s highlights a core question: Should you try to get the LLMs behave more deterministic, or should you give them access to deterministic tools?
Visit the source: ben-evans.com