Unbundling AI

This analysis explores the product challenges of Large Language Models (LLMs) like ChatGPT, which promise to be universal tools that can handle any query but face significant usability obstacles. The key problem is that while LLMs solved the "output" challenge that plagued voice assistants like Alexa (which could understand anything but only answer pre-programmed queries), they introduced new issues around accuracy and user interface design.

The core challenge is twofold: a "science problem" where LLMs provide statistically probable rather than definitively correct answers (leading to hallucinations), and a "product problem" where natural language interfaces may not be optimal for complex tasks. Unlike traditional software like Excel or Photoshop that offer iterative, visual editing processes, LLMs operate as "black boxes" where users input prompts and receive outputs without transparency or granular control over the process.

Key takeaways for product managers include: 1) Design interfaces that communicate uncertainty rather than false confidence (avoiding the "three paragraphs with apparent certainty" problem), 2) Recognize that natural language isn't always the best interface even for AI systems, 3) Focus on use cases where statistical accuracy is acceptable rather than binary correctness, and 4) Consider how to transform the current "Battleship-like" user experience into more iterative, process-oriented workflows. The technology works best as an "infinite intern" providing first drafts that require human oversight rather than final answers.