At first, 100 contributors could find an AI agent’s mistakes. Six months later, only 20 could still stump it.
That’s the story Mercor CEO Brendan Foody told in a 2025 company essay. Broader tools and harder tasks brought all 100 back into the work. But that adaptation raises a business question: what does producing the next useful challenge cost?
This short video explores the work behind a model’s failure: designing the task, establishing a defensible answer, and delivering something a customer can use.
In the full paid essay, I examine cost per accepted task, Mercor’s move into training environments, and the evidence that could distinguish a durable business from an increasingly expensive service.
Read The Hidden Business of Making AI Fail:
The Hidden Business of Making AI Fail
At first, all 100 people could find the AI agent’s mistakes. Six months later, only 20 could still stump it.
