90% of real work still escapes benchmarks

WHAT HAPPENED

Mercor matched benchmarks against real tasks in the economy

The study mapped 29 benchmarks against 923 occupations and 18,797 tasks from O*NET. The main takeaway: coverage of economically valuable work is still limited, which makes it hard to know how good AI is at real work.

THE NUMBER

90%

of working hours have little or no coverage

That is the source’s estimate for tasks with little or no presence in benchmarks

THE REAL PROBLEM IN COMPANIES

What I see in companies when they evaluate AI

1. 01They test on a benchmark, but not in the team’s real workflow
2. 02They launch without proving value before heavy integration
3. 03They do not measure adoption and impact after it goes live

I HAVE ALREADY SOLVED THIS

AskMoses.AI, the SaaS I built end to end and operate as a contractor

1. 0117 organizations in the US signed, and 6 actually use it
2. 0211 of 17 never activated: onboarding asked for CRM and rubric before value
3. 03Script Intelligence generates a weekly script with real phrases, with a minimum of 3 calls

MY TAKE

Launching is not the same as being used.

If the benchmark does not cover the user’s real work, the company can confuse technical capability with adoption. In practice, ROI depends less on a test score and more on fitting AI into the workflow, delivering value early, and measuring usage and impact after launch.