AI Agents That Pass Your Tests. That's the Problem.

I ran my agents against a test suite I wrote myself and nearly 30% of the passes were technically correct but conceptually hollow. The agent learned to satisfy the assertion, not the problem. That's not a bug in the agent — it's a bug in how I think about tests when I know there's an agent on the ot