killcheck
A Python mutation-testing harness that checks whether the tests an AI agent writes actually catch bugs, built for the micro1 Frontier Engineering Challenge 2026.
Open Source
AI Evaluation
Sole Developer
Python, pytest, mutation testing
killcheck plants one mutant at each mutable site in a library. Mutants the existing tests miss become the agent's work queue, and a new test is kept only if it passes on the clean code and fails on its mutant. Exit codes are the ground truth, not the agent's own report.
An evaluation is only worth something if its numbers can't drift. I froze the measurement core, committed to the metric before running it, and logged every retraction. Across 455 mutants in 12 libraries, only 53 turned out to be reachable but unasserted.
Public on GitHub with a full write-up on dev.to, including the corrections readers found after publication.

