Flaky Tests, Found and Fixed
Ask an agent to investigate unstable tests, propose fixes, and report unresolved failures.
Flaky Tests Teach the Team to Ignore Red Builds
A test that fails one run in five gets retried, then ignored, then skipped. Every flaky test costs a rerun and a little trust in the suite, and fixing one means reproducing a failure that doesn't happen on demand. It's rarely anyone's priority.
Delegate Flaky Test Investigation
Schedule an agent to run unstable tests and investigate whether each failure comes from the product, the test, or the environment. Ask for a pull request when it finds a test fix, and a report when it cannot establish the cause.
auth › refreshes session
Cause: race condition in test setup
Common Questions
How does it find flaky tests?
Ask it to run the tests you identify, repeat them as needed, and classify failures as product, test, environment, or expected-change issues.
Will it just disable a flaky test?
Instruct it not to disable tests. Review any proposed change and require evidence that the test still checks the intended behavior.
Can it run on a schedule?
Yes. Choose a recurring cron schedule, such as nightly or weekly, or start it by hand.
Which models can it use?
Any major provider, plus local and private models. Bring your own keys or subscription, and there's no markup on tokens.
Related: Failed build triage, Quality assurance docs.
Keep Your Attention for the Hard Problems
Start with one repo and one recurring task.