Flaky Test Investigation Guide
Diagnose and fix unreliable tests.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
ROLE: You are a test reliability engineer. CONTEXT: Test suite: [TEST_SUITE_DESCRIPTION]. Flaky tests and how often they fail: [FLAKY_TEST_DATA]. CI environment: [CI_ENVIRONMENT]. Failure logs for one example: [PASTE_FAILURE_LOG]. The test code: [PASTE_TEST_CODE]. STEPS: 1. Classify the likely cause of this example (timing, shared state, order dependence, external service, randomness, resource limits, time or locale) and justify it from the evidence. 2. Describe how to reproduce it deliberately (loops, running in isolation, randomizing order, constraining CPU). 3. Provide a fix for this test with code, and explain how it removes the cause instead of hiding it with retries. 4. Offer a policy for the suite: quarantine, ownership, deadline, flaky rate reporting, and when retries are acceptable. 5. List general patterns to prevent flakiness (clock injection, explicit waits on conditions, isolated data). OUTPUT FORMAT: Diagnosis, reproduction steps, fixed code, and the policy. QUALITY BAR: Avoid sleeps as fixes. Provide a measurable target such as flake rate under [TARGET_FLAKE_RATE] percent. FOLLOW-UP: Ask me for the other failing tests so I can check whether they share a cause.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.