Model Fairness Check (Defensive Audit)
Check a model for uneven performance across groups.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
[VERIFY] is an editorial flag, not a blank to fill. Wherever the prompt shows [VERIFY], check that item against a reliable source before you use it.
Plan a fairness check for my model. Model purpose and decision it informs: [PURPOSE_DECISION]. Groups that could be affected and what data I have about them: [GROUPS_DATA]. Legal context I'm aware of: [LEGAL_CONTEXT]. Define the harms of concern, then plan analyses: performance metrics by group (error rates, false positive and negative rates, calibration), differences with confidence intervals, and small-sample caution. Explain that different fairness definitions can conflict and that choices require input from stakeholders. Provide Python code outline for sliced metrics and a report template. Recommend documenting limits and involving affected people. Mark legal matters [VERIFY] for [JURISDICTION] and recommend a qualified adviser; this is not legal advice. Add a worked example with invented group metrics showing how to compute and present false positive and false negative rates with intervals, and a list of questions to put to stakeholders before choosing which disparity matters most. Include a note on small groups and the limits of what can be concluded. Add a worked example with invented group metrics: group A with 4,200 cases and a false positive rate of 8.1 percent, group B with 380 cases and 12.6 percent. Show how to compute the rates, a confidence interval for each (state the method), the difference and its interval, and the conclusion permitted by a small group (the interval is wide, so the data are compatible with a modest difference or none). Label it an illustration. Add a stakeholder meeting agenda of 45 minutes with decisions needed: which harm matters most, which metric reflects it, what difference would trigger action, and what mitigation options exist. Provide a record template for the audit, with the date, data version, metrics, limits, decisions and next review, and remind me that legal questions need qualified advice. Finish with a list of the follow-up analyses I could run if the first results show a gap (a different threshold, more data for the small group, error analysis of the affected cases).
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.