Fairness Audit · People Analytics

A model that flags who might leave can't be allowed to flag one group more than another without anyone checking

A risk score is only usable in HR if it's defensible. I ran a disparate-impact audit on the same 294 held-out employees behind the live prototype — a chi-square test of independence plus the EEOC's four-fifths rule, across the three demographic fields the dataset provides. Gender passes clean. Age doesn't, and that's reported here rather than left out.

Computed directly from the scored held-out set embedded in the live intervention prototype (n=294), not a separate or re-run sample — the same 84 flagged employees referenced throughout this case study.

Method

Chi-square test of independence — tests whether being flagged as at-risk is statistically independent of group membership (gender, age band, marital status). A low p-value means the flag rate really does differ by group, not by chance.
Four-fifths (80%) rule — the EEOC's standard adverse-impact screen: if any group's selection rate falls below 80% of the highest-selected group's rate, that's a flag for review, independent of statistical significance.
Age band split at 40 — the standard cut used under the U.S. Age Discrimination in Employment Act (ADEA), not an arbitrary threshold.

Results

Attribute	Flag rate by group	χ²	p-value	4/5ths ratio	Verdict
Gender	Male 27.5% (n=178) Female 30.2% (n=116)	0.13	0.720	0.912	Pass
Age band	40+ 14.9% (n=101) Under 40 35.8% (n=193)	13.19	<0.001	0.415	Adverse impact
Marital status	Divorced 12.5% (n=64) Married 24.8% (n=133) Single 44.3% (n=97)	20.82	<0.001	0.282	Adverse impact

Passing on gender doesn't make the model fair — it makes it fair on the one dimension that happened to pass. The other two needed to be looked at, not assumed away.

What this means

Gender: clean

Flag rate is nearly identical (27.5% vs. 30.2%), the difference isn't statistically significant (p=0.72), and the adverse-impact ratio (0.912) sits comfortably above the 0.80 threshold. No action needed here.

Age: a real disparity

Employees under 40 are flagged at more than double the rate of employees 40+ (35.8% vs. 14.9%), and it's not noise (p<0.001). An adverse-impact ratio of 0.415 is well below the 0.80 line — this is exactly the pattern the four-fifths rule exists to catch.

Likely driver, not proof of bias

This dataset's real attrition is itself concentrated in early-tenure, early-career employees — a well-documented pattern, not unique to this model. The score is likely picking up a genuine signal (tenure, career stage) that correlates with age, rather than using age as a feature directly.

Correlation doesn't remove the obligation

Disparate impact is a statistical fact, not an intent test — a legitimate underlying driver doesn't exempt a flagging system from disproportionately affecting a protected group. That's precisely why this is reported as a finding, not reasoned away.

What I'd do next with this finding

Trace the drivers, not just the outcome. The live prototype already surfaces a per-employee, SHAP-style contribution breakdown — the next step is aggregating those top factors specifically for the under-40 flagged group to see whether age-correlated features (tenure, total working years, number of companies worked) are doing most of the work.
Keep the human-in-the-loop guardrail non-negotiable for this exact reason. The model was already designed to flag, not decide — this finding is the concrete argument for why that boundary matters, not just a principle.
Re-run this audit before any expansion in scope. If this score ever fed anything beyond an HRBP check-in prompt — comp, promotion, or workforce-planning decisions — this exact test would need to pass, with legal counsel involved, before that expansion happens.