Algorithmic Bias Review

An inventory of your models and automated decisions, agreed fairness metrics, disparity testing, documented limitations, release gates and the mitigations and audits that fix unfair outcomes.

Free · Mac (Apple silicon) and Windows · v1.0.5 · Other platforms

Then open the Marketplace in the app, search for Algorithmic Bias Review, and click Install. It goes into your own files, and works offline from then on.

What it installs.

1

Pillar

1

Area

50

Projects

194

Starter tasks

📊 Data & Analytics

Algorithmic Bias Review

Checking models and automated decisions for unfair outcomes across groups, documenting limitations and fixing harmful patterns.

  1. Inventory of models and automated decisions
  2. Register of groups and sensitive attributes to test
  3. Fairness metric choice for each decision type
  4. Harm and stakeholder map for one high-risk decision
  5. Bias review charter and sign-off roles
  6. Baseline disparity report for the riskiest model
  7. Group labels when attributes are not collected
  8. Model card for a production model
  9. Datasheet for a training dataset
  10. Pre-release bias review gate
  11. Quarterly fairness report to the risk committee
  12. Monthly outcome disparity check on live decisions
  13. Bias findings log and remediation tracker
  14. Complaint and appeal review for automated decisions
  15. Annual re-review of every high-risk model
  16. Algorithmic regulation and standards watch
  17. Human override monitoring by group
  18. Fairness tests in the model training pipeline
  19. Fairness metrics and their trade-offs
  20. Intersectional analysis with small groups
  21. Sources of bias in data and labels
  22. Disparate impact law and guidance with counsel
  23. Explainability methods for bias investigation
  24. Bias testing for language model features
  25. Algorithmic audit case study reading group
  26. Removing or replacing a proxy variable
  27. Reweighting or resampling training data
  28. Group-aware decision threshold review
  29. Fixing a biased target label
  30. Closing a data gap for an under-served group
  31. Choosing a fairness testing toolkit
  32. Vendor model bias due diligence
  33. Go or no-go decision on a problem model
  34. Independent third-party bias audit
  35. Pre-launch fairness red-team session
  36. Bias inquiry response rehearsal
  37. Annual algorithmic transparency statement
  38. Board briefing on algorithmic fairness risk
  39. Bias review for an academic predictive study
  40. First bias review programme for a large organisation
  41. Hiring and promotion tool bias review
  42. Credit and lending decision fairness review
  43. Clinical risk score equity review
  44. Public service eligibility and fraud flag review
  45. Answering customers' AI fairness questionnaires
  46. Fairness across countries and populations
  47. Causal fairness analysis of a decision
  48. Feedback loop and long-term effect study
  49. Community panel for an automated decision
  50. Publishing a bias audit method or benchmark

Where it starts.

Each project arrives with its purpose, milestones and first tasks written out. Here is how the first three read.

Inventory of models and automated decisions

This is a team plan, best installed into a shared workspace so everyone with a role in it can see and own their tasks. Most organisations cannot say how many models make or shape decisions about people, and bias reviews then only reach the systems someone happened to mention. A single inventory listing each model, the decision it feeds, who is affected and who owns it is the scope every later review depends on.

First tasks

  • List the models and scoring rules your own team runs today
  • Ask procurement for every vendor tool that scores or ranks people
  • Write three criteria that make a decision high risk
  • Tier each entry and name its business owner
  • Add newly proposed models and vendor tools to the inventory @recurring(monthly:9)

Register of groups and sensitive attributes to test

Every bias review needs to know which groups it is comparing, and the answer depends on the decision, the jurisdiction and the population affected. Writing down the characteristics in scope, such as sex, age, ethnicity, disability and their proxies, with counsel's view on which are legally protected where you operate, stops each analyst choosing their own list.

First tasks

  • Draft the characteristics you think each high-risk decision should be tested on
  • Ask counsel which are legally protected for these decisions and places
  • Note likely proxies for each characteristic in your data
  • Record where each attribute comes from and who may see it
  • Ask counsel whether the protected list has changed @recurring(yearly)

Fairness metric choice for each decision type

Demographic parity, equal opportunity, predictive parity and calibration cannot all hold at once when base rates differ, so choosing a metric is a value judgement, not a technical default. Recording which metric applies to each decision, and why, means a disparity finding can be argued about on its merits rather than on the yardstick.

First tasks

  • Class each high-risk decision as benefit, burden or ranking
  • Write down which error hurts people most for each class
  • Choose a primary and secondary metric and justify each in a paragraph
  • Agree a tolerance for each metric with the decision owner

Make it yours.

Install Algorithmic Bias Review, then tell Aurum the date you're aiming for. It checks whether the plan fits the weeks you actually have.

Free · Mac (Apple silicon) and Windows · v1.0.5 · Other platforms

Blueprints are general information to help you organise your own work. They are not medical, financial, legal or other professional advice. See the Terms.