Model Validation and Model Risk

Model validation run as a craft: a model inventory and tiering, conceptual soundness and replication, benchmarks and outcomes analysis, findings that get fixed, ongoing monitoring and documentation a model risk team accepts.

Free · Mac (Apple silicon) and Windows · v1.0.5 · Other platforms

Then open the Marketplace in the app, search for Model Validation and Model Risk, and click Install. It goes into your own files, and works offline from then on.

Before you start: this plan is general information to help your team organise its work. It is not financial, investment, tax, accounting or legal advice, and it does not recommend any product, investment or deal. Rules differ between countries, so check decisions that matter with qualified, licensed advisers in the countries concerned. Investing can lose money, and raising or managing other people's money is regulated in most countries.

What it installs.

1

Pillar

1

Area

50

Projects

200

Starter tasks

🧮 Quantitative Finance & Trading

Model Validation and Model Risk

Validating quantitative models: independent review, assumptions and limits, benchmarking, ongoing monitoring and documentation a model risk team accepts.

  1. Written definition of what counts as a model
  2. Model inventory with owners and uses
  3. Model risk tiering method
  4. Model ownership and independence map
  5. Reading the model risk guidance your firm follows
  6. Model documentation standard a validator accepts
  7. Validation report template and rating scale
  8. Model assumptions and limitations register
  9. Scoping memo for the next validation
  10. Validation findings tracker and remediation follow-up
  11. Ongoing model performance monitoring plan
  12. Annual model review cycle
  13. Model change control and revalidation triggers
  14. Model risk committee meetings
  15. Quarterly inventory attestation
  16. Model overlays and adjustments log
  17. Model risk report to senior management
  18. Validation pipeline and capacity plan
  19. Use before validation exceptions log
  20. Conceptual soundness review practice
  21. Independent replication of model outputs
  22. Benchmark and challenger model build
  23. Outcomes analysis of model forecasts
  24. Sensitivity and stability testing of model inputs
  25. Model input data validation
  26. Writing findings that get fixed
  27. Reading list for model risk practice
  28. Vendor and third-party model validation approach
  29. Spreadsheet and end-user tool controls
  30. Model interdependency map
  31. Approved uses and use limits statement
  32. Decommissioning retired models
  33. Compensating controls for models with open findings
  34. Pre-submission readiness check before validation
  35. Internal audit or supervisory review of model risk
  36. Model go-live approval meeting
  37. Annual model risk report to the board
  38. Responding to a validation that rejected your model
  39. First 90 days as a model validation analyst
  40. Moving from front-office research to model validation
  41. Preparing your first model for independent review
  42. One-person validation for a small fund or startup
  43. Independent validation engagement as a consultant
  44. Stepping up to model risk manager
  45. Validating machine learning models
  46. Generative AI and language model tool review
  47. Pricing model validation and model reserves
  48. Credit risk model validation
  49. Model risk appetite and aggregate model risk
  50. Quantifying uncertainty in a single model's output

Where it starts.

Each project arrives with its purpose, milestones and first tasks written out. Here is how the first three read.

Written definition of what counts as a model

Most model risk arguments start with whether a thing is a model at all: a pricing spreadsheet, a rules engine, a vendor score or a regression a researcher runs every Friday. A one-page definition, with worked examples of what is in and what is out, settles that argument once and decides what goes in the inventory and what gets validated.

First tasks

  • Each desk lead lists ten tools the team uses that might be models
  • Draft a definition built on inputs, method and decision-relevant outputs
  • Class each of the ten tools against the draft definition with a reason
  • Ask the model risk lead to agree or amend the definition

Model inventory with owners and uses

A validation function cannot manage risk in models it does not know exist, and inventories built from memory miss the spreadsheet in the corner and the script copied between desks. Building one inventory with an owner, a developer, the uses, the inputs and the validation status for every model is the base everything else in this area stands on.

First tasks

  • Set out the inventory fields and agree them with the model risk lead
  • Send each team a short request to declare its models and owners
  • Link each entry to its documentation and code repository
  • Flag every entry with no named owner or no documentation

Model risk tiering method

Validating every model to the same depth wastes scarce reviewer time on low-stakes tools and starves the models that move capital, prices or reported numbers. A written tiering method, scoring materiality, complexity and reliance, decides how deep each validation goes and how often it repeats.

First tasks

  • Draft materiality, complexity and reliance criteria with a scoring scale
  • Validation lead tests the scoring on five models the team knows well
  • Score every inventory entry and assign a tier
  • Send tier assignments to model owners for comment

Make it yours.

Install Model Validation and Model Risk, then tell Aurum the date you're aiming for. It checks whether the plan fits the weeks you actually have.

Free · Mac (Apple silicon) and Windows · v1.0.5 · Other platforms

Blueprints are general information to help you organise your own work. They are not medical, financial, legal or other professional advice. See the Terms.