AI Feature Product Management

Shaping AI features from use case to trusted release: quality bars and golden evaluation sets, failure catalogues, cost and latency budgets, weekly output reviews, model changes, guardrails and the launch gate.

Free · Mac (Apple silicon) and Windows · v1.0.5 · Other platforms

Then open the Marketplace in the app, search for AI Feature Product Management, and click Install. It goes into your own files, and works offline from then on.

What it installs.

1

Pillar

1

Area

50

Projects

200

Starter tasks

🧩 Product Management

AI Feature Product Management

Shaping AI-powered features, from choosing use cases and setting quality bars to evaluation, cost, failure handling and user trust.

  1. AI use case inventory scored by value and risk
  2. Rules, classic ML or language model choice
  3. Written quality bar with acceptable error rates
  4. Golden evaluation set from real examples
  5. Failure mode catalogue for the AI feature
  6. Data and privacy check for model inputs
  7. Cost per task estimate before building
  8. One-page AI feature brief
  9. Human baseline for the task today
  10. Weekly evaluation run against the golden set
  11. Weekly sample review of live AI outputs
  12. Thumbs and problem report triage
  13. Prompt and model change log
  14. Monthly AI cost and latency review
  15. AI incident log and response routine
  16. Evaluation set refresh from production cases
  17. Model provider updates and deprecation watch
  18. Abuse and jailbreak case library
  19. Quarterly AI feature health review
  20. How language models fail, for product managers
  21. Writing grading rubrics for AI outputs
  22. Prompt writing basics for product managers
  23. Tokens, context windows and pricing units
  24. Reading model benchmarks with scepticism
  25. Model-graded evaluation and its limits
  26. Sample sizes for comparing model variants
  27. Choosing a model provider on your own evals
  28. Large and small model routing decision
  29. Retrieval, fine-tuning or prompting decision
  30. Human review points for high-stakes outputs
  31. Interface patterns for uncertain AI output
  32. Fallback behaviour when the model fails
  33. Latency budget and streaming decision
  34. Cutting cost per task by a third
  35. Input and output guardrails layer
  36. AI disclosure and labelling in the product
  37. Internal dogfood fortnight for the AI feature
  38. Pre-launch red-team day
  39. AI launch quality gate and go or no-go
  40. Planned migration to a new model version
  41. Thirty-day post-launch AI quality review
  42. First AI feature for a founder without ML staff
  43. Engineer stepping into AI feature ownership
  44. Adding AI to a mature product carefully
  45. Answering customer AI security questionnaires
  46. Briefing sales and support on AI limits
  47. Agent workflows with tool permissions and stop points
  48. Implicit quality signals from user behaviour
  49. Shared evaluation harness across AI features
  50. User memory and personalisation consent controls

Where it starts.

Each project arrives with its purpose, milestones and first tasks written out. Here is how the first three read.

AI use case inventory scored by value and risk

Teams adding AI usually start from what the model can do rather than the work customers need done, and end up with a chat box nobody asked for. Listing every candidate use case against the user task it speeds up, how often that task happens, what a wrong answer would cost and whether the data exists turns a vague mandate to add AI into a short, defensible list.

First tasks

  • List every AI idea raised in the last six months from tickets, sales calls and the team
  • Ask the agent to suggest use cases from your top twenty support ticket themes
  • Score each idea for frequency, value, harm of errors and data on hand
  • Choose a top three and write one line on why each made the cut

Rules, classic ML or language model choice

Plenty of problems sold as AI work are better solved by a lookup table, a pattern match or a simple classifier that is cheaper, faster and fully predictable. Before committing to a language model for the top use case, test whether a simpler approach reaches the bar, and record why the model earns its cost if it does.

First tasks

  • Write the use case as one input, one output and what makes an output correct
  • Ask an engineer how a rules or simple classifier version would work
  • Run twenty real examples through both approaches
  • Record the chosen approach with cost, speed and accuracy side by side

Written quality bar with acceptable error rates

Every AI feature makes mistakes, so the useful question is which mistakes, how often and at what cost. A quality bar written before building, such as 'no invented account numbers, fewer than one in twenty summaries missing a key action', gives engineers a target and gives you an honest launch criterion instead of a demo that felt good.

First tasks

  • Write three examples each of a cosmetic, a serious and an unacceptable error
  • Agree a maximum rate for each severity with design and engineering
  • Name the errors that block launch at any rate
  • Revisit the quality bar against the latest production data @recurring(quarterly)

Make it yours.

Install AI Feature Product Management, then tell Aurum the date you're aiming for. It checks whether the plan fits the weeks you actually have.

Free · Mac (Apple silicon) and Windows · v1.0.5 · Other platforms

Blueprints are general information to help you organise your own work. They are not medical, financial, legal or other professional advice. See the Terms.