Skip to content

1. Core Activities & Roles (Validation)

Purpose

Overview of the core activities and role assignments during the Validation phase, including the Validation Pilot (PoV) and Business Case preparation.

1. Core Activities

Validation Pilot

A small-scale experiment to test whether the AI understands the specific business context.

  • Assemble Test Set: Collect 50–100 representative real-world examples
  • Baseline Measurement: How do humans or existing systems perform currently?
  • AI Experiment: Have the AI process the same examples
  • Success Criterion: Does the AI score a sufficient result (>90%) on the test set?

Reliability Testing

Statistical check whether the results are stable and not based on chance.

  • Reproducibility: Does the AI give consistent answers when repeated?
  • Edge Cases: How does the system respond to unusual or extreme input?
  • Bias Detection: Are there systematic errors in certain categories?

Cost Overview

A complete estimate of investment and operational costs.

Investment Costs

  • People: Development, training, management (FTEs)
  • Technology: Licences, cloud infrastructure, tools
  • Data: Cleaning, labelling, enrichment

Operational Costs (per month/year)

  • Usage Costs: Cloud/API costs per task or transaction
  • Maintenance: Monitoring, updates, support
  • Risk: Potential costs of errors or incidents

Return on Investment (ROI)

  • Time Savings: How many hours do we save per week/month?
  • Quality Improvement: Fewer errors, higher customer satisfaction
  • Revenue Growth: New opportunities, faster turnaround

2. Team & Roles

Role Responsibility in Validation
Data Scientist Responsible: Performing the Validation Pilot and reliability testing.
AI Product Manager Accountable: Owner of the business case and ROI calculation (Cost Overview).
Business Sponsor Consulted: Validates the test set and success criteria.
Finance Consulted: Reviews the cost estimate and ROI calculation.
Stakeholders Informed: Receive updates on progress.

Templates:

See also: Phase 2 Overview · Deliverables

Next step: Run the Validation Pilot and document the results in the Validation report. → Use the Validation Report as your starting point. → See also: Business Case | Gate 2 Checklist