1. Traceability¶

1. Objective¶

Traceability ensures that we can always explain why an AI system produced a particular output. This is essential for auditing, debugging, incident analysis and compliance with the EU AI Act.

2. The Traceability Pyramid¶

 ┌───────────────┐
 │ Goal          │  Why are we building this?
 │ Definition    │
 └───────┬───────┘
         │
 ┌───────v───────┐
 │ Specification │  How must it behave?
 │ (Contract)    │
 └───────┬───────┘
         │
 ┌───────v───────┐
 │ Steering      │  Which prompts/configs steer it?
 │ Instructions  │
 └───────┬───────┘
         │
 ┌───────v───────┐
 │ Golden Set    │  How have we tested?
 │ (Tests)       │
 └───────┬───────┘
         │
 ┌───────v───────┐
 │ Validation    │  What were the results?
 │ Report        │
 └───────────────┘

Each layer must be traceable back to the layer above it.

3. Traceability Matrix¶

The traceability matrix links requirements to implementation to tests.

Structure¶

Minimum Fields¶

4. Runtime Traceability (Logging)¶

In addition to documentation traceability, runtime logging is essential.

What Do We Log?¶

Per interaction, at minimum (see Evidence Standards):

For systems that execute tasks autonomously, we additionally record which actions were taken, within which pre-established boundaries, and whether human intervention or approval took place.

Logging per Risk Level¶

Retention¶

• Minimal/Limited: 90 days standard
• High Risk: 12 months or longer (depending on regulations)

5. Incident Analysis with Traceability¶

When an incident occurs, we follow the traceability chain back:

Analysis Procedure¶

1. Identify the output: Which response caused the problem?
2. Retrieve logging: Request-ID, input, model, sources
3. Check Steering Instructions: Was the correct version active?
4. Compare with specification: Did the output comply with the spec?
5. Check Golden Set: Had we tested this scenario?
6. Back to Goal Definition: Was this behaviour intended or a gap?

Root Cause Categories¶

6. Traceability for Audit¶

EU AI Act Requirements (High Risk)¶

• All decisions must be traceable
• Documentation must be available to supervisory authorities
• Changes to the system must be documented

Audit-Ready Package¶

For each production release:

7. Tooling Suggestions¶

8. Traceability Checklist¶

9. Related Modules¶

• Evidence Standards
• Traceability Template
• Validation Report
• Incident Response

Next step: Set up the traceability matrix and link it to the Gate 4 Checklist → See also: Evidence Standards