Sauvegarde de l'état (réparation)

This commit is contained in:
Reviflow Bot committed 2026-01-31 11:00:30 +01:00
commit 7c04fffe77
434 files changed
+83923

No files matched your search

@@ -0,0 +1,407 @@
# Test Design and Risk Assessment - Validation Checklist
## Prerequisites (Mode-Dependent)
**System-Level Mode (Phase 3):**
- [ ] PRD exists with functional and non-functional requirements
- [ ] ADR (Architecture Decision Record) exists
- [ ] Architecture document available (architecture.md or tech-spec)
- [ ] Requirements are testable and unambiguous
**Epic-Level Mode (Phase 4):**
- [ ] Story markdown with clear acceptance criteria exists
- [ ] PRD or epic documentation available
- [ ] Architecture documents available (test-design-architecture.md + test-design-qa.md from Phase 3, if exists)
- [ ] Requirements are testable and unambiguous
## Process Steps
### Step 1: Context Loading
- [ ] PRD.md read and requirements extracted
- [ ] Epics.md or specific epic documentation loaded
- [ ] Story markdown with acceptance criteria analyzed
- [ ] Architecture documents reviewed (if available)
- [ ] Existing test coverage analyzed
- [ ] Knowledge base fragments loaded (risk-governance, probability-impact, test-levels, test-priorities)
### Step 2: Risk Assessment
- [ ] Genuine risks identified (not just features)
- [ ] Risks classified by category (TECH/SEC/PERF/DATA/BUS/OPS)
- [ ] Probability scored (1-3 for each risk)
- [ ] Impact scored (1-3 for each risk)
- [ ] Risk scores calculated (probability × impact)
- [ ] High-priority risks (score ≥6) flagged
- [ ] Mitigation plans defined for high-priority risks
- [ ] Owners assigned for each mitigation
- [ ] Timelines set for mitigations
- [ ] Residual risk documented
### Step 3: Coverage Design
- [ ] Acceptance criteria broken into atomic scenarios
- [ ] Test levels selected (E2E/API/Component/Unit)
- [ ] No duplicate coverage across levels
- [ ] Priority levels assigned (P0/P1/P2/P3)
- [ ] P0 scenarios meet strict criteria (blocks core + high risk + no workaround)
- [ ] Data prerequisites identified
- [ ] Tooling requirements documented
- [ ] Execution order defined (smoke → P0 → P1 → P2/P3)
### Step 4: Deliverables Generation
- [ ] Risk assessment matrix created
- [ ] Coverage matrix created
- [ ] Execution order documented
- [ ] Resource estimates calculated
- [ ] Quality gate criteria defined
- [ ] Output file written to correct location
- [ ] Output file uses template structure
## Output Validation
### Risk Assessment Matrix
- [ ] All risks have unique IDs (R-001, R-002, etc.)
- [ ] Each risk has category assigned
- [ ] Probability values are 1, 2, or 3
- [ ] Impact values are 1, 2, or 3
- [ ] Scores calculated correctly (P × I)
- [ ] High-priority risks (≥6) clearly marked
- [ ] Mitigation strategies specific and actionable
### Coverage Matrix
- [ ] All requirements mapped to test levels
- [ ] Priorities assigned to all scenarios
- [ ] Risk linkage documented
- [ ] Test counts realistic
- [ ] Owners assigned where applicable
- [ ] No duplicate coverage (same behavior at multiple levels)
### Execution Strategy
**CRITICAL: Keep execution strategy simple, avoid redundancy**
- [ ] **Simple structure**: PR / Nightly / Weekly (NOT complex smoke/P0/P1/P2 tiers)
- [ ] **PR execution**: All functional tests unless significant infrastructure overhead
- [ ] **Nightly/Weekly**: Only performance, chaos, long-running, manual tests
- [ ] **No redundancy**: Don't re-list all tests (already in coverage plan)
- [ ] **Philosophy stated**: "Run everything in PRs if <15 min, defer only if expensive/long"
- [ ] **Playwright parallelization noted**: 100s of tests in 10-15 min
### Resource Estimates
**CRITICAL: Use intervals/ranges, NOT exact numbers**
- [ ] P0 effort provided as interval range (e.g., "~25-40 hours" NOT "36 hours")
- [ ] P1 effort provided as interval range (e.g., "~20-35 hours" NOT "27 hours")
- [ ] P2 effort provided as interval range (e.g., "~10-30 hours" NOT "15.5 hours")
- [ ] P3 effort provided as interval range (e.g., "~2-5 hours" NOT "2.5 hours")
- [ ] Total effort provided as interval range (e.g., "~55-110 hours" NOT "81 hours")
- [ ] Timeline provided as week range (e.g., "~1.5-3 weeks" NOT "11 days")
- [ ] Estimates include setup time and account for complexity variations
- [ ] **No false precision**: Avoid exact calculations like "18 tests × 2 hours = 36 hours"
### Quality Gate Criteria
- [ ] P0 pass rate threshold defined (should be 100%)
- [ ] P1 pass rate threshold defined (typically ≥95%)
- [ ] High-risk mitigation completion required
- [ ] Coverage targets specified (≥80% recommended)
## Quality Checks
### Evidence-Based Assessment
- [ ] Risk assessment based on documented evidence
- [ ] No speculation on business impact
- [ ] Assumptions clearly documented
- [ ] Clarifications requested where needed
- [ ] Historical data referenced where available
### Risk Classification Accuracy
- [ ] TECH risks are architecture/integration issues
- [ ] SEC risks are security vulnerabilities
- [ ] PERF risks are performance/scalability concerns
- [ ] DATA risks are data integrity issues
- [ ] BUS risks are business/revenue impacts
- [ ] OPS risks are deployment/operational issues
### Priority Assignment Accuracy
**CRITICAL: Priority classification is separate from execution timing**
- [ ] **Priority sections (P0/P1/P2/P3) do NOT include execution context** (e.g., no "Run on every commit" in headers)
- [ ] **Priority sections have only "Criteria" and "Purpose"** (no "Execution:" field)
- [ ] **Execution Strategy section** is separate and handles timing based on infrastructure overhead
- [ ] P0: Truly blocks core functionality + High-risk (≥6) + No workaround
- [ ] P1: Important features + Medium-risk (3-4) + Common workflows
- [ ] P2: Secondary features + Low-risk (1-2) + Edge cases
- [ ] P3: Nice-to-have + Exploratory + Benchmarks
- [ ] **Note at top of Test Coverage Plan**: Clarifies P0/P1/P2/P3 = priority/risk, NOT execution timing
### Test Level Selection
- [ ] E2E used only for critical paths
- [ ] API tests cover complex business logic
- [ ] Component tests for UI interactions
- [ ] Unit tests for edge cases and algorithms
- [ ] No redundant coverage
## Integration Points
### Knowledge Base Integration
- [ ] risk-governance.md consulted
- [ ] probability-impact.md applied
- [ ] test-levels-framework.md referenced
- [ ] test-priorities-matrix.md used
- [ ] Additional fragments loaded as needed
### Status File Integration
- [ ] Test design logged in Quality & Testing Progress
- [ ] Epic number and scope documented
- [ ] Completion timestamp recorded
### Workflow Dependencies
- [ ] Can proceed to `*atdd` workflow with P0 scenarios
- [ ] `*atdd` is a separate workflow and must be run explicitly (not auto-run)
- [ ] Can proceed to `automate` workflow with full coverage plan
- [ ] Risk assessment informs `gate` workflow criteria
- [ ] Integrates with `ci` workflow execution order
## System-Level Mode: Two-Document Validation
**When in system-level mode (PRD + ADR input), validate BOTH documents:**
### test-design-architecture.md
- [ ] **Purpose statement** at top (serves as contract with Architecture team)
- [ ] **Executive Summary** with scope, business context, architecture decisions, risk summary
- [ ] **Quick Guide** section with three tiers:
- [ ] 🚨 BLOCKERS - Team Must Decide (Sprint 0 critical path items)
- [ ] ⚠️ HIGH PRIORITY - Team Should Validate (recommendations for approval)
- [ ] 📋 INFO ONLY - Solutions Provided (no decisions needed)
- [ ] **Risk Assessment** section - **ACTIONABLE**
- [ ] Total risks identified count
- [ ] High-priority risks table (score ≥6) with all columns: Risk ID, Category, Description, Probability, Impact, Score, Mitigation, Owner, Timeline
- [ ] Medium and low-priority risks tables
- [ ] Risk category legend included
- [ ] **Testability Concerns and Architectural Gaps** section - **ACTIONABLE**
- [ ] **Sub-section: 🚨 ACTIONABLE CONCERNS** at TOP
- [ ] Blockers to Fast Feedback table (WHAT architecture must provide)
- [ ] Architectural Improvements Needed (WHAT must be changed)
- [ ] Each concern has: Owner, Timeline, Impact
- [ ] **Sub-section: Testability Assessment Summary** at BOTTOM (FYI)
- [ ] What Works Well (passing items)
- [ ] Accepted Trade-offs (no action required)
- [ ] This section only included if worth mentioning; otherwise omitted
- [ ] **Risk Mitigation Plans** for all high-priority risks (≥6)
- [ ] Each plan has: Strategy (numbered steps), Owner, Timeline, Status, Verification
- [ ] **Only Backend/DevOps/Arch/Security mitigations** (production code changes)
- [ ] QA-owned mitigations belong in QA doc instead
- [ ] **Assumptions and Dependencies** section
- [ ] **Architectural assumptions only** (SLO targets, replication lag, system design)
- [ ] Assumptions list (numbered)
- [ ] Dependencies list with required dates
- [ ] Risks to plan with impact and contingency
- [ ] QA execution assumptions belong in QA doc instead
- [ ] **NO test implementation code** (long examples belong in QA doc)
- [ ] **NO test scripts** (no Playwright test(...) blocks, no assertions, no test setup code)
- [ ] **NO NFR test examples** (NFR sections describe WHAT to test, not HOW to test)
- [ ] **NO test scenario checklists** (belong in QA doc)
- [ ] **NO bloat or repetition** (consolidate repeated notes, avoid over-explanation)
- [ ] **Cross-references to QA doc** where appropriate (instead of duplication)
- [ ] **RECIPE SECTIONS NOT IN ARCHITECTURE DOC:**
- [ ] NO "Test Levels Strategy" section (unit/integration/E2E split belongs in QA doc only)
- [ ] NO "NFR Testing Approach" section with detailed test procedures (belongs in QA doc only)
- [ ] NO "Test Environment Requirements" section (belongs in QA doc only)
- [ ] NO "Recommendations for Sprint 0" section with test framework setup (belongs in QA doc only)
- [ ] NO "Quality Gate Criteria" section (pass rates, coverage targets belong in QA doc only)
- [ ] NO "Tool Selection" section (Playwright, k6, etc. belongs in QA doc only)
### test-design-qa.md
**REQUIRED SECTIONS:**
- [ ] **Purpose statement** at top (test execution recipe)
- [ ] **Executive Summary** with risk summary and coverage summary
- [ ] **Dependencies & Test Blockers** section in POSITION 2 (right after Executive Summary)
- [ ] Backend/Architecture dependencies listed (what QA needs from other teams)
- [ ] QA infrastructure setup listed (factories, fixtures, environments)
- [ ] Code example with playwright-utils if config.tea_use_playwright_utils is true
- [ ] Test from '@seontechnologies/playwright-utils/api-request/fixtures'
- [ ] Expect from '@playwright/test' (playwright-utils does not re-export expect)
- [ ] Code examples include assertions (no unused imports)
- [ ] **Risk Assessment** section (brief, references Architecture doc)
- [ ] High-priority risks table
- [ ] Medium/low-priority risks table
- [ ] Each risk shows "QA Test Coverage" column (how QA validates)
- [ ] **Test Coverage Plan** with P0/P1/P2/P3 sections
- [ ] Priority sections have ONLY "Criteria" (no execution context)
- [ ] Note at top: "P0/P1/P2/P3 = priority, NOT execution timing"
- [ ] Test tables with columns: Test ID | Requirement | Test Level | Risk Link | Notes
- [ ] **Execution Strategy** section (organized by TOOL TYPE)
- [ ] Every PR: Playwright tests (~10-15 min)
- [ ] Nightly: k6 performance tests (~30-60 min)
- [ ] Weekly: Chaos & long-running (~hours)
- [ ] Philosophy: "Run everything in PRs unless expensive/long-running"
- [ ] **QA Effort Estimate** section (QA effort ONLY)
- [ ] Interval-based estimates (e.g., "~1-2 weeks" NOT "36 hours")
- [ ] NO DevOps, Backend, Data Eng, Finance effort
- [ ] NO Sprint breakdowns (too prescriptive)
- [ ] **Appendix A: Code Examples & Tagging**
- [ ] **Appendix B: Knowledge Base References**
**DON'T INCLUDE (bloat):**
- [ ] ❌ NO Quick Reference section
- [ ] ❌ NO System Architecture Summary
- [ ] ❌ NO Test Environment Requirements as separate section (integrate into Dependencies)
- [ ] ❌ NO Testability Assessment section (covered in Dependencies)
- [ ] ❌ NO Test Levels Strategy section (obvious from test scenarios)
- [ ] ❌ NO NFR Readiness Summary
- [ ] ❌ NO Quality Gate Criteria section (teams decide for themselves)
- [ ] ❌ NO Follow-on Workflows section (BMAD commands self-explanatory)
- [ ] ❌ NO Approval section
- [ ] ❌ NO Infrastructure/DevOps/Finance effort tables (out of scope)
- [ ] ❌ NO Sprint 0/1/2/3 breakdown tables
- [ ] ❌ NO Next Steps section
### Cross-Document Consistency
- [ ] Both documents reference same risks by ID (R-001, R-002, etc.)
- [ ] Both documents use consistent priority levels (P0, P1, P2, P3)
- [ ] Both documents reference same Sprint 0 blockers
- [ ] No duplicate content (cross-reference instead)
- [ ] Dates and authors match across documents
- [ ] ADR and PRD references consistent
### Document Quality (Anti-Bloat Check)
**CRITICAL: Check for bloat and repetition across BOTH documents**
- [ ] **No repeated notes 10+ times** (e.g., "Timing is pessimistic until R-005 fixed" on every section)
- [ ] **Repeated information consolidated** (write once at top, reference briefly if needed)
- [ ] **No excessive detail** that doesn't add value (obvious concepts, redundant examples)
- [ ] **Focus on unique/critical info** (only document what's different from standard practice)
- [ ] **Architecture doc**: Concerns-focused, NOT implementation-focused
- [ ] **QA doc**: Implementation-focused, NOT theory-focused
- [ ] **Clear separation**: Architecture = WHAT and WHY, QA = HOW
- [ ] **Professional tone**: No AI slop markers
- [ ] Avoid excessive ✅/❌ emojis (use sparingly, only when adding clarity)
- [ ] Avoid "absolutely", "excellent", "fantastic", overly enthusiastic language
- [ ] Write professionally and directly
- [ ] **Architecture doc length**: Target ~150-200 lines max (focus on actionable concerns only)
- [ ] **QA doc length**: Keep concise, remove bloat sections
### Architecture Doc Structure (Actionable-First Principle)
**CRITICAL: Validate structure follows actionable-first, FYI-last principle**
- [ ] **Actionable sections at TOP:**
- [ ] Quick Guide (🚨 BLOCKERS first, then ⚠️ HIGH PRIORITY, then 📋 INFO ONLY last)
- [ ] Risk Assessment (high-priority risks ≥6 at top)
- [ ] Testability Concerns (concerns/blockers at top, passing items at bottom)
- [ ] Risk Mitigation Plans (for high-priority risks ≥6)
- [ ] **FYI sections at BOTTOM:**
- [ ] Testability Assessment Summary (what works well - only if worth mentioning)
- [ ] Assumptions and Dependencies
- [ ] **ASRs categorized correctly:**
- [ ] Actionable ASRs included in 🚨 or ⚠️ sections
- [ ] FYI ASRs included in 📋 section or omitted if obvious
## Completion Criteria
**All must be true:**
- [ ] All prerequisites met
- [ ] All process steps completed
- [ ] All output validations passed
- [ ] All quality checks passed
- [ ] All integration points verified
- [ ] Output file(s) complete and well-formatted
- [ ] **System-level mode:** Both documents validated (if applicable)
- [ ] **Epic-level mode:** Single document validated (if applicable)
- [ ] Team review scheduled (if required)
## Post-Workflow Actions
**User must complete:**
1. [ ] Review risk assessment with team
2. [ ] Prioritize mitigation for high-priority risks (score ≥6)
3. [ ] Allocate resources per estimates
4. [ ] Run `*atdd` workflow to generate P0 tests (separate workflow; not auto-run)
5. [ ] Set up test data factories and fixtures
6. [ ] Schedule team review of test design document
**Recommended next workflows:**
1. [ ] Run `atdd` workflow for P0 test generation
2. [ ] Run `framework` workflow if not already done
3. [ ] Run `ci` workflow to configure pipeline stages
## Rollback Procedure
If workflow fails:
1. [ ] Delete output file
2. [ ] Review error logs
3. [ ] Fix missing context (PRD, architecture docs)
4. [ ] Clarify ambiguous requirements
5. [ ] Retry workflow
## Notes
### Common Issues
**Issue**: Too many P0 tests
- **Solution**: Apply strict P0 criteria - must block core AND high risk AND no workaround
**Issue**: Risk scores all high
- **Solution**: Differentiate between high-impact (3) and degraded (2) impacts
**Issue**: Duplicate coverage across levels
- **Solution**: Use test pyramid - E2E for critical paths only
**Issue**: Resource estimates too high or too precise
- **Solution**:
- Invest in fixtures/factories to reduce per-test setup time
- Use interval ranges (e.g., "~55-110 hours") instead of exact numbers (e.g., "81 hours")
- Widen intervals if high uncertainty exists
**Issue**: Execution order section too complex or redundant
- **Solution**:
- Default: Run everything in PRs (<15 min with Playwright parallelization)
- Only defer to nightly/weekly if expensive (k6, chaos, 4+ hour tests)
- Don't create smoke/P0/P1/P2/P3 tier structure
- Don't re-list all tests (already in coverage plan)
### Best Practices
- Base risk assessment on evidence, not assumptions
- High-priority risks (≥6) require immediate mitigation
- P0 tests should cover <10% of total scenarios
- Avoid testing same behavior at multiple levels
- **Use interval-based estimates** (e.g., "~25-40 hours") instead of exact numbers to avoid false precision and provide flexibility
- **Keep execution strategy simple**: Default to "run everything in PRs" (<15 min with Playwright), only defer if expensive/long-running
- **Avoid execution order redundancy**: Don't create complex tier structures or re-list tests
---
**Checklist Complete**: Sign off when all items validated.
**Completed by:** {name}
**Date:** {date}
**Epic:** {epic title}
**Notes:** {additional notes}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,213 @@
# Test Design for Architecture: {Feature Name}
**Purpose:** Architectural concerns, testability gaps, and NFR requirements for review by Architecture/Dev teams. Serves as a contract between QA and Engineering on what must be addressed before test development begins.
**Date:** {date}
**Author:** {author}
**Status:** Architecture Review Pending
**Project:** {project_name}
**PRD Reference:** {prd_link}
**ADR Reference:** {adr_link}
---
## Executive Summary
**Scope:** {Brief description of feature scope}
**Business Context** (from PRD):
- **Revenue/Impact:** {Business metrics if applicable}
- **Problem:** {Problem being solved}
- **GA Launch:** {Target date or timeline}
**Architecture** (from ADR {adr_number}):
- **Key Decision 1:** {e.g., OAuth 2.1 authentication}
- **Key Decision 2:** {e.g., Centralized MCP Server pattern}
- **Key Decision 3:** {e.g., Stack: TypeScript, SDK v1.x}
**Expected Scale** (from ADR):
- {RPS, volume, users, etc.}
**Risk Summary:**
- **Total risks**: {N}
- **High-priority (≥6)**: {N} risks requiring immediate mitigation
- **Test effort**: ~{N} tests (~{X} weeks for 1 QA, ~{Y} weeks for 2 QAs)
---
## Quick Guide
### 🚨 BLOCKERS - Team Must Decide (Can't Proceed Without)
**Sprint 0 Critical Path** - These MUST be completed before QA can write integration tests:
1. **{Blocker ID}: {Blocker Title}** - {What architecture must provide} (recommended owner: {Team/Role})
2. **{Blocker ID}: {Blocker Title}** - {What architecture must provide} (recommended owner: {Team/Role})
3. **{Blocker ID}: {Blocker Title}** - {What architecture must provide} (recommended owner: {Team/Role})
**What we need from team:** Complete these {N} items in Sprint 0 or test development is blocked.
---
### ⚠️ HIGH PRIORITY - Team Should Validate (We Provide Recommendation, You Approve)
1. **{Risk ID}: {Title}** - {Recommendation + who should approve} (Sprint {N})
2. **{Risk ID}: {Title}** - {Recommendation + who should approve} (Sprint {N})
3. **{Risk ID}: {Title}** - {Recommendation + who should approve} (Sprint {N})
**What we need from team:** Review recommendations and approve (or suggest changes).
---
### 📋 INFO ONLY - Solutions Provided (Review, No Decisions Needed)
1. **Test strategy**: {Test level split} ({Rationale})
2. **Tooling**: {Test frameworks and utilities}
3. **Tiered CI/CD**: {Execution tiers with timing}
4. **Coverage**: ~{N} test scenarios prioritized P0-P3 with risk-based classification
5. **Quality gates**: {Pass criteria}
**What we need from team:** Just review and acknowledge (we already have the solution).
---
## For Architects and Devs - Open Topics 👷
### Risk Assessment
**Total risks identified**: {N} ({X} high-priority score ≥6, {Y} medium, {Z} low)
#### High-Priority Risks (Score ≥6) - IMMEDIATE ATTENTION
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner | Timeline |
|---------|----------|-------------|-------------|--------|-------|------------|-------|----------|
| **{R-ID}** | **{CAT}** | {Description} | {1-3} | {1-3} | **{Score}** | {Mitigation strategy} | {Owner} | {Date} |
#### Medium-Priority Risks (Score 3-5)
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner |
|---------|----------|-------------|-------------|--------|-------|------------|-------|
| {R-ID} | {CAT} | {Description} | {1-3} | {1-3} | {Score} | {Mitigation} | {Owner} |
#### Low-Priority Risks (Score 1-2)
| Risk ID | Category | Description | Probability | Impact | Score | Action |
|---------|----------|-------------|-------------|--------|-------|--------|
| {R-ID} | {CAT} | {Description} | {1-3} | {1-3} | {Score} | Monitor |
#### Risk Category Legend
- **TECH**: Technical/Architecture (flaws, integration, scalability)
- **SEC**: Security (access controls, auth, data exposure)
- **PERF**: Performance (SLA violations, degradation, resource limits)
- **DATA**: Data Integrity (loss, corruption, inconsistency)
- **BUS**: Business Impact (UX harm, logic errors, revenue)
- **OPS**: Operations (deployment, config, monitoring)
---
### Testability Concerns and Architectural Gaps
**🚨 ACTIONABLE CONCERNS - Architecture Team Must Address**
{If system has critical testability concerns, list them here. If architecture supports testing well, state "No critical testability concerns identified" and skip to Testability Assessment Summary}
#### 1. Blockers to Fast Feedback (WHAT WE NEED FROM ARCHITECTURE)
| Concern | Impact | What Architecture Must Provide | Owner | Timeline |
|---------|--------|--------------------------------|-------|----------|
| **{Concern name}** | {Impact on testing} | {Specific architectural change needed} | {Team} | {Sprint} |
**Example:**
- **No API for test data seeding** → Cannot parallelize tests → Provide POST /test/seed endpoint (Backend, Sprint 0)
#### 2. Architectural Improvements Needed (WHAT SHOULD BE CHANGED)
{List specific improvements that would make the system more testable}
1. **{Improvement name}**
- **Current problem**: {What's wrong}
- **Required change**: {What architecture must do}
- **Impact if not fixed**: {Consequences}
- **Owner**: {Team}
- **Timeline**: {Sprint}
---
### Testability Assessment Summary
**📊 CURRENT STATE - FYI**
{Only include this section if there are passing items worth mentioning. Otherwise omit.}
#### What Works Well
- ✅ {Passing item 1} (e.g., "API-first design supports parallel test execution")
- ✅ {Passing item 2} (e.g., "Feature flags enable test isolation")
- ✅ {Passing item 3}
#### Accepted Trade-offs (No Action Required)
For {Feature} Phase 1, the following trade-offs are acceptable:
- **{Trade-off 1}** - {Why acceptable for now}
- **{Trade-off 2}** - {Why acceptable for now}
{This is technical debt OR acceptable for Phase 1} that {should be revisited post-GA OR maintained as-is}
---
### Risk Mitigation Plans (High-Priority Risks ≥6)
**Purpose**: Detailed mitigation strategies for all {N} high-priority risks (score ≥6). These risks MUST be addressed before {GA launch date or milestone}.
#### {R-ID}: {Risk Description} (Score: {Score}) - {CRITICALITY LEVEL}
**Mitigation Strategy:**
1. {Step 1}
2. {Step 2}
3. {Step 3}
**Owner:** {Owner}
**Timeline:** {Sprint or date}
**Status:** Planned / In Progress / Complete
**Verification:** {How to verify mitigation is effective}
---
{Repeat for all high-priority risks}
---
### Assumptions and Dependencies
#### Assumptions
1. {Assumption about architecture or requirements}
2. {Assumption about team or timeline}
3. {Assumption about scope or constraints}
#### Dependencies
1. {Dependency} - Required by {date/sprint}
2. {Dependency} - Required by {date/sprint}
#### Risks to Plan
- **Risk**: {Risk to the test plan itself}
- **Impact**: {How it affects testing}
- **Contingency**: {Backup plan}
---
**End of Architecture Document**
**Next Steps for Architecture Team:**
1. Review Quick Guide (🚨/⚠️/📋) and prioritize blockers
2. Assign owners and timelines for high-priority risks (≥6)
3. Validate assumptions and dependencies
4. Provide feedback to QA on testability gaps
**Next Steps for QA Team:**
1. Wait for Sprint 0 blockers to be resolved
2. Refer to companion QA doc (test-design-qa.md) for test scenarios
3. Begin test infrastructure setup (factories, fixtures, environments)
@@ -0,0 +1,286 @@
# Test Design for QA: {Feature Name}
**Purpose:** Test execution recipe for QA team. Defines what to test, how to test it, and what QA needs from other teams.
**Date:** {date}
**Author:** {author}
**Status:** Draft
**Project:** {project_name}
**Related:** See Architecture doc (test-design-architecture.md) for testability concerns and architectural blockers.
---
## Executive Summary
**Scope:** {Brief description of testing scope}
**Risk Summary:**
- Total Risks: {N} ({X} high-priority score ≥6, {Y} medium, {Z} low)
- Critical Categories: {Categories with most high-priority risks}
**Coverage Summary:**
- P0 tests: ~{N} (critical paths, security)
- P1 tests: ~{N} (important features, integration)
- P2 tests: ~{N} (edge cases, regression)
- P3 tests: ~{N} (exploratory, benchmarks)
- **Total**: ~{N} tests (~{X}-{Y} weeks with 1 QA)
---
## Dependencies & Test Blockers
**CRITICAL:** QA cannot proceed without these items from other teams.
### Backend/Architecture Dependencies (Sprint 0)
**Source:** See Architecture doc "Quick Guide" for detailed mitigation plans
1. **{Dependency 1}** - {Team} - {Timeline}
- {What QA needs}
- {Why it blocks testing}
2. **{Dependency 2}** - {Team} - {Timeline}
- {What QA needs}
- {Why it blocks testing}
### QA Infrastructure Setup (Sprint 0)
1. **Test Data Factories** - QA
- {Entity} factory with faker-based randomization
- Auto-cleanup fixtures for parallel safety
2. **Test Environments** - QA
- Local: {Setup details}
- CI/CD: {Setup details}
- Staging: {Setup details}
**Example factory pattern:**
```typescript
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
import { expect } from '@playwright/test';
import { faker } from '@faker-js/faker';
test('example test @p0', async ({ apiRequest }) => {
const testData = {
id: `test-${faker.string.uuid()}`,
email: faker.internet.email(),
};
const { status } = await apiRequest({
method: 'POST',
path: '/api/resource',
body: testData,
});
expect(status).toBe(201);
});
```
---
## Risk Assessment
**Note:** Full risk details in Architecture doc. This section summarizes risks relevant to QA test planning.
### High-Priority Risks (Score ≥6)
| Risk ID | Category | Description | Score | QA Test Coverage |
|---------|----------|-------------|-------|------------------|
| **{R-ID}** | {CAT} | {Brief description} | **{Score}** | {How QA validates this risk} |
### Medium/Low-Priority Risks
| Risk ID | Category | Description | Score | QA Test Coverage |
|---------|----------|-------------|-------|------------------|
| {R-ID} | {CAT} | {Brief description} | {Score} | {How QA validates this risk} |
---
## Test Coverage Plan
**IMPORTANT:** P0/P1/P2/P3 = **priority and risk level** (what to focus on if time-constrained), NOT execution timing. See "Execution Strategy" for when tests run.
### P0 (Critical)
**Criteria:** Blocks core functionality + High risk (≥6) + No workaround + Affects majority of users
| Test ID | Requirement | Test Level | Risk Link | Notes |
|---------|-------------|------------|-----------|-------|
| **P0-001** | {Requirement} | {Level} | {R-ID} | {Notes} |
| **P0-002** | {Requirement} | {Level} | {R-ID} | {Notes} |
**Total P0:** ~{N} tests
---
### P1 (High)
**Criteria:** Important features + Medium risk (3-4) + Common workflows + Workaround exists but difficult
| Test ID | Requirement | Test Level | Risk Link | Notes |
|---------|-------------|------------|-----------|-------|
| **P1-001** | {Requirement} | {Level} | {R-ID} | {Notes} |
| **P1-002** | {Requirement} | {Level} | {R-ID} | {Notes} |
**Total P1:** ~{N} tests
---
### P2 (Medium)
**Criteria:** Secondary features + Low risk (1-2) + Edge cases + Regression prevention
| Test ID | Requirement | Test Level | Risk Link | Notes |
|---------|-------------|------------|-----------|-------|
| **P2-001** | {Requirement} | {Level} | {R-ID} | {Notes} |
**Total P2:** ~{N} tests
---
### P3 (Low)
**Criteria:** Nice-to-have + Exploratory + Performance benchmarks + Documentation validation
| Test ID | Requirement | Test Level | Notes |
|---------|-------------|------------|-------|
| **P3-001** | {Requirement} | {Level} | {Notes} |
**Total P3:** ~{N} tests
---
## Execution Strategy
**Philosophy:** Run everything in PRs unless there's significant infrastructure overhead. Playwright with parallelization is extremely fast (100s of tests in ~10-15 min).
**Organized by TOOL TYPE:**
### Every PR: Playwright Tests (~10-15 min)
**All functional tests** (from any priority level):
- All E2E, API, integration, unit tests using Playwright
- Parallelized across {N} shards
- Total: ~{N} Playwright tests (includes P0, P1, P2, P3)
**Why run in PRs:** Fast feedback, no expensive infrastructure
### Nightly: k6 Performance Tests (~30-60 min)
**All performance tests** (from any priority level):
- Load, stress, spike, endurance tests
- Total: ~{N} k6 tests (may include P0, P1, P2)
**Why defer to nightly:** Expensive infrastructure (k6 Cloud), long-running (10-40 min per test)
### Weekly: Chaos & Long-Running (~hours)
**Special infrastructure tests** (from any priority level):
- Multi-region failover (requires AWS Fault Injection Simulator)
- Disaster recovery (backup restore, 4+ hours)
- Endurance tests (4+ hours runtime)
**Why defer to weekly:** Very expensive infrastructure, very long-running, infrequent validation sufficient
**Manual tests** (excluded from automation):
- DevOps validation (deployment, monitoring)
- Finance validation (cost alerts)
- Documentation validation
---
## QA Effort Estimate
**QA test development effort only** (excludes DevOps, Backend, Data Eng, Finance work):
| Priority | Count | Effort Range | Notes |
|----------|-------|--------------|-------|
| P0 | ~{N} | ~{X}-{Y} weeks | Complex setup (security, performance, multi-step) |
| P1 | ~{N} | ~{X}-{Y} weeks | Standard coverage (integration, API tests) |
| P2 | ~{N} | ~{X}-{Y} days | Edge cases, simple validation |
| P3 | ~{N} | ~{X}-{Y} days | Exploratory, benchmarks |
| **Total** | ~{N} | **~{X}-{Y} weeks** | **1 QA engineer, full-time** |
**Assumptions:**
- Includes test design, implementation, debugging, CI integration
- Excludes ongoing maintenance (~10% effort)
- Assumes test infrastructure (factories, fixtures) ready
**Dependencies from other teams:**
- See "Dependencies & Test Blockers" section for what QA needs from Backend, DevOps, Data Eng
---
## Appendix A: Code Examples & Tagging
**Playwright Tags for Selective Execution:**
```typescript
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
import { expect } from '@playwright/test';
// P0 critical test
test('@P0 @API @Security unauthenticated request returns 401', async ({ apiRequest }) => {
const { status, body } = await apiRequest({
method: 'POST',
path: '/api/endpoint',
body: { data: 'test' },
skipAuth: true,
});
expect(status).toBe(401);
expect(body.error).toContain('unauthorized');
});
// P1 integration test
test('@P1 @Integration data syncs correctly', async ({ apiRequest }) => {
// Seed data
await apiRequest({
method: 'POST',
path: '/api/seed',
body: { /* test data */ },
});
// Validate
const { status, body } = await apiRequest({
method: 'GET',
path: '/api/resource',
});
expect(status).toBe(200);
expect(body).toHaveProperty('data');
});
```
**Run specific tags:**
```bash
# Run only P0 tests
npx playwright test --grep @P0
# Run P0 + P1 tests
npx playwright test --grep "@P0|@P1"
# Run only security tests
npx playwright test --grep @Security
# Run all Playwright tests in PR (default)
npx playwright test
```
---
## Appendix B: Knowledge Base References
- **Risk Governance**: `risk-governance.md` - Risk scoring methodology
- **Test Priorities Matrix**: `test-priorities-matrix.md` - P0-P3 criteria
- **Test Levels Framework**: `test-levels-framework.md` - E2E vs API vs Unit selection
- **Test Quality**: `test-quality.md` - Definition of Done (no hard waits, <300 lines, <1.5 min)
---
**Generated by:** BMad TEA Agent
**Workflow:** `_bmad/bmm/testarch/test-design`
**Version:** 4.0 (BMad v6)
@@ -0,0 +1,294 @@
# Test Design: Epic {epic_num} - {epic_title}
**Date:** {date}
**Author:** {user_name}
**Status:** Draft / Approved
---
## Executive Summary
**Scope:** {design_level} test design for Epic {epic_num}
**Risk Summary:**
- Total risks identified: {total_risks}
- High-priority risks (≥6): {high_priority_count}
- Critical categories: {top_categories}
**Coverage Summary:**
- P0 scenarios: {p0_count} ({p0_hours} hours)
- P1 scenarios: {p1_count} ({p1_hours} hours)
- P2/P3 scenarios: {p2p3_count} ({p2p3_hours} hours)
- **Total effort**: {total_hours} hours (~{total_days} days)
---
## Risk Assessment
### High-Priority Risks (Score ≥6)
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner | Timeline |
| ------- | -------- | ------------- | ----------- | ------ | ----- | ------------ | ------- | -------- |
| R-001 | SEC | {description} | 2 | 3 | 6 | {mitigation} | {owner} | {date} |
| R-002 | PERF | {description} | 3 | 2 | 6 | {mitigation} | {owner} | {date} |
### Medium-Priority Risks (Score 3-4)
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner |
| ------- | -------- | ------------- | ----------- | ------ | ----- | ------------ | ------- |
| R-003 | TECH | {description} | 2 | 2 | 4 | {mitigation} | {owner} |
| R-004 | DATA | {description} | 1 | 3 | 3 | {mitigation} | {owner} |
### Low-Priority Risks (Score 1-2)
| Risk ID | Category | Description | Probability | Impact | Score | Action |
| ------- | -------- | ------------- | ----------- | ------ | ----- | ------- |
| R-005 | OPS | {description} | 1 | 2 | 2 | Monitor |
| R-006 | BUS | {description} | 1 | 1 | 1 | Monitor |
### Risk Category Legend
- **TECH**: Technical/Architecture (flaws, integration, scalability)
- **SEC**: Security (access controls, auth, data exposure)
- **PERF**: Performance (SLA violations, degradation, resource limits)
- **DATA**: Data Integrity (loss, corruption, inconsistency)
- **BUS**: Business Impact (UX harm, logic errors, revenue)
- **OPS**: Operations (deployment, config, monitoring)
---
## Test Coverage Plan
### P0 (Critical) - Run on every commit
**Criteria**: Blocks core journey + High risk (≥6) + No workaround
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
| ------------- | ---------- | --------- | ---------- | ----- | ------- |
| {requirement} | E2E | R-001 | 3 | QA | {notes} |
| {requirement} | API | R-002 | 5 | QA | {notes} |
**Total P0**: {p0_count} tests, {p0_hours} hours
### P1 (High) - Run on PR to main
**Criteria**: Important features + Medium risk (3-4) + Common workflows
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
| ------------- | ---------- | --------- | ---------- | ----- | ------- |
| {requirement} | API | R-003 | 4 | QA | {notes} |
| {requirement} | Component | - | 6 | DEV | {notes} |
**Total P1**: {p1_count} tests, {p1_hours} hours
### P2 (Medium) - Run nightly/weekly
**Criteria**: Secondary features + Low risk (1-2) + Edge cases
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
| ------------- | ---------- | --------- | ---------- | ----- | ------- |
| {requirement} | API | R-004 | 8 | QA | {notes} |
| {requirement} | Unit | - | 15 | DEV | {notes} |
**Total P2**: {p2_count} tests, {p2_hours} hours
### P3 (Low) - Run on-demand
**Criteria**: Nice-to-have + Exploratory + Performance benchmarks
| Requirement | Test Level | Test Count | Owner | Notes |
| ------------- | ---------- | ---------- | ----- | ------- |
| {requirement} | E2E | 2 | QA | {notes} |
| {requirement} | Unit | 8 | DEV | {notes} |
**Total P3**: {p3_count} tests, {p3_hours} hours
---
## Execution Order
### Smoke Tests (<5 min)
**Purpose**: Fast feedback, catch build-breaking issues
- [ ] {scenario} (30s)
- [ ] {scenario} (45s)
- [ ] {scenario} (1min)
**Total**: {smoke_count} scenarios
### P0 Tests (<10 min)
**Purpose**: Critical path validation
- [ ] {scenario} (E2E)
- [ ] {scenario} (API)
- [ ] {scenario} (API)
**Total**: {p0_count} scenarios
### P1 Tests (<30 min)
**Purpose**: Important feature coverage
- [ ] {scenario} (API)
- [ ] {scenario} (Component)
**Total**: {p1_count} scenarios
### P2/P3 Tests (<60 min)
**Purpose**: Full regression coverage
- [ ] {scenario} (Unit)
- [ ] {scenario} (API)
**Total**: {p2p3_count} scenarios
---
## Resource Estimates
### Test Development Effort
| Priority | Count | Hours/Test | Total Hours | Notes |
| --------- | ----------------- | ---------- | ----------------- | ----------------------- |
| P0 | {p0_count} | 2.0 | {p0_hours} | Complex setup, security |
| P1 | {p1_count} | 1.0 | {p1_hours} | Standard coverage |
| P2 | {p2_count} | 0.5 | {p2_hours} | Simple scenarios |
| P3 | {p3_count} | 0.25 | {p3_hours} | Exploratory |
| **Total** | **{total_count}** | **-** | **{total_hours}** | **~{total_days} days** |
### Prerequisites
**Test Data:**
- {factory_name} factory (faker-based, auto-cleanup)
- {fixture_name} fixture (setup/teardown)
**Tooling:**
- {tool} for {purpose}
- {tool} for {purpose}
**Environment:**
- {env_requirement}
- {env_requirement}
---
## Quality Gate Criteria
### Pass/Fail Thresholds
- **P0 pass rate**: 100% (no exceptions)
- **P1 pass rate**: ≥95% (waivers required for failures)
- **P2/P3 pass rate**: ≥90% (informational)
- **High-risk mitigations**: 100% complete or approved waivers
### Coverage Targets
- **Critical paths**: ≥80%
- **Security scenarios**: 100%
- **Business logic**: ≥70%
- **Edge cases**: ≥50%
### Non-Negotiable Requirements
- [ ] All P0 tests pass
- [ ] No high-risk (≥6) items unmitigated
- [ ] Security tests (SEC category) pass 100%
- [ ] Performance targets met (PERF category)
---
## Mitigation Plans
### R-001: {Risk Description} (Score: 6)
**Mitigation Strategy:** {detailed_mitigation}
**Owner:** {owner}
**Timeline:** {date}
**Status:** Planned / In Progress / Complete
**Verification:** {how_to_verify}
### R-002: {Risk Description} (Score: 6)
**Mitigation Strategy:** {detailed_mitigation}
**Owner:** {owner}
**Timeline:** {date}
**Status:** Planned / In Progress / Complete
**Verification:** {how_to_verify}
---
## Assumptions and Dependencies
### Assumptions
1. {assumption}
2. {assumption}
3. {assumption}
### Dependencies
1. {dependency} - Required by {date}
2. {dependency} - Required by {date}
### Risks to Plan
- **Risk**: {risk_to_plan}
- **Impact**: {impact}
- **Contingency**: {contingency}
---
---
## Follow-on Workflows (Manual)
- Run `*atdd` to generate failing P0 tests (separate workflow; not auto-run).
- Run `*automate` for broader coverage once implementation exists.
---
## Approval
**Test Design Approved By:**
- [ ] Product Manager: {name} Date: {date}
- [ ] Tech Lead: {name} Date: {date}
- [ ] QA Lead: {name} Date: {date}
**Comments:**
---
---
---
## Appendix
### Knowledge Base References
- `risk-governance.md` - Risk classification framework
- `probability-impact.md` - Risk scoring methodology
- `test-levels-framework.md` - Test level selection
- `test-priorities-matrix.md` - P0-P3 prioritization
### Related Documents
- PRD: {prd_link}
- Epic: {epic_link}
- Architecture: {arch_link}
- Tech Spec: {tech_spec_link}
---
**Generated by**: BMad TEA Agent - Test Architect Module
**Workflow**: `_bmad/bmm/testarch/test-design`
**Version**: 4.0 (BMad v6)
@@ -0,0 +1,69 @@
# Test Architect workflow: test-design
name: testarch-test-design
description: "Dual-mode workflow: (1) System-level testability review in Solutioning phase, or (2) Epic-level test planning in Implementation phase. Auto-detects mode based on project phase."
author: "BMad"
# Critical variables from config
config_source: "{project-root}/_bmad/bmm/config.yaml"
output_folder: "{config_source}:output_folder"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/bmm/workflows/testarch/test-design"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
# Note: Template selection is mode-based (see instructions.md Step 1.5):
# - System-level: test-design-architecture-template.md + test-design-qa-template.md
# - Epic-level: test-design-template.md (unchanged)
template: "{installed_path}/test-design-template.md"
# Variables and inputs
variables:
design_level: "full" # full, targeted, minimal - scope of design effort
mode: "auto-detect" # auto-detect (default), system-level, epic-level
# Output configuration
# Note: Actual output file determined dynamically based on mode detection
# Declared outputs for new workflow format
outputs:
# System-Level Mode (Phase 3) - TWO documents
- id: test-design-architecture
description: "System-level test architecture: Architectural concerns, testability gaps, NFR requirements for Architecture/Dev teams"
path: "{output_folder}/test-design-architecture.md"
mode: system-level
audience: architecture
- id: test-design-qa
description: "System-level test design: Test execution recipe, coverage plan, Sprint 0 setup for QA team"
path: "{output_folder}/test-design-qa.md"
mode: system-level
audience: qa
# Epic-Level Mode (Phase 4) - ONE document (unchanged)
- id: epic-level
description: "Epic-level test plan (Phase 4)"
path: "{output_folder}/test-design-epic-{epic_num}.md"
mode: epic-level
# Note: No default_output_file - mode detection determines which outputs to write
# Required tools
required_tools:
- read_file # Read PRD, epics, stories, architecture docs
- write_file # Create test design document
- list_files # Find related documentation
- search_repo # Search for existing tests and patterns
tags:
- qa
- planning
- test-architect
- risk-assessment
- coverage
execution_hints:
interactive: false # Minimize prompts
autonomous: true # Proceed without user input unless blocked
iterative: true