feat: Add a foundational set of BMM, BMB, TEA, and Core workflows, agent definitions, and supporting templates.

This commit is contained in:
Michael committed 2026-02-20 17:22:00 +01:00
commit 6a0ddd7e36
724 files changed
+110361

No files matched your search

+74
View File
@@ -0,0 +1,74 @@
# TEA Workflow Step Files
This folder contains the Test Architect (TEA) workflows converted to step-file architecture for strict LLM compliance. Each workflow is tri-modal (create, edit, validate) and uses small, ordered step files instead of a single monolithic instruction file.
## Why Step Files
- Enforces sequential execution and prevents improvisation
- Keeps context small and focused per step
- Makes validation and edits deterministic
## Standard Layout (per workflow)
```
<workflow>/
├── workflow.md # Mode routing (create / edit / validate)
├── workflow-plan.md # Design reference for step order and intent
├── workflow.yaml # Installer metadata
├── instructions.md # Short entrypoint / summary
├── checklist.md # Validation criteria for outputs
├── steps-c/ # Create mode steps
├── steps-e/ # Edit mode steps
├── steps-v/ # Validate mode steps
├── templates/ # Output templates (if applicable)
└── validation-report-*.md # Validator outputs (latest run)
```
## Modes
- **Create (steps-c/):** Primary execution flow to generate outputs
- **Edit (steps-e/):** Structured edits to existing outputs
- **Validate (steps-v/):** Checklist-based validation of outputs
## Execution Rules (Summary)
- Load **one step at a time**. Do not preload future steps.
- Follow the **MANDATORY SEQUENCE** exactly in each step.
- Do not skip steps, reorder, or improvise.
- If a step writes outputs, do so **before** loading the next step.
## Step Naming Conventions
- `step-01-*.md` is the init step (no menus unless explicitly required).
- `step-01b-*.md` is a continuation/resume step if the workflow is continuable.
- `step-0X-*.md` are sequential create-mode steps.
- `steps-v/step-01-validate.md` is the validate mode entrypoint.
- `steps-e/step-01-assess.md` is the edit mode entrypoint.
## Validation
- Each workflow has a latest `validation-report-*.md` in its folder.
- Validation uses the BMad Builder workflow validator (workflow-builder).
- The goal is 100% compliance with no warnings.
## References
- Step-file architecture: `docs/explanation/step-file-architecture.md`
- Subprocess patterns: `docs/explanation/subprocess-architecture.md`
## TEA Workflows
- test-design
- automate
- atdd
- test-review
- trace
- framework
- ci
- nfr-assess
## Notes
- `workflow.md` is the canonical entrypoint. `instructions.md` is a short summary for quick context.
- Output files typically use `{test_artifacts}` or `{project-root}` variables.
- If a workflow produces multiple artifacts (e.g., system-level vs epic-level), the step file will specify which templates and output paths to use.
@@ -0,0 +1,369 @@
---
stepsCompleted: []
lastStep: ''
lastSaved: ''
---
# ATDD Checklist - Epic {epic_num}, Story {story_num}: {story_title}
**Date:** {date}
**Author:** {user_name}
**Primary Test Level:** {primary_level}
---
## Story Summary
{Brief 2-3 sentence summary of the user story}
**As a** {user_role}
**I want** {feature_description}
**So that** {business_value}
---
## Acceptance Criteria
{List all testable acceptance criteria from the story}
1. {Acceptance criterion 1}
2. {Acceptance criterion 2}
3. {Acceptance criterion 3}
---
## Failing Tests Created (RED Phase)
### E2E Tests ({e2e_test_count} tests)
**File:** `{e2e_test_file_path}` ({line_count} lines)
{List each E2E test with its current status and expected failure reason}
- ✅ **Test:** {test_name}
- **Status:** RED - {failure_reason}
- **Verifies:** {what_this_test_validates}
### API Tests ({api_test_count} tests)
**File:** `{api_test_file_path}` ({line_count} lines)
{List each API test with its current status and expected failure reason}
- ✅ **Test:** {test_name}
- **Status:** RED - {failure_reason}
- **Verifies:** {what_this_test_validates}
### Component Tests ({component_test_count} tests)
**File:** `{component_test_file_path}` ({line_count} lines)
{List each component test with its current status and expected failure reason}
- ✅ **Test:** {test_name}
- **Status:** RED - {failure_reason}
- **Verifies:** {what_this_test_validates}
---
## Data Factories Created
{List all data factory files created with their exports}
### {Entity} Factory
**File:** `tests/support/factories/{entity}.factory.ts`
**Exports:**
- `create{Entity}(overrides?)` - Create single entity with optional overrides
- `create{Entity}s(count)` - Create array of entities
**Example Usage:**
```typescript
const user = createUser({ email: 'specific@example.com' });
const users = createUsers(5); // Generate 5 random users
```
---
## Fixtures Created
{List all test fixture files created with their fixture names and descriptions}
### {Feature} Fixtures
**File:** `tests/support/fixtures/{feature}.fixture.ts`
**Fixtures:**
- `{fixtureName}` - {description_of_what_fixture_provides}
- **Setup:** {what_setup_does}
- **Provides:** {what_test_receives}
- **Cleanup:** {what_cleanup_does}
**Example Usage:**
```typescript
import { test } from './fixtures/{feature}.fixture';
test('should do something', async ({ {fixtureName} }) => {
// {fixtureName} is ready to use with auto-cleanup
});
```
---
## Mock Requirements
{Document external services that need mocking and their requirements}
### {Service Name} Mock
**Endpoint:** `{HTTP_METHOD} {endpoint_url}`
**Success Response:**
```json
{
{success_response_example}
}
```
**Failure Response:**
```json
{
{failure_response_example}
}
```
**Notes:** {any_special_mock_requirements}
---
## Required data-testid Attributes
{List all data-testid attributes required in UI implementation for test stability}
### {Page or Component Name}
- `{data-testid-name}` - {description_of_element}
- `{data-testid-name}` - {description_of_element}
**Implementation Example:**
```tsx
<button data-testid="login-button">Log In</button>
<input data-testid="email-input" type="email" />
<div data-testid="error-message">{errorText}</div>
```
---
## Implementation Checklist
{Map each failing test to concrete implementation tasks that will make it pass}
### Test: {test_name_1}
**File:** `{test_file_path}`
**Tasks to make this test pass:**
- [ ] {Implementation task 1}
- [ ] {Implementation task 2}
- [ ] {Implementation task 3}
- [ ] Add required data-testid attributes: {list_of_testids}
- [ ] Run test: `{test_execution_command}`
- [ ] ✅ Test passes (green phase)
**Estimated Effort:** {effort_estimate} hours
---
### Test: {test_name_2}
**File:** `{test_file_path}`
**Tasks to make this test pass:**
- [ ] {Implementation task 1}
- [ ] {Implementation task 2}
- [ ] {Implementation task 3}
- [ ] Add required data-testid attributes: {list_of_testids}
- [ ] Run test: `{test_execution_command}`
- [ ] ✅ Test passes (green phase)
**Estimated Effort:** {effort_estimate} hours
---
## Running Tests
```bash
# Run all failing tests for this story
{test_command_all}
# Run specific test file
{test_command_specific_file}
# Run tests in headed mode (see browser)
{test_command_headed}
# Debug specific test
{test_command_debug}
# Run tests with coverage
{test_command_coverage}
```
---
## Red-Green-Refactor Workflow
### RED Phase (Complete) ✅
**TEA Agent Responsibilities:**
- ✅ All tests written and failing
- ✅ Fixtures and factories created with auto-cleanup
- ✅ Mock requirements documented
- ✅ data-testid requirements listed
- ✅ Implementation checklist created
**Verification:**
- All tests run and fail as expected
- Failure messages are clear and actionable
- Tests fail due to missing implementation, not test bugs
---
### GREEN Phase (DEV Team - Next Steps)
**DEV Agent Responsibilities:**
1. **Pick one failing test** from implementation checklist (start with highest priority)
2. **Read the test** to understand expected behavior
3. **Implement minimal code** to make that specific test pass
4. **Run the test** to verify it now passes (green)
5. **Check off the task** in implementation checklist
6. **Move to next test** and repeat
**Key Principles:**
- One test at a time (don't try to fix all at once)
- Minimal implementation (don't over-engineer)
- Run tests frequently (immediate feedback)
- Use implementation checklist as roadmap
**Progress Tracking:**
- Check off tasks as you complete them
- Share progress in daily standup
---
### REFACTOR Phase (DEV Team - After All Tests Pass)
**DEV Agent Responsibilities:**
1. **Verify all tests pass** (green phase complete)
2. **Review code for quality** (readability, maintainability, performance)
3. **Extract duplications** (DRY principle)
4. **Optimize performance** (if needed)
5. **Ensure tests still pass** after each refactor
6. **Update documentation** (if API contracts change)
**Key Principles:**
- Tests provide safety net (refactor with confidence)
- Make small refactors (easier to debug if tests fail)
- Run tests after each change
- Don't change test behavior (only implementation)
**Completion:**
- All tests pass
- Code quality meets team standards
- No duplications or code smells
- Ready for code review and story approval
---
## Next Steps
1. **Share this checklist and failing tests** with the dev workflow (manual handoff)
2. **Review this checklist** with team in standup or planning
3. **Run failing tests** to confirm RED phase: `{test_command_all}`
4. **Begin implementation** using implementation checklist as guide
5. **Work one test at a time** (red → green for each)
6. **Share progress** in daily standup
7. **When all tests pass**, refactor code for quality
8. **When refactoring complete**, manually update story status to 'done' in sprint-status.yaml
---
## Knowledge Base References Applied
This ATDD workflow consulted the following knowledge fragments:
- **fixture-architecture.md** - Test fixture patterns with setup/teardown and auto-cleanup using Playwright's `test.extend()`
- **data-factories.md** - Factory patterns using `@faker-js/faker` for random test data generation with overrides support
- **component-tdd.md** - Component test strategies using Playwright Component Testing
- **network-first.md** - Route interception patterns (intercept BEFORE navigation to prevent race conditions)
- **test-quality.md** - Test design principles (Given-When-Then, one assertion per test, determinism, isolation)
- **test-levels-framework.md** - Test level selection framework (E2E vs API vs Component vs Unit)
See `tea-index.csv` for complete knowledge fragment mapping.
---
## Test Execution Evidence
### Initial Test Run (RED Phase Verification)
**Command:** `{test_command_all}`
**Results:**
```
{paste_test_run_output_showing_all_tests_failing}
```
**Summary:**
- Total tests: {total_test_count}
- Passing: 0 (expected)
- Failing: {total_test_count} (expected)
- Status: ✅ RED phase verified
**Expected Failure Messages:**
{list_expected_failure_messages_for_each_test}
---
## Notes
{Any additional notes, context, or special considerations for this story}
- {Note 1}
- {Note 2}
- {Note 3}
---
## Contact
**Questions or Issues?**
- Ask in team standup
- Tag @{tea_agent_username} in Slack/Discord
- Refer to `./bmm/docs/tea-README.md` for workflow documentation
- Consult `./bmm/testarch/knowledge` for testing best practices
---
**Generated by BMad TEA Agent** - {date}
@@ -0,0 +1,374 @@
# ATDD Workflow Validation Checklist
Use this checklist to validate that the ATDD workflow has been executed correctly and all deliverables meet quality standards.
## Prerequisites
Before starting this workflow, verify:
- [ ] Story approved with clear acceptance criteria (AC must be testable)
- [ ] Development sandbox/environment ready
- [ ] Framework scaffolding exists (run `framework` workflow if missing)
- [ ] Test framework configuration available (playwright.config.ts or cypress.config.ts)
- [ ] Package.json has test dependencies installed (Playwright or Cypress)
**Halt if missing:** Framework scaffolding or story acceptance criteria
---
## Step 1: Story Context and Requirements
- [ ] Story markdown file loaded and parsed successfully
- [ ] All acceptance criteria identified and extracted
- [ ] Affected systems and components identified
- [ ] Technical constraints documented
- [ ] Framework configuration loaded (playwright.config.ts or cypress.config.ts)
- [ ] Test directory structure identified from config
- [ ] Existing fixture patterns reviewed for consistency
- [ ] Similar test patterns searched and found in `{test_dir}`
- [ ] Knowledge base fragments loaded:
- [ ] `fixture-architecture.md`
- [ ] `data-factories.md`
- [ ] `component-tdd.md`
- [ ] `network-first.md`
- [ ] `test-quality.md`
---
## Step 2: Test Level Selection and Strategy
- [ ] Each acceptance criterion analyzed for appropriate test level
- [ ] Test level selection framework applied (E2E vs API vs Component vs Unit)
- [ ] E2E tests: Critical user journeys and multi-system integration identified
- [ ] API tests: Business logic and service contracts identified
- [ ] Component tests: UI component behavior and interactions identified
- [ ] Unit tests: Pure logic and edge cases identified (if applicable)
- [ ] Duplicate coverage avoided (same behavior not tested at multiple levels unnecessarily)
- [ ] Tests prioritized using P0-P3 framework (if test-design document exists)
- [ ] Primary test level set in `primary_level` variable (typically E2E or API)
- [ ] Test levels documented in ATDD checklist
---
## Step 3: Failing Tests Generated
### Test File Structure Created
- [ ] Test files organized in appropriate directories:
- [ ] `tests/e2e/` for end-to-end tests
- [ ] `tests/api/` for API tests
- [ ] `tests/component/` for component tests
- [ ] `tests/support/` for infrastructure (fixtures, factories, helpers)
### E2E Tests (If Applicable)
- [ ] E2E test files created in `tests/e2e/`
- [ ] All tests follow Given-When-Then format
- [ ] Tests use `data-testid` selectors (not CSS classes or fragile selectors)
- [ ] One assertion per test (atomic test design)
- [ ] No hard waits or sleeps (explicit waits only)
- [ ] Network-first pattern applied (route interception BEFORE navigation)
- [ ] Tests fail initially (RED phase verified by local test run)
- [ ] Failure messages are clear and actionable
### API Tests (If Applicable)
- [ ] API test files created in `tests/api/`
- [ ] Tests follow Given-When-Then format
- [ ] API contracts validated (request/response structure)
- [ ] HTTP status codes verified
- [ ] Response body validation includes all required fields
- [ ] Error cases tested (400, 401, 403, 404, 500)
- [ ] Tests fail initially (RED phase verified)
### Component Tests (If Applicable)
- [ ] Component test files created in `tests/component/`
- [ ] Tests follow Given-When-Then format
- [ ] Component mounting works correctly
- [ ] Interaction testing covers user actions (click, hover, keyboard)
- [ ] State management within component validated
- [ ] Props and events tested
- [ ] Tests fail initially (RED phase verified)
### Test Quality Validation
- [ ] All tests use Given-When-Then structure with clear comments
- [ ] All tests have descriptive names explaining what they test
- [ ] No duplicate tests (same behavior tested multiple times)
- [ ] No flaky patterns (race conditions, timing issues)
- [ ] No test interdependencies (tests can run in any order)
- [ ] Tests are deterministic (same input always produces same result)
---
## Step 4: Data Infrastructure Built
### Data Factories Created
- [ ] Factory files created in `tests/support/factories/`
- [ ] All factories use `@faker-js/faker` for random data generation (no hardcoded values)
- [ ] Factories support overrides for specific test scenarios
- [ ] Factories generate complete valid objects matching API contracts
- [ ] Helper functions for bulk creation provided (e.g., `createUsers(count)`)
- [ ] Factory exports are properly typed (TypeScript)
### Test Fixtures Created
- [ ] Fixture files created in `tests/support/fixtures/`
- [ ] All fixtures use Playwright's `test.extend()` pattern
- [ ] Fixtures have setup phase (arrange test preconditions)
- [ ] Fixtures provide data to tests via `await use(data)`
- [ ] Fixtures have teardown phase with auto-cleanup (delete created data)
- [ ] Fixtures are composable (can use other fixtures if needed)
- [ ] Fixtures are isolated (each test gets fresh data)
- [ ] Fixtures are type-safe (TypeScript types defined)
### Mock Requirements Documented
- [ ] External service mocking requirements identified
- [ ] Mock endpoints documented with URLs and methods
- [ ] Success response examples provided
- [ ] Failure response examples provided
- [ ] Mock requirements documented in ATDD checklist for DEV team
### data-testid Requirements Listed
- [ ] All required data-testid attributes identified from E2E tests
- [ ] data-testid list organized by page or component
- [ ] Each data-testid has clear description of element it targets
- [ ] data-testid list included in ATDD checklist for DEV team
---
## Step 5: Implementation Checklist Created
- [ ] Implementation checklist created with clear structure
- [ ] Each failing test mapped to concrete implementation tasks
- [ ] Tasks include:
- [ ] Route/component creation
- [ ] Business logic implementation
- [ ] API integration
- [ ] data-testid attribute additions
- [ ] Error handling
- [ ] Test execution command
- [ ] Completion checkbox
- [ ] Red-Green-Refactor workflow documented in checklist
- [ ] RED phase marked as complete (TEA responsibility)
- [ ] GREEN phase tasks listed for DEV team
- [ ] REFACTOR phase guidance provided
- [ ] Execution commands provided:
- [ ] Run all tests: `npm run test:e2e`
- [ ] Run specific test file
- [ ] Run in headed mode
- [ ] Debug specific test
- [ ] Estimated effort included (hours or story points)
---
## Step 6: Deliverables Generated
### ATDD Checklist Document Created
- [ ] Output file created at `{test_artifacts}/atdd-checklist-{story_id}.md`
- [ ] Document follows template structure from `atdd-checklist-template.md`
- [ ] Document includes all required sections:
- [ ] Story summary
- [ ] Acceptance criteria breakdown
- [ ] Failing tests created (paths and line counts)
- [ ] Data factories created
- [ ] Fixtures created
- [ ] Mock requirements
- [ ] Required data-testid attributes
- [ ] Implementation checklist
- [ ] Red-green-refactor workflow
- [ ] Execution commands
- [ ] Next steps for DEV team
- [ ] Output shared with DEV workflow (manual handoff; not auto-consumed)
### All Tests Verified to Fail (RED Phase)
- [ ] Full test suite run locally before finalizing
- [ ] All tests fail as expected (RED phase confirmed)
- [ ] No tests passing before implementation (if passing, test is invalid)
- [ ] Failure messages documented in ATDD checklist
- [ ] Failures are due to missing implementation, not test bugs
- [ ] Test run output captured for reference
### Summary Provided
- [ ] Summary includes:
- [ ] Story ID
- [ ] Primary test level
- [ ] Test counts (E2E, API, Component)
- [ ] Test file paths
- [ ] Factory count
- [ ] Fixture count
- [ ] Mock requirements count
- [ ] data-testid count
- [ ] Implementation task count
- [ ] Estimated effort
- [ ] Next steps for DEV team
- [ ] Output file path
- [ ] Knowledge base references applied
---
## Quality Checks
### Test Design Quality
- [ ] Tests are readable (clear Given-When-Then structure)
- [ ] Tests are maintainable (use factories and fixtures, not hardcoded data)
- [ ] Tests are isolated (no shared state between tests)
- [ ] Tests are deterministic (no race conditions or flaky patterns)
- [ ] Tests are atomic (one assertion per test)
- [ ] Tests are fast (no unnecessary waits or delays)
### Knowledge Base Integration
- [ ] fixture-architecture.md patterns applied to all fixtures
- [ ] data-factories.md patterns applied to all factories
- [ ] network-first.md patterns applied to E2E tests with network requests
- [ ] component-tdd.md patterns applied to component tests
- [ ] test-quality.md principles applied to all test design
### Code Quality
- [ ] All TypeScript types are correct and complete
- [ ] No linting errors in generated test files
- [ ] Consistent naming conventions followed
- [ ] Imports are organized and correct
- [ ] Code follows project style guide
---
## Integration Points
### With DEV Agent
- [ ] ATDD checklist provides clear implementation guidance
- [ ] Implementation tasks are granular and actionable
- [ ] data-testid requirements are complete and clear
- [ ] Mock requirements include all necessary details
- [ ] Execution commands work correctly
### With Story Workflow
- [ ] Story ID correctly referenced in output files
- [ ] Acceptance criteria from story accurately reflected in tests
- [ ] Technical constraints from story considered in test design
### With Framework Workflow
- [ ] Test framework configuration correctly detected and used
- [ ] Directory structure matches framework setup
- [ ] Fixtures and helpers follow established patterns
- [ ] Naming conventions consistent with framework standards
### With test-design Workflow (If Available)
- [ ] P0 scenarios from test-design prioritized in ATDD
- [ ] Risk assessment from test-design considered in test coverage
- [ ] Coverage strategy from test-design aligned with ATDD tests
---
## Completion Criteria
All of the following must be true before marking this workflow as complete:
- [ ] **Story acceptance criteria analyzed** and mapped to appropriate test levels
- [ ] **Failing tests created** at all appropriate levels (E2E, API, Component)
- [ ] **Given-When-Then format** used consistently across all tests
- [ ] **RED phase verified** by local test run (all tests failing as expected)
- [ ] **Network-first pattern** applied to E2E tests with network requests
- [ ] **Data factories created** using faker (no hardcoded test data)
- [ ] **Fixtures created** with auto-cleanup in teardown
- [ ] **Mock requirements documented** for external services
- [ ] **data-testid attributes listed** for DEV team
- [ ] **Implementation checklist created** mapping tests to code tasks
- [ ] **Red-green-refactor workflow documented** in ATDD checklist
- [ ] **Execution commands provided** and verified to work
- [ ] **ATDD checklist document created** and saved to correct location
- [ ] **Output file formatted correctly** using template structure
- [ ] **Knowledge base references applied** and documented in summary
- [ ] **No test quality issues** (flaky patterns, race conditions, hardcoded data)
---
## Common Issues and Resolutions
### Issue: Tests pass before implementation
**Problem:** A test passes even though no implementation code exists yet.
**Resolution:**
- Review test to ensure it's testing actual behavior, not mocked/stubbed behavior
- Check if test is accidentally using existing functionality
- Verify test assertions are correct and meaningful
- Rewrite test to fail until implementation is complete
### Issue: Network-first pattern not applied
**Problem:** Route interception happens after navigation, causing race conditions.
**Resolution:**
- Move `await page.route()` calls BEFORE `await page.goto()`
- Review `network-first.md` knowledge fragment
- Update all E2E tests to follow network-first pattern
### Issue: Hardcoded test data in tests
**Problem:** Tests use hardcoded strings/numbers instead of factories.
**Resolution:**
- Replace all hardcoded data with factory function calls
- Use `faker` for all random data generation
- Update data-factories to support all required test scenarios
### Issue: Fixtures missing auto-cleanup
**Problem:** Fixtures create data but don't clean it up in teardown.
**Resolution:**
- Add cleanup logic after `await use(data)` in fixture
- Call deletion/cleanup functions in teardown
- Verify cleanup works by checking database/storage after test run
### Issue: Tests have multiple assertions
**Problem:** Tests verify multiple behaviors in single test (not atomic).
**Resolution:**
- Split into separate tests (one assertion per test)
- Each test should verify exactly one behavior
- Use descriptive test names to clarify what each test verifies
### Issue: Tests depend on execution order
**Problem:** Tests fail when run in isolation or different order.
**Resolution:**
- Remove shared state between tests
- Each test should create its own test data
- Use fixtures for consistent setup across tests
- Verify tests can run with `.only` flag
---
## Notes for TEA Agent
- **Preflight halt is critical:** Do not proceed if story has no acceptance criteria or framework is missing
- **RED phase verification is mandatory:** Tests must fail before sharing with DEV team
- **Network-first pattern:** Route interception BEFORE navigation prevents race conditions
- **One assertion per test:** Atomic tests provide clear failure diagnosis
- **Auto-cleanup is non-negotiable:** Every fixture must clean up data in teardown
- **Use knowledge base:** Load relevant fragments (fixture-architecture, data-factories, network-first, component-tdd, test-quality) for guidance
- **Share with DEV agent:** ATDD checklist provides implementation roadmap from red to green
@@ -0,0 +1,45 @@
<!-- Powered by BMAD-CORE™ -->
# Acceptance Test-Driven Development (ATDD)
**Workflow ID**: `_bmad/tea/testarch/atdd`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Generates **failing acceptance tests** before implementation (TDD red phase), plus an implementation checklist. Produces tests at appropriate levels (E2E/API/Component) with supporting fixtures and helpers.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order without skipping
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `test_dir`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/atdd/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.
@@ -0,0 +1,143 @@
---
name: 'step-01-preflight-and-context'
description: 'Verify prerequisites and load story, framework, and knowledge base'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-02-generation-mode.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
---
# Step 1: Preflight & Context Loading
## STEP GOAL
Verify prerequisites and load all required inputs before generating failing tests.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if requirements are missing
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Prerequisites (Hard Requirements)
- Story approved with **clear acceptance criteria**
- Test framework configured (`playwright.config.ts` or `cypress.config.ts`)
- Development environment available
If any are missing: **HALT** and notify the user.
---
## 2. Load Story Context
- Read story markdown from `{story_file}` (or ask user if not provided)
- Extract acceptance criteria and constraints
- Identify affected components and integrations
---
## 3. Load Framework & Existing Patterns
- Read framework config
- Inspect `{test_dir}` for existing test patterns, fixtures, helpers
## 3.5 Read TEA Config Flags
From `{config_source}`:
- `tea_use_playwright_utils`
- `tea_browser_automation`
---
## 4. Load Knowledge Base Fragments
Use `{knowledgeIndex}` to load:
**Core (always):**
- `data-factories.md`
- `component-tdd.md`
- `test-quality.md`
- `test-healing-patterns.md`
- `selector-resilience.md`
- `timing-debugging.md`
**Playwright Utils (if enabled):**
- `overview.md`, `api-request.md`, `network-recorder.md`, `auth-session.md`, `intercept-network-call.md`, `recurse.md`, `log.md`, `file-utils.md`, `network-error-monitor.md`, `fixtures-composition.md`
**Playwright CLI (if tea_browser_automation is "cli" or "auto"):**
- `playwright-cli.md`
**MCP Patterns (if tea_browser_automation is "mcp" or "auto"):**
- (existing MCP-related fragments, if any are added in future)
**Traditional Patterns (if utils disabled):**
- `fixture-architecture.md`
- `network-first.md`
---
## 5. Confirm Inputs
Summarize loaded inputs and confirm with the user. Then proceed.
---
## 6. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight-and-context']
lastStep: 'step-01-preflight-and-context'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight-and-context'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight-and-context'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,96 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Display Progress Dashboard
Display progress with ✅/⬜ indicators:
1. ✅/⬜ Preflight & Context (step-01-preflight-and-context)
2. ✅/⬜ Generation Mode (step-02-generation-mode)
3. ✅/⬜ Test Strategy (step-03-test-strategy)
4. ✅/⬜ Generate Tests + Aggregate (step-04c-aggregate)
5. ✅/⬜ Validate & Complete (step-05-validate-and-complete)
---
### 3. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight-and-context'` → load `./step-02-generation-mode.md`
- `'step-02-generation-mode'` → load `./step-03-test-strategy.md`
- `'step-03-test-strategy'` → load `./step-04-generate-tests.md`
- `'step-04c-aggregate'` → load `./step-05-validate-and-complete.md`
- `'step-05-validate-and-complete'` → **Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
@@ -0,0 +1,120 @@
---
name: 'step-02-generation-mode'
description: 'Choose AI generation or recording mode'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-03-test-strategy.md'
---
# Step 2: Generation Mode Selection
## STEP GOAL
Choose the appropriate generation mode for ATDD tests.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Default Mode: AI Generation
Use AI generation when:
- Acceptance criteria are clear
- Scenarios are standard (CRUD, auth, API, navigation)
Proceed directly to test strategy if this applies.
---
## 2. Optional Mode: Recording (Complex UI)
Use recording when UI interactions need live browser verification.
**Tool selection based on `config.tea_browser_automation`:**
If `auto`:
> **Note:** `${timestamp}` is a placeholder the agent should replace with a unique value (e.g., epoch seconds) for session isolation.
- **Simple recording** (snapshot selectors, capture structure): Use CLI
- `playwright-cli -s=tea-atdd-${timestamp} open <url>` → `playwright-cli -s=tea-atdd-${timestamp} snapshot` → extract refs
- **Complex recording** (drag/drop, wizards, multi-step state): Use MCP
- Full browser automation with rich tool semantics
- **Fallback:** If preferred tool unavailable, use the other; if neither, skip recording
If `cli`:
- Use Playwright CLI for all recording
- `playwright-cli -s=tea-atdd-${timestamp} open <url>`, `snapshot`, `screenshot`, `click <ref>`, etc.
If `mcp`:
- Use Playwright MCP tools for all recording (current behavior)
- Confirm MCP availability, record selectors and interactions
If `none`:
- Skip recording mode entirely, use AI generation from documentation
---
## 3. Confirm Mode
State the chosen mode and why. Then proceed.
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-generation-mode']
lastStep: 'step-02-generation-mode'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-generation-mode'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-generation-mode'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,101 @@
---
name: 'step-03-test-strategy'
description: 'Map acceptance criteria to test levels and priorities'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-04-generate-tests.md'
---
# Step 3: Test Strategy
## STEP GOAL
Translate acceptance criteria into a prioritized, level-appropriate test plan.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Avoid duplicate coverage across levels
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Map Acceptance Criteria
- Convert each acceptance criterion into test scenarios
- Include negative and edge cases where risk is high
---
## 2. Select Test Levels
Choose the best level per scenario:
- **E2E** for critical user journeys
- **API** for business logic and service contracts
- **Component** for UI behavior
---
## 3. Prioritize Tests
Assign P0–P3 priorities using risk and business impact.
---
## 4. Confirm Red Phase Requirements
Ensure all tests are designed to **fail before implementation** (TDD red phase).
---
## 5. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-test-strategy']
lastStep: 'step-03-test-strategy'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-test-strategy'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-test-strategy'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,229 @@
---
name: 'step-04-generate-tests'
description: 'Orchestrate parallel FAILING test generation (TDD red phase)'
nextStepFile: './step-04c-aggregate.md'
---
# Step 4: Orchestrate Parallel FAILING Test Generation
## STEP GOAL
Launch parallel subprocesses to generate FAILING API and E2E tests simultaneously (TDD RED PHASE) for maximum performance.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Launch TWO subprocesses in PARALLEL
- ✅ Generate FAILING tests only (TDD red phase)
- ✅ Wait for BOTH subprocesses to complete
- ❌ Do NOT generate sequential tests (use subprocesses)
- ❌ Do NOT generate passing tests (this is red phase)
- ❌ Do NOT proceed until both subprocesses finish
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Wait for subprocess outputs
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, acceptance criteria from Step 1, test strategy from Step 3
- Focus: subprocess orchestration only
- Limits: do not generate tests directly (delegate to subprocesses)
- Dependencies: Steps 1-3 outputs
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Prepare Subprocess Inputs
**Generate unique timestamp** for temp file naming:
```javascript
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
```
**Prepare input context for both subprocesses:**
```javascript
const subprocessContext = {
story_acceptance_criteria: /* from Step 1 */,
test_strategy: /* from Step 3 */,
knowledge_fragments_loaded: /* list of fragments */,
config: {
test_framework: config.test_framework,
use_playwright_utils: config.tea_use_playwright_utils,
browser_automation: config.tea_browser_automation
},
timestamp: timestamp
};
```
---
### 2. Launch Subprocess A: Failing API Test Generation
**Launch subprocess in parallel:**
- **Subprocess File:** `./step-04a-subprocess-api-failing.md`
- **Output File:** `/tmp/tea-atdd-api-tests-${timestamp}.json`
- **Context:** Pass `subprocessContext`
- **Execution:** PARALLEL (non-blocking)
- **TDD Phase:** RED (failing tests)
**System Action:**
```
🚀 Launching Subprocess A: FAILING API Test Generation (RED PHASE)
📝 Output: /tmp/tea-atdd-api-tests-${timestamp}.json
🔴 TDD Phase: RED (tests will fail until feature implemented)
⏳ Status: Running in parallel...
```
---
### 3. Launch Subprocess B: Failing E2E Test Generation
**Launch subprocess in parallel:**
- **Subprocess File:** `./step-04b-subprocess-e2e-failing.md`
- **Output File:** `/tmp/tea-atdd-e2e-tests-${timestamp}.json`
- **Context:** Pass `subprocessContext`
- **Execution:** PARALLEL (non-blocking)
- **TDD Phase:** RED (failing tests)
**System Action:**
```
🚀 Launching Subprocess B: FAILING E2E Test Generation (RED PHASE)
📝 Output: /tmp/tea-atdd-e2e-tests-${timestamp}.json
🔴 TDD Phase: RED (tests will fail until feature implemented)
⏳ Status: Running in parallel...
```
---
### 4. Wait for Both Subprocesses to Complete
**Monitor subprocess execution:**
```
⏳ Waiting for subprocesses to complete...
├── Subprocess A (API RED): Running... ⟳
└── Subprocess B (E2E RED): Running... ⟳
[... time passes ...]
├── Subprocess A (API RED): Complete ✅
└── Subprocess B (E2E RED): Complete ✅
✅ All subprocesses completed successfully!
```
**Verify both outputs exist:**
```javascript
const apiOutputExists = fs.existsSync(`/tmp/tea-atdd-api-tests-${timestamp}.json`);
const e2eOutputExists = fs.existsSync(`/tmp/tea-atdd-e2e-tests-${timestamp}.json`);
if (!apiOutputExists || !e2eOutputExists) {
throw new Error('One or both subprocess outputs missing!');
}
```
---
### 5. TDD Red Phase Report
**Display TDD status:**
```
🔴 TDD RED PHASE: Failing Tests Generated
✅ Both subprocesses completed:
- API Tests: Generated with test.skip()
- E2E Tests: Generated with test.skip()
📋 All tests assert EXPECTED behavior
📋 All tests will FAIL until feature implemented
📋 This is INTENTIONAL (TDD red phase)
Next: Aggregation will verify TDD compliance
```
---
### 6. Performance Report
**Display performance metrics:**
```
🚀 Performance Report:
- Execution Mode: PARALLEL (2 subprocesses)
- API Test Generation: ~X minutes
- E2E Test Generation: ~Y minutes
- Total Elapsed: ~max(X, Y) minutes
- Sequential Would Take: ~(X + Y) minutes
- Performance Gain: ~50% faster!
```
---
### 7. Proceed to Aggregation
**Load aggregation step:**
Load next step: `{nextStepFile}`
The aggregation step (4C) will:
- Read both subprocess outputs
- Verify TDD red phase compliance (all tests have test.skip())
- Write all test files to disk
- Generate ATDD checklist
- Calculate summary statistics
---
## EXIT CONDITION
Proceed to Step 4C (Aggregation) when:
- ✅ Subprocess A (API failing tests) completed successfully
- ✅ Subprocess B (E2E failing tests) completed successfully
- ✅ Both output files exist and are valid JSON
- ✅ TDD red phase status reported
**Do NOT proceed if:**
- ❌ One or both subprocesses failed
- ❌ Output files missing or corrupted
- ❌ Subprocess generated passing tests (wrong - must be failing)
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Both subprocesses launched successfully
- Both subprocesses completed without errors
- Output files generated and valid
- Tests generated with test.skip() (TDD red phase)
- Parallel execution achieved ~50% performance gain
### ❌ SYSTEM FAILURE:
- Failed to launch subprocesses
- One or both subprocesses failed
- Output files missing or invalid
- Tests generated without test.skip() (wrong phase)
- Attempted sequential generation instead of parallel
**Master Rule:** TDD RED PHASE requires FAILING tests (with test.skip()). Parallel subprocess execution is MANDATORY for performance.
@@ -0,0 +1,215 @@
---
name: 'step-04a-subprocess-api-failing'
description: 'Subprocess: Generate FAILING API tests (TDD red phase)'
subprocess: true
outputFile: '/tmp/tea-atdd-api-tests-{{timestamp}}.json'
---
# Subprocess 4A: Generate Failing API Tests (TDD Red Phase)
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with E2E failing test generation.
**What you have from parent workflow:**
- Story acceptance criteria from Step 1
- Test strategy and scenarios from Step 3
- Knowledge fragments loaded: api-request, data-factories, api-testing-patterns
- Config: test framework, Playwright Utils enabled/disabled
**Your task:** Generate API tests that will FAIL because the feature is not implemented yet (TDD RED PHASE).
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subprocess file before acting
- ✅ Generate FAILING API tests ONLY
- ✅ Tests MUST fail when run (feature not implemented yet)
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate E2E tests (that's subprocess 4B)
- ❌ Do NOT generate passing tests (this is TDD red phase)
- ❌ Do NOT run tests (that's step 5)
---
## SUBPROCESS TASK
### 1. Identify API Endpoints from Acceptance Criteria
From the story acceptance criteria (Step 1 output), identify:
- Which API endpoints will be created for this story
- Expected request/response contracts
- Authentication requirements
- Expected status codes and error scenarios
**Example Acceptance Criteria:**
```
Story: User Registration
- As a user, I can POST to /api/users/register with email and password
- System returns 201 Created with user object
- System returns 400 Bad Request if email already exists
- System returns 422 Unprocessable Entity if validation fails
```
### 2. Generate FAILING API Test Files
For each API endpoint, create test file in `tests/api/[feature].spec.ts`:
**Test Structure (ATDD - Red Phase):**
```typescript
import { test, expect } from '@playwright/test';
// If Playwright Utils enabled:
// import { apiRequest } from '@playwright-utils/api';
test.describe('[Story Name] API Tests (ATDD)', () => {
test.skip('[P0] should register new user successfully', async ({ request }) => {
// THIS TEST WILL FAIL - Endpoint not implemented yet
const response = await request.post('/api/users/register', {
data: {
email: 'newuser@example.com',
password: 'SecurePass123!',
},
});
// Expect 201 but will get 404 (endpoint doesn't exist)
expect(response.status()).toBe(201);
const user = await response.json();
expect(user).toMatchObject({
id: expect.any(Number),
email: 'newuser@example.com',
});
});
test.skip('[P1] should return 400 if email exists', async ({ request }) => {
// THIS TEST WILL FAIL - Endpoint not implemented yet
const response = await request.post('/api/users/register', {
data: {
email: 'existing@example.com',
password: 'SecurePass123!',
},
});
expect(response.status()).toBe(400);
const error = await response.json();
expect(error.message).toContain('Email already exists');
});
});
```
**CRITICAL ATDD Requirements:**
- ✅ Use `test.skip()` to mark tests as intentionally failing (red phase)
- ✅ Write assertions for EXPECTED behavior (even though not implemented)
- ✅ Use realistic test data (not placeholder data)
- ✅ Test both happy path and error scenarios from acceptance criteria
- ✅ Use `apiRequest()` helper if Playwright Utils enabled
- ✅ Use data factories for test data (from data-factories fragment)
- ✅ Include priority tags [P0], [P1], [P2], [P3]
**Why test.skip():**
- Tests are written correctly for EXPECTED behavior
- But we know they'll fail because feature isn't implemented
- `test.skip()` documents this is intentional (TDD red phase)
- Once feature is implemented, remove `test.skip()` to verify green phase
### 3. Track Fixture Needs
Identify fixtures needed for API tests:
- Authentication fixtures (if endpoints require auth)
- Data factories (user data, etc.)
- API client configurations
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-atdd-api-tests-{{timestamp}}.json`
```json
{
"success": true,
"subprocess": "atdd-api-tests",
"tests": [
{
"file": "tests/api/user-registration.spec.ts",
"content": "[full TypeScript test file content with test.skip()]",
"description": "ATDD API tests for user registration (RED PHASE)",
"expected_to_fail": true,
"acceptance_criteria_covered": [
"User can register with email/password",
"System returns 201 on success",
"System returns 400 if email exists"
],
"priority_coverage": {
"P0": 1,
"P1": 2,
"P2": 0,
"P3": 0
}
}
],
"fixture_needs": ["userDataFactory"],
"knowledge_fragments_used": ["api-request", "data-factories", "api-testing-patterns"],
"test_count": 3,
"tdd_phase": "RED",
"summary": "Generated 3 FAILING API tests for user registration story"
}
```
**On Error:**
```json
{
"success": false,
"subprocess": "atdd-api-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subprocess completes when:
- ✅ All API endpoints from acceptance criteria have test files
- ✅ All tests use `test.skip()` (documented failing tests)
- ✅ All tests assert EXPECTED behavior (not placeholder assertions)
- ✅ JSON output written to temp file
- ✅ Fixture needs tracked
**Subprocess terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBPROCESS SUCCESS METRICS
### ✅ SUCCESS:
- All API tests generated with test.skip()
- Tests assert expected behavior (not placeholders)
- JSON output valid and complete
- No E2E/component/unit tests included (out of scope)
- Tests follow knowledge fragment patterns
### ❌ FAILURE:
- Generated passing tests (wrong - this is RED phase)
- Tests without test.skip() (will break CI)
- Placeholder assertions (expect(true).toBe(true))
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output
@@ -0,0 +1,244 @@
---
name: 'step-04b-subprocess-e2e-failing'
description: 'Subprocess: Generate FAILING E2E tests (TDD red phase)'
subprocess: true
outputFile: '/tmp/tea-atdd-e2e-tests-{{timestamp}}.json'
---
# Subprocess 4B: Generate Failing E2E Tests (TDD Red Phase)
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with API failing test generation.
**What you have from parent workflow:**
- Story acceptance criteria from Step 1
- Test strategy and user journey scenarios from Step 3
- Knowledge fragments loaded: fixture-architecture, network-first, selector-resilience
- Config: test framework, Playwright Utils enabled/disabled
**Your task:** Generate E2E tests that will FAIL because the feature UI is not implemented yet (TDD RED PHASE).
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subprocess file before acting
- ✅ Generate FAILING E2E tests ONLY
- ✅ Tests MUST fail when run (UI not implemented yet)
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate API tests (that's subprocess 4A)
- ❌ Do NOT generate passing tests (this is TDD red phase)
- ❌ Do NOT run tests (that's step 5)
---
## SUBPROCESS TASK
### 1. Identify User Journeys from Acceptance Criteria
From the story acceptance criteria (Step 1 output), identify:
- Which UI flows will be created for this story
- User interactions required
- Expected visual states
- Success/error messages expected
**Example Acceptance Criteria:**
```
Story: User Registration
- As a user, I can navigate to /register page
- I can fill in email and password fields
- I can click "Register" button
- System shows success message and redirects to dashboard
- System shows error if email already exists
```
### 2. Browser Interaction (Selector Verification)
**Automation mode:** `config.tea_browser_automation`
If `auto` (fall back to MCP if CLI unavailable; if neither available, generate from best practices):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-atdd-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-atdd-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}` → `page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}` → `page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-atdd-{{timestamp}} close` when done
If `cli` (CLI only — do NOT fall back to MCP; generate from best practices if CLI unavailable):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-atdd-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-atdd-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}` → `page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}` → `page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-atdd-{{timestamp}} close` when done
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-atdd-{{timestamp}} close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
If `mcp`:
- Use MCP tools for selector verification (current behavior)
If `none`:
- Generate selectors from best practices without browser verification
### 3. Generate FAILING E2E Test Files
For each user journey, create test file in `tests/e2e/[feature].spec.ts`:
**Test Structure (ATDD - Red Phase):**
```typescript
import { test, expect } from '@playwright/test';
test.describe('[Story Name] E2E User Journey (ATDD)', () => {
test.skip('[P0] should complete user registration successfully', async ({ page }) => {
// THIS TEST WILL FAIL - UI not implemented yet
await page.goto('/register');
// Expect registration form but will get 404 or missing elements
await page.fill('[name="email"]', 'newuser@example.com');
await page.fill('[name="password"]', 'SecurePass123!');
await page.click('button:has-text("Register")');
// Expect success message and redirect
await expect(page.getByText('Registration successful!')).toBeVisible();
await page.waitForURL('/dashboard');
});
test.skip('[P1] should show error if email exists', async ({ page }) => {
// THIS TEST WILL FAIL - UI not implemented yet
await page.goto('/register');
await page.fill('[name="email"]', 'existing@example.com');
await page.fill('[name="password"]', 'SecurePass123!');
await page.click('button:has-text("Register")');
// Expect error message
await expect(page.getByText('Email already exists')).toBeVisible();
});
});
```
**CRITICAL ATDD Requirements:**
- ✅ Use `test.skip()` to mark tests as intentionally failing (red phase)
- ✅ Write assertions for EXPECTED UI behavior (even though not implemented)
- ✅ Use resilient selectors: getByRole, getByText, getByLabel (from selector-resilience)
- ✅ Follow network-first patterns if API calls involved (from network-first)
- ✅ Test complete user journeys from acceptance criteria
- ✅ Include priority tags [P0], [P1], [P2], [P3]
- ✅ Use proper TypeScript types
- ✅ Deterministic waits (no hard sleeps)
**Why test.skip():**
- Tests are written correctly for EXPECTED UI behavior
- But we know they'll fail because UI isn't implemented
- `test.skip()` documents this is intentional (TDD red phase)
- Once UI is implemented, remove `test.skip()` to verify green phase
### 4. Track Fixture Needs
Identify fixtures needed for E2E tests:
- Authentication fixtures (if journey requires logged-in state)
- Network mocks (if API calls involved)
- Test data fixtures
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-atdd-e2e-tests-{{timestamp}}.json`
```json
{
"success": true,
"subprocess": "atdd-e2e-tests",
"tests": [
{
"file": "tests/e2e/user-registration.spec.ts",
"content": "[full TypeScript test file content with test.skip()]",
"description": "ATDD E2E tests for user registration journey (RED PHASE)",
"expected_to_fail": true,
"acceptance_criteria_covered": [
"User can navigate to /register",
"User can fill registration form",
"System shows success message on registration",
"System shows error if email exists"
],
"priority_coverage": {
"P0": 1,
"P1": 1,
"P2": 0,
"P3": 0
}
}
],
"fixture_needs": ["registrationPageMock"],
"knowledge_fragments_used": ["fixture-architecture", "network-first", "selector-resilience"],
"test_count": 2,
"tdd_phase": "RED",
"summary": "Generated 2 FAILING E2E tests for user registration story"
}
```
**On Error:**
```json
{
"success": false,
"subprocess": "atdd-e2e-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subprocess completes when:
- ✅ All user journeys from acceptance criteria have test files
- ✅ All tests use `test.skip()` (documented failing tests)
- ✅ All tests assert EXPECTED UI behavior (not placeholder assertions)
- ✅ Resilient selectors used (getByRole, getByText)
- ✅ JSON output written to temp file
- ✅ Fixture needs tracked
**Subprocess terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBPROCESS SUCCESS METRICS
### ✅ SUCCESS:
- All E2E tests generated with test.skip()
- Tests assert expected UI behavior (not placeholders)
- Resilient selectors used (getByRole, getByText)
- JSON output valid and complete
- No API/component/unit tests included (out of scope)
- Tests follow knowledge fragment patterns
### ❌ FAILURE:
- Generated passing tests (wrong - this is RED phase)
- Tests without test.skip() (will break CI)
- Placeholder assertions (expect(true).toBe(true))
- Brittle selectors used (CSS classes, XPath)
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output
@@ -0,0 +1,354 @@
---
name: 'step-04c-aggregate'
description: 'Aggregate subprocess outputs and complete ATDD test infrastructure'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-05-validate-and-complete.md'
---
# Step 4C: Aggregate ATDD Test Generation Results
## STEP GOAL
Read outputs from parallel subprocesses (API + E2E failing test generation), aggregate results, verify TDD red phase compliance, and create supporting infrastructure.
---
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Read subprocess outputs from temp files
- ✅ Verify all tests are marked with test.skip() (TDD red phase)
- ✅ Generate shared fixtures based on fixture needs
- ✅ Write all generated test files to disk
- ❌ Do NOT remove test.skip() (that's done after feature implementation)
- ❌ Do NOT run tests yet (that's step 5 - verify they fail)
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, subprocess outputs from temp files
- Focus: aggregation and TDD validation
- Limits: do not execute future steps
- Dependencies: Step 4A and 4B subprocess outputs
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Read Subprocess Outputs
**Read API test subprocess output:**
```javascript
const apiTestsPath = '/tmp/tea-atdd-api-tests-{{timestamp}}.json';
const apiTestsOutput = JSON.parse(fs.readFileSync(apiTestsPath, 'utf8'));
```
**Read E2E test subprocess output:**
```javascript
const e2eTestsPath = '/tmp/tea-atdd-e2e-tests-{{timestamp}}.json';
const e2eTestsOutput = JSON.parse(fs.readFileSync(e2eTestsPath, 'utf8'));
```
**Verify both subprocesses succeeded:**
- Check `apiTestsOutput.success === true`
- Check `e2eTestsOutput.success === true`
- If either failed, report error and stop (don't proceed)
---
### 2. Verify TDD Red Phase Compliance
**CRITICAL TDD Validation:**
**Check API tests:**
```javascript
apiTestsOutput.tests.forEach((test) => {
// Verify test.skip() is present
if (!test.content.includes('test.skip(')) {
throw new Error(`ATDD ERROR: ${test.file} missing test.skip() - tests MUST be skipped in red phase!`);
}
// Verify not placeholder assertions
if (test.content.includes('expect(true).toBe(true)')) {
throw new Error(`ATDD ERROR: ${test.file} has placeholder assertions - must assert EXPECTED behavior!`);
}
// Verify expected_to_fail flag
if (!test.expected_to_fail) {
throw new Error(`ATDD ERROR: ${test.file} not marked as expected_to_fail!`);
}
});
```
**Check E2E tests:**
```javascript
e2eTestsOutput.tests.forEach((test) => {
// Same validation as API tests
if (!test.content.includes('test.skip(')) {
throw new Error(`ATDD ERROR: ${test.file} missing test.skip() - tests MUST be skipped in red phase!`);
}
if (test.content.includes('expect(true).toBe(true)')) {
throw new Error(`ATDD ERROR: ${test.file} has placeholder assertions!`);
}
if (!test.expected_to_fail) {
throw new Error(`ATDD ERROR: ${test.file} not marked as expected_to_fail!`);
}
});
```
**If validation passes:**
```
✅ TDD Red Phase Validation: PASS
- All tests use test.skip()
- All tests assert expected behavior (not placeholders)
- All tests marked as expected_to_fail
```
---
### 3. Write All Test Files to Disk
**Write API test files:**
```javascript
apiTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created (RED): ${test.file}`);
});
```
**Write E2E test files:**
```javascript
e2eTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created (RED): ${test.file}`);
});
```
---
### 4. Aggregate Fixture Needs
**Collect all fixture needs from both subprocesses:**
```javascript
const allFixtureNeeds = [...apiTestsOutput.fixture_needs, ...e2eTestsOutput.fixture_needs];
// Remove duplicates
const uniqueFixtures = [...new Set(allFixtureNeeds)];
```
---
### 5. Generate Fixture Infrastructure
**Create fixtures needed by ATDD tests:**
(Similar to automate workflow, but may be simpler for ATDD since feature not implemented)
**Minimal fixtures for TDD red phase:**
```typescript
// tests/fixtures/test-data.ts
export const testUserData = {
email: 'test@example.com',
password: 'SecurePass123!',
};
```
Note: More complete fixtures will be needed when moving to green phase.
---
### 6. Generate ATDD Checklist
**Create ATDD checklist document:**
```markdown
# ATDD Checklist: [Story Name]
## TDD Red Phase (Current)
✅ Failing tests generated
- API Tests: {api_test_count} tests (all skipped)
- E2E Tests: {e2e_test_count} tests (all skipped)
## Acceptance Criteria Coverage
{list all acceptance criteria with test coverage}
## Next Steps (TDD Green Phase)
After implementing the feature:
1. Remove `test.skip()` from all test files
2. Run tests: `npm test`
3. Verify tests PASS (green phase)
4. If any tests fail:
- Either fix implementation (feature bug)
- Or fix test (test bug)
5. Commit passing tests
## Implementation Guidance
Feature endpoints to implement:
{list endpoints from API tests}
UI components to implement:
{list UI flows from E2E tests}
```
**Save checklist:**
```javascript
fs.writeFileSync(`{test_artifacts}/atdd-checklist-{story-id}.md`, checklistContent, 'utf8');
```
---
### 7. Calculate Summary Statistics
**Aggregate test counts:**
```javascript
const summary = {
tdd_phase: 'RED',
total_tests: apiTestsOutput.test_count + e2eTestsOutput.test_count,
api_tests: apiTestsOutput.test_count,
e2e_tests: e2eTestsOutput.test_count,
all_tests_skipped: true,
expected_to_fail: true,
fixtures_created: uniqueFixtures.length,
acceptance_criteria_covered: [
...apiTestsOutput.tests.flatMap((t) => t.acceptance_criteria_covered),
...e2eTestsOutput.tests.flatMap((t) => t.acceptance_criteria_covered),
],
knowledge_fragments_used: [...apiTestsOutput.knowledge_fragments_used, ...e2eTestsOutput.knowledge_fragments_used],
subprocess_execution: 'PARALLEL (API + E2E)',
performance_gain: '~50% faster than sequential',
};
```
**Store summary for Step 5:**
```javascript
fs.writeFileSync('/tmp/tea-atdd-summary-{{timestamp}}.json', JSON.stringify(summary, null, 2), 'utf8');
```
---
## OUTPUT SUMMARY
Display to user:
```
✅ ATDD Test Generation Complete (TDD RED PHASE)
🔴 TDD Red Phase: Failing Tests Generated
📊 Summary:
- Total Tests: {total_tests} (all with test.skip())
- API Tests: {api_tests} (RED)
- E2E Tests: {e2e_tests} (RED)
- Fixtures Created: {fixtures_created}
- All tests will FAIL until feature implemented
✅ Acceptance Criteria Coverage:
{list all covered criteria}
🚀 Performance: Parallel execution ~50% faster than sequential
📂 Generated Files:
- tests/api/[feature].spec.ts (with test.skip())
- tests/e2e/[feature].spec.ts (with test.skip())
- tests/fixtures/test-data.ts
- {test_artifacts}/atdd-checklist-{story-id}.md
📝 Next Steps:
1. Implement the feature
2. Remove test.skip() from tests
3. Run tests → verify PASS (green phase)
4. Commit passing tests
✅ Ready for validation (Step 5 - verify tests fail as expected)
```
---
## EXIT CONDITION
Proceed to Step 5 when:
- ✅ All test files written to disk (API + E2E)
- ✅ All tests verified to have test.skip()
- ✅ All fixtures created
- ✅ ATDD checklist generated
- ✅ Summary statistics calculated and saved
- ✅ Output displayed to user
---
### 8. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04c-aggregate']
lastStep: 'step-04c-aggregate'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04c-aggregate'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04c-aggregate'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Both subprocesses succeeded
- All tests have test.skip() (TDD red phase compliant)
- All tests assert expected behavior (not placeholders)
- All test files written to disk
- ATDD checklist generated
### ❌ SYSTEM FAILURE:
- One or both subprocesses failed
- Tests missing test.skip() (would break CI)
- Tests have placeholder assertions
- Test files not written to disk
- ATDD checklist missing
**Master Rule:** TDD RED PHASE requires ALL tests to use test.skip() and assert expected behavior.
@@ -0,0 +1,95 @@
---
name: 'step-05-validate-and-complete'
description: 'Validate ATDD outputs and summarize'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
---
# Step 5: Validate & Complete
## STEP GOAL
Validate ATDD outputs and provide a completion summary.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Validate against the checklist
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validation
Use `checklist.md` to validate:
- Prerequisites satisfied
- Test files created correctly
- Checklist matches acceptance criteria
- Tests are designed to fail before implementation
- [ ] CLI sessions cleaned up (no orphaned browsers)
- [ ] Temp artifacts stored in `{test_artifacts}/` not random locations
Fix any gaps before completion.
---
## 2. Completion Summary
Report:
- Test files created
- Checklist output path
- Key risks or assumptions
- Next recommended workflow (e.g., implementation or `automate`)
---
## 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-05-validate-and-complete']
lastStep: 'step-05-validate-and-complete'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-05-validate-and-complete'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-05-validate-and-complete'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target
@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update
@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/atdd-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced
@@ -0,0 +1,73 @@
---
validationDate: 2026-01-27
workflowName: testarch-atdd
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/atdd
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-atdd
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 101 lines [GOOD]
- steps-c/step-02-generation-mode.md: 71 lines [GOOD]
- steps-c/step-03-test-strategy.md: 70 lines [GOOD]
- steps-c/step-04-generate-tests.md: 70 lines [GOOD]
- steps-c/step-05-validate-and-complete.md: 61 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: atdd-checklist-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-04-generate-tests.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,116 @@
---
validationDate: 2026-01-27
workflowName: testarch-atdd
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/atdd
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-atdd
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 100 lines [GOOD]
- steps-c/step-02-generation-mode.md: 70 lines [GOOD]
- steps-c/step-03-test-strategy.md: 69 lines [GOOD]
- steps-c/step-04-generate-tests.md: 69 lines [GOOD]
- steps-c/step-05-validate-and-complete.md: 60 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight-and-context.md: Init [PASS]
- steps-c/step-02-generation-mode.md: Middle [PASS]
- steps-c/step-03-test-strategy.md: Middle [PASS]
- steps-c/step-04-generate-tests.md: Middle [PASS]
- steps-c/step-05-validate-and-complete.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: atdd-checklist-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-04-generate-tests.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subprocess Optimization Opportunities
- No high-priority subprocess optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 8 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,21 @@
# Workflow Plan: testarch-atdd
## Create Mode (steps-c)
- step-01-preflight-and-context.md
- step-02-generation-mode.md
- step-03-test-strategy.md
- step-04-generate-tests.md
- step-05-validate-and-complete.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- {test_artifacts}/atdd-checklist-{story_id}.md
- Failing acceptance tests under {project-root}/tests
@@ -0,0 +1,41 @@
---
name: testarch-atdd
description: 'Generate failing acceptance tests before implementation using TDD red-green-refactor cycle'
web_bundle: true
---
# Acceptance Test-Driven Development (ATDD)
**Goal:** Generate failing acceptance tests before implementation using TDD red-green-refactor cycle
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight-and-context.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`
@@ -0,0 +1,47 @@
# Test Architect workflow: atdd
name: testarch-atdd
description: "Generate failing acceptance tests before implementation using TDD red-green-refactor cycle"
author: "BMad"
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/atdd"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
template: "{installed_path}/atdd-checklist-template.md"
# Variables and inputs
variables:
test_dir: "{project-root}/tests" # Root test directory
# Output configuration
default_output_file: "{test_artifacts}/atdd-checklist-{story_id}.md"
# Required tools
required_tools:
- read_file # Read story markdown, framework config
- write_file # Create test files, checklist, factory stubs
- create_directory # Create test directories
- list_files # Find existing fixtures and helpers
- search_repo # Search for similar test patterns
tags:
- qa
- atdd
- test-architect
- tdd
- red-green-refactor
execution_hints:
interactive: false # Minimize prompts
autonomous: true # Proceed without user input unless blocked
iterative: true
␍
@@ -0,0 +1,582 @@
# Automate Workflow Validation Checklist
Use this checklist to validate that the automate workflow has been executed correctly and all deliverables meet quality standards.
## Prerequisites
Before starting this workflow, verify:
- [ ] Framework scaffolding configured (playwright.config.ts or cypress.config.ts exists)
- [ ] Test directory structure exists (tests/ folder with subdirectories)
- [ ] Package.json has test framework dependencies installed
**Halt only if:** Framework scaffolding is completely missing (run `framework` workflow first)
**Note:** BMad artifacts (story, tech-spec, PRD) are OPTIONAL - workflow can run without them
**Note:** `automate` generates tests; it does not run `*atdd` or `*test-review`. If ATDD outputs exist, use them as input and avoid duplicate coverage.
---
## Step 1: Execution Mode Determination and Context Loading
### Mode Detection
- [ ] Execution mode correctly determined:
- [ ] BMad-Integrated Mode (story_file variable set) OR
- [ ] Standalone Mode (target_feature or target_files set) OR
- [ ] Auto-discover Mode (no targets specified)
### BMad Artifacts (If Available - OPTIONAL)
- [ ] Story markdown loaded (if `{story_file}` provided)
- [ ] Acceptance criteria extracted from story (if available)
- [ ] Tech-spec.md loaded (if `{use_tech_spec}` true and file exists)
- [ ] Test-design.md loaded (if `{use_test_design}` true and file exists)
- [ ] PRD.md loaded (if `{use_prd}` true and file exists)
- [ ] **Note**: Absence of BMad artifacts does NOT halt workflow
### Framework Configuration
- [ ] Test framework config loaded (playwright.config.ts or cypress.config.ts)
- [ ] Test directory structure identified from `{test_dir}`
- [ ] Existing test patterns reviewed
- [ ] Test runner capabilities noted (parallel execution, fixtures, etc.)
### Coverage Analysis
- [ ] Existing test files searched in `{test_dir}` (if `{analyze_coverage}` true)
- [ ] Tested features vs untested features identified
- [ ] Coverage gaps mapped (tests to source files)
- [ ] Existing fixture and factory patterns checked
### Knowledge Base Fragments Loaded
- [ ] `test-levels-framework.md` - Test level selection
- [ ] `test-priorities.md` - Priority classification (P0-P3)
- [ ] `fixture-architecture.md` - Fixture patterns with auto-cleanup
- [ ] `data-factories.md` - Factory patterns using faker
- [ ] `selective-testing.md` - Targeted test execution strategies
- [ ] `ci-burn-in.md` - Flaky test detection patterns
- [ ] `test-quality.md` - Test design principles
---
## Step 2: Automation Targets Identification
### Target Determination
**BMad-Integrated Mode (if story available):**
- [ ] Acceptance criteria mapped to test scenarios
- [ ] Features implemented in story identified
- [ ] Existing ATDD tests checked (if any)
- [ ] Expansion beyond ATDD planned (edge cases, negative paths)
**Standalone Mode (if no story):**
- [ ] Specific feature analyzed (if `{target_feature}` specified)
- [ ] Specific files analyzed (if `{target_files}` specified)
- [ ] Features auto-discovered (if `{auto_discover_features}` true)
- [ ] Features prioritized by:
- [ ] No test coverage (highest priority)
- [ ] Complex business logic
- [ ] External integrations (API, database, auth)
- [ ] Critical user paths (login, checkout, etc.)
### Test Level Selection
- [ ] Test level selection framework applied (from `test-levels-framework.md`)
- [ ] E2E tests identified: Critical user journeys, multi-system integration
- [ ] API tests identified: Business logic, service contracts, data transformations
- [ ] Component tests identified: UI behavior, interactions, state management
- [ ] Unit tests identified: Pure logic, edge cases, error handling
### Duplicate Coverage Avoidance
- [ ] Same behavior NOT tested at multiple levels unnecessarily
- [ ] E2E used for critical happy path only
- [ ] API tests used for business logic variations
- [ ] Component tests used for UI interaction edge cases
- [ ] Unit tests used for pure logic edge cases
### Priority Assignment
- [ ] Test priorities assigned using `test-priorities.md` framework
- [ ] P0 tests: Critical paths, security-critical, data integrity
- [ ] P1 tests: Important features, integration points, error handling
- [ ] P2 tests: Edge cases, less-critical variations, performance
- [ ] P3 tests: Nice-to-have, rarely-used features, exploratory
- [ ] Priority variables respected:
- [ ] `{include_p0}` = true (always include)
- [ ] `{include_p1}` = true (high priority)
- [ ] `{include_p2}` = true (medium priority)
- [ ] `{include_p3}` = false (low priority, skip by default)
### Coverage Plan Created
- [ ] Test coverage plan documented
- [ ] What will be tested at each level listed
- [ ] Priorities assigned to each test
- [ ] Coverage strategy clear (critical-paths, comprehensive, or selective)
---
## Step 3: Test Infrastructure Generated
### Fixture Architecture
- [ ] Existing fixtures checked in `tests/support/fixtures/`
- [ ] Fixture architecture created/enhanced (if `{generate_fixtures}` true)
- [ ] All fixtures use Playwright's `test.extend()` pattern
- [ ] All fixtures have auto-cleanup in teardown
- [ ] Common fixtures created/enhanced:
- [ ] authenticatedUser (with auto-delete)
- [ ] apiRequest (authenticated client)
- [ ] mockNetwork (external service mocking)
- [ ] testDatabase (with auto-cleanup)
### Data Factories
- [ ] Existing factories checked in `tests/support/factories/`
- [ ] Factory architecture created/enhanced (if `{generate_factories}` true)
- [ ] All factories use `@faker-js/faker` for random data (no hardcoded values)
- [ ] All factories support overrides for specific scenarios
- [ ] Common factories created/enhanced:
- [ ] User factory (email, password, name, role)
- [ ] Product factory (name, price, SKU)
- [ ] Order factory (items, total, status)
- [ ] Cleanup helpers provided (e.g., deleteUser(), deleteProduct())
### Helper Utilities
- [ ] Existing helpers checked in `tests/support/helpers/` (if `{update_helpers}` true)
- [ ] Common utilities created/enhanced:
- [ ] waitFor (polling for complex conditions)
- [ ] retry (retry helper for flaky operations)
- [ ] testData (test data generation)
- [ ] assertions (custom assertion helpers)
---
## Step 4: Test Files Generated
### Test File Structure
- [ ] Test files organized correctly:
- [ ] `tests/e2e/` for E2E tests
- [ ] `tests/api/` for API tests
- [ ] `tests/component/` for component tests
- [ ] `tests/unit/` for unit tests
- [ ] `tests/support/` for fixtures/factories/helpers
### E2E Tests (If Applicable)
- [ ] E2E test files created in `tests/e2e/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags ([P0], [P1], [P2], [P3]) in test name
- [ ] All tests use data-testid selectors (not CSS classes)
- [ ] One assertion per test (atomic design)
- [ ] No hard waits or sleeps (explicit waits only)
- [ ] Network-first pattern applied (route interception BEFORE navigation)
- [ ] Clear Given-When-Then comments in test code
### API Tests (If Applicable)
- [ ] API test files created in `tests/api/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags in test name
- [ ] API contracts validated (request/response structure)
- [ ] HTTP status codes verified
- [ ] Response body validation includes required fields
- [ ] Error cases tested (400, 401, 403, 404, 500)
- [ ] JWT token format validated (if auth tests)
### Component Tests (If Applicable)
- [ ] Component test files created in `tests/component/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags in test name
- [ ] Component mounting works correctly
- [ ] Interaction testing covers user actions (click, hover, keyboard)
- [ ] State management validated
- [ ] Props and events tested
### Unit Tests (If Applicable)
- [ ] Unit test files created in `tests/unit/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags in test name
- [ ] Pure logic tested (no dependencies)
- [ ] Edge cases covered
- [ ] Error handling tested
### Quality Standards Enforced
- [ ] All tests use Given-When-Then format with clear comments
- [ ] All tests have descriptive names with priority tags
- [ ] No duplicate tests (same behavior tested multiple times)
- [ ] No flaky patterns (race conditions, timing issues)
- [ ] No test interdependencies (tests can run in any order)
- [ ] Tests are deterministic (same input always produces same result)
- [ ] All tests use data-testid selectors (E2E tests)
- [ ] No hard waits: `await page.waitForTimeout()` (forbidden)
- [ ] No conditional flow: `if (await element.isVisible())` (forbidden)
- [ ] No try-catch for test logic (only for cleanup)
- [ ] No hardcoded test data (use factories with faker)
- [ ] No page object classes (tests are direct and simple)
- [ ] No shared state between tests
### Network-First Pattern Applied
- [ ] Route interception set up BEFORE navigation (E2E tests with network requests)
- [ ] `page.route()` called before `page.goto()` to prevent race conditions
- [ ] Network-first pattern verified in all E2E tests that make API calls
---
## Step 5: Test Validation and Healing (NEW - Phase 2.5)
### Healing Configuration
- [ ] Healing configuration checked:
- [ ] `{auto_validate}` setting noted (default: true)
- [ ] `{auto_heal_failures}` setting noted (default: false)
- [ ] `{max_healing_iterations}` setting noted (default: 3)
- [ ] `{use_mcp_healing}` setting noted (default: true)
### Healing Knowledge Fragments Loaded (If Healing Enabled)
- [ ] `test-healing-patterns.md` loaded (common failure patterns and fixes)
- [ ] `selector-resilience.md` loaded (selector refactoring guide)
- [ ] `timing-debugging.md` loaded (race condition fixes)
### Test Execution and Validation
- [ ] Generated tests executed (if `{auto_validate}` true)
- [ ] Test results captured:
- [ ] Total tests run
- [ ] Passing tests count
- [ ] Failing tests count
- [ ] Error messages and stack traces captured
### Healing Loop (If Enabled and Tests Failed)
- [ ] Healing loop entered (if `{auto_heal_failures}` true AND tests failed)
- [ ] For each failing test:
- [ ] Failure pattern identified (selector, timing, data, network, hard wait)
- [ ] Appropriate healing strategy applied:
- [ ] Stale selector → Replaced with data-testid or ARIA role
- [ ] Race condition → Added network-first interception or state waits
- [ ] Dynamic data → Replaced hardcoded values with regex/dynamic generation
- [ ] Network error → Added route mocking
- [ ] Hard wait → Replaced with event-based wait
- [ ] Healed test re-run to validate fix
- [ ] Iteration count tracked (max 3 attempts)
### Unfixable Tests Handling
- [ ] Tests that couldn't be healed after 3 iterations marked with `test.fixme()` (if `{mark_unhealable_as_fixme}` true)
- [ ] Detailed comment added to test.fixme() tests:
- [ ] What failure occurred
- [ ] What healing was attempted (3 iterations)
- [ ] Why healing failed
- [ ] Manual investigation steps needed
- [ ] Original test logic preserved in comments
### Healing Report Generated
- [ ] Healing report generated (if healing attempted)
- [ ] Report includes:
- [ ] Auto-heal enabled status
- [ ] Healing mode (MCP-assisted or Pattern-based)
- [ ] Iterations allowed (max_healing_iterations)
- [ ] Validation results (total, passing, failing)
- [ ] Successfully healed tests (count, file:line, fix applied)
- [ ] Unable to heal tests (count, file:line, reason)
- [ ] Healing patterns applied (selector fixes, timing fixes, data fixes)
- [ ] Knowledge base references used
---
## Step 6: Documentation and Scripts Updated
### Test README Updated
- [ ] `tests/README.md` created or updated (if `{update_readme}` true)
- [ ] Test suite structure overview included
- [ ] Test execution instructions provided (all, specific files, by priority)
- [ ] Fixture usage examples provided
- [ ] Factory usage examples provided
- [ ] Priority tagging convention explained ([P0], [P1], [P2], [P3])
- [ ] How to write new tests documented
- [ ] Common patterns documented
- [ ] Anti-patterns documented (what to avoid)
### package.json Scripts Updated
- [ ] package.json scripts added/updated (if `{update_package_scripts}` true)
- [ ] `test:e2e` script for all E2E tests
- [ ] `test:e2e:p0` script for P0 tests only
- [ ] `test:e2e:p1` script for P0 + P1 tests
- [ ] `test:api` script for API tests
- [ ] `test:component` script for component tests
- [ ] `test:unit` script for unit tests (if applicable)
### Test Suite Executed
- [ ] Test suite run locally (if `{run_tests_after_generation}` true)
- [ ] Test results captured (passing/failing counts)
- [ ] No flaky patterns detected (tests are deterministic)
- [ ] Setup requirements documented (if any)
- [ ] Known issues documented (if any)
---
## Step 6: Automation Summary Generated
### Automation Summary Document
- [ ] Output file created at `{output_summary}`
- [ ] Document includes execution mode (BMad-Integrated, Standalone, Auto-discover)
- [ ] Feature analysis included (source files, coverage gaps) - Standalone mode
- [ ] Tests created listed (E2E, API, Component, Unit) with counts and paths
- [ ] Infrastructure created listed (fixtures, factories, helpers)
- [ ] Test execution instructions provided
- [ ] Coverage analysis included:
- [ ] Total test count
- [ ] Priority breakdown (P0, P1, P2, P3 counts)
- [ ] Test level breakdown (E2E, API, Component, Unit counts)
- [ ] Coverage percentage (if calculated)
- [ ] Coverage status (acceptance criteria covered, gaps identified)
- [ ] Definition of Done checklist included
- [ ] Next steps provided
- [ ] Recommendations included (if Standalone mode)
### Summary Provided to User
- [ ] Concise summary output provided
- [ ] Total tests created across test levels
- [ ] Priority breakdown (P0, P1, P2, P3 counts)
- [ ] Infrastructure counts (fixtures, factories, helpers)
- [ ] Test execution command provided
- [ ] Output file path provided
- [ ] Next steps listed
---
## Quality Checks
### Test Design Quality
- [ ] Tests are readable (clear Given-When-Then structure)
- [ ] Tests are maintainable (use factories/fixtures, not hardcoded data)
- [ ] Tests are isolated (no shared state between tests)
- [ ] Tests are deterministic (no race conditions or flaky patterns)
- [ ] Tests are atomic (one assertion per test)
- [ ] Tests are fast (no unnecessary waits or delays)
- [ ] Tests are lean (files under {max_file_lines} lines)
### Knowledge Base Integration
- [ ] Test level selection framework applied (from `test-levels-framework.md`)
- [ ] Priority classification applied (from `test-priorities.md`)
- [ ] Fixture architecture patterns applied (from `fixture-architecture.md`)
- [ ] Data factory patterns applied (from `data-factories.md`)
- [ ] Selective testing strategies considered (from `selective-testing.md`)
- [ ] Flaky test detection patterns considered (from `ci-burn-in.md`)
- [ ] Test quality principles applied (from `test-quality.md`)
### Code Quality
- [ ] All TypeScript types are correct and complete
- [ ] No linting errors in generated test files
- [ ] Consistent naming conventions followed
- [ ] Imports are organized and correct
- [ ] Code follows project style guide
- [ ] No console.log or debug statements in test code
---
## Integration Points
### With Framework Workflow
- [ ] Test framework configuration detected and used
- [ ] Directory structure matches framework setup
- [ ] Fixtures and helpers follow established patterns
- [ ] Naming conventions consistent with framework standards
### With BMad Workflows (If Available - OPTIONAL)
**With Story Workflow:**
- [ ] Story ID correctly referenced in output (if story available)
- [ ] Acceptance criteria from story reflected in tests (if story available)
- [ ] Technical constraints from story considered (if story available)
**With test-design Workflow:**
- [ ] P0 scenarios from test-design prioritized (if test-design available)
- [ ] Risk assessment from test-design considered (if test-design available)
- [ ] Coverage strategy aligned with test-design (if test-design available)
**With atdd Workflow:**
- [ ] ATDD artifacts provided or located (manual handoff; `atdd` not auto-run)
- [ ] Existing ATDD tests checked (if story had ATDD workflow run)
- [ ] Expansion beyond ATDD planned (edge cases, negative paths)
- [ ] No duplicate coverage with ATDD tests
### With CI Pipeline
- [ ] Tests can run in CI environment
- [ ] Tests are parallelizable (no shared state)
- [ ] Tests have appropriate timeouts
- [ ] Tests clean up their data (no CI environment pollution)
---
## Completion Criteria
All of the following must be true before marking this workflow as complete:
- [ ] **Execution mode determined** (BMad-Integrated, Standalone, or Auto-discover)
- [ ] **Framework configuration loaded** and validated
- [ ] **Coverage analysis completed** (gaps identified if analyze_coverage true)
- [ ] **Automation targets identified** (what needs testing)
- [ ] **Test levels selected** appropriately (E2E, API, Component, Unit)
- [ ] **Duplicate coverage avoided** (same behavior not tested at multiple levels)
- [ ] **Test priorities assigned** (P0, P1, P2, P3)
- [ ] **Fixture architecture created/enhanced** with auto-cleanup
- [ ] **Data factories created/enhanced** using faker (no hardcoded data)
- [ ] **Helper utilities created/enhanced** (if needed)
- [ ] **Test files generated** at appropriate levels (E2E, API, Component, Unit)
- [ ] **Given-When-Then format used** consistently across all tests
- [ ] **Priority tags added** to all test names ([P0], [P1], [P2], [P3])
- [ ] **data-testid selectors used** in E2E tests (not CSS classes)
- [ ] **Network-first pattern applied** (route interception before navigation)
- [ ] **Quality standards enforced** (no hard waits, no flaky patterns, self-cleaning, deterministic)
- [ ] **Test README updated** with execution instructions and patterns
- [ ] **package.json scripts updated** with test execution commands
- [ ] **Test suite run locally** (if run_tests_after_generation true)
- [ ] **Tests validated** (if auto_validate enabled)
- [ ] **Failures healed** (if auto_heal_failures enabled and tests failed)
- [ ] **Healing report generated** (if healing attempted)
- [ ] **Unfixable tests marked** with test.fixme() and detailed comments (if any)
- [ ] **Automation summary created** and saved to correct location
- [ ] **Output file formatted correctly**
- [ ] **Knowledge base references applied** and documented (including healing fragments if used)
- [ ] **No test quality issues** (flaky patterns, race conditions, hardcoded data, page objects)
---
## Common Issues and Resolutions
### Issue: BMad artifacts not found
**Problem:** Story, tech-spec, or PRD files not found when variables are set.
**Resolution:**
- **automate does NOT require BMad artifacts** - they are OPTIONAL enhancements
- If files not found, switch to Standalone Mode automatically
- Analyze source code directly without BMad context
- Continue workflow without halting
### Issue: Framework configuration not found
**Problem:** No playwright.config.ts or cypress.config.ts found.
**Resolution:**
- **HALT workflow** - framework is required
- Message: "Framework scaffolding required. Run `bmad tea *framework` first."
- User must run framework workflow before automate
### Issue: No automation targets identified
**Problem:** Neither story, target_feature, nor target_files specified, and auto-discover finds nothing.
**Resolution:**
- Check if source_dir variable is correct
- Verify source code exists in project
- Ask user to specify target_feature or target_files explicitly
- Provide examples: `target_feature: "src/auth/"` or `target_files: "src/auth/login.ts,src/auth/session.ts"`
### Issue: Duplicate coverage detected
**Problem:** Same behavior tested at multiple levels (E2E + API + Component).
**Resolution:**
- Review test level selection framework (test-levels-framework.md)
- Use E2E for critical happy path ONLY
- Use API for business logic variations
- Use Component for UI edge cases
- Remove redundant tests that duplicate coverage
### Issue: Tests have hardcoded data
**Problem:** Tests use hardcoded email addresses, passwords, or other data.
**Resolution:**
- Replace all hardcoded data with factory function calls
- Use faker for all random data generation
- Update data-factories to support all required test scenarios
- Example: `createUser({ email: faker.internet.email() })`
### Issue: Tests are flaky
**Problem:** Tests fail intermittently, pass on retry.
**Resolution:**
- Remove all hard waits (`page.waitForTimeout()`)
- Use explicit waits (`page.waitForSelector()`)
- Apply network-first pattern (route interception before navigation)
- Remove conditional flow (`if (await element.isVisible())`)
- Ensure tests are deterministic (no race conditions)
- Run burn-in loop (10 iterations) to detect flakiness
### Issue: Fixtures don't clean up data
**Problem:** Test data persists after test run, causing test pollution.
**Resolution:**
- Ensure all fixtures have cleanup in teardown phase
- Cleanup happens AFTER `await use(data)`
- Call deletion/cleanup functions (deleteUser, deleteProduct, etc.)
- Verify cleanup works by checking database/storage after test run
### Issue: Tests too slow
**Problem:** Tests take longer than 90 seconds (max_test_duration).
**Resolution:**
- Remove unnecessary waits and delays
- Use parallel execution where possible
- Mock external services (don't make real API calls)
- Use API tests instead of E2E for business logic
- Optimize test data creation (use in-memory database, etc.)
---
## Notes for TEA Agent
- **automate is flexible:** Can work with or without BMad artifacts (story, tech-spec, PRD are OPTIONAL)
- **Standalone mode is powerful:** Analyze any codebase and generate tests independently
- **Auto-discover mode:** Scan codebase for features needing tests when no targets specified
- **Framework is the ONLY hard requirement:** HALT if framework config missing, otherwise proceed
- **Avoid duplicate coverage:** E2E for critical paths only, API/Component for variations
- **Priority tagging enables selective execution:** P0 tests run on every commit, P1 on PR, P2 nightly
- **Network-first pattern prevents race conditions:** Route interception BEFORE navigation
- **No page objects:** Keep tests simple, direct, and maintainable
- **Use knowledge base:** Load relevant fragments (test-levels, test-priorities, fixture-architecture, data-factories, healing patterns) for guidance
- **Deterministic tests only:** No hard waits, no conditional flow, no flaky patterns allowed
- **Optional healing:** auto_heal_failures disabled by default (opt-in for automatic test healing)
- **Graceful degradation:** Healing works without Playwright MCP (pattern-based fallback)
- **Unfixable tests handled:** Mark with test.fixme() and detailed comments (not silently broken)
@@ -0,0 +1,50 @@
<!-- Powered by BMAD-CORE™ -->
# Test Automation Expansion
**Workflow ID**: `_bmad/tea/testarch/automate`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Expands test automation coverage by generating prioritized tests at the appropriate level (E2E, API, Component, Unit) with supporting fixtures and helpers.
Modes:
- **BMad-Integrated**: Uses story/PRD/test-design artifacts when available
- **Standalone**: Analyzes existing codebase without BMad artifacts
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture** for disciplined execution:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order without skipping
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `test_dir`, `source_dir`, `coverage_target`, `standalone_mode`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/automate/steps-c/step-01-preflight-and-context.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/automate/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.
@@ -0,0 +1,161 @@
---
name: 'step-01-preflight-and-context'
description: 'Determine mode, verify framework, and load context and knowledge'
outputFile: '{test_artifacts}/automation-summary.md'
nextStepFile: './step-02-identify-targets.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
---
# Step 1: Preflight & Context Loading
## STEP GOAL
Determine execution mode, verify framework readiness, and load the necessary artifacts and knowledge fragments.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if framework scaffolding is missing
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Verify Framework
Ensure a framework exists:
- `playwright.config.ts` or `cypress.config.ts`
- `package.json` includes test dependencies
If missing: **HALT** with message "Run `framework` workflow first."
---
## 2. Determine Execution Mode
- **BMad-Integrated** if story/tech-spec/test-design artifacts are provided or found
- **Standalone** if only source code is available
- If unclear, ask the user which mode to use
---
## 3. Load Context
### BMad-Integrated (if available)
- Story with acceptance criteria
- PRD and/or tech spec
- Test-design document (if exists)
### Standalone
- Skip artifacts; proceed to codebase analysis
### Always Load
- Test framework config
- Existing test structure in `{test_dir}`
- Existing tests (for coverage gaps)
### Read TEA Config Flags
- From `{config_source}` read `tea_use_playwright_utils`
- From `{config_source}` read `tea_browser_automation`
---
## 4. Load Knowledge Base Fragments
Use `{knowledgeIndex}` and load only what is required.
**Core (always load):**
- `test-levels-framework.md`
- `test-priorities-matrix.md`
- `data-factories.md`
- `selective-testing.md`
- `ci-burn-in.md`
- `test-quality.md`
**Playwright Utils (if enabled):**
- `overview.md`, `api-request.md`, `network-recorder.md`, `auth-session.md`, `intercept-network-call.md`, `recurse.md`, `log.md`, `file-utils.md`, `burn-in.md`, `network-error-monitor.md`, `fixtures-composition.md`
**Traditional Patterns (if Playwright Utils disabled):**
- `fixture-architecture.md`
- `network-first.md`
**Healing (if auto-heal enabled):**
- `test-healing-patterns.md`
- `selector-resilience.md`
- `timing-debugging.md`
**Playwright CLI (if tea_browser_automation is "cli" or "auto"):**
- `playwright-cli.md`
**MCP Patterns (if tea_browser_automation is "mcp" or "auto"):**
- (existing MCP-related fragments, if any are added in future)
---
## 5. Confirm Inputs
Summarize loaded artifacts, framework, and knowledge fragments, then proceed.
---
## 6. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight-and-context']
lastStep: 'step-01-preflight-and-context'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight-and-context'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight-and-context'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,94 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/automation-summary.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Display Progress Dashboard
Display progress with ✅/⬜ indicators:
1. ✅/⬜ Preflight & Context (step-01-preflight-and-context)
2. ✅/⬜ Identify Targets (step-02-identify-targets)
3. ✅/⬜ Generate Tests + Aggregate (step-03c-aggregate)
4. ✅/⬜ Validate & Summarize (step-04-validate-and-summarize)
---
### 3. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight-and-context'` → load `./step-02-identify-targets.md`
- `'step-02-identify-targets'` → load `./step-03-generate-tests.md`
- `'step-03c-aggregate'` → load `./step-04-validate-and-summarize.md`
- `'step-04-validate-and-summarize'` → **Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
@@ -0,0 +1,133 @@
---
name: 'step-02-identify-targets'
description: 'Identify automation targets and create coverage plan'
outputFile: '{test_artifacts}/automation-summary.md'
nextStepFile: './step-03-generate-tests.md'
---
# Step 2: Identify Automation Targets
## STEP GOAL
Determine what needs to be tested and select appropriate test levels and priorities.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Avoid duplicate coverage across test levels
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Determine Targets
**BMad-Integrated:**
- Map acceptance criteria to test scenarios
- Check for existing ATDD outputs to avoid duplication
- Expand coverage with edge cases and negative paths
**Standalone:**
- If specific target feature/files are provided, focus there
- Otherwise auto-discover features in `{source_dir}`
- Prioritize critical paths, integrations, and untested logic
**Browser Exploration (if `tea_browser_automation` is `cli` or `auto`):**
> **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip browser exploration and rely on code/doc analysis.
Use CLI to explore the application and identify testable pages/flows:
1. `playwright-cli -s=tea-automate open <target_url>`
2. `playwright-cli -s=tea-automate snapshot` → capture page structure and element refs
3. Analyze snapshot output to identify testable elements and flows
4. `playwright-cli -s=tea-automate close`
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-automate close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
---
## 2. Choose Test Levels
Use `test-levels-framework.md` to select:
- **E2E** for critical user journeys
- **API** for business logic and service contracts
- **Component** for UI behavior
- **Unit** for pure logic and edge cases
---
## 3. Assign Priorities
Use `test-priorities-matrix.md`:
- P0: Critical path + high risk
- P1: Important flows + medium/high risk
- P2: Secondary + edge cases
- P3: Optional/rare scenarios
---
## 4. Coverage Plan
Produce a concise coverage plan:
- Targets by test level
- Priority assignments
- Justification for coverage scope (critical-paths/comprehensive/selective)
---
## 5. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-identify-targets']
lastStep: 'step-02-identify-targets'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-identify-targets'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-identify-targets'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,199 @@
---
name: 'step-03-generate-tests'
description: 'Orchestrate parallel test generation via subprocesses'
nextStepFile: './step-03c-aggregate.md'
---
# Step 3: Orchestrate Parallel Test Generation
## STEP GOAL
Launch parallel subprocesses to generate API and E2E tests simultaneously for maximum performance.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Launch TWO subprocesses in PARALLEL
- ✅ Wait for BOTH subprocesses to complete
- ❌ Do NOT generate tests sequentially (use subprocesses)
- ❌ Do NOT proceed until both subprocesses finish
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Wait for subprocess outputs
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, coverage plan from Step 2, knowledge fragments
- Focus: subprocess orchestration only
- Limits: do not generate tests directly (delegate to subprocesses)
- Dependencies: Step 2 outputs (coverage plan, target features)
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Prepare Subprocess Inputs
**Generate unique timestamp** for temp file naming:
```javascript
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
```
**Prepare input context for both subprocesses:**
```javascript
const subprocessContext = {
features: /* from Step 2 coverage plan */,
knowledge_fragments_loaded: /* list of fragments */,
config: {
test_framework: config.test_framework,
use_playwright_utils: config.tea_use_playwright_utils,
browser_automation: config.tea_browser_automation // "auto" | "cli" | "mcp" | "none"
},
timestamp: timestamp
};
```
---
### 2. Launch Subprocess A: API Test Generation
**Launch subprocess in parallel:**
- **Subprocess File:** `./step-03a-subprocess-api.md`
- **Output File:** `/tmp/tea-automate-api-tests-${timestamp}.json`
- **Context:** Pass `subprocessContext`
- **Execution:** PARALLEL (non-blocking)
**System Action:**
```
🚀 Launching Subprocess A: API Test Generation
📝 Output: /tmp/tea-automate-api-tests-${timestamp}.json
⏳ Status: Running in parallel...
```
---
### 3. Launch Subprocess B: E2E Test Generation
**Launch subprocess in parallel:**
- **Subprocess File:** `./step-03b-subprocess-e2e.md`
- **Output File:** `/tmp/tea-automate-e2e-tests-${timestamp}.json`
- **Context:** Pass `subprocessContext`
- **Execution:** PARALLEL (non-blocking)
**System Action:**
```
🚀 Launching Subprocess B: E2E Test Generation
📝 Output: /tmp/tea-automate-e2e-tests-${timestamp}.json
⏳ Status: Running in parallel...
```
---
### 4. Wait for Both Subprocesses to Complete
**Monitor subprocess execution:**
```
⏳ Waiting for subprocesses to complete...
├── Subprocess A (API): Running... ⟳
└── Subprocess B (E2E): Running... ⟳
[... time passes ...]
├── Subprocess A (API): Complete ✅
└── Subprocess B (E2E): Complete ✅
✅ All subprocesses completed successfully!
```
**Verify both outputs exist:**
```javascript
const apiOutputExists = fs.existsSync(`/tmp/tea-automate-api-tests-${timestamp}.json`);
const e2eOutputExists = fs.existsSync(`/tmp/tea-automate-e2e-tests-${timestamp}.json`);
if (!apiOutputExists || !e2eOutputExists) {
throw new Error('One or both subprocess outputs missing!');
}
```
---
### 5. Performance Report
**Display performance metrics:**
```
🚀 Performance Report:
- Execution Mode: PARALLEL (2 subprocesses)
- API Test Generation: ~X minutes
- E2E Test Generation: ~Y minutes
- Total Elapsed: ~max(X, Y) minutes
- Sequential Would Take: ~(X + Y) minutes
- Performance Gain: ~50% faster!
```
---
### 6. Proceed to Aggregation
**Load aggregation step:**
Load next step: `{nextStepFile}`
The aggregation step (3C) will:
- Read both subprocess outputs
- Write all test files to disk
- Generate shared fixtures and helpers
- Calculate summary statistics
---
## EXIT CONDITION
Proceed to Step 3C (Aggregation) when:
- ✅ Subprocess A (API tests) completed successfully
- ✅ Subprocess B (E2E tests) completed successfully
- ✅ Both output files exist and are valid JSON
- ✅ Performance metrics displayed
**Do NOT proceed if:**
- ❌ One or both subprocesses failed
- ❌ Output files missing or corrupted
- ❌ Timeout occurred (subprocesses took too long)
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Both subprocesses launched successfully
- Both subprocesses completed without errors
- Output files generated and valid
- Parallel execution achieved ~50% performance gain
### ❌ SYSTEM FAILURE:
- Failed to launch subprocesses
- One or both subprocesses failed
- Output files missing or invalid
- Attempted sequential generation instead of parallel
**Master Rule:** Parallel subprocess execution is MANDATORY for performance.
@@ -0,0 +1,183 @@
---
name: 'step-03a-subprocess-api'
description: 'Subprocess: Generate API tests only'
subprocess: true
outputFile: '/tmp/tea-automate-api-tests-{{timestamp}}.json'
---
# Subprocess 3A: Generate API Tests
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with E2E test generation.
**What you have from parent workflow:**
- Target features/components identified in Step 2
- Knowledge fragments loaded: api-request, data-factories, api-testing-patterns
- Config: test framework, Playwright Utils enabled/disabled
- Coverage plan: which API endpoints need testing
**Your task:** Generate API tests ONLY (not E2E, not fixtures, not other test types).
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subprocess file before acting
- ✅ Generate API tests ONLY
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate E2E tests (that's subprocess 3B)
- ❌ Do NOT run tests (that's step 4)
- ❌ Do NOT generate fixtures yet (that's step 3C aggregation)
---
## SUBPROCESS TASK
### 1. Identify API Endpoints
From the coverage plan (Step 2 output), identify:
- Which API endpoints need test coverage
- Expected request/response formats
- Authentication requirements
- Error scenarios to test
### 2. Generate API Test Files
For each API endpoint, create test file in `tests/api/[feature].spec.ts`:
**Test Structure:**
```typescript
import { test, expect } from '@playwright/test';
// If Playwright Utils enabled:
// import { apiRequest } from '@playwright-utils/api';
test.describe('[Feature] API Tests', () => {
test('[P0] should handle successful [operation]', async ({ request }) => {
// Use apiRequest helper if Playwright Utils enabled
// Otherwise use standard request fixture
const response = await request.post('/api/endpoint', {
data: {
/* test data */
},
});
expect(response.status()).toBe(200);
expect(await response.json()).toMatchObject({
/* expected */
});
});
test('[P1] should handle [error scenario]', async ({ request }) => {
// Test error handling
});
});
```
**Requirements:**
- ✅ Use `apiRequest()` helper if Playwright Utils enabled (from api-request fragment)
- ✅ Use data factories for test data (from data-factories fragment)
- ✅ Follow API testing patterns (from api-testing-patterns fragment)
- ✅ Include priority tags [P0], [P1], [P2], [P3]
- ✅ Test both happy path and error scenarios
- ✅ Use proper TypeScript types
- ✅ Deterministic assertions (no timing dependencies)
### 3. Track Fixture Needs
Identify fixtures needed for API tests:
- Authentication fixtures (auth tokens, API keys)
- Data factories (user data, product data, etc.)
- API client configurations
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-automate-api-tests-{{timestamp}}.json`
```json
{
"success": true,
"subprocess": "api-tests",
"tests": [
{
"file": "tests/api/auth.spec.ts",
"content": "[full TypeScript test file content]",
"description": "API tests for authentication endpoints",
"priority_coverage": {
"P0": 3,
"P1": 2,
"P2": 1,
"P3": 0
}
},
{
"file": "tests/api/checkout.spec.ts",
"content": "[full TypeScript test file content]",
"description": "API tests for checkout endpoints",
"priority_coverage": {
"P0": 2,
"P1": 3,
"P2": 1,
"P3": 0
}
}
],
"fixture_needs": ["authToken", "userDataFactory", "productDataFactory"],
"knowledge_fragments_used": ["api-request", "data-factories", "api-testing-patterns"],
"test_count": 12,
"summary": "Generated 12 API test cases covering 3 features"
}
```
**On Error:**
```json
{
"success": false,
"subprocess": "api-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subprocess completes when:
- ✅ All API endpoints have test files generated
- ✅ All tests follow knowledge fragment patterns
- ✅ JSON output written to temp file
- ✅ Fixture needs tracked
**Subprocess terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBPROCESS SUCCESS METRICS
### ✅ SUCCESS:
- All API tests generated following patterns
- JSON output valid and complete
- No E2E/component/unit tests included (out of scope)
### ❌ FAILURE:
- Generated tests other than API tests
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output
- Ran tests (not subprocess responsibility)
@@ -0,0 +1,213 @@
---
name: 'step-03b-subprocess-e2e'
description: 'Subprocess: Generate E2E tests only'
subprocess: true
outputFile: '/tmp/tea-automate-e2e-tests-{{timestamp}}.json'
---
# Subprocess 3B: Generate E2E Tests
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with API test generation.
**What you have from parent workflow:**
- Target features/user journeys identified in Step 2
- Knowledge fragments loaded: fixture-architecture, network-first, selector-resilience
- Config: test framework, Playwright Utils enabled/disabled
- Coverage plan: which user journeys need E2E testing
**Your task:** Generate E2E tests ONLY (not API, not fixtures, not other test types).
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subprocess file before acting
- ✅ Generate E2E tests ONLY
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate API tests (that's subprocess 3A)
- ❌ Do NOT run tests (that's step 4)
- ❌ Do NOT generate fixtures yet (that's step 3C aggregation)
---
## SUBPROCESS TASK
### 1. Identify User Journeys
From the coverage plan (Step 2 output), identify:
- Which user journeys need E2E coverage
- Critical user paths (authentication, checkout, profile, etc.)
- UI interactions required
- Expected visual states
### 2. Browser Interaction (Selector Verification)
**Automation mode:** `config.tea_browser_automation`
If `auto` (fall back to MCP if CLI unavailable; if neither available, generate from best practices):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-automate-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-automate-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}` → `page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}` → `page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-automate-{{timestamp}} close` when done
If `cli` (CLI only — do NOT fall back to MCP; generate from best practices if CLI unavailable):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-automate-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-automate-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}` → `page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}` → `page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-automate-{{timestamp}} close` when done
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-automate-{{timestamp}} close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
If `mcp`:
- Use MCP tools for selector verification (current behavior)
If `none`:
- Generate selectors from best practices without browser verification
### 3. Generate E2E Test Files
For each user journey, create test file in `tests/e2e/[feature].spec.ts`:
**Test Structure:**
```typescript
import { test, expect } from '@playwright/test';
test.describe('[Feature] E2E User Journey', () => {
test('[P0] should complete [user journey]', async ({ page }) => {
// Navigate to starting point
await page.goto('/feature');
// Interact with UI
await page.getByRole('button', { name: 'Submit' }).click();
// Assert expected state
await expect(page.getByText('Success')).toBeVisible();
});
test('[P1] should handle [edge case]', async ({ page }) => {
// Test edge case scenario
});
});
```
**Requirements:**
- ✅ Follow fixture architecture patterns (from fixture-architecture fragment)
- ✅ Use network-first patterns: intercept before navigate (from network-first fragment)
- ✅ Use resilient selectors: getByRole, getByText, getByLabel (from selector-resilience fragment)
- ✅ Include priority tags [P0], [P1], [P2], [P3]
- ✅ Test complete user journeys (not isolated clicks)
- ✅ Use proper TypeScript types
- ✅ Deterministic waits (no hard sleeps, use expect().toBeVisible())
### 4. Track Fixture Needs
Identify fixtures needed for E2E tests:
- Page object models (if complex)
- Authentication fixtures (logged-in user state)
- Network mocks/intercepts
- Test data fixtures
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-automate-e2e-tests-{{timestamp}}.json`
```json
{
"success": true,
"subprocess": "e2e-tests",
"tests": [
{
"file": "tests/e2e/authentication.spec.ts",
"content": "[full TypeScript test file content]",
"description": "E2E tests for user authentication journey",
"priority_coverage": {
"P0": 2,
"P1": 3,
"P2": 2,
"P3": 0
}
},
{
"file": "tests/e2e/checkout.spec.ts",
"content": "[full TypeScript test file content]",
"description": "E2E tests for checkout journey",
"priority_coverage": {
"P0": 3,
"P1": 2,
"P2": 1,
"P3": 0
}
}
],
"fixture_needs": ["authenticatedUserFixture", "paymentMockFixture", "checkoutDataFixture"],
"knowledge_fragments_used": ["fixture-architecture", "network-first", "selector-resilience"],
"test_count": 15,
"summary": "Generated 15 E2E test cases covering 5 user journeys"
}
```
**On Error:**
```json
{
"success": false,
"subprocess": "e2e-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subprocess completes when:
- ✅ All user journeys have E2E test files generated
- ✅ All tests follow knowledge fragment patterns
- ✅ JSON output written to temp file
- ✅ Fixture needs tracked
**Subprocess terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBPROCESS SUCCESS METRICS
### ✅ SUCCESS:
- All E2E tests generated following patterns
- JSON output valid and complete
- No API/component/unit tests included (out of scope)
- Resilient selectors used (getByRole, getByText)
- Network-first patterns applied (intercept before navigate)
### ❌ FAILURE:
- Generated tests other than E2E tests
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output
- Ran tests (not subprocess responsibility)
- Used brittle selectors (CSS classes, XPath)
@@ -0,0 +1,325 @@
---
name: 'step-03c-aggregate'
description: 'Aggregate subprocess outputs and complete test infrastructure'
outputFile: '{test_artifacts}/automation-summary.md'
nextStepFile: './step-04-validate-and-summarize.md'
---
# Step 3C: Aggregate Test Generation Results
## STEP GOAL
Read outputs from parallel subprocesses (API + E2E test generation), aggregate results, and create supporting infrastructure (fixtures, helpers).
---
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Read subprocess outputs from temp files
- ✅ Generate shared fixtures based on fixture needs from both subprocesses
- ✅ Write all generated test files to disk
- ❌ Do NOT regenerate tests (use subprocess outputs)
- ❌ Do NOT run tests yet (that's step 4)
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, subprocess outputs from temp files
- Focus: aggregation and fixture generation only
- Limits: do not execute future steps
- Dependencies: Step 3A and 3B subprocess outputs
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Read Subprocess Outputs
**Read API test subprocess output:**
```javascript
const apiTestsPath = '/tmp/tea-automate-api-tests-{{timestamp}}.json';
const apiTestsOutput = JSON.parse(fs.readFileSync(apiTestsPath, 'utf8'));
```
**Read E2E test subprocess output:**
```javascript
const e2eTestsPath = '/tmp/tea-automate-e2e-tests-{{timestamp}}.json';
const e2eTestsOutput = JSON.parse(fs.readFileSync(e2eTestsPath, 'utf8'));
```
**Verify both subprocesses succeeded:**
- Check `apiTestsOutput.success === true`
- Check `e2eTestsOutput.success === true`
- If either failed, report error and stop (don't proceed)
---
### 2. Write All Test Files to Disk
**Write API test files:**
```javascript
apiTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created: ${test.file}`);
});
```
**Write E2E test files:**
```javascript
e2eTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created: ${test.file}`);
});
```
---
### 3. Aggregate Fixture Needs
**Collect all fixture needs from both subprocesses:**
```javascript
const allFixtureNeeds = [...apiTestsOutput.fixture_needs, ...e2eTestsOutput.fixture_needs];
// Remove duplicates
const uniqueFixtures = [...new Set(allFixtureNeeds)];
```
**Categorize fixtures:**
- **Authentication fixtures:** authToken, authenticatedUserFixture, etc.
- **Data factories:** userDataFactory, productDataFactory, etc.
- **Network mocks:** paymentMockFixture, apiResponseMocks, etc.
- **Test helpers:** wait/retry/assertion helpers
---
### 4. Generate Fixture Infrastructure
**Create or update fixture files based on needs:**
**A) Authentication Fixtures** (`tests/fixtures/auth.ts`):
```typescript
import { test as base } from '@playwright/test';
export const test = base.extend({
authenticatedUser: async ({ page }, use) => {
// Login logic
await page.goto('/login');
await page.fill('[name="email"]', 'test@example.com');
await page.fill('[name="password"]', 'password');
await page.click('button[type="submit"]');
await page.waitForURL('/dashboard');
await use(page);
},
authToken: async ({ request }, use) => {
// Get auth token for API tests
const response = await request.post('/api/auth/login', {
data: { email: 'test@example.com', password: 'password' },
});
const { token } = await response.json();
await use(token);
},
});
```
**B) Data Factories** (`tests/fixtures/data-factories.ts`):
```typescript
import { faker } from '@faker-js/faker';
export const createUserData = (overrides = {}) => ({
name: faker.person.fullName(),
email: faker.internet.email(),
...overrides,
});
export const createProductData = (overrides = {}) => ({
name: faker.commerce.productName(),
price: faker.number.int({ min: 10, max: 1000 }),
...overrides,
});
```
**C) Network Mocks** (`tests/fixtures/network-mocks.ts`):
```typescript
import { Page } from '@playwright/test';
export const mockPaymentSuccess = async (page: Page) => {
await page.route('/api/payment/**', (route) => {
route.fulfill({
status: 200,
body: JSON.stringify({ success: true, transactionId: '12345' }),
});
});
};
```
**D) Helper Utilities** (`tests/fixtures/helpers.ts`):
```typescript
import { expect, Page } from '@playwright/test';
export const waitForApiResponse = async (page: Page, urlPattern: string) => {
return page.waitForResponse((response) => response.url().includes(urlPattern) && response.ok());
};
```
---
### 5. Calculate Summary Statistics
**Aggregate test counts:**
```javascript
const summary = {
total_tests: apiTestsOutput.test_count + e2eTestsOutput.test_count,
api_tests: apiTestsOutput.test_count,
e2e_tests: e2eTestsOutput.test_count,
fixtures_created: uniqueFixtures.length,
api_test_files: apiTestsOutput.tests.length,
e2e_test_files: e2eTestsOutput.tests.length,
priority_coverage: {
P0: /* sum P0 tests from both */,
P1: /* sum P1 tests from both */,
P2: /* sum P2 tests from both */,
P3: /* sum P3 tests from both */
},
knowledge_fragments_used: [
...apiTestsOutput.knowledge_fragments_used,
...e2eTestsOutput.knowledge_fragments_used
],
subprocess_execution: 'PARALLEL (API + E2E)',
performance_gain: '~50% faster than sequential'
};
```
**Store summary for Step 4:**
Save summary to temp file for validation step:
```javascript
fs.writeFileSync('/tmp/tea-automate-summary-{{timestamp}}.json', JSON.stringify(summary, null, 2), 'utf8');
```
---
### 6. Optional Cleanup
**Clean up subprocess temp files** (optional - can keep for debugging):
```javascript
fs.unlinkSync(apiTestsPath);
fs.unlinkSync(e2eTestsPath);
console.log('✅ Subprocess temp files cleaned up');
```
---
## OUTPUT SUMMARY
Display to user:
```
✅ Test Generation Complete (Parallel Execution)
📊 Summary:
- Total Tests: {total_tests}
- API Tests: {api_tests} ({api_test_files} files)
- E2E Tests: {e2e_tests} ({e2e_test_files} files)
- Fixtures Created: {fixtures_created}
- Priority Coverage:
- P0 (Critical): {P0} tests
- P1 (High): {P1} tests
- P2 (Medium): {P2} tests
- P3 (Low): {P3} tests
🚀 Performance: Parallel execution ~50% faster than sequential
📂 Generated Files:
- tests/api/[feature].spec.ts
- tests/e2e/[feature].spec.ts
- tests/fixtures/auth.ts
- tests/fixtures/data-factories.ts
- tests/fixtures/network-mocks.ts
- tests/fixtures/helpers.ts
✅ Ready for validation (Step 4)
```
---
## EXIT CONDITION
Proceed to Step 4 when:
- ✅ All test files written to disk (API + E2E)
- ✅ All fixtures and helpers created
- ✅ Summary statistics calculated and saved
- ✅ Output displayed to user
---
### 7. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03c-aggregate']
lastStep: 'step-03c-aggregate'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03c-aggregate'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03c-aggregate'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Both subprocesses succeeded
- All test files written to disk
- Fixtures generated based on subprocess needs
- Summary complete and accurate
### ❌ SYSTEM FAILURE:
- One or both subprocesses failed
- Test files not written to disk
- Fixtures missing or incomplete
- Summary missing or inaccurate
**Master Rule:** Do NOT proceed to Step 4 if aggregation incomplete.
@@ -0,0 +1,95 @@
---
name: 'step-04-validate-and-summarize'
description: 'Validate outputs and produce automation summary'
outputFile: '{test_artifacts}/automation-summary.md'
---
# Step 4: Validate & Summarize
## STEP GOAL
Validate generated outputs and produce a concise automation summary.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Validate against the checklist before completion
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validate
Use `checklist.md` to validate:
- Framework readiness
- Coverage mapping
- Test quality and structure
- Fixtures, factories, helpers
- [ ] CLI sessions cleaned up (no orphaned browsers)
- [ ] Temp artifacts stored in `{test_artifacts}/` not random locations
Fix gaps before proceeding.
---
## 2. Summary Output
Write `{outputFile}` including:
- Coverage plan by test level and priority
- Files created/updated
- Key assumptions and risks
- Next recommended workflow (e.g., `test-review` or `trace`)
---
## 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04-validate-and-summarize']
lastStep: 'step-04-validate-and-summarize'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04-validate-and-summarize'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04-validate-and-summarize'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target
@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update
@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/automate-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced
@@ -0,0 +1,72 @@
---
validationDate: 2026-01-27
workflowName: testarch-automate
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/automate
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-automate
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 113 lines [GOOD]
- steps-c/step-02-identify-targets.md: 85 lines [GOOD]
- steps-c/step-03-generate-tests.md: 76 lines [GOOD]
- steps-c/step-04-validate-and-summarize.md: 62 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- No templates found in workflow root
- Steps with outputFile in frontmatter:
- steps-c/step-04-validate-and-summarize.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,114 @@
---
validationDate: 2026-01-27
workflowName: testarch-automate
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/automate
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-automate
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 112 lines [GOOD]
- steps-c/step-02-identify-targets.md: 84 lines [GOOD]
- steps-c/step-03-generate-tests.md: 75 lines [GOOD]
- steps-c/step-04-validate-and-summarize.md: 61 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight-and-context.md: Init [PASS]
- steps-c/step-02-identify-targets.md: Middle [PASS]
- steps-c/step-03-generate-tests.md: Middle [PASS]
- steps-c/step-04-validate-and-summarize.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: NONE
- Steps with outputFile in frontmatter:
- steps-c/step-04-validate-and-summarize.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subprocess Optimization Opportunities
- No high-priority subprocess optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 7 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,20 @@
# Workflow Plan: testarch-automate
## Create Mode (steps-c)
- step-01-preflight-and-context.md
- step-02-identify-targets.md
- step-03-generate-tests.md
- step-04-validate-and-summarize.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- {test_artifacts}/automation-summary.md
- Test files under {project-root}/tests
@@ -0,0 +1,41 @@
---
name: testarch-automate
description: 'Expand test automation coverage after implementation or analyze existing codebase to generate comprehensive test suite'
web_bundle: true
---
# Test Automation Expansion
**Goal:** Expand test automation coverage after implementation or analyze existing codebase to generate comprehensive test suite
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight-and-context.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`
@@ -0,0 +1,54 @@
# Test Architect workflow: automate
name: testarch-automate
description: "Expand test automation coverage after implementation or analyze existing codebase to generate comprehensive test suite"
author: "BMad"
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/automate"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
template: false
# Variables and inputs
variables:
# Execution mode and targeting
standalone_mode: true # Can work without BMad artifacts (true) or integrate with BMad (false)
coverage_target: "critical-paths" # critical-paths, comprehensive, selective
# Directory paths
test_dir: "{project-root}/tests" # Root test directory
source_dir: "{project-root}" # Source code directory (customize if needed, e.g., {project-root}/src or {project-root}/lib)
# Output configuration
default_output_file: "{test_artifacts}/automation-summary.md"
# Required tools
required_tools:
- read_file # Read source code, existing tests, BMad artifacts
- write_file # Create test files, fixtures, factories, summaries
- create_directory # Create test directories
- list_files # Discover features and existing tests
- search_repo # Find coverage gaps and patterns
- glob # Find test files and source files
tags:
- qa
- automation
- test-architect
- regression
- coverage
execution_hints:
interactive: false # Minimize prompts
autonomous: true # Proceed without user input unless blocked
iterative: true
␍
@@ -0,0 +1,247 @@
# CI/CD Pipeline Setup - Validation Checklist
## Prerequisites
- [ ] Git repository initialized (`.git/` exists)
- [ ] Git remote configured (`git remote -v` shows origin)
- [ ] Test framework configured (`playwright.config._` or `cypress.config._`)
- [ ] Local tests pass (`npm run test:e2e` succeeds)
- [ ] Team agrees on CI platform
- [ ] Access to CI platform settings (if updating)
Note: CI setup is typically a one-time task per repo and can be run any time after the test framework is configured.
## Process Steps
### Step 1: Preflight Checks
- [ ] Git repository validated
- [ ] Framework configuration detected
- [ ] Local test execution successful
- [ ] CI platform detected or selected
- [ ] Node version identified (.nvmrc or default)
- [ ] No blocking issues found
### Step 2: CI Pipeline Configuration
- [ ] CI configuration file created (`.github/workflows/test.yml` or `.gitlab-ci.yml`)
- [ ] File is syntactically valid (no YAML errors)
- [ ] Correct framework commands configured
- [ ] Node version matches project
- [ ] Test directory paths correct
### Step 3: Parallel Sharding
- [ ] Matrix strategy configured (4 shards default)
- [ ] Shard syntax correct for framework
- [ ] fail-fast set to false
- [ ] Shard count appropriate for test suite size
### Step 4: Burn-In Loop
- [ ] Burn-in job created
- [ ] 10 iterations configured
- [ ] Proper exit on failure (`|| exit 1`)
- [ ] Runs on appropriate triggers (PR, cron)
- [ ] Failure artifacts uploaded
### Step 5: Caching Configuration
- [ ] Dependency cache configured (npm/yarn)
- [ ] Cache key uses lockfile hash
- [ ] Browser cache configured (Playwright/Cypress)
- [ ] Restore-keys defined for fallback
- [ ] Cache paths correct for platform
### Step 6: Artifact Collection
- [ ] Artifacts upload on failure only
- [ ] Correct artifact paths (test-results/, traces/, etc.)
- [ ] Retention days set (30 default)
- [ ] Artifact names unique per shard
- [ ] No sensitive data in artifacts
### Step 7: Retry Logic
- [ ] Retry action/strategy configured
- [ ] Max attempts: 2-3
- [ ] Timeout appropriate (30 min)
- [ ] Retry only on transient errors
### Step 8: Helper Scripts
- [ ] `scripts/test-changed.sh` created
- [ ] `scripts/ci-local.sh` created
- [ ] `scripts/burn-in.sh` created (optional)
- [ ] Scripts are executable (`chmod +x`)
- [ ] Scripts use correct test commands
- [ ] Shebang present (`#!/bin/bash`)
### Step 9: Documentation
- [ ] `docs/ci.md` created with pipeline guide
- [ ] `docs/ci-secrets-checklist.md` created
- [ ] Required secrets documented
- [ ] Setup instructions clear
- [ ] Troubleshooting section included
- [ ] Badge URLs provided (optional)
## Output Validation
### Configuration Validation
- [ ] CI file loads without errors
- [ ] All paths resolve correctly
- [ ] No hardcoded values (use env vars)
- [ ] Triggers configured (push, pull_request, schedule)
- [ ] Platform-specific syntax correct
### Execution Validation
- [ ] First CI run triggered (push to remote)
- [ ] Pipeline starts without errors
- [ ] All jobs appear in CI dashboard
- [ ] Caching works (check logs for cache hit)
- [ ] Tests execute in parallel
- [ ] Artifacts collected on failure
### Performance Validation
- [ ] Lint stage: <2 minutes
- [ ] Test stage (per shard): <10 minutes
- [ ] Burn-in stage: <30 minutes
- [ ] Total pipeline: <45 minutes
- [ ] Cache reduces install time by 2-5 minutes
## Quality Checks
### Best Practices Compliance
- [ ] Burn-in loop follows production patterns
- [ ] Parallel sharding configured optimally
- [ ] Failure-only artifact collection
- [ ] Selective testing enabled (optional)
- [ ] Retry logic handles transient failures only
- [ ] No secrets in configuration files
### Knowledge Base Alignment
- [ ] Burn-in pattern matches `ci-burn-in.md`
- [ ] Selective testing matches `selective-testing.md`
- [ ] Artifact collection matches `visual-debugging.md`
- [ ] Test quality matches `test-quality.md`
### Security Checks
- [ ] No credentials in CI configuration
- [ ] Secrets use platform secret management
- [ ] Environment variables for sensitive data
- [ ] Artifact retention appropriate (not too long)
- [ ] No debug output exposing secrets
## Integration Points
### Status File Integration
- [ ] CI setup logged in Quality & Testing Progress section
- [ ] Status updated with completion timestamp
- [ ] Platform and configuration noted
### Knowledge Base Integration
- [ ] Relevant knowledge fragments loaded
- [ ] Patterns applied from knowledge base
- [ ] Documentation references knowledge base
- [ ] Knowledge base references in README
### Workflow Dependencies
- [ ] `framework` workflow completed first
- [ ] Can proceed to `atdd` workflow after CI setup
- [ ] Can proceed to `automate` workflow
- [ ] CI integrates with `gate` workflow
## Completion Criteria
**All must be true:**
- [ ] All prerequisites met
- [ ] All process steps completed
- [ ] All output validations passed
- [ ] All quality checks passed
- [ ] All integration points verified
- [ ] First CI run successful
- [ ] Performance targets met
- [ ] Documentation complete
## Post-Workflow Actions
**User must complete:**
1. [ ] Commit CI configuration
2. [ ] Push to remote repository
3. [ ] Configure required secrets in CI platform
4. [ ] Open PR to trigger first CI run
5. [ ] Monitor and verify pipeline execution
6. [ ] Adjust parallelism if needed (based on actual run times)
7. [ ] Set up notifications (optional)
**Recommended next workflows:**
1. [ ] Run `atdd` workflow for test generation
2. [ ] Run `automate` workflow for coverage expansion
3. [ ] Run `gate` workflow for quality gates
## Rollback Procedure
If workflow fails:
1. [ ] Delete CI configuration file
2. [ ] Remove helper scripts directory
3. [ ] Remove documentation (docs/ci.md, etc.)
4. [ ] Clear CI platform secrets (if added)
5. [ ] Review error logs
6. [ ] Fix issues and retry workflow
## Notes
### Common Issues
**Issue**: CI file syntax errors
- **Solution**: Validate YAML syntax online or with linter
**Issue**: Tests fail in CI but pass locally
- **Solution**: Use `scripts/ci-local.sh` to mirror CI environment
**Issue**: Caching not working
- **Solution**: Check cache key formula, verify paths
**Issue**: Burn-in too slow
- **Solution**: Reduce iterations or run on cron only
### Platform-Specific
**GitHub Actions:**
- Secrets: Repository Settings → Secrets and variables → Actions
- Runners: Ubuntu latest recommended
- Concurrency limits: 20 jobs for free tier
**GitLab CI:**
- Variables: Project Settings → CI/CD → Variables
- Runners: Shared or project-specific
- Pipeline quota: 400 minutes/month free tier
---
**Checklist Complete**: Sign off when all items validated.
**Completed by:** {name}
**Date:** {date}
**Platform:** {GitHub Actions, GitLab CI, Other}
**Notes:** {notes}
@@ -0,0 +1,198 @@
# GitHub Actions CI/CD Pipeline for Test Execution
# Generated by BMad TEA Agent - Test Architect Module
# Optimized for: Playwright/Cypress, Parallel Sharding, Burn-In Loop
name: Test Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
schedule:
# Weekly burn-in on Sundays at 2 AM UTC
- cron: "0 2 * * 0"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# Lint stage - Code quality checks
lint:
name: Lint
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- name: Determine Node version
id: node-version
run: |
if [ -f .nvmrc ]; then
echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
echo "Using Node from .nvmrc"
else
echo "value=24" >> "$GITHUB_OUTPUT"
echo "Using default Node 24 (current LTS)"
fi
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ steps.node-version.outputs.value }}
cache: "npm"
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
# Test stage - Parallel execution with sharding
test:
name: Test (Shard ${{ matrix.shard }})
runs-on: ubuntu-latest
timeout-minutes: 30
needs: lint
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
- name: Determine Node version
id: node-version
run: |
if [ -f .nvmrc ]; then
echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
echo "Using Node from .nvmrc"
else
echo "value=22" >> "$GITHUB_OUTPUT"
echo "Using default Node 22 (current LTS)"
fi
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ steps.node-version.outputs.value }}
cache: "npm"
- name: Cache Playwright browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-playwright-
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
- name: Run tests (shard ${{ matrix.shard }}/4)
run: npm run test:e2e -- --shard=${{ matrix.shard }}/4
- name: Upload test results
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ matrix.shard }}
path: |
test-results/
playwright-report/
retention-days: 30
# Burn-in stage - Flaky test detection
burn-in:
name: Burn-In (Flaky Detection)
runs-on: ubuntu-latest
timeout-minutes: 60
needs: test
# Only run burn-in on PRs to main/develop or on schedule
if: github.event_name == 'pull_request' || github.event_name == 'schedule'
steps:
- uses: actions/checkout@v4
- name: Determine Node version
id: node-version
run: |
if [ -f .nvmrc ]; then
echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
echo "Using Node from .nvmrc"
else
echo "value=22" >> "$GITHUB_OUTPUT"
echo "Using default Node 22 (current LTS)"
fi
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ steps.node-version.outputs.value }}
cache: "npm"
- name: Cache Playwright browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
- name: Run burn-in loop (10 iterations)
run: |
echo "🔥 Starting burn-in loop - detecting flaky tests"
for i in {1..10}; do
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🔥 Burn-in iteration $i/10"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
npm run test:e2e || exit 1
done
echo "✅ Burn-in complete - no flaky tests detected"
- name: Upload burn-in failure artifacts
if: failure()
uses: actions/upload-artifact@v4
with:
name: burn-in-failures
path: |
test-results/
playwright-report/
retention-days: 30
# Report stage - Aggregate and publish results
report:
name: Test Report
runs-on: ubuntu-latest
needs: [test, burn-in]
if: always()
steps:
- name: Download all artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
- name: Generate summary
run: |
echo "## Test Execution Summary" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- **Status**: ${{ needs.test.result }}" >> $GITHUB_STEP_SUMMARY
echo "- **Burn-in**: ${{ needs.burn-in.result }}" >> $GITHUB_STEP_SUMMARY
echo "- **Shards**: 4" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "${{ needs.burn-in.result }}" == "failure" ]; then
echo "⚠️ **Flaky tests detected** - Review burn-in artifacts" >> $GITHUB_STEP_SUMMARY
fi
@@ -0,0 +1,149 @@
# GitLab CI/CD Pipeline for Test Execution
# Generated by BMad TEA Agent - Test Architect Module
# Optimized for: Playwright/Cypress, Parallel Sharding, Burn-In Loop
stages:
- lint
- test
- burn-in
- report
variables:
# Disable git depth for accurate change detection
GIT_DEPTH: 0
# Use npm ci for faster, deterministic installs
npm_config_cache: "$CI_PROJECT_DIR/.npm"
# Playwright browser cache
PLAYWRIGHT_BROWSERS_PATH: "$CI_PROJECT_DIR/.cache/ms-playwright"
# Default Node version when .nvmrc is missing
DEFAULT_NODE_VERSION: "24"
# Caching configuration
cache:
key:
files:
- package-lock.json
paths:
- .npm/
- .cache/ms-playwright/
- node_modules/
# Lint stage - Code quality checks
lint:
stage: lint
image: node:$DEFAULT_NODE_VERSION
before_script:
- |
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "$DEFAULT_NODE_VERSION")
echo "Using Node $NODE_VERSION"
npm install -g n
n "$NODE_VERSION"
node -v
- npm ci
script:
- npm run lint
timeout: 5 minutes
# Test stage - Parallel execution with sharding
.test-template: &test-template
stage: test
image: node:$DEFAULT_NODE_VERSION
needs:
- lint
before_script:
- |
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "$DEFAULT_NODE_VERSION")
echo "Using Node $NODE_VERSION"
npm install -g n
n "$NODE_VERSION"
node -v
- npm ci
- npx playwright install --with-deps chromium
artifacts:
when: on_failure
paths:
- test-results/
- playwright-report/
expire_in: 30 days
timeout: 30 minutes
test:shard-1:
<<: *test-template
script:
- npm run test:e2e -- --shard=1/4
test:shard-2:
<<: *test-template
script:
- npm run test:e2e -- --shard=2/4
test:shard-3:
<<: *test-template
script:
- npm run test:e2e -- --shard=3/4
test:shard-4:
<<: *test-template
script:
- npm run test:e2e -- --shard=4/4
# Burn-in stage - Flaky test detection
burn-in:
stage: burn-in
image: node:$DEFAULT_NODE_VERSION
needs:
- test:shard-1
- test:shard-2
- test:shard-3
- test:shard-4
# Only run burn-in on merge requests to main/develop or on schedule
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "schedule"'
before_script:
- |
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "$DEFAULT_NODE_VERSION")
echo "Using Node $NODE_VERSION"
npm install -g n
n "$NODE_VERSION"
node -v
- npm ci
- npx playwright install --with-deps chromium
script:
- |
echo "🔥 Starting burn-in loop - detecting flaky tests"
for i in {1..10}; do
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🔥 Burn-in iteration $i/10"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
npm run test:e2e || exit 1
done
echo "✅ Burn-in complete - no flaky tests detected"
artifacts:
when: on_failure
paths:
- test-results/
- playwright-report/
expire_in: 30 days
timeout: 60 minutes
# Report stage - Aggregate results
report:
stage: report
image: alpine:latest
needs:
- test:shard-1
- test:shard-2
- test:shard-3
- test:shard-4
- burn-in
when: always
script:
- |
echo "## Test Execution Summary"
echo ""
echo "- Pipeline: $CI_PIPELINE_ID"
echo "- Shards: 4"
echo "- Branch: $CI_COMMIT_REF_NAME"
echo ""
echo "View detailed results in job artifacts"
@@ -0,0 +1,45 @@
<!-- Powered by BMAD-CORE™ -->
# CI/CD Pipeline Setup
**Workflow ID**: `_bmad/tea/testarch/ci`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Scaffold a production-ready CI/CD quality pipeline with test execution, burn-in loops for flaky detection, parallel sharding, artifact collection, and notifications.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `ci_platform`, `test_dir`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/ci/steps-c/step-01-preflight.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/ci/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.
@@ -0,0 +1,117 @@
---
name: 'step-01-preflight'
description: 'Verify prerequisites and detect CI platform'
nextStepFile: './step-02-generate-pipeline.md'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 1: Preflight Checks
## STEP GOAL
Verify CI prerequisites and determine target CI platform.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if requirements fail
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Verify Git Repository
- `.git/` exists
- Remote configured (if available)
If missing: **HALT** with "Git repository required for CI/CD setup."
---
## 2. Verify Test Framework
- `playwright.config.*` or `cypress.config.*` exists
- Framework installed in `package.json`
If missing: **HALT** with "Run `framework` workflow first."
---
## 3. Ensure Tests Pass Locally
- Run the main test command (e.g., `npm run test:e2e`)
- If failing: **HALT** and request fixes before CI setup
---
## 4. Detect CI Platform
- Check for existing CI config:
- `.github/workflows/*.yml` (GitHub Actions)
- `.gitlab-ci.yml` (GitLab CI)
- `.circleci/config.yml` (Circle CI)
- `Jenkinsfile`
- If found, ask whether to update or replace
- If not found, infer from git remote (github.com → GitHub Actions)
- Respect `ci_platform` if explicitly set
---
## 5. Read Environment Context
- Read `.nvmrc` if present (default to Node 24+ LTS if missing)
- Read `package.json` for dependency caching strategy
---
### 6. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight']
lastStep: 'step-01-preflight'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,110 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing progress document, displaying progress, verifying previously created artifacts, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Verify Previously Created Artifacts
Since this is a file-creation workflow, verify that artifacts from completed steps still exist on disk:
- If `step-02-generate-pipeline` is in `stepsCompleted`, check that the pipeline config file exists (e.g., `.github/workflows/test.yml` or equivalent)
- If any expected artifact is missing, warn the user and suggest re-running from the step that creates it
---
### 3. Display Progress Dashboard
Display:
"📋 **Workflow Resume — CI/CD Pipeline Setup**
**Last saved:** {lastSaved}
**Steps completed:** {stepsCompleted.length} of 4
1. Preflight Checks (step-01-preflight) — {✅ if in stepsCompleted, ⬜ otherwise}
2. Generate Pipeline (step-02-generate-pipeline) — {✅ if in stepsCompleted, ⬜ otherwise}
3. Configure Quality Gates (step-03-configure-quality-gates) — {✅ if in stepsCompleted, ⬜ otherwise}
4. Validate & Summary (step-04-validate-and-summary) — {✅ if in stepsCompleted, ⬜ otherwise}"
---
### 4. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight'` → Load `./step-02-generate-pipeline.md`
- `'step-02-generate-pipeline'` → Load `./step-03-configure-quality-gates.md`
- `'step-03-configure-quality-gates'` → Load `./step-04-validate-and-summary.md`
- `'step-04-validate-and-summary'` → **Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps. Use it as reference for remaining steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Previously created artifacts verified
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
- Re-executing completed steps
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
@@ -0,0 +1,107 @@
---
name: 'step-02-generate-pipeline'
description: 'Generate CI pipeline configuration'
nextStepFile: './step-03-configure-quality-gates.md'
pipelineOutputFile: '{project-root}/.github/workflows/test.yml'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 2: Generate CI Pipeline
## STEP GOAL
Create platform-specific CI configuration with test execution, sharding, burn-in, and artifacts.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Select Template
Choose template based on platform:
- GitHub Actions → `.github/workflows/test.yml`
- GitLab CI → `.gitlab-ci.yml`
- Circle CI → `.circleci/config.yml`
- Jenkins → `Jenkinsfile`
Use templates from `{installed_path}` when available (e.g., `github-actions-template.yaml`, `gitlab-ci-template.yaml`).
---
## 2. Pipeline Stages
Include stages:
- lint
- test (parallel shards)
- burn-in (flaky detection)
- report (aggregate + publish)
---
## 3. Test Execution
- Parallel sharding enabled
- CI retries configured
- Capture artifacts (HTML report, JUnit XML, traces/videos on failure)
- Cache dependencies (node_modules / pnpm / npm cache)
Write the selected pipeline configuration to `{pipelineOutputFile}` (or adjust the path if a non-GitHub platform was chosen).
---
### 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-generate-pipeline']
lastStep: 'step-02-generate-pipeline'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-generate-pipeline'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-generate-pipeline'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,100 @@
---
name: 'step-03-configure-quality-gates'
description: 'Configure burn-in, quality gates, and notifications'
nextStepFile: './step-04-validate-and-summary.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 3: Quality Gates & Notifications
## STEP GOAL
Configure burn-in loops, quality thresholds, and notification hooks.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Burn-In Configuration
Use `{knowledgeIndex}` to load `ci-burn-in.md` guidance:
- Run N-iteration burn-in for flaky detection
- Gate promotion based on burn-in stability
---
## 2. Quality Gates
Define:
- Minimum pass rates (P0 = 100%, P1 ≥ 95%)
- Fail CI on critical test failures
- Optional: require traceability or nfr-assess output before release
---
## 3. Notifications
Configure:
- Failure notifications (Slack/email)
- Artifact links
---
### 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-configure-quality-gates']
lastStep: 'step-03-configure-quality-gates'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-configure-quality-gates'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-configure-quality-gates'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,92 @@
---
name: 'step-04-validate-and-summary'
description: 'Validate pipeline and summarize'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 4: Validate & Summarize
## STEP GOAL
Validate CI configuration and report completion details.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validation
Validate against `checklist.md`:
- Config file created
- Stages and sharding configured
- Burn-in and artifacts enabled
- Secrets/variables documented
Fix gaps before completion.
---
## 2. Completion Summary
Report:
- CI platform and config path
- Key stages enabled
- Artifacts and notifications
- Next steps (set secrets, run pipeline)
---
### 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04-validate-and-summary']
lastStep: 'step-04-validate-and-summary'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04-validate-and-summary'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04-validate-and-summary'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target
@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update
@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/ci-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced
@@ -0,0 +1,72 @@
---
validationDate: 2026-01-27
workflowName: testarch-ci
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/ci
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-ci
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight.md: 87 lines [GOOD]
- steps-c/step-02-generate-pipeline.md: 75 lines [GOOD]
- steps-c/step-03-configure-quality-gates.md: 67 lines [GOOD]
- steps-c/step-04-validate-and-summary.md: 60 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- No templates found in workflow root
- Steps with outputFile in frontmatter:
- steps-c/step-02-generate-pipeline.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,114 @@
---
validationDate: 2026-01-27
workflowName: testarch-ci
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/ci
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-ci
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight.md: 86 lines [GOOD]
- steps-c/step-02-generate-pipeline.md: 74 lines [GOOD]
- steps-c/step-03-configure-quality-gates.md: 66 lines [GOOD]
- steps-c/step-04-validate-and-summary.md: 59 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight.md: Init [PASS]
- steps-c/step-02-generate-pipeline.md: Middle [PASS]
- steps-c/step-03-configure-quality-gates.md: Middle [PASS]
- steps-c/step-04-validate-and-summary.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: NONE
- Steps with outputFile in frontmatter:
- steps-c/step-02-generate-pipeline.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subprocess Optimization Opportunities
- No high-priority subprocess optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 7 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,20 @@
# Workflow Plan: testarch-ci
## Create Mode (steps-c)
- step-01-preflight.md
- step-02-generate-pipeline.md
- step-03-configure-quality-gates.md
- step-04-validate-and-summary.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- CI config (e.g., {project-root}/.github/workflows/test.yml)
- Pipeline guidance and artifacts configuration
@@ -0,0 +1,41 @@
---
name: testarch-ci
description: 'Scaffold CI/CD quality pipeline with test execution, burn-in loops, and artifact collection'
web_bundle: true
---
# CI/CD Pipeline Setup
**Goal:** Scaffold CI/CD quality pipeline with test execution, burn-in loops, and artifact collection
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`
@@ -0,0 +1,47 @@
# Test Architect workflow: ci
name: testarch-ci
description: "Scaffold CI/CD quality pipeline with test execution, burn-in loops, and artifact collection"
author: "BMad"
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/ci"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
# Variables and inputs
variables:
ci_platform: "auto" # auto, github-actions, gitlab-ci, circle-ci, jenkins - user can override
test_dir: "{project-root}/tests" # Root test directory
# Output configuration
default_output_file: "{project-root}/.github/workflows/test.yml" # GitHub Actions default
# Required tools
required_tools:
- read_file # Read .nvmrc, package.json, framework config
- write_file # Create CI config, scripts, documentation
- create_directory # Create .github/workflows/ or .gitlab-ci/ directories
- list_files # Detect existing CI configuration
- search_repo # Find test files for selective testing
tags:
- qa
- ci-cd
- test-architect
- pipeline
- automation
execution_hints:
interactive: false # Minimize prompts, auto-detect when possible
autonomous: true # Proceed without user input unless blocked
iterative: true
␍
@@ -0,0 +1,320 @@
# Test Framework Setup - Validation Checklist
This checklist ensures the framework workflow completes successfully and all deliverables meet quality standards.
---
## Prerequisites
Before starting the workflow:
- [ ] Project root contains valid `package.json`
- [ ] No existing modern E2E framework detected (`playwright.config.*`, `cypress.config.*`)
- [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, etc.)
- [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable
- [ ] User has write permissions to create directories and files
---
## Process Steps
### Step 1: Preflight Checks
- [ ] package.json successfully read and parsed
- [ ] Project type extracted correctly
- [ ] Bundler identified (or marked as N/A for backend projects)
- [ ] No framework conflicts detected
- [ ] Architecture documents located (if available)
### Step 2: Framework Selection
- [ ] Framework auto-detection logic executed
- [ ] Framework choice justified (Playwright vs Cypress)
- [ ] Framework preference respected (if explicitly set)
- [ ] User notified of framework selection and rationale
### Step 3: Directory Structure
- [ ] `tests/` root directory created
- [ ] `tests/e2e/` directory created (or user's preferred structure)
- [ ] `tests/support/` directory created (critical pattern)
- [ ] `tests/support/fixtures/` directory created
- [ ] `tests/support/fixtures/factories/` directory created
- [ ] `tests/support/helpers/` directory created
- [ ] `tests/support/page-objects/` directory created (if applicable)
- [ ] All directories have correct permissions
**Note**: Test organization is flexible (e2e/, api/, integration/). The **support/** folder is the key pattern.
### Step 4: Configuration Files
- [ ] Framework config file created (`playwright.config.ts` or `cypress.config.ts`)
- [ ] Config file uses TypeScript (if `use_typescript: true`)
- [ ] Timeouts configured correctly (action: 15s, navigation: 30s, test: 60s)
- [ ] Base URL configured with environment variable fallback
- [ ] Trace/screenshot/video set to retain-on-failure
- [ ] Multiple reporters configured (HTML + JUnit + console)
- [ ] Parallel execution enabled
- [ ] CI-specific settings configured (retries, workers)
- [ ] Config file is syntactically valid (no compilation errors)
### Step 5: Environment Configuration
- [ ] `.env.example` created in project root
- [ ] `TEST_ENV` variable defined
- [ ] `BASE_URL` variable defined with default
- [ ] `API_URL` variable defined (if applicable)
- [ ] Authentication variables defined (if applicable)
- [ ] Feature flag variables defined (if applicable)
- [ ] `.nvmrc` created with appropriate Node version
### Step 6: Fixture Architecture
- [ ] `tests/support/fixtures/index.ts` created
- [ ] Base fixture extended from Playwright/Cypress
- [ ] Type definitions for fixtures created
- [ ] mergeTests pattern implemented (if multiple fixtures)
- [ ] Auto-cleanup logic included in fixtures
- [ ] Fixture architecture follows knowledge base patterns
### Step 7: Data Factories
- [ ] At least one factory created (e.g., UserFactory)
- [ ] Factories use @faker-js/faker for realistic data
- [ ] Factories track created entities (for cleanup)
- [ ] Factories implement `cleanup()` method
- [ ] Factories integrate with fixtures
- [ ] Factories follow knowledge base patterns
### Step 8: Sample Tests
- [ ] Example test file created (`tests/e2e/example.spec.ts`)
- [ ] Test uses fixture architecture
- [ ] Test demonstrates data factory usage
- [ ] Test uses proper selector strategy (data-testid)
- [ ] Test follows Given-When-Then structure
- [ ] Test includes proper assertions
- [ ] Network interception demonstrated (if applicable)
### Step 9: Helper Utilities
- [ ] API helper created (if API testing needed)
- [ ] Network helper created (if network mocking needed)
- [ ] Auth helper created (if authentication needed)
- [ ] Helpers follow functional patterns
- [ ] Helpers have proper error handling
### Step 10: Documentation
- [ ] `tests/README.md` created
- [ ] Setup instructions included
- [ ] Running tests section included
- [ ] Architecture overview section included
- [ ] Best practices section included
- [ ] CI integration section included
- [ ] Knowledge base references included
- [ ] Troubleshooting section included
### Step 11: Package.json Updates
- [ ] Minimal test script added to package.json: `test:e2e`
- [ ] Test framework dependency added (if not already present)
- [ ] Type definitions added (if TypeScript)
- [ ] Users can extend with additional scripts as needed
---
## Output Validation
### Configuration Validation
- [ ] Config file loads without errors
- [ ] Config file passes linting (if linter configured)
- [ ] Config file uses correct syntax for chosen framework
- [ ] All paths in config resolve correctly
- [ ] Reporter output directories exist or are created on test run
### Test Execution Validation
- [ ] Sample test runs successfully
- [ ] Test execution produces expected output (pass/fail)
- [ ] Test artifacts generated correctly (traces, screenshots, videos)
- [ ] Test report generated successfully
- [ ] No console errors or warnings during test run
### Directory Structure Validation
- [ ] All required directories exist
- [ ] Directory structure matches framework conventions
- [ ] No duplicate or conflicting directories
- [ ] Directories accessible with correct permissions
### File Integrity Validation
- [ ] All generated files are syntactically correct
- [ ] No placeholder text left in files (e.g., "TODO", "FIXME")
- [ ] All imports resolve correctly
- [ ] No hardcoded credentials or secrets in files
- [ ] All file paths use correct separators for OS
---
## Quality Checks
### Code Quality
- [ ] Generated code follows project coding standards
- [ ] TypeScript types are complete and accurate (no `any` unless necessary)
- [ ] No unused imports or variables
- [ ] Consistent code formatting (matches project style)
- [ ] No linting errors in generated files
### Best Practices Compliance
- [ ] Fixture architecture follows pure function → fixture → mergeTests pattern
- [ ] Data factories implement auto-cleanup
- [ ] Network interception occurs before navigation
- [ ] Selectors use data-testid strategy
- [ ] Artifacts only captured on failure
- [ ] Tests follow Given-When-Then structure
- [ ] No hard-coded waits or sleeps
### Knowledge Base Alignment
- [ ] Fixture pattern matches `fixture-architecture.md`
- [ ] Data factories match `data-factories.md`
- [ ] Network handling matches `network-first.md`
- [ ] Config follows `playwright-config.md` or `test-config.md`
- [ ] Test quality matches `test-quality.md`
### Security Checks
- [ ] No credentials in configuration files
- [ ] .env.example contains placeholders, not real values
- [ ] Sensitive test data handled securely
- [ ] API keys and tokens use environment variables
- [ ] No secrets committed to version control
---
## Integration Points
### Status File Integration
- [ ] Framework initialization logged in Quality & Testing Progress section
- [ ] Status file updated with completion timestamp
- [ ] Status file shows framework: Playwright or Cypress
### Knowledge Base Integration
- [ ] Relevant knowledge fragments identified from tea-index.csv
- [ ] Knowledge fragments successfully loaded
- [ ] Patterns from knowledge base applied correctly
- [ ] Knowledge base references included in documentation
### Workflow Dependencies
- [ ] Can proceed to `ci` workflow after completion
- [ ] Can proceed to `test-design` workflow after completion
- [ ] Can proceed to `atdd` workflow after completion
- [ ] Framework setup compatible with downstream workflows
---
## Completion Criteria
**All of the following must be true:**
- [ ] All prerequisite checks passed
- [ ] All process steps completed without errors
- [ ] All output validations passed
- [ ] All quality checks passed
- [ ] All integration points verified
- [ ] Sample test executes successfully
- [ ] User can run `npm run test:e2e` without errors
- [ ] Documentation is complete and accurate
- [ ] No critical issues or blockers identified
---
## Post-Workflow Actions
**User must complete:**
1. [ ] Copy `.env.example` to `.env`
2. [ ] Fill in environment-specific values in `.env`
3. [ ] Run `npm install` to install test dependencies
4. [ ] Run `npm run test:e2e` to verify setup
5. [ ] Review `tests/README.md` for project-specific guidance
**Recommended next workflows:**
1. [ ] Run `ci` workflow to set up CI/CD pipeline
2. [ ] Run `test-design` workflow to plan test coverage
3. [ ] Run `atdd` workflow when ready to develop stories
---
## Rollback Procedure
If workflow fails and needs to be rolled back:
1. [ ] Delete `tests/` directory
2. [ ] Remove test scripts from package.json
3. [ ] Delete `.env.example` (if created)
4. [ ] Delete `.nvmrc` (if created)
5. [ ] Delete framework config file
6. [ ] Remove test dependencies from package.json (if added)
7. [ ] Run `npm install` to clean up node_modules
---
## Notes
### Common Issues
**Issue**: Config file has TypeScript errors
- **Solution**: Ensure `@playwright/test` or `cypress` types are installed
**Issue**: Sample test fails to run
- **Solution**: Check BASE_URL in .env, ensure app is running
**Issue**: Fixture cleanup not working
- **Solution**: Verify cleanup() is called in fixture teardown
**Issue**: Network interception not working
- **Solution**: Ensure route setup occurs before page.goto()
### Framework-Specific Considerations
**Playwright:**
- Requires Node.js 18+
- Browser binaries auto-installed on first run
- Trace viewer requires running `npx playwright show-trace`
**Cypress:**
- Requires Node.js 18+
- Cypress app opens on first run
- Component testing requires additional setup
### Version Compatibility
- [ ] Node.js version matches .nvmrc
- [ ] Framework version compatible with Node.js version
- [ ] TypeScript version compatible with framework
- [ ] All peer dependencies satisfied
---
**Checklist Complete**: Sign off when all items checked and validated.
**Completed by:** {name}
**Date:** {date}
**Framework:** { Playwright / Cypress or something else}
**Notes:** {notes}
@@ -0,0 +1,45 @@
<!-- Powered by BMAD-CORE™ -->
# Test Framework Setup
**Workflow ID**: `_bmad/tea/testarch/framework`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Initialize a production-ready test framework (Playwright or Cypress) with fixtures, helpers, configuration, and best practices.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order without skipping
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `test_dir`, `use_typescript`, `framework_preference`, `project_size`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/framework/steps-c/step-01-preflight.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/framework/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.
@@ -0,0 +1,100 @@
---
name: 'step-01-preflight'
description: 'Verify prerequisites and gather project context'
nextStepFile: './step-02-select-framework.md'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 1: Preflight Checks
## STEP GOAL
Verify the project is ready for framework scaffolding and gather key context.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if preflight requirements fail
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validate Prerequisites
- `package.json` exists in project root
- No existing E2E framework (`playwright.config.*`, `cypress.config.*`, `cypress.json`)
- Architecture/stack context available (project type, bundler, dependencies)
If any fail, **HALT** and report the missing requirement.
---
## 2. Gather Project Context
- Read `package.json` to identify framework, bundler, dependencies
- Check for architecture docs (`architecture.md`, `tech-spec*.md`) if available
- Note auth requirements and APIs (if documented)
---
## 3. Confirm Findings
Summarize:
- Project type and bundler
- Whether a framework is already installed
- Any relevant context docs found
---
### 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight']
lastStep: 'step-01-preflight'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,116 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing progress document, verifying previously created artifacts still exist on disk, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Verify Previously Created Artifacts
Since this workflow creates code files, verify that artifacts from completed steps still exist on disk:
- If `step-01-preflight` completed: Confirm `package.json` still exists
- If `step-03-scaffold-framework` completed: Confirm directory structure and config files exist
- If `step-04-docs-and-scripts` completed: Confirm `{test_dir}/README.md` exists
If any expected artifacts are missing, warn the user and suggest re-running from the step that created them.
---
### 3. Display Progress Dashboard
Display:
"📋 **Workflow Resume — Test Framework Setup**
**Last saved:** {lastSaved}
**Steps completed:** {stepsCompleted.length} of 5
1. ✅/⬜ Preflight Checks (step-01-preflight)
2. ✅/⬜ Select Framework (step-02-select-framework)
3. ✅/⬜ Scaffold Framework (step-03-scaffold-framework)
4. ✅/⬜ Docs & Scripts (step-04-docs-and-scripts)
5. ✅/⬜ Validate & Summary (step-05-validate-and-summary)"
---
### 4. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight'` → `./step-02-select-framework.md`
- `'step-02-select-framework'` → `./step-03-scaffold-framework.md`
- `'step-03-scaffold-framework'` → `./step-04-docs-and-scripts.md`
- `'step-04-docs-and-scripts'` → `./step-05-validate-and-summary.md`
- `'step-05-validate-and-summary'` → **Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Previously created artifacts verified on disk
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Not verifying existing artifacts
- Incorrect progress display
- Routing to wrong step
- Re-executing completed steps
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
@@ -0,0 +1,98 @@
---
name: 'step-02-select-framework'
description: 'Select Playwright or Cypress and justify choice'
nextStepFile: './step-03-scaffold-framework.md'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 2: Framework Selection
## STEP GOAL
Choose the most appropriate framework and document the rationale.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Selection Logic
Default to **Playwright** unless strong reasons suggest Cypress.
**Playwright recommended when:**
- Large or complex repo
- Multi-browser support needed
- Heavy API + UI integration
- CI speed/parallelism is important
**Cypress recommended when:**
- Small team prioritizes DX
- Component testing focus
- Simpler setup needed
Respect `framework_preference` if explicitly set.
---
## 2. Announce Decision
State the selected framework and reasoning.
---
### 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-select-framework']
lastStep: 'step-02-select-framework'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-select-framework'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-select-framework'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,145 @@
---
name: 'step-03-scaffold-framework'
description: 'Create directory structure, config, fixtures, factories, and sample tests'
nextStepFile: './step-04-docs-and-scripts.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 3: Scaffold Framework
## STEP GOAL
Generate the test directory structure, configuration files, fixtures, factories, helpers, and sample tests.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Apply knowledge base patterns where required
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Create Directory Structure
Create:
- `{test_dir}/e2e/`
- `{test_dir}/support/fixtures/`
- `{test_dir}/support/helpers/`
- `{test_dir}/support/page-objects/` (optional)
---
## 2. Generate Framework Config
Create `playwright.config.ts` or `cypress.config.ts` with:
- **Timeouts**: action 15s, navigation 30s, test 60s
- **Base URL**: env fallback (`BASE_URL`)
- **Artifacts**: retain-on-failure (trace/screenshot/video)
- **Reporters**: HTML + JUnit + console
- **Parallelism**: enabled (CI tuned)
Use TypeScript if `use_typescript: true`.
---
## 3. Environment & Node
Create:
- `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`
- `.nvmrc` using current LTS Node (prefer Node 24+)
---
## 4. Fixtures & Factories
Read `{config_source}` and use `{knowledgeIndex}` to load fragments based on `config.tea_use_playwright_utils`:
**If Playwright Utils enabled:**
- `overview.md`, `fixtures-composition.md`, `auth-session.md`, `api-request.md`, `burn-in.md`, `network-error-monitor.md`, `data-factories.md`
- Recommend installing `@seontechnologies/playwright-utils`
**If disabled:**
- `fixture-architecture.md`, `data-factories.md`, `network-first.md`, `playwright-config.md`, `test-quality.md`
Implement:
- Fixture index with `mergeTests`
- Auto-cleanup hooks
- Faker-based data factories with overrides
---
## 5. Sample Tests & Helpers
Create example tests in `{test_dir}/e2e/` demonstrating:
- Given/When/Then format
- data-testid selector strategy
- Factory usage
- Network interception pattern (if applicable)
Create helpers for:
- API clients (if needed)
- Network utilities
- Auth helpers
---
### 6. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-scaffold-framework']
lastStep: 'step-03-scaffold-framework'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-scaffold-framework'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-scaffold-framework'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,93 @@
---
name: 'step-04-docs-and-scripts'
description: 'Document setup and add package.json scripts'
nextStepFile: './step-05-validate-and-summary.md'
outputFile: '{test_dir}/README.md'
progressFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 4: Documentation & Scripts
## STEP GOAL
Create `tests/README.md` and update `package.json` scripts.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. tests/README.md
Create `{outputFile}` and include:
- Setup instructions
- Running tests (local/headed/debug)
- Architecture overview (fixtures, factories, helpers)
- Best practices (selectors, isolation, cleanup)
- CI integration notes
- Knowledge base references
---
## 2. package.json Scripts
Add at minimum:
- `test:e2e`: framework execution command
---
### 3. Save Progress
**Save this step's accumulated work to `{progressFile}`.**
- **If `{progressFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04-docs-and-scripts']
lastStep: 'step-04-docs-and-scripts'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{progressFile}` already exists**, update:
- Add `'step-04-docs-and-scripts'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04-docs-and-scripts'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,93 @@
---
name: 'step-05-validate-and-summary'
description: 'Validate against checklist and summarize'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 5: Validate & Summarize
## STEP GOAL
Validate framework setup and provide a completion summary.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validation
Validate against `checklist.md`:
- Preflight success
- Directory structure created
- Config correctness
- Fixtures/factories created
- Docs and scripts present
Fix any gaps before completion.
---
## 2. Completion Summary
Report:
- Framework selected
- Artifacts created
- Next steps (install deps, run tests)
- Knowledge fragments applied
---
### 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-05-validate-and-summary']
lastStep: 'step-05-validate-and-summary'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-05-validate-and-summary'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-05-validate-and-summary'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target
@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update
@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/framework-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced
@@ -0,0 +1,73 @@
---
validationDate: 2026-01-27
workflowName: testarch-framework
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/framework
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-framework
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight.md: 69 lines [GOOD]
- steps-c/step-02-select-framework.md: 66 lines [GOOD]
- steps-c/step-03-scaffold-framework.md: 107 lines [GOOD]
- steps-c/step-04-docs-and-scripts.md: 63 lines [GOOD]
- steps-c/step-05-validate-and-summary.md: 61 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- No templates found in workflow root
- Steps with outputFile in frontmatter:
- steps-c/step-04-docs-and-scripts.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,116 @@
---
validationDate: 2026-01-27
workflowName: testarch-framework
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/framework
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-framework
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight.md: 68 lines [GOOD]
- steps-c/step-02-select-framework.md: 65 lines [GOOD]
- steps-c/step-03-scaffold-framework.md: 106 lines [GOOD]
- steps-c/step-04-docs-and-scripts.md: 62 lines [GOOD]
- steps-c/step-05-validate-and-summary.md: 60 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight.md: Init [PASS]
- steps-c/step-02-select-framework.md: Middle [PASS]
- steps-c/step-03-scaffold-framework.md: Middle [PASS]
- steps-c/step-04-docs-and-scripts.md: Middle [PASS]
- steps-c/step-05-validate-and-summary.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: NONE
- Steps with outputFile in frontmatter:
- steps-c/step-04-docs-and-scripts.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subprocess Optimization Opportunities
- No high-priority subprocess optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 8 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,22 @@
# Workflow Plan: testarch-framework
## Create Mode (steps-c)
- step-01-preflight.md
- step-02-select-framework.md
- step-03-scaffold-framework.md
- step-04-docs-and-scripts.md
- step-05-validate-and-summary.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- Framework config (playwright.config.ts or cypress.config.ts)
- {project-root}/tests/README.md
- Test support scaffolding under {project-root}/tests
@@ -0,0 +1,41 @@
---
name: testarch-framework
description: 'Initialize production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, and configuration'
web_bundle: true
---
# Test Framework Setup
**Goal:** Initialize production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, and configuration
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`
@@ -0,0 +1,49 @@
# Test Architect workflow: framework
name: testarch-framework
description: "Initialize production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, and configuration"
author: "BMad"
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/framework"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
# Variables and inputs
variables:
test_dir: "{project-root}/tests" # Root test directory
use_typescript: true # Prefer TypeScript configuration
framework_preference: "auto" # auto, playwright, cypress - user can override auto-detection
project_size: "auto" # auto, small, large - influences framework recommendation
# Output configuration
default_output_file: "{test_dir}/README.md" # Main deliverable is test setup README
# Required tools
required_tools:
- read_file # Read package.json, existing configs
- write_file # Create config files, helpers, fixtures, tests
- create_directory # Create test directory structure
- list_files # Check for existing framework
- search_repo # Find architecture docs
tags:
- qa
- setup
- test-architect
- framework
- initialization
execution_hints:
interactive: false # Minimize prompts; auto-detect when possible
autonomous: true # Proceed without user input unless blocked
iterative: true
␍
@@ -0,0 +1,407 @@
# Non-Functional Requirements Assessment - Validation Checklist
**Workflow:** `testarch-nfr`
**Purpose:** Ensure comprehensive and evidence-based NFR assessment with actionable recommendations
---
Note: `nfr-assess` evaluates existing evidence; it does not run tests or CI workflows.
## Prerequisites Validation
- [ ] Implementation is deployed and accessible for evaluation
- [ ] Evidence sources are available (test results, metrics, logs, CI results)
- [ ] NFR categories are determined (performance, security, reliability, maintainability, custom)
- [ ] Evidence directories exist and are accessible (`test_results_dir`, `metrics_dir`, `logs_dir`)
- [ ] Knowledge base is loaded (nfr-criteria, ci-burn-in, test-quality)
---
## Context Loading
- [ ] Tech-spec.md loaded successfully (if available)
- [ ] PRD.md loaded (if available)
- [ ] Story file loaded (if applicable)
- [ ] Relevant knowledge fragments loaded from `tea-index.csv`:
- [ ] `nfr-criteria.md`
- [ ] `ci-burn-in.md`
- [ ] `test-quality.md`
- [ ] `playwright-config.md` (if using Playwright)
---
## NFR Categories and Thresholds
### Performance
- [ ] Response time threshold defined or marked as UNKNOWN
- [ ] Throughput threshold defined or marked as UNKNOWN
- [ ] Resource usage thresholds defined or marked as UNKNOWN
- [ ] Scalability requirements defined or marked as UNKNOWN
### Security
- [ ] Authentication requirements defined or marked as UNKNOWN
- [ ] Authorization requirements defined or marked as UNKNOWN
- [ ] Data protection requirements defined or marked as UNKNOWN
- [ ] Vulnerability management thresholds defined or marked as UNKNOWN
- [ ] Compliance requirements identified (GDPR, HIPAA, PCI-DSS, etc.)
### Reliability
- [ ] Availability (uptime) threshold defined or marked as UNKNOWN
- [ ] Error rate threshold defined or marked as UNKNOWN
- [ ] MTTR (Mean Time To Recovery) threshold defined or marked as UNKNOWN
- [ ] Fault tolerance requirements defined or marked as UNKNOWN
- [ ] Disaster recovery requirements defined (RTO, RPO) or marked as UNKNOWN
### Maintainability
- [ ] Test coverage threshold defined or marked as UNKNOWN
- [ ] Code quality threshold defined or marked as UNKNOWN
- [ ] Technical debt threshold defined or marked as UNKNOWN
- [ ] Documentation completeness threshold defined or marked as UNKNOWN
### Custom NFR Categories (if applicable)
- [ ] Custom NFR category 1: Thresholds defined or marked as UNKNOWN
- [ ] Custom NFR category 2: Thresholds defined or marked as UNKNOWN
- [ ] Custom NFR category 3: Thresholds defined or marked as UNKNOWN
---
## Evidence Gathering
### Performance Evidence
- [ ] Load test results collected (JMeter, k6, Gatling, etc.)
- [ ] Application metrics collected (response times, throughput, resource usage)
- [ ] APM data collected (New Relic, Datadog, Dynatrace, etc.)
- [ ] Lighthouse reports collected (if web app)
- [ ] Playwright performance traces collected (if applicable)
### Security Evidence
- [ ] SAST results collected (SonarQube, Checkmarx, Veracode, etc.)
- [ ] DAST results collected (OWASP ZAP, Burp Suite, etc.)
- [ ] Dependency scanning results collected (Snyk, Dependabot, npm audit)
- [ ] Penetration test reports collected (if available)
- [ ] Security audit logs collected
- [ ] Compliance audit results collected (if applicable)
### Reliability Evidence
- [ ] Uptime monitoring data collected (Pingdom, UptimeRobot, StatusCake)
- [ ] Error logs collected
- [ ] Error rate metrics collected
- [ ] CI burn-in results collected (stability over time)
- [ ] Chaos engineering test results collected (if available)
- [ ] Failover/recovery test results collected (if available)
- [ ] Incident reports and postmortems collected (if applicable)
### Maintainability Evidence
- [ ] Code coverage reports collected (Istanbul, NYC, c8, JaCoCo)
- [ ] Static analysis results collected (ESLint, SonarQube, CodeClimate)
- [ ] Technical debt metrics collected
- [ ] Documentation audit results collected
- [ ] Test review report collected (from test-review workflow, if available)
- [ ] Git metrics collected (code churn, commit frequency, etc.)
---
## NFR Assessment with Deterministic Rules
### Performance Assessment
- [ ] Response time assessed against threshold
- [ ] Throughput assessed against threshold
- [ ] Resource usage assessed against threshold
- [ ] Scalability assessed against requirements
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, metric name)
### Security Assessment
- [ ] Authentication strength assessed against requirements
- [ ] Authorization controls assessed against requirements
- [ ] Data protection assessed against requirements
- [ ] Vulnerability management assessed against thresholds
- [ ] Compliance assessed against requirements
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, scan result)
### Reliability Assessment
- [ ] Availability (uptime) assessed against threshold
- [ ] Error rate assessed against threshold
- [ ] MTTR assessed against threshold
- [ ] Fault tolerance assessed against requirements
- [ ] Disaster recovery assessed against requirements (RTO, RPO)
- [ ] CI burn-in assessed (stability over time)
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, monitoring data)
### Maintainability Assessment
- [ ] Test coverage assessed against threshold
- [ ] Code quality assessed against threshold
- [ ] Technical debt assessed against threshold
- [ ] Documentation completeness assessed against threshold
- [ ] Test quality assessed (from test-review, if available)
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, coverage report)
### Custom NFR Assessment (if applicable)
- [ ] Custom NFR 1 assessed against threshold with justification
- [ ] Custom NFR 2 assessed against threshold with justification
- [ ] Custom NFR 3 assessed against threshold with justification
---
## Status Classification Validation
### PASS Criteria Verified
- [ ] Evidence exists for PASS status
- [ ] Evidence meets or exceeds threshold
- [ ] No concerns flagged in evidence
- [ ] Quality is acceptable
### CONCERNS Criteria Verified
- [ ] Threshold is UNKNOWN (documented) OR
- [ ] Evidence is MISSING or INCOMPLETE (documented) OR
- [ ] Evidence is close to threshold (within 10%, documented) OR
- [ ] Evidence shows intermittent issues (documented)
### FAIL Criteria Verified
- [ ] Evidence exists BUT does not meet threshold (documented) OR
- [ ] Critical evidence is MISSING (documented) OR
- [ ] Evidence shows consistent failures (documented) OR
- [ ] Quality is unacceptable (documented)
### No Threshold Guessing
- [ ] All thresholds are either defined or marked as UNKNOWN
- [ ] No thresholds were guessed or inferred
- [ ] All UNKNOWN thresholds result in CONCERNS status
---
## Quick Wins and Recommended Actions
### Quick Wins Identified
- [ ] Low-effort, high-impact improvements identified for CONCERNS/FAIL
- [ ] Configuration changes (no code changes) identified
- [ ] Optimization opportunities identified (caching, indexing, compression)
- [ ] Monitoring additions identified (detect issues before failures)
### Recommended Actions
- [ ] Specific remediation steps provided (not generic advice)
- [ ] Priority assigned (CRITICAL, HIGH, MEDIUM, LOW)
- [ ] Estimated effort provided (hours, days)
- [ ] Owner suggestions provided (dev, ops, security)
### Monitoring Hooks
- [ ] Performance monitoring suggested (APM, synthetic monitoring)
- [ ] Error tracking suggested (Sentry, Rollbar, error logs)
- [ ] Security monitoring suggested (intrusion detection, audit logs)
- [ ] Alerting thresholds suggested (notify before breach)
### Fail-Fast Mechanisms
- [ ] Circuit breakers suggested for reliability
- [ ] Rate limiting suggested for performance
- [ ] Validation gates suggested for security
- [ ] Smoke tests suggested for maintainability
---
## Deliverables Generated
### NFR Assessment Report
- [ ] File created at `{test_artifacts}/nfr-assessment.md`
- [ ] Template from `nfr-report-template.md` used
- [ ] Executive summary included (overall status, critical issues)
- [ ] Assessment by category included (performance, security, reliability, maintainability)
- [ ] Evidence for each NFR documented
- [ ] Status classifications documented (PASS/CONCERNS/FAIL)
- [ ] Findings summary included (PASS count, CONCERNS count, FAIL count)
- [ ] Quick wins section included
- [ ] Recommended actions section included
- [ ] Evidence gaps checklist included
### Gate YAML Snippet (if enabled)
- [ ] YAML snippet generated
- [ ] Date included
- [ ] Categories status included (performance, security, reliability, maintainability)
- [ ] Overall status included (PASS/CONCERNS/FAIL)
- [ ] Issue counts included (critical, high, medium, concerns)
- [ ] Blockers flag included (true/false)
- [ ] Recommendations included
### Evidence Checklist (if enabled)
- [ ] All NFRs with MISSING or INCOMPLETE evidence listed
- [ ] Owners assigned for evidence collection
- [ ] Suggested evidence sources provided
- [ ] Deadlines set for evidence collection
### Updated Story File (if enabled and requested)
- [ ] "NFR Assessment" section added to story markdown
- [ ] Link to NFR assessment report included
- [ ] Overall status and critical issues included
- [ ] Gate status included
---
## Quality Assurance
### Accuracy Checks
- [ ] All NFR categories assessed (none skipped)
- [ ] All thresholds documented (defined or UNKNOWN)
- [ ] All evidence sources documented (file paths, metric names)
- [ ] Status classifications are deterministic and consistent
- [ ] No false positives (status correctly assigned)
- [ ] No false negatives (all issues identified)
### Completeness Checks
- [ ] All NFR categories covered (performance, security, reliability, maintainability, custom)
- [ ] All evidence sources checked (test results, metrics, logs, CI results)
- [ ] All status types used appropriately (PASS, CONCERNS, FAIL)
- [ ] All NFRs with CONCERNS/FAIL have recommendations
- [ ] All evidence gaps have owners and deadlines
### Actionability Checks
- [ ] Recommendations are specific (not generic)
- [ ] Remediation steps are clear and actionable
- [ ] Priorities are assigned (CRITICAL, HIGH, MEDIUM, LOW)
- [ ] Effort estimates are provided (hours, days)
- [ ] Owners are suggested (dev, ops, security)
---
## Integration with BMad Artifacts
### With tech-spec.md
- [ ] Tech spec loaded for NFR requirements and thresholds
- [ ] Performance targets extracted
- [ ] Security requirements extracted
- [ ] Reliability SLAs extracted
- [ ] Architectural decisions considered
### With test-design.md
- [ ] Test design loaded for NFR test plan
- [ ] Test priorities referenced (P0/P1/P2/P3)
- [ ] Assessment aligned with planned NFR validation
### With PRD.md
- [ ] PRD loaded for product-level NFR context
- [ ] User experience goals considered
- [ ] Unstated requirements checked
- [ ] Product-level SLAs referenced
---
## Quality Gates Validation
### Release Blocker (FAIL)
- [ ] Critical NFR status checked (security, reliability)
- [ ] Performance failures assessed for user impact
- [ ] Release blocker flagged if critical NFR has FAIL status
### PR Blocker (HIGH CONCERNS)
- [ ] High-priority NFR status checked
- [ ] Multiple CONCERNS assessed
- [ ] PR blocker flagged if HIGH priority issues exist
### Warning (CONCERNS)
- [ ] Any NFR with CONCERNS status flagged
- [ ] Missing or incomplete evidence documented
- [ ] Warning issued to address before next release
### Pass (PASS)
- [ ] All NFRs have PASS status
- [ ] No blockers or concerns exist
- [ ] Ready for release confirmed
---
## Non-Prescriptive Validation
- [ ] NFR categories adapted to team needs
- [ ] Thresholds appropriate for project context
- [ ] Assessment criteria customized as needed
- [ ] Teams can extend with custom NFR categories
- [ ] Integration with external tools supported (New Relic, Datadog, SonarQube, JIRA)
---
## Documentation and Communication
- [ ] NFR assessment report is readable and well-formatted
- [ ] Tables render correctly in markdown
- [ ] Code blocks have proper syntax highlighting
- [ ] Links are valid and accessible
- [ ] Recommendations are clear and prioritized
- [ ] Overall status is prominent and unambiguous
- [ ] Executive summary provides quick understanding
---
## Final Validation
- [ ] All prerequisites met
- [ ] All NFR categories assessed with evidence (or gaps documented)
- [ ] No thresholds were guessed (all defined or UNKNOWN)
- [ ] Status classifications are deterministic and justified
- [ ] Quick wins identified for all CONCERNS/FAIL
- [ ] Recommended actions are specific and actionable
- [ ] Evidence gaps documented with owners and deadlines
- [ ] NFR assessment report generated and saved
- [ ] Gate YAML snippet generated (if enabled)
- [ ] Evidence checklist generated (if enabled)
- [ ] Workflow completed successfully
---
## Sign-Off
**NFR Assessment Status:**
- [ ] ✅ PASS - All NFRs meet requirements, ready for release
- [ ] ⚠️ CONCERNS - Some NFRs have concerns, address before next release
- [ ] ❌ FAIL - Critical NFRs not met, BLOCKER for release
**Next Actions:**
- If PASS ✅: Proceed to `*gate` workflow or release
- If CONCERNS ⚠️: Address HIGH/CRITICAL issues, re-run `*nfr-assess`
- If FAIL ❌: Resolve FAIL status NFRs, re-run `*nfr-assess`
**Critical Issues:** {COUNT}
**High Priority Issues:** {COUNT}
**Concerns:** {COUNT}
---
<!-- Powered by BMAD-CORE™ -->
@@ -0,0 +1,43 @@
# Non-Functional Requirements Assessment
**Workflow:** `testarch-nfr`
**Version:** 5.0 (Step-File Architecture)
---
## Overview
Assess non-functional requirements (performance, security, reliability, maintainability) with evidence-based validation and deterministic PASS/CONCERNS/FAIL outcomes.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `custom_nfr_categories`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/nfr-assess/steps-c/step-01-load-context.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/nfr-assess/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.
@@ -0,0 +1,468 @@
---
stepsCompleted: []
lastStep: ''
lastSaved: ''
---
# NFR Assessment - {FEATURE_NAME}
**Date:** {DATE}
**Story:** {STORY_ID} (if applicable)
**Overall Status:** {OVERALL_STATUS} {STATUS_ICON}
---
Note: This assessment summarizes existing evidence; it does not run tests or CI workflows.
## Executive Summary
**Assessment:** {PASS_COUNT} PASS, {CONCERNS_COUNT} CONCERNS, {FAIL_COUNT} FAIL
**Blockers:** {BLOCKER_COUNT} {BLOCKER_DESCRIPTION}
**High Priority Issues:** {HIGH_PRIORITY_COUNT} {HIGH_PRIORITY_DESCRIPTION}
**Recommendation:** {OVERALL_RECOMMENDATION}
---
## Performance Assessment
### Response Time (p95)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Throughput
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Resource Usage
- **CPU Usage**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **Memory Usage**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
### Scalability
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Security Assessment
### Authentication Strength
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
- **Recommendation:** {RECOMMENDATION} (if CONCERNS or FAIL)
### Authorization Controls
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Data Protection
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Vulnerability Management
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION} (e.g., "0 critical, <3 high vulnerabilities")
- **Actual:** {ACTUAL_DESCRIPTION} (e.g., "0 critical, 1 high, 5 medium vulnerabilities")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Snyk scan results - scan-2025-10-14.json")
- **Findings:** {FINDINGS_DESCRIPTION}
### Compliance (if applicable)
- **Status:** {STATUS} {STATUS_ICON}
- **Standards:** {COMPLIANCE_STANDARDS} (e.g., "GDPR, HIPAA, PCI-DSS")
- **Actual:** {ACTUAL_COMPLIANCE_STATUS}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Reliability Assessment
### Availability (Uptime)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "99.9%")
- **Actual:** {ACTUAL_VALUE} (e.g., "99.95%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Uptime monitoring - uptime-report-2025-10-14.csv")
- **Findings:** {FINDINGS_DESCRIPTION}
### Error Rate
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "<0.1%")
- **Actual:** {ACTUAL_VALUE} (e.g., "0.05%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Error logs - logs/errors-2025-10.log")
- **Findings:** {FINDINGS_DESCRIPTION}
### MTTR (Mean Time To Recovery)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "<15 minutes")
- **Actual:** {ACTUAL_VALUE} (e.g., "12 minutes")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Incident reports - incidents/")
- **Findings:** {FINDINGS_DESCRIPTION}
### Fault Tolerance
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### CI Burn-In (Stability)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "100 consecutive successful runs")
- **Actual:** {ACTUAL_VALUE} (e.g., "150 consecutive successful runs")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "CI burn-in results - ci-burn-in-2025-10-14.log")
- **Findings:** {FINDINGS_DESCRIPTION}
### Disaster Recovery (if applicable)
- **RTO (Recovery Time Objective)**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **RPO (Recovery Point Objective)**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
---
## Maintainability Assessment
### Test Coverage
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., ">=80%")
- **Actual:** {ACTUAL_VALUE} (e.g., "87%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Coverage report - coverage/lcov-report/index.html")
- **Findings:** {FINDINGS_DESCRIPTION}
### Code Quality
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., ">=85/100")
- **Actual:** {ACTUAL_VALUE} (e.g., "92/100")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "SonarQube analysis - sonarqube-report-2025-10-14.pdf")
- **Findings:** {FINDINGS_DESCRIPTION}
### Technical Debt
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "<5% debt ratio")
- **Actual:** {ACTUAL_VALUE} (e.g., "3.2% debt ratio")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "CodeClimate analysis - codeclimate-2025-10-14.json")
- **Findings:** {FINDINGS_DESCRIPTION}
### Documentation Completeness
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., ">=90%")
- **Actual:** {ACTUAL_VALUE} (e.g., "95%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Documentation audit - docs-audit-2025-10-14.md")
- **Findings:** {FINDINGS_DESCRIPTION}
### Test Quality (from test-review, if available)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Test review report - test-review-2025-10-14.md")
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Custom NFR Assessments (if applicable)
### {CUSTOM_NFR_NAME_1}
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### {CUSTOM_NFR_NAME_2}
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Quick Wins
{QUICK_WIN_COUNT} quick wins identified for immediate implementation:
1. **{QUICK_WIN_TITLE_1}** ({NFR_CATEGORY}) - {PRIORITY} - {ESTIMATED_EFFORT}
- {QUICK_WIN_DESCRIPTION}
- No code changes needed / Minimal code changes
2. **{QUICK_WIN_TITLE_2}** ({NFR_CATEGORY}) - {PRIORITY} - {ESTIMATED_EFFORT}
- {QUICK_WIN_DESCRIPTION}
---
## Recommended Actions
### Immediate (Before Release) - CRITICAL/HIGH Priority
1. **{ACTION_TITLE_1}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
- {SPECIFIC_STEPS}
- {VALIDATION_CRITERIA}
2. **{ACTION_TITLE_2}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
- {SPECIFIC_STEPS}
- {VALIDATION_CRITERIA}
### Short-term (Next Sprint) - MEDIUM Priority
1. **{ACTION_TITLE_3}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
2. **{ACTION_TITLE_4}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
### Long-term (Backlog) - LOW Priority
1. **{ACTION_TITLE_5}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
---
## Monitoring Hooks
{MONITORING_HOOK_COUNT} monitoring hooks recommended to detect issues before failures:
### Performance Monitoring
- [ ] {MONITORING_TOOL_1} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
- [ ] {MONITORING_TOOL_2} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
### Security Monitoring
- [ ] {MONITORING_TOOL_3} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
### Reliability Monitoring
- [ ] {MONITORING_TOOL_4} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
### Alerting Thresholds
- [ ] {ALERT_DESCRIPTION} - Notify when {THRESHOLD_CONDITION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
---
## Fail-Fast Mechanisms
{FAIL_FAST_COUNT} fail-fast mechanisms recommended to prevent failures:
### Circuit Breakers (Reliability)
- [ ] {CIRCUIT_BREAKER_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
### Rate Limiting (Performance)
- [ ] {RATE_LIMITING_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
### Validation Gates (Security)
- [ ] {VALIDATION_GATE_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
### Smoke Tests (Maintainability)
- [ ] {SMOKE_TEST_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
---
## Evidence Gaps
{EVIDENCE_GAP_COUNT} evidence gaps identified - action required:
- [ ] **{NFR_NAME_1}** ({NFR_CATEGORY})
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
- **Suggested Evidence:** {SUGGESTED_EVIDENCE_SOURCE}
- **Impact:** {IMPACT_DESCRIPTION}
- [ ] **{NFR_NAME_2}** ({NFR_CATEGORY})
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
- **Suggested Evidence:** {SUGGESTED_EVIDENCE_SOURCE}
- **Impact:** {IMPACT_DESCRIPTION}
---
## Findings Summary
**Based on ADR Quality Readiness Checklist (8 categories, 29 criteria)**
| Category | Criteria Met | PASS | CONCERNS | FAIL | Overall Status |
| ------------------------------------------------ | ------------------ | ---------------- | -------------------- | ---------------- | ----------------------------------- |
| 1. Testability & Automation | {T_MET}/4 | {T_PASS} | {T_CONCERNS} | {T_FAIL} | {T_STATUS} {T_ICON} |
| 2. Test Data Strategy | {TD_MET}/3 | {TD_PASS} | {TD_CONCERNS} | {TD_FAIL} | {TD_STATUS} {TD_ICON} |
| 3. Scalability & Availability | {SA_MET}/4 | {SA_PASS} | {SA_CONCERNS} | {SA_FAIL} | {SA_STATUS} {SA_ICON} |
| 4. Disaster Recovery | {DR_MET}/3 | {DR_PASS} | {DR_CONCERNS} | {DR_FAIL} | {DR_STATUS} {DR_ICON} |
| 5. Security | {SEC_MET}/4 | {SEC_PASS} | {SEC_CONCERNS} | {SEC_FAIL} | {SEC_STATUS} {SEC_ICON} |
| 6. Monitorability, Debuggability & Manageability | {MON_MET}/4 | {MON_PASS} | {MON_CONCERNS} | {MON_FAIL} | {MON_STATUS} {MON_ICON} |
| 7. QoS & QoE | {QOS_MET}/4 | {QOS_PASS} | {QOS_CONCERNS} | {QOS_FAIL} | {QOS_STATUS} {QOS_ICON} |
| 8. Deployability | {DEP_MET}/3 | {DEP_PASS} | {DEP_CONCERNS} | {DEP_FAIL} | {DEP_STATUS} {DEP_ICON} |
| **Total** | **{TOTAL_MET}/29** | **{TOTAL_PASS}** | **{TOTAL_CONCERNS}** | **{TOTAL_FAIL}** | **{OVERALL_STATUS} {OVERALL_ICON}** |
**Criteria Met Scoring:**
- ≥26/29 (90%+) = Strong foundation
- 20-25/29 (69-86%) = Room for improvement
- <20/29 (<69%) = Significant gaps
---
## Gate YAML Snippet
```yaml
nfr_assessment:
date: '{DATE}'
story_id: '{STORY_ID}'
feature_name: '{FEATURE_NAME}'
adr_checklist_score: '{TOTAL_MET}/29' # ADR Quality Readiness Checklist
categories:
testability_automation: '{T_STATUS}'
test_data_strategy: '{TD_STATUS}'
scalability_availability: '{SA_STATUS}'
disaster_recovery: '{DR_STATUS}'
security: '{SEC_STATUS}'
monitorability: '{MON_STATUS}'
qos_qoe: '{QOS_STATUS}'
deployability: '{DEP_STATUS}'
overall_status: '{OVERALL_STATUS}'
critical_issues: { CRITICAL_COUNT }
high_priority_issues: { HIGH_COUNT }
medium_priority_issues: { MEDIUM_COUNT }
concerns: { CONCERNS_COUNT }
blockers: { BLOCKER_BOOLEAN } # true/false
quick_wins: { QUICK_WIN_COUNT }
evidence_gaps: { EVIDENCE_GAP_COUNT }
recommendations:
- '{RECOMMENDATION_1}'
- '{RECOMMENDATION_2}'
- '{RECOMMENDATION_3}'
```
---
## Related Artifacts
- **Story File:** {STORY_FILE_PATH} (if applicable)
- **Tech Spec:** {TECH_SPEC_PATH} (if available)
- **PRD:** {PRD_PATH} (if available)
- **Test Design:** {TEST_DESIGN_PATH} (if available)
- **Evidence Sources:**
- Test Results: {TEST_RESULTS_DIR}
- Metrics: {METRICS_DIR}
- Logs: {LOGS_DIR}
- CI Results: {CI_RESULTS_PATH}
---
## Recommendations Summary
**Release Blocker:** {RELEASE_BLOCKER_SUMMARY}
**High Priority:** {HIGH_PRIORITY_SUMMARY}
**Medium Priority:** {MEDIUM_PRIORITY_SUMMARY}
**Next Steps:** {NEXT_STEPS_DESCRIPTION}
---
## Sign-Off
**NFR Assessment:**
- Overall Status: {OVERALL_STATUS} {OVERALL_ICON}
- Critical Issues: {CRITICAL_COUNT}
- High Priority Issues: {HIGH_COUNT}
- Concerns: {CONCERNS_COUNT}
- Evidence Gaps: {EVIDENCE_GAP_COUNT}
**Gate Status:** {GATE_STATUS} {GATE_ICON}
**Next Actions:**
- If PASS ✅: Proceed to `*gate` workflow or release
- If CONCERNS ⚠️: Address HIGH/CRITICAL issues, re-run `*nfr-assess`
- If FAIL ❌: Resolve FAIL status NFRs, re-run `*nfr-assess`
**Generated:** {DATE}
**Workflow:** testarch-nfr v4.0
---
<!-- Powered by BMAD-CORE™ -->
@@ -0,0 +1,126 @@
---
name: 'step-01-load-context'
description: 'Load NFR requirements, evidence sources, and knowledge base'
nextStepFile: './step-02-define-thresholds.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 1: Load Context & Knowledge Base
## STEP GOAL
Gather NFR requirements, evidence sources, and knowledge fragments needed for assessment.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if implementation or evidence is unavailable
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Prerequisites
- Implementation accessible for evaluation
- Evidence sources available (test results, metrics, logs)
If missing: **HALT** and request the missing inputs.
---
## 2. Load Configuration
From `{config_source}`:
- Read `tea_browser_automation`
---
## 3. Load Knowledge Base Fragments
From `{knowledgeIndex}` load:
- `adr-quality-readiness-checklist.md`
- `ci-burn-in.md`
- `test-quality.md`
- `playwright-config.md`
- `error-handling.md`
**Playwright CLI (if `tea_browser_automation` is "cli" or "auto"):**
- `playwright-cli.md`
**MCP Patterns (if `tea_browser_automation` is "mcp" or "auto"):**
- (existing MCP-related fragments, if any are added in future)
---
## 4. Load Artifacts
If available, read:
- `tech-spec.md` (primary NFRs)
- `PRD.md` (product-level NFRs)
- `story` or `test-design` docs (feature-level NFRs)
---
## 5. Confirm Inputs
Summarize loaded NFR sources and evidence availability.
---
## 6. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-load-context']
lastStep: 'step-01-load-context'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-load-context'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-load-context'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,106 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- Read the entire step file before acting
- Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- Follow the MANDATORY SEQUENCE exactly
- Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` -- array of completed step names
- `lastStep` -- last completed step name
- `lastSaved` -- timestamp of last save
**If `{outputFile}` does not exist**, display:
"No previous progress found. There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Display Progress Dashboard
Display progress with checkmark/empty indicators:
```
NFR Assessment - Resume Progress:
1. Load Context (step-01-load-context) [completed/pending]
2. Define Thresholds (step-02-define-thresholds) [completed/pending]
3. Gather Evidence (step-03-gather-evidence) [completed/pending]
4. Evaluate & Aggregate (step-04e-aggregate-nfr) [completed/pending]
5. Generate Report (step-05-generate-report) [completed/pending]
Last saved: {lastSaved}
```
---
### 3. Route to Next Step
Based on `lastStep`, load the next incomplete step:
| lastStep | Next Step File |
| --------------------------- | --------------------------------- |
| `step-01-load-context` | `./step-02-define-thresholds.md` |
| `step-02-define-thresholds` | `./step-03-gather-evidence.md` |
| `step-03-gather-evidence` | `./step-04-evaluate-and-score.md` |
| `step-04e-aggregate-nfr` | `./step-05-generate-report.md` |
| `step-05-generate-report` | **Workflow already complete.** |
**If `lastStep` is the final step** (`step-05-generate-report`), display: "All steps completed. Use **[C] Create** to start fresh, **[V] Validate** to review outputs, or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "Unknown progress state (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## SYSTEM SUCCESS/FAILURE METRICS
### SUCCESS:
- Output document loaded and parsed correctly
- Progress dashboard displayed accurately
- Routed to correct next step
### FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
@@ -0,0 +1,107 @@
---
name: 'step-02-define-thresholds'
description: 'Identify NFR categories and thresholds'
nextStepFile: './step-03-gather-evidence.md'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 2: Define NFR Categories & Thresholds
## STEP GOAL
Establish the NFR categories to assess and the thresholds used for validation.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Never guess thresholds
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Select Categories
Use the ADR Quality Readiness Checklist (8 categories):
1. Testability & Automation
2. Test Data Strategy
3. Scalability & Availability
4. Disaster Recovery
5. Security
6. Monitorability/Debuggability/Manageability
7. QoS/QoE
8. Deployability
Add any `custom_nfr_categories` if provided.
---
## 2. Define Thresholds
For each category, extract thresholds from:
- tech-spec (primary)
- PRD (secondary)
- story or test-design (feature-specific)
If a threshold is unknown, mark it **UNKNOWN** and plan to report **CONCERNS**.
---
## 3. Confirm NFR Matrix
List each NFR category with its threshold or UNKNOWN status.
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-define-thresholds']
lastStep: 'step-02-define-thresholds'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-define-thresholds'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-define-thresholds'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,108 @@
---
name: 'step-03-gather-evidence'
description: 'Collect evidence for each NFR category'
nextStepFile: './step-04-evaluate-and-score.md'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 3: Gather Evidence
## STEP GOAL
Collect measurable evidence to evaluate each NFR category.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Evidence Sources
Collect evidence for:
- **Performance**: load tests, metrics, response time data
- **Security**: scans, auth tests, vuln reports
- **Reliability**: error rates, burn-in runs, failover tests
- **Maintainability**: test quality, code health signals
- **Other categories**: logs, monitoring, DR drills, deployability checks
---
## 2. Browser-Based Evidence Collection (if `tea_browser_automation` is `cli` or `auto`)
> **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip browser-based evidence collection.
For performance and security categories, CLI can gather live evidence:
**Performance evidence (page load, response times):**
1. `playwright-cli -s=tea-nfr open <target_url>`
2. `playwright-cli -s=tea-nfr network` → capture response times and payload sizes
3. `playwright-cli -s=tea-nfr screenshot --filename={test_artifacts}/nfr/perf-<page>.png`
4. `playwright-cli -s=tea-nfr close`
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-nfr close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
Store artifacts under `{test_artifacts}/nfr/`
---
## 3. Evidence Gaps
If evidence is missing for a category, mark that category as **CONCERNS**.
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-gather-evidence']
lastStep: 'step-03-gather-evidence'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-gather-evidence'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-gather-evidence'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,140 @@
---
name: 'step-04-evaluate-and-score'
description: 'Orchestrate parallel NFR domain assessments (4 subprocesses)'
nextStepFile: './step-04e-aggregate-nfr.md'
---
# Step 4: Orchestrate Parallel NFR Assessment
## STEP GOAL
Launch 4 parallel subprocesses to assess independent NFR domains simultaneously for maximum performance.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Launch FOUR subprocesses in PARALLEL
- ✅ Wait for ALL subprocesses to complete
- ❌ Do NOT assess NFRs sequentially (use subprocesses)
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Wait for subprocess outputs
- 📖 Load the next step only when instructed
---
## MANDATORY SEQUENCE
### 1. Prepare Subprocess Inputs
**Generate unique timestamp:**
```javascript
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
```
**Prepare context:**
```javascript
const subprocessContext = {
system_context: /* from Step 1 */,
nfr_thresholds: /* from Step 2 */,
evidence_gathered: /* from Step 3 */,
timestamp: timestamp
};
```
---
### 2. Launch 4 Parallel NFR Subprocesses
**Subprocess A: Security Assessment**
- File: `./step-04a-subprocess-security.md`
- Output: `/tmp/tea-nfr-security-${timestamp}.json`
- Status: Running... ⟳
**Subprocess B: Performance Assessment**
- File: `./step-04b-subprocess-performance.md`
- Output: `/tmp/tea-nfr-performance-${timestamp}.json`
- Status: Running... ⟳
**Subprocess C: Reliability Assessment**
- File: `./step-04c-subprocess-reliability.md`
- Output: `/tmp/tea-nfr-reliability-${timestamp}.json`
- Status: Running... ⟳
**Subprocess D: Scalability Assessment**
- File: `./step-04d-subprocess-scalability.md`
- Output: `/tmp/tea-nfr-scalability-${timestamp}.json`
- Status: Running... ⟳
---
### 3. Wait for All Subprocesses
```
⏳ Waiting for 4 NFR subprocesses to complete...
├── Subprocess A (Security): Running... ⟳
├── Subprocess B (Performance): Running... ⟳
├── Subprocess C (Reliability): Running... ⟳
└── Subprocess D (Scalability): Running... ⟳
[... time passes ...]
✅ All 4 NFR subprocesses completed!
```
---
### 4. Performance Report
```
🚀 Performance Report:
- Execution Mode: PARALLEL (4 NFR domains)
- Total Elapsed: ~max(all subprocesses) minutes
- Sequential Would Take: ~sum(all subprocesses) minutes
- Performance Gain: ~67% faster!
```
---
### 5. Proceed to Aggregation
Load next step: `{nextStepFile}`
The aggregation step will:
- Read all 4 NFR domain outputs
- Calculate overall risk level
- Aggregate compliance status
- Identify cross-domain risks
- Generate executive summary
---
## EXIT CONDITION
Proceed when all 4 subprocesses completed and outputs exist.
---
## 🚨 SYSTEM SUCCESS METRICS
### ✅ SUCCESS:
- All 4 NFR subprocesses completed
- Parallel execution achieved ~67% performance gain
### ❌ FAILURE:
- One or more subprocesses failed
- Sequential assessment instead of parallel
@@ -0,0 +1,138 @@
---
name: 'step-04a-subprocess-security'
description: 'Subprocess: Security NFR assessment'
subprocess: true
outputFile: '/tmp/tea-nfr-security-{{timestamp}}.json'
---
# Subprocess 4A: Security NFR Assessment
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with other NFR domain assessments.
**Your task:** Assess SECURITY NFR domain only.
---
## MANDATORY EXECUTION RULES
- ✅ Assess SECURITY only (not performance, reliability, scalability)
- ✅ Output structured JSON to temp file
- ❌ Do NOT assess other NFR domains
---
## SUBPROCESS TASK
### 1. Security Assessment Categories
**Assess the following security dimensions:**
**A) Authentication & Authorization:**
- OAuth2/JWT implementation
- Session management
- Multi-factor authentication
- Role-based access control (RBAC)
**B) Data Protection:**
- Encryption at rest
- Encryption in transit (HTTPS/TLS)
- Sensitive data handling (PII, passwords)
- Database encryption
**C) Input Validation:**
- SQL injection prevention
- XSS prevention
- CSRF protection
- Input sanitization
**D) API Security:**
- Rate limiting
- API authentication
- CORS configuration
- Security headers
**E) Secrets Management:**
- Environment variables for secrets
- No hardcoded credentials
- Secret rotation policies
- Key management systems
### 2. Risk Assessment
For each category, determine status:
- **PASS**: Properly implemented
- **CONCERN**: Partially implemented or weak
- **FAIL**: Not implemented or critical vulnerability
- **N/A**: Not applicable to this system
### 3. Compliance Check
**Common compliance standards:**
- SOC2
- GDPR
- HIPAA
- PCI-DSS
- ISO 27001
---
## OUTPUT FORMAT
```json
{
"domain": "security",
"risk_level": "MEDIUM",
"findings": [
{
"category": "Authentication",
"status": "PASS",
"description": "OAuth2 with JWT tokens implemented",
"evidence": ["src/auth/oauth.ts", "JWT refresh token rotation"],
"recommendations": []
},
{
"category": "Data Encryption",
"status": "CONCERN",
"description": "Database encryption at rest not enabled",
"evidence": ["Database config shows no encryption"],
"recommendations": ["Enable database encryption at rest", "Use AWS RDS encryption or equivalent", "Implement key rotation policy"]
},
{
"category": "Input Validation",
"status": "FAIL",
"description": "SQL injection vulnerability in search endpoint",
"evidence": ["src/api/search.ts:42 - direct SQL concatenation"],
"recommendations": ["URGENT: Use parameterized queries", "Add input sanitization library", "Implement WAF rules"]
}
],
"compliance": {
"SOC2": "PARTIAL",
"GDPR": "PASS",
"HIPAA": "N/A",
"PCI-DSS": "FAIL"
},
"priority_actions": [
"Fix SQL injection vulnerability (URGENT)",
"Enable database encryption within 30 days",
"Implement rate limiting for all APIs"
],
"summary": "Security posture is MEDIUM risk with 1 critical vulnerability requiring immediate attention"
}
```
---
## EXIT CONDITION
Subprocess completes when JSON output written to temp file.
**Subprocess terminates here.**
@@ -0,0 +1,84 @@
---
name: 'step-04b-subprocess-performance'
description: 'Subprocess: Performance NFR assessment'
subprocess: true
outputFile: '/tmp/tea-nfr-performance-{{timestamp}}.json'
---
# Subprocess 4B: Performance NFR Assessment
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with other NFR domain assessments.
**Your task:** Assess PERFORMANCE NFR domain only.
---
## SUBPROCESS TASK
### 1. Performance Assessment Categories
**A) Response Times:**
- API response times (<200ms target)
- Page load times (<2s target)
- Time to interactive (<3s target)
**B) Throughput:**
- Requests per second capacity
- Concurrent user support
- Database query performance
**C) Resource Usage:**
- Memory consumption
- CPU utilization
- Database connection pooling
**D) Optimization:**
- Caching strategies
- CDN usage
- Code splitting/lazy loading
- Database indexing
---
## OUTPUT FORMAT
```json
{
"domain": "performance",
"risk_level": "LOW",
"findings": [
{
"category": "Response Times",
"status": "PASS",
"description": "API endpoints respond in <150ms (P95)",
"evidence": ["Load testing results show 140ms P95"],
"recommendations": []
},
{
"category": "Caching",
"status": "CONCERN",
"description": "No CDN for static assets",
"evidence": ["Static files served from origin"],
"recommendations": ["Implement CDN (CloudFront/Cloudflare)", "Cache static assets for 1 year"]
}
],
"compliance": {
"SLA_99.9": "PASS",
"SLA_99.99": "CONCERN"
},
"priority_actions": ["Implement CDN for static assets", "Add database query caching for frequent reads"],
"summary": "Performance is acceptable with minor optimization opportunities"
}
```
---
## EXIT CONDITION
Subprocess completes when JSON output written to temp file.
@@ -0,0 +1,85 @@
---
name: 'step-04c-subprocess-reliability'
description: 'Subprocess: Reliability NFR assessment'
subprocess: true
outputFile: '/tmp/tea-nfr-reliability-{{timestamp}}.json'
---
# Subprocess 4C: Reliability NFR Assessment
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with other NFR domain assessments.
**Your task:** Assess RELIABILITY NFR domain only.
---
## SUBPROCESS TASK
### 1. Reliability Assessment Categories
**A) Error Handling:**
- Try-catch blocks for critical operations
- Graceful degradation
- Circuit breakers
- Retry mechanisms
**B) Monitoring & Observability:**
- Logging implementation
- Error tracking (Sentry/Datadog)
- Health check endpoints
- Alerting systems
**C) Fault Tolerance:**
- Database failover
- Service redundancy
- Backup strategies
- Disaster recovery plan
**D) Uptime & Availability:**
- SLA targets
- Historical uptime
- Incident response
---
## OUTPUT FORMAT
```json
{
"domain": "reliability",
"risk_level": "LOW",
"findings": [
{
"category": "Error Handling",
"status": "PASS",
"description": "Comprehensive error handling with circuit breakers",
"evidence": ["Circuit breaker pattern in src/services/", "Retry logic implemented"],
"recommendations": []
},
{
"category": "Monitoring",
"status": "CONCERN",
"description": "No APM (Application Performance Monitoring) tool",
"evidence": ["Logging present but no distributed tracing"],
"recommendations": ["Implement APM (Datadog/New Relic)", "Add distributed tracing"]
}
],
"compliance": {
"SLA_99.9": "PASS"
},
"priority_actions": ["Implement APM for better observability"],
"summary": "Reliability is good with minor monitoring gaps"
}
```
---
## EXIT CONDITION
Subprocess completes when JSON output written to temp file.
@@ -0,0 +1,88 @@
---
name: 'step-04d-subprocess-scalability'
description: 'Subprocess: Scalability NFR assessment'
subprocess: true
outputFile: '/tmp/tea-nfr-scalability-{{timestamp}}.json'
---
# Subprocess 4D: Scalability NFR Assessment
## SUBPROCESS CONTEXT
This is an **isolated subprocess** running in parallel with other NFR domain assessments.
**Your task:** Assess SCALABILITY NFR domain only.
---
## SUBPROCESS TASK
### 1. Scalability Assessment Categories
**A) Horizontal Scaling:**
- Stateless architecture
- Load balancer configuration
- Container orchestration (K8s)
- Auto-scaling policies
**B) Vertical Scaling:**
- Resource allocation
- Database size limits
- Memory management
- CPU optimization
**C) Data Scaling:**
- Database partitioning/sharding
- Read replicas
- Caching layers
- Data archival strategy
**D) Traffic Handling:**
- CDN for static assets
- Rate limiting
- Queue systems for async work
- WebSocket scaling
---
## OUTPUT FORMAT
```json
{
"domain": "scalability",
"risk_level": "MEDIUM",
"findings": [
{
"category": "Horizontal Scaling",
"status": "PASS",
"description": "Stateless architecture with container orchestration",
"evidence": ["Docker + Kubernetes setup", "Auto-scaling configured"],
"recommendations": []
},
{
"category": "Data Scaling",
"status": "CONCERN",
"description": "No database sharding strategy for large data growth",
"evidence": ["Single database instance", "No partitioning"],
"recommendations": ["Plan database sharding strategy", "Implement read replicas", "Consider database clustering"]
}
],
"compliance": {
"1M_users": "PASS",
"10M_users": "CONCERN",
"100M_users": "FAIL"
},
"priority_actions": ["Design database sharding strategy for future growth", "Implement read replicas for read-heavy workloads"],
"summary": "Scalability is good up to 1M users, concerns for 10M+ users"
}
```
---
## EXIT CONDITION
Subprocess completes when JSON output written to temp file.
@@ -0,0 +1,247 @@
---
name: 'step-04e-aggregate-nfr'
description: 'Aggregate NFR domain assessments into executive summary'
nextStepFile: './step-05-generate-report.md'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 4E: Aggregate NFR Assessment Results
## STEP GOAL
Read outputs from 4 parallel NFR subprocesses, calculate overall risk level, aggregate compliance status, and identify cross-domain risks.
---
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Read all 4 subprocess outputs
- ✅ Calculate overall risk level
- ❌ Do NOT re-assess NFRs (use subprocess outputs)
---
## MANDATORY SEQUENCE
### 1. Read All Subprocess Outputs
```javascript
const domains = ['security', 'performance', 'reliability', 'scalability'];
const assessments = {};
domains.forEach((domain) => {
const outputPath = `/tmp/tea-nfr-${domain}-{{timestamp}}.json`;
assessments[domain] = JSON.parse(fs.readFileSync(outputPath, 'utf8'));
});
```
---
### 2. Calculate Overall Risk Level
**Risk hierarchy:** HIGH > MEDIUM > LOW > NONE
```javascript
const riskLevels = { HIGH: 3, MEDIUM: 2, LOW: 1, NONE: 0 };
const domainRisks = domains.map((d) => assessments[d].risk_level);
const maxRiskValue = Math.max(...domainRisks.map((r) => riskLevels[r]));
const overallRisk = Object.keys(riskLevels).find((k) => riskLevels[k] === maxRiskValue);
```
**Risk assessment:**
- If ANY domain is HIGH → overall is HIGH
- If ANY domain is MEDIUM (and none HIGH) → overall is MEDIUM
- If ALL domains are LOW/NONE → overall is LOW
---
### 3. Aggregate Compliance Status
```javascript
const allCompliance = {};
domains.forEach((domain) => {
const compliance = assessments[domain].compliance;
Object.entries(compliance).forEach(([standard, status]) => {
if (!allCompliance[standard]) {
allCompliance[standard] = [];
}
allCompliance[standard].push({ domain, status });
});
});
// Determine overall compliance per standard
const complianceSummary = {};
Object.entries(allCompliance).forEach(([standard, statuses]) => {
const hasFail = statuses.some((s) => s.status === 'FAIL');
const hasPartial = statuses.some((s) => s.status === 'PARTIAL' || s.status === 'CONCERN');
complianceSummary[standard] = hasFail ? 'FAIL' : hasPartial ? 'PARTIAL' : 'PASS';
});
```
---
### 4. Identify Cross-Domain Risks
**Look for risks that span multiple domains:**
```javascript
const crossDomainRisks = [];
// Example: Performance + Scalability issue
const perfConcerns = assessments.performance.findings.filter((f) => f.status !== 'PASS');
const scaleConcerns = assessments.scalability.findings.filter((f) => f.status !== 'PASS');
if (perfConcerns.length > 0 && scaleConcerns.length > 0) {
crossDomainRisks.push({
domains: ['performance', 'scalability'],
description: 'Performance issues may worsen under scale',
impact: 'HIGH',
});
}
// Example: Security + Reliability issue
const securityFails = assessments.security.findings.filter((f) => f.status === 'FAIL');
const reliabilityConcerns = assessments.reliability.findings.filter((f) => f.status !== 'PASS');
if (securityFails.length > 0 && reliabilityConcerns.length > 0) {
crossDomainRisks.push({
domains: ['security', 'reliability'],
description: 'Security vulnerabilities may cause reliability incidents',
impact: 'CRITICAL',
});
}
```
---
### 5. Aggregate Priority Actions
```javascript
const allPriorityActions = domains.flatMap((domain) =>
assessments[domain].priority_actions.map((action) => ({
domain,
action,
urgency: assessments[domain].risk_level === 'HIGH' ? 'URGENT' : 'NORMAL',
})),
);
// Sort by urgency
const prioritizedActions = allPriorityActions.sort((a, b) => (a.urgency === 'URGENT' ? -1 : 1));
```
---
### 6. Generate Executive Summary
```javascript
const executiveSummary = {
overall_risk: overallRisk,
assessment_date: new Date().toISOString(),
domain_assessments: assessments,
compliance_summary: complianceSummary,
cross_domain_risks: crossDomainRisks,
priority_actions: prioritizedActions,
risk_breakdown: {
security: assessments.security.risk_level,
performance: assessments.performance.risk_level,
reliability: assessments.reliability.risk_level,
scalability: assessments.scalability.risk_level,
},
subprocess_execution: 'PARALLEL (4 NFR domains)',
performance_gain: '~67% faster than sequential',
};
// Save for Step 5 (report generation)
fs.writeFileSync('/tmp/tea-nfr-summary-{{timestamp}}.json', JSON.stringify(executiveSummary, null, 2), 'utf8');
```
---
### 7. Display Summary to User
```
✅ NFR Assessment Complete (Parallel Execution)
🎯 Overall Risk Level: {overallRisk}
📊 Domain Risk Breakdown:
- Security: {security_risk}
- Performance: {performance_risk}
- Reliability: {reliability_risk}
- Scalability: {scalability_risk}
✅ Compliance Summary:
{list standards with PASS/PARTIAL/FAIL}
⚠️ Cross-Domain Risks: {cross_domain_risk_count}
🎯 Priority Actions: {priority_action_count}
🚀 Performance: Parallel execution ~67% faster
✅ Ready for report generation (Step 5)
```
---
---
### 8. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04e-aggregate-nfr']
lastStep: 'step-04e-aggregate-nfr'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04e-aggregate-nfr'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04e-aggregate-nfr'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
---
## EXIT CONDITION
Proceed to Step 5 when:
- ✅ All subprocess outputs read
- ✅ Overall risk calculated
- ✅ Compliance aggregated
- ✅ Summary saved
- ✅ Progress saved to output document
Load next step: `{nextStepFile}`
---
## 🚨 SYSTEM SUCCESS METRICS
### ✅ SUCCESS:
- All 4 NFR domains aggregated correctly
- Overall risk level determined
- Executive summary complete
### ❌ FAILURE:
- Failed to read subprocess outputs
- Risk calculation incorrect
@@ -0,0 +1,97 @@
---
name: 'step-05-generate-report'
description: 'Create NFR report and validation summary'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 5: Generate Report & Validate
## STEP GOAL
Produce the NFR assessment report and validate completeness.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Report Generation
Use `nfr-report-template.md` to produce `{outputFile}` containing:
- Category results (PASS/CONCERNS/FAIL)
- Evidence summary
- Remediation actions
- Gate-ready YAML snippet (if applicable)
---
## 2. Validation
Validate against `checklist.md` and fix gaps.
- [ ] CLI sessions cleaned up (no orphaned browsers)
---
## 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-05-generate-report']
lastStep: 'step-05-generate-report'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-05-generate-report'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-05-generate-report'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
---
## 4. Completion Summary
Report:
- Overall NFR status
- Critical blockers or waivers needed
- Next recommended workflow (`trace` or release gate)
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.
@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target
@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update
@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/nfr-assess-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced
@@ -0,0 +1,73 @@
---
validationDate: 2026-01-27
workflowName: testarch-nfr
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/nfr-assess
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-nfr
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-load-context.md: 78 lines [GOOD]
- steps-c/step-02-define-thresholds.md: 75 lines [GOOD]
- steps-c/step-03-gather-evidence.md: 58 lines [GOOD]
- steps-c/step-04-evaluate-and-score.md: 61 lines [GOOD]
- steps-c/step-05-generate-report.md: 64 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: nfr-report-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-05-generate-report.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,116 @@
---
validationDate: 2026-01-27
workflowName: testarch-nfr
workflowPath: /Users/murat.ozcan/opensource/bmad-method-test-architecture-enterprise/src/workflows/testarch/nfr-assess
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-nfr
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-load-context.md: 77 lines [GOOD]
- steps-c/step-02-define-thresholds.md: 74 lines [GOOD]
- steps-c/step-03-gather-evidence.md: 57 lines [GOOD]
- steps-c/step-04-evaluate-and-score.md: 60 lines [GOOD]
- steps-c/step-05-generate-report.md: 63 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-load-context.md: Init [PASS]
- steps-c/step-02-define-thresholds.md: Middle [PASS]
- steps-c/step-03-gather-evidence.md: Middle [PASS]
- steps-c/step-04-evaluate-and-score.md: Middle [PASS]
- steps-c/step-05-generate-report.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: nfr-report-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-05-generate-report.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subprocess Optimization Opportunities
- No high-priority subprocess optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 8 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)
@@ -0,0 +1,19 @@
# Workflow Plan: testarch-nfr
## Create Mode (steps-c)
- step-01-load-context.md
- step-02-define-thresholds.md
- step-03-gather-evidence.md
- step-04-evaluate-and-score.md
- step-05-generate-report.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- {test_artifacts}/nfr-assessment.md
@@ -0,0 +1,41 @@
---
name: testarch-nfr
description: 'Assess non-functional requirements (performance, security, reliability, maintainability) before release with evidence-based validation'
web_bundle: true
---
# Non-Functional Requirements Assessment
**Goal:** Assess non-functional requirements (performance, security, reliability, maintainability) before release with evidence-based validation
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-load-context.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`
@@ -0,0 +1,49 @@
# Test Architect workflow: nfr-assess
name: testarch-nfr
description: "Assess non-functional requirements (performance, security, reliability, maintainability) before release with evidence-based validation"
author: "BMad"
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/nfr-assess"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
template: "{installed_path}/nfr-report-template.md"
# Variables and inputs
variables:
# NFR category assessment (defaults to all categories)
custom_nfr_categories: "" # Optional additional categories beyond standard (security, performance, reliability, maintainability)
# Output configuration
default_output_file: "{test_artifacts}/nfr-assessment.md"
# Required tools
required_tools:
- read_file # Read story, test results, metrics, logs, BMad artifacts
- write_file # Create NFR assessment, gate YAML, evidence checklist
- list_files # Discover test results, metrics, logs
- search_repo # Find NFR-related tests and evidence
- glob # Find result files matching patterns
tags:
- qa
- nfr
- test-architect
- performance
- security
- reliability
execution_hints:
interactive: false # Minimize prompts
autonomous: true # Proceed without user input unless blocked
iterative: true
␍
@@ -0,0 +1,197 @@
# Teach Me Testing - Quality Checklist
## Workflow Quality Standards
Use this checklist to validate the teaching workflow meets quality standards.
---
## Foundation Quality
- [ ] **workflow.md** exists with proper frontmatter
- [ ] Tri-modal routing logic present (Create/Edit/Validate)
- [ ] Configuration loading references correct module (TEA)
- [ ] First step path correct (`./steps-c/step-01-init.md`)
- [ ] Folder structure complete (steps-c/, steps-e/, steps-v/, data/, templates/)
---
## Template Quality
- [ ] **progress-template.yaml** has complete schema
- [ ] All 7 sessions defined with proper structure
- [ ] Session status tracking fields present (not-started/in-progress/completed)
- [ ] stepsCompleted array for continuation tracking
- [ ] **session-notes-template.md** has all required sections
- [ ] **certificate-template.md** includes all 7 sessions
---
## Step File Quality (CREATE mode)
### Initialization Steps
- [ ] **step-01-init.md** checks for existing progress file
- [ ] Continuation detection logic works correctly
- [ ] **step-01b-continue.md** loads progress and routes to session menu
- [ ] Progress dashboard displays completion status
### Assessment Step
- [ ] **step-02-assess.md** gathers role, experience, goals
- [ ] Validation for role (QA/Dev/Lead/VP)
- [ ] Validation for experience (beginner/intermediate/experienced)
- [ ] Assessment data written to progress file
### Session Menu Hub
- [ ] **step-03-session-menu.md** displays all 7 sessions
- [ ] Completion indicators shown (✓ completed, 🔄 in-progress, ⬜ not-started)
- [ ] Branching logic routes to selected session (1-7)
- [ ] Exit logic (X) routes to completion if all done, otherwise saves and exits
### Session Steps (1-7)
- [ ] Each session loads relevant TEA docs just-in-time
- [ ] Teaching content presented (mostly autonomous)
- [ ] Quiz validation with ≥70% threshold
- [ ] Session notes artifact generated
- [ ] Progress file updated (status, score, artifact path)
- [ ] Returns to session menu hub after completion
### Completion Step
- [ ] **step-05-completion.md** verifies all 7 sessions complete
- [ ] Certificate generated with accurate data
- [ ] Final progress file update (certificate_generated: true)
- [ ] Congratulations message shown
---
## Data File Quality
- [ ] **curriculum.yaml** defines all 7 sessions
- [ ] **role-paths.yaml** maps role customizations
- [ ] **session-content-map.yaml** references TEA docs/fragments/URLs correctly
- [ ] **quiz-questions.yaml** has questions for all sessions
- [ ] **tea-resources-index.yaml** has complete documentation index
---
## Content Quality
### TEA Documentation Integration
- [ ] Local file paths correct (`/docs/*.md`, `/src/testarch/knowledge/*.md`)
- [ ] Online URLs correct (<https://bmad-code-org.github.io/...>)
- [ ] GitHub fragment links correct
- [ ] Triple reference system (local + online + GitHub) implemented
### Role-Based Content
- [ ] QA examples present (practical testing focus)
- [ ] Dev examples present (integration/TDD focus)
- [ ] Lead examples present (architecture/patterns focus)
- [ ] VP examples present (strategy/metrics focus)
### Quiz Quality
- [ ] Questions test understanding, not memorization
- [ ] 3-5 questions per session
- [ ] Mix of difficulty levels
- [ ] Clear correct answers with explanations
---
## Error Handling
- [ ] Corrupted progress file detection
- [ ] Backup and recovery options
- [ ] Missing TEA docs fallback (Web-Browsing)
- [ ] Quiz failure recovery (review or continue)
- [ ] Session interruption handling (auto-save)
---
## User Experience
- [ ] Clear navigation instructions
- [ ] Progress visibility (completion percentage, next recommended)
- [ ] Auto-save after each session
- [ ] Resume capability works seamlessly
- [ ] Exit options clear at all decision points
---
## State Management
- [ ] stepsCompleted array updated correctly
- [ ] Session tracking accurate (status, dates, scores)
- [ ] Completion percentage calculated correctly
- [ ] Next recommended session logic works
- [ ] lastStep and lastContinued timestamps updated
---
## Validation Mode
- [ ] **step-v-01-validate.md** checks all quality standards
- [ ] Generates validation report
- [ ] Identifies issues clearly
- [ ] Provides remediation suggestions
---
## Edit Mode
- [ ] **step-e-01-assess-workflow.md** identifies what to edit
- [ ] **step-e-02-apply-edits.md** applies modifications safely
- [ ] Preserves workflow integrity during edits
---
## Documentation
- [ ] **instructions.md** clear and complete
- [ ] **checklist.md** (this file) comprehensive
- [ ] README (if present) accurate
- [ ] Inline comments in complex logic
---
## Performance
- [ ] Just-in-time loading (not loading all docs upfront)
- [ ] Session steps complete in reasonable time (<5 min)
- [ ] Quiz validation fast (<1 min)
- [ ] Progress file writes efficient
---
## Security
- [ ] No hardcoded credentials
- [ ] File paths use variables
- [ ] Progress files private to user
- [ ] No sensitive data in session notes
---
## Completion Criteria
✅ **Workflow is ready for deployment when:**
- All checkboxes above are checked
- All step files exist and follow standards
- All templates present and correct
- Data files complete and accurate
- Error handling robust
- User experience smooth
- Documentation complete
---
**Validation Date:** **\*\***\_\_\_**\*\***
**Validated By:** **\*\***\_\_\_**\*\***
**Issues Found:** **\*\***\_\_\_**\*\***
**Status:** ⬜ Ready for Production | ⬜ Needs Revisions
@@ -0,0 +1,129 @@
# TEA Academy Curriculum Structure
# Defines the 7-session learning path with objectives and content mappings
sessions:
- id: session-01-quickstart
name: "Quick Start"
duration: "30 min"
difficulty: beginner
objective: "Get immediate value by seeing TEA in action"
description: "TEA Lite intro, run automate workflow, understand engagement models"
recommended_for:
- beginner
- intermediate
- experienced
prerequisites: []
- id: session-02-concepts
name: "Core Concepts"
duration: "45 min"
difficulty: beginner
objective: "Understand WHY behind TEA principles"
description: "Risk-based testing, DoD, testing as engineering philosophy"
recommended_for:
- beginner
- intermediate
prerequisites: []
- id: session-03-architecture
name: "Architecture & Patterns"
duration: "60 min"
difficulty: intermediate
objective: "Understand TEA patterns and architecture"
description: "Fixtures, network-first patterns, data factories, step-file architecture"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-04-test-design
name: "Test Design"
duration: "60 min"
difficulty: intermediate
objective: "Learn risk assessment and coverage planning"
description: "Test Design workflow, risk/testability assessment, coverage planning"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-05-atdd-automate
name: "ATDD & Automate"
duration: "60 min"
difficulty: intermediate
objective: "Generate tests with TDD red-green approach"
description: "ATDD workflow (red phase), Automate workflow, component TDD, API testing"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-06-quality-trace
name: "Quality & Trace"
duration: "45 min"
difficulty: intermediate
objective: "Audit quality and ensure traceability"
description: "Test Review (5 dimensions), Trace workflow, quality metrics"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-07-advanced
name: "Advanced Patterns"
duration: "ongoing"
difficulty: advanced
objective: "Deep-dive into specific knowledge fragments"
description: "Menu-driven exploration of 35 knowledge fragments organized by category"
recommended_for:
- experienced
prerequisites: []
# Learning Paths by Experience Level
learning_paths:
beginner:
recommended_sequence:
- session-01-quickstart
- session-02-concepts
- session-03-architecture
- session-04-test-design
- session-05-atdd-automate
- session-06-quality-trace
- session-07-advanced
skip_optional: []
intermediate:
recommended_sequence:
- session-01-quickstart
- session-02-concepts
- session-03-architecture
- session-04-test-design
- session-05-atdd-automate
- session-06-quality-trace
- session-07-advanced
skip_optional:
- session-01-quickstart # Can skip if already familiar
certificate_eligible_if_skipped: false
experienced:
recommended_sequence:
- session-02-concepts
- session-03-architecture
- session-04-test-design
- session-05-atdd-automate
- session-06-quality-trace
- session-07-advanced
skip_optional:
- session-01-quickstart
certificate_eligible_if_skipped: false
# Completion Requirements
completion:
minimum_sessions: 7 # All sessions required for certificate
passing_score: 70 # Minimum quiz score to pass session
average_score_threshold: 70 # Minimum average for certificate
certificate_note: "Certificate eligibility requires completion.minimum_sessions. If intermediate.skip_optional or experienced.skip_optional sessions are skipped, certificate eligibility is forfeited."
@@ -0,0 +1,206 @@
# Quiz Questions Bank
# Organized by session with questions, answers, and explanations
session-01-quickstart:
passing_score: 70
questions:
- id: q1-purpose
question: "What is the primary purpose of TEA?"
options:
A: "Replace all testing tools with a single framework"
B: "Make testing expertise accessible through structured workflows and knowledge"
C: "Automate 100% of test writing"
D: "Only works for Playwright tests"
correct: B
explanation: "TEA makes testing expertise accessible and scalable through workflows and knowledge fragments. It's not about replacing tools or automating everything."
- id: q2-risk-matrix
question: "What does the P0-P3 risk matrix help with?"
options:
A: "Prioritizing test coverage based on criticality"
B: "Grading test code quality"
C: "Measuring test execution speed"
D: "Tracking bug severity"
correct: A
explanation: "P0-P3 helps prioritize what to test based on risk (Probability × Impact). P0 = critical features like login, P3 = nice-to-have like tooltips."
- id: q3-engagement
question: "Which TEA engagement model is best for quick value in 30 minutes?"
options:
A: "TEA Enterprise"
B: "TEA Lite"
C: "TEA Integrated"
D: "TEA Brownfield"
correct: B
explanation: "TEA Lite is the 30-minute quick start approach. Enterprise and Integrated are more comprehensive."
session-02-concepts:
passing_score: 70
questions:
- id: q1-p0-priority
question: "In the P0-P3 matrix, what priority level should login/authentication have?"
options:
A: "P3 - Low priority"
B: "P2 - Medium priority"
C: "P1 - High priority"
D: "P0 - Critical priority"
correct: D
explanation: "Login/authentication is P0 - critical. Business fails if broken. High usage, high impact, business-critical."
- id: q2-hard-waits
question: "What is the problem with using sleep(5000) instead of waitFor conditions?"
options:
A: "It makes tests slower"
B: "It's a hard wait that doesn't react to state changes (violates DoD)"
C: "It uses too much memory"
D: "It's not supported in modern frameworks"
correct: B
explanation: "Hard waits don't react to state changes - they guess timing. Use waitFor to react to conditions. This violates TEA Definition of Done."
- id: q3-self-cleaning
question: "What does 'self-cleaning tests' mean in TEA Definition of Done?"
options:
A: "Tests automatically fix their own bugs"
B: "Tests delete/deactivate entities they create during testing"
C: "Tests run faster by cleaning up code"
D: "Tests remove old test files"
correct: B
explanation: "Self-cleaning means tests delete/deactivate entities they created. No manual cleanup required."
session-03-architecture:
passing_score: 70
questions:
- id: q1-fixtures
question: "What is the main benefit of fixture composition?"
options:
A: "Faster test execution"
B: "DRY - define once, reuse everywhere"
C: "Better error messages"
D: "Automatic screenshot capture"
correct: B
explanation: "Fixture composition allows you to define setup once and reuse everywhere. DRY principle for test setup."
- id: q2-network-first
question: "Why is 'network-first' better than mocking after the action?"
options:
A: "It's faster"
B: "It prevents race conditions"
C: "It uses less memory"
D: "It's easier to write"
correct: B
explanation: "Setting up network interception BEFORE the action prevents race conditions. The mock is ready when the action triggers."
- id: q3-step-file
question: "What pattern does this teaching workflow use?"
options:
A: "Page Object Model"
B: "Behavior Driven Development"
C: "Step-File Architecture"
D: "Test Pyramid"
correct: C
explanation: "This workflow uses step-file architecture: micro-file design, just-in-time loading, sequential enforcement."
session-04-test-design:
passing_score: 70
questions:
- id: q1-test-design-purpose
question: "What does the Test Design workflow help you do?"
options:
A: "Write tests faster"
B: "Plan tests BEFORE writing them"
C: "Run tests in parallel"
D: "Debug test failures"
correct: B
explanation: "Test Design workflow helps you plan tests before writing them. Design before code, like architecture before implementation."
- id: q2-risk-calculation
question: "How do you calculate risk?"
options:
A: "Probability + Impact"
B: "Probability × Impact"
C: "Probability - Impact"
D: "Probability / Impact"
correct: B
explanation: "Risk = Probability × Impact. Multiply the likelihood of failure by the impact of failure."
- id: q3-p0-coverage
question: "For P0 features, which test levels should you use?"
options:
A: "Only E2E tests"
B: "Only unit tests"
C: "Unit + Integration + E2E (comprehensive)"
D: "Manual testing only"
correct: C
explanation: "P0 features need comprehensive coverage: Unit + Integration + E2E. High confidence for critical features."
session-05-atdd-automate:
passing_score: 70
questions:
- id: q1-red-phase
question: "What is the 'red' phase in TDD?"
options:
A: "Tests fail (code doesn't exist yet)"
B: "Tests pass"
C: "Code is refactored"
D: "Tests are deleted"
correct: A
explanation: "Red phase: Tests fail because the code doesn't exist yet. Write tests first, then implement."
- id: q2-atdd-vs-automate
question: "What's the difference between ATDD and Automate workflows?"
options:
A: "ATDD generates E2E, Automate generates API tests"
B: "ATDD writes tests first (red phase), Automate tests existing code"
C: "ATDD is faster than Automate"
D: "They're the same workflow"
correct: B
explanation: "ATDD writes failing tests first (red phase), then you implement. Automate generates tests for existing code (coverage expansion)."
- id: q3-api-testing
question: "Why use pure API tests without a browser?"
options:
A: "They look prettier"
B: "They're easier to debug"
C: "They're faster and test business logic directly"
D: "They're required by TEA"
correct: C
explanation: "Pure API tests are faster (no browser overhead) and test business logic directly without UI complexity."
session-06-quality-trace:
passing_score: 70
questions:
- id: q1-five-dimensions
question: "What are the 5 dimensions in Test Review workflow?"
options:
A: "Speed, cost, coverage, bugs, time"
B: "Determinism, Isolation, Assertions, Structure, Performance"
C: "Unit, integration, E2E, manual, exploratory"
D: "P0, P1, P2, P3, P4"
correct: B
explanation: "Test Review evaluates 5 dimensions: Determinism (no flakiness), Isolation (parallel-safe), Assertions (correct checks), Structure (readable/maintainable organization), Performance (speed)."
- id: q2-release-gate
question: "When should the Trace workflow gate decision be RED (block release)?"
options:
A: "Any test failures exist"
B: "P0 gaps exist (critical requirements not tested)"
C: "Code coverage is below 80%"
D: "Tests are slow"
correct: B
explanation: "RED gate when P0 gaps exist - critical requirements not tested. Don't ship if critical features lack test coverage."
- id: q3-metrics
question: "Which metric matters most for quality?"
options:
A: "Total line coverage %"
B: "Number of tests written"
C: "P0/P1 coverage %"
D: "Test file count"
correct: C
explanation: "P0/P1 coverage matters most - it measures coverage of critical/high-priority features. Total line coverage is a vanity metric."
session-07-advanced:
# No quiz - exploratory session
# Score: 100 (completion based, not quiz based)
passing_score: 100
questions: []
@@ -0,0 +1,136 @@
# Role-Based Content Customization
# Defines how teaching examples and focus areas adapt based on learner role
roles:
qa:
display_name: "QA Engineer"
focus_areas:
- Practical testing workflow usage
- Test framework setup and maintenance
- Test quality and coverage metrics
- CI/CD integration
example_contexts:
- "Expanding test coverage for existing features"
- "Setting up test framework for new project"
- "Reducing flaky tests in CI pipeline"
- "Improving test execution speed"
recommended_sessions:
- session-01-quickstart
- session-02-concepts
- session-03-architecture
- session-05-atdd-automate
- session-06-quality-trace
teaching_adaptations:
session-01-quickstart: "Focus on Automate workflow - quickly expand coverage"
session-02-concepts: "Emphasize P0-P3 for defending coverage decisions"
session-03-architecture: "Fixture patterns for maintainable test suites"
session-04-test-design: "Test design for planning coverage expansion"
session-05-atdd-automate: "ATDD and Automate for test generation"
session-06-quality-trace: "Test Review for quality metrics reporting"
session-07-advanced: "Playwright Utils for advanced testing patterns"
dev:
display_name: "Software Developer"
focus_areas:
- Integration testing perspective
- TDD approach
- Test-driven development workflow
- Unit and integration tests
example_contexts:
- "Writing tests alongside feature development"
- "Using ATDD to drive implementation"
- "Integrating tests into development workflow"
- "Testing APIs and business logic"
recommended_sessions:
- session-01-quickstart
- session-02-concepts
- session-05-atdd-automate
- session-03-architecture
- session-04-test-design
teaching_adaptations:
session-01-quickstart: "Focus on ATDD - tests drive implementation"
session-02-concepts: "Connect DoD to code quality standards"
session-03-architecture: "Fixtures as code patterns, like dependency injection"
session-04-test-design: "Risk assessment before writing code"
session-05-atdd-automate: "Red-green-refactor TDD cycle"
session-06-quality-trace: "Test quality like code quality - refactoring applies"
session-07-advanced: "API testing patterns, component TDD"
lead:
display_name: "Tech Lead / Engineering Manager"
focus_areas:
- Test architecture decisions
- Team testing patterns
- Framework and tooling choices
- Quality standards enforcement
example_contexts:
- "Establishing team testing standards"
- "Choosing test architecture patterns"
- "Code review for test quality"
- "Scaling test automation across team"
recommended_sessions:
- session-01-quickstart
- session-03-architecture
- session-04-test-design
- session-06-quality-trace
- session-07-advanced
teaching_adaptations:
session-01-quickstart: "TEA as team standard - scalable patterns"
session-02-concepts: "DoD as code review checklist - enforce quality"
session-03-architecture: "Architecture patterns for team consistency"
session-04-test-design: "Test design as planning phase in development"
session-05-atdd-automate: "ATDD for team TDD adoption"
session-06-quality-trace: "Test Review for quality metrics and team standards"
session-07-advanced: "Step-file architecture, fixture patterns, CI governance"
vp:
display_name: "VP Engineering / Director"
focus_areas:
- Testing strategy and ROI
- Quality metrics that matter
- Team scalability
- Risk management through testing
example_contexts:
- "Justifying test automation investment"
- "Scaling testing across multiple teams"
- "Quality metrics for stakeholder reporting"
- "Risk mitigation through test coverage"
recommended_sessions:
- session-01-quickstart
- session-02-concepts
- session-04-test-design
- session-06-quality-trace
teaching_adaptations:
session-01-quickstart: "TEA scales testing without scaling headcount"
session-02-concepts: "Risk-based testing aligns engineering with business impact"
session-03-architecture: "Architecture patterns reduce maintenance costs"
session-04-test-design: "Test design makes risk visible to stakeholders"
session-05-atdd-automate: "ATDD reduces defect rates early"
session-06-quality-trace: "Quality metrics: P0/P1 coverage, not vanity metrics"
session-07-advanced: "Governance patterns, CI orchestration, NFR assessment"
# Role-Based Example Types
example_types:
qa:
- "Test suite maintenance scenarios"
- "Coverage expansion projects"
- "Flaky test debugging"
- "CI pipeline configuration"
dev:
- "Feature development with TDD"
- "API integration testing"
- "Unit test patterns"
- "Mocking and stubbing"
lead:
- "Team architecture decisions"
- "Code review scenarios"
- "Standard enforcement"
- "Tooling selection"
vp:
- "ROI calculations"
- "Quality dashboards"
- "Risk reporting"
- "Team scaling strategies"
@@ -0,0 +1,207 @@
# Session Content Mapping
# Maps each session to specific TEA documentation, knowledge fragments, and online resources
base_paths:
tea_docs: "/docs"
tea_knowledge: "/src/testarch/knowledge"
online_base: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise"
github_knowledge: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/tree/main/src/testarch/knowledge"
sessions:
session-01-quickstart:
docs:
- path: "/docs/tutorials/tea-lite-quickstart.md"
title: "TEA Lite Quickstart"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/tutorials/tea-lite-quickstart/"
- path: "/docs/explanation/tea-overview.md"
title: "TEA Overview"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/tea-overview/"
- path: "/docs/how-to/workflows/run-automate.md"
title: "Run Automate Workflow"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-automate/"
knowledge_fragments: []
online_references:
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/"
workflows_referenced:
- automate
key_concepts:
- "What is TEA"
- "TEA Lite approach"
- "Engagement models"
- "9 workflows overview"
session-02-concepts:
docs:
- path: "/docs/explanation/testing-as-engineering.md"
title: "Testing as Engineering"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/testing-as-engineering/"
- path: "/docs/explanation/risk-based-testing.md"
title: "Risk-Based Testing"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/risk-based-testing/"
- path: "/docs/explanation/test-quality-standards.md"
title: "Test Quality Standards"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/test-quality-standards/"
knowledge_fragments:
- path: "/src/testarch/knowledge/test-quality.md"
title: "Test Quality (DoD Execution Limits)"
- path: "/src/testarch/knowledge/probability-impact.md"
title: "Probability × Impact Scoring"
online_references:
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/testing-as-engineering/"
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/risk-based-testing/"
workflows_referenced: []
key_concepts:
- "Testing as engineering philosophy"
- "P0-P3 risk matrix"
- "Probability × Impact scoring"
- "Definition of Done (7 principles)"
session-03-architecture:
docs:
- path: "/docs/explanation/fixture-architecture.md"
title: "Fixture Architecture"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/fixture-architecture/"
- path: "/docs/explanation/network-first-patterns.md"
title: "Network-First Patterns"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/network-first-patterns/"
- path: "/docs/explanation/step-file-architecture.md"
title: "Step-File Architecture"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/step-file-architecture/"
knowledge_fragments:
- path: "/src/testarch/knowledge/fixture-architecture.md"
title: "Fixture Architecture Patterns"
- path: "/src/testarch/knowledge/network-first.md"
title: "Network-First Implementation"
- path: "/src/testarch/knowledge/data-factories.md"
title: "Data Factories Pattern"
online_references:
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/explanation/fixture-architecture/"
workflows_referenced:
- framework
key_concepts:
- "Fixture composition"
- "Network interception patterns"
- "Data factory pattern"
- "Step-file architecture"
session-04-test-design:
docs:
- path: "/docs/how-to/workflows/run-test-design.md"
title: "Run Test Design Workflow"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-test-design/"
knowledge_fragments:
- path: "/src/testarch/knowledge/test-levels-framework.md"
title: "Test Levels Framework"
- path: "/src/testarch/knowledge/test-priorities-matrix.md"
title: "Test Priorities Matrix"
online_references:
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-test-design/"
workflows_referenced:
- test-design
key_concepts:
- "Test Design workflow steps"
- "Risk/testability assessment"
- "Coverage planning"
- "Test levels (unit/integration/E2E)"
session-05-atdd-automate:
docs:
- path: "/docs/how-to/workflows/run-atdd.md"
title: "Run ATDD Workflow"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-atdd/"
- path: "/docs/how-to/workflows/run-automate.md"
title: "Run Automate Workflow"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-automate/"
knowledge_fragments:
- path: "/src/testarch/knowledge/component-tdd.md"
title: "Component TDD Red-Green Loop"
- path: "/src/testarch/knowledge/api-testing-patterns.md"
title: "API Testing Patterns"
- path: "/src/testarch/knowledge/api-request.md"
title: "API Request Utility"
online_references:
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-atdd/"
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-automate/"
workflows_referenced:
- atdd
- automate
key_concepts:
- "ATDD workflow (red phase)"
- "TDD red-green-refactor"
- "Automate workflow (coverage expansion)"
- "API testing without browser"
session-06-quality-trace:
docs:
- path: "/docs/how-to/workflows/run-test-review.md"
title: "Run Test Review Workflow"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-test-review/"
- path: "/docs/how-to/workflows/run-trace.md"
title: "Run Trace Workflow"
url: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-trace/"
knowledge_fragments: []
online_references:
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-test-review/"
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/how-to/workflows/run-trace/"
workflows_referenced:
- test-review
- trace
key_concepts:
- "5 dimensions of test quality"
- "Quality scoring (0-100)"
- "Requirements traceability"
- "Release gate decisions"
session-07-advanced:
docs: []
knowledge_fragments:
categories:
testing_patterns:
- fixture-architecture.md
- network-first.md
- data-factories.md
- component-tdd.md
- api-testing-patterns.md
- test-healing-patterns.md
- selector-resilience.md
- timing-debugging.md
playwright_utils:
- api-request.md
- network-recorder.md
- intercept-network-call.md
- recurse.md
- log.md
- file-utils.md
- burn-in.md
- network-error-monitor.md
- contract-testing.md
browser_automation:
- playwright-cli.md
configuration_governance:
- playwright-config.md
- ci-burn-in.md
- selective-testing.md
- feature-flags.md
- risk-governance.md
quality_frameworks:
- test-quality.md
- test-levels-framework.md
- test-priorities-matrix.md
- nfr-criteria.md
auth_security:
- email-auth.md
- auth-session.md
- error-handling.md
online_references:
- "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise/reference/knowledge-base/"
- "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/tree/main/src/testarch/knowledge"
workflows_referenced: []
key_concepts:
- "Menu-driven fragment exploration"
- "Just-in-time deep-dive learning"
- "35 knowledge fragments organized by category"
@@ -0,0 +1,359 @@
# TEA Resources Index
# Comprehensive index of TEA documentation, knowledge fragments, and online resources
base_urls:
online_docs: "https://bmad-code-org.github.io/bmad-method-test-architecture-enterprise"
github_repo: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise"
github_knowledge: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/tree/main/src/testarch/knowledge"
# Public Documentation (32 files)
documentation:
tutorials:
- name: "Getting Started with Test Architect"
local: "/docs/tutorials/tea-lite-quickstart.md"
online: "/tutorials/tea-lite-quickstart/"
description: "30-minute quick start guide to TEA Lite"
how_to_guides:
workflows:
- name: "Set Up Test Framework"
local: "/docs/how-to/workflows/setup-test-framework.md"
online: "/how-to/workflows/setup-test-framework/"
workflow: framework
- name: "Set Up CI Pipeline"
local: "/docs/how-to/workflows/setup-ci.md"
online: "/how-to/workflows/setup-ci/"
workflow: ci
- name: "Test Design"
local: "/docs/how-to/workflows/run-test-design.md"
online: "/how-to/workflows/run-test-design/"
workflow: test-design
- name: "ATDD"
local: "/docs/how-to/workflows/run-atdd.md"
online: "/how-to/workflows/run-atdd/"
workflow: atdd
- name: "Automate"
local: "/docs/how-to/workflows/run-automate.md"
online: "/how-to/workflows/run-automate/"
workflow: automate
- name: "Test Review"
local: "/docs/how-to/workflows/run-test-review.md"
online: "/how-to/workflows/run-test-review/"
workflow: test-review
- name: "Trace"
local: "/docs/how-to/workflows/run-trace.md"
online: "/how-to/workflows/run-trace/"
workflow: trace
- name: "NFR Assessment"
local: "/docs/how-to/workflows/run-nfr-assess.md"
online: "/how-to/workflows/run-nfr-assess/"
workflow: nfr-assess
customization:
- name: "Configure Browser Automation"
local: "/docs/how-to/customization/configure-browser-automation.md"
online: "/how-to/customization/configure-browser-automation/"
- name: "Integrate Playwright Utils with TEA"
local: "/docs/how-to/customization/integrate-playwright-utils.md"
online: "/how-to/customization/integrate-playwright-utils/"
brownfield:
- name: "Running TEA for Enterprise Projects"
local: "/docs/how-to/brownfield/use-tea-for-enterprise.md"
online: "/how-to/brownfield/use-tea-for-enterprise/"
- name: "Using TEA with Existing Tests"
local: "/docs/how-to/brownfield/use-tea-with-existing-tests.md"
online: "/how-to/brownfield/use-tea-with-existing-tests/"
explanation:
- name: "TEA Overview"
local: "/docs/explanation/tea-overview.md"
online: "/explanation/tea-overview/"
topics: ["Architecture", "Engagement models"]
- name: "Testing as Engineering"
local: "/docs/explanation/testing-as-engineering.md"
online: "/explanation/testing-as-engineering/"
topics: ["Philosophy", "Design principles"]
- name: "Engagement Models"
local: "/docs/explanation/engagement-models.md"
online: "/explanation/engagement-models/"
topics: ["Lite", "Solo", "Integrated", "Enterprise", "Brownfield"]
- name: "Risk-Based Testing"
local: "/docs/explanation/risk-based-testing.md"
online: "/explanation/risk-based-testing/"
topics: ["P0-P3 matrix", "Probability × Impact"]
- name: "Test Quality Standards"
local: "/docs/explanation/test-quality-standards.md"
online: "/explanation/test-quality-standards/"
topics: ["Definition of Done", "7 principles"]
- name: "Knowledge Base System"
local: "/docs/explanation/knowledge-base-system.md"
online: "/explanation/knowledge-base-system/"
topics: ["Fragment management", "35 fragments"]
- name: "Network-First Patterns"
local: "/docs/explanation/network-first-patterns.md"
online: "/explanation/network-first-patterns/"
topics: ["Network interception", "Race condition prevention"]
- name: "Fixture Architecture"
local: "/docs/explanation/fixture-architecture.md"
online: "/explanation/fixture-architecture/"
topics: ["Composition", "mergeTests pattern"]
- name: "Step-File Architecture"
local: "/docs/explanation/step-file-architecture.md"
online: "/explanation/step-file-architecture/"
topics: ["Micro-file design", "JIT loading", "Sequential enforcement"]
- name: "Subprocess Architecture"
local: "/docs/explanation/subprocess-architecture.md"
online: "/explanation/subprocess-architecture/"
topics: ["Parallel execution", "Context optimization"]
reference:
- name: "Commands"
local: "/docs/reference/commands.md"
online: "/reference/commands/"
- name: "Configuration"
local: "/docs/reference/configuration.md"
online: "/reference/configuration/"
- name: "Knowledge Base"
local: "/docs/reference/knowledge-base.md"
online: "/reference/knowledge-base/"
github_link: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/tree/main/src/testarch/knowledge"
- name: "Troubleshooting"
local: "/docs/reference/troubleshooting.md"
online: "/reference/troubleshooting/"
# Knowledge Fragments (34 files)
knowledge_fragments:
testing_patterns:
- name: "fixture-architecture"
path: "/src/testarch/knowledge/fixture-architecture.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/fixture-architecture.md"
description: "Composable fixture patterns and mergeTests"
- name: "fixtures-composition"
path: "/src/testarch/knowledge/fixtures-composition.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/fixtures-composition.md"
description: "mergeTests composition patterns for combining utilities"
- name: "network-first"
path: "/src/testarch/knowledge/network-first.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/network-first.md"
description: "Network interception safeguards"
- name: "data-factories"
path: "/src/testarch/knowledge/data-factories.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/data-factories.md"
description: "Data seeding and setup patterns"
- name: "component-tdd"
path: "/src/testarch/knowledge/component-tdd.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/component-tdd.md"
description: "TDD red-green-refactor loop"
- name: "api-testing-patterns"
path: "/src/testarch/knowledge/api-testing-patterns.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/api-testing-patterns.md"
description: "Pure API testing without browser"
- name: "test-healing-patterns"
path: "/src/testarch/knowledge/test-healing-patterns.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/test-healing-patterns.md"
description: "Auto-fix common test failures"
- name: "selector-resilience"
path: "/src/testarch/knowledge/selector-resilience.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/selector-resilience.md"
description: "Robust selectors that don't break"
- name: "timing-debugging"
path: "/src/testarch/knowledge/timing-debugging.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/timing-debugging.md"
description: "Race condition fixes"
playwright_utils:
- name: "overview"
path: "/src/testarch/knowledge/overview.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/overview.md"
description: "Playwright Utils overview and installation"
- name: "api-request"
path: "/src/testarch/knowledge/api-request.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/api-request.md"
description: "Typed HTTP client with schema validation"
- name: "network-recorder"
path: "/src/testarch/knowledge/network-recorder.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/network-recorder.md"
description: "HAR record and playback"
- name: "intercept-network-call"
path: "/src/testarch/knowledge/intercept-network-call.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/intercept-network-call.md"
description: "Network spy and stub utilities"
- name: "recurse"
path: "/src/testarch/knowledge/recurse.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/recurse.md"
description: "Async polling for eventual consistency"
- name: "log"
path: "/src/testarch/knowledge/log.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/log.md"
description: "Test report logging utilities"
- name: "file-utils"
path: "/src/testarch/knowledge/file-utils.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/file-utils.md"
description: "CSV/XLSX/PDF/ZIP validation"
- name: "burn-in"
path: "/src/testarch/knowledge/burn-in.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/burn-in.md"
description: "Smart test selection via git diff"
- name: "network-error-monitor"
path: "/src/testarch/knowledge/network-error-monitor.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/network-error-monitor.md"
description: "HTTP 4xx/5xx detection"
- name: "contract-testing"
path: "/src/testarch/knowledge/contract-testing.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/contract-testing.md"
description: "Pact publishing and provider verification"
- name: "visual-debugging"
path: "/src/testarch/knowledge/visual-debugging.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/visual-debugging.md"
description: "Trace viewer workflows and debugging artifacts"
configuration_governance:
- name: "playwright-config"
path: "/src/testarch/knowledge/playwright-config.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/playwright-config.md"
description: "Environment and timeout guardrails"
- name: "ci-burn-in"
path: "/src/testarch/knowledge/ci-burn-in.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/ci-burn-in.md"
description: "CI orchestration and smart selection"
- name: "selective-testing"
path: "/src/testarch/knowledge/selective-testing.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/selective-testing.md"
description: "Tag and grep filters"
- name: "feature-flags"
path: "/src/testarch/knowledge/feature-flags.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/feature-flags.md"
description: "Feature flag governance and cleanup"
- name: "risk-governance"
path: "/src/testarch/knowledge/risk-governance.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/risk-governance.md"
description: "Risk scoring matrix and gate rules"
- name: "adr-quality-readiness-checklist"
path: "/src/testarch/knowledge/adr-quality-readiness-checklist.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/adr-quality-readiness-checklist.md"
description: "Quality readiness checklist for decisions and reviews"
quality_frameworks:
- name: "test-quality"
path: "/src/testarch/knowledge/test-quality.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/test-quality.md"
description: "Definition of Done execution limits"
- name: "test-levels-framework"
path: "/src/testarch/knowledge/test-levels-framework.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/test-levels-framework.md"
description: "Unit/Integration/E2E selection criteria"
- name: "test-priorities-matrix"
path: "/src/testarch/knowledge/test-priorities-matrix.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/test-priorities-matrix.md"
description: "P0-P3 coverage targets"
- name: "probability-impact"
path: "/src/testarch/knowledge/probability-impact.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/probability-impact.md"
description: "Probability × impact scoring definitions"
- name: "nfr-criteria"
path: "/src/testarch/knowledge/nfr-criteria.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/nfr-criteria.md"
description: "Non-functional requirements assessment"
auth_security:
- name: "email-auth"
path: "/src/testarch/knowledge/email-auth.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/email-auth.md"
description: "Magic link extraction and auth state"
- name: "auth-session"
path: "/src/testarch/knowledge/auth-session.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/auth-session.md"
description: "Token persistence and multi-user auth"
- name: "error-handling"
path: "/src/testarch/knowledge/error-handling.md"
github: "https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/testarch/knowledge/error-handling.md"
description: "Exception handling and retry validation"
# Quick Reference Maps
session_to_resources:
session-01:
primary_docs: ["tea-lite-quickstart", "tea-overview", "run-automate"]
fragments: []
session-02:
primary_docs: ["testing-as-engineering", "risk-based-testing", "test-quality-standards"]
fragments: ["test-quality", "probability-impact"]
session-03:
primary_docs: ["fixture-architecture", "network-first-patterns", "step-file-architecture"]
fragments: ["fixture-architecture", "network-first", "data-factories"]
session-04:
primary_docs: ["run-test-design"]
fragments: ["test-levels-framework", "test-priorities-matrix"]
session-05:
primary_docs: ["run-atdd", "run-automate"]
fragments: ["component-tdd", "api-testing-patterns", "api-request"]
session-06:
primary_docs: ["run-test-review", "run-trace"]
fragments: []
session-07:
primary_docs: []
fragments: [] # All 35 fragments available via menu-driven exploration
# Web-Browsing Fallback Strategy
fallback_urls:
playwright_docs: "https://playwright.dev/docs/intro"
jest_docs: "https://jestjs.io/docs/getting-started"
cypress_docs: "https://docs.cypress.io/guides/overview/why-cypress"
vitest_docs: "https://vitest.dev/guide/"
testing_library: "https://testing-library.com/docs/"
Loaded 100 of 724 files, more files were not shown because too many files have changed in this diff. Show more