mirror of
https://github.com/R0m1k3/CollectFlow.git
synced 2026-10-24 07:28:10 +02:00
feat: Add a foundational set of BMM, BMB, TEA, and Core workflows, agent definitions, and supporting templates.
This commit is contained in:
Michael committed
commit
6a0ddd7e36
724 files changed
+110361
No files matched your search
@@ -0,0 +1,161 @@
|
||||
---
|
||||
name: 'step-01-preflight-and-context'
|
||||
description: 'Determine mode, verify framework, and load context and knowledge'
|
||||
outputFile: '{test_artifacts}/automation-summary.md'
|
||||
nextStepFile: './step-02-identify-targets.md'
|
||||
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
|
||||
---
|
||||
|
||||
# Step 1: Preflight & Context Loading
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Determine execution mode, verify framework readiness, and load the necessary artifacts and knowledge fragments.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
- 🚫 Halt if framework scaffolding is missing
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, loaded artifacts, and knowledge fragments
|
||||
- Focus: this step's goal only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: prior steps' outputs (if any)
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
## 1. Verify Framework
|
||||
|
||||
Ensure a framework exists:
|
||||
|
||||
- `playwright.config.ts` or `cypress.config.ts`
|
||||
- `package.json` includes test dependencies
|
||||
|
||||
If missing: **HALT** with message "Run `framework` workflow first."
|
||||
|
||||
---
|
||||
|
||||
## 2. Determine Execution Mode
|
||||
|
||||
- **BMad-Integrated** if story/tech-spec/test-design artifacts are provided or found
|
||||
- **Standalone** if only source code is available
|
||||
- If unclear, ask the user which mode to use
|
||||
|
||||
---
|
||||
|
||||
## 3. Load Context
|
||||
|
||||
### BMad-Integrated (if available)
|
||||
|
||||
- Story with acceptance criteria
|
||||
- PRD and/or tech spec
|
||||
- Test-design document (if exists)
|
||||
|
||||
### Standalone
|
||||
|
||||
- Skip artifacts; proceed to codebase analysis
|
||||
|
||||
### Always Load
|
||||
|
||||
- Test framework config
|
||||
- Existing test structure in `{test_dir}`
|
||||
- Existing tests (for coverage gaps)
|
||||
|
||||
### Read TEA Config Flags
|
||||
|
||||
- From `{config_source}` read `tea_use_playwright_utils`
|
||||
- From `{config_source}` read `tea_browser_automation`
|
||||
|
||||
---
|
||||
|
||||
## 4. Load Knowledge Base Fragments
|
||||
|
||||
Use `{knowledgeIndex}` and load only what is required.
|
||||
|
||||
**Core (always load):**
|
||||
|
||||
- `test-levels-framework.md`
|
||||
- `test-priorities-matrix.md`
|
||||
- `data-factories.md`
|
||||
- `selective-testing.md`
|
||||
- `ci-burn-in.md`
|
||||
- `test-quality.md`
|
||||
|
||||
**Playwright Utils (if enabled):**
|
||||
|
||||
- `overview.md`, `api-request.md`, `network-recorder.md`, `auth-session.md`, `intercept-network-call.md`, `recurse.md`, `log.md`, `file-utils.md`, `burn-in.md`, `network-error-monitor.md`, `fixtures-composition.md`
|
||||
|
||||
**Traditional Patterns (if Playwright Utils disabled):**
|
||||
|
||||
- `fixture-architecture.md`
|
||||
- `network-first.md`
|
||||
|
||||
**Healing (if auto-heal enabled):**
|
||||
|
||||
- `test-healing-patterns.md`
|
||||
- `selector-resilience.md`
|
||||
- `timing-debugging.md`
|
||||
|
||||
**Playwright CLI (if tea_browser_automation is "cli" or "auto"):**
|
||||
|
||||
- `playwright-cli.md`
|
||||
|
||||
**MCP Patterns (if tea_browser_automation is "mcp" or "auto"):**
|
||||
|
||||
- (existing MCP-related fragments, if any are added in future)
|
||||
|
||||
---
|
||||
|
||||
## 5. Confirm Inputs
|
||||
|
||||
Summarize loaded artifacts, framework, and knowledge fragments, then proceed.
|
||||
|
||||
---
|
||||
|
||||
## 6. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-01-preflight-and-context']
|
||||
lastStep: 'step-01-preflight-and-context'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-01-preflight-and-context'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-01-preflight-and-context'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section.
|
||||
|
||||
Load next step: `{nextStepFile}`
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Step completed in full with required outputs
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Skipped sequence steps or missing outputs
|
||||
**Master Rule:** Skipping steps is FORBIDDEN.
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
name: 'step-01b-resume'
|
||||
description: 'Resume interrupted workflow from last completed step'
|
||||
outputFile: '{test_artifacts}/automation-summary.md'
|
||||
---
|
||||
|
||||
# Step 1b: Resume Workflow
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: Output document with progress frontmatter
|
||||
- Focus: Load progress and route to next step
|
||||
- Limits: Do not re-execute completed steps
|
||||
- Dependencies: Output document must exist from a previous run
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly.
|
||||
|
||||
### 1. Load Output Document
|
||||
|
||||
Read `{outputFile}` and parse YAML frontmatter for:
|
||||
|
||||
- `stepsCompleted` — array of completed step names
|
||||
- `lastStep` — last completed step name
|
||||
- `lastSaved` — timestamp of last save
|
||||
|
||||
**If `{outputFile}` does not exist**, display:
|
||||
|
||||
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
|
||||
|
||||
**THEN:** Halt. Do not proceed.
|
||||
|
||||
---
|
||||
|
||||
### 2. Display Progress Dashboard
|
||||
|
||||
Display progress with ✅/⬜ indicators:
|
||||
|
||||
1. ✅/⬜ Preflight & Context (step-01-preflight-and-context)
|
||||
2. ✅/⬜ Identify Targets (step-02-identify-targets)
|
||||
3. ✅/⬜ Generate Tests + Aggregate (step-03c-aggregate)
|
||||
4. ✅/⬜ Validate & Summarize (step-04-validate-and-summarize)
|
||||
|
||||
---
|
||||
|
||||
### 3. Route to Next Step
|
||||
|
||||
Based on `lastStep`, load the next incomplete step:
|
||||
|
||||
- `'step-01-preflight-and-context'` → load `./step-02-identify-targets.md`
|
||||
- `'step-02-identify-targets'` → load `./step-03-generate-tests.md`
|
||||
- `'step-03c-aggregate'` → load `./step-04-validate-and-summarize.md`
|
||||
- `'step-04-validate-and-summarize'` → **Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
|
||||
|
||||
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
|
||||
|
||||
**Otherwise**, load the identified step file, read completely, and execute.
|
||||
|
||||
The existing content in `{outputFile}` provides context from previously completed steps.
|
||||
|
||||
---
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Output document loaded and parsed correctly
|
||||
- Progress dashboard displayed accurately
|
||||
- Routed to correct next step
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Not loading output document
|
||||
- Incorrect progress display
|
||||
- Routing to wrong step
|
||||
|
||||
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
|
||||
@@ -0,0 +1,133 @@
|
||||
---
|
||||
name: 'step-02-identify-targets'
|
||||
description: 'Identify automation targets and create coverage plan'
|
||||
outputFile: '{test_artifacts}/automation-summary.md'
|
||||
nextStepFile: './step-03-generate-tests.md'
|
||||
---
|
||||
|
||||
# Step 2: Identify Automation Targets
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Determine what needs to be tested and select appropriate test levels and priorities.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
- 🚫 Avoid duplicate coverage across test levels
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, loaded artifacts, and knowledge fragments
|
||||
- Focus: this step's goal only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: prior steps' outputs (if any)
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
## 1. Determine Targets
|
||||
|
||||
**BMad-Integrated:**
|
||||
|
||||
- Map acceptance criteria to test scenarios
|
||||
- Check for existing ATDD outputs to avoid duplication
|
||||
- Expand coverage with edge cases and negative paths
|
||||
|
||||
**Standalone:**
|
||||
|
||||
- If specific target feature/files are provided, focus there
|
||||
- Otherwise auto-discover features in `{source_dir}`
|
||||
- Prioritize critical paths, integrations, and untested logic
|
||||
|
||||
**Browser Exploration (if `tea_browser_automation` is `cli` or `auto`):**
|
||||
|
||||
> **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip browser exploration and rely on code/doc analysis.
|
||||
|
||||
Use CLI to explore the application and identify testable pages/flows:
|
||||
|
||||
1. `playwright-cli -s=tea-automate open <target_url>`
|
||||
2. `playwright-cli -s=tea-automate snapshot` → capture page structure and element refs
|
||||
3. Analyze snapshot output to identify testable elements and flows
|
||||
4. `playwright-cli -s=tea-automate close`
|
||||
|
||||
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-automate close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
|
||||
|
||||
---
|
||||
|
||||
## 2. Choose Test Levels
|
||||
|
||||
Use `test-levels-framework.md` to select:
|
||||
|
||||
- **E2E** for critical user journeys
|
||||
- **API** for business logic and service contracts
|
||||
- **Component** for UI behavior
|
||||
- **Unit** for pure logic and edge cases
|
||||
|
||||
---
|
||||
|
||||
## 3. Assign Priorities
|
||||
|
||||
Use `test-priorities-matrix.md`:
|
||||
|
||||
- P0: Critical path + high risk
|
||||
- P1: Important flows + medium/high risk
|
||||
- P2: Secondary + edge cases
|
||||
- P3: Optional/rare scenarios
|
||||
|
||||
---
|
||||
|
||||
## 4. Coverage Plan
|
||||
|
||||
Produce a concise coverage plan:
|
||||
|
||||
- Targets by test level
|
||||
- Priority assignments
|
||||
- Justification for coverage scope (critical-paths/comprehensive/selective)
|
||||
|
||||
---
|
||||
|
||||
## 5. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-02-identify-targets']
|
||||
lastStep: 'step-02-identify-targets'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-02-identify-targets'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-02-identify-targets'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section.
|
||||
|
||||
Load next step: `{nextStepFile}`
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Step completed in full with required outputs
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Skipped sequence steps or missing outputs
|
||||
**Master Rule:** Skipping steps is FORBIDDEN.
|
||||
@@ -0,0 +1,199 @@
|
||||
---
|
||||
name: 'step-03-generate-tests'
|
||||
description: 'Orchestrate parallel test generation via subprocesses'
|
||||
nextStepFile: './step-03c-aggregate.md'
|
||||
---
|
||||
|
||||
# Step 3: Orchestrate Parallel Test Generation
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Launch parallel subprocesses to generate API and E2E tests simultaneously for maximum performance.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
- ✅ Launch TWO subprocesses in PARALLEL
|
||||
- ✅ Wait for BOTH subprocesses to complete
|
||||
- ❌ Do NOT generate tests sequentially (use subprocesses)
|
||||
- ❌ Do NOT proceed until both subprocesses finish
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Wait for subprocess outputs
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, coverage plan from Step 2, knowledge fragments
|
||||
- Focus: subprocess orchestration only
|
||||
- Limits: do not generate tests directly (delegate to subprocesses)
|
||||
- Dependencies: Step 2 outputs (coverage plan, target features)
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
### 1. Prepare Subprocess Inputs
|
||||
|
||||
**Generate unique timestamp** for temp file naming:
|
||||
|
||||
```javascript
|
||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
```
|
||||
|
||||
**Prepare input context for both subprocesses:**
|
||||
|
||||
```javascript
|
||||
const subprocessContext = {
|
||||
features: /* from Step 2 coverage plan */,
|
||||
knowledge_fragments_loaded: /* list of fragments */,
|
||||
config: {
|
||||
test_framework: config.test_framework,
|
||||
use_playwright_utils: config.tea_use_playwright_utils,
|
||||
browser_automation: config.tea_browser_automation // "auto" | "cli" | "mcp" | "none"
|
||||
},
|
||||
timestamp: timestamp
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Launch Subprocess A: API Test Generation
|
||||
|
||||
**Launch subprocess in parallel:**
|
||||
|
||||
- **Subprocess File:** `./step-03a-subprocess-api.md`
|
||||
- **Output File:** `/tmp/tea-automate-api-tests-${timestamp}.json`
|
||||
- **Context:** Pass `subprocessContext`
|
||||
- **Execution:** PARALLEL (non-blocking)
|
||||
|
||||
**System Action:**
|
||||
|
||||
```
|
||||
🚀 Launching Subprocess A: API Test Generation
|
||||
📝 Output: /tmp/tea-automate-api-tests-${timestamp}.json
|
||||
⏳ Status: Running in parallel...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Launch Subprocess B: E2E Test Generation
|
||||
|
||||
**Launch subprocess in parallel:**
|
||||
|
||||
- **Subprocess File:** `./step-03b-subprocess-e2e.md`
|
||||
- **Output File:** `/tmp/tea-automate-e2e-tests-${timestamp}.json`
|
||||
- **Context:** Pass `subprocessContext`
|
||||
- **Execution:** PARALLEL (non-blocking)
|
||||
|
||||
**System Action:**
|
||||
|
||||
```
|
||||
🚀 Launching Subprocess B: E2E Test Generation
|
||||
📝 Output: /tmp/tea-automate-e2e-tests-${timestamp}.json
|
||||
⏳ Status: Running in parallel...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. Wait for Both Subprocesses to Complete
|
||||
|
||||
**Monitor subprocess execution:**
|
||||
|
||||
```
|
||||
⏳ Waiting for subprocesses to complete...
|
||||
├── Subprocess A (API): Running... ⟳
|
||||
└── Subprocess B (E2E): Running... ⟳
|
||||
|
||||
[... time passes ...]
|
||||
|
||||
├── Subprocess A (API): Complete ✅
|
||||
└── Subprocess B (E2E): Complete ✅
|
||||
|
||||
✅ All subprocesses completed successfully!
|
||||
```
|
||||
|
||||
**Verify both outputs exist:**
|
||||
|
||||
```javascript
|
||||
const apiOutputExists = fs.existsSync(`/tmp/tea-automate-api-tests-${timestamp}.json`);
|
||||
const e2eOutputExists = fs.existsSync(`/tmp/tea-automate-e2e-tests-${timestamp}.json`);
|
||||
|
||||
if (!apiOutputExists || !e2eOutputExists) {
|
||||
throw new Error('One or both subprocess outputs missing!');
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Performance Report
|
||||
|
||||
**Display performance metrics:**
|
||||
|
||||
```
|
||||
🚀 Performance Report:
|
||||
- Execution Mode: PARALLEL (2 subprocesses)
|
||||
- API Test Generation: ~X minutes
|
||||
- E2E Test Generation: ~Y minutes
|
||||
- Total Elapsed: ~max(X, Y) minutes
|
||||
- Sequential Would Take: ~(X + Y) minutes
|
||||
- Performance Gain: ~50% faster!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Proceed to Aggregation
|
||||
|
||||
**Load aggregation step:**
|
||||
Load next step: `{nextStepFile}`
|
||||
|
||||
The aggregation step (3C) will:
|
||||
|
||||
- Read both subprocess outputs
|
||||
- Write all test files to disk
|
||||
- Generate shared fixtures and helpers
|
||||
- Calculate summary statistics
|
||||
|
||||
---
|
||||
|
||||
## EXIT CONDITION
|
||||
|
||||
Proceed to Step 3C (Aggregation) when:
|
||||
|
||||
- ✅ Subprocess A (API tests) completed successfully
|
||||
- ✅ Subprocess B (E2E tests) completed successfully
|
||||
- ✅ Both output files exist and are valid JSON
|
||||
- ✅ Performance metrics displayed
|
||||
|
||||
**Do NOT proceed if:**
|
||||
|
||||
- ❌ One or both subprocesses failed
|
||||
- ❌ Output files missing or corrupted
|
||||
- ❌ Timeout occurred (subprocesses took too long)
|
||||
|
||||
---
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Both subprocesses launched successfully
|
||||
- Both subprocesses completed without errors
|
||||
- Output files generated and valid
|
||||
- Parallel execution achieved ~50% performance gain
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Failed to launch subprocesses
|
||||
- One or both subprocesses failed
|
||||
- Output files missing or invalid
|
||||
- Attempted sequential generation instead of parallel
|
||||
|
||||
**Master Rule:** Parallel subprocess execution is MANDATORY for performance.
|
||||
@@ -0,0 +1,183 @@
|
||||
---
|
||||
name: 'step-03a-subprocess-api'
|
||||
description: 'Subprocess: Generate API tests only'
|
||||
subprocess: true
|
||||
outputFile: '/tmp/tea-automate-api-tests-{{timestamp}}.json'
|
||||
---
|
||||
|
||||
# Subprocess 3A: Generate API Tests
|
||||
|
||||
## SUBPROCESS CONTEXT
|
||||
|
||||
This is an **isolated subprocess** running in parallel with E2E test generation.
|
||||
|
||||
**What you have from parent workflow:**
|
||||
|
||||
- Target features/components identified in Step 2
|
||||
- Knowledge fragments loaded: api-request, data-factories, api-testing-patterns
|
||||
- Config: test framework, Playwright Utils enabled/disabled
|
||||
- Coverage plan: which API endpoints need testing
|
||||
|
||||
**Your task:** Generate API tests ONLY (not E2E, not fixtures, not other test types).
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read this entire subprocess file before acting
|
||||
- ✅ Generate API tests ONLY
|
||||
- ✅ Output structured JSON to temp file
|
||||
- ✅ Follow knowledge fragment patterns
|
||||
- ❌ Do NOT generate E2E tests (that's subprocess 3B)
|
||||
- ❌ Do NOT run tests (that's step 4)
|
||||
- ❌ Do NOT generate fixtures yet (that's step 3C aggregation)
|
||||
|
||||
---
|
||||
|
||||
## SUBPROCESS TASK
|
||||
|
||||
### 1. Identify API Endpoints
|
||||
|
||||
From the coverage plan (Step 2 output), identify:
|
||||
|
||||
- Which API endpoints need test coverage
|
||||
- Expected request/response formats
|
||||
- Authentication requirements
|
||||
- Error scenarios to test
|
||||
|
||||
### 2. Generate API Test Files
|
||||
|
||||
For each API endpoint, create test file in `tests/api/[feature].spec.ts`:
|
||||
|
||||
**Test Structure:**
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
// If Playwright Utils enabled:
|
||||
// import { apiRequest } from '@playwright-utils/api';
|
||||
|
||||
test.describe('[Feature] API Tests', () => {
|
||||
test('[P0] should handle successful [operation]', async ({ request }) => {
|
||||
// Use apiRequest helper if Playwright Utils enabled
|
||||
// Otherwise use standard request fixture
|
||||
const response = await request.post('/api/endpoint', {
|
||||
data: {
|
||||
/* test data */
|
||||
},
|
||||
});
|
||||
|
||||
expect(response.status()).toBe(200);
|
||||
expect(await response.json()).toMatchObject({
|
||||
/* expected */
|
||||
});
|
||||
});
|
||||
|
||||
test('[P1] should handle [error scenario]', async ({ request }) => {
|
||||
// Test error handling
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
|
||||
- ✅ Use `apiRequest()` helper if Playwright Utils enabled (from api-request fragment)
|
||||
- ✅ Use data factories for test data (from data-factories fragment)
|
||||
- ✅ Follow API testing patterns (from api-testing-patterns fragment)
|
||||
- ✅ Include priority tags [P0], [P1], [P2], [P3]
|
||||
- ✅ Test both happy path and error scenarios
|
||||
- ✅ Use proper TypeScript types
|
||||
- ✅ Deterministic assertions (no timing dependencies)
|
||||
|
||||
### 3. Track Fixture Needs
|
||||
|
||||
Identify fixtures needed for API tests:
|
||||
|
||||
- Authentication fixtures (auth tokens, API keys)
|
||||
- Data factories (user data, product data, etc.)
|
||||
- API client configurations
|
||||
|
||||
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
|
||||
|
||||
---
|
||||
|
||||
## OUTPUT FORMAT
|
||||
|
||||
Write JSON to temp file: `/tmp/tea-automate-api-tests-{{timestamp}}.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"subprocess": "api-tests",
|
||||
"tests": [
|
||||
{
|
||||
"file": "tests/api/auth.spec.ts",
|
||||
"content": "[full TypeScript test file content]",
|
||||
"description": "API tests for authentication endpoints",
|
||||
"priority_coverage": {
|
||||
"P0": 3,
|
||||
"P1": 2,
|
||||
"P2": 1,
|
||||
"P3": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"file": "tests/api/checkout.spec.ts",
|
||||
"content": "[full TypeScript test file content]",
|
||||
"description": "API tests for checkout endpoints",
|
||||
"priority_coverage": {
|
||||
"P0": 2,
|
||||
"P1": 3,
|
||||
"P2": 1,
|
||||
"P3": 0
|
||||
}
|
||||
}
|
||||
],
|
||||
"fixture_needs": ["authToken", "userDataFactory", "productDataFactory"],
|
||||
"knowledge_fragments_used": ["api-request", "data-factories", "api-testing-patterns"],
|
||||
"test_count": 12,
|
||||
"summary": "Generated 12 API test cases covering 3 features"
|
||||
}
|
||||
```
|
||||
|
||||
**On Error:**
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"subprocess": "api-tests",
|
||||
"error": "Error message describing what went wrong",
|
||||
"partial_output": {
|
||||
/* any tests generated before error */
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## EXIT CONDITION
|
||||
|
||||
Subprocess completes when:
|
||||
|
||||
- ✅ All API endpoints have test files generated
|
||||
- ✅ All tests follow knowledge fragment patterns
|
||||
- ✅ JSON output written to temp file
|
||||
- ✅ Fixture needs tracked
|
||||
|
||||
**Subprocess terminates here.** Parent workflow will read output and proceed to aggregation.
|
||||
|
||||
---
|
||||
|
||||
## 🚨 SUBPROCESS SUCCESS METRICS
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- All API tests generated following patterns
|
||||
- JSON output valid and complete
|
||||
- No E2E/component/unit tests included (out of scope)
|
||||
|
||||
### ❌ FAILURE:
|
||||
|
||||
- Generated tests other than API tests
|
||||
- Did not follow knowledge fragment patterns
|
||||
- Invalid or missing JSON output
|
||||
- Ran tests (not subprocess responsibility)
|
||||
@@ -0,0 +1,213 @@
|
||||
---
|
||||
name: 'step-03b-subprocess-e2e'
|
||||
description: 'Subprocess: Generate E2E tests only'
|
||||
subprocess: true
|
||||
outputFile: '/tmp/tea-automate-e2e-tests-{{timestamp}}.json'
|
||||
---
|
||||
|
||||
# Subprocess 3B: Generate E2E Tests
|
||||
|
||||
## SUBPROCESS CONTEXT
|
||||
|
||||
This is an **isolated subprocess** running in parallel with API test generation.
|
||||
|
||||
**What you have from parent workflow:**
|
||||
|
||||
- Target features/user journeys identified in Step 2
|
||||
- Knowledge fragments loaded: fixture-architecture, network-first, selector-resilience
|
||||
- Config: test framework, Playwright Utils enabled/disabled
|
||||
- Coverage plan: which user journeys need E2E testing
|
||||
|
||||
**Your task:** Generate E2E tests ONLY (not API, not fixtures, not other test types).
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read this entire subprocess file before acting
|
||||
- ✅ Generate E2E tests ONLY
|
||||
- ✅ Output structured JSON to temp file
|
||||
- ✅ Follow knowledge fragment patterns
|
||||
- ❌ Do NOT generate API tests (that's subprocess 3A)
|
||||
- ❌ Do NOT run tests (that's step 4)
|
||||
- ❌ Do NOT generate fixtures yet (that's step 3C aggregation)
|
||||
|
||||
---
|
||||
|
||||
## SUBPROCESS TASK
|
||||
|
||||
### 1. Identify User Journeys
|
||||
|
||||
From the coverage plan (Step 2 output), identify:
|
||||
|
||||
- Which user journeys need E2E coverage
|
||||
- Critical user paths (authentication, checkout, profile, etc.)
|
||||
- UI interactions required
|
||||
- Expected visual states
|
||||
|
||||
### 2. Browser Interaction (Selector Verification)
|
||||
|
||||
**Automation mode:** `config.tea_browser_automation`
|
||||
|
||||
If `auto` (fall back to MCP if CLI unavailable; if neither available, generate from best practices):
|
||||
|
||||
- Open the target page first, then verify selectors with a snapshot:
|
||||
`playwright-cli -s=tea-automate-{{timestamp}} open <target_url>`
|
||||
`playwright-cli -s=tea-automate-{{timestamp}} snapshot` → map refs to Playwright locators
|
||||
- ref `{role: "button", name: "Submit"}` → `page.getByRole('button', { name: 'Submit' })`
|
||||
- ref `{role: "textbox", name: "Email"}` → `page.getByRole('textbox', { name: 'Email' })`
|
||||
- `playwright-cli -s=tea-automate-{{timestamp}} close` when done
|
||||
|
||||
If `cli` (CLI only — do NOT fall back to MCP; generate from best practices if CLI unavailable):
|
||||
|
||||
- Open the target page first, then verify selectors with a snapshot:
|
||||
`playwright-cli -s=tea-automate-{{timestamp}} open <target_url>`
|
||||
`playwright-cli -s=tea-automate-{{timestamp}} snapshot` → map refs to Playwright locators
|
||||
- ref `{role: "button", name: "Submit"}` → `page.getByRole('button', { name: 'Submit' })`
|
||||
- ref `{role: "textbox", name: "Email"}` → `page.getByRole('textbox', { name: 'Email' })`
|
||||
- `playwright-cli -s=tea-automate-{{timestamp}} close` when done
|
||||
|
||||
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-automate-{{timestamp}} close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
|
||||
|
||||
If `mcp`:
|
||||
|
||||
- Use MCP tools for selector verification (current behavior)
|
||||
|
||||
If `none`:
|
||||
|
||||
- Generate selectors from best practices without browser verification
|
||||
|
||||
### 3. Generate E2E Test Files
|
||||
|
||||
For each user journey, create test file in `tests/e2e/[feature].spec.ts`:
|
||||
|
||||
**Test Structure:**
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('[Feature] E2E User Journey', () => {
|
||||
test('[P0] should complete [user journey]', async ({ page }) => {
|
||||
// Navigate to starting point
|
||||
await page.goto('/feature');
|
||||
|
||||
// Interact with UI
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
|
||||
// Assert expected state
|
||||
await expect(page.getByText('Success')).toBeVisible();
|
||||
});
|
||||
|
||||
test('[P1] should handle [edge case]', async ({ page }) => {
|
||||
// Test edge case scenario
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
|
||||
- ✅ Follow fixture architecture patterns (from fixture-architecture fragment)
|
||||
- ✅ Use network-first patterns: intercept before navigate (from network-first fragment)
|
||||
- ✅ Use resilient selectors: getByRole, getByText, getByLabel (from selector-resilience fragment)
|
||||
- ✅ Include priority tags [P0], [P1], [P2], [P3]
|
||||
- ✅ Test complete user journeys (not isolated clicks)
|
||||
- ✅ Use proper TypeScript types
|
||||
- ✅ Deterministic waits (no hard sleeps, use expect().toBeVisible())
|
||||
|
||||
### 4. Track Fixture Needs
|
||||
|
||||
Identify fixtures needed for E2E tests:
|
||||
|
||||
- Page object models (if complex)
|
||||
- Authentication fixtures (logged-in user state)
|
||||
- Network mocks/intercepts
|
||||
- Test data fixtures
|
||||
|
||||
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
|
||||
|
||||
---
|
||||
|
||||
## OUTPUT FORMAT
|
||||
|
||||
Write JSON to temp file: `/tmp/tea-automate-e2e-tests-{{timestamp}}.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"subprocess": "e2e-tests",
|
||||
"tests": [
|
||||
{
|
||||
"file": "tests/e2e/authentication.spec.ts",
|
||||
"content": "[full TypeScript test file content]",
|
||||
"description": "E2E tests for user authentication journey",
|
||||
"priority_coverage": {
|
||||
"P0": 2,
|
||||
"P1": 3,
|
||||
"P2": 2,
|
||||
"P3": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"file": "tests/e2e/checkout.spec.ts",
|
||||
"content": "[full TypeScript test file content]",
|
||||
"description": "E2E tests for checkout journey",
|
||||
"priority_coverage": {
|
||||
"P0": 3,
|
||||
"P1": 2,
|
||||
"P2": 1,
|
||||
"P3": 0
|
||||
}
|
||||
}
|
||||
],
|
||||
"fixture_needs": ["authenticatedUserFixture", "paymentMockFixture", "checkoutDataFixture"],
|
||||
"knowledge_fragments_used": ["fixture-architecture", "network-first", "selector-resilience"],
|
||||
"test_count": 15,
|
||||
"summary": "Generated 15 E2E test cases covering 5 user journeys"
|
||||
}
|
||||
```
|
||||
|
||||
**On Error:**
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"subprocess": "e2e-tests",
|
||||
"error": "Error message describing what went wrong",
|
||||
"partial_output": {
|
||||
/* any tests generated before error */
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## EXIT CONDITION
|
||||
|
||||
Subprocess completes when:
|
||||
|
||||
- ✅ All user journeys have E2E test files generated
|
||||
- ✅ All tests follow knowledge fragment patterns
|
||||
- ✅ JSON output written to temp file
|
||||
- ✅ Fixture needs tracked
|
||||
|
||||
**Subprocess terminates here.** Parent workflow will read output and proceed to aggregation.
|
||||
|
||||
---
|
||||
|
||||
## 🚨 SUBPROCESS SUCCESS METRICS
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- All E2E tests generated following patterns
|
||||
- JSON output valid and complete
|
||||
- No API/component/unit tests included (out of scope)
|
||||
- Resilient selectors used (getByRole, getByText)
|
||||
- Network-first patterns applied (intercept before navigate)
|
||||
|
||||
### ❌ FAILURE:
|
||||
|
||||
- Generated tests other than E2E tests
|
||||
- Did not follow knowledge fragment patterns
|
||||
- Invalid or missing JSON output
|
||||
- Ran tests (not subprocess responsibility)
|
||||
- Used brittle selectors (CSS classes, XPath)
|
||||
@@ -0,0 +1,325 @@
|
||||
---
|
||||
name: 'step-03c-aggregate'
|
||||
description: 'Aggregate subprocess outputs and complete test infrastructure'
|
||||
outputFile: '{test_artifacts}/automation-summary.md'
|
||||
nextStepFile: './step-04-validate-and-summarize.md'
|
||||
---
|
||||
|
||||
# Step 3C: Aggregate Test Generation Results
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Read outputs from parallel subprocesses (API + E2E test generation), aggregate results, and create supporting infrastructure (fixtures, helpers).
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
- ✅ Read subprocess outputs from temp files
|
||||
- ✅ Generate shared fixtures based on fixture needs from both subprocesses
|
||||
- ✅ Write all generated test files to disk
|
||||
- ❌ Do NOT regenerate tests (use subprocess outputs)
|
||||
- ❌ Do NOT run tests yet (that's step 4)
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, subprocess outputs from temp files
|
||||
- Focus: aggregation and fixture generation only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: Step 3A and 3B subprocess outputs
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
### 1. Read Subprocess Outputs
|
||||
|
||||
**Read API test subprocess output:**
|
||||
|
||||
```javascript
|
||||
const apiTestsPath = '/tmp/tea-automate-api-tests-{{timestamp}}.json';
|
||||
const apiTestsOutput = JSON.parse(fs.readFileSync(apiTestsPath, 'utf8'));
|
||||
```
|
||||
|
||||
**Read E2E test subprocess output:**
|
||||
|
||||
```javascript
|
||||
const e2eTestsPath = '/tmp/tea-automate-e2e-tests-{{timestamp}}.json';
|
||||
const e2eTestsOutput = JSON.parse(fs.readFileSync(e2eTestsPath, 'utf8'));
|
||||
```
|
||||
|
||||
**Verify both subprocesses succeeded:**
|
||||
|
||||
- Check `apiTestsOutput.success === true`
|
||||
- Check `e2eTestsOutput.success === true`
|
||||
- If either failed, report error and stop (don't proceed)
|
||||
|
||||
---
|
||||
|
||||
### 2. Write All Test Files to Disk
|
||||
|
||||
**Write API test files:**
|
||||
|
||||
```javascript
|
||||
apiTestsOutput.tests.forEach((test) => {
|
||||
fs.writeFileSync(test.file, test.content, 'utf8');
|
||||
console.log(`✅ Created: ${test.file}`);
|
||||
});
|
||||
```
|
||||
|
||||
**Write E2E test files:**
|
||||
|
||||
```javascript
|
||||
e2eTestsOutput.tests.forEach((test) => {
|
||||
fs.writeFileSync(test.file, test.content, 'utf8');
|
||||
console.log(`✅ Created: ${test.file}`);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Aggregate Fixture Needs
|
||||
|
||||
**Collect all fixture needs from both subprocesses:**
|
||||
|
||||
```javascript
|
||||
const allFixtureNeeds = [...apiTestsOutput.fixture_needs, ...e2eTestsOutput.fixture_needs];
|
||||
|
||||
// Remove duplicates
|
||||
const uniqueFixtures = [...new Set(allFixtureNeeds)];
|
||||
```
|
||||
|
||||
**Categorize fixtures:**
|
||||
|
||||
- **Authentication fixtures:** authToken, authenticatedUserFixture, etc.
|
||||
- **Data factories:** userDataFactory, productDataFactory, etc.
|
||||
- **Network mocks:** paymentMockFixture, apiResponseMocks, etc.
|
||||
- **Test helpers:** wait/retry/assertion helpers
|
||||
|
||||
---
|
||||
|
||||
### 4. Generate Fixture Infrastructure
|
||||
|
||||
**Create or update fixture files based on needs:**
|
||||
|
||||
**A) Authentication Fixtures** (`tests/fixtures/auth.ts`):
|
||||
|
||||
```typescript
|
||||
import { test as base } from '@playwright/test';
|
||||
|
||||
export const test = base.extend({
|
||||
authenticatedUser: async ({ page }, use) => {
|
||||
// Login logic
|
||||
await page.goto('/login');
|
||||
await page.fill('[name="email"]', 'test@example.com');
|
||||
await page.fill('[name="password"]', 'password');
|
||||
await page.click('button[type="submit"]');
|
||||
await page.waitForURL('/dashboard');
|
||||
|
||||
await use(page);
|
||||
},
|
||||
|
||||
authToken: async ({ request }, use) => {
|
||||
// Get auth token for API tests
|
||||
const response = await request.post('/api/auth/login', {
|
||||
data: { email: 'test@example.com', password: 'password' },
|
||||
});
|
||||
const { token } = await response.json();
|
||||
|
||||
await use(token);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
**B) Data Factories** (`tests/fixtures/data-factories.ts`):
|
||||
|
||||
```typescript
|
||||
import { faker } from '@faker-js/faker';
|
||||
|
||||
export const createUserData = (overrides = {}) => ({
|
||||
name: faker.person.fullName(),
|
||||
email: faker.internet.email(),
|
||||
...overrides,
|
||||
});
|
||||
|
||||
export const createProductData = (overrides = {}) => ({
|
||||
name: faker.commerce.productName(),
|
||||
price: faker.number.int({ min: 10, max: 1000 }),
|
||||
...overrides,
|
||||
});
|
||||
```
|
||||
|
||||
**C) Network Mocks** (`tests/fixtures/network-mocks.ts`):
|
||||
|
||||
```typescript
|
||||
import { Page } from '@playwright/test';
|
||||
|
||||
export const mockPaymentSuccess = async (page: Page) => {
|
||||
await page.route('/api/payment/**', (route) => {
|
||||
route.fulfill({
|
||||
status: 200,
|
||||
body: JSON.stringify({ success: true, transactionId: '12345' }),
|
||||
});
|
||||
});
|
||||
};
|
||||
```
|
||||
|
||||
**D) Helper Utilities** (`tests/fixtures/helpers.ts`):
|
||||
|
||||
```typescript
|
||||
import { expect, Page } from '@playwright/test';
|
||||
|
||||
export const waitForApiResponse = async (page: Page, urlPattern: string) => {
|
||||
return page.waitForResponse((response) => response.url().includes(urlPattern) && response.ok());
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Calculate Summary Statistics
|
||||
|
||||
**Aggregate test counts:**
|
||||
|
||||
```javascript
|
||||
const summary = {
|
||||
total_tests: apiTestsOutput.test_count + e2eTestsOutput.test_count,
|
||||
api_tests: apiTestsOutput.test_count,
|
||||
e2e_tests: e2eTestsOutput.test_count,
|
||||
fixtures_created: uniqueFixtures.length,
|
||||
api_test_files: apiTestsOutput.tests.length,
|
||||
e2e_test_files: e2eTestsOutput.tests.length,
|
||||
priority_coverage: {
|
||||
P0: /* sum P0 tests from both */,
|
||||
P1: /* sum P1 tests from both */,
|
||||
P2: /* sum P2 tests from both */,
|
||||
P3: /* sum P3 tests from both */
|
||||
},
|
||||
knowledge_fragments_used: [
|
||||
...apiTestsOutput.knowledge_fragments_used,
|
||||
...e2eTestsOutput.knowledge_fragments_used
|
||||
],
|
||||
subprocess_execution: 'PARALLEL (API + E2E)',
|
||||
performance_gain: '~50% faster than sequential'
|
||||
};
|
||||
```
|
||||
|
||||
**Store summary for Step 4:**
|
||||
Save summary to temp file for validation step:
|
||||
|
||||
```javascript
|
||||
fs.writeFileSync('/tmp/tea-automate-summary-{{timestamp}}.json', JSON.stringify(summary, null, 2), 'utf8');
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Optional Cleanup
|
||||
|
||||
**Clean up subprocess temp files** (optional - can keep for debugging):
|
||||
|
||||
```javascript
|
||||
fs.unlinkSync(apiTestsPath);
|
||||
fs.unlinkSync(e2eTestsPath);
|
||||
console.log('✅ Subprocess temp files cleaned up');
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## OUTPUT SUMMARY
|
||||
|
||||
Display to user:
|
||||
|
||||
```
|
||||
✅ Test Generation Complete (Parallel Execution)
|
||||
|
||||
📊 Summary:
|
||||
- Total Tests: {total_tests}
|
||||
- API Tests: {api_tests} ({api_test_files} files)
|
||||
- E2E Tests: {e2e_tests} ({e2e_test_files} files)
|
||||
- Fixtures Created: {fixtures_created}
|
||||
- Priority Coverage:
|
||||
- P0 (Critical): {P0} tests
|
||||
- P1 (High): {P1} tests
|
||||
- P2 (Medium): {P2} tests
|
||||
- P3 (Low): {P3} tests
|
||||
|
||||
🚀 Performance: Parallel execution ~50% faster than sequential
|
||||
|
||||
📂 Generated Files:
|
||||
- tests/api/[feature].spec.ts
|
||||
- tests/e2e/[feature].spec.ts
|
||||
- tests/fixtures/auth.ts
|
||||
- tests/fixtures/data-factories.ts
|
||||
- tests/fixtures/network-mocks.ts
|
||||
- tests/fixtures/helpers.ts
|
||||
|
||||
✅ Ready for validation (Step 4)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## EXIT CONDITION
|
||||
|
||||
Proceed to Step 4 when:
|
||||
|
||||
- ✅ All test files written to disk (API + E2E)
|
||||
- ✅ All fixtures and helpers created
|
||||
- ✅ Summary statistics calculated and saved
|
||||
- ✅ Output displayed to user
|
||||
|
||||
---
|
||||
|
||||
### 7. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-03c-aggregate']
|
||||
lastStep: 'step-03c-aggregate'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-03c-aggregate'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-03c-aggregate'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section.
|
||||
|
||||
Load next step: `{nextStepFile}`
|
||||
|
||||
---
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Both subprocesses succeeded
|
||||
- All test files written to disk
|
||||
- Fixtures generated based on subprocess needs
|
||||
- Summary complete and accurate
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- One or both subprocesses failed
|
||||
- Test files not written to disk
|
||||
- Fixtures missing or incomplete
|
||||
- Summary missing or inaccurate
|
||||
|
||||
**Master Rule:** Do NOT proceed to Step 4 if aggregation incomplete.
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
name: 'step-04-validate-and-summarize'
|
||||
description: 'Validate outputs and produce automation summary'
|
||||
outputFile: '{test_artifacts}/automation-summary.md'
|
||||
---
|
||||
|
||||
# Step 4: Validate & Summarize
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Validate generated outputs and produce a concise automation summary.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
- ✅ Validate against the checklist before completion
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, loaded artifacts, and knowledge fragments
|
||||
- Focus: this step's goal only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: prior steps' outputs (if any)
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
## 1. Validate
|
||||
|
||||
Use `checklist.md` to validate:
|
||||
|
||||
- Framework readiness
|
||||
- Coverage mapping
|
||||
- Test quality and structure
|
||||
- Fixtures, factories, helpers
|
||||
- [ ] CLI sessions cleaned up (no orphaned browsers)
|
||||
- [ ] Temp artifacts stored in `{test_artifacts}/` not random locations
|
||||
|
||||
Fix gaps before proceeding.
|
||||
|
||||
---
|
||||
|
||||
## 2. Summary Output
|
||||
|
||||
Write `{outputFile}` including:
|
||||
|
||||
- Coverage plan by test level and priority
|
||||
- Files created/updated
|
||||
- Key assumptions and risks
|
||||
- Next recommended workflow (e.g., `test-review` or `trace`)
|
||||
|
||||
---
|
||||
|
||||
## 3. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-04-validate-and-summarize']
|
||||
lastStep: 'step-04-validate-and-summarize'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-04-validate-and-summarize'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-04-validate-and-summarize'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section.
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Step completed in full with required outputs
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Skipped sequence steps or missing outputs
|
||||
**Master Rule:** Skipping steps is FORBIDDEN.
|
||||
Reference in new issue
Block a user