mirror of
https://github.com/R0m1k3/xtremflow.git
synced 2026-10-24 15:38:04 +02:00
feat: security overhaul (proxy ssrf fix, streaming sanitization, docker optimization)
This commit is contained in:
Michael committed
1 parent
6d87ada2b8
commit
8df9a37baa
810 files changed
+131090
-151
No files matched your search
@@ -0,0 +1,87 @@
|
||||
# Test Review - Validation Checklist
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [ ] Test suite accessible
|
||||
- [ ] Test results available
|
||||
- [ ] Feature list known
|
||||
- [ ] Access to CI/CD pipeline
|
||||
|
||||
---
|
||||
|
||||
## Metrics Collection
|
||||
|
||||
- [ ] Test counts by type gathered
|
||||
- [ ] Pass rates calculated
|
||||
- [ ] Average durations measured
|
||||
- [ ] Flaky tests identified
|
||||
- [ ] Slow tests identified
|
||||
- [ ] Disabled tests listed
|
||||
|
||||
---
|
||||
|
||||
## Quality Assessment
|
||||
|
||||
- [ ] Determinism evaluated
|
||||
- [ ] Isolation checked
|
||||
- [ ] Speed benchmarked
|
||||
- [ ] Readability assessed
|
||||
- [ ] Anti-patterns documented
|
||||
|
||||
---
|
||||
|
||||
## Coverage Analysis
|
||||
|
||||
- [ ] All features listed
|
||||
- [ ] Test coverage mapped
|
||||
- [ ] P0/P1 coverage verified
|
||||
- [ ] Critical gaps identified
|
||||
- [ ] Gap priorities assigned
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure Review
|
||||
|
||||
- [ ] CI integration verified
|
||||
- [ ] Results visibility confirmed
|
||||
- [ ] Failure blocking assessed
|
||||
- [ ] Fixture quality evaluated
|
||||
- [ ] Maintenance burden estimated
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
- [ ] Findings prioritized
|
||||
- [ ] Effort estimated
|
||||
- [ ] Immediate actions identified
|
||||
- [ ] Short-term improvements listed
|
||||
- [ ] Long-term strategy outlined
|
||||
|
||||
---
|
||||
|
||||
## Report
|
||||
|
||||
- [ ] Executive summary complete
|
||||
- [ ] Metrics section complete
|
||||
- [ ] Quality assessment documented
|
||||
- [ ] Coverage analysis included
|
||||
- [ ] Recommendations actionable
|
||||
- [ ] Appendices populated
|
||||
|
||||
---
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] All test types reviewed
|
||||
- [ ] Critical gaps identified
|
||||
- [ ] Actionable recommendations provided
|
||||
- [ ] Report delivered to stakeholders
|
||||
|
||||
---
|
||||
|
||||
**Completed by:** {name}
|
||||
**Date:** {date}
|
||||
**Tests Reviewed:** {count}
|
||||
@@ -0,0 +1,272 @@
|
||||
<!-- Powered by BMAD-CORE™ -->
|
||||
|
||||
# Test Review
|
||||
|
||||
**Workflow ID**: `_bmad/bmgd/gametest/test-review`
|
||||
**Version**: 1.0 (BMad v6)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Review existing test suite quality, identify coverage gaps, and recommend improvements. Regular test review prevents test rot and maintains test value over time.
|
||||
|
||||
**Knowledge Base Reference**: `knowledge/regression-testing.md`, `knowledge/test-priorities.md`
|
||||
|
||||
---
|
||||
|
||||
## Preflight Requirements
|
||||
|
||||
- ✅ Test suite exists (some tests to review)
|
||||
- ✅ Access to test execution results
|
||||
- ✅ Understanding of game features
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Gather Test Suite Metrics
|
||||
|
||||
### Actions
|
||||
|
||||
1. **Count Tests by Type**
|
||||
|
||||
| Type | Count | Pass Rate | Avg Duration |
|
||||
| -------------------- | ----- | --------- | ------------ |
|
||||
| Unit | | | |
|
||||
| Integration | | | |
|
||||
| Play Mode/Functional | | | |
|
||||
| Performance | | | |
|
||||
| **Total** | | | |
|
||||
|
||||
2. **Analyze Test Results**
|
||||
- Recent pass rate (last 10 runs)
|
||||
- Flaky tests (inconsistent results)
|
||||
- Slow tests (> 30s individual)
|
||||
- Disabled/skipped tests
|
||||
|
||||
3. **Map Coverage**
|
||||
- Features with tests
|
||||
- Features without tests
|
||||
- Critical paths covered
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Assess Test Quality
|
||||
|
||||
### Quality Criteria
|
||||
|
||||
For each test, evaluate:
|
||||
|
||||
| Criterion | Good | Bad |
|
||||
| ----------------- | -------------------------------- | ---------------------------- |
|
||||
| **Deterministic** | Same input = same result | Flaky, timing-dependent |
|
||||
| **Isolated** | No shared state | Tests affect each other |
|
||||
| **Fast** | < 5s (unit), < 30s (integration) | Minutes per test |
|
||||
| **Readable** | Clear intent, good names | Cryptic, no comments |
|
||||
| **Maintained** | Up-to-date, passing | Disabled, stale |
|
||||
| **Valuable** | Tests real behavior | Tests implementation details |
|
||||
|
||||
### Anti-Pattern Detection
|
||||
|
||||
Look for these common issues:
|
||||
|
||||
```
|
||||
❌ Hard-coded waits
|
||||
await Task.Delay(5000); // Bad
|
||||
await WaitUntil(() => condition); // Good
|
||||
|
||||
❌ Shared test state
|
||||
static bool wasSetup; // Dangerous
|
||||
[SetUp] void Setup() { /* fresh state */ } // Good
|
||||
|
||||
❌ Testing private implementation
|
||||
var result = obj.GetPrivateField(); // Bad
|
||||
var result = obj.PublicBehavior(); // Good
|
||||
|
||||
❌ Missing cleanup
|
||||
var go = Instantiate(prefab); // Leaks
|
||||
var go = Instantiate(prefab);
|
||||
AddCleanup(() => Destroy(go)); // Good
|
||||
|
||||
❌ Assertion-free tests
|
||||
void Test() { DoSomething(); } // What does it test?
|
||||
void Test() { DoSomething(); Assert.That(...); } // Clear
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Identify Coverage Gaps
|
||||
|
||||
### Critical Areas to Verify
|
||||
|
||||
| Area | P0 Coverage | P1 Coverage | Gap? |
|
||||
| ------------- | ----------- | ----------- | ---- |
|
||||
| Core Loop | | | |
|
||||
| Save/Load | | | |
|
||||
| Progression | | | |
|
||||
| Combat | | | |
|
||||
| UI/Menus | | | |
|
||||
| Multiplayer | | | |
|
||||
| Platform Cert | | | |
|
||||
|
||||
### Gap Identification Process
|
||||
|
||||
1. List all game features
|
||||
2. Check if each feature has tests
|
||||
3. Assess test depth (happy path only vs edge cases)
|
||||
4. Prioritize gaps by risk
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Review Test Infrastructure
|
||||
|
||||
### Framework Health
|
||||
|
||||
- [ ] Tests run in CI
|
||||
- [ ] Results are visible to team
|
||||
- [ ] Failures block deployments
|
||||
- [ ] Test data is versioned
|
||||
- [ ] Fixtures are reusable
|
||||
- [ ] Helpers reduce duplication
|
||||
|
||||
### Maintenance Burden
|
||||
|
||||
- How often do tests need updates?
|
||||
- Are updates proportional to code changes?
|
||||
- Do refactors break tests unnecessarily?
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Generate Recommendations
|
||||
|
||||
### Priority Matrix
|
||||
|
||||
| Finding | Severity | Effort | Recommendation |
|
||||
| --------- | -------------- | -------------- | -------------- |
|
||||
| {finding} | {High/Med/Low} | {High/Med/Low} | {action} |
|
||||
|
||||
### Common Recommendations
|
||||
|
||||
**For Flaky Tests**:
|
||||
|
||||
- Replace `Thread.Sleep` with explicit waits
|
||||
- Add proper synchronization
|
||||
- Isolate test state
|
||||
|
||||
**For Slow Tests**:
|
||||
|
||||
- Move to nightly builds
|
||||
- Optimize test setup
|
||||
- Mock expensive dependencies
|
||||
|
||||
**For Coverage Gaps**:
|
||||
|
||||
- Prioritize P0/P1 features
|
||||
- Add smoke tests first
|
||||
- Use test-design workflow
|
||||
|
||||
**For Maintenance Issues**:
|
||||
|
||||
- Refactor common patterns
|
||||
- Create test utilities
|
||||
- Improve documentation
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Generate Test Review Report
|
||||
|
||||
### Report Structure
|
||||
|
||||
```markdown
|
||||
# Test Review Report: {Project Name}
|
||||
|
||||
## Executive Summary
|
||||
|
||||
- Overall health: {Good/Needs Work/Critical}
|
||||
- Key findings: {3-5 bullet points}
|
||||
- Recommended actions: {prioritized list}
|
||||
|
||||
## Metrics
|
||||
|
||||
### Test Suite Statistics
|
||||
|
||||
[Tables from Step 1]
|
||||
|
||||
### Recent History
|
||||
|
||||
[Pass rates, trends]
|
||||
|
||||
## Quality Assessment
|
||||
|
||||
### Strengths
|
||||
|
||||
- {What's working well}
|
||||
|
||||
### Issues Found
|
||||
|
||||
| Issue | Severity | Tests Affected | Fix |
|
||||
| ----- | -------- | -------------- | --- |
|
||||
| | | | |
|
||||
|
||||
## Coverage Analysis
|
||||
|
||||
### Current Coverage
|
||||
|
||||
[Feature coverage table]
|
||||
|
||||
### Critical Gaps
|
||||
|
||||
[Prioritized list of missing coverage]
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate (This Sprint)
|
||||
|
||||
1. {Fix critical issues}
|
||||
|
||||
### Short-term (This Milestone)
|
||||
|
||||
1. {Address major gaps}
|
||||
|
||||
### Long-term (Ongoing)
|
||||
|
||||
1. {Improve infrastructure}
|
||||
|
||||
## Appendix
|
||||
|
||||
### Flaky Tests
|
||||
|
||||
[List with failure patterns]
|
||||
|
||||
### Slow Tests
|
||||
|
||||
[List with durations]
|
||||
|
||||
### Disabled Tests
|
||||
|
||||
[List with reasons]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Review Frequency
|
||||
|
||||
| Review Type | Frequency | Scope | Owner |
|
||||
| ----------- | --------- | ------------------------ | --------- |
|
||||
| Quick Check | Weekly | Pass rates, flaky tests | QA |
|
||||
| Full Review | Monthly | Coverage, quality | Tech Lead |
|
||||
| Deep Dive | Quarterly | Infrastructure, strategy | Team |
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. **Test Review Report** - Comprehensive analysis
|
||||
2. **Action Items** - Prioritized improvements
|
||||
3. **Coverage Matrix** - Visual gap identification
|
||||
4. **Technical Debt List** - Tests needing refactor
|
||||
|
||||
---
|
||||
|
||||
## Validation
|
||||
|
||||
Refer to `checklist.md` for validation criteria.
|
||||
@@ -0,0 +1,203 @@
|
||||
# Test Review Report: {PROJECT_NAME}
|
||||
|
||||
**Review Date**: {DATE}
|
||||
**Reviewer**: {REVIEWER}
|
||||
**Period Covered**: {START_DATE} to {END_DATE}
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
### Overall Health: {Good | Needs Work | Critical}
|
||||
|
||||
### Key Findings
|
||||
|
||||
1. {Finding 1}
|
||||
2. {Finding 2}
|
||||
3. {Finding 3}
|
||||
|
||||
### Recommended Actions
|
||||
|
||||
1. {High priority action}
|
||||
2. {Medium priority action}
|
||||
3. {Ongoing improvement}
|
||||
|
||||
---
|
||||
|
||||
## Test Suite Metrics
|
||||
|
||||
### Test Distribution
|
||||
|
||||
| Type | Count | % of Total |
|
||||
| -------------------- | ----- | ---------- |
|
||||
| Unit Tests | | |
|
||||
| Integration Tests | | |
|
||||
| Play Mode/Functional | | |
|
||||
| Performance Tests | | |
|
||||
| **Total** | | 100% |
|
||||
|
||||
### Execution Metrics
|
||||
|
||||
| Metric | Current | Previous | Trend |
|
||||
| -------------- | ------- | -------- | ----- |
|
||||
| Pass Rate | | | {↑↓→} |
|
||||
| Avg Duration | | | {↑↓→} |
|
||||
| Flaky Tests | | | {↑↓→} |
|
||||
| Disabled Tests | | | {↑↓→} |
|
||||
|
||||
### Recent Run History
|
||||
|
||||
| Date | Passed | Failed | Skipped | Duration |
|
||||
| ---- | ------ | ------ | ------- | -------- |
|
||||
| | | | | |
|
||||
| | | | | |
|
||||
| | | | | |
|
||||
|
||||
---
|
||||
|
||||
## Quality Assessment
|
||||
|
||||
### Strengths
|
||||
|
||||
- {What the test suite does well}
|
||||
- {Good patterns observed}
|
||||
- {Areas of strong coverage}
|
||||
|
||||
### Issues Found
|
||||
|
||||
| Issue | Severity | Count | Example | Recommended Fix |
|
||||
| ------------------ | -------- | ----- | ------- | --------------- |
|
||||
| Flaky tests | High | | | |
|
||||
| Slow tests | Medium | | | |
|
||||
| Missing cleanup | Medium | | | |
|
||||
| Hard-coded waits | Low | | | |
|
||||
| Unclear assertions | Low | | | |
|
||||
|
||||
### Anti-Patterns Detected
|
||||
|
||||
| Pattern | Occurrences | Impact | Fix Effort |
|
||||
| --------- | ----------- | ------ | ---------- |
|
||||
| {pattern} | | | |
|
||||
|
||||
---
|
||||
|
||||
## Coverage Analysis
|
||||
|
||||
### Feature Coverage Matrix
|
||||
|
||||
| Feature | P0 Tests | P1 Tests | P2 Tests | Gap? |
|
||||
| ------------- | -------- | -------- | -------- | ---- |
|
||||
| Core Loop | | | | |
|
||||
| Combat/Action | | | | |
|
||||
| Movement | | | | |
|
||||
| UI/Menus | | | | |
|
||||
| Save/Load | | | | |
|
||||
| Progression | | | | |
|
||||
| Multiplayer | | | | |
|
||||
| Audio | | | | |
|
||||
| Platform | | | | |
|
||||
|
||||
### Critical Gaps
|
||||
|
||||
| Gap | Risk | Impact | Priority to Fix |
|
||||
| ----------------------- | ------------ | ----------- | --------------- |
|
||||
| {feature without tests} | {risk level} | {if breaks} | P{0-3} |
|
||||
|
||||
### Coverage by Priority
|
||||
|
||||
```
|
||||
P0 Coverage: {X}% ████████░░
|
||||
P1 Coverage: {X}% ██████░░░░
|
||||
P2 Coverage: {X}% ████░░░░░░
|
||||
P3 Coverage: {X}% ██░░░░░░░░
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure Review
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
| Aspect | Status | Notes |
|
||||
| ----------------- | ------- | ----- |
|
||||
| Tests in CI | {✅/❌} | |
|
||||
| Results visible | {✅/❌} | |
|
||||
| Failures block | {✅/❌} | |
|
||||
| Nightly runs | {✅/❌} | |
|
||||
| Performance tests | {✅/❌} | |
|
||||
|
||||
### Test Infrastructure Quality
|
||||
|
||||
| Component | Quality | Notes |
|
||||
| -------------- | ---------------- | ----- |
|
||||
| Fixtures | {Good/Fair/Poor} | |
|
||||
| Helpers | {Good/Fair/Poor} | |
|
||||
| Data factories | {Good/Fair/Poor} | |
|
||||
| Documentation | {Good/Fair/Poor} | |
|
||||
|
||||
### Maintenance Burden
|
||||
|
||||
- Test update frequency: {high/medium/low}
|
||||
- Brittleness score: {high/medium/low}
|
||||
- Developer friction: {high/medium/low}
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate (This Sprint)
|
||||
|
||||
| Action | Effort | Impact | Owner |
|
||||
| ------------------------- | ------- | ------ | ----- |
|
||||
| {fix critical flaky test} | {hours} | High | |
|
||||
| {add P0 test for X} | {hours} | High | |
|
||||
|
||||
### Short-term (This Milestone)
|
||||
|
||||
| Action | Effort | Impact | Owner |
|
||||
| ----------------------------- | ------ | ------ | ----- |
|
||||
| {refactor slow tests} | {days} | Medium | |
|
||||
| {add integration tests for Y} | {days} | Medium | |
|
||||
|
||||
### Long-term (Ongoing)
|
||||
|
||||
| Action | Effort | Impact | Notes |
|
||||
| ----------------------------- | ------- | ------ | ----- |
|
||||
| {improve test infrastructure} | {weeks} | High | |
|
||||
| {expand coverage to Z} | {weeks} | Medium | |
|
||||
|
||||
---
|
||||
|
||||
## Appendices
|
||||
|
||||
### Appendix A: Flaky Tests
|
||||
|
||||
| Test Name | Failure Rate | Failure Pattern | Fix Priority |
|
||||
| --------- | ------------ | --------------- | ------------ |
|
||||
| | | | |
|
||||
|
||||
### Appendix B: Slow Tests
|
||||
|
||||
| Test Name | Duration | Type | Action |
|
||||
| --------- | -------- | ---- | -------------------------- |
|
||||
| | | | {optimize/move to nightly} |
|
||||
|
||||
### Appendix C: Disabled Tests
|
||||
|
||||
| Test Name | Disabled Since | Reason | Action |
|
||||
| --------- | -------------- | ------ | ------------ |
|
||||
| | | | {fix/delete} |
|
||||
|
||||
### Appendix D: Technical Debt
|
||||
|
||||
| Item | Description | Effort | Priority |
|
||||
| ---- | ----------- | ------ | -------- |
|
||||
| | | | |
|
||||
|
||||
---
|
||||
|
||||
## Next Review
|
||||
|
||||
**Scheduled**: {DATE}
|
||||
**Focus Areas**: {areas to pay attention to}
|
||||
**Success Criteria**: {what would make next review "green"}
|
||||
@@ -0,0 +1,46 @@
|
||||
# Game QA workflow: test-review
|
||||
name: gametest-test-review
|
||||
description: "Review test quality, coverage, and identify gaps in game testing"
|
||||
author: "BMad"
|
||||
|
||||
# Critical variables from config
|
||||
config_source: "{project-root}/_bmad/bmgd/config.yaml"
|
||||
output_folder: "{config_source}:output_folder"
|
||||
user_name: "{config_source}:user_name"
|
||||
communication_language: "{config_source}:communication_language"
|
||||
document_output_language: "{config_source}:document_output_language"
|
||||
date: system-generated
|
||||
|
||||
# Workflow components
|
||||
installed_path: "{project-root}/_bmad/bmgd/workflows/gametest/test-review"
|
||||
instructions: "{installed_path}/instructions.md"
|
||||
validation: "{installed_path}/checklist.md"
|
||||
template: "{installed_path}/test-review-template.md"
|
||||
|
||||
# Variables and inputs
|
||||
variables:
|
||||
review_scope: "full" # full, targeted, quick
|
||||
game_engine: "auto" # auto, unity, unreal, godot
|
||||
|
||||
# Output configuration
|
||||
default_output_file: "{output_folder}/test-review-report.md"
|
||||
|
||||
# Required tools
|
||||
required_tools:
|
||||
- read_file
|
||||
- write_file
|
||||
- list_files
|
||||
- search_repo
|
||||
- glob
|
||||
|
||||
tags:
|
||||
- qa
|
||||
- review
|
||||
- coverage
|
||||
- quality
|
||||
- maintenance
|
||||
|
||||
execution_hints:
|
||||
interactive: false
|
||||
autonomous: true
|
||||
iterative: true
|
||||
Reference in new issue
Block a user