ADR-206: Autonomous Research Pipeline — Multi-Agent Workflow with Project Genesis Handoff

Status

Proposed — 2026-02-16

Context

Problem Statement

CODITECT's system prompt v7.0 defines a 3-phase research pipeline that generates 9 markdown artifacts, 6 JSX dashboards, and 15-25 follow-up prompts for technology evaluation and architecture analysis. Currently, this pipeline runs as a single monolithic session — one agent produces all artifacts sequentially, with no parallelization, no dependency management, and no integration with project creation workflows.

This creates several problems:

1. Token Inefficiency: A single agent generating all 20+ artifacts exhausts context windows, loses coherence on later artifacts, and wastes tokens re-reading earlier artifacts for cross-referencing.
2. No Reusability: The research output exists as flat files with no connection to project creation (/new-project), forcing manual handoff.
3. No Quality Gates: Artifacts are not validated against CODITECT standards during generation.
4. No Parallelization: Independent artifacts (e.g., glossary and mermaid diagrams) are generated sequentially despite having no dependencies.
5. No Interactive Intake: Users cannot specify URLs, document paths, or research parameters interactively.

Prior Art

• Agent Labs (Brainqub3): External tool evaluated 2026-02-16. Generates same artifact structure (9 markdown + 6 JSX) but as monolithic output. Conditional Go recommendation — good for offline validation but not integrated.
• /new-project command: 5-phase workflow (Discovery → Submodule → Planning → Structure → QA) that creates production-ready projects. Currently has no pre-discovery research input mechanism.
• MoE Workflow Pattern: Established in skills/moe-task-execution for multi-agent task routing with dependency awareness.

Requirement

Build an embedded framework feature that:

1. Accepts interactive input (URLs, document paths, GitHub repos, artifact directories)
2. Orchestrates multi-agent parallel artifact generation with dependency management
3. Validates all outputs against CODITECT standards
4. Hands off completed research artifacts directly to /new-project as project genesis input
5. Follows all CODITECT automation principles (self-provisioning, complete execution, no manual steps)

Decision

Architecture: 4-Phase Pipeline with Project Genesis Handoff

We adopt a 4-phase pipeline architecture that extends the existing 3-phase research system with an interactive intake phase and a project genesis handoff:

Phase 0: INTAKE (Interactive)
    ↓
Phase 1: RESEARCH (9 Markdown Artifacts — Batched Parallel)
    ↓
Phase 2: VISUALIZATION (6 JSX Dashboards — Batched Parallel)
    ↓
Phase 3: IDEATION (15-25 Follow-up Prompts)
    ↓
Phase 4: GENESIS (Handoff to /new-project) [Optional]

Phase 0: Interactive Intake

Command: /research-pipeline (aliases: /deep-dive, /rp)

Interactive intake collects:

Input	Method	Required
Topic/subject	Positional arg or interactive	Yes
URLs for web research	--urls flag or interactive prompt	No
GitHub repository URLs	--github flag or interactive prompt	No
Document directory path	--docs flag or interactive prompt	No
Original research folder	--originals flag or interactive prompt	No
Output directory	--output flag (default: auto-generated)	No
Extended mode	--extended flag (6 dashboards vs 4)	No
Skip phases	--skip-phase N flag	No
Handoff to /new-project	--genesis flag	No

When no flags provided, the command enters interactive mode using AskUserQuestion to gather all inputs.

Phase 1: Research Artifact Generation (Multi-Agent, Batched Parallel)

Dependency Graph:

Batch 0 (Foundation — must complete first):
  └── Web Research Agent (crawls URLs, fetches GitHub, reads documents)
      Output: research-context.json (structured findings)

Batch 1 (Core Analysis — parallel, depends on Batch 0):
  ├── Quick Start Generator → 1-2-3-detailed-quick-start.md
  ├── CODITECT Impact Analyzer → coditect-impact.md
  └── Executive Summary Writer → executive-summary.md

Batch 2 (Design Documents — parallel, depends on Batch 0 + Batch 1):
  ├── SDD Generator → sdd.md
  ├── TDD Generator → tdd.md
  └── C4 Architecture Modeler → c4-architecture.md

Batch 3 (Reference Materials — parallel, depends on Batch 0):
  ├── ADR Generator → adrs/ (3-7 ADRs)
  ├── Glossary Builder → glossary.md
  └── Mermaid Diagram Creator → mermaid-diagrams.md

Rationale for Batch Ordering:

• Batch 0: Web research must complete first — all other agents consume its output.
• Batch 1 before Batch 2: Executive summary and impact analysis inform SDD/TDD design decisions. Quick start validates that core concepts are understood before detailed design.
• Batch 3 parallel with Batch 2: ADRs, glossary, and diagrams depend only on Batch 0 research context, not on design documents. They can run alongside Batch 2.

Agent-to-Model Routing:

Agent	Default Model	Regulatory Override
Web Research	Sonnet	—
Quick Start Generator	Haiku	—
CODITECT Impact Analyzer	Sonnet	Opus (if healthcare/fintech)
Executive Summary Writer	Sonnet	Opus (if board-level)
SDD Generator	Sonnet	Opus (if compliance-critical)
TDD Generator	Sonnet	—
C4 Architecture Modeler	Sonnet	—
ADR Generator	Sonnet	Opus (if compliance ADRs)
Glossary Builder	Haiku	—
Mermaid Diagram Creator	Haiku	—

Phase 2: Visualization (JSX Dashboard Generation)

Dependency: All Phase 1 artifacts must be complete.

Batch 1 (Core Dashboards — parallel):
  ├── Tech Architecture Analyzer → tech-architecture-analyzer.jsx
  ├── Strategic Fit Dashboard → strategic-fit-dashboard.jsx
  ├── Integration Playbook → coditect-integration-playbook.jsx
  └── Executive Decision Brief → executive-decision-brief.jsx

Batch 2 (Extended — parallel, if --extended):
  ├── Competitive Comparison → competitive-comparison.jsx
  └── Implementation Planner → implementation-planner.jsx

Input: Each dashboard agent receives ALL Phase 1 artifacts as structured JSON input (not raw markdown). A Phase 1 Aggregator agent processes all markdown artifacts into a unified data structure before dispatching to dashboard agents.

Phase 3: Ideation (Follow-up Prompt Generation)

Dependency: All Phase 1 + Phase 2 artifacts complete.

Single agent generates 15-25 categorized prompts across 6 categories:

1. Architecture Deep-Dives
2. Compliance & Regulatory
3. Multi-Agent Orchestration
4. Competitive & Market Intelligence
5. Product Feature Extraction
6. Risk & Mitigation

Phase 4: Project Genesis Handoff (Optional)

Trigger: --genesis flag or user confirmation after Phase 3.

Mechanism: Transform research artifacts into /new-project input format:

# Research artifacts become project discovery input
project_brief = {
    "source": "research-pipeline",
    "topic": research_topic,
    "executive_summary": artifacts["executive-summary.md"],
    "architecture": artifacts["sdd.md"],
    "technical_design": artifacts["tdd.md"],
    "adrs": artifacts["adrs/"],
    "c4_model": artifacts["c4-architecture.md"],
    "impact_analysis": artifacts["coditect-impact.md"],
    "glossary": artifacts["glossary.md"],
    "diagrams": artifacts["mermaid-diagrams.md"],
    "quick_start": artifacts["1-2-3-detailed-quick-start.md"],
    "dashboards": artifacts["dashboards/"],
    "follow_up_prompts": artifacts["follow-up-prompts.md"],
}

# Hand off to /new-project Step 1 (Discovery)
# The discovery specialist receives this as pre-populated input
# instead of conducting a blank-slate interview

The key insight: research artifacts ARE discovery output. The /new-project discovery phase (Step 1) can accept pre-populated project briefs, skipping the interview and proceeding directly to:

• Step 2: Submodule Creation
• Step 3: Project Planning (augmented by research SDD/TDD)
• Step 4: Structure Optimization (informed by C4 architecture)
• Step 5: QA Verification

Orchestration Model

Pattern: Orchestrator-Workers with batched parallelization.

class ResearchPipelineOrchestrator:
    """Coordinates 4-phase research pipeline with dependency management."""

    async def execute(self, config: PipelineConfig) -> PipelineResult:
        # Phase 0: Intake (interactive or CLI)
        inputs = await self.collect_inputs(config)

        # Phase 1: Research (batched parallel)
        context = await self.run_batch(0, [WebResearchAgent(inputs)])
        batch_1 = await self.run_batch(1, [
            QuickStartAgent(context),
            ImpactAnalyzerAgent(context),
            ExecSummaryAgent(context),
        ])
        batch_2_3 = await asyncio.gather(
            self.run_batch(2, [
                SDDAgent(context, batch_1),
                TDDAgent(context, batch_1),
                C4Agent(context, batch_1),
            ]),
            self.run_batch(3, [
                ADRAgent(context),
                GlossaryAgent(context),
                MermaidAgent(context),
            ]),
        )

        # Phase 1 Quality Gate
        artifacts = self.aggregate_artifacts(batch_1, *batch_2_3)
        quality = await self.validate_quality(artifacts)
        if quality.score < 0.7:
            artifacts = await self.retry_failed(quality.failures)

        # Phase 2: Visualization
        structured_data = await self.aggregate_for_viz(artifacts)
        dashboards = await self.run_batch(4, [
            TechArchDashboard(structured_data),
            StrategicFitDashboard(structured_data),
            IntegrationPlaybook(structured_data),
            ExecBriefDashboard(structured_data),
        ])
        if config.extended:
            dashboards += await self.run_batch(5, [
                CompetitiveDashboard(structured_data),
                ImplementationPlanner(structured_data),
            ])

        # Phase 3: Ideation
        prompts = await self.run_single(IdeationAgent(artifacts, dashboards))

        # Phase 4: Genesis (optional)
        if config.genesis:
            await self.handoff_to_new_project(artifacts, dashboards, prompts)

        # Phase 5: Package (always)
        manifest = await self.generate_manifest(artifacts, dashboards, prompts)

        # Phase 5b: Promote (optional, ADR-207)
        if config.auto_promote:
            promotion = await self.run_single(ArtifactPromoterAgent(
                staging_dir=config.output_dir,
                manifest=manifest,
                dry_run=not config.force_promote,
            ))

        # Phase 5c: Registry (after promotion, ADR-207)
        if config.auto_promote and promotion.approved:
            await self.update_manifest_registry(promotion.manifest)

        return PipelineResult(artifacts, dashboards, prompts, manifest)

Error Handling

Failure	Strategy
Web research timeout	Proceed with available data, flag gaps
Artifact agent fails	Retry once with Opus, then continue without
Quality gate fails	Re-run failed artifacts with enhanced prompt
JSX validation fails	Retry with stricter design system constraints
Genesis handoff fails	Save artifacts, provide manual handoff instructions

Standards Integration

The pipeline enforces these CODITECT standards during generation:

Standard	Enforcement Point
CODITECT-STANDARD-AUTOMATION	All phases — self-provisioning, complete execution
CODITECT-STANDARD-TRACK-NOMENCLATURE	ADR numbering, task ID assignment
Artifact Template Standard (NEW)	Phase 1 — markdown structure validation
JSX Design System Standard (NEW)	Phase 2 — Tailwind theme, accessibility
CODITECT-STANDARD-RESEARCH-PIPELINE (NEW)	All phases — pipeline configuration

File Organization

{output-dir}/
├── README.md                          # Pipeline summary and index
├── research-context.json              # Structured web research findings
├── 1-2-3-detailed-quick-start.md      # Artifact 1
├── coditect-impact.md                 # Artifact 2
├── executive-summary.md               # Artifact 3
├── sdd.md                             # Artifact 4
├── tdd.md                             # Artifact 5
├── c4-architecture.md                 # Artifact 9
├── glossary.md                        # Artifact 7
├── mermaid-diagrams.md                # Artifact 8
├── follow-up-prompts.md               # Phase 3
├── adrs/                              # Artifact 6
│   ├── ADR-001-*.md
│   └── ADR-00N-*.md
├── dashboards/                        # Phase 2
│   ├── tech-architecture-analyzer.jsx
│   ├── strategic-fit-dashboard.jsx
│   ├── coditect-integration-playbook.jsx
│   ├── executive-decision-brief.jsx
│   ├── competitive-comparison.jsx     # Extended only
│   └── implementation-planner.jsx     # Extended only
└── pipeline-report.json               # Execution metrics

Phase 5b: Promote (ADR-207)

After artifact generation completes, the pipeline optionally invokes the artifact-promoter agent to move high-value artifacts from the gitignored staging directory to permanent git-tracked locations.

Trigger: --auto-promote flag on /research-pipeline, or manual invocation via /research-promote.

Workflow:

Phase 5: PACKAGE
    ├── Generate MANIFEST.yaml in staging directory
    └── Validate against research-manifest-v1.schema.json
        ↓
Phase 5b: PROMOTE (ADR-207)
    ├── Read MANIFEST.yaml from staging
    ├── Auto-classify research category (1 of 6)
    ├── Apply promotion rules matrix (ALWAYS/CONDITIONALLY/NEVER)
    ├── Present dry-run plan for user approval
    ├── Copy eligible artifacts to permanent locations
    ├── Update frontmatter metadata in promoted files
    └── Rename per ADR-207 naming conventions
        ↓
Phase 5c: REGISTRY (ADR-207)
    ├── Generate/update YAML manifest at internal/research/manifests/
    ├── Validate manifest against JSON schema
    └── Update internal/research/manifests/README.md registry

Safety: Phase 5b always shows a dry-run plan before executing. Staging files are copied, never moved — the staging directory remains intact for backup.

Agent: artifact-promoter (see agents/artifact-promoter.md) Command: /research-promote (see commands/research-promote.md) Rules: ADR-207 Section 4 (Promotion Rules)

Consequences

Positive

1. 10x faster artifact generation — Parallel batch execution vs sequential single-agent.
2. Higher quality — Specialized agents per artifact type produce better-focused output.
3. Project genesis integration — Research directly feeds project creation, eliminating manual handoff.
4. Embedded framework feature — Available to all CODITECT projects via /research-pipeline.
5. Token efficiency — Model routing (Haiku for glossary, Opus for compliance ADRs) reduces costs 40-60%.
6. Standards compliance — Automated validation during generation, not after.
7. Reproducible — Pipeline configuration + inputs can be re-run to refresh artifacts.

Negative

1. Complexity — 16+ agent definitions, orchestrator script, command definition, 3 new standards.
2. Token cost per run — Multi-agent overhead (~15x multiplier per Section 8.1 of system prompt). A full pipeline run may consume 100K+ tokens.
3. Maintenance surface — Agent prompts must stay synchronized with system prompt v7.0+ updates.

Neutral

1. Backward compatible — Existing /new-project workflow unchanged; research pipeline is additive.
2. Agent Labs displacement — Brainqub3 agent-labs becomes redundant for this use case once pipeline is complete.

Alternatives Considered

Alternative 1: Enhance Agent Labs Integration

Use Brainqub3 agent-labs as the generation engine, wrapping it with CODITECT intake and handoff.

Rejected because: Agent Labs runs externally (ChatGPT), produces monolithic output without parallelization, and has no CODITECT standards integration. Wrapping would require more effort than building natively.

Alternative 2: Single-Agent Sequential Pipeline

Keep the current approach but add intake and genesis handoff phases.

Rejected because: Does not address token efficiency, quality, or parallelization problems. Context window exhaustion on large research topics remains.

Alternative 3: Full Async Event-Driven Pipeline

Use NATS/Redis Streams event bus for agent coordination with fully dynamic dependency resolution.

Rejected because: Over-engineered for a well-defined dependency graph. The batch structure is predictable and does not require dynamic scheduling. Simplicity First (Principle 1).

Implementation Plan

Components to Create

Component	Type	Track
/research-pipeline	Command	K (Workflow)
research-pipeline-orchestrator.py	Script	K
research-web-crawler	Agent	K
research-quick-start-generator	Agent	K
research-impact-analyzer	Agent	K
research-exec-summary-writer	Agent	K
research-sdd-generator	Agent	K
research-tdd-generator	Agent	K
research-c4-modeler	Agent	K
research-adr-generator	Agent	K
research-glossary-builder	Agent	K
research-mermaid-creator	Agent	K
research-artifact-aggregator	Agent	K
research-dashboard-generator	Agent	K
research-ideation-generator	Agent	K
research-quality-validator	Agent	K
WF-RESEARCH-PIPELINE.md	Workflow	K
CODITECT-STANDARD-RESEARCH-PIPELINE.md	Standard	K
artifact-promoter	Agent	K
/research-promote	Command	K
research-manifest-v1.schema.json	Schema	K

Modifications to Existing Components

Component	Change
/new-project	Accept --from-research <path> to skip discovery with pre-populated brief
project-discovery-specialist	Accept structured research brief as alternative to interactive interview

Dependencies

• ADR-206 (this document)
• ADR-207 (Research Artifact Organization Taxonomy — promotion rules, manifests)
• System prompt v7.0 (artifact specifications)
• CODITECT-STANDARD-AUTOMATION (execution principles)
• /new-project command (genesis handoff target)

---

Author: Claude (Sonnet 4.5) Requested by: Hal Casteel Date: 2026-02-16