Skip to main content

Market Data Aggregation

Aggregate market data from multiple sources into unified research database for analysis

Complexity: Complex | Duration: 30m+ | Category: Research/Intelligence

Tags: data-aggregation data-integration ETL research-database normalization

Workflow Diagram​

Steps​

Step 1: Source identification​

Agent: research

agent - Identify authoritative data sources (Gartner, IDC, govt stats)

Step 2: Schema design​

Agent: backend

architect - Design unified schema for multi-source data integration

Step 3: Extraction logic​

Agent: data

engineering - Build extractors for each data source format

Step 4: Deduplication​

Agent: data

engineering - Implement entity resolution and duplicate detection

Step 5: Normalization​

Agent: data

engineering - Normalize metrics, currencies, time periods across sources

Step 6: Quality validation​

Agent: data

engineering - Validate data quality, completeness, consistency

Step 7: Storage optimization​

Agent: backend

architect - Optimize storage for analytical queries

Step 8: API creation​

Agent: backend

architect - Build query API for research access to aggregated data

Usage​

To execute this workflow:

/workflow research/intelligence/market-data-aggregation.workflow

See other workflows in this category for related automation patterns.