Skip to main content

AI Deep Research Agent

import { CodeFetcher } from '../../../components/CodeFetcher'

Let's build a AI Deep Research Agent that:​

  • Deep Web Research: Automatically searches the web, extracts content, and synthesizes findings
  • Iterative Research Process: Supports multiple layers of research depth for comprehensive exploration
  • Event-Driven Architecture: Built using Motia Framework's event system for robust workflow management
  • Parallel Processing: Efficiently processes search results and content extraction
  • API Endpoints: REST API access for initiating research and retrieving reports
  • Stateful Processing: Maintains research state throughout the entire process

The Steps​

<Tabs items={['analyze-content', 'compile-report', 'extract-content', 'follow-up-research', 'generate-queries', 'report-api', 'research-api', 'search-web', 'status-api']}>

🚀 Features​

  • Deep Web Research: Automatically searches the web, extracts content, and synthesizes findings
  • Iterative Research Process: Supports multiple layers of research depth for comprehensive exploration
  • Event-Driven Architecture: Built using Motia Framework's event system for robust workflow management
  • Parallel Processing: Efficiently processes search results and content extraction
  • API Endpoints: REST API access for initiating research and retrieving reports
  • Stateful Processing: Maintains research state throughout the entire process

📋 Prerequisites​

  • Node.js v18 or later
  • npm or pnpm
  • API keys for:

🛠️ Installation​

  1. Clone the repository:

    git clone https://github.com/MotiaDev/motia-examples
    cd examples/ai-deep-research-agent

  2. Install dependencies:

    pnpm install
    # or
    npm install

  3. Configure environment variables:

    cp .env.example .env

    Update .env with your API keys:

    # Required
    OPENAI_API_KEY=your-openai-api-key-here
    FIRECRAWL_API_KEY=your-firecrawl-api-key-here

    # Optional
    # OPENAI_MODEL=gpt-4o
    # FIRECRAWL_BASE_URL=http://your-firecrawl-instance-url

🏗️ Architecture​

AI Deep Research Agent

🚦 API Endpoints​

Start Research​

POST /research
Content-Type: application/json

{
  "query": "The research topic or question",
  "breadth": 4,  // Number of search queries to generate (1-10)
  "depth": 2     // Depth of research iterations (1-5)
}

Response:

{
  "message": "Research process started",
  "requestId": "unique-trace-id"
}

Check Research Status​

GET /research/status?requestId=unique-trace-id

Response:

{
  "message": "Research status retrieved successfully",
  "requestId": "unique-trace-id",
  "originalQuery": "The research topic or question",
  "status": "in-progress",
  "progress": {
    "currentDepth": 1,
    "totalDepth": 2,
    "percentComplete": 50
  },
  "reportAvailable": false
}

Get Research Report​

GET /research/report?requestId=unique-trace-id

Response:

{
  "message": "Research report retrieved successfully",
  "report": {
    "title": "Research Report Title",
    "overview": "Executive summary...",
    "sections": [
      {
        "title": "Section Title",
        "content": "Section content..."
      }
    ],
    "keyTakeaways": [
      "Key takeaway 1",
      "Key takeaway 2"
    ],
    "sources": [
      {
        "title": "Source Title",
        "url": "Source URL"
      }
    ],
    "originalQuery": "The research topic or question",
    "metadata": {
      "depthUsed": 2,
      "completedAt": "2025-03-18T16:45:30Z"
    }
  },
  "requestId": "unique-trace-id"
}

🏃‍♂️ Running the Application​

  1. Start the development server:

    pnpm dev

  2. Access the Motia Workbench:

    http://localhost:3000

  3. Make a test request:

    curl --request POST \
    --url http://localhost:3000/research \
    --header 'Content-Type: application/json' \
    --data '{
       "query": "Advancements in renewable energy storage",
       "depth": 1,
       "breadth": 1
    }'

🙏 Acknowledgments​