theia AI Agent Development

Overview

theia AI provides a framework for creating custom AI agents that can interact with your IDE, understand domain-specific concepts, and automate workflows.

Agent Fundamentals

What is an Agent?

An agent in theia AI is a specialized AI assistant that:

• Has a specific role/purpose (coding, testing, documentation, etc.)
• Can use tools to interact with the IDE and external systems
• Follows custom prompts tailored to your domain
• Can delegate to other agents for complex tasks

Agent Types in theia IDE

1. Coder - AI coding assistant

   • Code generation
   • Refactoring
   • Bug fixing
   • Test creation

2. App Tester - Browser application testing

   • E2E testing with Playwright
   • Bug detection
   • Test case generation
   • Autonomous testing loops

3. Architect - System design assistance

   • Architecture discussions
   • Design patterns
   • Technical planning

4. Custom Agents - Domain-specific agents

   • Blog post formatting (example)
   • Domain model manipulation
   • Custom workflows

Creating Custom Agents

Method 1: UI-Based Creation

1. Open AI Configuration View

   Settings → AI Configuration → Agents tab

2. Add Custom Agent

   • Click "Add Custom Agent"
   • Choose storage location:
     • User home (personal use)
     • workspace directory (team sharing)

3. Configure Agent

   ID: agent-id-name
   Name: Display Name
   Description: What this agent does

4. Edit Prompt Template

   • Click "Edit" next to agent
   • Opens prompt editor in IDE
   • Write system prompt with instructions

Method 2: Programmatic Creation

// Define agent class
@injectable()
export class CustomAgent extends AbstractStreamParsingAgent {
  override async getLanguageModelRequirements(): Promise<LanguageModelRequirement[]> {
    return [{
      purpose: 'custom-agent',
      identifier: 'anthropic/claude-sonnet-4'
    }];
  }

  override async getSystemMessageTemplate(): Promise<string> {
    return `
      You are an agent that helps with [specific task].
      
      Your capabilities:
      - [Capability 1]
      - [Capability 2]
      
      Workflow:
      1. [Step 1]
      2. [Step 2]
    `;
  }
}

Prompt Engineering for Agents

System Prompt Structure

# Role Definition
You are an agent that [primary function].

# Capabilities
- [List tools and functions available]
- [List data access capabilities]

# Workflow
1. [First step with clear instructions]
2. [Second step]
   - Sub-instruction
   - Constraint
3. [Final step]

# Output Format
[Specify desired output structure]

# Constraints
- [Important limitation 1]
- [Important limitation 2]

# Tool Functions
[List available tool functions]

Example: Blog Post Converter Agent

You are an agent that helps users convert existing content 
into a well-formatted blog post.

Workflow:
1. Read the source document
2. Extract content and images
3. Apply Hugo format with metadata:
   - Title
   - Date
   - Tags
   - Author
   - Description
4. Fix internal links (relative)
5. Embed images correctly
6. **STOP** after each step for user review

Output:
- Formatted markdown file
- Images in correct directory
- Valid frontmatter metadata

Tool Functions

What are Tool Functions?

Tool functions enable agents to interact with:

• workspace files and directories
• External systems (via MCP)
• IDE services
• Domain-specific logic

Built-in Tool Functions

workspace Operations

// File operations
file_read(path: string): Promise<string>
file_write(path: string, content: string): Promise<void>
file_search(pattern: string): Promise<string[]>

// Directory operations
dir_list(path: string): Promise<FileEntry[]>
workspace_structure(): Promise<TreeNode>

Context Retrieval

// Get file contents
retrieve_file(path: string)

// Search in workspace
search_workspace(query: string)

// Get file validation info
validate_file(path: string)

Editing Operations

// Propose changes
propose_change(file: string, old: string, new: string)

// Apply edits
apply_edit(file: string, edits: Edit[])

Custom Tool Functions

Creating Tool Functions

1. Implement Tool Provider

@injectable()
export class ListTasksTool implements ToolProvider {
  getTool(): Tool {
    return {
      name: 'list_tasks',
      description: 'List all tasks in the workspace',
      parameters: {
        type: 'object',
        properties: {}
      },
      handler: async () => {
        // Implementation
        const taskDir = 'task-definitions';
        const files = await this.fileService.readDir(taskDir);
        const tasks = await Promise.all(
          files.map(f => this.readTaskFile(f))
        );
        return { tasks };
      }
    };
  }
}

2. Register in Module

export default new ContainerModule(bind => {
  bind(ToolProvider).to(ListTasksTool);
  bind(ToolProvider).to(CreateTaskTool);
  bind(ToolProvider).to(GetTaskTool);
});

3. Reference in Prompt

Available Functions:
- list_tasks(): Returns all tasks in workspace
- get_task(id: string): Get specific task details
- create_task(definition: TaskDef): Create new task

Tool Function Best Practices

1. Clear Descriptions

   • Explain what the function does
   • Specify when to use it
   • Document parameters

2. JSON Schema for Parameters

   {
     "type": "object",
     "properties": {
       "id": {
         "type": "string",
         "description": "The task ID"
       }
     },
     "required": ["id"]
   }

3. Error Handling

   handler: async (args) => {
     try {
       const result = await this.doWork(args);
       return { success: true, data: result };
     } catch (error) {
       return { 
         success: false, 
         error: error.message 
       };
     }
   }

4. Idempotency

   • Safe to call multiple times
   • Use file locking if needed
   • Check state before mutations

Agent-to-Agent Delegation

Why Delegate?

• Break complex tasks into subtasks
• Leverage specialized agents
• Maintain separation of concerns
• Enable autonomous workflows

Delegation Pattern

// In main agent's prompt:
When you need to [specific task], delegate to [agent-name]:

Use the delegation tool function:
delegate_to_agent({
  agent: "agent-id",
  task: "Specific instruction",
  context: "Relevant information"
})

Example: Testing + Bug Reporting

# App Tester Agent

Workflow:
1. Test the application
2. Identify bugs
3. **Delegate to bug-reporter** for each bug found:
   - Call: delegate_to_agent("bug-reporter", {
       title: "Bug description",
       steps: [...],
       expected: "...",
       actual: "..."
     })
4. Continue testing

# Bug Reporter Agent

Purpose: Report bugs to GitHub

Workflow:
1. Receive bug details
2. Format using template
3. Create GitHub issue
4. Return issue URL

Implementing Delegation

// Register delegation tool
@injectable()
export class AgentDelegationTool implements ToolProvider {
  @inject(AgentService)
  protected agentService: AgentService;

  getTool(): Tool {
    return {
      name: 'delegate_to_agent',
      description: 'Delegate task to another agent',
      parameters: {
        type: 'object',
        properties: {
          agent_id: { type: 'string' },
          task: { type: 'string' },
          context: { type: 'object' }
        },
        required: ['agent_id', 'task']
      },
      handler: async (args) => {
        const agent = this.agentService.getAgent(args.agent_id);
        const result = await agent.execute(
          args.task,
          args.context
        );
        return result;
      }
    };
  }
}

Context Management

Providing Context to Agents

1. Drag and Drop Files

// In chat UI, users can drag files
// Agent receives file references

2. Variable References

User: @coder create tests for ${file:src/calculator.ts}

Agent receives:
- File path
- File contents
- Context about the file

3. Project Information

# .theia/ai/project-info.md

Project: Calculator App
Technology: React + TypeScript

Testing Guidelines:
- Tests in same directory as source
- Use .spec.ts extension
- Example: src/calculator.spec.ts

Conventions:
- Follow existing test patterns
- Mock external dependencies

4. workspace Variables

// Available in prompts:
${workspace.root}      // /home/user/project
${workspace.name}      // project
${file.current}        // Currently open file
${file.selection}      // Selected text

Project-Specific Augmentation

# In system prompt:

## Project Context

The following project-specific information augments your knowledge:

${prompt_project_info}

This variable loads: .theia/ai/project-info.md

Use this information to:
- Follow project conventions
- Use correct file locations
- Apply project-specific patterns

Advanced Agent Patterns

1. Multi-Step Workflows

Your workflow has multiple steps. After EACH step:

1. Present results to user
2. STOP and wait for confirmation
3. Only proceed when user says "next" or "continue"

Steps:
1. Analyze input → STOP
2. Generate solution → STOP
3. Apply changes → STOP
4. Verify results → STOP

2. Iterative Refinement

class IterativeAgent {
  async execute(task: string): Promise<Result> {
    let iteration = 0;
    let result = await this.initialAttempt(task);
    
    while (!result.acceptable && iteration < MAX_ITERATIONS) {
      const feedback = await this.evaluate(result);
      result = await this.refine(result, feedback);
      iteration++;
    }
    
    return result;
  }
}

3. Autonomous Loops

You will test the application autonomously:

1. Generate test case
2. Execute test
3. Record results
4. If bug found → delegate to bug-reporter
5. Generate next test case
6. Repeat until coverage complete or max iterations

Stop conditions:
- All features tested
- 20 test cases executed
- User interrupts

4. Context-Aware Agents

@injectable()
export class ContextAwareAgent extends AbstractAgent {
  @inject(workspaceService)
  protected workspace: workspaceService;
  
  override async getSystemMessageTemplate(): Promise<string> {
    // Load dynamic context
    const projectInfo = await this.loadProjectInfo();
    const recentFiles = await this.getRecentFiles();
    
    return `
      ${basePrompt}
      
      Current Project: ${projectInfo}
      Recent Activity: ${recentFiles}
    `;
  }
}

Testing Agents

Manual Testing

1. Create test workspace
2. Invoke agent with sample tasks
3. Verify outputs
4. Check tool calls
5. Validate error handling

Automated Testing

describe('CustomAgent', () => {
  let agent: CustomAgent;
  
  beforeEach(() => {
    agent = createTestAgent(CustomAgent);
  });
  
  it('should process simple request', async () => {
    const result = await agent.execute('test task');
    expect(result).toMatchObject({
      success: true,
      output: expect.any(String)
    });
  });
  
  it('should call correct tools', async () => {
    const spy = jest.spyOn(agent, 'callTool');
    await agent.execute('list all tasks');
    expect(spy).toHaveBeenCalledWith('list_tasks');
  });
});

Evaluation Criteria

• Accuracy: Correct understanding of requests
• Tool Usage: Appropriate tool selection
• Error Handling: Graceful failure recovery
• Output Quality: Useful, formatted responses
• Performance: Response time, token efficiency

Agent Configuration

llm Selection

// In agent definition:
getLanguageModelRequirements() {
  return [{
    purpose: 'agent-purpose',
    identifier: 'anthropic/claude-sonnet-4',
    // or: 'openai/gpt-4'
    // or: 'local/mistral-7b'
  }];
}

Agent Settings

# .theia/agents/my-agent.yaml
id: my-agent
name: My Custom Agent
description: Does specific tasks
llm:
  provider: anthropic
  model: claude-sonnet-4
  temperature: 0.7
  max_tokens: 4000
tools:
  - workspace_operations
  - custom_tools
prompt_template: ./prompts/my-agent.md

Best Practices

1. Clear Role Definition

❌ "You are a helpful assistant"
✅ "You are a testing agent that autonomously tests web applications using Playwright"

2. Specific Instructions

❌ "Help the user"
✅ "Follow these steps:
    1. Read the application code
    2. Generate 5 test cases covering core functionality
    3. Execute each test using the Playwright MCP server
    4. Report results in structured format"

3. Boundary Setting

Do NOT:
- Make assumptions about file locations
- Execute destructive operations without confirmation
- Generate code without understanding requirements

ALWAYS:
- Verify file exists before reading
- Ask for clarification when ambiguous
- Provide reasoning for decisions

4. Tool Function Documentation

{
  name: 'create_task',
  description: `
    Creates a new task definition in the workspace.
    
    Use when: User requests task creation
    Don't use when: Task already exists (check first)
    
    Parameters:
    - name: Human-readable task name
    - inputs: Array of input types
    - outputs: Array of output types
    
    Returns: Task ID and file path
  `,
  // ...
}

5. Error Communication

When errors occur:

1. Explain what went wrong in plain language
2. Provide specific error details if helpful
3. Suggest corrective actions
4. Ask if user wants to retry or take different approach

Example:
"I encountered an error reading the file 'tasks/task-1.json':
File not found. This might mean:
- The task doesn't exist yet
- It's in a different directory
- The file was deleted

Would you like me to:
a) Create this task
b) Search for it in other locations
c) List all existing tasks"

Integration Patterns

With MCP Servers

# In agent prompt:

You have access to GitHub via MCP:

Functions:
${mcp:github:*}

Use these to:
- Create issues: create_issue(title, body, labels)
- List PRs: list_pull_requests(state, author)
- Add comments: add_comment(issue_number, body)

With IDE Services

@injectable()
export class IntegratedAgent extends AbstractAgent {
  @inject(MessageService)
  protected messages: MessageService;
  
  @inject(workspaceService)
  protected workspace: workspaceService;
  
  @inject(editorManager)
  protected editorManager: editorManager;
  
  // Use IDE services in tool handlers
}

With External APIs

export class APIIntegratedTool implements ToolProvider {
  getTool(): Tool {
    return {
      name: 'fetch_external_data',
      handler: async (args) => {
        const response = await fetch(args.url, {
          headers: { 'Authorization': `Bearer ${args.token}` }
        });
        return await response.json();
      }
    };
  }
}

Deployment

Sharing Agents

workspace-Level (Team Sharing)

project/
├── .theia/
│   ├── agents/
│   │   └── team-agent.yaml
│   └── prompts/
│       └── team-agent.md
└── ... (commit to Git)

User-Level (Personal)

~/.theia/
├── agents/
│   └── my-agent.yaml
└── prompts/
    └── my-agent.md

Distributing Custom Agents

// As theia extension
export default new ContainerModule(bind => {
  // Register agent
  bind(Agent).to(CustomAgent);
  
  // Register tools
  bind(ToolProvider).to(CustomTool1);
  bind(ToolProvider).to(CustomTool2);
});

Summary

Creating effective AI agents in theia requires:

1. Clear purpose - Define specific role and capabilities
2. Good prompts - Detailed instructions and workflows
3. Right tools - Provide necessary functions and access
4. Context management - Supply relevant project information
5. Error handling - Graceful failures and user communication
6. Testing - Validate behavior with real scenarios
7. Iteration - Refine based on usage and feedback

The framework provides flexibility to create agents ranging from simple assistants to complex autonomous systems with multi-agent coordination.