Audit Model Documentation

Overview

The Audit model provides comprehensive tracking of all system activities for compliance, security, and debugging purposes. It implements an immutable audit trail with blockchain-style hash chaining for integrity verification.

Model Structure

Core Fields

Field	Type	Description	Constraints
id	UUID	Unique audit entry identifier	Primary key, auto-generated
timestamp	DateTime	When the event occurred	Required, immutable
tenant_id	UUID	Tenant context (system-wide = 00000000)	Foreign key to Tenant
shard_key	String	Partitioning key for distribution	Format: {tenant_id}-{YYYYMMDD}
actor	AuditActor	Who performed the action	Required
impersonated_by	ImpersonationInfo (Optional)	Admin acting as user	Tracked for security
event	AuditEvent	What action was performed	Required
resource	AuditResource	Target of the action	Required
outcome	AuditOutcome	Success or failure details	Required
lineage	AuditLineage	Event correlation tracking	Required
context	AuditContext	Request and environment info	Required
security	SecurityInfo	Security-relevant metadata	Required
compliance	ComplianceInfo	Regulatory compliance data	Required
metrics	PerformanceMetrics	Performance measurements	Optional
integrity	IntegrityInfo	Hash chain validation	Required

AuditActor Structure

enum AuditActor {
    Human {
        user_id: Uuid,
        tenant_id: Uuid,
        email: String,
        roles: Vec<String>,
        permissions: PermissionSet,
        authentication_method: AuthMethod,
        session_id: Uuid,
    },
    System {
        component: String,
        version: String,
        host: String,
    },
    Agent {
        agent_id: Uuid,
        agent_type: String,
        model: String,
        provider: String,
    }
}

AuditEvent Structure

enum AuditEvent {
    DataOperation {
        operation: String, // CREATE, READ, UPDATE, DELETE
        entity_type: String,
        entity_id: Option<Uuid>,
        changes: Option<FieldChanges>,
        bulk_count: Option<usize>,
        data_classification: DataClassification,
    },
    AuthenticationEvent {
        event_type: String, // LOGIN, LOGOUT, FAILED_LOGIN, TOKEN_REFRESH
        method: String,
        mfa_used: bool,
        failure_reason: Option<String>,
    },
    SystemEvent {
        event_type: String,
        component: String,
        details: serde_json::Value,
    },
}

AuditOutcome Enum

enum AuditOutcome {
    Success,
    Failure {
        error_code: String,
        error_message: String,
    }
}

Database Schema

Primary Storage Pattern

# V2 Format (Current)
Key: {tenant_id}/audit_log/v2/{YYYY}/{MM}/{DD}/{shard_key}/{timestamp_nanos}:{event_id}
Value: JSON serialized AuditLog object

# Legacy V1 Format (Deprecated)
Key: {tenant_id}/audit/logs/{uuid}

Index Patterns

# Actor index
{tenant_id}/audit_idx/actor/{actor_id}/{timestamp}:{event_id}

# Cross-tenant system operations
00000000-0000-0000-0000-000000000000/audit_idx/cross_tenant/{affected_tenant_id}/{timestamp}:{event_id}

# Date-based indexes
{tenant_id}/audit_idx/by_date/{YYYYMMDD}/{timestamp}:{event_id}

# Action-based indexes
{tenant_id}/audit_idx/by_action/{action}/{timestamp}:{event_id}

# User-based indexes (legacy)
{tenant_id}/audit/idx/by_user/{user_id}/{timestamp}:{event_id}

Sequence Tracking

# Global sequence counter
{tenant_id}/audit_sequence -> u64

# Latest hash for chain integrity
{tenant_id}/audit_latest_hash -> String

Integrity Features

Hash Chaining

Each audit entry contains:

• hash: SHA-256 of current entry
• previous_hash: Hash of previous entry
• merkle_root: Optional merkle tree root for batch verification

Signature Support

• Digital signatures for high-value operations
• Witness attestations for critical events
• Timestamp authority integration

Example Data

Standard Audit Entry

{
  "id": "8820da6f-ec1e-4c34-a7d9-e5adeb5dc5d3",
  "timestamp": "2025-08-11T19:54:26.280989703Z",
  "tenant_id": "00000000-0000-0000-0000-000000000000",
  "shard_key": "00000000-0000-0000-0000-000000000000-20250811",
  "actor": {
    "type": "Human",
    "user_id": "00000000-0000-0000-0000-000000000001",
    "tenant_id": "c77bcf41-32c9-48de-9f3d-36217423248c",
    "email": "user@example.com",
    "roles": [],
    "authentication_method": {
      "ApiKey": {
        "key_id": "afd9a82d-d9ab-4c2c-b1e8-5222ada8fc9b"
      }
    },
    "session_id": "161a0026-c26f-4363-8aa3-5c7de24cf496"
  },
  "event": {
    "type": "DataOperation",
    "operation": "Create",
    "entity_type": "tenants",
    "data_classification": "Internal"
  },
  "outcome": "Success",
  "context": {
    "environment": "Production",
    "request_metadata": {
      "ip_address": "2603:9001:5f0:4850:216:3eff:fef3:7de4",
      "user_agent": "curl/7.88.1"
    }
  }
}

Related Code

Source Files

• Model Definition: /src/models/audit_event.rs
• Actor Types: /src/models/audit_actors.rs
• Repository: /src/db/repositories/audit_repository.rs
• Service: /src/services/audit_service.rs
• API Handler: /src/api/handlers/audit/mod.rs
• Tests: /src/services/tests/audit_service_tests.rs

Key Methods

impl AuditService {
    async fn log_event(&self, event: AuditEvent) -> Result<()>
    async fn query_events(&self, filters: AuditFilters) -> Result<Vec<AuditLog>>
    async fn verify_integrity(&self, start: DateTime, end: DateTime) -> Result<bool>
    async fn export_for_compliance(&self, request: ExportRequest) -> Result<ExportedData>
}

API Endpoints

Query Audit Logs

• GET /api/audit/logs?start_date={date}&end_date={date}&actor={uuid}&action={string}
• Response: Paginated Vec<AuditLog>
• Auth: Admin or Auditor role

Export Audit Report

• POST /api/audit/export
• Body: ExportRequest
• Response: CSV/JSON download
• Auth: Admin or Compliance role

Verify Integrity

• POST /api/audit/verify
• Body: { start_date, end_date }
• Response: { valid: bool, details }
• Auth: Admin only

Compliance Features

Retention Policies

• Default: 7 years (2555 days)
• Legal Hold: Prevents deletion
• Auto-archival: After 90 days
• GDPR: PII redaction supported

Frameworks Supported

• SOC 2 Type II
• ISO 27001
• HIPAA (with encryption)
• GDPR (with PII controls)

Export Formats

• JSON (full fidelity)
• CSV (flattened)
• CEF (Common Event Format)
• SIEM integration ready

Security Patterns

Authentication Events

// Login Success
actor: Human { email: "user@example.com", ... }
event: AuthenticationEvent { 
    event_type: "LOGIN",
    method: "password",
    mfa_used: true 
}

// Failed Login
outcome: Failure {
    error_code: "AUTH_FAILED",
    error_message: "Invalid credentials"
}

Data Operations

// Create Operation
event: DataOperation {
    operation: "CREATE",
    entity_type: "users",
    entity_id: Some(user_uuid),
    changes: None, // No previous state
    data_classification: "PII"
}

// Update Operation with Changes
changes: Some({
    "role": {
        "old": "developer",
        "new": "admin"
    }
})

Performance Considerations

Sharding Strategy

• Partition by tenant + date
• Enables parallel queries
• Efficient retention management
• Optimized for time-range queries

Index Strategy

• Actor index for user activity reports
• Date index for compliance queries
• Action index for security analysis
• Cross-tenant for system operations

Write Performance

• Asynchronous logging
• Batch writes when possible
• Minimal impact on operations
• Queue overflow protection

Query Patterns

User Activity

// All actions by a specific user
/{tenant_id}/audit_idx/actor/{user_id}/*

Daily Compliance Report

// All events for a specific date
/{tenant_id}/audit_idx/by_date/{YYYYMMDD}/*

Security Investigation

// Failed login attempts
/{tenant_id}/audit_idx/by_action/FAILED_LOGIN/*

Best Practices

1. Never modify audit entries after creation
2. Always include correlation IDs for related events
3. Batch read operations for efficiency
4. Use indexes rather than scanning all entries
5. Implement retention to manage storage growth

---

Last Updated: 2025-08-29 Version: 2.0 (V2 format)