Skip to main content

ADR-011-v4: Audit & Compliance - Part 2 (Technical)

Document Specification Block​

Document: ADR-011-v4-audit-compliance-part2-technical
Version: 1.0.0
Purpose: Constrain AI implementation with exact technical specifications for audit and compliance
Audience: AI agents, developers implementing the system
Date Created: 2025-08-31
Date Modified: 2025-08-31
QA Review Date: 2025-08-31
Status: APPROVED

Table of Contents​

  1. Constraints
  2. Dependencies
  3. Component Architecture
  4. Data Models
  5. Implementation Patterns
  6. API Specifications
  7. Testing Requirements
  8. Performance Benchmarks
  9. Security Controls
  10. Logging and Error Handling
  11. References
  12. Approval Signatures

1. Constraints​

CONSTRAINT: Mandatory Audit Events​

Every API request, data modification, authentication attempt, and AI operation MUST generate an audit event. No exceptions.

CONSTRAINT: Immutable Storage​

Audit events MUST be append-only. Updates or deletions are forbidden except through compliant retention policies.

CONSTRAINT: Multi-Tenant Isolation​

Each tenant's audit data MUST be cryptographically isolated using tenant-prefixed keys in FoundationDB.

CONSTRAINT: GDPR Compliance​

System MUST support data export, selective deletion, and anonymization while maintaining legally required audit trails.

CONSTRAINT: Performance Requirements​

Audit system MUST NOT add more than 10ms latency to any operation and MUST handle 10,000 events/second per region.

↑ Back to Top

2. Dependencies​

cargo.toml Dependencies​

[dependencies]
# Core async runtime
tokio = { version = "1.35", features = ["full"] }

# Database
foundationdb = { version = "0.8", features = ["embedded-fdb-include"] }

# Web framework
actix-web = "4.4"
actix-rt = "2.9"

# Serialization
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
bincode = "1.3"

# Error handling
anyhow = "1.0"
thiserror = "1.0"

# Utilities
uuid = { version = "1.6", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["json"] }

# Cryptography
sha2 = "0.10"
chacha20poly1305 = "0.10"

# JWT Authentication
jsonwebtoken = "9.2"

[dev-dependencies]
# Testing
tokio-test = "0.4"

↑ Back to Top

3. Component Architecture​

// File: src/services/audit_service.rs
use std::sync::Arc;
use anyhow::Result;
use uuid::Uuid;
use foundationdb::Database;

// Core audit service structure
pub struct AuditService {
    db: Arc<Database>,
    enricher: Arc<EventEnricher>,
    integrity: Arc<IntegrityService>,
    retention: Arc<RetentionManager>,
}

// File: src/services/audit_service.rs
// Event flow pipeline
impl AuditService {
    pub async fn log_event(&self, event: AuditEvent) -> Result<Uuid> {
        // 1. Enrich with context
        let enriched = self.enricher.enrich(event).await?;
        
        // 2. Calculate integrity hash
        let hashed = self.integrity.add_proof(enriched).await?;
        
        // 3. Store in FoundationDB
        let repo = AuditLogRepository::new(self.db.clone(), hashed.tenant_id);
        repo.store_event(&hashed).await?;
        
        // 4. Async retention check
        let tenant_id = hashed.tenant_id;
        let retention = self.retention.clone();
        tokio::spawn(async move {
            if let Err(e) = retention.check_policies(tenant_id).await {
                tracing::error!("Retention check failed: {}", e);
            }
        });
        
        Ok(hashed.id)
    }
}

↑ Back to Top

4. Data Models​

Audit Event Structure​

// File: src/models/audit_event.rs
use chrono::{DateTime, Utc};
use serde::{Serialize, Deserialize};
use uuid::Uuid;

#[derive(Serialize, Deserialize)]
pub struct AuditEvent {
    // Identity
    pub id: Uuid,
    pub timestamp: DateTime<Utc>,
    pub tenant_id: Uuid,
    pub shard_key: String, // Format: "{tenant_id}-{YYYYMMDD}"
    
    // Actor context
    pub actor: AuditActor,
    pub impersonated_by: Option<Box<AuditActor>>,
    
    // Event details
    pub event: AuditEventType,
    pub resource: ResourceDescriptor,
    pub outcome: EventOutcome,
    
    // Compliance metadata
    pub compliance: ComplianceMetadata,
    pub integrity: IntegrityProof,
}

// File: src/models/compliance.rs
// GDPR-compliant data subject request
#[derive(Serialize, Deserialize)]
pub struct DataSubjectRequest {
    pub id: Uuid,
    pub tenant_id: Uuid,
    pub subject_id: Uuid,
    pub request_type: DsrType,
    pub status: DsrStatus,
    pub data_categories: Vec<DataCategory>,
}

// Retention policy configuration
#[derive(Serialize, Deserialize)]
pub struct RetentionPolicy {
    pub id: String,
    pub data_type: DataType,
    pub retention_period: Duration,
    pub action: RetentionAction,
}

↑ Back to Top

5. Implementation Patterns​

Repository Pattern for Audit Storage​

// File: src/db/repositories/audit_log_repository.rs
use foundationdb::{Database, Transaction};
use anyhow::Result;
use uuid::Uuid;

impl AuditLogRepository {
    // Store with integrity chain
    pub async fn store_event(&self, event: &AuditEvent) -> Result<()> {
        let tx = self.db.create_transaction()?;
        
        // Get previous hash for chain
        let prev_hash = self.get_latest_hash(&tx).await?;
        let event = event.with_integrity(prev_hash);
        
        // Store event
        let key = format!("{}/audit/{}/{}", 
            self.tenant_id, 
            event.timestamp.format("%Y%m%d"),
            event.id
        );
        
        let value = bincode::serialize(&event)?;
        tx.set(&key, &value);
        
        // Update indices
        self.update_actor_index(&tx, &event).await?;
        self.update_resource_index(&tx, &event).await?;
        
        tx.commit().await?;
        Ok(())
    }
    
    // GDPR-compliant data export
    pub async fn export_user_data(&self, user_id: Uuid) -> Result<UserDataExport> {
        let mut export = UserDataExport::new(user_id);
        
        // Collect all data categories
        for category in DataCategory::iter() {
            let data = self.collect_category_data(user_id, category).await?;
            export.add_category(category, data);
        }
        
        Ok(export)
    }
}

Compliance Service Implementation​

impl ComplianceService {
    // Process GDPR request
    pub async fn process_dsr(&self, request: DataSubjectRequest) -> Result<()> {
        match request.request_type {
            DsrType::Access => self.handle_access_request(request).await,
            DsrType::Erasure => self.handle_erasure_request(request).await,
            DsrType::Portability => self.handle_portability_request(request).await,
            _ => Err(anyhow!("Unsupported request type"))
        }
    }
    
    // Selective deletion with audit preservation
    async fn handle_erasure_request(&self, request: DataSubjectRequest) -> Result<()> {
        let tx = self.db.create_transaction()?;
        
        // Find erasable data (not legally required)
        let erasable = self.find_erasable_data(&tx, request.subject_id).await?;
        
        for key in erasable {
            // Anonymize instead of delete for audit trail
            let mut record = self.get_record(&tx, &key).await?;
            record.anonymize();
            tx.set(&key, &bincode::serialize(&record)?);
        }
        
        tx.commit().await?;
        Ok(())
    }
}

↑ Back to Top

6. API Specifications​

JWT Authentication Middleware​

// File: src/middleware/auth.rs
use actix_web::{FromRequest, HttpRequest, dev::Payload};
use jsonwebtoken::{decode, DecodingKey, Validation};

#[derive(Debug, Deserialize)]
pub struct Claims {
    pub user_id: Uuid,
    pub tenant_id: Uuid,
    pub roles: Vec<String>,
    pub exp: usize,
}

impl FromRequest for Claims {
    type Error = actix_web::Error;
    type Future = Ready<Result<Self, Self::Error>>;

    fn from_request(req: &HttpRequest, _: &mut Payload) -> Self::Future {
        let auth_header = req.headers().get("Authorization");
        
        match auth_header {
            Some(header) => {
                let token = header.to_str().unwrap().replace("Bearer ", "");
                let key = DecodingKey::from_secret(JWT_SECRET.as_bytes());
                
                match decode::<Claims>(&token, &key, &Validation::default()) {
                    Ok(token_data) => ready(Ok(token_data.claims)),
                    Err(_) => ready(Err(ErrorUnauthorized("Invalid token"))),
                }
            }
            None => ready(Err(ErrorUnauthorized("Missing authorization header"))),
        }
    }
}

Audit Query Endpoints​

// File: src/api/handlers/audit_routes.rs
use actix_web::{web::{Query, Json}, get, post, Result};
use std::sync::Arc;

// Query audit events
#[get("/audit/events")]
pub async fn query_events(
    Query(params): Query<AuditQueryParams>,
    Extension(audit_service): Extension<Arc<AuditService>>,
    Claims(claims): Claims,
) -> Result<Json<Vec<AuditEvent>>> {
    // Validate tenant access
    validate_tenant_access(&claims, params.tenant_id)?;
    
    let events = audit_service
        .query_events(params.tenant_id, params.start, params.end, params.limit)
        .await?;
        
    Ok(Json(events))
}

// GDPR data export
#[post("/compliance/gdpr/export")]
pub async fn export_user_data(
    Json(request): Json<ExportRequest>,
    Extension(compliance_service): Extension<Arc<ComplianceService>>,
    Claims(claims): Claims,
) -> Result<Json<ExportResponse>> {
    // Verify requester is subject or has permission
    verify_gdpr_permission(&claims, request.subject_id)?;
    
    let export = compliance_service
        .export_user_data(request.subject_id)
        .await?;
        
    Ok(Json(ExportResponse {
        export_id: export.id,
        download_url: generate_secure_url(export.id),
        expires_at: Utc::now() + Duration::hours(24),
    }))
}

// Compliance report generation
#[post("/compliance/reports/generate")]
pub async fn generate_report(
    Json(request): Json<ReportRequest>,
    Extension(report_service): Extension<Arc<ReportService>>,
    Claims(claims): Claims,
) -> Result<Json<ReportResponse>> {
    validate_compliance_officer(&claims)?;
    
    let report = report_service
        .generate_report(request.report_type, request.period)
        .await?;
        
    Ok(Json(report))
}

↑ Back to Top

7. Testing Requirements​

Test Coverage Requirements​

  • Unit Test Coverage: ≥90% of all code paths
  • Integration Test Coverage: ≥80% of component interactions
  • Critical Path Coverage: 100% for audit, authentication, and compliance operations
  • Performance Test Coverage: All endpoints must be benchmarked

Unit Tests​

// File: src/services/audit_service_tests.rs
#[cfg(test)]
mod tests {
    use super::*;
    
    #[tokio::test]
    async fn test_audit_event_integrity_chain() {
        let service = setup_test_service().await;
        
        // Create chain of events
        let event1 = create_test_event();
        let id1 = service.log_event(event1).await.unwrap();
        
        let event2 = create_test_event();
        let id2 = service.log_event(event2).await.unwrap();
        
        // Verify chain integrity
        let stored1 = service.get_event(id1).await.unwrap();
        let stored2 = service.get_event(id2).await.unwrap();
        
        assert_eq!(stored2.integrity.previous_hash, stored1.integrity.hash);
    }
    
    #[tokio::test]
    async fn test_gdpr_data_export() {
        let service = setup_test_service().await;
        let user_id = Uuid::new_v4();
        
        // Create various user data
        create_user_audit_trail(&service, user_id).await;
        
        // Export data
        let export = service.export_user_data(user_id).await.unwrap();
        
        // Verify all categories present
        assert!(export.has_category(DataCategory::Profile));
        assert!(export.has_category(DataCategory::Activity));
        assert!(export.has_category(DataCategory::AuditLogs));
    }
    
    #[tokio::test]
    async fn test_retention_policy_enforcement() {
        let service = setup_test_service().await;
        
        // Create old events
        let old_event = create_event_with_age(Duration::days(100));
        service.log_event(old_event).await.unwrap();
        
        // Run retention
        service.enforce_retention_policies().await.unwrap();
        
        // Verify anonymization
        let events = service.query_old_events(Duration::days(91)).await.unwrap();
        assert!(events[0].is_anonymized());
    }
}

Integration Tests​

#[tokio::test]
async fn test_compliance_workflow() {
    let app = setup_test_app().await;
    
    // Submit GDPR request
    let response = app.post("/compliance/gdpr/request")
        .json(&json!({
            "request_type": "access",
            "subject_id": "user123"
        }))
        .send()
        .await
        .unwrap();
        
    assert_eq!(response.status(), 200);
    
    // Process request
    let request_id = response.json::<DsrResponse>().await.unwrap().id;
    
    // Check status
    let status = app.get(&format!("/compliance/gdpr/status/{}", request_id))
        .send()
        .await
        .unwrap()
        .json::<DsrStatus>()
        .await
        .unwrap();
        
    assert_eq!(status, DsrStatus::Completed);
}

↑ Back to Top

7. Performance Benchmarks​

Required Performance Metrics​

// Benchmark configuration
const BENCHMARK_EVENTS: usize = 1_000_000;
const TARGET_THROUGHPUT: usize = 10_000; // events/second
const MAX_LATENCY_MS: u64 = 10;

#[bench]
fn bench_event_ingestion(b: &mut Bencher) {
    let runtime = tokio::runtime::Runtime::new().unwrap();
    let service = runtime.block_on(setup_service());
    
    b.iter(|| {
        runtime.block_on(async {
            let events = generate_events(1000);
            for event in events {
                service.log_event(event).await.unwrap();
            }
        })
    });
}

// Performance requirements
impl PerformanceRequirements {
    pub fn validate() -> bool {
        let metrics = run_benchmarks();
        
        metrics.throughput >= TARGET_THROUGHPUT &&
        metrics.p99_latency_ms <= MAX_LATENCY_MS &&
        metrics.storage_per_event <= 100 // bytes after compression
    }
}

↑ Back to Top

8. Security Controls​

Access Control​

// Role-based access to audit data
impl AuditAccessControl {
    pub fn can_read_audit(&self, claims: &Claims, tenant_id: Uuid) -> bool {
        claims.tenant_id == tenant_id && 
        (claims.roles.contains("admin") || 
         claims.roles.contains("auditor") ||
         claims.roles.contains("compliance_officer"))
    }
    
    pub fn can_export_data(&self, claims: &Claims, subject_id: Uuid) -> bool {
        claims.user_id == subject_id || // Own data
        claims.roles.contains("privacy_officer") ||
        claims.roles.contains("admin")
    }
}

// Encryption for sensitive data
impl EncryptionService {
    pub fn encrypt_pii(&self, data: &str) -> Result<Vec<u8>> {
        let key = self.get_tenant_key()?;
        let nonce = generate_nonce();
        
        let cipher = ChaCha20Poly1305::new(&key);
        cipher.encrypt(&nonce, data.as_bytes())
            .map_err(|e| anyhow!("Encryption failed: {}", e))
    }
}

↑ Back to Top

10. Logging and Error Handling​

Logging Pattern​

// File: src/logging/mod.rs
use tracing::{info, warn, error, debug};
use serde_json::json;

pub fn log_audit_event(event: &AuditEvent) {
    info!(
        correlation_id = %event.lineage.correlation_id,
        tenant_id = %event.tenant_id,
        event_type = ?event.event,
        actor = %event.actor.display_name(),
        "Audit event processed"
    );
}

pub fn log_compliance_action(action: &str, user_id: Uuid, result: &Result<()>) {
    match result {
        Ok(_) => info!(
            action = action,
            user_id = %user_id,
            status = "success",
            "Compliance action completed"
        ),
        Err(e) => error!(
            action = action,
            user_id = %user_id,
            error = %e,
            status = "failed",
            "Compliance action failed"
        ),
    }
}

Error Handling with User-Friendly Messages​

// File: src/errors/compliance_errors.rs
use thiserror::Error;
use actix_web::{ResponseError, HttpResponse};

#[derive(Error, Debug)]
pub enum ComplianceError {
    #[error("You don't have permission to view audit logs. Please contact your administrator to request 'Auditor' role access.")]
    AuditAccessDenied,
    
    #[error("Your data export request couldn't be completed. This is usually due to high system load. Please try again in 5 minutes or contact support.")]
    ExportFailed(#[source] anyhow::Error),
    
    #[error("The compliance report couldn't be generated because some data is still being processed. The report will be ready in approximately 10 minutes.")]
    ReportGenerationInProgress,
    
    #[error("This data cannot be deleted due to legal retention requirements. It will be automatically removed on {date} when the retention period expires.")]
    RetentionPolicyConflict { date: String },
    
    #[error("The requested data subject operation is not supported: {operation}")]
    UnsupportedDsrOperation { operation: String },
}

impl ResponseError for ComplianceError {
    fn error_response(&self) -> HttpResponse {
        match self {
            Self::AuditAccessDenied => HttpResponse::Forbidden().json(json!({
                "error": "access_denied",
                "message": self.to_string(),
                "action_required": "Request 'Auditor' role from administrator"
            })),
            Self::ExportFailed(_) => HttpResponse::ServiceUnavailable().json(json!({
                "error": "export_failed", 
                "message": self.to_string(),
                "retry_after": 300
            })),
            Self::ReportGenerationInProgress => HttpResponse::Accepted().json(json!({
                "error": "report_pending",
                "message": self.to_string(),
                "estimated_time": 600
            })),
            Self::RetentionPolicyConflict { .. } => HttpResponse::Conflict().json(json!({
                "error": "retention_conflict",
                "message": self.to_string()
            })),
            Self::UnsupportedDsrOperation { .. } => HttpResponse::BadRequest().json(json!({
                "error": "unsupported_operation",
                "message": self.to_string()
            })),
        }
    }
}

↑ Back to Top

11. References​

Version Compatibility​

  • FoundationDB: 7.1.0+ required for transaction tagging
  • Rust: 1.75.0+ for async trait support
  • Actix-web: 4.4+ for enhanced middleware
  • Tokio: 1.35+ for improved performance

↑ Back to Top

12. Approval Signatures​

Technical Sign-off​

ComponentOwnerApprovedDate
ArchitectureSession5✓2025-08-31
ImplementationPending--
Security ReviewPending--
Performance TestPending--

Implementation Checklist​

  • Audit event models implemented
  • Repository pattern with FDB integration
  • GDPR compliance endpoints
  • Retention policy engine
  • Integration tests passing
  • Performance benchmarks met
  • Security controls validated

↑ Back to Top