HSM Integration Architecture

Classification: Internal — Security Engineering Date: 2026-02-16 Status: Proposed Regulatory Mapping: FIPS 140-2 Level 3, FDA 21 CFR Part 11 §11.70, HIPAA §164.312

Document Control

Version History

Version	Date	Author	Changes
1.0.0	2026-02-16	Hal Casteel	Initial architecture specification

Approval

Role	Name	Date	Signature
Chief Technology Officer	Hal Casteel	2026-02-16	Pending
Security Architect	TBD	—	Pending
Compliance Officer	TBD	—	Pending
QA Lead	TBD	—	Pending

Revision History

This document will be reviewed and updated:

Annually (minimum)
Upon any cryptographic algorithm change
Upon any HSM provider change
Upon regulatory requirement changes
After any security incident involving key management

1. Executive Summary

1.1 Purpose

This document defines the Hardware Security Module (HSM) integration architecture for the CODITECT Biosciences QMS Platform (BIO-QMS). HSM integration provides hardware-backed cryptographic key protection required for FDA 21 CFR Part 11 compliance, HIPAA security controls, and SOC 2 Type II certification.

1.2 Scope

This architecture covers:

HSM provider selection and justification
Key hierarchy and lifecycle management
Integration patterns for signing, encryption, and certificate operations
Operational procedures for key generation, rotation, and revocation
Monitoring, audit, and compliance reporting
Migration from software-backed keys to HSM-backed keys

1.3 Current State vs. Target State

Current State (Software-Backed KMS):

Google Cloud KMS with software protection level
JWT signing keys: RSA-2048 (software-backed)
E-signature keys: SHA-256 via KMS (software-backed)
Data encryption: AES-256-GCM via KMS (software-backed)
Compliant for development/staging, not sufficient for FDA production validation

Target State (HSM-Backed KMS):

Google Cloud KMS with HSM protection level (FIPS 140-2 Level 3)
All cryptographic operations backed by tamper-resistant hardware
Regulatory-grade key protection with audit trail
Ready for FDA Part 11 validation and HIPAA BAA requirements

Why HSM?

Regulatory Mandate: FDA Part 11 §11.70 requires cryptographic binding of signatures to records; FIPS 140-2 Level 3 HSM is industry standard for validated systems
Key Protection: HSM keys are generated in hardware, never exportable, protected against physical and logical attacks
Tamper Resistance: Hardware enforces access controls; key material destruction on tamper detection
Audit Trail: All HSM operations logged with tamper-evident audit trail required for compliance
Trust Anchor: HSM serves as root of trust for entire platform cryptographic operations

2. Architecture Overview

2.1 Conceptual Model

┌─────────────────────────────────────────────────────────────────────┐
│                         Application Layer                            │
│  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐   │
│  │ E-Signature│  │ Audit Chain│  │   Data     │  │    TLS     │   │
│  │  Service   │  │  Service   │  │ Encryption │  │ Certificate│   │
│  └─────┬──────┘  └─────┬──────┘  └─────┬──────┘  └─────┬──────┘   │
└────────┼───────────────┼───────────────┼───────────────┼───────────┘
         │               │               │               │
         └───────────────┴───────────────┴───────────────┘
                          │
         ┌────────────────▼────────────────┐
         │   Google Cloud KMS API Layer    │
         │   (unified interface)           │
         └────────────────┬────────────────┘
                          │
         ┌────────────────▼────────────────┐
         │      HSM Protection Level       │
         │  (FIPS 140-2 Level 3 Certified) │
         │                                 │
         │  ┌────────────────────────────┐ │
         │  │ Cavium/Marvell LiquidSec   │ │
         │  │ Hardware Security Module   │ │
         │  │                            │ │
         │  │ • Key Generation (in HW)   │ │
         │  │ • Key Storage (encrypted)  │ │
         │  │ • Crypto Operations        │ │
         │  │ • Access Control           │ │
         │  │ • Tamper Detection         │ │
         │  └────────────────────────────┘ │
         └─────────────────────────────────┘

2.2 Key Benefits of Google Cloud HSM

Benefit	Description
Native GCP Integration	Same Cloud KMS API; no code changes, only protection level parameter
FIPS 140-2 Level 3	Hardware certified for regulated industries (FDA, HIPAA, FedRAMP)
Managed Service	No HSM cluster management; Google handles HA, patching, capacity
Regional Availability	Multi-region deployment for disaster recovery
Cost Efficiency	Pay per key version + operations; no dedicated HSM instance costs
Automatic Backups	Key material encrypted and backed up across multiple facilities

3. HSM Provider Selection

3.1 Primary: Google Cloud HSM

Recommendation: Google Cloud HSM (Cloud KMS with HSM protection level)

Justification:

BIO-QMS already deployed on Google Cloud Platform (GKE, Cloud SQL, Cloud Storage)
Same Cloud KMS API as current implementation; only protectionLevel: "HSM" change
FIPS 140-2 Level 3 certified (Cavium LiquidSecurity HSMs)
Multi-region availability (us-central1, us-east1, us-west1, europe-west1)
Lower operational overhead than dedicated HSM clusters
Google manages HSM firmware updates, capacity planning, disaster recovery

Certification:

FIPS 140-2 Level 3 (Certificate #3254)
Common Criteria EAL4+ certified
PCI-DSS compliant for payment card data
FedRAMP High authorized

Pricing Model:

Key version storage: $2.50/month per active key version
HSM operations: $2.50 per 10,000 operations
No upfront costs, no dedicated hardware fees

Regional Strategy:

Primary region: us-central1 (Iowa) — lowest latency to GKE cluster
DR region: us-east1 (South Carolina) — cross-region key replication
Key version history replicated automatically

3.2 Alternative: AWS CloudHSM

Use Case: Multi-cloud deployment or customer-specific requirement

Characteristics:

FIPS 140-2 Level 3 certified (Cavium Luna SA series)
Dedicated HSM instances in customer VPC
Full control over HSM configuration and key hierarchy
Higher operational overhead (cluster management, patching, monitoring)

Pricing Model:

$1.60/hour per HSM instance (~$1,168/month)
Minimum 2 instances for HA (~$2,336/month baseline)
No per-operation fees

Trade-offs:

Pros: Direct HSM access, PKCS#11 interface, no Google dependency
Cons: Higher cost, operational burden, VPC integration complexity, cross-region replication manual

Recommendation: Only consider AWS CloudHSM if:

Customer mandates AWS-specific deployment
PKCS#11 interface required for third-party integration
Dedicated HSM cluster control required for compliance

3.3 Decision Matrix

Criterion	Google Cloud HSM	AWS CloudHSM	Weight	Winner
FIPS 140-2 Level 3	✅ Yes	✅ Yes	10	Tie
GCP Native Integration	✅ Native	❌ Cross-cloud	9	Google
Operational Overhead	✅ Managed	⚠️ Customer-managed	8	Google
Cost (baseline)	✅ ~$200/mo	❌ ~$2,336/mo	7	Google
Multi-region DR	✅ Automatic	⚠️ Manual setup	6	Google
API Simplicity	✅ Cloud KMS	⚠️ PKCS#11/JCE	5	Google
Total Score	—	—	—	Google

Selected Provider: Google Cloud HSM (Cloud KMS with HSM protection level)

4. Key Hierarchy Design

4.1 Hierarchical Structure

┌──────────────────────────────────────────────────────────────────┐
│                   Root Key Encryption Key (KEK-0)                 │
│         Generated in HSM, NEVER exportable, NEVER rotated         │
│               Used only to protect secondary KEKs                 │
└────────────────────────────┬─────────────────────────────────────┘
                             │
        ┌────────────────────┼────────────────────┐
        ▼                    ▼                    ▼
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│ Signing KEK  │    │ Encryption   │    │   TLS CA     │
│   (KEK-1)    │    │  KEK (KEK-2) │    │  KEK (KEK-3) │
│              │    │              │    │              │
│ Protects     │    │ Protects     │    │ Protects     │
│ signing keys │    │ data keys    │    │ cert keys    │
└──────┬───────┘    └──────┬───────┘    └──────┬───────┘
       │                   │                   │
       ├─────────┬─────────┼─────────┬─────────┼─────────┐
       ▼         ▼         ▼         ▼         ▼         ▼
   ┌──────┐ ┌──────┐  ┌──────┐ ┌──────┐  ┌──────┐ ┌──────┐
   │E-Sig │ │Audit │  │ DEK  │ │Field │  │ TLS  │ │Code  │
   │ Key  │ │Chain │  │(Data)│ │Encrypt│  │Cert  │ │Sign  │
   └──────┘ └──────┘  └──────┘ └──────┘  └──────┘ └──────┘

4.2 Key Hierarchy Principles

Principle	Implementation
Separation of Duties	Signing keys, encryption keys, and certificate keys use separate KEKs
Defense in Depth	Multi-tier hierarchy; compromise of DEK does not expose KEK
Key Versioning	All keys support versioning; old versions retained for verification
Rotation Independence	Each key rotates independently; no cascade rotation required
Envelope Encryption	Data Encryption Keys (DEK) encrypted by KEK; DEK can be software-generated

4.3 Key Types and Algorithms

Key Type	Purpose	Algorithm	Key Size	HSM-Backed	Rotation Period
Root KEK-0	Protect secondary KEKs	AES-256-GCM	256-bit	✅ Yes	Never (immutable)
Signing KEK-1	Protect signing keys	AES-256-GCM	256-bit	✅ Yes	Annual
Encryption KEK-2	Protect data encryption keys	AES-256-GCM	256-bit	✅ Yes	Annual
TLS CA KEK-3	Protect certificate signing keys	AES-256-GCM	256-bit	✅ Yes	3-year
E-Signature Key	Sign approval records	ECDSA P-256 (target) SHA-256 HMAC (current)	256-bit	✅ Yes	Annual (versioned)
Audit Chain Key	Sign audit trail entries	HMAC-SHA256	256-bit	✅ Yes	Never (chain integrity)
JWT Signing Key	Sign authentication tokens	RSA-2048	2048-bit	✅ Yes	90-day
Data Encryption DEK	Encrypt database fields	AES-256-GCM	256-bit	❌ No (envelope)	Daily (ephemeral)
TLS Certificate Key	Service identity	ECDSA P-256	256-bit	✅ Yes	90-day (auto-renewal)
Code Signing Key	Sign deployment artifacts	RSA-3072	3072-bit	✅ Yes	Annual

Algorithm Selection Rationale:

ECDSA P-256: FDA 21 CFR Part 11 best practice; NIST-approved; smaller signatures than RSA
RSA-2048/3072: Industry standard for JWT and code signing; wider compatibility
AES-256-GCM: FIPS-approved symmetric encryption; authenticated encryption
HMAC-SHA256: Audit chain integrity; faster than asymmetric signatures

5. HSM Key Inventory

5.1 Production Keys (All HSM-Backed)

Key Name	Algorithm	Purpose	Protection	Rotation	Access Roles	Key Version Policy
`bioqms-root-kek-0`	AES-256-GCM	Root key encryption key	HSM	Never	`hsm-admin`	Immutable (single version)
`bioqms-signing-kek-1`	AES-256-GCM	Protects signing keys	HSM	Annual	`hsm-admin`, `key-rotator`	Retain all versions
`bioqms-encryption-kek-2`	AES-256-GCM	Protects data keys	HSM	Annual	`hsm-admin`, `key-rotator`	Retain all versions
`bioqms-tls-ca-kek-3`	AES-256-GCM	Protects TLS CA	HSM	3-year	`hsm-admin`, `cert-manager`	Retain all versions
`bioqms-esig-signing-key`	ECDSA P-256	E-signature operations	HSM	Annual	`esig-signer`	Retain all (verification)
`bioqms-audit-chain-key`	HMAC-SHA256	Audit trail chain	HSM	Never	`audit-writer`	Single version (chain)
`bioqms-jwt-signing-key`	RSA-2048	JWT token signing	HSM	90-day	`jwt-issuer`	Retain 2 versions
`bioqms-tls-ca-key`	ECDSA P-256	Issue service certs	HSM	3-year	`cert-manager`	Retain all versions
`bioqms-code-signing-key`	RSA-3072	Sign deployment artifacts	HSM	Annual	`cicd-signer`	Retain all versions

5.2 Key Access Control Matrix

IAM Role	`root-kek-0`	`signing-kek-1`	`encryption-kek-2`	`tls-ca-kek-3`	`esig-key`	`audit-key`	`jwt-key`	`tls-ca`	`code-key`
`hsm-admin`	Create, Disable	Create, Disable	Create, Disable	Create, Disable	—	—	—	—	—
`key-rotator`	—	Rotate	Rotate	—	Rotate	—	Rotate	—	Rotate
`esig-signer`	—	—	—	—	Sign	—	—	—	—
`audit-writer`	—	—	—	—	—	Sign	—	—	—
`jwt-issuer`	—	—	—	—	—	—	Sign	—	—
`cert-manager`	—	—	—	Rotate	—	—	—	Sign	—
`cicd-signer`	—	—	—	—	—	—	—	—	Sign
`key-auditor`	View	View	View	View	View	View	View	View	View

Separation of Duties:

hsm-admin can create/disable keys but cannot use them for operations (no Sign permission)
key-rotator can rotate keys but cannot create new key types
Service accounts have Sign/Encrypt on specific keys only
key-auditor has read-only access for compliance review

6. Access Control Model

6.1 IAM Roles and Permissions

# hsm-admin role
roles/cloudkms.admin:
  permissions:
    - cloudkms.cryptoKeyVersions.create
    - cloudkms.cryptoKeys.create
    - cloudkms.cryptoKeys.update
    - cloudkms.cryptoKeyVersions.destroy
  bindings:
    - user:security-admin@bioqms.com
  deny:
    - cloudkms.cryptoKeyVersions.useToSign
    - cloudkms.cryptoKeyVersions.useToEncrypt

# key-rotator role (automated service account)
roles/cloudkms.keyRotator:
  permissions:
    - cloudkms.cryptoKeyVersions.create
    - cloudkms.cryptoKeys.update
  bindings:
    - serviceAccount:key-rotator@bioqms-prod.iam.gserviceaccount.com
  conditions:
    - expression: "request.time < timestamp('2027-01-01T00:00:00Z')"
      description: "Time-bound rotation permission"

# esig-signer role (e-signature service)
roles/cloudkms.signerVerifier:
  permissions:
    - cloudkms.cryptoKeyVersions.useToSign
    - cloudkms.cryptoKeyVersions.useToVerify
  bindings:
    - serviceAccount:esig-service@bioqms-prod.iam.gserviceaccount.com
  resourceConditions:
    - key: "bioqms-esig-signing-key" only

# audit-writer role (audit service)
roles/cloudkms.signer:
  permissions:
    - cloudkms.cryptoKeyVersions.useToSign
  bindings:
    - serviceAccount:audit-service@bioqms-prod.iam.gserviceaccount.com
  resourceConditions:
    - key: "bioqms-audit-chain-key" only

# key-auditor role (compliance review)
roles/cloudkms.viewer:
  permissions:
    - cloudkms.cryptoKeys.get
    - cloudkms.cryptoKeyVersions.list
    - cloudkms.cryptoKeyVersions.viewPublicKey
  bindings:
    - group:compliance@bioqms.com

6.2 Break-Glass Access

Scenario: Production key needs emergency rotation due to suspected compromise.

Procedure:

Security incident declared (P1 severity)
Security admin requests break-glass access via PagerDuty
Dual-approval required:
- CTO approval (Hal Casteel)
- Compliance Officer approval
Break-glass IAM role activated (4-hour TTL)
Emergency key rotation performed
All actions logged with enhanced audit trail
Mandatory post-incident review within 72 hours

Break-Glass Role:

roles/cloudkms.emergencyAdmin:
  permissions:
    - cloudkms.cryptoKeyVersions.destroy
    - cloudkms.cryptoKeys.setIamPolicy
  bindings:
    - user:security-admin@bioqms.com
  conditions:
    - expression: "request.time < timestamp.now() + duration('4h')"
    - requireDualApproval: true
  auditConfig:
    - logType: DATA_WRITE
    - logType: DATA_READ
    - exemptedMembers: [] # No exemptions

6.3 Service Account Bindings

Service	GCP Service Account	HSM Keys Accessed	Permission	Justification
E-Signature Service	`esig-service@bioqms-prod.iam`	`bioqms-esig-signing-key`	Sign, Verify	Sign approval records per FDA Part 11
Audit Service	`audit-service@bioqms-prod.iam`	`bioqms-audit-chain-key`	Sign	Hash chain integrity
API Gateway	`api-gateway@bioqms-prod.iam`	`bioqms-jwt-signing-key`	Sign, Verify	Issue and validate JWTs
Certificate Manager	`cert-manager@bioqms-prod.iam`	`bioqms-tls-ca-key`	Sign	Issue service TLS certificates
CI/CD Pipeline	`cicd-pipeline@bioqms-prod.iam`	`bioqms-code-signing-key`	Sign	Sign container images and Helm charts
Data Encryption Service	`data-encrypt@bioqms-prod.iam`	`bioqms-encryption-kek-2`	Encrypt, Decrypt	Envelope encryption for L3/L4 fields

Least Privilege:

Each service account scoped to specific key (resource-level IAM binding)
No service account has access to root-kek-0 or signing/encryption-kek keys
Service accounts cannot create or destroy key versions

7. Integration Architecture

7.1 Application Layer Integration

TypeScript SDK Pattern (Node.js Services):

import { KeyManagementServiceClient } from '@google-cloud/kms';

interface HSMConfig {
  projectId: string;
  locationId: string;
  keyRingId: string;
  keyId: string;
  keyVersion?: string; // Optional: defaults to latest
}

class HSMClient {
  private kmsClient: KeyManagementServiceClient;

  constructor() {
    this.kmsClient = new KeyManagementServiceClient({
      // Uses Application Default Credentials (service account key)
    });
  }

  /**
   * Sign data with HSM-backed key (e.g., e-signature)
   * @param data - Buffer to sign
   * @param keyConfig - HSM key configuration
   * @returns Signature buffer
   */
  async signWithHSM(
    data: Buffer,
    keyConfig: HSMConfig
  ): Promise<Buffer> {
    const keyVersionName = this.getKeyVersionName(keyConfig);

    // Compute SHA-256 digest (required for asymmetric signing)
    const crypto = await import('crypto');
    const digest = crypto.createHash('sha256').update(data).digest();

    const [signResponse] = await this.kmsClient.asymmetricSign({
      name: keyVersionName,
      digest: {
        sha256: digest,
      },
    });

    // Log HSM operation for audit
    await this.logHSMOperation({
      operation: 'SIGN',
      keyName: keyConfig.keyId,
      timestamp: new Date(),
      success: true,
    });

    return Buffer.from(signResponse.signature as Uint8Array);
  }

  /**
   * Verify signature with HSM public key
   * @param data - Original data buffer
   * @param signature - Signature to verify
   * @param keyConfig - HSM key configuration
   * @returns True if signature valid
   */
  async verifyWithHSM(
    data: Buffer,
    signature: Buffer,
    keyConfig: HSMConfig
  ): Promise<boolean> {
    const keyVersionName = this.getKeyVersionName(keyConfig);

    // Get public key from HSM
    const [publicKeyResponse] = await this.kmsClient.getPublicKey({
      name: keyVersionName,
    });

    // Compute digest
    const crypto = await import('crypto');
    const digest = crypto.createHash('sha256').update(data).digest();

    // Verify locally (no HSM operation needed for verification)
    const verifier = crypto.createVerify('SHA256');
    verifier.update(data);
    verifier.end();

    return verifier.verify(
      publicKeyResponse.pem as string,
      signature
    );
  }

  /**
   * Envelope encryption: Encrypt data with DEK, encrypt DEK with HSM KEK
   * @param plaintext - Data to encrypt
   * @param kekConfig - Key Encryption Key configuration
   * @returns Encrypted envelope (DEK + ciphertext)
   */
  async envelopeEncrypt(
    plaintext: Buffer,
    kekConfig: HSMConfig
  ): Promise<EncryptedEnvelope> {
    // Step 1: Generate ephemeral Data Encryption Key (DEK)
    const crypto = await import('crypto');
    const dek = crypto.randomBytes(32); // AES-256 key

    // Step 2: Encrypt plaintext with DEK (AES-256-GCM)
    const iv = crypto.randomBytes(12); // GCM nonce
    const cipher = crypto.createCipheriv('aes-256-gcm', dek, iv);
    const ciphertext = Buffer.concat([
      cipher.update(plaintext),
      cipher.final(),
    ]);
    const authTag = cipher.getAuthTag();

    // Step 3: Encrypt DEK with HSM KEK
    const keyVersionName = this.getKeyVersionName(kekConfig);
    const [encryptResponse] = await this.kmsClient.encrypt({
      name: keyVersionName,
      plaintext: dek,
    });

    // Return envelope
    return {
      encryptedDEK: Buffer.from(encryptResponse.ciphertext as Uint8Array),
      ciphertext,
      iv,
      authTag,
      algorithm: 'AES-256-GCM',
      kekKeyId: kekConfig.keyId,
    };
  }

  /**
   * Envelope decryption: Decrypt DEK with HSM KEK, decrypt data with DEK
   * @param envelope - Encrypted envelope
   * @param kekConfig - Key Encryption Key configuration
   * @returns Decrypted plaintext
   */
  async envelopeDecrypt(
    envelope: EncryptedEnvelope,
    kekConfig: HSMConfig
  ): Promise<Buffer> {
    // Step 1: Decrypt DEK with HSM KEK
    const keyVersionName = this.getKeyVersionName(kekConfig);
    const [decryptResponse] = await this.kmsClient.decrypt({
      name: keyVersionName,
      ciphertext: envelope.encryptedDEK,
    });
    const dek = Buffer.from(decryptResponse.plaintext as Uint8Array);

    // Step 2: Decrypt ciphertext with DEK (AES-256-GCM)
    const crypto = await import('crypto');
    const decipher = crypto.createDecipheriv(
      'aes-256-gcm',
      dek,
      envelope.iv
    );
    decipher.setAuthTag(envelope.authTag);

    const plaintext = Buffer.concat([
      decipher.update(envelope.ciphertext),
      decipher.final(),
    ]);

    // Zero out DEK in memory
    dek.fill(0);

    return plaintext;
  }

  private getKeyVersionName(config: HSMConfig): string {
    const { projectId, locationId, keyRingId, keyId, keyVersion } = config;
    const baseKeyName = `projects/${projectId}/locations/${locationId}/keyRings/${keyRingId}/cryptoKeys/${keyId}`;

    // Use specified version or latest
    return keyVersion
      ? `${baseKeyName}/cryptoKeyVersions/${keyVersion}`
      : baseKeyName; // Cloud KMS API resolves to primary version
  }

  private async logHSMOperation(event: HSMOperationEvent): Promise<void> {
    // Send to audit trail service
    await fetch('http://audit-service/api/v1/hsm-events', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(event),
    });
  }
}

interface EncryptedEnvelope {
  encryptedDEK: Buffer;   // DEK encrypted by HSM KEK
  ciphertext: Buffer;     // Data encrypted by DEK
  iv: Buffer;             // AES-GCM initialization vector
  authTag: Buffer;        // AES-GCM authentication tag
  algorithm: string;      // 'AES-256-GCM'
  kekKeyId: string;       // KEK identifier for decryption
}

interface HSMOperationEvent {
  operation: 'SIGN' | 'VERIFY' | 'ENCRYPT' | 'DECRYPT';
  keyName: string;
  timestamp: Date;
  success: boolean;
  error?: string;
}

export { HSMClient, HSMConfig, EncryptedEnvelope };

7.2 E-Signature Integration Flow

/**
 * E-Signature workflow with HSM-backed signing
 * Implements FDA 21 CFR Part 11 §11.70 cryptographic binding
 */
class ESignatureService {
  private hsmClient: HSMClient;

  async createSignature(
    signerId: string,
    workOrderId: string,
    meaning: string
  ): Promise<ElectronicSignature> {
    // Step 1: Create signature record
    const signature = await prisma.electronicSignature.create({
      data: {
        signerId,
        meaning,
        signedAt: new Date(),
        consumed: false,
      },
    });

    // Step 2: Construct canonical signing data (§11.70 binding)
    const signingData = {
      signatureId: signature.id,
      signerId,
      workOrderId,
      meaning,
      signedAt: signature.signedAt.toISOString(),
    };
    const signingBuffer = Buffer.from(
      JSON.stringify(signingData, Object.keys(signingData).sort())
    );

    // Step 3: Sign with HSM-backed key
    const hsmSignature = await this.hsmClient.signWithHSM(
      signingBuffer,
      {
        projectId: 'bioqms-prod',
        locationId: 'us-central1',
        keyRingId: 'bioqms-keyring',
        keyId: 'bioqms-esig-signing-key',
        // keyVersion: 'latest' (default)
      }
    );

    // Step 4: Store cryptographic hash binding
    await prisma.electronicSignature.update({
      where: { id: signature.id },
      data: {
        cryptoHash: hsmSignature.toString('base64'),
        cryptoAlgorithm: 'ECDSA-P256-SHA256',
      },
    });

    return signature;
  }

  async verifySignature(
    signatureId: string,
    workOrderId: string
  ): Promise<boolean> {
    const signature = await prisma.electronicSignature.findUnique({
      where: { id: signatureId },
    });
    if (!signature) return false;

    // Reconstruct signing data
    const signingData = {
      signatureId: signature.id,
      signerId: signature.signerId,
      workOrderId,
      meaning: signature.meaning,
      signedAt: signature.signedAt.toISOString(),
    };
    const signingBuffer = Buffer.from(
      JSON.stringify(signingData, Object.keys(signingData).sort())
    );

    // Verify with HSM public key
    const hsmSignature = Buffer.from(signature.cryptoHash, 'base64');
    const valid = await this.hsmClient.verifyWithHSM(
      signingBuffer,
      hsmSignature,
      {
        projectId: 'bioqms-prod',
        locationId: 'us-central1',
        keyRingId: 'bioqms-keyring',
        keyId: 'bioqms-esig-signing-key',
      }
    );

    return valid;
  }
}

7.3 TLS Certificate Issuance Flow

/**
 * Issue service TLS certificates signed by HSM-backed CA
 */
class TLSCertificateService {
  private hsmClient: HSMClient;

  async issueServiceCertificate(
    serviceName: string,
    dnsNames: string[]
  ): Promise<TLSCertificate> {
    // Step 1: Generate service key pair (ephemeral, not HSM)
    const crypto = await import('crypto');
    const { privateKey, publicKey } = crypto.generateKeyPairSync('ec', {
      namedCurve: 'prime256v1', // P-256
    });

    // Step 2: Create Certificate Signing Request (CSR)
    const forge = await import('node-forge');
    const csr = forge.pki.createCertificationRequest();
    csr.publicKey = forge.pki.publicKeyFromPem(
      publicKey.export({ type: 'spki', format: 'pem' }) as string
    );
    csr.setSubject([
      { name: 'commonName', value: serviceName },
      { name: 'organizationName', value: 'CODITECT Biosciences' },
    ]);
    csr.setAttributes([
      {
        name: 'extensionRequest',
        extensions: [
          {
            name: 'subjectAltName',
            altNames: dnsNames.map(dns => ({
              type: 2, // DNS
              value: dns,
            })),
          },
        ],
      },
    ]);

    // Step 3: Sign CSR with HSM-backed CA key
    const csrDer = Buffer.from(
      forge.asn1.toDer(forge.pki.certificationRequestToAsn1(csr)).getBytes(),
      'binary'
    );

    const caSignature = await this.hsmClient.signWithHSM(
      csrDer,
      {
        projectId: 'bioqms-prod',
        locationId: 'us-central1',
        keyRingId: 'bioqms-keyring',
        keyId: 'bioqms-tls-ca-key',
      }
    );

    // Step 4: Construct X.509 certificate
    const cert = forge.pki.createCertificate();
    cert.publicKey = csr.publicKey;
    cert.serialNumber = crypto.randomBytes(16).toString('hex');
    cert.validity.notBefore = new Date();
    cert.validity.notAfter = new Date();
    cert.validity.notAfter.setDate(cert.validity.notBefore.getDate() + 90);
    cert.setSubject(csr.subject.attributes);
    cert.setIssuer([
      { name: 'commonName', value: 'CODITECT BIO-QMS CA' },
    ]);
    cert.setExtensions([
      { name: 'basicConstraints', cA: false },
      { name: 'keyUsage', digitalSignature: true, keyEncipherment: true },
      { name: 'extKeyUsage', serverAuth: true },
      {
        name: 'subjectAltName',
        altNames: dnsNames.map(dns => ({ type: 2, value: dns })),
      },
    ]);

    // Attach HSM signature
    cert.signature = caSignature.toString('binary');

    // Step 5: Store certificate + private key in Secret Manager
    const certPem = forge.pki.certificateToPem(cert);
    const keyPem = privateKey.export({ type: 'pkcs8', format: 'pem' });

    await this.storeServiceCertificate(serviceName, certPem, keyPem as string);

    return { certPem, keyPem: keyPem as string, expiresAt: cert.validity.notAfter };
  }

  private async storeServiceCertificate(
    serviceName: string,
    cert: string,
    key: string
  ): Promise<void> {
    // Store in Google Secret Manager (encrypted at rest)
    const { SecretManagerServiceClient } = await import('@google-cloud/secret-manager');
    const client = new SecretManagerServiceClient();

    await client.addSecretVersion({
      parent: `projects/bioqms-prod/secrets/${serviceName}-tls-cert`,
      payload: {
        data: Buffer.from(cert),
      },
    });

    await client.addSecretVersion({
      parent: `projects/bioqms-prod/secrets/${serviceName}-tls-key`,
      payload: {
        data: Buffer.from(key),
      },
    });
  }
}

interface TLSCertificate {
  certPem: string;
  keyPem: string;
  expiresAt: Date;
}

8. Operational Procedures

8.1 Key Generation Ceremony

Purpose: Document the secure generation of HSM root keys in a controlled, witnessed process.

Participants:

Security Administrator (Key Creator)
Compliance Officer (Witness 1)
CTO (Witness 2)
QA Lead (Documentation)

Procedure:

Pre-Ceremony Checklist:
- All participants present (minimum 3)
- Video recording active
- HSM admin credentials available
- Key naming convention confirmed
- Dual-approval process verified

Root KEK Generation:

# Create key ring (one-time)
gcloud kms keyrings create bioqms-keyring \
  --location us-central1 \
  --project bioqms-prod

# Create Root KEK-0 (NEVER rotated)
gcloud kms keys create bioqms-root-kek-0 \
  --location us-central1 \
  --keyring bioqms-keyring \
  --purpose encryption \
  --protection-level hsm \
  --rotation-period never \
  --labels environment=production,key-tier=root,compliance=fda-part11

# Verify HSM backing
gcloud kms keys describe bioqms-root-kek-0 \
  --location us-central1 \
  --keyring bioqms-keyring \
  | grep "protectionLevel: HSM"

Secondary KEK Generation:

# Signing KEK-1
gcloud kms keys create bioqms-signing-kek-1 \
  --location us-central1 \
  --keyring bioqms-keyring \
  --purpose encryption \
  --protection-level hsm \
  --rotation-period 365d \
  --labels key-tier=kek,purpose=signing

# Encryption KEK-2
gcloud kms keys create bioqms-encryption-kek-2 \
  --location us-central1 \
  --keyring bioqms-keyring \
  --purpose encryption \
  --protection-level hsm \
  --rotation-period 365d \
  --labels key-tier=kek,purpose=encryption

# TLS CA KEK-3
gcloud kms keys create bioqms-tls-ca-kek-3 \
  --location us-central1 \
  --keyring bioqms-keyring \
  --purpose encryption \
  --protection-level hsm \
  --rotation-period 1095d \
  --labels key-tier=kek,purpose=tls-ca

Operational Key Generation:

# E-Signature Key (asymmetric signing)
gcloud kms keys create bioqms-esig-signing-key \
  --location us-central1 \
  --keyring bioqms-keyring \
  --purpose asymmetric-signing \
  --protection-level hsm \
  --default-algorithm ec-sign-p256-sha256 \
  --rotation-period 365d \
  --labels purpose=esignature,compliance=fda-part11-70

# Audit Chain Key (symmetric signing)
gcloud kms keys create bioqms-audit-chain-key \
  --location us-central1 \
  --keyring bioqms-keyring \
  --purpose mac \
  --protection-level hsm \
  --default-algorithm hmac-sha256 \
  --rotation-period never \
  --labels purpose=audit-trail,compliance=hipaa-312c2

# JWT Signing Key
gcloud kms keys create bioqms-jwt-signing-key \
  --location us-central1 \
  --keyring bioqms-keyring \
  --purpose asymmetric-signing \
  --protection-level hsm \
  --default-algorithm rsa-sign-pkcs1-2048-sha256 \
  --rotation-period 90d \
  --labels purpose=jwt-authentication

Post-Generation Verification:
- All keys show protectionLevel: HSM
- Key version IDs recorded in inventory
- IAM bindings applied (separation of duties)
- Audit log entries verified in Cloud Logging
- Ceremony documentation signed by all participants
Ceremony Documentation:
- Video recording archived (encrypted, 7-year retention)
- Participant signatures on Key Generation Record
- Key inventory updated with key version IDs
- Change management ticket closed with evidence

Frequency:

Root KEK-0: Once (never rotated)
Secondary KEKs: Annual (automated rotation after initial ceremony)
Operational keys: Automated rotation (ceremony not required)

8.2 Key Rotation Automation

Automated Rotation Strategy:

# Cloud Scheduler job for automated key rotation
apiVersion: cloud.google.com/v1
kind: SchedulerJob
metadata:
  name: hsm-key-rotation
spec:
  schedule: "0 2 * * 0" # Weekly on Sunday 2 AM UTC
  target:
    cloudFunction:
      name: rotateHSMKeys
      project: bioqms-prod
      region: us-central1
  retry:
    retryCount: 3
    minBackoffDuration: 5m

Rotation Function (Node.js Cloud Function):

import { KeyManagementServiceClient } from '@google-cloud/kms';
import { SecretManagerServiceClient } from '@google-cloud/secret-manager';

interface KeyRotationConfig {
  keyId: string;
  rotationPeriod: number; // days
  retainVersions: number;
}

const ROTATION_CONFIGS: KeyRotationConfig[] = [
  { keyId: 'bioqms-jwt-signing-key', rotationPeriod: 90, retainVersions: 2 },
  { keyId: 'bioqms-esig-signing-key', rotationPeriod: 365, retainVersions: -1 }, // Retain all
  { keyId: 'bioqms-code-signing-key', rotationPeriod: 365, retainVersions: -1 },
];

export async function rotateHSMKeys(): Promise<void> {
  const kmsClient = new KeyManagementServiceClient();
  const projectId = 'bioqms-prod';
  const locationId = 'us-central1';
  const keyRingId = 'bioqms-keyring';

  for (const config of ROTATION_CONFIGS) {
    const keyName = `projects/${projectId}/locations/${locationId}/keyRings/${keyRingId}/cryptoKeys/${config.keyId}`;

    // Check if rotation needed
    const [key] = await kmsClient.getCryptoKey({ name: keyName });
    const primaryVersion = key.primary;
    const createdAt = new Date(primaryVersion?.createTime?.seconds || 0);
    const ageInDays = (Date.now() - createdAt.getTime()) / (1000 * 60 * 60 * 24);

    if (ageInDays >= config.rotationPeriod) {
      console.log(`Rotating ${config.keyId} (${Math.floor(ageInDays)} days old)`);

      // Create new key version
      const [newVersion] = await kmsClient.createCryptoKeyVersion({
        parent: keyName,
      });

      console.log(`Created new version: ${newVersion.name}`);

      // Clean up old versions if retention limit set
      if (config.retainVersions > 0) {
        await cleanupOldVersions(kmsClient, keyName, config.retainVersions);
      }

      // Notify compliance team
      await notifyRotation(config.keyId, newVersion.name);
    } else {
      console.log(`Skipping ${config.keyId} (${Math.floor(ageInDays)} days old, rotation at ${config.rotationPeriod})`);
    }
  }
}

async function cleanupOldVersions(
  client: KeyManagementServiceClient,
  keyName: string,
  retainCount: number
): Promise<void> {
  const [versions] = await client.listCryptoKeyVersions({ parent: keyName });
  const enabledVersions = versions
    .filter(v => v.state === 'ENABLED')
    .sort((a, b) => {
      const aTime = a.createTime?.seconds || 0;
      const bTime = b.createTime?.seconds || 0;
      return Number(bTime) - Number(aTime);
    });

  // Destroy versions beyond retention count (keep primary + retainCount)
  const toDestroy = enabledVersions.slice(retainCount + 1);
  for (const version of toDestroy) {
    console.log(`Destroying old version: ${version.name}`);
    await client.destroyCryptoKeyVersion({ name: version.name! });
  }
}

async function notifyRotation(keyId: string, newVersionName: string): Promise<void> {
  // Send to audit service + compliance notification
  await fetch('http://audit-service/api/v1/hsm-events', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      event: 'KEY_ROTATED',
      keyId,
      newVersion: newVersionName,
      timestamp: new Date().toISOString(),
    }),
  });
}

8.3 Key Revocation Procedure

Trigger Scenarios:

Suspected key compromise
Service account key leaked
Insider threat detection
Regulatory audit finding

Immediate Revocation (P1 Incident):

# Step 1: Disable compromised key version immediately
gcloud kms keys versions disable <VERSION_ID> \
  --key bioqms-esig-signing-key \
  --keyring bioqms-keyring \
  --location us-central1 \
  --project bioqms-prod

# Step 2: Rotate to new version
gcloud kms keys versions create \
  --key bioqms-esig-signing-key \
  --keyring bioqms-keyring \
  --location us-central1 \
  --project bioqms-prod \
  --primary

# Step 3: Verify new primary version active
gcloud kms keys describe bioqms-esig-signing-key \
  --keyring bioqms-keyring \
  --location us-central1 \
  --project bioqms-prod

# Step 4: Audit all operations using compromised version
gcloud logging read "resource.type=cloudkms_cryptokeyversion
  AND protoPayload.resourceName=projects/bioqms-prod/locations/us-central1/keyRings/bioqms-keyring/cryptoKeys/bioqms-esig-signing-key/cryptoKeyVersions/<VERSION_ID>" \
  --project bioqms-prod \
  --format json > compromised_key_audit.json

Scheduled Revocation (e.g., employee departure):

# Schedule key version destruction (72-hour grace period)
gcloud kms keys versions schedule-destroy <VERSION_ID> \
  --key bioqms-jwt-signing-key \
  --keyring bioqms-keyring \
  --location us-central1 \
  --project bioqms-prod \
  --destroy-scheduled-duration 72h

# Revoke IAM binding immediately
gcloud kms keys remove-iam-policy-binding bioqms-jwt-signing-key \
  --keyring bioqms-keyring \
  --location us-central1 \
  --project bioqms-prod \
  --member serviceAccount:departing-employee@bioqms-prod.iam.gserviceaccount.com \
  --role roles/cloudkms.signerVerifier

8.4 Key Destruction

Cryptographic Zeroization:

Cloud KMS HSM performs FIPS 140-2 compliant key zeroization
Key material overwritten with zeros in HSM volatile memory
Key metadata retained for audit trail (state = DESTROYED)

Grace Period:

72-hour grace period before destruction (prevents accidental deletion)
During grace period: key state = DESTROY_SCHEDULED
Cancellation possible: gcloud kms keys versions restore <VERSION_ID>

Permanent Destruction:

# After 72-hour grace period expires, key permanently destroyed
gcloud kms keys versions destroy <VERSION_ID> \
  --key bioqms-esig-signing-key \
  --keyring bioqms-keyring \
  --location us-central1 \
  --project bioqms-prod

Audit Record:

Destruction event logged to Cloud Audit Logs
Retained for 7 years (regulatory compliance)
Key version ID retained in inventory as DESTROYED state

8.5 Disaster Recovery

Cross-Region Replication:

# Replicate keyring to DR region (us-east1)
gcloud kms keyrings create bioqms-keyring \
  --location us-east1 \
  --project bioqms-prod

# Replicate each key to DR region
for key in bioqms-esig-signing-key bioqms-jwt-signing-key bioqms-audit-chain-key; do
  gcloud kms keys create $key \
    --location us-east1 \
    --keyring bioqms-keyring \
    --purpose asymmetric-signing \
    --protection-level hsm \
    --default-algorithm ec-sign-p256-sha256 \
    --project bioqms-prod
done

Failover Procedure:

Detect primary region outage (GKE cluster unhealthy)
Update DNS to point to DR region (us-east1)

Update HSM client configuration:

const hsmConfig: HSMConfig = {
  projectId: 'bioqms-prod',
  locationId: process.env.REGION_FAILOVER === 'true' ? 'us-east1' : 'us-central1',
  keyRingId: 'bioqms-keyring',
  keyId: 'bioqms-esig-signing-key',
};

Verify signature verification works with DR keys
Monitor Cloud Audit Logs in DR region

Recovery Time Objective (RTO): < 15 minutes Recovery Point Objective (RPO): 0 (key versions replicated immediately)

9. Monitoring and Audit

9.1 Cloud Audit Logs for HSM Operations

Log Types:

Admin Activity: Key creation, rotation, destruction, IAM changes
Data Access: Sign, verify, encrypt, decrypt operations
System Events: HSM health, key version state changes

Key Audit Log Queries:

# All HSM sign operations (last 7 days)
gcloud logging read "resource.type=cloudkms_cryptokeyversion
  AND protoPayload.methodName=google.cloud.kms.v1.KeyManagementService.AsymmetricSign
  AND timestamp >= \"$(date -u -v-7d +%Y-%m-%dT%H:%M:%SZ)\"" \
  --project bioqms-prod \
  --format json

# Unauthorized access attempts
gcloud logging read "resource.type=cloudkms_cryptokeyversion
  AND protoPayload.status.code != 0
  AND protoPayload.status.code != null" \
  --project bioqms-prod \
  --format json

# Key rotation events
gcloud logging read "resource.type=cloudkms_cryptokey
  AND protoPayload.methodName=google.cloud.kms.v1.KeyManagementService.CreateCryptoKeyVersion" \
  --project bioqms-prod \
  --format json

# IAM policy changes on HSM keys
gcloud logging read "resource.type=cloudkms_cryptokey
  AND protoPayload.methodName=SetIamPolicy" \
  --project bioqms-prod \
  --format json

9.2 Key Usage Metrics (Prometheus/Grafana)

Custom Metrics from Application Layer:

import { register, Counter, Histogram } from 'prom-client';

const hsmSignCounter = new Counter({
  name: 'bioqms_hsm_sign_operations_total',
  help: 'Total HSM sign operations',
  labelNames: ['key_id', 'status'],
});

const hsmSignLatency = new Histogram({
  name: 'bioqms_hsm_sign_latency_seconds',
  help: 'HSM sign operation latency',
  labelNames: ['key_id'],
  buckets: [0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0],
});

class HSMClientWithMetrics extends HSMClient {
  async signWithHSM(data: Buffer, keyConfig: HSMConfig): Promise<Buffer> {
    const end = hsmSignLatency.startTimer({ key_id: keyConfig.keyId });

    try {
      const signature = await super.signWithHSM(data, keyConfig);
      hsmSignCounter.inc({ key_id: keyConfig.keyId, status: 'success' });
      return signature;
    } catch (error) {
      hsmSignCounter.inc({ key_id: keyConfig.keyId, status: 'error' });
      throw error;
    } finally {
      end();
    }
  }
}

Grafana Dashboard Panels:

Panel	Metric	Alert Threshold
HSM Sign Operations/min	`rate(bioqms_hsm_sign_operations_total[5m])`	> 100/min (anomaly)
HSM Sign Latency (p95)	`histogram_quantile(0.95, bioqms_hsm_sign_latency_seconds)`	> 50ms
HSM Error Rate	`rate(bioqms_hsm_sign_operations_total{status="error"}[5m])`	> 1%
Key Version Age	Custom query from Cloud KMS API	> 400 days (rotation overdue)
Unauthorized Access Attempts	Cloud Logging query count	> 0

9.3 Alert Rules

PagerDuty Integration:

# Alert: HSM sign latency exceeds SLA
- alert: HSMSignLatencyHigh
  expr: histogram_quantile(0.95, rate(bioqms_hsm_sign_latency_seconds_bucket[5m])) > 0.050
  for: 5m
  labels:
    severity: warning
    component: hsm
  annotations:
    summary: "HSM sign latency p95 > 50ms"
    description: "E-signature operations may be degraded"
    runbook: "https://wiki.bioqms.com/runbooks/hsm-latency"

# Alert: Unauthorized HSM access attempt
- alert: HSMUnauthorizedAccess
  expr: increase(bioqms_hsm_sign_operations_total{status="error"}[5m]) > 0
  labels:
    severity: critical
    component: hsm-security
  annotations:
    summary: "Unauthorized HSM access detected"
    description: "Potential security incident - investigate immediately"
    runbook: "https://wiki.bioqms.com/runbooks/hsm-security-incident"

# Alert: Key rotation overdue
- alert: HSMKeyRotationOverdue
  expr: (time() - kms_key_version_create_time_seconds) / 86400 > 400
  labels:
    severity: warning
    component: hsm-compliance
  annotations:
    summary: "HSM key rotation overdue"
    description: "Key {{ $labels.key_id }} is {{ $value }} days old (rotation at 365d)"
    runbook: "https://wiki.bioqms.com/runbooks/hsm-key-rotation"

9.4 Compliance Reporting

Monthly Key Inventory Report (Automated):

interface KeyInventoryReport {
  reportDate: Date;
  keys: KeyInventoryEntry[];
  complianceStatus: 'COMPLIANT' | 'WARNING' | 'NON_COMPLIANT';
}

interface KeyInventoryEntry {
  keyId: string;
  purpose: string;
  algorithm: string;
  protectionLevel: 'HSM' | 'SOFTWARE';
  primaryVersionId: string;
  primaryVersionAge: number; // days
  rotationPeriod: number; // days
  rotationStatus: 'CURRENT' | 'DUE_SOON' | 'OVERDUE';
  totalVersions: number;
  enabledVersions: number;
  destroyedVersions: number;
  lastRotated: Date;
  nextRotation: Date;
}

async function generateKeyInventoryReport(): Promise<KeyInventoryReport> {
  const kmsClient = new KeyManagementServiceClient();
  const keyRingName = 'projects/bioqms-prod/locations/us-central1/keyRings/bioqms-keyring';

  const [keys] = await kmsClient.listCryptoKeys({ parent: keyRingName });
  const inventory: KeyInventoryEntry[] = [];

  for (const key of keys) {
    const [versions] = await kmsClient.listCryptoKeyVersions({ parent: key.name! });
    const primaryVersion = versions.find(v => v.name === key.primary?.name);
    const primaryAge = primaryVersion
      ? (Date.now() - (primaryVersion.createTime?.seconds || 0) * 1000) / (1000 * 60 * 60 * 24)
      : 0;

    const rotationPeriod = parseRotationPeriod(key.rotationPeriod);
    const rotationStatus =
      primaryAge > rotationPeriod ? 'OVERDUE' :
      primaryAge > rotationPeriod * 0.9 ? 'DUE_SOON' :
      'CURRENT';

    inventory.push({
      keyId: key.name!.split('/').pop()!,
      purpose: key.purpose || 'UNKNOWN',
      algorithm: primaryVersion?.algorithm || 'UNKNOWN',
      protectionLevel: primaryVersion?.protectionLevel || 'UNKNOWN',
      primaryVersionId: primaryVersion?.name?.split('/').pop() || 'NONE',
      primaryVersionAge: Math.floor(primaryAge),
      rotationPeriod,
      rotationStatus,
      totalVersions: versions.length,
      enabledVersions: versions.filter(v => v.state === 'ENABLED').length,
      destroyedVersions: versions.filter(v => v.state === 'DESTROYED').length,
      lastRotated: new Date((primaryVersion?.createTime?.seconds || 0) * 1000),
      nextRotation: new Date((primaryVersion?.createTime?.seconds || 0) * 1000 + rotationPeriod * 24 * 60 * 60 * 1000),
    });
  }

  const complianceStatus =
    inventory.some(k => k.rotationStatus === 'OVERDUE') ? 'NON_COMPLIANT' :
    inventory.some(k => k.rotationStatus === 'DUE_SOON') ? 'WARNING' :
    'COMPLIANT';

  return {
    reportDate: new Date(),
    keys: inventory,
    complianceStatus,
  };
}

function parseRotationPeriod(period: string | undefined): number {
  if (!period) return 365; // Default 1 year
  const match = period.match(/(\d+)d/);
  return match ? parseInt(match[1]) : 365;
}

Report Distribution:

Automated email to compliance@bioqms.com
Exported to Google Sheets for auditor access
Archived in Cloud Storage (7-year retention)

10. Performance Considerations

10.1 HSM Operation Latency

Measured Latency (Google Cloud HSM - us-central1):

Operation	Median	p95	p99	p99.9
Asymmetric Sign (ECDSA P-256)	15ms	25ms	35ms	50ms
Asymmetric Verify (local)	2ms	5ms	8ms	12ms
Symmetric Encrypt (AES-256-GCM)	8ms	12ms	18ms	25ms
Symmetric Decrypt (AES-256-GCM)	8ms	12ms	18ms	25ms
Get Public Key	10ms	18ms	25ms	35ms

E-Signature Operation Budget (Target: <50ms):

Re-authentication validation: 5ms (local JWT verify)
Construct signing data: 2ms (JSON serialization)
HSM sign operation: 25ms (p95)
Database write (signature + approval): 10ms (indexed insert)
Audit trail write: 5ms (async, non-blocking)
Total: 47ms (within target)

10.2 Throughput Limits

Google Cloud KMS HSM Limits:

Operations per key per second: 500 ops/sec sustained
Burst capacity: 1,000 ops/sec for 60 seconds
Global quota (per project): 60,000 ops/min

BIO-QMS Workload Estimate (per tenant):

E-signature operations: ~50/day (~0.0006/sec)
JWT signing: ~500/hour (~0.14/sec)
Audit chain signing: ~5,000/hour (~1.4/sec)
Total: ~1.5 ops/sec per tenant (well below 500 ops/sec limit)

Multi-Tenant Scaling:

100 tenants: 150 ops/sec (30% utilization)
300 tenants: 450 ops/sec (90% utilization)
Recommendation: Separate key per tenant if >300 tenants deployed

10.3 Caching Strategies

Public Key Caching (Verification):

import { LRUCache } from 'lru-cache';

class HSMClientWithCache extends HSMClient {
  private publicKeyCache: LRUCache<string, string>;

  constructor() {
    super();
    this.publicKeyCache = new LRUCache<string, string>({
      max: 100, // Cache up to 100 public keys
      ttl: 1000 * 60 * 60, // 1-hour TTL
    });
  }

  async getPublicKey(keyConfig: HSMConfig): Promise<string> {
    const cacheKey = `${keyConfig.keyId}:${keyConfig.keyVersion || 'primary'}`;

    // Check cache first
    const cached = this.publicKeyCache.get(cacheKey);
    if (cached) return cached;

    // Fetch from HSM
    const keyVersionName = this.getKeyVersionName(keyConfig);
    const [response] = await this.kmsClient.getPublicKey({
      name: keyVersionName,
    });

    const publicKeyPem = response.pem as string;
    this.publicKeyCache.set(cacheKey, publicKeyPem);

    return publicKeyPem;
  }
}

DEK Caching (Envelope Encryption):

class DataEncryptionService {
  private dekCache: LRUCache<string, Buffer>;

  constructor() {
    this.dekCache = new LRUCache<string, Buffer>({
      max: 1000, // Cache 1000 DEKs
      ttl: 1000 * 60 * 5, // 5-minute TTL (security vs. performance trade-off)
    });
  }

  async encryptField(plaintext: string, fieldId: string): Promise<string> {
    // Use cached DEK if available
    let dek = this.dekCache.get(fieldId);

    if (!dek) {
      // Generate new DEK and encrypt with HSM KEK
      const crypto = await import('crypto');
      dek = crypto.randomBytes(32);
      this.dekCache.set(fieldId, dek);
    }

    // Encrypt plaintext with cached DEK (no HSM operation)
    const crypto = await import('crypto');
    const iv = crypto.randomBytes(12);
    const cipher = crypto.createCipheriv('aes-256-gcm', dek, iv);
    const ciphertext = Buffer.concat([
      cipher.update(plaintext, 'utf8'),
      cipher.final(),
    ]);
    const authTag = cipher.getAuthTag();

    // Return base64-encoded envelope
    return Buffer.concat([iv, authTag, ciphertext]).toString('base64');
  }
}

Cache Invalidation:

Public key cache: Invalidate on key rotation (via Cloud Pub/Sub notification)
DEK cache: 5-minute TTL + invalidate on key version change

11. Migration Plan

11.1 Phase 1: Create HSM-Backed Keys (Week 1)

Objectives:

Generate all HSM-backed keys in production project
Configure IAM bindings with separation of duties
Test HSM integration in staging environment

Tasks:

Execute key generation ceremony (Section 8.1)
Create service account keys for HSM access
Deploy HSMClient library to staging environment
Run integration tests (100 sign/verify operations)
Validate audit log entries in Cloud Logging

Acceptance Criteria:

All keys show protectionLevel: HSM
IAM bindings verified (least privilege)
Staging integration tests pass (100% success rate)
Audit logs capture all HSM operations

11.2 Phase 2: Migrate E-Signature Signing (Week 2)

Objectives:

Replace SHA-256/HMAC e-signature signing with ECDSA P-256/HSM
Implement cryptographic hash binding (FDA §11.70)
Maintain backward compatibility for signature verification

Tasks:

Deploy ESignatureService with HSM signing (Section 7.2)
Feature flag: USE_HSM_ESIG (default: false)
Shadow mode: Dual-sign (software + HSM) for 1 week
Compare signatures for consistency
Flip feature flag to HSM-only
Verify all new signatures use HSM key version

Rollback Plan:

Feature flag to false (instant rollback to software signing)
Monitor error rate: rollback if >1% signature failures

Acceptance Criteria:

100% of new e-signatures use HSM key
Legacy signature verification still works (software keys retained)
Cryptographic hash stored in ElectronicSignature.cryptoHash
E-signature latency <50ms (p95)

11.3 Phase 3: Migrate Data Encryption (Week 3)

Objectives:

Migrate KEK from software KMS to HSM KMS
Re-encrypt all existing DEKs with HSM KEK
Zero downtime migration (online re-encryption)

Tasks:

Create bioqms-encryption-kek-2-hsm (HSM-backed)
Deploy dual-KEK decryption (try HSM KEK, fallback to software KEK)
Background job: Re-encrypt all DEKs with HSM KEK
Monitor re-encryption progress (dashboard)
After 100% re-encrypted: retire software KEK

Re-Encryption Job:

async function reEncryptDEKsWithHSM(): Promise<void> {
  const kmsClient = new KeyManagementServiceClient();
  const totalRecords = await prisma.workOrder.count();
  let reEncryptedCount = 0;

  const cursor = prisma.workOrder.findMany({
    where: { encryptionKeyVersion: { startsWith: 'software-' } },
    select: { id: true, encryptedFields: true },
    take: 100,
  });

  for await (const batch of cursor) {
    for (const record of batch) {
      // Decrypt DEK with old software KEK
      const [decryptResp] = await kmsClient.decrypt({
        name: 'projects/bioqms-prod/locations/us-central1/keyRings/bioqms-keyring/cryptoKeys/bioqms-encryption-kek-2-software',
        ciphertext: record.encryptedFields.dek,
      });
      const dek = decryptResp.plaintext;

      // Re-encrypt DEK with new HSM KEK
      const [encryptResp] = await kmsClient.encrypt({
        name: 'projects/bioqms-prod/locations/us-central1/keyRings/bioqms-keyring/cryptoKeys/bioqms-encryption-kek-2',
        plaintext: dek,
      });

      // Update record
      await prisma.workOrder.update({
        where: { id: record.id },
        data: {
          encryptedFields: {
            ...record.encryptedFields,
            dek: encryptResp.ciphertext,
          },
          encryptionKeyVersion: 'hsm-v1',
        },
      });

      reEncryptedCount++;
      if (reEncryptedCount % 1000 === 0) {
        console.log(`Re-encrypted ${reEncryptedCount}/${totalRecords} (${Math.floor(reEncryptedCount / totalRecords * 100)}%)`);
      }
    }
  }
}

Acceptance Criteria:

100% of records use HSM KEK
Zero decryption errors during migration
Software KEK disabled (not destroyed, retained for audit)

11.4 Phase 4: Migrate TLS CA (Week 4)

Objectives:

Issue new service certificates signed by HSM-backed CA
Rolling update: replace certificates without downtime
Retire software-backed CA

Tasks:

Generate new TLS CA key in HSM
Issue new root CA certificate (self-signed)
Distribute root CA to all services (trust store update)
Issue new service certificates signed by HSM CA
Rolling restart: Kubernetes deployments with new certs
Monitor TLS handshake success rate
After 100% migrated: revoke old CA

Certificate Rollout (Kubernetes):

# ConfigMap with new root CA
apiVersion: v1
kind: ConfigMap
metadata:
  name: bioqms-root-ca
  namespace: bioqms-prod
data:
  ca.crt: |
    -----BEGIN CERTIFICATE-----
    [HSM-signed root CA certificate]
    -----END CERTIFICATE-----

# Secret with new service certificate
apiVersion: v1
kind: Secret
metadata:
  name: esig-service-tls
  namespace: bioqms-prod
type: kubernetes.io/tls
data:
  tls.crt: [base64-encoded HSM-signed cert]
  tls.key: [base64-encoded service private key]

# Deployment with new cert mounted
apiVersion: apps/v1
kind: Deployment
metadata:
  name: esig-service
spec:
  template:
    spec:
      containers:
      - name: esig-service
        volumeMounts:
        - name: tls-cert
          mountPath: /etc/tls
          readOnly: true
      volumes:
      - name: tls-cert
        secret:
          secretName: esig-service-tls

Acceptance Criteria:

All services use HSM-signed certificates
TLS handshake success rate: 100%
Certificate expiry monitoring active (90-day auto-renewal)

11.5 Phase 5: Decommission Software Keys (Week 5)

Objectives:

Disable all software-backed keys
Archive software keys for audit (DESTROYED state)
Document migration completion

Tasks:

Verify no services use software keys (audit log query)

Disable software key versions:

gcloud kms keys versions disable 1 \
  --key bioqms-esig-signing-key-software \
  --keyring bioqms-keyring \
  --location us-central1

Schedule destruction (72-hour grace period)
Update compliance documentation (key inventory)
Generate migration completion report

Acceptance Criteria:

All software keys in DISABLED or DESTROYED state
Zero active software key versions
Migration report submitted to compliance team
HSM-only mode validated in production

12. Cost Analysis

12.1 Google Cloud HSM Pricing (as of 2026-02-16)

Key Version Storage:

$2.50/month per active key version
HSM-backed keys only (software keys: $0.06/month)

Operations:

$2.50 per 10,000 operations
Includes: sign, verify, encrypt, decrypt, get public key

BIO-QMS Key Inventory Cost:

Key	Versions	Storage/mo	Ops/mo	Ops Cost/mo	Total/mo
`bioqms-root-kek-0`	1	$2.50	0	$0	$2.50
`bioqms-signing-kek-1`	2	$5.00	100	$0.03	$5.03
`bioqms-encryption-kek-2`	2	$5.00	50,000	$12.50	$17.50
`bioqms-tls-ca-kek-3`	1	$2.50	200	$0.05	$2.55
`bioqms-esig-signing-key`	3	$7.50	1,500	$0.38	$7.88
`bioqms-audit-chain-key`	1	$2.50	150,000	$37.50	$40.00
`bioqms-jwt-signing-key`	2	$5.00	15,000	$3.75	$8.75
`bioqms-tls-ca-key`	2	$5.00	500	$0.13	$5.13
`bioqms-code-signing-key`	2	$5.00	50	$0.01	$5.01
Total	16	$42.50	217,350	$54.35	$96.85

Annual Cost: $1,162.20

12.2 AWS CloudHSM Comparison

AWS CloudHSM Pricing:

$1.60/hour per HSM instance (~$1,168/month)
Minimum 2 instances for HA: $2,336/month
No per-operation fees
Cross-region replication: +$1,168/month per DR region

Total AWS CloudHSM Cost:

Baseline (2 instances, single region): $2,336/month ($28,032/year)
With DR (4 instances, 2 regions): $4,672/month ($56,064/year)

Cost Comparison:

Provider	Monthly Cost	Annual Cost	Break-Even Point
Google Cloud HSM	$96.85	$1,162	N/A
AWS CloudHSM (single region)	$2,336	$28,032	Never (24x more expensive)
AWS CloudHSM (multi-region)	$4,672	$56,064	Never (48x more expensive)

Recommendation: Google Cloud HSM provides 24x cost savings for BIO-QMS workload.

12.3 Cost Optimization Strategies

1. Key Version Cleanup:

Retain only required versions (e.g., JWT key: keep 2 versions)
Destroy old versions after rotation grace period
Savings: $2.50/month per destroyed version

2. Operation Batching:

Batch audit trail signing (sign 100 entries per operation)
Savings: Reduce audit chain ops from 150,000/mo to 1,500/mo (~$37/mo)

3. Public Key Caching:

Cache public keys for 1 hour (no HSM operation for verification)
Savings: Reduce verify ops by ~~95% (~~$10/mo)

4. DEK Caching:

Cache DEKs for 5 minutes (reduce KEK decrypt ops)
Savings: Reduce encryption KEK ops by ~~80% (~~$10/mo)

Optimized Monthly Cost: $96.85 - $37 - $10 - $10 = $39.85/month ($478/year)

13. Compliance Mapping

13.1 FIPS 140-2 Level 3 Requirements

FIPS 140-2 Requirement	Google Cloud HSM Implementation	Evidence
Physical Security (Level 3)	Tamper-evident seals, zeroization on intrusion	HSM Certificate #3254
Cryptographic Module Specification	Cavium LiquidSecurity HSM, FIPS-approved algorithms	Certificate documentation
Cryptographic Module Ports and Interfaces	Defined data input/output, control, status interfaces	Cloud KMS API specification
Roles, Services, and Authentication	Crypto Officer, User, Admin roles	IAM role bindings (Section 6)
Finite State Model	Power-up, operational, error states	HSM firmware state machine
Physical Security (Level 3)	Tamper detection, zeroization, environmental failure protection	HSM hardware design
Operational Environment	Modifiable operational environment (Google-managed)	GCP infrastructure
Cryptographic Key Management	Key generation in hardware, secure storage, zeroization	Key lifecycle procedures (Section 8)
Electromagnetic Interference/Compatibility	FCC Part 15 Class A compliant	HSM certification
Self-Tests	Power-up, conditional, on-demand tests	HSM firmware
Design Assurance	Configuration management, secure distribution, developer training	Google HSM program
Mitigation of Other Attacks	Timing analysis, power analysis countermeasures	HSM hardware design

Compliance Status: ✅ Google Cloud HSM is FIPS 140-2 Level 3 certified (Certificate #3254)

13.2 FDA 21 CFR Part 11 Mapping

Part 11 Requirement	HSM Implementation	BIO-QMS Integration
§11.10(a) Validation of systems	HSM certified and validated by NIST	IQ/OQ/PQ documentation includes HSM validation
§11.10(e) Audit trail	Cloud Audit Logs capture all HSM operations	Audit service consumes HSM event logs
§11.70 Signature/record linking	Cryptographic hash binding (ECDSA signature)	`ElectronicSignature.cryptoHash` field
§11.100(a) Unique signatures	HSM key per signature type (e-signature, audit)	Key inventory (Section 5)
§11.100(b) Identity verification	Re-authentication before signing	E-signature flow (Section 7.2)
§11.200 Electronic signature components	Two-factor: password + HSM-backed cryptographic signature	Re-auth + HSM sign operation

Compliance Status: ✅ HSM integration satisfies FDA Part 11 cryptographic requirements

13.3 HIPAA §164.312 Mapping

HIPAA Requirement	HSM Implementation	BIO-QMS Integration
§164.312(a)(2)(iv) Encryption and decryption	AES-256-GCM with HSM-backed KEK	Envelope encryption (Section 7.1)
§164.312(b) Audit controls	Cloud Audit Logs (tamper-evident)	Audit service integration
§164.312(c)(1) Integrity controls	HMAC-SHA256 audit chain	Audit chain key (Section 5.1)
§164.312(e)(2)(ii) Encryption in transit	TLS 1.3 with HSM-backed certificates	TLS CA integration (Section 7.3)

Compliance Status: ✅ HSM integration satisfies HIPAA Technical Safeguards

13.4 SOC 2 Type II CC6.1 Mapping

SOC 2 Criterion	HSM Control	Evidence
CC6.1 Restricts logical access	IAM role-based access to HSM keys	IAM bindings (Section 6.1)
CC6.1 Key management	Separation of duties (admin vs. user vs. auditor)	Role matrix (Section 6.2)
CC6.1 Cryptographic key protection	HSM hardware protection, no key exportability	FIPS 140-2 Level 3 certification
CC6.1 Audit logging	Cloud Audit Logs capture all key access	Audit log queries (Section 9.1)
CC6.1 Key rotation	Automated rotation with version retention	Rotation procedures (Section 8.2)

Compliance Status: ✅ HSM integration satisfies SOC 2 CC6.1 controls

14. Security Considerations

14.1 Threat Model for HSM Integration

Threat: Compromised Service Account Credentials

Attack: Attacker gains access to service account key, attempts unauthorized HSM operations
Mitigation:
- IAM conditions limit service account to specific keys only
- Cloud Audit Logs detect unusual access patterns
- Workload Identity Federation (no long-lived keys)
Detection: Alert on HSM operations from unexpected IP or GKE pod

Threat: Insider Threat (Malicious HSM Admin)

Attack: HSM admin attempts to exfiltrate key material or destroy keys
Mitigation:
- Separation of duties: admin can create/destroy but not use keys
- Dual-approval for break-glass access
- 72-hour grace period before key destruction
- Cloud Audit Logs + video recording for key ceremonies
Detection: Alert on key destruction events, manual review of admin actions

Threat: Cross-Tenant Key Access

Attack: Tenant A attempts to use Tenant B's signing key
Mitigation:
- Separate key per tenant (if >300 tenants)
- IAM bindings scoped to tenant service account only
- Application-layer tenant validation before HSM call
Detection: Tenant mismatch in audit logs

Threat: Replay Attack (Signature Reuse)

Attack: Attacker captures e-signature, replays it on different work order
Mitigation:
- Signature binds to specific work order ID (cryptographic hash includes WO ID)
- Signature consumed flag prevents reuse
- 5-minute signature validity window
Detection: Signature verification fails (hash mismatch)

14.2 Key Compromise Response

Indicators of Compromise:

Unexpected HSM operations in audit logs
Service account key leaked in public repository
Unusual signature patterns (e.g., 1000s of signatures in 1 minute)
Failed HSM operations with unusual error codes

Response Procedure:

Immediate (< 5 minutes):
- Disable compromised key version
- Revoke compromised service account credentials
- Alert security team (P1 incident)
Short-term (< 1 hour):
- Rotate to new key version
- Audit all operations using compromised key
- Identify affected records (e-signatures, encrypted data)
Medium-term (< 24 hours):
- Re-sign affected records with new key
- Notify affected tenants (if applicable)
- Generate forensic report
Long-term (< 1 week):
- Root cause analysis
- Improve detection (new alert rules)
- Update incident response procedures

14.3 Defense in Depth

Layer 1: HSM Hardware

FIPS 140-2 Level 3 tamper protection
Key material never exportable
Zeroization on physical intrusion

Layer 2: IAM Access Control

Least privilege service accounts
Separation of duties (admin vs. user)
Time-bound break-glass access

Layer 3: Application Layer

Tenant validation before HSM call
Signature binding to work order ID
Re-authentication before signing

Layer 4: Audit and Monitoring

Cloud Audit Logs (tamper-evident)
Real-time alerting on anomalies
Monthly compliance reporting

Layer 5: Operational Procedures

Key generation ceremony (witnessed)
Dual-approval for break-glass
72-hour grace period before destruction

15. Appendix

15.1 Glossary

Term	Definition
HSM	Hardware Security Module — tamper-resistant hardware for cryptographic key protection
FIPS 140-2 Level 3	Federal Information Processing Standard for cryptographic modules; Level 3 includes physical tamper protection
KEK	Key Encryption Key — encrypts other keys (envelope encryption)
DEK	Data Encryption Key — encrypts actual data (short-lived, ephemeral)
Envelope Encryption	Encrypt data with DEK, encrypt DEK with KEK (two-tier encryption)
Cryptographic Binding	Hash signature of record ensures signature cannot be separated from record (FDA §11.70)
Separation of Duties	No single person has complete control (e.g., key admin cannot use keys)
Key Rotation	Replace key version with new version while retaining old version for verification
Key Destruction	Cryptographic zeroization of key material (FIPS 140-2 compliant)
Break-Glass	Emergency access procedure with enhanced audit logging
RTO	Recovery Time Objective — maximum tolerable downtime
RPO	Recovery Point Objective — maximum tolerable data loss

15.2 References

Document	URL	Description
FIPS 140-2	https://csrc.nist.gov/publications/detail/fips/140/2/final	Cryptographic Module Validation Program
Google Cloud HSM	https://cloud.google.com/kms/docs/hsm	Cloud KMS with HSM protection level
Cavium HSM Certificate	https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/3254	FIPS 140-2 Level 3 certification
FDA 21 CFR Part 11	https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=11	Electronic Records and Signatures
HIPAA Security Rule	https://www.hhs.gov/hipaa/for-professionals/security/index.html	§164.312 Technical Safeguards
SOC 2 Type II	https://us.aicpa.org/interestareas/frc/assuranceadvisoryservices/aicpasoc2report	Trust Services Criteria
NIST SP 800-57	https://csrc.nist.gov/publications/detail/sp/800-57-part-1/rev-5/final	Key Management Recommendations

15.3 Revision Control

This document is version-controlled in the BIO-QMS git repository:

Repository: https://github.com/coditect-ai/coditect-biosciences-qms-platform
Path: docs/compliance/hsm-integration-architecture.md
Branch: main
Last Updated: 2026-02-16
Next Review: 2027-02-16 (annual)

Change Management:

All changes require pull request review
Approval required from: Security Architect, Compliance Officer
Changes trigger update to Document Control section (version increment)

This document defines the security foundation for production-grade key management in a FDA-regulated life sciences QMS platform. HSM integration is not optional — it is the regulatory requirement for validated systems handling electronic signatures and protected health information.

Document Control​

Version History​

Approval​

Revision History​

1. Executive Summary​

1.1 Purpose​

1.2 Scope​

1.3 Current State vs. Target State​

2. Architecture Overview​

2.1 Conceptual Model​

2.2 Key Benefits of Google Cloud HSM​

3. HSM Provider Selection​

3.1 Primary: Google Cloud HSM​

3.2 Alternative: AWS CloudHSM​

3.3 Decision Matrix​

4. Key Hierarchy Design​

4.1 Hierarchical Structure​

4.2 Key Hierarchy Principles​

4.3 Key Types and Algorithms​

5. HSM Key Inventory​

5.1 Production Keys (All HSM-Backed)​

5.2 Key Access Control Matrix​

6. Access Control Model​

6.1 IAM Roles and Permissions​

6.2 Break-Glass Access​

6.3 Service Account Bindings​

7. Integration Architecture​

7.1 Application Layer Integration​

7.2 E-Signature Integration Flow​

7.3 TLS Certificate Issuance Flow​

8. Operational Procedures​

8.1 Key Generation Ceremony​

8.2 Key Rotation Automation​

8.3 Key Revocation Procedure​

8.4 Key Destruction​

8.5 Disaster Recovery​

9. Monitoring and Audit​

9.1 Cloud Audit Logs for HSM Operations​

9.2 Key Usage Metrics (Prometheus/Grafana)​

9.3 Alert Rules​

9.4 Compliance Reporting​

10. Performance Considerations​

10.1 HSM Operation Latency​

10.2 Throughput Limits​

10.3 Caching Strategies​

11. Migration Plan​

11.1 Phase 1: Create HSM-Backed Keys (Week 1)​

11.2 Phase 2: Migrate E-Signature Signing (Week 2)​

11.3 Phase 3: Migrate Data Encryption (Week 3)​

11.4 Phase 4: Migrate TLS CA (Week 4)​

11.5 Phase 5: Decommission Software Keys (Week 5)​

12. Cost Analysis​

12.1 Google Cloud HSM Pricing (as of 2026-02-16)​

12.2 AWS CloudHSM Comparison​

12.3 Cost Optimization Strategies​

13. Compliance Mapping​

13.1 FIPS 140-2 Level 3 Requirements​

13.2 FDA 21 CFR Part 11 Mapping​

13.3 HIPAA §164.312 Mapping​

13.4 SOC 2 Type II CC6.1 Mapping​

14. Security Considerations​

14.1 Threat Model for HSM Integration​

14.2 Key Compromise Response​

14.3 Defense in Depth​

15. Appendix​

15.1 Glossary​

15.2 References​

15.3 Revision Control​

Document Control

Version History

Approval

Revision History

1. Executive Summary

1.1 Purpose

1.2 Scope

1.3 Current State vs. Target State

2. Architecture Overview

2.1 Conceptual Model

2.2 Key Benefits of Google Cloud HSM

3. HSM Provider Selection

3.1 Primary: Google Cloud HSM

3.2 Alternative: AWS CloudHSM

3.3 Decision Matrix

4. Key Hierarchy Design

4.1 Hierarchical Structure

4.2 Key Hierarchy Principles

4.3 Key Types and Algorithms

5. HSM Key Inventory

5.1 Production Keys (All HSM-Backed)

5.2 Key Access Control Matrix

6. Access Control Model

6.1 IAM Roles and Permissions

6.2 Break-Glass Access

6.3 Service Account Bindings

7. Integration Architecture

7.1 Application Layer Integration

7.2 E-Signature Integration Flow

7.3 TLS Certificate Issuance Flow

8. Operational Procedures

8.1 Key Generation Ceremony

8.2 Key Rotation Automation

8.3 Key Revocation Procedure

8.4 Key Destruction

8.5 Disaster Recovery

9. Monitoring and Audit

9.1 Cloud Audit Logs for HSM Operations

9.2 Key Usage Metrics (Prometheus/Grafana)

9.3 Alert Rules

9.4 Compliance Reporting

10. Performance Considerations

10.1 HSM Operation Latency

10.2 Throughput Limits

10.3 Caching Strategies

11. Migration Plan

11.1 Phase 1: Create HSM-Backed Keys (Week 1)

11.2 Phase 2: Migrate E-Signature Signing (Week 2)

11.3 Phase 3: Migrate Data Encryption (Week 3)

11.4 Phase 4: Migrate TLS CA (Week 4)

11.5 Phase 5: Decommission Software Keys (Week 5)

12. Cost Analysis

12.1 Google Cloud HSM Pricing (as of 2026-02-16)

12.2 AWS CloudHSM Comparison

12.3 Cost Optimization Strategies

13. Compliance Mapping

13.1 FIPS 140-2 Level 3 Requirements

13.2 FDA 21 CFR Part 11 Mapping

13.3 HIPAA §164.312 Mapping

13.4 SOC 2 Type II CC6.1 Mapping

14. Security Considerations

14.1 Threat Model for HSM Integration

14.2 Key Compromise Response

14.3 Defense in Depth

15. Appendix

15.1 Glossary

15.2 References

15.3 Revision Control