Skip to main content

OpenTofu Migration - Next Steps and Implementation Plan

Date: December 1, 2025 Status: Planning Phase - Ready to Execute Current State: Manual Infrastructure (100% Functional) Target State: Infrastructure as Code (IaC) with OpenTofu


Executive Summary

Why Migrate to OpenTofu?

The staging environment was successfully deployed using manual gcloud commands. While functional, this approach has significant limitations:

AspectManual (Current)OpenTofu (Target)
Reproducibility❌ Tribal knowledge only✅ Code-defined, version-controlled
Drift Detection❌ No visibility✅ Automatic detection via tofu plan
Disaster Recovery❌ Manual recreation✅ Re-apply configuration
Environment Parity❌ Manual synchronization✅ Same code, different variables
Change Tracking❌ No audit trail✅ Git history
Team Collaboration❌ Knowledge silos✅ Shared codebase

Investment Required: 1-2 hours (one-time) Benefits: Permanent infrastructure reproducibility and drift prevention Risk: Low - Import existing resources without recreating


Current Infrastructure Inventory

What We Manually Created (December 1, 2025)

Cloud SQL PostgreSQL:

Instance Name: coditect-db
Version: POSTGRES_16
Tier: db-f1-micro
Region: us-central1
Network: projects/coditect-cloud-infra/global/networks/default
Private IP: 10.28.0.3
SSL: Disabled (staging only)
Database: coditect
User: coditect_app

Redis Memorystore:

Instance Name: coditect-redis-staging
Version: redis_7_0
Tier: BASIC
Memory: 1GB
Region: us-central1
Network: default
Host: 10.164.210.91
AUTH: Disabled (staging only)

GKE Resources:

Cluster: coditect-cluster
Namespace: coditect-staging
Deployment: coditect-backend (2 replicas)
Service: coditect-backend (LoadBalancer)
External IP: 136.114.0.156
ConfigMaps: backend-config
Secrets: backend-secrets

Docker Images:

Repository: us-central1-docker.pkg.dev/coditect-cloud-infra/coditect-backend
Latest: coditect-cloud-backend:v1.0.3-staging

Networking:

VPC: default
Subnetworks: Auto-created
Private Service Connection: Established for Cloud SQL
LoadBalancer: GKE-managed (136.114.0.156)

Existing OpenTofu Module Structure

Location: submodules/cloud/coditect-cloud-infra/opentofu/

Expected Directory Structure:

opentofu/
├── modules/
│ ├── database/ # Cloud SQL module
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── redis/ # Redis Memorystore module
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── kubernetes/ # GKE cluster module
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── networking/ # VPC and networking
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
├── environments/
│ ├── staging/ # Need to create
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ ├── terraform.tfvars
│ │ └── backend.tf
│ └── production/ # Future
│ └── (same structure)
└── backend-config/
└── staging-backend.hcl

Migration Strategy: Import Existing Resources

Philosophy: Import existing resources into OpenTofu state WITHOUT recreating them.

3-Phase Approach:

Phase 1: State Import (30 minutes)

Import manually-created resources into OpenTofu state

Phase 2: Validation (15 minutes)

Verify tofu plan shows no changes

Phase 3: GitOps Integration (15 minutes)

Commit configuration and establish workflow


Phase 1: Import Existing Resources

Step 1.1: Create Staging Environment Configuration

File: opentofu/environments/staging/main.tf

terraform {
required_version = ">= 1.6.0"

required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.0"
}
}

backend "gcs" {
bucket = "coditect-terraform-state"
prefix = "staging"
}
}

provider "google" {
project = var.project_id
region = var.region
}

# Cloud SQL PostgreSQL
module "database" {
source = "../../modules/database"

project_id = var.project_id
region = var.region
instance_name = "coditect-db"
database_version = "POSTGRES_16"
tier = "db-f1-micro"
network = "default"
database_name = "coditect"
database_user = "coditect_app"
enable_ssl = false # Staging only
}

# Redis Memorystore
module "redis" {
source = "../../modules/redis"

project_id = var.project_id
region = var.region
instance_name = "coditect-redis-staging"
redis_version = "REDIS_7_0"
tier = "BASIC"
memory_size_gb = 1
network = "default"
enable_auth = false # Staging only
}

# GKE Cluster (if managing cluster via OpenTofu)
# Note: If cluster already managed elsewhere, omit this
module "kubernetes" {
source = "../../modules/kubernetes"

project_id = var.project_id
region = var.region
cluster_name = "coditect-cluster"
network = "default"
min_nodes = 1
max_nodes = 3
machine_type = "e2-standard-4"
}

File: opentofu/environments/staging/variables.tf

variable "project_id" {
description = "GCP Project ID"
type = string
default = "coditect-cloud-infra"
}

variable "region" {
description = "GCP Region"
type = string
default = "us-central1"
}

variable "environment" {
description = "Environment name"
type = string
default = "staging"
}

File: opentofu/environments/staging/terraform.tfvars

project_id  = "coditect-cloud-infra"
region = "us-central1"
environment = "staging"

Step 1.2: Initialize OpenTofu

cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging

# Initialize OpenTofu
tofu init

# Expected output:
# Initializing the backend...
# Initializing provider plugins...
# OpenTofu has been successfully initialized!

Step 1.3: Import Cloud SQL Instance

# Import Cloud SQL instance
tofu import module.database.google_sql_database_instance.main \
coditect-cloud-infra/coditect-db

# Import database
tofu import module.database.google_sql_database.main \
coditect-cloud-infra/coditect-db/coditect

# Import database user
tofu import module.database.google_sql_user.main \
coditect-cloud-infra/coditect-db/coditect_app

Verification:

tofu state list | grep database
# Should show:
# module.database.google_sql_database_instance.main
# module.database.google_sql_database.main
# module.database.google_sql_user.main

Step 1.4: Import Redis Instance

# Import Redis instance
tofu import module.redis.google_redis_instance.main \
coditect-cloud-infra/us-central1/coditect-redis-staging

Verification:

tofu state list | grep redis
# Should show:
# module.redis.google_redis_instance.main

Step 1.5: Import GKE Cluster (Optional)

Decision Point: If GKE cluster is managed separately (e.g., via Google Cloud Console or separate IaC), you may choose NOT to import it.

If importing:

tofu import module.kubernetes.google_container_cluster.main \
coditect-cloud-infra/us-central1/coditect-cluster

Alternative: Manage GKE cluster separately and only manage Cloud SQL + Redis via OpenTofu.


Phase 2: Validation (Zero Changes Expected)

Step 2.1: Run OpenTofu Plan

cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging

tofu plan

Expected Output:

OpenTofu will perform the following actions:

No changes. Your infrastructure matches the configuration.

OpenTofu has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

If Changes Detected:

This indicates configuration drift. Common causes:

  1. Missing configuration parameters - Add parameters to match current state
  2. Default values changed - Explicitly set values in configuration
  3. Computed values - Mark as lifecycle { ignore_changes = [...] }

Example Fix:

resource "google_sql_database_instance" "main" {
# ... other configuration ...

lifecycle {
ignore_changes = [
# Ignore auto-assigned values
connection_name,
ip_address,
]
}
}

Step 2.2: Validate State Accuracy

# Show Cloud SQL instance
tofu state show module.database.google_sql_database_instance.main

# Verify matches actual resource
gcloud sql instances describe coditect-db

# Show Redis instance
tofu state show module.redis.google_redis_instance.main

# Verify matches actual resource
gcloud redis instances describe coditect-redis-staging --region=us-central1

Acceptance Criteria:

  • ✅ All imported resources appear in tofu state list
  • tofu plan shows zero changes
  • tofu state show matches gcloud describe output

Phase 3: GitOps Integration

Step 3.1: Configure Remote State Backend

Create GCS bucket for state:

# Create bucket (if not exists)
gcloud storage buckets create gs://coditect-terraform-state \
--project=coditect-cloud-infra \
--location=us-central1 \
--uniform-bucket-level-access

# Enable versioning
gcloud storage buckets update gs://coditect-terraform-state \
--versioning

File: opentofu/environments/staging/backend.tf

terraform {
backend "gcs" {
bucket = "coditect-terraform-state"
prefix = "staging"
}
}

Migrate local state to GCS:

cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging

# Reconfigure backend
tofu init -migrate-state

# Confirm migration when prompted

Step 3.2: Commit Configuration to Git

cd submodules/cloud/coditect-cloud-infra

# Add OpenTofu configuration
git add opentofu/environments/staging/
git add opentofu/modules/ # If modules don't exist yet

# Commit
git commit -m "feat: Add OpenTofu staging environment configuration

- Import existing Cloud SQL and Redis resources
- Configure remote state backend (GCS)
- Establish IaC for staging infrastructure
- Zero-change validation confirmed

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>"

# Push
git push origin main

Step 3.3: Update Master Repository Pointer

cd /Users/halcasteel/PROJECTS/coditect-rollout-master

# Update submodule pointer
git add submodules/cloud/coditect-cloud-infra

# Commit pointer update
git commit -m "chore: Update coditect-cloud-infra submodule - OpenTofu staging config

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>"

# Push
git push origin main

Operational Workflows

Daily: Check for Drift

cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging

# Run plan to detect any manual changes
tofu plan

# If changes detected, review and decide:
# 1. Update OpenTofu config to match manual change
# 2. Apply OpenTofu config to revert manual change

Making Infrastructure Changes

Old Way (Manual):

gcloud sql instances patch coditect-db --tier=db-n1-standard-1

New Way (IaC):

# 1. Edit configuration
vim opentofu/environments/staging/main.tf
# Change: tier = "db-n1-standard-1"

# 2. Preview changes
tofu plan

# 3. Apply changes
tofu apply

# 4. Commit configuration
git add opentofu/environments/staging/main.tf
git commit -m "feat: Upgrade staging database tier to db-n1-standard-1"
git push

Creating Production Environment

# Copy staging configuration
cp -r opentofu/environments/staging opentofu/environments/production

# Update production-specific values
vim opentofu/environments/production/terraform.tfvars

# Production differences:
# - Enable SSL on Cloud SQL
# - Enable Redis AUTH
# - Larger instance sizes
# - Different database names/users
# - Reserved static IP for LoadBalancer

# Initialize production
cd opentofu/environments/production
tofu init

# Create production resources
tofu apply

Module Development (If Modules Don't Exist)

Cloud SQL Module

File: opentofu/modules/database/main.tf

resource "google_sql_database_instance" "main" {
name = var.instance_name
database_version = var.database_version
region = var.region
project = var.project_id

settings {
tier = var.tier

ip_configuration {
ipv4_enabled = false
private_network = "projects/${var.project_id}/global/networks/${var.network}"
require_ssl = var.enable_ssl
}

backup_configuration {
enabled = true
start_time = "03:00"
}
}

deletion_protection = true
}

resource "google_sql_database" "main" {
name = var.database_name
instance = google_sql_database_instance.main.name
project = var.project_id
}

resource "google_sql_user" "main" {
name = var.database_user
instance = google_sql_database_instance.main.name
password = var.database_password
project = var.project_id
}

File: opentofu/modules/database/variables.tf

variable "project_id" {
description = "GCP Project ID"
type = string
}

variable "region" {
description = "GCP Region"
type = string
}

variable "instance_name" {
description = "Cloud SQL instance name"
type = string
}

variable "database_version" {
description = "Database version (e.g., POSTGRES_16)"
type = string
}

variable "tier" {
description = "Machine tier (e.g., db-f1-micro)"
type = string
}

variable "network" {
description = "VPC network name"
type = string
}

variable "database_name" {
description = "Database name to create"
type = string
}

variable "database_user" {
description = "Database user to create"
type = string
}

variable "database_password" {
description = "Database user password"
type = string
sensitive = true
}

variable "enable_ssl" {
description = "Require SSL connections"
type = bool
default = true
}

File: opentofu/modules/database/outputs.tf

output "instance_name" {
description = "Cloud SQL instance name"
value = google_sql_database_instance.main.name
}

output "private_ip_address" {
description = "Private IP address"
value = google_sql_database_instance.main.private_ip_address
}

output "connection_name" {
description = "Connection name"
value = google_sql_database_instance.main.connection_name
}

output "database_name" {
description = "Database name"
value = google_sql_database.main.name
}

Redis Module

File: opentofu/modules/redis/main.tf

resource "google_redis_instance" "main" {
name = var.instance_name
memory_size_gb = var.memory_size_gb
redis_version = var.redis_version
tier = var.tier
region = var.region
project = var.project_id
authorized_network = "projects/${var.project_id}/global/networks/${var.network}"

auth_enabled = var.enable_auth

redis_configs = {
maxmemory-policy = "allkeys-lru"
}
}

File: opentofu/modules/redis/variables.tf

variable "project_id" {
description = "GCP Project ID"
type = string
}

variable "region" {
description = "GCP Region"
type = string
}

variable "instance_name" {
description = "Redis instance name"
type = string
}

variable "redis_version" {
description = "Redis version (e.g., REDIS_7_0)"
type = string
}

variable "tier" {
description = "Service tier (BASIC or STANDARD_HA)"
type = string
}

variable "memory_size_gb" {
description = "Memory size in GB"
type = number
}

variable "network" {
description = "VPC network name"
type = string
}

variable "enable_auth" {
description = "Enable Redis AUTH"
type = bool
default = true
}

File: opentofu/modules/redis/outputs.tf

output "instance_name" {
description = "Redis instance name"
value = google_redis_instance.main.name
}

output "host" {
description = "Redis host IP"
value = google_redis_instance.main.host
}

output "port" {
description = "Redis port"
value = google_redis_instance.main.port
}

output "auth_string" {
description = "Redis AUTH string"
value = google_redis_instance.main.auth_string
sensitive = true
}

Security Considerations

Secret Management

NEVER commit to Git:

  • Database passwords
  • Redis AUTH tokens
  • Service account keys
  • API keys

Use GCP Secret Manager:

# Store database password
gcloud secrets create staging-db-password \
--data-file=- <<< "YOUR_PASSWORD_HERE" \
--project=coditect-cloud-infra

# Grant OpenTofu service account access
gcloud secrets add-iam-policy-binding staging-db-password \
--member="serviceAccount:terraform@coditect-cloud-infra.iam.gserviceaccount.com" \
--role="roles/secretmanager.secretAccessor"

Reference in OpenTofu:

data "google_secret_manager_secret_version" "db_password" {
secret = "staging-db-password"
project = var.project_id
}

module "database" {
source = "../../modules/database"
# ... other config ...
database_password = data.google_secret_manager_secret_version.db_password.secret_data
}

State File Security

GCS bucket ACLs:

# Restrict access to terraform service account only
gcloud storage buckets add-iam-policy-binding gs://coditect-terraform-state \
--member="serviceAccount:terraform@coditect-cloud-infra.iam.gserviceaccount.com" \
--role="roles/storage.objectAdmin"

Enable encryption:

# Use customer-managed encryption key (CMEK)
gcloud storage buckets update gs://coditect-terraform-state \
--default-encryption-key=projects/coditect-cloud-infra/locations/us-central1/keyRings/terraform/cryptoKeys/state-encryption

Rollback Plan

If OpenTofu migration causes issues:

Option 1: Remove from State (Keep Resources)

# Remove resource from OpenTofu management
tofu state rm module.database.google_sql_database_instance.main

# Resource continues to exist, just not managed by OpenTofu
# Can manage manually again

Option 2: Destroy and Recreate Manually

# Destroy OpenTofu-managed resource
tofu destroy -target=module.database.google_sql_database_instance.main

# Recreate manually using gcloud commands
gcloud sql instances create coditect-db ...

Option 3: Rollback State

# List state versions
gcloud storage ls gs://coditect-terraform-state/staging/

# Download previous state version
gcloud storage cp gs://coditect-terraform-state/staging/default.tfstate#VERSION ./terraform.tfstate

# Replace current state
tofu state push terraform.tfstate

Timeline and Effort Estimates

Phase 1: Import Existing Resources (30 minutes)

TaskTimeDeliverable
Create staging environment configuration10 minmain.tf, variables.tf, terraform.tfvars
Initialize OpenTofu5 min.terraform/ directory
Import Cloud SQL instance5 minCloud SQL in state
Import Redis instance5 minRedis in state
Import GKE cluster (optional)5 minGKE in state

Phase 2: Validation (15 minutes)

TaskTimeDeliverable
Run tofu plan5 minZero-change confirmation
Verify state accuracy5 minState matches reality
Fix any drift issues5 minUpdated configuration

Phase 3: GitOps Integration (15 minutes)

TaskTimeDeliverable
Configure remote state backend5 minGCS bucket + backend.tf
Migrate local state to GCS5 minState in GCS
Commit configuration to Git5 minIaC version-controlled

Total Time: 1 hour (conservative estimate)


Success Metrics

Immediate (Day 1):

  • ✅ All staging resources imported into OpenTofu state
  • tofu plan shows zero changes
  • ✅ Configuration committed to Git
  • ✅ Remote state stored in GCS

Week 1:

  • ✅ Team using tofu plan for drift detection
  • ✅ Infrastructure changes via IaC (not manual)
  • ✅ Documented operational workflows

Month 1:

  • ✅ Production environment created from IaC
  • ✅ Zero manual infrastructure changes
  • ✅ Disaster recovery tested (recreate from code)

Production Readiness Gaps (Address Before Production)

P0 (Must fix):

  1. Enable SSL on Cloud SQL

    enable_ssl = true
  2. Enable Redis AUTH

    enable_auth = true
  3. Use GCP Secret Manager for all secrets

    • Database passwords
    • Redis AUTH tokens
    • Django SECRET_KEY
  4. Configure Cloud KMS for license signing

    module "kms" {
    source = "../../modules/kms"
    keyring_name = "license-signing-keyring"
    key_name = "license-signing-key"
    }

P1 (Before production):

  1. Reserved static IP for LoadBalancer

    gcloud compute addresses create coditect-prod-lb-ip --global
  2. Specific ALLOWED_HOSTS domains

    • No wildcards (*) in production
    • Exact domain list
  3. Automated database backups

    backup_configuration {
    enabled = true
    start_time = "03:00"
    point_in_time_recovery_enabled = true
    }
  4. Monitoring and alerting

    • Cloud Monitoring dashboards
    • Alerting policies for downtime
    • SLO/SLA tracking

P2 (Nice to have):

  1. Multi-region deployment
  2. Read replicas for database
  3. Redis Cluster mode (STANDARD_HA tier)
  4. CI/CD automation (GitHub Actions)

Next Immediate Actions

Tomorrow (2 hours):

  1. Verify OpenTofu modules exist

    cd submodules/cloud/coditect-cloud-infra
    ls opentofu/modules/
  2. Create staging environment configuration (if modules exist)

    • Use module structure above
    • Reference existing modules
  3. Initialize and import resources

    • Follow Phase 1 steps
    • Import Cloud SQL and Redis
  4. Validate zero-change plan

    • Run tofu plan
    • Fix any drift
  5. Commit to Git

    • Version-control configuration
    • Update master repository pointer

This Week:

  1. Production planning

    • Design production architecture
    • Plan security hardening
    • Configure monitoring/alerting
  2. OpenTofu production environment

    • Create opentofu/environments/production/
    • Apply production-specific settings
  3. Disaster recovery testing

    • Destroy and recreate staging from code
    • Verify complete reproducibility

Benefits Realized After Migration

Immediate:

  • ✅ Complete infrastructure reproducibility
  • ✅ Drift detection (manual changes visible)
  • ✅ Version-controlled infrastructure
  • ✅ Audit trail via Git history

Short-term (Week 1):

  • ✅ Faster environment creation (staging → production)
  • ✅ Safer infrastructure changes (preview with tofu plan)
  • ✅ Team collaboration (shared codebase)

Long-term (Month 1+):

  • ✅ Disaster recovery capability (recreate from code)
  • ✅ Compliance (infrastructure documented)
  • ✅ Cost optimization (identify unused resources)
  • ✅ Multi-environment parity (staging = production)

Documentation References

Related Documents:

External Resources:


Created: December 1, 2025, 3:45 AM EST Status: Ready to Execute Estimated Effort: 1-2 hours Next Step: Verify OpenTofu module structure in coditect-cloud-infra repository

For Questions: See related documentation or CLAUDE.md for AI agent context