OpenTofu Migration - Next Steps and Implementation Plan
Date: December 1, 2025 Status: Planning Phase - Ready to Execute Current State: Manual Infrastructure (100% Functional) Target State: Infrastructure as Code (IaC) with OpenTofu
Executive Summary
Why Migrate to OpenTofu?
The staging environment was successfully deployed using manual gcloud commands. While functional, this approach has significant limitations:
| Aspect | Manual (Current) | OpenTofu (Target) |
|---|---|---|
| Reproducibility | ❌ Tribal knowledge only | ✅ Code-defined, version-controlled |
| Drift Detection | ❌ No visibility | ✅ Automatic detection via tofu plan |
| Disaster Recovery | ❌ Manual recreation | ✅ Re-apply configuration |
| Environment Parity | ❌ Manual synchronization | ✅ Same code, different variables |
| Change Tracking | ❌ No audit trail | ✅ Git history |
| Team Collaboration | ❌ Knowledge silos | ✅ Shared codebase |
Investment Required: 1-2 hours (one-time) Benefits: Permanent infrastructure reproducibility and drift prevention Risk: Low - Import existing resources without recreating
Current Infrastructure Inventory
What We Manually Created (December 1, 2025)
Cloud SQL PostgreSQL:
Instance Name: coditect-db
Version: POSTGRES_16
Tier: db-f1-micro
Region: us-central1
Network: projects/coditect-cloud-infra/global/networks/default
Private IP: 10.28.0.3
SSL: Disabled (staging only)
Database: coditect
User: coditect_app
Redis Memorystore:
Instance Name: coditect-redis-staging
Version: redis_7_0
Tier: BASIC
Memory: 1GB
Region: us-central1
Network: default
Host: 10.164.210.91
AUTH: Disabled (staging only)
GKE Resources:
Cluster: coditect-cluster
Namespace: coditect-staging
Deployment: coditect-backend (2 replicas)
Service: coditect-backend (LoadBalancer)
External IP: 136.114.0.156
ConfigMaps: backend-config
Secrets: backend-secrets
Docker Images:
Repository: us-central1-docker.pkg.dev/coditect-cloud-infra/coditect-backend
Latest: coditect-cloud-backend:v1.0.3-staging
Networking:
VPC: default
Subnetworks: Auto-created
Private Service Connection: Established for Cloud SQL
LoadBalancer: GKE-managed (136.114.0.156)
Existing OpenTofu Module Structure
Location: submodules/cloud/coditect-cloud-infra/opentofu/
Expected Directory Structure:
opentofu/
├── modules/
│ ├── database/ # Cloud SQL module
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── redis/ # Redis Memorystore module
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── kubernetes/ # GKE cluster module
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── networking/ # VPC and networking
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
├── environments/
│ ├── staging/ # Need to create
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ ├── terraform.tfvars
│ │ └── backend.tf
│ └── production/ # Future
│ └── (same structure)
└── backend-config/
└── staging-backend.hcl
Migration Strategy: Import Existing Resources
Philosophy: Import existing resources into OpenTofu state WITHOUT recreating them.
3-Phase Approach:
Phase 1: State Import (30 minutes)
Import manually-created resources into OpenTofu state
Phase 2: Validation (15 minutes)
Verify tofu plan shows no changes
Phase 3: GitOps Integration (15 minutes)
Commit configuration and establish workflow
Phase 1: Import Existing Resources
Step 1.1: Create Staging Environment Configuration
File: opentofu/environments/staging/main.tf
terraform {
required_version = ">= 1.6.0"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.0"
}
}
backend "gcs" {
bucket = "coditect-terraform-state"
prefix = "staging"
}
}
provider "google" {
project = var.project_id
region = var.region
}
# Cloud SQL PostgreSQL
module "database" {
source = "../../modules/database"
project_id = var.project_id
region = var.region
instance_name = "coditect-db"
database_version = "POSTGRES_16"
tier = "db-f1-micro"
network = "default"
database_name = "coditect"
database_user = "coditect_app"
enable_ssl = false # Staging only
}
# Redis Memorystore
module "redis" {
source = "../../modules/redis"
project_id = var.project_id
region = var.region
instance_name = "coditect-redis-staging"
redis_version = "REDIS_7_0"
tier = "BASIC"
memory_size_gb = 1
network = "default"
enable_auth = false # Staging only
}
# GKE Cluster (if managing cluster via OpenTofu)
# Note: If cluster already managed elsewhere, omit this
module "kubernetes" {
source = "../../modules/kubernetes"
project_id = var.project_id
region = var.region
cluster_name = "coditect-cluster"
network = "default"
min_nodes = 1
max_nodes = 3
machine_type = "e2-standard-4"
}
File: opentofu/environments/staging/variables.tf
variable "project_id" {
description = "GCP Project ID"
type = string
default = "coditect-cloud-infra"
}
variable "region" {
description = "GCP Region"
type = string
default = "us-central1"
}
variable "environment" {
description = "Environment name"
type = string
default = "staging"
}
File: opentofu/environments/staging/terraform.tfvars
project_id = "coditect-cloud-infra"
region = "us-central1"
environment = "staging"
Step 1.2: Initialize OpenTofu
cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging
# Initialize OpenTofu
tofu init
# Expected output:
# Initializing the backend...
# Initializing provider plugins...
# OpenTofu has been successfully initialized!
Step 1.3: Import Cloud SQL Instance
# Import Cloud SQL instance
tofu import module.database.google_sql_database_instance.main \
coditect-cloud-infra/coditect-db
# Import database
tofu import module.database.google_sql_database.main \
coditect-cloud-infra/coditect-db/coditect
# Import database user
tofu import module.database.google_sql_user.main \
coditect-cloud-infra/coditect-db/coditect_app
Verification:
tofu state list | grep database
# Should show:
# module.database.google_sql_database_instance.main
# module.database.google_sql_database.main
# module.database.google_sql_user.main
Step 1.4: Import Redis Instance
# Import Redis instance
tofu import module.redis.google_redis_instance.main \
coditect-cloud-infra/us-central1/coditect-redis-staging
Verification:
tofu state list | grep redis
# Should show:
# module.redis.google_redis_instance.main
Step 1.5: Import GKE Cluster (Optional)
Decision Point: If GKE cluster is managed separately (e.g., via Google Cloud Console or separate IaC), you may choose NOT to import it.
If importing:
tofu import module.kubernetes.google_container_cluster.main \
coditect-cloud-infra/us-central1/coditect-cluster
Alternative: Manage GKE cluster separately and only manage Cloud SQL + Redis via OpenTofu.
Phase 2: Validation (Zero Changes Expected)
Step 2.1: Run OpenTofu Plan
cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging
tofu plan
Expected Output:
OpenTofu will perform the following actions:
No changes. Your infrastructure matches the configuration.
OpenTofu has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.
If Changes Detected:
This indicates configuration drift. Common causes:
- Missing configuration parameters - Add parameters to match current state
- Default values changed - Explicitly set values in configuration
- Computed values - Mark as
lifecycle { ignore_changes = [...] }
Example Fix:
resource "google_sql_database_instance" "main" {
# ... other configuration ...
lifecycle {
ignore_changes = [
# Ignore auto-assigned values
connection_name,
ip_address,
]
}
}
Step 2.2: Validate State Accuracy
# Show Cloud SQL instance
tofu state show module.database.google_sql_database_instance.main
# Verify matches actual resource
gcloud sql instances describe coditect-db
# Show Redis instance
tofu state show module.redis.google_redis_instance.main
# Verify matches actual resource
gcloud redis instances describe coditect-redis-staging --region=us-central1
Acceptance Criteria:
- ✅ All imported resources appear in
tofu state list - ✅
tofu planshows zero changes - ✅
tofu state showmatchesgcloud describeoutput
Phase 3: GitOps Integration
Step 3.1: Configure Remote State Backend
Create GCS bucket for state:
# Create bucket (if not exists)
gcloud storage buckets create gs://coditect-terraform-state \
--project=coditect-cloud-infra \
--location=us-central1 \
--uniform-bucket-level-access
# Enable versioning
gcloud storage buckets update gs://coditect-terraform-state \
--versioning
File: opentofu/environments/staging/backend.tf
terraform {
backend "gcs" {
bucket = "coditect-terraform-state"
prefix = "staging"
}
}
Migrate local state to GCS:
cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging
# Reconfigure backend
tofu init -migrate-state
# Confirm migration when prompted
Step 3.2: Commit Configuration to Git
cd submodules/cloud/coditect-cloud-infra
# Add OpenTofu configuration
git add opentofu/environments/staging/
git add opentofu/modules/ # If modules don't exist yet
# Commit
git commit -m "feat: Add OpenTofu staging environment configuration
- Import existing Cloud SQL and Redis resources
- Configure remote state backend (GCS)
- Establish IaC for staging infrastructure
- Zero-change validation confirmed
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>"
# Push
git push origin main
Step 3.3: Update Master Repository Pointer
cd /Users/halcasteel/PROJECTS/coditect-rollout-master
# Update submodule pointer
git add submodules/cloud/coditect-cloud-infra
# Commit pointer update
git commit -m "chore: Update coditect-cloud-infra submodule - OpenTofu staging config
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>"
# Push
git push origin main
Operational Workflows
Daily: Check for Drift
cd submodules/cloud/coditect-cloud-infra/opentofu/environments/staging
# Run plan to detect any manual changes
tofu plan
# If changes detected, review and decide:
# 1. Update OpenTofu config to match manual change
# 2. Apply OpenTofu config to revert manual change
Making Infrastructure Changes
Old Way (Manual):
gcloud sql instances patch coditect-db --tier=db-n1-standard-1
New Way (IaC):
# 1. Edit configuration
vim opentofu/environments/staging/main.tf
# Change: tier = "db-n1-standard-1"
# 2. Preview changes
tofu plan
# 3. Apply changes
tofu apply
# 4. Commit configuration
git add opentofu/environments/staging/main.tf
git commit -m "feat: Upgrade staging database tier to db-n1-standard-1"
git push
Creating Production Environment
# Copy staging configuration
cp -r opentofu/environments/staging opentofu/environments/production
# Update production-specific values
vim opentofu/environments/production/terraform.tfvars
# Production differences:
# - Enable SSL on Cloud SQL
# - Enable Redis AUTH
# - Larger instance sizes
# - Different database names/users
# - Reserved static IP for LoadBalancer
# Initialize production
cd opentofu/environments/production
tofu init
# Create production resources
tofu apply
Module Development (If Modules Don't Exist)
Cloud SQL Module
File: opentofu/modules/database/main.tf
resource "google_sql_database_instance" "main" {
name = var.instance_name
database_version = var.database_version
region = var.region
project = var.project_id
settings {
tier = var.tier
ip_configuration {
ipv4_enabled = false
private_network = "projects/${var.project_id}/global/networks/${var.network}"
require_ssl = var.enable_ssl
}
backup_configuration {
enabled = true
start_time = "03:00"
}
}
deletion_protection = true
}
resource "google_sql_database" "main" {
name = var.database_name
instance = google_sql_database_instance.main.name
project = var.project_id
}
resource "google_sql_user" "main" {
name = var.database_user
instance = google_sql_database_instance.main.name
password = var.database_password
project = var.project_id
}
File: opentofu/modules/database/variables.tf
variable "project_id" {
description = "GCP Project ID"
type = string
}
variable "region" {
description = "GCP Region"
type = string
}
variable "instance_name" {
description = "Cloud SQL instance name"
type = string
}
variable "database_version" {
description = "Database version (e.g., POSTGRES_16)"
type = string
}
variable "tier" {
description = "Machine tier (e.g., db-f1-micro)"
type = string
}
variable "network" {
description = "VPC network name"
type = string
}
variable "database_name" {
description = "Database name to create"
type = string
}
variable "database_user" {
description = "Database user to create"
type = string
}
variable "database_password" {
description = "Database user password"
type = string
sensitive = true
}
variable "enable_ssl" {
description = "Require SSL connections"
type = bool
default = true
}
File: opentofu/modules/database/outputs.tf
output "instance_name" {
description = "Cloud SQL instance name"
value = google_sql_database_instance.main.name
}
output "private_ip_address" {
description = "Private IP address"
value = google_sql_database_instance.main.private_ip_address
}
output "connection_name" {
description = "Connection name"
value = google_sql_database_instance.main.connection_name
}
output "database_name" {
description = "Database name"
value = google_sql_database.main.name
}
Redis Module
File: opentofu/modules/redis/main.tf
resource "google_redis_instance" "main" {
name = var.instance_name
memory_size_gb = var.memory_size_gb
redis_version = var.redis_version
tier = var.tier
region = var.region
project = var.project_id
authorized_network = "projects/${var.project_id}/global/networks/${var.network}"
auth_enabled = var.enable_auth
redis_configs = {
maxmemory-policy = "allkeys-lru"
}
}
File: opentofu/modules/redis/variables.tf
variable "project_id" {
description = "GCP Project ID"
type = string
}
variable "region" {
description = "GCP Region"
type = string
}
variable "instance_name" {
description = "Redis instance name"
type = string
}
variable "redis_version" {
description = "Redis version (e.g., REDIS_7_0)"
type = string
}
variable "tier" {
description = "Service tier (BASIC or STANDARD_HA)"
type = string
}
variable "memory_size_gb" {
description = "Memory size in GB"
type = number
}
variable "network" {
description = "VPC network name"
type = string
}
variable "enable_auth" {
description = "Enable Redis AUTH"
type = bool
default = true
}
File: opentofu/modules/redis/outputs.tf
output "instance_name" {
description = "Redis instance name"
value = google_redis_instance.main.name
}
output "host" {
description = "Redis host IP"
value = google_redis_instance.main.host
}
output "port" {
description = "Redis port"
value = google_redis_instance.main.port
}
output "auth_string" {
description = "Redis AUTH string"
value = google_redis_instance.main.auth_string
sensitive = true
}
Security Considerations
Secret Management
NEVER commit to Git:
- Database passwords
- Redis AUTH tokens
- Service account keys
- API keys
Use GCP Secret Manager:
# Store database password
gcloud secrets create staging-db-password \
--data-file=- <<< "YOUR_PASSWORD_HERE" \
--project=coditect-cloud-infra
# Grant OpenTofu service account access
gcloud secrets add-iam-policy-binding staging-db-password \
--member="serviceAccount:terraform@coditect-cloud-infra.iam.gserviceaccount.com" \
--role="roles/secretmanager.secretAccessor"
Reference in OpenTofu:
data "google_secret_manager_secret_version" "db_password" {
secret = "staging-db-password"
project = var.project_id
}
module "database" {
source = "../../modules/database"
# ... other config ...
database_password = data.google_secret_manager_secret_version.db_password.secret_data
}
State File Security
GCS bucket ACLs:
# Restrict access to terraform service account only
gcloud storage buckets add-iam-policy-binding gs://coditect-terraform-state \
--member="serviceAccount:terraform@coditect-cloud-infra.iam.gserviceaccount.com" \
--role="roles/storage.objectAdmin"
Enable encryption:
# Use customer-managed encryption key (CMEK)
gcloud storage buckets update gs://coditect-terraform-state \
--default-encryption-key=projects/coditect-cloud-infra/locations/us-central1/keyRings/terraform/cryptoKeys/state-encryption
Rollback Plan
If OpenTofu migration causes issues:
Option 1: Remove from State (Keep Resources)
# Remove resource from OpenTofu management
tofu state rm module.database.google_sql_database_instance.main
# Resource continues to exist, just not managed by OpenTofu
# Can manage manually again
Option 2: Destroy and Recreate Manually
# Destroy OpenTofu-managed resource
tofu destroy -target=module.database.google_sql_database_instance.main
# Recreate manually using gcloud commands
gcloud sql instances create coditect-db ...
Option 3: Rollback State
# List state versions
gcloud storage ls gs://coditect-terraform-state/staging/
# Download previous state version
gcloud storage cp gs://coditect-terraform-state/staging/default.tfstate#VERSION ./terraform.tfstate
# Replace current state
tofu state push terraform.tfstate
Timeline and Effort Estimates
Phase 1: Import Existing Resources (30 minutes)
| Task | Time | Deliverable |
|---|---|---|
| Create staging environment configuration | 10 min | main.tf, variables.tf, terraform.tfvars |
| Initialize OpenTofu | 5 min | .terraform/ directory |
| Import Cloud SQL instance | 5 min | Cloud SQL in state |
| Import Redis instance | 5 min | Redis in state |
| Import GKE cluster (optional) | 5 min | GKE in state |
Phase 2: Validation (15 minutes)
| Task | Time | Deliverable |
|---|---|---|
Run tofu plan | 5 min | Zero-change confirmation |
| Verify state accuracy | 5 min | State matches reality |
| Fix any drift issues | 5 min | Updated configuration |
Phase 3: GitOps Integration (15 minutes)
| Task | Time | Deliverable |
|---|---|---|
| Configure remote state backend | 5 min | GCS bucket + backend.tf |
| Migrate local state to GCS | 5 min | State in GCS |
| Commit configuration to Git | 5 min | IaC version-controlled |
Total Time: 1 hour (conservative estimate)
Success Metrics
Immediate (Day 1):
- ✅ All staging resources imported into OpenTofu state
- ✅
tofu planshows zero changes - ✅ Configuration committed to Git
- ✅ Remote state stored in GCS
Week 1:
- ✅ Team using
tofu planfor drift detection - ✅ Infrastructure changes via IaC (not manual)
- ✅ Documented operational workflows
Month 1:
- ✅ Production environment created from IaC
- ✅ Zero manual infrastructure changes
- ✅ Disaster recovery tested (recreate from code)
Production Readiness Gaps (Address Before Production)
P0 (Must fix):
-
Enable SSL on Cloud SQL
enable_ssl = true -
Enable Redis AUTH
enable_auth = true -
Use GCP Secret Manager for all secrets
- Database passwords
- Redis AUTH tokens
- Django SECRET_KEY
-
Configure Cloud KMS for license signing
module "kms" {
source = "../../modules/kms"
keyring_name = "license-signing-keyring"
key_name = "license-signing-key"
}
P1 (Before production):
-
Reserved static IP for LoadBalancer
gcloud compute addresses create coditect-prod-lb-ip --global -
Specific ALLOWED_HOSTS domains
- No wildcards (
*) in production - Exact domain list
- No wildcards (
-
Automated database backups
backup_configuration {
enabled = true
start_time = "03:00"
point_in_time_recovery_enabled = true
} -
Monitoring and alerting
- Cloud Monitoring dashboards
- Alerting policies for downtime
- SLO/SLA tracking
P2 (Nice to have):
- Multi-region deployment
- Read replicas for database
- Redis Cluster mode (STANDARD_HA tier)
- CI/CD automation (GitHub Actions)
Next Immediate Actions
Tomorrow (2 hours):
-
✅ Verify OpenTofu modules exist
cd submodules/cloud/coditect-cloud-infra
ls opentofu/modules/ -
✅ Create staging environment configuration (if modules exist)
- Use module structure above
- Reference existing modules
-
✅ Initialize and import resources
- Follow Phase 1 steps
- Import Cloud SQL and Redis
-
✅ Validate zero-change plan
- Run
tofu plan - Fix any drift
- Run
-
✅ Commit to Git
- Version-control configuration
- Update master repository pointer
This Week:
-
✅ Production planning
- Design production architecture
- Plan security hardening
- Configure monitoring/alerting
-
✅ OpenTofu production environment
- Create
opentofu/environments/production/ - Apply production-specific settings
- Create
-
✅ Disaster recovery testing
- Destroy and recreate staging from code
- Verify complete reproducibility
Benefits Realized After Migration
Immediate:
- ✅ Complete infrastructure reproducibility
- ✅ Drift detection (manual changes visible)
- ✅ Version-controlled infrastructure
- ✅ Audit trail via Git history
Short-term (Week 1):
- ✅ Faster environment creation (staging → production)
- ✅ Safer infrastructure changes (preview with
tofu plan) - ✅ Team collaboration (shared codebase)
Long-term (Month 1+):
- ✅ Disaster recovery capability (recreate from code)
- ✅ Compliance (infrastructure documented)
- ✅ Cost optimization (identify unused resources)
- ✅ Multi-environment parity (staging = production)
Documentation References
Related Documents:
- staging-quick-reference.md - Current staging infrastructure
- deployment-night-summary.md - Manual deployment process
- staging-deployment-guide.md - Step-by-step manual deployment
- infrastructure-pivot-summary.md - OpenTofu migration rationale
External Resources:
Created: December 1, 2025, 3:45 AM EST Status: Ready to Execute Estimated Effort: 1-2 hours Next Step: Verify OpenTofu module structure in coditect-cloud-infra repository
For Questions: See related documentation or CLAUDE.md for AI agent context