Skip to main content

CI/CL End-to-End (MVP)

1. Scope​

This document defines the end-to-end Continuous Integration and Continuous Learning (CI/CL) process for the Coditect Flow Platform MVP. It is anchored to existing CODITECT infrastructure patterns and OpenTofu workflows.

2. Definitions​

  • CI (Continuous Integration): Automated build, test, and security validation for code and infrastructure changes.
  • CL (Continuous Learning): Controlled feedback and evaluation loop that improves workflows, models, and operations based on production telemetry and audits.
  • CD (Deployment): Included as a required CI output stage because MVP must ship to GKE. CD is described explicitly to remove ambiguity.

3. Repositories and Pipelines in Scope​

  • submodules/cloud/coditect-cloud-infra (OpenTofu + GKE + Cloud SQL + Redis)
  • coditect-flow-platform (new platform: Rust runtime + Workbench UI)
  • submodules/cloud/coditect-cloud-backend (legacy reference, deployment status tracked)
  • submodules/cloud/coditect-cloud-frontend (legacy reference for UI patterns)

4. CI Pipeline Overview (Code)​

The code CI pipeline applies to the platform runtime, control plane, and UI.

4.1 Required CI Stages​

  1. Lint and format checks
  2. Unit tests
  3. Integration tests
  4. Security scans (SAST + dependency audit)
  5. Container build
  6. Container vulnerability scan
  7. Artifact signing and promotion

4.2 Required CI Artifacts​

  • Test reports (JUnit or equivalent)
  • Coverage reports
  • SBOM (Software Bill of Materials)
  • Signed container image digest

5. CI Pipeline Overview (Infrastructure)​

Infrastructure CI is based on existing OpenTofu workflows in submodules/cloud/coditect-cloud-infra.

5.1 Validation and Plan​

  • Workflow: .github/workflows/tofu-validate.yml
  • Workflow: .github/workflows/tofu-plan.yml
  • Inputs: opentofu/environments/{dev,staging,production}
  • Outputs: tfplan and plan-readable.txt artifacts

5.2 Apply and Verification​

  • Workflow: .github/workflows/tofu-apply.yml
  • Manual approval required unless auto_approve is set.
  • Post-deploy verification checks: GKE cluster, Cloud SQL instance, Redis instance.

5.3 Drift Detection​

  • Workflow: .github/workflows/tofu-drift-detection.yml
  • Scheduled daily, creates issues on drift detection.

6. CD (Deployment) Workflow (MVP)​

Deployment targets GKE and uses Artifact Registry.

6.1 Deployment Steps​

  1. Build container images (Cloud Build or GitHub Actions)
  2. Push to Artifact Registry with immutability enabled
  3. Deploy to GKE using Kustomize overlays
  4. Verify service health and ingress
  5. Publish deployment summary and audit record

6.2 Required Environments​

  • dev: continuous deployment on merge to main
  • staging: manual promotion from dev
  • production: manual approval gate + change window

6.3 Rollback​

  • Roll back to last known good image digest
  • Record rollback in audit chain

7. Continuous Learning (CL) Workflow​

CL is mandatory for the AI-first platform and must be auditable.

7.1 Inputs​

  • Trace data from OpenTelemetry
  • Error budgets and SLO violations
  • Audit logs and compliance events
  • User feedback and session logs

7.2 CL Stages​

  1. Ingest and normalize telemetry
  2. Evaluate workflow quality and latency regressions
  3. Generate improvement candidates
  4. Require human approval for changes that affect production behavior
  5. Deploy improvements through CI/CD

7.3 Outputs​

  • Updated workflow templates
  • Provider routing adjustments
  • Updated operational runbooks

8. Security and Compliance Gates​

  • All CI/CD steps must emit audit logs.
  • Production deploys require explicit approval.
  • Secrets are stored only in GCP Secret Manager.
  • No production deploys are allowed without passing security scans.

9. Observability Requirements​

  • CI and CD events must emit structured logs.
  • Every deployment must be traceable by commit hash, artifact digest, and environment.
  • SLO violations trigger CL review.

10. Known Gaps from Existing Pipelines​

  • tofu-validate.yml runs tflint in a terraform directory that does not exist in the infra repo.
  • Application CI/CD is not fully centralized for the new platform; MVP must add unified workflows.

11. Owner Decisions Required​

  • Confirm the system of record for build artifacts.
  • Confirm which projects host the GKE cluster and Artifact Registry.
  • Confirm which pipeline system is authoritative for CD (Cloud Build vs GitHub Actions).

12. Compliance Evidence​

  • Plan artifacts: stored for 30 days.
  • Apply logs: stored for 90 days.
  • Drift detection issues: permanent record.