Contract Testing Skill

Contract Testing Skill

When to Use This Skill

Use this skill when implementing contract testing patterns in your codebase.

How to Use This Skill

1. Review the patterns and examples below
2. Apply the relevant patterns to your implementation
3. Follow the best practices outlined in this skill

Consumer-driven contract testing patterns ensuring API compatibility between services with Pact and schema validation.

Pact Overview

Consumer-Driven Workflow

1. Consumer writes test defining expected API behavior
2. Pact generates contract from test
3. Contract published to Pact Broker
4. Provider verifies contract against implementation
5. Both sides remain compatible

JavaScript/TypeScript Implementation

Consumer Side Setup

npm install --save-dev @pact-foundation/pact

// consumer/tests/userApi.pact.spec.ts
import { Pact, Matchers } from '@pact-foundation/pact'
import { UserApiClient } from '../src/UserApiClient'
import path from 'path'

const { like, eachLike, regex } = Matchers

describe('User API Consumer', () => {
  const provider = new Pact({
    consumer: 'UserWebApp',
    provider: 'UserService',
    port: 1234,
    log: path.resolve(process.cwd(), 'logs', 'pact.log'),
    dir: path.resolve(process.cwd(), 'pacts'),
    logLevel: 'warn'
  })

  beforeAll(() => provider.setup())
  afterEach(() => provider.verify())
  afterAll(() => provider.finalize())

  describe('GET /users/:id', () => {
    const expectedUser = {
      id: 1,
      name: 'John Doe',
      email: 'john@example.com',
      createdAt: '2024-01-15T10:30:00Z'
    }

    beforeEach(() => {
      return provider.addInteraction({
        state: 'user with id 1 exists',
        uponReceiving: 'a request for user 1',
        withRequest: {
          method: 'GET',
          path: '/users/1',
          headers: {
            Accept: 'application/json',
            Authorization: regex(/Bearer .+/, 'Bearer token123')
          }
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json'
          },
          body: like(expectedUser)
        }
      })
    })

    it('returns the user', async () => {
      const client = new UserApiClient(`http://localhost:${provider.opts.port}`)
      const user = await client.getUser(1)

      expect(user.id).toBe(1)
      expect(user.name).toBe('John Doe')
    })
  })

  describe('GET /users', () => {
    beforeEach(() => {
      return provider.addInteraction({
        state: 'users exist',
        uponReceiving: 'a request for all users',
        withRequest: {
          method: 'GET',
          path: '/users',
          query: {
            page: '1',
            limit: '10'
          }
        },
        willRespondWith: {
          status: 200,
          body: {
            users: eachLike({
              id: like(1),
              name: like('User Name'),
              email: regex(/\S+@\S+\.\S+/, 'user@example.com')
            }),
            total: like(100),
            page: like(1),
            limit: like(10)
          }
        }
      })
    })

    it('returns paginated users', async () => {
      const client = new UserApiClient(`http://localhost:${provider.opts.port}`)
      const result = await client.getUsers({ page: 1, limit: 10 })

      expect(result.users).toHaveLength(1)
      expect(result.total).toBe(100)
    })
  })

  describe('POST /users', () => {
    beforeEach(() => {
      return provider.addInteraction({
        state: 'no users exist',
        uponReceiving: 'a request to create a user',
        withRequest: {
          method: 'POST',
          path: '/users',
          headers: {
            'Content-Type': 'application/json'
          },
          body: {
            name: 'New User',
            email: 'new@example.com'
          }
        },
        willRespondWith: {
          status: 201,
          body: {
            id: like(1),
            name: 'New User',
            email: 'new@example.com',
            createdAt: regex(
              /\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z/,
              '2024-01-15T10:30:00Z'
            )
          }
        }
      })
    })

    it('creates a user', async () => {
      const client = new UserApiClient(`http://localhost:${provider.opts.port}`)
      const user = await client.createUser({
        name: 'New User',
        email: 'new@example.com'
      })

      expect(user.id).toBeDefined()
      expect(user.name).toBe('New User')
    })
  })
})

Provider Side Verification

// provider/tests/pact.provider.spec.ts
import { Verifier } from '@pact-foundation/pact'
import { app } from '../src/app'
import { Server } from 'http'

describe('Pact Verification', () => {
  let server: Server
  const port = 3001

  beforeAll((done) => {
    server = app.listen(port, done)
  })

  afterAll((done) => {
    server.close(done)
  })

  it('validates the expectations of UserWebApp', async () => {
    const verifier = new Verifier({
      provider: 'UserService',
      providerBaseUrl: `http://localhost:${port}`,

      // Local pact files
      pactUrls: ['../consumer/pacts/userweb-userservice.json'],

      // OR from Pact Broker
      pactBrokerUrl: process.env.PACT_BROKER_URL,
      pactBrokerToken: process.env.PACT_BROKER_TOKEN,
      consumerVersionSelectors: [
        { mainBranch: true },
        { deployedOrReleased: true }
      ],

      // Provider states
      stateHandlers: {
        'user with id 1 exists': async () => {
          await db.users.create({
            id: 1,
            name: 'John Doe',
            email: 'john@example.com'
          })
        },
        'users exist': async () => {
          await db.users.createMany([
            { id: 1, name: 'User 1', email: 'user1@example.com' },
            { id: 2, name: 'User 2', email: 'user2@example.com' }
          ])
        },
        'no users exist': async () => {
          await db.users.deleteAll()
        }
      },

      // Publish results
      publishVerificationResult: process.env.CI === 'true',
      providerVersion: process.env.GIT_SHA,
      providerVersionBranch: process.env.GIT_BRANCH
    })

    await verifier.verifyProvider()
  })
})

Python Implementation

Consumer (pytest-pact)

# consumer/tests/test_user_api_pact.py
import pytest
from pact import Consumer, Provider, Like, EachLike, Regex
from user_client import UserApiClient

pact = Consumer('UserWebApp').has_pact_with(
    Provider('UserService'),
    pact_dir='./pacts'
)

@pytest.fixture
def client():
    pact.start_service()
    yield UserApiClient(pact.uri)
    pact.stop_service()

def test_get_user(client):
    expected = {
        'id': 1,
        'name': 'John Doe',
        'email': 'john@example.com'
    }

    (pact
        .given('user with id 1 exists')
        .upon_receiving('a request for user 1')
        .with_request('GET', '/users/1')
        .will_respond_with(200, body=Like(expected)))

    with pact:
        user = client.get_user(1)
        assert user['id'] == 1

def test_get_users(client):
    (pact
        .given('users exist')
        .upon_receiving('a request for all users')
        .with_request('GET', '/users', query={'page': '1', 'limit': '10'})
        .will_respond_with(200, body={
            'users': EachLike({
                'id': Like(1),
                'name': Like('User'),
                'email': Regex(r'\S+@\S+\.\S+', 'user@example.com')
            }),
            'total': Like(100)
        }))

    with pact:
        result = client.get_users(page=1, limit=10)
        assert len(result['users']) >= 1

Provider Verification (Python)

# provider/tests/test_pact_provider.py
import pytest
from pact import Verifier
from app import create_app
from db import db

@pytest.fixture(scope='module')
def app():
    app = create_app(testing=True)
    with app.app_context():
        db.create_all()
        yield app
        db.drop_all()

def test_provider_verification(app):
    verifier = Verifier(
        provider='UserService',
        provider_base_url='http://localhost:5000'
    )

    # Setup provider states
    @verifier.state_handler('user with id 1 exists')
    def user_exists():
        db.session.add(User(id=1, name='John Doe', email='john@example.com'))
        db.session.commit()

    @verifier.state_handler('no users exist')
    def no_users():
        User.query.delete()
        db.session.commit()

    # Verify
    success, logs = verifier.verify_pacts(
        './pacts/userweb-userservice.json',
        verbose=True
    )

    assert success

Pact Broker

Publishing Contracts

# Publish pact file to broker
pact-broker publish ./pacts \
  --consumer-app-version=$(git rev-parse HEAD) \
  --branch=$(git branch --show-current) \
  --broker-base-url=$PACT_BROKER_URL \
  --broker-token=$PACT_BROKER_TOKEN

# Tag with environment
pact-broker create-version-tag \
  --pacticipant=UserWebApp \
  --version=$(git rev-parse HEAD) \
  --tag=production

Can I Deploy

# Check if safe to deploy
pact-broker can-i-deploy \
  --pacticipant=UserWebApp \
  --version=$(git rev-parse HEAD) \
  --to-environment=production

# Check both services
pact-broker can-i-deploy \
  --pacticipant=UserWebApp \
  --version=$(git rev-parse HEAD) \
  --pacticipant=UserService \
  --version=$(git rev-parse HEAD) \
  --to-environment=production

CI Integration

GitHub Actions

# Consumer CI
name: Consumer Contract Tests

on: [push, pull_request]

jobs:
  contract-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci
      - run: npm run test:pact

      - name: Publish Pacts
        if: github.ref == 'refs/heads/main'
        run: |
          npx pact-broker publish ./pacts \
            --consumer-app-version=${{ github.sha }} \
            --branch=${{ github.ref_name }} \
            --broker-base-url=${{ secrets.PACT_BROKER_URL }} \
            --broker-token=${{ secrets.PACT_BROKER_TOKEN }}

---
# Provider CI
name: Provider Contract Verification

on:
  push:
  workflow_dispatch:
    inputs:
      pact_url:
        description: 'Pact URL to verify'
        required: false

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci
      - run: npm run test:pact:provider
        env:
          PACT_BROKER_URL: ${{ secrets.PACT_BROKER_URL }}
          PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
          GIT_SHA: ${{ github.sha }}
          GIT_BRANCH: ${{ github.ref_name }}

      - name: Can I Deploy
        run: |
          npx pact-broker can-i-deploy \
            --pacticipant=UserService \
            --version=${{ github.sha }} \
            --to-environment=production

Webhook Triggers

# Pact Broker webhook configuration
{
  "events": ["contract_content_changed"],
  "request": {
    "method": "POST",
    "url": "https://api.github.com/repos/org/provider-repo/dispatches",
    "headers": {
      "Authorization": "Bearer ${githubToken}",
      "Content-Type": "application/json"
    },
    "body": {
      "event_type": "pact_changed",
      "client_payload": {
        "pact_url": "${pactbroker.pactUrl}"
      }
    }
  }
}

Schema Validation

JSON Schema Contracts

// schemas/user.schema.json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["id", "name", "email"],
  "properties": {
    "id": { "type": "integer" },
    "name": { "type": "string", "minLength": 1 },
    "email": { "type": "string", "format": "email" },
    "createdAt": { "type": "string", "format": "date-time" }
  },
  "additionalProperties": false
}

// Validate in tests
import Ajv from 'ajv'
import addFormats from 'ajv-formats'
import userSchema from './schemas/user.schema.json'

const ajv = new Ajv()
addFormats(ajv)
const validate = ajv.compile(userSchema)

test('API response matches schema', async () => {
  const response = await api.getUser(1)
  const valid = validate(response)
  expect(valid).toBe(true)
  if (!valid) console.log(validate.errors)
})

Usage Examples

Setup Consumer Contract Tests

Apply contract-testing skill to implement Pact consumer tests for REST API client

Configure Provider Verification

Apply contract-testing skill to setup provider verification with state handlers and CI integration

Implement Can-I-Deploy

Apply contract-testing skill to add deployment safety checks with Pact Broker can-i-deploy

Success Output

When successful, this skill MUST output:

✅ SKILL COMPLETE: contract-testing

Completed:
- [x] Consumer Pact tests implemented
- [x] Provider verification passing
- [x] Contracts published to Pact Broker
- [x] CI integration configured
- [x] can-i-deploy checks enabled

Outputs:
- consumer/tests/userApi.pact.spec.ts (Consumer tests)
- provider/tests/pact.provider.spec.ts (Provider verification)
- pacts/userweb-userservice.json (Generated contract)
- .github/workflows/contract-tests.yml (CI pipeline)

Verification: PASSED (all interactions verified)

Completion Checklist

Before marking this skill as complete, verify:

• Consumer tests generate pact file successfully
• All consumer interactions have provider states
• Provider verification passes all expectations
• State handlers properly setup test data
• Pact contracts published to broker
• CI runs contract tests on every push
• can-i-deploy checks gate deployments
• Webhooks trigger provider verification on contract changes
• Both consumer and provider teams can deploy independently
• Matchers used appropriately (like, eachLike, regex)

Failure Indicators

This skill has FAILED if:

• ❌ Consumer tests fail to generate pact file
• ❌ Provider verification fails with mismatches
• ❌ State handlers don't setup required data
• ❌ Pact Broker unreachable or contracts not published
• ❌ can-i-deploy returns "Computer says no"
• ❌ CI doesn't run contract tests
• ❌ Provider not notified of contract changes
• ❌ Breaking changes deployed without verification
• ❌ Matchers too strict (exact match where like() needed)

When NOT to Use

Do NOT use this skill when:

• Single monolithic application (no service boundaries, use integration tests instead)
• Services deployed together always (use integration-testing instead)
• GraphQL APIs (use schema stitching validation instead)
• gRPC services (use protobuf compatibility checks instead)
• Internal library changes (use semantic versioning + tests)
• No independent deployment needed
• Team too small to justify broker infrastructure
• Services communicate only via async messaging (use message contract testing)

Anti-Patterns (Avoid)

Anti-Pattern	Problem	Solution
No provider states	Tests fail unpredictably	Define states for all scenarios
Exact matching everything	Brittle tests	Use like(), eachLike() for flexibility
Not publishing contracts	No deployment safety checks	Publish on main branch always
Skipping can-i-deploy	Breaking changes reach production	Gate deployments with verification
No webhook triggers	Provider not notified of changes	Configure broker webhooks
Testing implementation details	Couples services too tightly	Test contract, not internals
Shared pact files manually	Version conflicts	Use Pact Broker as source of truth
No versioning strategy	Can't track compatibility	Tag contracts with versions/environments

Principles

This skill embodies:

• #2 Security First - Ensures API compatibility before deployment
• #3 Separation of Concerns - Consumer and provider test independently
• #5 Eliminate Ambiguity - Clear contract defines expectations
• #8 No Assumptions - Verifies compatibility, doesn't assume
• #10 Test Everything - Both sides validate the contract

Full Standard: CODITECT-STANDARD-AUTOMATION.md