Visual Regression Skill

Visual Regression Skill

When to Use This Skill

Use this skill when implementing visual regression patterns in your codebase.

How to Use This Skill

1. Review the patterns and examples below
2. Apply the relevant patterns to your implementation
3. Follow the best practices outlined in this skill

Visual regression testing patterns for catching unintended UI changes with automated screenshot comparison.

Percy

Setup

npm install --save-dev @percy/cli @percy/playwright
# or
npm install --save-dev @percy/cypress

Playwright Integration

// playwright.config.ts
import { defineConfig } from '@playwright/test'

export default defineConfig({
  use: {
    // Consistent viewport for screenshots
    viewport: { width: 1280, height: 720 }
  }
})

// tests/visual.spec.ts
import { test } from '@playwright/test'
import percySnapshot from '@percy/playwright'

test.describe('Visual Regression', () => {
  test('homepage visual test', async ({ page }) => {
    await page.goto('/')
    await page.waitForLoadState('networkidle')

    // Hide dynamic content
    await page.evaluate(() => {
      document.querySelectorAll('[data-testid="timestamp"]').forEach(el => {
        (el as HTMLElement).style.visibility = 'hidden'
      })
    })

    await percySnapshot(page, 'Homepage')
  })

  test('responsive breakpoints', async ({ page }) => {
    await page.goto('/dashboard')

    // Desktop
    await page.setViewportSize({ width: 1280, height: 720 })
    await percySnapshot(page, 'Dashboard - Desktop')

    // Tablet
    await page.setViewportSize({ width: 768, height: 1024 })
    await percySnapshot(page, 'Dashboard - Tablet')

    // Mobile
    await page.setViewportSize({ width: 375, height: 667 })
    await percySnapshot(page, 'Dashboard - Mobile')
  })

  test('component states', async ({ page }) => {
    await page.goto('/components')

    // Default state
    await percySnapshot(page, 'Button - Default')

    // Hover state
    await page.hover('[data-testid="primary-button"]')
    await percySnapshot(page, 'Button - Hover')

    // Focus state
    await page.focus('[data-testid="primary-button"]')
    await percySnapshot(page, 'Button - Focus')
  })
})

Cypress Integration

// cypress/e2e/visual.cy.ts
describe('Visual Regression', () => {
  it('captures homepage', () => {
    cy.visit('/')
    cy.percySnapshot('Homepage')
  })

  it('captures with options', () => {
    cy.visit('/dashboard')

    cy.percySnapshot('Dashboard', {
      widths: [375, 768, 1280],
      minHeight: 1024,
      percyCSS: `
        [data-testid="timestamp"] { visibility: hidden; }
        .ad-banner { display: none; }
      `
    })
  })
})

Percy Configuration

# .percy.yml
version: 2
snapshot:
  widths: [375, 768, 1280]
  min-height: 1024
  percy-css: |
    [data-testid="timestamp"] { visibility: hidden; }
    .dynamic-content { visibility: hidden; }

discovery:
  network-idle-timeout: 500
  disable-cache: true

upload:
  files: '**/*.html'
  ignore: '**/node_modules/**'

CI Integration

# .github/workflows/visual.yml
name: Visual Tests

on: [push, pull_request]

jobs:
  percy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci
      - run: npx playwright install --with-deps

      - name: Percy Test
        run: npx percy exec -- npx playwright test tests/visual/
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

Chromatic (Storybook)

Setup

npm install --save-dev chromatic
npx chromatic --project-token=<token>

Storybook Stories

// Button.stories.tsx
import type { Meta, StoryObj } from '@storybook/react'
import { Button } from './Button'

const meta: Meta<typeof Button> = {
  title: 'Components/Button',
  component: Button,
  parameters: {
    chromatic: {
      viewports: [375, 768, 1280],
      delay: 300 // Wait for animations
    }
  }
}

export default meta
type Story = StoryObj<typeof Button>

export const Primary: Story = {
  args: {
    variant: 'primary',
    children: 'Click me'
  }
}

export const Secondary: Story = {
  args: {
    variant: 'secondary',
    children: 'Click me'
  }
}

export const Loading: Story = {
  args: {
    variant: 'primary',
    loading: true,
    children: 'Loading...'
  },
  parameters: {
    chromatic: {
      disableSnapshot: false // Capture loading state
    }
  }
}

export const AllStates: Story = {
  render: () => (
    <div style={{ display: 'flex', gap: '1rem', flexWrap: 'wrap' }}>
      <Button variant="primary">Primary</Button>
      <Button variant="secondary">Secondary</Button>
      <Button variant="primary" disabled>Disabled</Button>
      <Button variant="primary" loading>Loading</Button>
    </div>
  )
}

Chromatic Configuration

// .storybook/main.js
module.exports = {
  stories: ['../src/**/*.stories.@(js|jsx|ts|tsx)'],
  addons: ['@storybook/addon-essentials'],
  framework: '@storybook/react-vite',
  features: {
    storyStoreV7: true
  }
}

// .storybook/preview.js
export const parameters = {
  chromatic: {
    // Global settings
    viewports: [375, 768, 1280],
    diffThreshold: 0.2
  }
}

Chromatic CI

# .github/workflows/chromatic.yml
name: Chromatic

on: push

jobs:
  chromatic:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # Required for TurboSnap

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci

      - name: Publish to Chromatic
        uses: chromaui/action@latest
        with:
          projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
          onlyChanged: true # TurboSnap
          exitZeroOnChanges: true

Playwright Screenshots

Built-in Screenshot Comparison

// playwright.config.ts
import { defineConfig } from '@playwright/test'

export default defineConfig({
  expect: {
    toHaveScreenshot: {
      maxDiffPixels: 100,
      maxDiffPixelRatio: 0.01,
      threshold: 0.2,
      animations: 'disabled'
    }
  },
  updateSnapshots: 'missing'
})

// tests/visual.spec.ts
import { test, expect } from '@playwright/test'

test.describe('Visual Tests', () => {
  test('full page screenshot', async ({ page }) => {
    await page.goto('/')
    await expect(page).toHaveScreenshot('homepage.png', {
      fullPage: true
    })
  })

  test('element screenshot', async ({ page }) => {
    await page.goto('/components')
    const card = page.getByTestId('product-card')
    await expect(card).toHaveScreenshot('product-card.png')
  })

  test('with mask', async ({ page }) => {
    await page.goto('/dashboard')
    await expect(page).toHaveScreenshot('dashboard.png', {
      mask: [
        page.getByTestId('timestamp'),
        page.getByTestId('avatar'),
        page.locator('.ad-banner')
      ]
    })
  })

  test('with custom threshold', async ({ page }) => {
    await page.goto('/chart')
    await expect(page.getByTestId('chart')).toHaveScreenshot('chart.png', {
      maxDiffPixelRatio: 0.05 // Allow 5% difference
    })
  })
})

Screenshot Organization

tests/
├── visual/
│   ├── homepage.spec.ts
│   ├── dashboard.spec.ts
│   └── components.spec.ts
├── visual.spec.ts-snapshots/
│   ├── homepage-chromium.png
│   ├── homepage-firefox.png
│   ├── homepage-webkit.png
│   └── dashboard-chromium.png

BackstopJS

Configuration

// backstop.config.js
module.exports = {
  id: 'my-project',
  viewports: [
    { label: 'phone', width: 375, height: 667 },
    { label: 'tablet', width: 768, height: 1024 },
    { label: 'desktop', width: 1280, height: 800 }
  ],
  scenarios: [
    {
      label: 'Homepage',
      url: 'http://localhost:3000/',
      delay: 500,
      hideSelectors: ['.timestamp', '.ad-banner'],
      removeSelectors: ['.cookie-notice']
    },
    {
      label: 'Dashboard',
      url: 'http://localhost:3000/dashboard',
      delay: 1000,
      clickSelector: '.load-data-button',
      postInteractionWait: 2000
    },
    {
      label: 'Modal Open',
      url: 'http://localhost:3000/',
      clickSelector: '#open-modal',
      postInteractionWait: 500
    }
  ],
  paths: {
    bitmaps_reference: 'backstop_data/bitmaps_reference',
    bitmaps_test: 'backstop_data/bitmaps_test',
    html_report: 'backstop_data/html_report'
  },
  engine: 'playwright',
  engineOptions: {
    browser: 'chromium',
    args: ['--no-sandbox']
  },
  report: ['browser', 'CI'],
  misMatchThreshold: 0.1,
  requireSameDimensions: true
}

Commands

# Create reference screenshots
npx backstop reference

# Run comparison
npx backstop test

# Approve changes
npx backstop approve

# Open report
npx backstop openReport

Baseline Management

Updating Baselines

# Playwright - Update all
npx playwright test --update-snapshots

# Playwright - Update specific
npx playwright test visual.spec.ts --update-snapshots

# Percy - Approve in dashboard
# https://percy.io/project/builds/123

# Chromatic - Approve in dashboard
# https://www.chromatic.com/builds?appId=xxx

Versioning Strategy

## Baseline Versioning

1. **Initial baseline** - Commit with feature PR
2. **Intentional changes** - Update and commit in same PR
3. **Review process** - Visual diff in PR comments
4. **Approval** - Team member approves visual changes

## When to Update
- Design system changes
- Layout modifications
- Typography updates
- Color scheme changes
- New components

Ignore Patterns

// Hide dynamic content before screenshots
async function prepareForScreenshot(page: Page) {
  await page.evaluate(() => {
    // Hide timestamps
    document.querySelectorAll('[data-visual-ignore]').forEach(el => {
      (el as HTMLElement).style.visibility = 'hidden'
    })

    // Remove animations
    const style = document.createElement('style')
    style.textContent = `
      *, *::before, *::after {
        animation-duration: 0s !important;
        transition-duration: 0s !important;
      }
    `
    document.head.appendChild(style)
  })

  // Wait for fonts
  await page.evaluate(() => document.fonts.ready)
}

Threshold Configuration

Per-Component Thresholds

// Different thresholds for different scenarios
const THRESHOLDS = {
  static: { maxDiffPixelRatio: 0.001 },  // Very strict
  charts: { maxDiffPixelRatio: 0.05 },    // Allow variation
  maps: { maxDiffPixelRatio: 0.1 },       // Higher tolerance
  animations: { maxDiffPixelRatio: 0.02 } // Moderate
}

test('chart with tolerance', async ({ page }) => {
  await page.goto('/analytics')
  await expect(page.getByTestId('chart')).toHaveScreenshot(
    'analytics-chart.png',
    THRESHOLDS.charts
  )
})

Success Output

When successful, this skill MUST output:

✅ SKILL COMPLETE: visual-regression

Completed:
- [x] Visual regression tool configured (Percy/Chromatic/Playwright)
- [x] Baseline screenshots captured
- [x] CI/CD pipeline integration verified
- [x] Dynamic content handling implemented
- [x] Threshold configuration optimized
- [x] Test suite executing successfully

Outputs:
- .percy.yml or chromatic config
- tests/visual/*.spec.ts
- .github/workflows/visual.yml
- Screenshot baselines in appropriate directory

Verification:
- Visual regression tests pass locally
- CI pipeline executes visual tests
- Baseline approval workflow functional

Completion Checklist

Before marking this skill as complete, verify:

• Visual regression tool installed and configured
• Test files created in tests/visual/ directory
• Baseline screenshots captured for all target viewports
• Dynamic content properly masked or hidden
• CI/CD pipeline integrated with visual testing service
• Threshold values tuned for appropriate sensitivity
• Documentation updated with visual testing workflow
• Team trained on baseline approval process
• All visual tests passing in CI
• Screenshot organization strategy documented

Failure Indicators

This skill has FAILED if:

• ❌ Visual regression tool fails to install or authenticate
• ❌ Baseline screenshots cannot be captured
• ❌ Dynamic content causes excessive false positives
• ❌ CI pipeline cannot connect to visual testing service
• ❌ Test execution times exceed acceptable limits (>10min)
• ❌ Screenshot comparison service quota exceeded
• ❌ Team cannot approve or reject baseline changes
• ❌ Multiple viewport configurations causing confusion
• ❌ Tests fail inconsistently due to timing issues

When NOT to Use

Do NOT use this skill when:

• Project has no UI components (use unit/integration testing instead)
• Visual changes are expected daily (too much baseline churn)
• Team lacks capacity for baseline review (use functional tests instead)
• Screenshot service costs exceed budget
• UI is entirely dynamic/procedural (no stable baselines)
• Project timeline doesn't allow for setup time
• Alternative: Use accessibility testing or functional e2e tests

Use alternative approaches for:

• Component unit testing → Jest/Vitest with Testing Library
• Functional testing → Playwright/Cypress without screenshots
• Accessibility → axe-core or Lighthouse CI
• Performance → Lighthouse or WebPageTest

Anti-Patterns (Avoid)

Anti-Pattern	Problem	Solution
Capturing entire pages without masking	Dynamic content causes false positives	Mask timestamps, ads, avatars with percyCSS
Overly sensitive thresholds (0.001)	Pixel-level changes break tests	Use 0.01-0.05 threshold for tolerance
Single viewport only	Mobile/responsive issues missed	Test 3+ viewports (mobile, tablet, desktop)
No wait for network idle	Incomplete renders cause flakes	Use waitForLoadState('networkidle')
Approving without review	Visual bugs slip through	Require human review of all diffs
Hardcoded delays (sleep)	Slow tests, timing issues	Use proper wait conditions
Testing animations mid-frame	Inconsistent snapshots	Disable animations or wait for completion
Baseline sprawl (100+ images)	Review fatigue, slow CI	Focus on critical user journeys

Principles

This skill embodies these CODITECT principles:

• #5 Eliminate Ambiguity - Clear success/failure through pixel-perfect diffs
• #6 Clear, Understandable, Explainable - Visual diffs are self-documenting
• #8 No Assumptions - Explicit baseline approval required
• #10 Iterative Refinement - Tune thresholds based on real results
• Automation - Catch visual regressions automatically in CI
• Quality Gates - Block deployment on visual breakage

Related Standards:

• CODITECT-STANDARD-AUTOMATION.md
• CODITECT-STANDARD-TESTING.md

Usage Examples

Setup Visual Testing Pipeline

Apply visual-regression skill to configure Percy with Playwright for component library

Integrate with Storybook

Apply visual-regression skill to setup Chromatic for Storybook with TurboSnap optimization

Handle Dynamic Content

Apply visual-regression skill to implement strategies for ignoring timestamps, ads, and user avatars