Skip to main content

Data Quality Checks

Implement comprehensive data quality validation including schema validation, null checks, range checks, uniqueness constraints, and referential integrity.

Complexity: Moderate | Duration: 15-30m | Category: Devops

Tags: data-engineering data-quality validation testing

Workflow Diagram​

Steps​

Step 1: Schema validation​

Agent: data

engineer - Check column names, data types, order

Step 2: Completeness checks​

Agent: data

engineer - Identify null values, missing records

Step 3: Uniqueness checks​

Agent: data

engineer - Validate primary keys, unique constraints

Step 4: Range checks​

Agent: data

engineer - Validate numeric ranges, date ranges

Step 5: Format checks​

Agent: data

engineer - Regex validation for emails, phone numbers, etc.

Step 6: Referential integrity​

Agent: data

engineer - Check foreign key relationships

Step 7: Statistical profiling​

Agent: data

engineer - Compute min, max, mean, stddev, percentiles

Step 8: Report generation​

Agent: testing

specialist - Generate data quality scorecard

Usage​

To execute this workflow:

/workflow devops/data-quality-checks.workflow

See other workflows in this category for related automation patterns.