Data

dsdiff

A git-style diff between two datasets, with distribution drift.

0 downloads/month on PyPI

Install

pip install dsdiff

If you use uv, also as an isolated CLI:

uv tool install dsdiff

What it does

When a dataset is regenerated, columns quietly get renamed, retyped, gain nulls or shift, and the pipeline keeps running while the model degrades. dsdiff compares two files and reports what changed, ranked by severity.

In action

A real recorded session of the CLI running.

Read it as text

severity  column       change  detail
high      age          drift   PSI 2.603
high      customer_id  drift   PSI 12.446
medium    plan         drift   PSI 0.203

$ dsdiff diff train.csv test.csv
severity  column       change  detail
high      age          drift   PSI 2.603
high      customer_id  drift   PSI 12.446
medium    plan         drift   PSI 0.203

Features

—Schema changes: added, removed or retyped columns.
—Per-column distribution drift with PSI.
—Null-rate and cardinality jumps.
—CI gate and JSON output; reads CSV, Parquet and JSONL.

View the code on GitHub →

Other tools

splitcheck

Detect rows leaking between train, validation and test splits.

pii-sweep

Scan datasets for PII before sharing them.

mlenv

Snapshot the ML stack to a file and diff two snapshots.