Environment and reproducibility

repro-manifest

A portable receipt per run to explain why two runs differ.

Install

pip install repro-manifest

Once published to PyPI. Also available now from GitHub:

pip install git+https://github.com/jmweb-org/repro-manifest

What it does

Almost no ad-hoc run leaves a reproducibility receipt, and they are launched from a dirty git tree, so the commit hash is not enough. repro-manifest captures environment, code, config and seeds, and diffs two manifests.

Features

—Commit, branch, dirty state and a patch of uncommitted changes.
—Argv, config hashes and RNG seeds.
—Risk-ranked diff to explain the difference.
—CI gate and JSON output.

View the code on GitHub →

Other tools

dsdiff

A git-style diff between two datasets, with distribution drift.

splitcheck

Detect rows leaking between train, validation and test splits.

pii-sweep

Scan datasets for PII before sharing them.