Serving and cost

tokenmeter

Count tokens and estimate prompt cost before sending.

Install

pip install tokenmeter-cli

Once published to PyPI. Also available now from GitHub:

pip install git+https://github.com/jmweb-org/tokenmeter

What it does

Prompt templates grow, an example gets added, the retrieved context balloons, and suddenly every call costs more than you thought. tokenmeter gives the exact count and a dollar estimate up front.

Features

—Exact counting with tiktoken for OpenAI encodings.
—Per-model cost estimate from a dated table.
—A file, a directory or stdin.
—Budget gate for CI.

View the code on GitHub →

Other tools

dsdiff

A git-style diff between two datasets, with distribution drift.

splitcheck

Detect rows leaking between train, validation and test splits.

pii-sweep

Scan datasets for PII before sharing them.