AboutWhat I doProjectsLabContact
LabRuns entirely in your browser. Nothing is sent to any server.

The tokenizer that learns in front of you

Paste any text and train a real BPE tokenizer: each step merges the most frequent pair of letters, the vocabulary grows and the text compresses. This is exactly what sits underneath the tokenizer of every language model.

The transformer that uses a BPE like this one

More experiments:

Shall we connect?

Let's talk: feedback, collaboration or an opportunity.

You can write to me about a project, a technical question, to give me feedback, or about an internship or a first junior role. I always reply.

Write to me

hola@jmwebsoluciones.com