Inside a transformer's attention

Attention extracted from the trained checkpoint of my handwritten transformer: pick a layer and head and see which characters the model attends to when predicting the next one. Not an illustration: the actual weights.