How many bits does a model need?

This lab's digit classifier, quantized to fewer bits in front of you: accuracy over 300 real images is measured in your browser for each format. From fp32 to int8 almost nothing is lost; at int2 the model collapses. This idea is what lets huge models run on a laptop.