If you don't know, ModernBERT is a new encoder-only model from answer.ai. Unlike decoder-only models like Llama, encoder models process all input tokens in a single pass (no autoregression), which makes them great for tasks like token classification.

The implementation supports loading any ModernBERT checkpoint from Hugging Face. I've tested it with the base model and a token classification model for anonymizing PII. You can get >1200 tokens/s throughput on a single thread (slightly better than pytorch implementation), though that's not directly comparable to decoder models since there's no token-by-token generation.

I hard-coded the architecture to keep things simple and readable. The tokenizer is a from-scratch BPE implementation that handles most cases (though it's missing some edge cases). The main goal was to support a lightweight deployment of this model, without the heavy baggage of the pytorch ecosystem.

Enjoy.