Show HN: Inseq – An Interpretability Toolkit for Generative Language Models

1 point

3 years ago

We recently open-sourced Inseq, a Python library built on top of transformers and Pytorch, aimed at democratizing and commoditizing post-hoc interpretability analysis for sequence generation models.

Inseq supports thousands of decoder-only and seq2seq models, with various attribution methods already baked in and many more to come. Attributing MetaAI's Galactica writing LaTeX formulas or GoogleAI Flan-T5 doing commonsense reasoning now takes only 3 lines of code!

The Inseq CLI improves the user experience when conducting global analyses by enabling batched attribution of examples and even entire datasets from the Hub directly from the console. Inseq is beginner-friendly but also fully extensible for advanced use cases, supporting attribution of custom functions and the extraction of step scores during generation.

With Inseq, we aim to centralize and standardize some practices of the interpretability community working on NLG and NMT, to enable fair and reproducible evaluation. The project is still in its infancy, and feedback/contributions are very much appreciated!