In 2D you can visualize all optimizers at once and see their behaviors around local minima etc.

In 3-15D you can visualize the loss landscape for the first two parameters being optimized. The loss landscape changes as the unobserved parameters change, and because of this, you can only visualize a single optimizer at a time.

This was a quick weekend project vibe coded with the help of codex. Github is here: https://github.com/olliepro/ND-Optimizer-Arena/tree/main