It runs fine on CPU, but can also run on Apple Silicon GPU, although I did not notice any performance improvements with GPU (tested on a M5 chip).

I think these local lightweight and multimodal models will open multiple possibilities for new software tools where privacy is essential.