Show HN: Chrome extension to run DeepSeek, LLMs and Whisper locally in browser

10 points

a year ago

Hi all, I open sourced my toy project that runs Generative AI models LOCALLY in the side panel of a Chrome extension.

The Chrome extension uses Transformers.js to run models in browser under the hood. I've integrated and tested these models so far. \1. LLM: Llama 3, Phi 3.5, Qwen 2.5, SmolLM2 \2. Reasoning: DeepSeek R1 \3. Multimodal LLM: Janus \4. Speech-to-Text: Whisper

On an M1 MacBook, DeepSeek R1 1.5B runs at ~30 tokens/sec

If you're interested in, you can download the extension from chrome web store or clone my github repository. \1. chrome web store: https://chromewebstore.google.com/detail/private-ai-assistan... \2. github: https://github.com/tantara/transformers.js-chrome \3. demo: https://www.youtube.com/watch?v=MSCDdFG5Lls

It's my first react project. Feedback is always welcome!

2 comments