1. brew install llama.cpp 2. ./llama-cli --hf-repo google/gemma-2-2b-it-GGUF \ --hf-file 2b_it_v2.gguf \ -p "Write a poem about cats as a labrador" -cnv

With this, I created a Local RAG Knowledge and Answering System with Google Gemma 2 2B and Marqo. Check it out: https://github.com/ellie-sleightholm/marqo-google-gemma2

Link also in the comments!