It leverages Gemini 2.0 Flash and Google's live multimodal API to analyze what's on screen and provide real-time assistance.
In this demo, my friend tries to add a payment method to Amazon, and the AI cursor walks them through the entire process with visual cues and spoken instructions.
I've also used it to interpret diagrams from research papers—curious to see what other use cases people find this useful for!