WebMarker is a JavaScript library used for adding visual markers and labels to elements on a web page. This can be used for Set-of-Mark prompting, which improves the visual grounding abilities of vision-enabled large language models such as GPT-4o, Claude 3.5, and Google Gemini 1.5. This library aims to:- Improve LLM performance on vision tasks referencing web pages
- Enable reliable web page interactions based on LLM responses