Show HN: rtrvr.ai – New Free SOTA AI Web Agent Beats Even Operator

8 points

a year ago

We just benchmarked our agent, rtrvr.ai, on the Halluminate (YC S25) Web Bench, and rtrvr.ai achieved a new State-of-the-Art performance with an 81% success rate. For perspective, this surpasses not only all other autonomous agents but also the human-intervention baseline of OpenAI's Operator (76.5%).

It also completes tasks an astonishing 7x faster than the next leading alternative.

This isn't just an incremental improvement; it's a validation of our core architectural philosophy. Our performance stems from two key differentiators:

- Local-First Operation: As a Chrome Extension, rtrvr.ai operates directly within the user's browser. This eliminates the latency, bot detection and access issues that plague cloud browser agents.

- DOM-Based Interaction: Instead of relying on brittle visual parsing (CUA), our agent interacts directly with the page's HTML structure, enabling skipping clicks and resilience to pop-ups and overlays. We also can just use the latest and fastest models such as Gemini Flash for superior performance.

This leads to a critical industry insight: Cloud Browser Agents are not a viable long-term solution for reliable web automation.

Our benchmark analysis shows that over 94% of rtrvr.ai's failures were "agent errors" (fixable AI logic), while only 5% were "infrastructure errors." For cloud agents, this ratio is often inverted. You can't build a reliable agent if you can't even guarantee access to the environment.

Finally it only cost us ~$40 to run this benchmark, whereas we estimate it cost >~$1k in infra costs for each agent for Halluminate.

The future of web automation won't be fought from remote data centers. It will be run symbiotically from your browser. Our results are the first major data point proving this thesis and putting the first nail in the coffin for cloud browser agents.

Full Report: https://www.rtrvr.ai/blog/web-bench-results

Or if you just want to tune into some Agentic-SMR of a web agent doing tasks online tune into the playlist: https://www.youtube.com/watch?v=HWPZI8PjuLY&list=PL5rk1YARPB...

Try out the magic of a working web agent yourself, install at: https://chromewebstore.google.com/detail/rtrvrai-ai-web-agen...

Bring your own API Key from ai.studio and use Google's Gemini Free Tier to use our web agent for free! We literally have a button that will get our agent to open AI Studio create key and configure itself all automatically.

2 comments