I'm Pablo from Objective. We're working on an AI-native Search platform. Today we're releasing Anton (https://www.objective.inc/anton) an API that you can use to evaluate relevance for any search system. We would love your feedback. Here's a quick demo by my co-founder Lance: https://youtu.be/geX8mwBGqIo
Why we created Anton: In every company we've worked, it was very slow and costly to spot check a new model, or to thoroughly evaluate a release candidate, or to compare a few competing approaches for how to improve search going forward. In each of those cases, we would absolutely have loved to have had Anton!
Today, in order to ship search that doesn't disappoint users, software engineers need to become experts in AI/ML, information retrieval, data engineering, and more. Folks in our team have worked for some of the largest companies out there (Apple, Google, Youtube, Amazon, Twitch, etc.), building products that you use every day. We've all had to build and stitch together similar components everywhere, and we're now building Objective to lift some of that burden for you. We want to make it super easy for every developer to build great search experiences into their app/site.
To build Anton, we spent a lot of time evaluating different LLMs, prompt engineering, fine-tuning and really throwing the kitchen sink at the LLM-as-a-judge problem. We've discovered many ways in which LLMs make it hard to make systematic progress, and only a few that didn't make us question our sanity. That work culminated in the creation of Anton. We hope this will provide you with a cost-effective way to get near-human quality relevance judgements, so that you can delight your users every day with search that actually understands them.
Can't wait to see what you'll build with it. And as always, would love feedback!