Benchmarks and comparison of LLM AI models and API hosting providers

152 points

2 years ago

Hi HN, ArtificialAnalysis.ai provides objective benchmarks and analysis of LLM AI models and API hosting providers so you can compare which to use in your next (or current) project.

The site consolidates different quality benchmarks, pricing information and our own technical benchmarking data. Technical benchmarking (throughput, latency) is conducted through sending API requests every 3 hours.

Check out the site at https://artificialanalysis.ai, and our twitter at https://twitter.com/ArtificialAnlys

Twitter thread with initial insights: https://twitter.com/ArtificialAnlys/status/17472648324397343...

All feedback is welcome and happy to discuss methodology, etc.

70 comments