LMArena Team

Predicting Human Preference with $100,000 in Prizes Overview LMSYS and Kaggle are launching a human preference prediction competition! You are challenged to predict which responses users will prefer in head-to-head battles between Large Language Models (LLMs). You’ll work with a dataset from the Chatbot Arena, containing conversations and

Authors Tianle Li* Wei-Lin Chiang* Evan Frick Lisa Dunlap Banghua Zhu Joseph E. Gonzalez Ion Stoica Building an affordable and reliable benchmark for LLM chatbots has become a critical challenge. A high-quality benchmark should 1) robustly separate model capability, 2) reflect human preference in real-world use cases, and 3) frequently

Authors Wei-Lin Chiang Tianle Li Joseph E. Gonzalez Ion Stoica Welcome to our latest update on the Chatbot Arena, our open evaluation platform to test the most advanced LLMs. We’re excited to share that over 130,000 votes that are now collected to rank the most capable 40+ models!

Since its launch three months ago, Chatbot Arena has become a widely cited LLM evaluation platform that emphasizes large-scale, community-based, and interactive human evaluation. In that short time span, we collected around 53K votes from 19K unique IP addresses for 22 models. In this blog post, we are releasing an

Introducing MT-Bench and Vicuna-33B