LMArena Team

You can now chat with your favorite vision-language models from OpenAI, Anthropic, Google, and most other major LLM providers to help discover how these models stack up against each other. Contributors: Christopher Chou* Lisa Dunlap* Wei-Lin Chiang Ying Sheng Lianmin Zheng Anastasios Angelopoulos Trevor Darrell Ion Stoica Joseph E. Gonzalez

Introducing Hard Prompts, a new and challenging category in the Chatbot Arena Leaderboard. Contributors: Tianle Li Wei-Lin Chiang Lisa Dunlap Background Introducing Hard Prompts, a new and challenging category in the Chatbot Arena Leaderboard. Over the past few months, the community has shown a growing interest in more challenging prompts

Authors Lisa Dunlap Evan Frick Tianle Li Isaac Ong Joseph E. Gonzalez Wei-Lin Chiang On April 18th, Meta released Llama 3, their newest open-weight large language model. Since then, Llama 3-70B has quickly risen to the top of the English Chatbot Arena leaderboard with over 50,000 battles. This remarkable

Predicting Human Preference with $100,000 in Prizes Overview LMSYS and Kaggle are launching a human preference prediction competition! You are challenged to predict which responses users will prefer in head-to-head battles between Large Language Models (LLMs). You’ll work with a dataset from the Chatbot Arena, containing conversations and

Authors Tianle Li* Wei-Lin Chiang* Evan Frick Lisa Dunlap Banghua Zhu Joseph E. Gonzalez Ion Stoica Building an affordable and reliable benchmark for LLM chatbots has become a critical challenge. A high-quality benchmark should 1) robustly separate model capability, 2) reflect human preference in real-world use cases, and 3) frequently