Arena Expert and Occupational Categories
The next frontier of large language model (LLM) evaluation lies in understanding how models perform when challenged by expert-level problems, drawn from real work, across diverse disciplines.
Research
Re-introducing Vision Arena Categories
Since we first introduced categories over two years ago, and Vision Arena last year, the AI evaluation landscape has evolved. New categories have been added, existing ones have been updated, and the leaderboards they power are becoming more insightful with each round of community input.
Introducing BiomedArena.AI: Evaluating LLMs for Biomedical Discovery
LMArena is honored to partner with the team at DataTecnica to advance the expansion of BiomedArena.ai: a new domain-specific evaluation track.
A Deep Dive into Recent Arena Data
Today, we're excited to release a new dataset of recent battles from LMArena! The dataset contains 140k conversations from the text arena.
Does Sentiment Matter Too?
Introducing Sentiment Control: Disentangling Sentiment and Substance
Contributors:
Connor Chen
Wei-Lin Chiang
Tianle Li
Anastasios Angelopoulos
Introduction
You may have noticed that recent models on Chatbot Arena appear more emotionally expressive than their predecessors. But does this added sentiment actually improve their rankings on the leaderboard? Our previous exploration revealed
How Many User Prompts are New?
We investigate 355,575 LLM battles from May 2024 to Dec 2024 to answer the following questions:
1. What proportion of prompts have never been seen before (aka “fresh”)?
2. What are common duplicate prompts?
3. How many prompts appear in widely used benchmarks?
WebDev Arena: A Live LLM Leaderboard for Web App Development
WebDev Arena allows users to test LLMs in a real-world coding task: building interactive web applications.