The Race
Four AI models compete head-to-head answering 300 MMLU-Pro Hard questions — one of the toughest academic benchmarks in AI. Each question is 10-way multiple choice requiring multi-step reasoning across subjects like math, physics, law, and more.
The Horses
Llama 3.2 3B — Meta's base model (3B params, 36.5% MMLU-Pro)
Qre Llama 3B — Qwerky's optimized 3B (~2x faster, same accuracy)
Qre Llama 8B — Qwerky's optimized 8B (~2x faster, same accuracy)
Llama 3.1 8B — Meta's base model (8B params, 44.3% MMLU-Pro)
How It Works
Models race around the track as they answer questions. Correct answers advance the horse forward. Wrong answers trigger a time penalty — and penalties increase with each mistake. The first model to complete all 300 questions wins!
Betting
Sign in to place virtual bets using tokens (everyone starts with 100). Betting uses a parimutuel system — the entire pool is split among winners proportional to their bets. Set up betting before the race starts, then watch the odds shift in real-time!
Built By
Qwerky AI optimizes Llama model inference using a custom SSM (State Space Model) architecture — delivering up to 2x throughput, 25-40% cost reduction, and 2x energy savings on NVIDIA hardware.
qwerky.ai · NVIDIA GTC 2026