Side-by-Side Benchmark Audit

Model Comparison Matrix

Select the benchmarked models we evaluated against the VaxBench suite. Compare overall scores, coding and reasoning performance, latency, throughput, and token usage.

Select Model A

Select Model B

Select Model C

Speed Champion

Cerebras

Gemma 4 31B

gemma-4-31b7/3/2026

Overall Score49.4%

Benchmark Suite Breakdown

Coding (76/167)45.5%

Reasoning (83/155)53.5%

Performance Metrics

Latency

645 ms

Throughput

2,015.5 tps

Total Tokens

89,946

Tokens / Task

279

Provider

Cerebras

GLM 4.7

zai-glm-4.77/3/2026

Overall Score41%

Benchmark Suite Breakdown

Coding (54/167)32.3%

Reasoning (78/155)50.3%

Performance Metrics

Latency

1,433 ms

Throughput

1,056.9 tps

Total Tokens

91,670

Tokens / Task

285

Provider

Cerebras

Highest OverallMost Efficient

Devin (self-benchmark)

GLM 5.2 High (self)

glm-5.2-high7/3/2026

Overall Score64.9%

Benchmark Suite Breakdown

Coding (96/167)57.5%

Reasoning (113/155)72.9%

Performance Metrics

Latency

0 ms

Throughput

0 tps

Total Tokens

Tokens / Task

Provider

Devin (self-benchmark)

How should I read these metrics?

Overall is the combined pass rate across the full VaxBench suite.Coding and Reasoning are pass@1 rates by domain.Latency is time-to-first-token in milliseconds, Throughput is tokens per second, and Tokens / Task is total completion tokens divided by the number of benchmark tasks (322 per run).