◂ BACK TO LEADERBOARD
▚ MODEL AUDIT // 8-bit MLX

// Gemma 4 26B-A4B

FLAWS — Minor Logic Flaws 80/100 tests: PASS
tok/sec49.48
tokens6876
TTFT—

▮ PILLAR BREAKDOWN

Complexity (O(1))
16/20
Concurrency / Races
18/20
Error Handling
16/20
Resource & State Safety
15/20
Test Integrity
15/20

✓ WENT RIGHT

  • Concurrency / Races (18/20)
  • Complexity (O(1)) (16/20)
  • Error Handling (16/20)

✗ WENT WRONG

  • No pillar fell below 14 — solid across the board.

▮ CRITICAL BUGS

▮ RECOMMENDED USE

DECISIVE: passes the TTS pipeline that Qwen 6-bit couldn't even parse. Uses REAL bounded concurrency (N worker tasks + Semaphore) — exactly Qwen's fatal flaw avoided. Gemma generalizes to multi-task orchestration where Qwen fails. Strong offload candidate for queue/pipeline work.

▮ REFACTORED PATCH

# TTFT capture: LM Studio returns timing under 'stats' or 'timings' with keys like
# 'time_to_first_token' / 'prompt_progress' — grade_run.py should dump resp['stats'] raw
# once to find the right key, then parse it.