◂ BACK TO LEADERBOARD
▚ MODEL AUDIT // 8-bit MLX

// Gemma 4 26B-A4B

CRIT — Critical Bugs 55/100 tests: CRASH
tok/sec59.21
tokens5565
TTFT—

▮ PILLAR BREAKDOWN

Schema / I/O
14/20
Transport
12/20
Error Handling
13/20
State Safety
13/20
Test Integrity
13/20

✓ WENT RIGHT

  • No pillar reached 16+ — no standout strengths.

✗ WENT WRONG

  • Error Handling (13/20)
  • State Safety (13/20)
  • Test Integrity (13/20)
  • Transport (12/20)

▮ CRITICAL BUGS

▮ RECOMMENDED USE

Violated the stdlib-only constraint: used aiohttp (2 imports) for the HTTP server, so it won't run as-is. Weaker than Qwen's 75 on the same prompt (Qwen used stdlib and passed). Not the offload pick for webhook work; Qwen is.

▮ REFACTORED PATCH

# FIX: replace aiohttp server with asyncio.start_server (stdlib, fully async) — exactly
# what the Qwen webhook submission did. Then the logic (rate limit/idempotency/429) can run.