◂ BACK TO LEADERBOARD
▚ MODEL AUDIT // 6-bit MLX

// Qwen 3.6 35B-A3B

CRIT — Critical Bugs 49/100 tests: CRASH
tok/sec69.87
tokens11996
TTFT0.84s

▮ PILLAR BREAKDOWN

Complexity (O(1))
10/20
Concurrency / Races
8/20
Error Handling
13/20
Resource & State Safety
12/20
Test Integrity
6/20

✓ WENT RIGHT

  • No pillar reached 16+ — no standout strengths.

✗ WENT WRONG

  • Error Handling (13/20)
  • Resource & State Safety (12/20)
  • Complexity (O(1)) (10/20)
  • Concurrency / Races (8/20)
  • Test Integrity (6/20)

▮ CRITICAL BUGS

▮ RECOMMENDED USE

NOT a safe offload for async-pipeline / queue work despite scoring 82 on the LFU exam. Same model+quant, different prompt: 82 -> 49. Use it for data-structure/ACID tasks; route queue/backpressure/retry work elsewhere (or to the cloud model).

▮ REFACTORED PATCH

# FIX 1 (the parse error): make on_event async (or don't await inside it).
# Simplest correct version:
async def on_event(self, callback):
    async with self._callbacks_lock:
        self._callbacks.append(callback)

# FIX 2 (real bounded concurrency): spawn N workers OR create_task per job
# gated by the semaphore. Option B (concurrency from the semaphore itself):
async def _worker_loop(self):
    while self._running:
        job = await self._queue.get()
        # do NOT hold the semaphore in the single worker; instead launch
        # each job as its own task, gated so at most max_concurrency run:
        async def _run(j):
            async with self._semaphore:
                await self._process_job(j)
                self._queue.task_done()
        asyncio.create_task(_run(job))
# (and add a test that submits >max_concurrency long jobs and asserts
# exactly max_concurrency run at once.)

# FIX 3: reject submit() after stop() (guard on self._running).
# FIX 4: bound _final_states (e.g. keep last N, or evict terminal >TTL).
# FIX 5: log callback errors instead of swallowing them silently.