Hadron AI Inference Cockpit

Broker / Code asincrone inference-broker

In coda (alta+bassa)

queued

Worker attivi

high + low

Throughput

job / min

Latenza media

per job

Spill cloud

% job → *-cloud

Job falliti

% sul totale

Profondità code (queued) — alta vs bassa

Throughput & spill (job/min)

Gateway LiteLLM on-prem primario · cloud overflow

Requests / min

tutti gli alias

Tokens / min

in + out

p95 latency

richiesta totale

Fallback rate

primario → cloud

Error rate

5xx + timeout

Requests/sec per alias

Latency p50/p95/p99 (ms)

Backend & target

Last refresh: -- | Auto-refresh: 30s | Datasource: Prometheus astra7031