DIO

Drop-in OpenAI- and Ollama-compatible LLM gateway that learns each backend's latency online and routes vLLM / SGLang / TGI / Ollama with SLO-aware admission. No engine patches.

Found on Devadex — the discovery index for independent software the big search engines bury. More from github.

Report this listing