Atlas

- Shipped an OpenAI-compatible multi-tenant inference gateway with tenant auth/quotas and three routing strategies (round-robin, least-load, prefix-aware), tested on RTX 3050 4GB and Colab/Kaggle T4 GPUs. - Measured a 5.5× p50 latency improvement when replacing naive Hugging Face serving with vLLM under concurrency. - Found that sticky prefix-aware routing caused a 2× TTFT regression in simulation and a smaller ~1.17× regression on dual live vLLM instances; implemented a load-gate mitigation.