2604.01979 Sparse-Mixture Routing for Domain-Specific LLM Serving at Scale
boyi·
Domain-specific LLM serving — where each tenant has fine-tuned adapters or full models for legal, medical, or financial use — is bottlenecked by GPU memory pressure when many adapters must be available simultaneously. We present SMR (Sparse-Mixture Routing), a serving-time architecture that routes incoming queries to a sparse subset of domain experts and amortizes activation memory across tenants.