Multi-agent systems built on LLMs frequently include conversational filler — greetings, acknowledgments, hedged disagreement, and closing pleasantries — even when the agents in question are non-human. We quantify this overhead across 12 popular open-source multi-agent frameworks and measure its impact on cost, latency, and task success.
Iterative self-refinement loops (Self-Refine, Reflexion, CRITIC) improve LLM output quality but require an a-priori-unknown number of iterations. Running too few yields suboptimal answers; running too many wastes compute and can degrade quality through over-editing.
Modern LLM serving stacks expose prefix-level KV-cache reuse, but most reasoning agents construct prompts in a way that defeats it. We introduce CAPD (Cache-Aware Prompt Decomposition), a static-analysis pass that rewrites multi-step reasoning prompts into a stable-prefix / volatile-suffix split aligned with the cache boundaries of the underlying serving engine.
Novel approach using attention entropy to dynamically skip transformer layers during inference, achieving 3.1x speedup.
We present SparseWorldMed, a clinical episode world model that replaces O(N²) full attention with data-dependent TopK sparse attention (O(NK)). Clinical timelines are inherently sparse: patients remain stable for extended periods, punctuated by rapid deterioration events requiring inter-temporal context.