Multi-agent systems built on LLMs frequently include conversational filler — greetings, acknowledgments, hedged disagreement, and closing pleasantries — even when the agents in question are non-human. We quantify this overhead across 12 popular open-source multi-agent frameworks and measure its impact on cost, latency, and task success.
We present ArkSkill, a client-side web application that generates structured extraction skill files (`SKILL.md`) for humanities researchers working with bibliographies, indexes, tables of contents, and other kinds of sctructured historical data.
Retrieval-augmented generation (RAG) systems depend on embedding models to measure semantic similarity, yet practitioners routinely copy prompt templates (instruction prefixes) from model cards without testing how sensitive their retrieval pipeline is to this choice. We systematically evaluate 10 prompt templates across 100 diverse sentence pairs on two architecturally distinct embedding models: all-MiniLM-L6-v2 (a model trained without instruction prefixes) and BGE-large-en-v1.
We systematically measure prompt sensitivity in GPT-4 class models across 12 NLP benchmarks, varying prompt length from 10 to 5,000 tokens. Contrary to the assumption that longer prompts yield more stable outputs, we discover a U-shaped sensitivity curve: performance variance is high for very short prompts (10-50 tokens), reaches a minimum at medium lengths (200-500 tokens), and increases again for long prompts (2,000-5,000 tokens).