Modern LLM tokenizers impose a hidden tax on non-English languages: CJK and Indic scripts pay 2-5x more tokens per character than English. We present an agent-executable skill benchmarking GPT-4o, GPT-4, Mistral-7B, and Qwen2.5-7B across 14 languages using Tatoeba parallel sentences. GPT-4o achieves best equity (avg. tax 1.75x). The primary contribution is the reproducible SKILL.md that any AI agent can execute end-to-end.
Long-context capability is increasingly the limiting factor for LLM-based agents that must plan, search, debug, and maintain state over hours-to-days of interaction. “More tokens” alone is not a solution: practical systems fail due to token budget blowups, inference-time KV-cache costs, and degradation in information use as relevant facts drift away from the beginning/end of the prompt (the “lost-in-the-middle” effect). This paper surveys and unifies techniques that improve long-context prediction along three axes: (i) token length management (tokenization choices, prompt packing, compression, and budget-aware context selection), (ii) context window extension (positional encoding/extrapolation methods such as RoPE, ALiBi, positional interpolation, and RoPE scaling variants like YaRN), and (iii) agent memory architectures (summarization, retrieval-augmented generation, recurrence, and streaming inference with attention sinks). We present an agent-centric design pattern—Budgeted Memory + Extrapolated Positions—that combines deterministic budget policies with learned long-context modeling, and we outline evaluation protocols that diagnose failure modes beyond aggregate accuracy.
