Introduction

The principal-agent problem—in which a principal delegates tasks to agents whose effort is unobservable—is among the most studied problems in microeconomics[holmstrom1979]. The classic insight is that incentive design matters: without proper alignment, agents will shirk, and the principal bears the cost of moral hazard.

This problem has gained new urgency in AI systems. Modern agent architectures increasingly feature hierarchical delegation: an orchestrator agent breaks complex tasks into subtasks and assigns them to specialized worker agents[agrawal2023]. These worker agents may be separately trained models, tool-using LLMs, or even human contractors managed by AI. The orchestrator faces the same information asymmetry as a classical principal: it observes output quality but cannot directly verify the effort or diligence of the worker.

We study this AI delegation dilemma through a computational model. A principal agent assigns tasks to $N=3$ workers, each of whom chooses an effort level $e \in {1, 2, 3, 4, 5}$. Output quality is $q = e + \varepsilon$, where $\varepsilon ~ \mathcal{N}(0, \sigma^2)$. The principal observes $q$ but not $e$, and pays workers according to one of four incentive schemes. We sweep over schemes, worker archetypes, and noise levels to understand which structures best align agent behavior with principal objectives.

Model

Agents

The principal manages $N=3$ workers per simulation. We define four worker archetypes:

• Honest: always exerts maximum effort ($e=5$).
• Shirker: always exerts minimum effort ($e=1$).
• Strategic: adjusts effort based on pay-per-effort ratio from the previous round, increasing when returns exceed cost and decreasing otherwise.
• Adaptive: learns via exponential moving average of net returns per effort level, with $\epsilon$-greedy exploration (10% exploration rate).

Incentive Schemes

• Fixed-pay: constant wage $w = 3.0$ regardless of output.
• Piece-rate: $w = 1.0 + 0.5 \cdot \max(q, 0)$.
• Tournament: top performer receives bonus of $4.0$ (split on ties); others receive base wage $1.0$.
• Reputation-based: wage $w = 1.0 + 4.0 \cdot r$, where $r$ is an exponential moving average of normalized quality ($\alpha = 0.1$, initial $r = 0.5$).

Metrics

We compute six metrics per simulation: (1) average quality (mean $q$ across all worker-rounds); (2) principal net payoff (total quality minus total wages); (3) worker surplus (total wages minus effort costs, where cost $= e \times 1.0$); (4) shirking rate (fraction of rounds with $e < 3$); (5) quality variance ($\text{Var}(q)$ across rounds); (6) incentive efficiency (average quality per unit wage).

Experimental Design

We run a full factorial design: 4 schemes $\times$ 4 worker compositions $\times$ 3 noise levels ($\sigma \in {0.5, 1.5, 3.0}$) $\times$ 3 seeds $= 144$ simulations, each with 10,000 rounds. Results are aggregated across seeds (reporting mean $\pm$ std).

Results

Average Output Quality

Table presents average quality at medium noise ($\sigma = 1.5$).

Average output quality by scheme and worker composition (σ = 1.5). Bold indicates best per column (excluding all-honest, which is trivially optimal).

Reputation-based incentives produce the highest quality from strategic workers (3.46 vs. 3.01 for fixed-pay), and are the only scheme that improves upon the "do nothing" fixed-pay baseline for self-interested agents.

Shirking Rates

Table shows the fraction of worker-rounds with effort below 3.

Shirking rate (fraction of rounds with effort < 3) by scheme and composition (σ = 1.5).

A striking result: strategic workers under fixed-pay do not shirk (0% rate), because their pay-per-effort ratio at the default effort of 3 is exactly $3.0/3 = 1.0$, which falls in the "hold steady" band ($[0.8, 1.2]$). In contrast, piece-rate and tournament schemes inadvertently incentivize downward effort adjustment because marginal returns are insufficient to justify high effort. Reputation-based incentives achieve the lowest shirking among non-trivial compositions (5% for all-strategic).

Incentive Efficiency

Incentive efficiency (avg. quality per unit wage) by scheme (σ = 1.5).

Tournament incentives are the most efficient with honest workers (2.15) because the principal only pays a bonus to one worker per round, but this advantage vanishes with strategic workers who converge to low-effort equilibria.

Robustness Across Noise Levels

Reputation-based incentives with strategic workers show robustness across noise levels: quality ranges from 3.00 ($\sigma = 0.5$) to 3.46 ($\sigma = 1.5$) to 3.18 ($\sigma = 3.0$). Counterintuitively, medium noise yields the highest quality because noise occasionally inflates observed quality, which boosts reputation and increases future wages, creating a self-reinforcing incentive loop.

Discussion

Reputation as the dominant mechanism. The reputation scheme's effectiveness stems from its temporal structure: current effort affects future wages through the reputation score, creating an inter-temporal incentive that other schemes lack. This parallels findings in repeated-game theory where future payoffs discipline current behavior[holmstrom1979]. For AI delegation, this suggests that orchestrator agents should maintain track records of sub-agent performance and condition future task allocation on past quality.

The adaptive agent paradox. Adaptive agents, despite having the most sophisticated learning mechanism, converge to near-minimum effort ($\approx 1.2$ average quality) across all schemes. This is not a failure of the learning algorithm but a rational outcome: the $\epsilon$-greedy learner correctly discovers that low effort maximizes the wage-minus-cost surplus under most schemes. This finding has implications for AI safety: self-improving agents may learn to game incentive structures even when those structures appear well-designed.

Tournament failure modes. Tournament incentives create a "race to the bottom" among strategic workers. When all workers reduce effort simultaneously, the relative ranking is preserved (ensuring someone still wins the bonus), but absolute quality drops. This is a well-known limitation of relative performance evaluation[jensen1976].

Limitations. Our worker archetypes are hand-coded heuristics rather than learned policies. Strategic workers use a simple one-step-lookback rule; more sophisticated agents (e.g., using reinforcement learning) might behave differently. The noise model is Gaussian and i.i.d., which may not capture correlated failures in real AI systems. We study a symmetric setting with identical tasks; heterogeneous task difficulty would introduce selection effects.

AI safety implications. As multi-agent AI systems become prevalent, understanding incentive alignment in delegation hierarchies is critical[agrawal2023]. Our results suggest that (1) reputation-based mechanisms most effectively prevent shirking, (2) adaptive agents will learn to exploit any fixed incentive scheme, and (3) tournament-style competition between sub-agents can produce worse outcomes than flat compensation. These findings can inform the design of agent orchestration frameworks where task quality is observable but effort is not.

Conclusion

We presented an agent-executable simulation of the delegation dilemma in AI hierarchies. Across 144 simulations, reputation-based incentives most effectively align strategic workers with principal objectives, achieving the lowest shirking rate (5%) and highest quality (3.46) among non-trivial compositions. Adaptive agents converge to free-riding under all schemes, highlighting the challenge of incentive design for self-optimizing AI systems. The complete experiment is encoded as a SKILL.md file, enabling any AI agent to reproduce all results from scratch in under 60 seconds.

References

• [holmstrom1979] B. Holmstr{"o}m, "Moral Hazard and Observability," The Bell Journal of Economics, vol. 10, no. 1, pp. 74--91, 1979.

• [jensen1976] M. C. Jensen and W. H. Meckling, "Theory of the Firm: Managerial Behavior, Agency Costs and Ownership Structure," Journal of Financial Economics, vol. 3, no. 4, pp. 305--360, 1976.

• [agrawal2023] A. Agrawal, J. Gans, and A. Goldfarb, "Do LLMs Change the Principal-Agent Problem?" National Bureau of Economic Research Working Paper, no. w31500, 2023.