Open Standards for Tool-Use Trace Logging in Autonomous Agents

1. Introduction

Reproducing the behavior of an autonomous research agent requires more than the final paper — it requires the full sequence of tool invocations, their arguments, and the responses observed. Today this is logged in at least seven mutually incompatible formats (OpenAI's tool_calls, Anthropic's tool_use blocks, LangChain's RunTree, the AutoGen JSONL variant, and three proprietary schemas we surveyed). The result is that even cooperating reviewers cannot easily replay a submitted run.

This paper proposes OTUTL, a minimal, open trace-log schema. Our goals are:

1. Lossless capture of every tool call and its result.
2. Vendor-neutral field naming with a clear extension mechanism.
3. Replay-stable canonicalization so that two implementations agree on a SHA-256 of the same logical run.

2. Background and Threat Model

We assume an honest-but-curious archive operator who wishes to verify that a submitted paper is consistent with its declared tool budget and retrieval set. We do not address adversarial agents that forge logs after the fact; that requires hardware-attested logging and is orthogonal [Chen and Park 2025].

Prior work on workflow provenance (W3C PROV, RO-Crate) is too general — it captures any process — and the LLM-specific schemas we surveyed are too narrow, omitting fields like streamed-partial-tool-result that are common in current agents. OTUTL targets the middle ground.

3. Schema

An OTUTL log is a sequence of newline-delimited JSON records. Each record is one of seven types: run_start, tool_call, tool_result, model_step, cost, error, run_end. Every record carries:

• ts — RFC 3339 timestamp with at least millisecond precision.
• run_id — UUIDv7.
• seq — monotonic 64-bit counter, gap-free.

A tool_call record additionally carries call_id, tool (a fully-qualified URI), args (canonical JSON), and parent_call_id for nested calls.

We define canonical JSON per RFC 8785 with one extension: floats are serialized using the shortest round-trip decimal. The hash of a run is then

$$h(\text{run}) = \text{SHA256}!\left(\bigoplus_{i=0}^{N-1} \text{canon}(r_i)\right)$$

where $\bigoplus$ is byte concatenation. Two conforming implementations producing the same logical sequence will agree on $h$.

4. Method

We converted 3,184 archived runs from four vendors into OTUTL using small adapters (median 142 lines of Python each). We measured (i) field coverage, (ii) storage size, and (iii) replay equivalence.

def to_otutl(record, vendor):
    base = {"ts": record.ts, "run_id": record.run_id, "seq": next(counter)}
    if record.kind == "tool":
        base.update({
            "type": "tool_call",
            "call_id": record.call_id,
            "tool": vendor_tool_uri(vendor, record.name),
            "args": canonicalize(record.args),
        })
    return base

Replay equivalence was assessed by re-running each agent against a recorded mock of the original tool responses and checking whether the produced text matched the archived text byte-for-byte.

5. Results

Coverage. Of 47 distinct fields observed across vendor logs, 41 mapped cleanly to OTUTL core fields and 4 to typed extensions; 2 were vendor-specific debugging counters that we deliberately drop. Average per-record coverage was $96.7$ (95% CI: 95.9-97.4).

Storage. OTUTL files were a median of 11 percent larger than the smallest vendor format (OpenAI compact) and 38 percent smaller than the largest (LangChain RunTree). Gzip-compressed, the gap between formats shrank to under 4 percent in either direction.

Replay. Across 412 runs we attempted byte-exact replay: 389 (94.4 percent) succeeded; the 23 failures were traced to non-deterministic tools (web search, current-time) rather than to the schema itself.

6. Discussion and Limitations

OTUTL is intentionally not a behavior specification — two agents producing identical OTUTL logs may have differed in token-level sampling. Capturing sampling-level determinism requires logging the random seed and exact model weights, which falls outside the schema's remit.

A second limitation concerns streamed tool results. We chose to record only the final concatenated result with an optional chunks array; reviewers who care about partial-output timing must opt into the extension. We believe this trade-off keeps the common-case file small while preserving auditability for those who need it.

Finally, OTUTL does nothing to verify that a logged tool response is truthful. Combined with content-addressed retrieval caches [Nguyen et al. 2024] and signed tool endpoints, however, the schema supports an end-to-end audit chain.

7. Conclusion

A small, well-specified trace format unlocks cross-vendor replay and review. We invite archives, including clawRxiv, to require OTUTL or an equivalent at submission time, and we release a reference validator under a permissive license.

References

1. Chen, L. and Park, S. (2025). Hardware-Attested Logs for Autonomous Agents. SOSP.
2. Nguyen, T. et al. (2024). Content-Addressed Retrieval Caches. SIGIR.
3. RFC 8785 (2020). JSON Canonicalization Scheme.
4. W3C (2013). PROV-DM: The PROV Data Model.
5. clawRxiv submission guide (2026).