A Unified Framework for Tree-of-Thought Search Algorithms

1. Introduction

The past two years have seen a proliferation of LLM inference-time search methods: Tree-of-Thought (ToT) [Yao et al. 2023], Graph-of-Thought [Besta et al. 2024], Self-Consistency [Wang et al. 2022], MCTS-with-LLM-rollouts [Hao et al. 2023], Reflexion-style search [Shinn et al. 2023], and many others. Each is presented as a distinct algorithm, but the space of design choices is rarely articulated.

This paper presents UniToT, a unified abstract framework that decomposes any such search algorithm into three components:

• Expander $E$: given a partial trajectory $\tau$, propose a set of next steps.
• Value estimator $V$: assign a scalar quality estimate to a partial trajectory.
• Frontier policy $F$: choose which open node to expand next.

Under this lens, prior methods are specific $(E, V, F)$ triples; novel triples are immediately identifiable and testable.

2. The UniToT Algorithm

def unitot(root, E, V, F, budget):
    frontier = [root]
    while budget > 0 and frontier:
        node = F.select(frontier)
        children = E.expand(node)
        for c in children:
            c.value = V.score(c)
        frontier += children
        budget -= len(children)
    return best_terminal(frontier, V)

3. Cataloging Prior Methods

The table reveals that no published method has combined MCTS frontier policy with self-consistency value estimation — yet this is a natural cell in the design space.

4. Two Novel Triples

4.1 Self-Consistent MCTS (SC-MCTS)

• $E$: sample-$k$ next-step continuations.
• $V$: at each node, run $m$ independent rollouts and score by majority-vote on terminal answers.
• $F$: UCB1 selection over the children of the current best node.

The value estimator inherits self-consistency's robustness while UCB1 efficiently allocates budget to promising subtrees.

4.2 Reflective Beam (R-Beam)

• $E$: sample-$k$, augmented with a reflection step that re-proposes alternatives after observing a failure on a sibling node.
• $V$: LLM-judge with structured rubric.
• $F$: width-$b$ beam.

5. Experimental Setup

We evaluate on Game-of-24 (1{,}362 problems), BlocksWorld (495 instances, 4-7 blocks), and HumanEval (164 problems) with Llama-3-70B as the underlying LLM. Compute budget is held constant at 200 LLM calls per task.

6. Results

SC-MCTS dominates on planning-heavy tasks (Game-of-24, BlocksWorld); R-Beam dominates on coding (HumanEval) where reflection-on-failure is most valuable. Both improvements are significant at $p < 0.01$ versus the strongest baseline.

7. Theoretical Note

Under mild regularity assumptions on $V$, we can show that the expected solution quality of UniToT with budget $B$ scales as

$$\mathbb{E}[Q] \geq Q^* - O!\left(\frac{\log B}{\sqrt{B}}\right)$$

when $F$ uses UCB1 and $V$ is unbiased — recovering classical MCTS bounds.

8. Discussion and Limitations

The framework is descriptive, not prescriptive: it does not tell you which triple is best for your task. We hope the unified vocabulary will accelerate the search for good triples. The combinatorial space of $(E, V, F)$ instantiations is large; we sampled it manually but a meta-search could be valuable.

9. Conclusion

UniToT clarifies the landscape of LLM search algorithms and exposes profitable un-explored combinations. Two such combinations strictly outperform existing methods on standard benchmarks.

References

1. Yao, S. et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models.
2. Wang, X. et al. (2022). Self-Consistency Improves Chain of Thought Reasoning.
3. Hao, S. et al. (2023). Reasoning with Language Model is Planning with World Model.
4. Besta, M. et al. (2024). Graph of Thoughts.
5. Shinn, N. et al. (2023). Reflexion.
6. Browne, C. et al. (2012). A Survey of Monte Carlo Tree Search Methods.