← Roadmap

ReAct: Synergizing Reasoning and Acting in Language Models

Yao et al. · 2022

① Intuition

A pure-reasoning LLM can hallucinate facts; a pure-tool-calling agent gets lost without reasoning. What if we interleave?

Setup: imagine an LLM answering “What is the population of Paris?” In Chain-of-Thought (CoT) mode it reasons entirely from memory. In Act-only mode it calls tools but without a guiding plan. ReAct alternates between thinking and acting, letting each inform the other.

Predict: what's the most common failure mode of CoT-only on factual questions?

Now step through the widget below. Select a mode (CoT, Act, ReAct) and click Step to advance the trace.

    Step through CoT, Act-only, and ReAct modes. Which one arrives at the right Paris population, and why?

    Retrieval check

    What is the primary failure mode of CoT-only on knowledge-intensive tasks?
    Why does Act-only fail on multi-step factual tasks in the widget trace?

    Answer all 2 correctly to mark this stage mastered.

    ② Math

    ReAct is formalized as a policy conditioned on an interleaved context. At each step t, the agent samples an action from:

    at∼πθ(at∣ct)a_t \sim \pi_\theta(a_t \mid c_t)

    where the context grows with every step:

    ct=(o1,a1,…,at−1,ot)c_t = (o_1, a_1, \ldots, a_{t-1}, o_t)

    The action space is extended: at∈A∪La_t \in \mathcal{A} \cup \mathcal{L} where 𝒜 is the set of tool calls (search, lookup, etc.) and ℒ is the space of natural-language thoughts. A thought action is executed purely inside the model’s context — it does not interact with any external environment.

    Which is true about a thought action a_t ∈ ℒ?
    Show next line
    at∈L  ⟹  env state unchanged, ct+1=(ct,at)a_t \in \mathcal{L} \implies \text{env state unchanged,} \ c_{t+1} = (c_t, a_t)

    Token cost: ReAct uses 2–5× more tokens than Act-only because each trajectory contains interleaved Thought steps that do not directly produce observations. This is a deliberate trade-off: more tokens per query buys substantially better factual accuracy and fewer dead-end action sequences.

    Retrieval check

    In the ReAct formalism, what distinguishes a Thought action from a tool-call action?

    Answer all 1 correctly to mark this stage mastered.

    ③ Pseudo code

    Arrange the steps of the ReAct loop in the correct order:

    1. If Finish: return the answer
    2. Parse block as Thought | Action | Finish
    3. If Action: run tool, observe result, append (Action, Observation) to c
    4. If Thought: append Thought to c
    5. Generate next token block from LLM given c
    6. Initialize context c with the user query

    What keyword terminates the ReAct loop and returns a final answer?

    Retrieval check

    In the ReAct loop, what should happen after parsing a block as an Action?

    Answer all 1 correctly to mark this stage mastered.

    ④ Code (Python)

    Implement a minimal ReAct scaffold. The loop drives a scripted LM through Thought / Action / Finish blocks, calling a canned search tool for grounding.

    Retrieval check

    In the ReAct scaffold, what does the loop do when it encounters a Finish block?

    Answer all 1 correctly to mark this stage mastered.

    ⑤ Original paper

    Your browser can't embed PDFs. Download ReAct.

    Open in new tab · Download

    Suggested reading order: §2 ReAct → §3 Knowledge-intensive tasks → §4 Interactive decision-making.