Autonomous Agents: ReAct, Memory & Tool-Calling Loops | AI & Machine Learning Engineering

Standard Large Language Models generate text auto-regressively in a single open-loop pass. While proficient at semantic translation and synthesis, static feedforward generation cannot query live external data, execute deterministic calculations, or dynamically correct mistakes during multi-step problem solving.

Autonomous Agents frame the LLM as a cognitive decision-making core embedded within a closed-loop environment. By interleaving verbal reasoning with external tool execution, agents plan trajectories, observe empirical environmental feedback, and iteratively solve complex multi-step objectives.

The ReAct LoopAgentic Systems

Thought_t -> Action_t -> Observation_t

Synergistic interleaving of verbal reasoning traces with deterministic tool executions solves hallucinations and ground-truth validation.

1. Cognitive Architecture of an Autonomous Agent#

An autonomous agent system consists of four primary subsystems:

Architecture Diagram

Rendering diagram...

The Reasoning Core (Brain): The foundation model responsible for semantic parsing, chain-of-thought planning, error reflection, and policy action selection.
Memory Architectures:

Short-Term (Working) Memory: The active context window containing recent conversation history, current system instructions, and immediate scratchpad thoughts.
Long-Term Memory: External retrieval systems (such as vector embeddings or document indices) storing semantic world knowledge and episodic records of past trajectories.
Procedural Memory: Fixed instructions, behavioral rules, few-shot demonstration exemplars, and strict JSON schemas governing tool capabilities.

Planning & Decomposition: Breaking down high-level ambiguous goals ("Analyze Q3 churn and summarize anomalies") into discrete, executable atomic sub-tasks.
Tool Execution Layer: Sandboxed interfaces enabling interaction with external systems, REST APIs, databases, and code interpreters.

2. The ReAct (Reasoning + Acting) Paradigm#

Introduced by Shunyu Yao et al. (2022), ReAct resolves the critical limitations of both pure reasoning (Chain-of-Thought) and pure acting:

Pure Chain-of-Thought suffers from hallucination because it cannot verify intermediate factual assertions against ground truth.
Pure Acting lacks strategic foresight, blindly executing commands without synthesizing context or recovering from unexpected failures.

The Formal ReAct Loop#

At discrete time step $t$ , the agent receives the complete historical trajectory:

\mathcal{H}_t = (S_{\text{sys}}, \text{Goal}, T_1, A_1, O_1, \dots, T_{t-1}, A_{t-1}, O_{t-1})

The step executes through three explicit phases:

Architecture Diagram

Rendering diagram...

Thought Generation ( $T_t$ ): The agent reasons verbally about its progress, notes missing information, and formulates an intent:

T_t = \text{LLM}(\mathcal{H}_t)

Action Selection ( $A_t$ ): The agent chooses an action name and structured arguments:

A_t = \text{ExtractAction}(T_t) = (\text{tool\_name}, \text{args})

Environmental Observation ( $O_t$ ): The external runtime executes the tool and captures standard output, return data, or exceptions:

O_t = \text{Runtime}(A_t)

The observation is concatenated into $\mathcal{H}_{t+1}$ . The loop terminates when the agent emits a special terminal action:

A_{\text{final}} = \text{Finish}(\text{answer})

3. Reflexion: Verbal Reinforcement Learning#

Traditional reinforcement learning optimizes policy weights via scalar numeric rewards and backpropagation. In contrast, Reflexion (Shinn et al., 2023) implements verbal reinforcement learning:

Architecture Diagram

Rendering diagram...

When an agent fails a task (e.g. unit tests fail or a tool returns an unrecoverable error), an internal Evaluator triggers a self-reflection prompt:

\text{Critique}_k = \text{LLM}(\text{Task}, \tau_k, \text{Error})

The critique is appended to an episodic memory buffer:

M_{\text{refl}} \leftarrow M_{\text{refl}} \cup \{ \text{Critique}_k \}

On the subsequent attempt $\tau_{k+1}$ , the agent receives $M_{\text{refl}}$ inside its prompt context, allowing it to bypass previous dead ends without altering a single neural network weight.

4. Tool Schemas and Self-Healing Execution Loops#

In production environments, tool calling requires strict interface contracts. Modern LLMs use JSON Schema declarations to define valid function signatures:

1
2
3
4
5
6
7
8
9
10
11
{
  "name": "lookup_customer",
  "description": "Fetch customer profile by unique ID",
  "parameters": {
    "type": "object",
    "properties": {
      "customer_id": { "type": "integer", "description": "The customer identifier" }
    },
    "required": ["customer_id"]
  }
}

Self-Healing Execution#

LLMs frequently generate invalid arguments, malformed JSON, or reference non-existent files. Rather than crashing the agent, robust runtimes intercept the error and route it back as an observation:

O_t = \text{"ExecutionError: parameter 'customer_id' must be an integer, got string 'CUST_42'"}

The LLM inspects the error in $T_{t+1}$ and automatically issues a corrected action $A_{t+1}$ .

5. From-Scratch Implementation: ReAct Agent Runtime#

Let us construct a complete, zero-dependency ReAct Agent runtime from first principles in both Python and Rust.

The agent features:

A registry of mock tools: calculate(expr) and get_weather(city).
A deterministic rule-based mock LLM brain simulating Chain-of-Thought parsing.
A full Thought -> Action -> Observation loop with step limits to prevent infinite cycles.

Python Implementation (Pure Standard Library)#

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
"""
Autonomous ReAct Agent Runtime from scratch in pure Python standard library.
Implements Thought-Action-Observation loops, tool dispatch, and step guards.
"""
import re
from typing import Dict, Any, Callable, Tuple

# 1. Tool Registry
class ToolRegistry:
    def __init__(self):
        self.tools: Dict[str, Callable[[str], str]] = {}

    def register(self, name: str, func: Callable[[str], str]):
        self.tools[name] = func

    def execute(self, name: str, arg: str) -> str:
        if name not in self.tools:
            return f"Error: Tool '{name}' does not exist. Available tools: {list(self.tools.keys())}"
        try:
            return self.tools[name](arg)
        except Exception as e:
            return f"ExecutionError: {type(e).__name__}: {str(e)}"

# Mock tool functions
def tool_calculate(expression: str) -> str:
    # Safe evaluation of basic arithmetic
    sanitized = re.sub(r"[^0-9+\-*/(). ]", "", expression)
    return str(eval(sanitized))

def tool_get_weather(city: str) -> str:
    db = {"tokyo": "22C, Sunny", "paris": "15C, Rainy", "new york": "18C, Clear"}
    clean_city = city.strip().lower()
    return db.get(clean_city, f"City '{city}' not found in weather records.")

# 2. Mock Agent Brain simulating ReAct generation
class MockReActLLM:
    """Simulates LLM response based on prompt state to demonstrate mechanics."""
    def generate_step(self, trajectory: str) -> str:
        if "Observation: 22C, Sunny" in trajectory:
            return "Thought: I have retrieved the weather for Tokyo.\nAction: finish[Tokyo is currently 22C and Sunny.]"
        elif "Observation:" in trajectory and "Action: weather[" in trajectory:
            return "Thought: I now have the weather data.\nAction: finish[Weather query complete.]"
        elif "Action: weather[" not in trajectory:
            return "Thought: To answer the question about Tokyo's weather, I need to check the weather tool.\nAction: weather[Tokyo]"
        return "Thought: I cannot proceed further.\nAction: finish[No solution found.]"

# 3. Agent Execution Engine
class ReActAgent:
    def __init__(self, registry: ToolRegistry, max_steps: int = 5):
        self.registry = registry
        self.max_steps = max_steps
        self.llm = MockReActLLM()

    def run(self, query: str) -> str:
        trajectory = f"Question: {query}\n"
        print(f"--- Starting ReAct Agent Run ---\nTask: {query}\n")

        for step in range(1, self.max_steps + 1):
            response = self.llm.generate_step(trajectory)
            trajectory += response + "\n"
            print(f"[Step {step}]\n{response}")

            # Parse action pattern: Action: tool_name[argument]
            match = re.search(r"Action:\s*([a-zA-Z0-9_]+)\[(.*?)\]", response)
            if not match:
                print("Error: No valid action found. Stopping.")
                break

            tool_name, tool_arg = match.group(1), match.group(2)

            # Check termination
            if tool_name.lower() == "finish":
                print(f"\nFinal Answer: {tool_arg}")
                return tool_arg

            # Execute tool
            obs = self.registry.execute(tool_name, tool_arg)
            obs_str = f"Observation: {obs}"
            print(f"{obs_str}\n")
            trajectory += obs_str + "\n"

        return "Agent stopped: Reached maximum step limit without terminal answer."

if __name__ == "__main__":
    tools = ToolRegistry()
    tools.register("calculate", tool_calculate)
    tools.register("weather", tool_get_weather)

    agent = ReActAgent(tools, max_steps=4)
    result = agent.run("What is the current weather condition in Tokyo?")

Rust Implementation (Pure Standard Library)#

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
// Autonomous ReAct Agent Loop from scratch in pure Rust standard library.
// Demonstrates Thought-Action-Observation orchestration and tool invocation.

use std::collections::HashMap;

struct ToolRegistry {
    tools: HashMap<String, Box<dyn Fn(&str) -> String>>,
}

impl ToolRegistry {
    fn new() -> Self {
        Self {
            tools: HashMap::new(),
        }
    }

    fn register<F>(&mut self, name: &str, func: F)
    where
        F: Fn(&str) -> String + 'static,
    {
        self.tools.insert(name.to_string(), Box::new(func));
    }

    fn execute(&self, name: &str, arg: &str) -> String {
        match self.tools.get(name) {
            Some(func) => func(arg),
            None => format!("Error: Tool '{}' not registered.", name),
        }
    }
}

// Mock LLM step engine
struct MockLLM;

impl MockLLM {
    fn generate_step(&self, history: &str) -> String {
        if history.contains("Observation: 22C, Sunny") {
            "Thought: I have the verified weather for Tokyo.\nAction: finish[Tokyo is 22C and Sunny.]".to_string()
        } else if !history.contains("Action: weather[") {
            "Thought: I must inspect current weather records for Tokyo.\nAction: weather[Tokyo]".to_string()
        } else {
            "Thought: Unrecognized state.\nAction: finish[Unable to resolve query.]".to_string()
        }
    }
}

// Action parser
struct ActionCommand {
    tool: String,
    argument: String,
}

fn parse_action(text: &str) -> Option<ActionCommand> {
    let action_marker = "Action:";
    if let Some(pos) = text.find(action_marker) {
        let remainder = &text[pos + action_marker.len()..].trim();
        if let Some(bracket_open) = remainder.find('[') {
            if let Some(bracket_close) = remainder.find(']') {
                let tool = remainder[..bracket_open].trim().to_string();
                let argument = remainder[bracket_open + 1..bracket_close].trim().to_string();
                return Some(ActionCommand { tool, argument });
            }
        }
    }
    None
}

fn main() {
    let mut registry = ToolRegistry::new();

    // Register tools
    registry.register("weather", |city| {
        if city.to_lowercase() == "tokyo" {
            "22C, Sunny".to_string()
        } else {
            "City not found".to_string()
        }
    });

    registry.register("calculate", |expr| {
        format!("Calculated: {}", expr)
    });

    let llm = MockLLM;
    let mut history = String::from("Question: What is the weather in Tokyo?\n");
    let max_steps = 4;

    println!("--- ReAct Agent Runtime (Rust) ---");
    for step in 1..=max_steps {
        let step_output = llm.generate_step(&history);
        history.push_str(&step_output);
        history.push('\n');

        println!("[Step {}]\n{}", step, step_output);

        if let Some(cmd) = parse_action(&step_output) {
            if cmd.tool == "finish" {
                println!("\nTerminated: Final Answer -> {}", cmd.argument);
                return;
            }

            let observation = registry.execute(&cmd.tool, &cmd.argument);
            let obs_line = format!("Observation: {}\n", observation);
            println!("{}", obs_line.trim());
            history.push_str(&obs_line);
        } else {
            println!("No actionable command found. Aborting.");
            break;
        }
    }
}

6. Multi-Agent Topologies#

When tasks grow beyond individual context capacities, multi-agent coordination architectures divide cognitive labor:

Topology	Mechanics	Primary Benefit	Failure Risk
Hierarchical (Manager-Worker)	Manager delegates sub-goals to specialist workers and aggregates results	Strict task decomposition and bounded worker contexts	Single point of failure at manager level
Peer Debate (Democratic)	Multiple agents argue differing hypotheses until consensus is reached	Mitigates individual hallucination and bias	High token cost and potential circular deadlocks
Swarm / Event-Driven	Autonomous agents publish events to a shared message bus asynchronously	Highly resilient, modular, and parallel	Non-deterministic race conditions and trace opacity

7. Knowledge Check#

Concept check

Question

question: "Why is the ReAct loop superior to unconstrained Chain-of-Thought for factual queries?"

options:

"(Recommended) Because ReAct grounds reasoning traces with empirical observations from external tools, eliminating unverified hallucinations"
"Because ReAct avoids using an LLM for token generation"
"Because ReAct guarantees zero latency during inference"
"Because ReAct permanently updates the foundation model weights after each step"

explanation: "Chain-of-Thought generates internal reasoning traces but cannot verify factual assertions against reality. ReAct executes tools after thoughts, injecting ground-truth environmental observations back into the context."

:::

Concept check

Question

question: "How does Reflexion differ from traditional Policy Gradient reinforcement learning?"

options:

"(Recommended) Reflexion optimizes policies through verbal critiques stored in working/episodic memory without computing mathematical gradients or modifying weights"
"Reflexion requires a quadratic programming solver"
"Reflexion only works with convolutional neural networks"
"Reflexion eliminates the need for short-term memory"

explanation: "Reflexion is a form of verbal reinforcement learning. It diagnoses failures through natural language critiques and stores them in episodic memory buffers, improving subsequent iterations purely through in-context learning without gradient descent."

:::