The Brittleness of Agentic Reasoning and Planning Using LLMs

The ability of Large Language Models (LLMs) to reason and make sequential decisions has been a topic of debate. The ReAct framework, introduced by Yao et al, claimed to enhance the reasoning and planning abilities of LLMs by interleaving reasoning traces with action execution. In the original paper, they explored the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner. The central claim was one of synergy between reasoning and action: how reasoning traces helped the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information. 

This is now a popular approach, with many researchers and practitioners adopting ReAct to improve LLM performance on tasks requiring reasoning and decision-making.

Recently, a group of researchers from Arizona State University decided to investigate the claims made by ReAct and examine the factors contributing to its perceived success. They were particularly interested in examining the impact of the structure of reasoning traces. A reasoning trace is a step-by-step explanation of the thought process or logic used to solve a problem or complete a task. It outlines the sequence of mental steps taken to arrive at a solution, providing insight into the reasoning behind each action or decision. In the context of the ReAct framework, a reasoning trace is interleaved with the actions taken by the AI model, with the goal of guiding the model's decision-making process.

To put it simply, a reasoning trace is like a "think-aloud" protocol, where the AI model verbalizes its thought process as it works through a problem, explaining why it takes each action and how it plans to proceed. This trace is meant to help the model make better decisions by providing a structured way of thinking about the task at hand.