"""

LLM-based Agent Research Review

1. Introduction

1.1 Historical Background

1.1.1 The Origin of Agents

The concept of "agents" originated from philosophy, describing entities with desires, beliefs, intentions, and the ability to take actions.
In the field of artificial intelligence, the term has been given a new meaning: intelligent entities with autonomy, reactivity, proactiveness, and social capabilities.

1.1.2 Evolution of Agent Design

Past works have mainly focused on enhancing specific capabilities of agents, such as symbolic reasoning or mastering specific tasks (e.g., chess, Go).
These studies have paid more attention to algorithm design and training strategies, neglecting the development of inherent general capabilities of models, such as knowledge memory, long-term planning, effective generalization, and efficient interaction.
It has been proven that enhancing the inherent capabilities of models is a key factor in promoting the further development of intelligent agents.

1.2 Current State and Future Trends

1.2.1 The Emergence of Large Language Models

The emergence of large language models (LLMs) has brought hope for the further development of intelligent agents.
LLMs have reached the second level of development from corpus, internet, perception, embodiment, and social attributes.
With the addition of perception and action spaces, LLMs can reach the third and fourth levels.
Further, multiple agents interacting, cooperating to solve more complex tasks, or reflecting real-world social behaviors can potentially reach the fifth level – agent society.

1.2.2 A Framework for LLM-based Agents

Inspired by Darwin's "Survival of the Fittest" law, the authors propose a universal framework for LLM-based intelligent agents.
The framework consists of three parts: the control end (Brain), the perception end (Perception), and the action end (Action).

2. The Components of LLM-based Agents

2.1 Control End: Brain

2.1.1 Natural Language Interaction

Language is a medium rich in information. LLMs have shown remarkable abilities in natural language generation and understanding.
LLMs can engage in multi-turn interactions with the outside world and achieve goals.

2.1.2 Understanding Implicit Meanings

LLMs can understand the implicit meanings behind language, such as the speaker's intentions and preferences.
This understanding helps the agent communicate and cooperate more efficiently.

2.1.3 Knowledge and Memory

LLMs, trained on a large amount of corpus, have the ability to store a vast amount of knowledge.
Memory modules store the agent's observations, thoughts, and action sequences, enabling the agent to reflect and apply past strategies.

2.1.4 Reasoning & Planning

Reasoning is crucial for the agent to make decisions and analyze complex tasks.
Planning involves breaking down complex tasks into more manageable subtasks and reflecting on the plan to evaluate its effectiveness.

2.1.5 Transferability & Generalization

LLMs possess strong transferability and generalization capabilities due to their knowledge of the world.
They can generalize to unknown tasks and learn in-context, extending their capabilities to multi-modal scenarios.

2.2 Perception End: Perception

2.2.1 Text Input

LLMs have the basic ability to process text input.

2.2.2 Visual Input

LLMs do not have visual perception capabilities but can understand discrete textual representations of visual information.
Common methods include converting visual input into text descriptions or encoding visual information for LLMs to understand.

2.2.3 Auditory Input

LLMs can utilize their excellent tool-calling capabilities to perceive audio information.
Audio can also be represented visually, such as through spectrograms, allowing for transfer of visual processing methods to the audio domain.

2.2.4 Other Inputs

LLMs are expected to have more diverse perception modules in the future, such as touch and smell.
They should also be able to perceive the overall environment, including temperature, humidity, and brightness.

2.3 Action End: Action

2.3.1 Text Output

LLMs have the basic ability to produce text output.

2.3.2 Tool Use

LLMs can use tools to improve robustness and avoid hallucinations.
They can also expand their action space by calling expert models or generating multi-modal actions.

2.3.3 Embodied Action

Embodiment refers to the ability of the agent to understand, modify, and update its own state while interacting with the environment.
Atomic actions include observation, manipulation, and navigation.

3. Applications of LLM-based Agents

3.1 Single Agent Scenarios

3.1.1 Task-Oriented

Agents help users handle daily basic tasks, requiring basic instruction understanding, task decomposition, and environmental interaction.

3.1.2 Innovation-Oriented

Agents show potential for autonomous inquiry in frontier scientific fields.

3.1.3 Lifecycle-Oriented

Agents can explore, learn, and use new skills in an open world, with Minecraft as an example.

3.2 Multi-Agent Scenarios

3.2.1 Cooperative Interactions

Cooperative agents can improve task efficiency and jointly improve decisions.
They can be either unordered or ordered in their cooperation.

3.2.2 Competitive Interactions

Agents interact competitively, through competition, negotiation, and debate, to improve the system's response quality.

3.3 Human-Agent Interactions

3.3.1 Instructor-Executor

Humans guide the agent with instructions and feedback, while the agent adjusts and optimizes based on these instructions.

3.3.2 Equal Partnership

Agents can demonstrate empathy and participate in tasks as equals with humans.

4. Agent Society: From Personality to Sociality

4.1 Social Behavior and Personality

4.1.1 Social Behavior

Individual behavior constitutes the foundation of the agent's operation and development, including input through perception and output through action.
Group behavior emerges when two or more agents interact spontaneously, including positive behavior like cooperation and negative behavior like conflict.

4.1.2 Personality

Personality includes cognitive, emotional, and character aspects. Agents show "human-like intelligence" by shaping their personality through interactions with groups and environments.

4.2 The Environment of the Agent Society

4.2.1 Text-Based Environment

LLMs rely on language as their input and output format, making text-based environments the most natural operating platform for agents.

4.2.2 Virtual Sandbox Environment

Virtual sandbox environments serve as platforms for simulating social interactions and behavior.

4.2.3 Physical Environment

The physical environment introduces rich sensory inputs and requires the agent to have adaptive behavior and executable motion control.

4.3 The Meaning of Simulation

A simulated society should be open, persistent, situational, and organized.
Simulated societies can be used to explore the capabilities of collective intelligence, accelerate the research of social sciences, and assist decision-making by simulating policy impacts.