Agents

Learn how to implement and use agents with R2R

This cookbook demonstrates how to use the agentic capabilities which ship by default in R2R. The R2R agent is an intelligent system that can formulate its own questions, search for information, and provide informed responses based on the retrieved context. It can be customized on the fly.

Agents in the R2R framework are still in beta. Please let us know what you like/dislike and what features you would like to see added.

Understanding R2R’s RAG Agent

R2R’s RAG agent is designed to provide powerful, context-aware responses by combining large language models with a search capability over your ingested documents. When you initialize an R2R application, it automatically creates a RAG assistant that’s ready to use. R2R plans to extend its agent functionality to mirror core features supported by OpenAI and more, including:

  • Multiple tool support (e.g., code interpreter, file search)
  • Persistent conversation threads
  • Complete end to end observability of agent interactions

R2R also provides support for local RAG capabilities, allowing you to create AI agents that can access and reason over your local document store, entirely offline.

The RAG agent is also available for use through the R2R API, specifically via the agent endpoint.

Using the RAG Agent

Now, let’s use the RAG agent to answer questions:

1def rag_agent_example():
2    # Prepare messages for the chat
3    messages = [
4        {"role": "system", "content": "You are a helpful assistant."},
5        {
6            "role": "user",
7            "content": "Who is the greatest philospher of all time?",
8        },
9        {
10            "role": "assistant",
11            "content": "Aristotle is widely considered the greatest philospher of all time.",
12        },
13        {
14            "role": "user",
15            "content": "Can you tell me more about him?",
16        },
17    ]
18
19    # Use the RAG assistant via the agent endpoint
20    response = client.agent(
21        messages=messages,
22        vector_search_settings={"search_limit": 5, "filters": {}},
23        kg_search_settings={"use_kg_search": False},
24        rag_generation_config={"max_tokens": 300}
25    )
26
27    print("RAG Assistant Response:")
28    # Note, response includes the full conversation, including steps taken by the assistant to produce the final result.
29    print(response)
30
31# Run the example
32rag_agent_example()
33
34# RAG Assistant Response:
35# {
36#    'results': [
37#        {'role': 'system', 'content': '## You are a helpful assistant that can search for information.\n\nWhen asked a question, perform a search to find relevant information and provide a response.\n\nThe response should contain line-item attributions to relevent search results, and be as informative if possible.\nIf no relevant results are found, then state that no results were found.\nIf no obvious question is present, then do not carry out a search, and instead ask for clarification.', 'name': None, 'function_call': None, 'tool_calls': None},
38#        {'role': 'system', 'content': 'You are a helpful assistant.', 'name': None, 'function_call': None, 'tool_calls': None},
39#        {'role': 'user', 'content': 'Who is the greatest philospher of all time?', 'name': None, 'function_call': None, 'tool_calls': None},
40#        {'role': 'assistant', 'content': 'Aristotle is widely considered the greatest philospher of all time.', 'name': None, 'function_call': None, 'tool_calls': None},
41#        {'role': 'user', 'content': 'Can you tell me more about him?', 'name': None, 'function_call': None, 'tool_calls': None},
42#        {'role': 'assistant', 'content': None, 'name': None, 'function_call': {'name': 'search', 'arguments': '{"query":"Aristotle biography"}'}, 'tool_calls': None},
43#        {'role': 'function', 'content': "1. Aristotle[A] (Greek: Ἀριστοτέλης Aristotélēs, pronounced [aristotélɛːs]; 384–322 BC) was an Ancient Greek philosopher and polymath. His writings cover a broad range of subjects spanning the natural sciences, philosophy, linguistics, economics, politics, psychology, and the arts. As the founder of the Peripatetic school of philosophy in the Lyceum in Athens, he began the wider Aristotelian tradition that followed, which set the groundwork for the development of modern science.\n2. Aristotle[A] (Greek: Ἀριστοτέλης Aristotélēs, pronounced [aristotélɛːs]; 384–322 BC) was an Ancient Greek philosopher and polymath. His writings cover a broad range of subjects spanning the natural sciences, philosophy, linguistics, economics, politics, psychology, and the arts. As the founder of the Peripatetic school of philosophy in the Lyceum in Athens, he began the wider Aristotelian tradition that followed, which set the groundwork for the development of modern science.\n3. Aristotle was born in 384 BC[C] in Stagira, Chalcidice,[2] about 55 km (34 miles) east of modern-day Thessaloniki.[3][4] His father, Nicomachus, was the personal physician to King Amyntas of Macedon. While he was young, Aristotle learned about biology and medical information, which was taught by his father.[5] Both of Aristotle's parents died when he was about thirteen, and Proxenus of Atarneus became his guardian.[6] Although little information about Aristotle's childhood has survived, he probably spent\n4. Aristotle was born in 384 BC[C] in Stagira, Chalcidice,[2] about 55 km (34 miles) east of modern-day Thessaloniki.[3][4] His father, Nicomachus, was the personal physician to King Amyntas of Macedon. While he was young, Aristotle learned about biology and medical information, which was taught by his father.[5] Both of Aristotle's parents died when he was about thirteen, and Proxenus of Atarneus became his guardian.[6] Although little information about Aristotle's childhood has survived, he probably spent\n5. Life\nIn general, the details of Aristotle's life are not well-established. The biographies written in ancient times are often speculative and historians only agree on a few salient points.[B]\n", 'name': 'search', 'function_call': None, 'tool_calls': None},
44#        {'role': 'assistant', 'content': "Aristotle (384–322 BC) was an Ancient Greek philosopher and polymath whose contributions have had a profound impact on various fields of knowledge. Here are some key points about his life and work:\n\n1. **Early Life**: Aristotle was born in 384 BC in Stagira, Chalcidice, which is near modern-day Thessaloniki, Greece. His father, Nicomachus, was the personal physician to King Amyntas of Macedon, which exposed Aristotle to medical and biological knowledge from a young age [C].\n\n2. **Education and Career**: After the death of his parents, Aristotle was sent to Athens to study at Plato's Academy, where he remained for about 20 years. After Plato's death, Aristotle left Athens and eventually became the tutor of Alexander the Great [C].\n\n3. **Philosophical Contributions**: Aristotle founded the Lyceum in Athens, where he established the Peripatetic school of philosophy. His works cover a wide range of subjects, including metaphysics, ethics, politics, logic, biology, and aesthetics. His writings laid the groundwork for many modern scientific and philosophical inquiries [A].\n\n4. **Legacy**: Aristotle's influence extends beyond philosophy to the natural sciences, linguistics, economics, and psychology. His method of systematic observation and analysis has been foundational to the development of modern science [A].\n\nAristotle's comprehensive approach to knowledge and his systematic methodology have earned him a lasting legacy as one of the greatest philosophers of all time.\n\nSources:\n- [A] Aristotle's broad range of writings and influence on modern science.\n- [C] Details about Aristotle's early life and education.", 'name': None, 'function_call': None, 'tool_calls': None}
45#    ]
46# }

In this example, the agent might formulate its own questions to gather more information before providing a response. For instance, it could ask the search tool about Aristotle’s scientific works or his methods of inquiry.

Streaming Responses

To see the agent’s thought process in real-time, you can use streaming responses:

1def streaming_rag_agent_example():
2    messages = [
3        {"role": "system", "content": "You are a helpful agent with access to a large knowledge base. You can ask questions to gather more information if needed."},
4        {"role": "user", "content": "What were Aristotle's main philosophical ideas?"},
5    ]
6
7    streaming_response = client.agent(
8        messages=messages,
9        vector_search_settings={"search_limit": 5, "filters": {}},
10        kg_search_settings={"use_kg_search": False},
11        rag_generation_config={"max_tokens": 300, "stream": True}
12    )
13
14    print("Streaming RAG Assistant Response:")
15    for chunk in streaming_response:
16        print(chunk, end="", flush=True)
17    print()  # New line after streaming response
18
19streaming_rag_agent_example()
20
21# Streaming RAG Assistant Response:
22
23# <function_call><name>search</name><arguments>{"query":"Aristotle's main philosophical ideas"}</arguments><results>1. Politics
24# Main article: Politics (Aristotle)
25# 2. Ethics
26# Main article: Aristotelian ethics
27# Aristotle considered ethics to be a practical rather than theoretical study, i.e., one aimed at becoming good and doing good rather than knowing for its own sake. He wrote several treatises on ethics, most notably including the Nicomachean Ethics.[117]
28# 3. Metaphysics
29# Main article: Metaphysics (Aristotle)
30# The word "metaphysics" appears to have been coined by the first century AD editor who assembled various small selections of Aristotle's works to the treatise we know by the name Metaphysics.[34] Aristotle called it "first philosophy", and distinguished it from mathematics and natural science (physics) as the contemplative (theoretikē) philosophy which is "theological" and studies the divine. He wrote in his Metaphysics (1026a16):
31# 4. Practical philosophy
32# Aristotle's practical philosophy covers areas such as ethics, politics, economics, and rhetoric.[40]
33# 5. Aristotle[A] (Greek: Ἀριστοτέλης Aristotélēs, pronounced [aristotélɛːs]; 384–322 BC) was an Ancient Greek philosopher and polymath. His writings cover a broad range of subjects spanning the natural sciences, philosophy, linguistics, economics, politics, psychology, and the arts. As the founder of the Peripatetic school of philosophy in the Lyceum in Athens, he began the wider Aristotelian tradition that followed, which set the groundwork for the development of modern science.
34# </results></function_call>Aristotle, an ancient Greek philosopher, made significant contributions across various fields of philosophy. Here are some of his main philosophical ideas:
35#
36# 1. **Metaphysics**: Aristotle's metaphysics, often referred to as "first philosophy," deals with the nature of reality, existence, and the fundamental nature of being. He introduced the concept of substance and essence, and his work laid the groundwork for later metaphysical thought [Metaphysics (Aristotle)].
37#
38# 2. **Ethics**: Aristotle's ethical philosophy is primarily outlined in his work "Nicomachean Ethics." He emphasized the concept of virtue ethics, which focuses on the development of good character traits (virtues) and living a life of moral excellence. He believed that the ultimate goal of human life is eudaimonia, often translated as "happiness" or "flourishing," achieved through virtuous living [Aristotelian ethics].
39#
40# 3. **Politics**: In his work "Politics," Aristotle explored the nature of human communities and the role of the state. He believed that humans are naturally political animals and that the state exists to promote the good life. He analyzed different forms of government and advocated for a constitutional government that balances the interests of different social classes [Politics (Aristotle)].
41#
42# 4. **Practical Philosophy**: Aristotle's practical philosophy encompasses ethics, politics, economics, and rhetoric. He believed that philosophy should be practical and aimed at improving human life and society [Practical philosophy].
43#
44# 5. **Natural Sciences**: Aristotle made substantial contributions to the natural sciences, including biology, physics, and astronomy. He conducted empirical observations and classified various forms of life, laying the foundation for the scientific method [Aristotle's contributions to natural sciences].
45
46# Aristotle's extensive body of work has had a profound and lasting impact on Western philosophy and science, influencing countless thinkers and shaping the development of various academic disciplines.

This will produce a streaming response, showing the agent’s thought process, including its search queries and the gradual construction of its response.

Customizing the RAG Agent

This section is still under development, please proceed carefully when customizing agents.
You can customize various aspects of the RAG agent’s behavior:

1custom_response = client.agent(
2    messages=[
3        {"role": "system", "content": "You are an expert on ancient Greek philosophy. Ask clarifying questions if needed."},
4        {"role": "user", "content": "Compare Aristotle's and Plato's views on the nature of reality."},
5    ],
6    vector_search_settings={"search_limit": 25, "use_hybrid_search": True, "filters": {"category": {"$eq": "philosophy"}}},
7    kg_search_settings={"use_kg_search": False},
8    rag_generation_config={
9        "max_tokens": 500,
10        "temperature": 0.7,
11        "model": "openai/gpt-4o"  # Assuming you have access to GPT-4
12    }
13)
14
15print("Customized RAG Agent Response:")
16print(custom_response)

This example demonstrates how to:

  • Set custom search filters
  • Enable hybrid search (combining vector and keyword search)
  • Adjust the number of search results
  • Customize the generation config
  • Use a specific model for the response

…

Conclusion

The R2R RAG assistant is a powerful tool that combines large language models with advanced search capabilities. By leveraging your ingested documents and the flexible agent endpoint, you can create dynamic, context-aware conversational experiences.

Key takeaways:

  1. R2R automatically creates a RAG assistant when initializing the application.
  2. The agent endpoint provides easy access to the RAG assistant’s capabilities.
  3. You can customize various aspects of the assistant’s behavior, including search settings and generation parameters.
  4. Streaming responses allow for real-time interaction and visibility into the assistant’s thought process.

Experiment with different settings and configurations to find the optimal setup for your specific use case. Remember to keep your ingested documents up-to-date to ensure the assistant has access to the most relevant and recent information.