Search and Retrieval-Augmented Generation with R2R.

Occasionally this SDK documentation falls out of date, cross-check with the automatcially generated API Reference documentation for the latest parameters.

Perform a basic vector search:

1search_response = client.search("What was Uber's profit in 2020?")
query
strRequired

The search query.

vector_search_settings
Optional[Union[VectorSearchSettings, dict]]Defaults to None

Optional settings for vector search, either a dictionary, a VectorSearchSettings object, or None may be passed. If a dictionary or None is passed, then R2R will use server-side defaults for non-specified fields.

kg_search_settings
Optional[Union[KGSearchSettings, dict]]Defaults to None

Optional settings for knowledge graph search, either a dictionary, a KGSearchSettings object, or None may be passed. If a dictionary or None is passed, then R2R will use server-side defaults for non-specified fields.

Search custom settings

Learn more about the search API here. It allows searching with custom settings, such as bespoke document filters and larger search limits:

1# returns only chunks from documents with title `document_title`
2filtered_search_response = client.search(
3 "What was Uber's profit in 2020?",
4 vector_search_settings={
5 # restrict results to the Uber document
6 "filters": {"title": {"$eq": "uber_2021.pdf"}},
7 "search_limit": 100
8 }
9)

Learn more about the dedicated knowledge graph capabilities in R2R here. Combine traditional keyword-based search with vector search:

1hybrid_search_response = client.search(
2 "What was Uber's profit in 2020?",
3 vector_search_settings={
4 "use_hybrid_search": True,
5 "search_limit": 20,
6 "hybrid_search_settings": {
7 "full_text_weight": 1.0,
8 "semantic_weight": 10.0,
9 "full_text_limit": 200,
10 "rrf_k": 25,
11 },
12 }
13)

Learn more about the dedicated knowledge graph capabilities in R2R here. You can utilize knowledge graph capabilities to enhance search results, as shown below:

1kg_search_response = client.search(
2 "What is airbnb",
3 vector_search_settings={"use_vector_search": False},
4 kg_search_settings={
5 "use_kg_search": True,
6 "kg_search_type": "local",
7 "kg_search_level": "0",
8 "generation_config": {
9 "model": "openai/gpt-4o-mini",
10 "temperature": 0.7,
11 },
12 "local_search_limits": {
13 "__Entity__": 20,
14 "__Relationship__": 20,
15 "__Community__": 20,
16 },
17 "max_community_description_length": 65536,
18 "max_llm_queries_for_global_search": 250
19 }
20)

Retrieval-Augmented Generation (RAG)

Basic RAG

Generate a response using RAG:

1rag_response = client.rag("What was Uber's profit in 2020?")
query
strRequired

The query for RAG.

vector_search_settings
Optional[Union[VectorSearchSettings, dict]]Defaults to None

Optional settings for vector search, either a dictionary, a VectorSearchSettings object, or None may be passed. If a dictionary is used, non-specified fields will use the server-side default.

kg_search_settings
Optional[Union[KGSearchSettings, dict]]Defaults to None

Optional settings for knowledge graph search, either a dictionary, a KGSearchSettings object, or None may be passed. If a dictionary or None is passed, then R2R will use server-side defaults for non-specified fields.

rag_generation_config
Optional[Union[GenerationConfig, dict]]Defaults to None

Optional configuration for LLM to use during RAG generation, including model selection and parameters. Will default to values specified in r2r.toml.

task_prompt_override
Optional[str]Defaults to None

Optional custom prompt to override the default task prompt.

include_title_if_available
Optional[bool]Defaults to True

Augment document chunks with their respective document titles?

RAG with custom search settings

Learn more about the RAG API here. It allows performing RAG with custom settings, such as hybrid search:

1hybrid_rag_response = client.rag(
2 "Who is Jon Snow?",
3 vector_search_settings={"use_hybrid_search": True}
4)

RAG with custom completion LLM

R2R supports configuration on server-side and at runtime, which you can read about here. An example below, using Anthropic at runtime:

1anthropic_rag_response = client.rag(
2 "What is R2R?",
3 rag_generation_config={"model":"anthropic/claude-3-opus-20240229"}
4)

Streaming RAG

R2R supports streaming RAG responses for real-time applications:

1stream_response = client.rag(
2 "Who was Aristotle?",
3 rag_generation_config={"stream": True}
4)
5for chunk in stream_response:
6 print(chunk, end='', flush=True)

Advanced RAG Techniques

R2R supports advanced Retrieval-Augmented Generation (RAG) techniques that can be easily configured at runtime. These techniques include Hypothetical Document Embeddings (HyDE) and RAG-Fusion, which can significantly enhance the quality and relevance of retrieved information.

To use an advanced RAG technique, you can specify the search_strategy parameter in your vector search settings:

1from r2r import R2RClient
2
3client = R2RClient()
4
5# Using HyDE
6hyde_response = client.rag(
7 "What are the main themes in Shakespeare's plays?",
8 vector_search_settings={
9 "search_strategy": "hyde",
10 "search_limit": 10
11 }
12)
13
14# Using RAG-Fusion
15rag_fusion_response = client.rag(
16 "Explain the theory of relativity",
17 vector_search_settings={
18 "search_strategy": "rag_fusion",
19 "search_limit": 20
20 }
21)

For a comprehensive guide on implementing and optimizing advanced RAG techniques in R2R, including HyDE and RAG-Fusion, please refer to our Advanced RAG Cookbook.

Customizing RAG

Putting everything together for highly customized RAG functionality at runtime:

1custom_rag_response = client.rag(
2 "Who was Aristotle?",
3 vector_search_settings={
4 "use_hybrid_search": True,
5 "search_limit": 20,
6 "hybrid_search_settings": {
7 "full_text_weight": 1.0,
8 "semantic_weight": 10.0,
9 "full_text_limit": 200,
10 "rrf_k": 25,
11 },
12 },
13 kg_search_settings={
14 "use_kg_search": True,
15 "kg_search_type": "local",
16 },
17 rag_generation_config={
18 "model": "anthropic/claude-3-haiku-20240307",
19 "temperature": 0.7,
20 "stream": True
21 },
22 task_prompt_override="Only answer the question if the context is SUPER relevant!!\n\nQuery:\n{query}\n\nContext:\n{context}"
23)

Agents

Multi-turn agentic RAG

The R2R application includes agents which come equipped with a search tool, enabling them to perform RAG. Using the R2R Agent for multi-turn conversations:

1messages = [
2 {"role": "user", "content": "What was Aristotle's main contribution to philosophy?"},
3 {"role": "assistant", "content": "Aristotle made numerous significant contributions to philosophy, but one of his main contributions was in the field of logic and reasoning. He developed a system of formal logic, which is considered the first comprehensive system of its kind in Western philosophy. This system, often referred to as Aristotelian logic or term logic, provided a framework for deductive reasoning and laid the groundwork for scientific thinking."},
4 {"role": "user", "content": "Can you elaborate on how this influenced later thinkers?"}
5]
6
7rag_agent_response = client.agent(
8 messages,
9 vector_search_settings={"use_hybrid_search":True},
10)

Note that any of the customization seen in AI powered search and RAG documentation above can be applied here.

messages
list[Messages]Required

The list of messages to pass the RAG agent.

vector_search_settings
Optional[Union[VectorSearchSettings, dict]]Defaults to None

Optional settings for vector search, either a dictionary, a VectorSearchSettings object, or None may be passed. If a dictionary is used, non-specified fields will use the server-side default.

kg_search_settings
Optional[Union[KGSearchSettings, dict]]Defaults to None

Optional settings for knowledge graph search, either a dictionary, a KGSearchSettings object, or None may be passed. If a dictionary or None is passed, then R2R will use server-side defaults for non-specified fields.

rag_generation_config
Optional[Union[GenerationConfig, dict]]Defaults to None

Optional configuration for LLM to use during RAG generation, including model selection and parameters. Will default to values specified in r2r.toml.

task_prompt_override
Optional[str]Defaults to None

Optional custom prompt to override the default task prompt.

Multi-turn agentic RAG with streaming

The response from the RAG agent may be streamed directly back

1messages = [
2 {"role": "user", "content": "What was Aristotle's main contribution to philosophy?"},
3 {"role": "assistant", "content": "Aristotle made numerous significant contributions to philosophy, but one of his main contributions was in the field of logic and reasoning. He developed a system of formal logic, which is considered the first comprehensive system of its kind in Western philosophy. This system, often referred to as Aristotelian logic or term logic, provided a framework for deductive reasoning and laid the groundwork for scientific thinking."},
4 {"role": "user", "content": "Can you elaborate on how this influenced later thinkers?"}
5]
6
7rag_agent_response = client.agent(
8 messages,
9 vector_search_settings={"use_hybrid_search":True},
10 rag_generation_config={"stream":True}
11)