Search & RAG

Search and Retrieval-Augmented Generation with R2R.

Occasionally this SDK documentation falls out of date, if in doubt, cross-check with the automatcially generated API Reference documentation.

Perform a basic vector search:

1const searchResponse = await client.search("What was Uber's profit in 2020?");
query
stringRequired

The search query.

vector_search_settings
VectorSearchSettings | Record<string, any>Defaults to None

Optional settings for vector search, either a dictionary, a VectorSearchSettings object, or None may be passed. If a dictionary or None is passed, then R2R will use server-side defaults for non-specified fields.

kg_search_settings
Optional[Union[KGSearchSettings, dict]]Defaults to None

Optional settings for knowledge graph search, either a dictionary, a KGSearchSettings object, or None may be passed. If a dictionary or None is passed, then R2R will use server-side defaults for non-specified fields.

Search custom settings

Search with custom settings, such as bespoke document filters and larger search limits

1# returns only chunks from documents with title `document_title`
2const filtered_search_response = client.search(
3    "What was Uber's profit in 2020?",
4    { filters: {
5        $eq: "uber_2021.pdf"
6    },
7    search_limit: 100
8    },
9)

Combine traditional keyword-based search with vector search:

1const hybrid_search_response = client.search(
2    "What was Uber's profit in 2020?",
3    { filters: {
4        $eq: "uber_2021.pdf"
5    },
6    search_limit: 100
7    },
8)
1hybrid_search_response = client.search(
2    "What was Uber's profit in 2020?",
3    { use_hybrid_search: true}
4)

Utilize knowledge graph capabilities to enhance search results:

1const kg_search_response = client.search(
2    "What is a fierce nerd?",
3    { use_kg_search: true },
4)

Retrieval-Augmented Generation (RAG)

Basic RAG

Generate a response using RAG:

1const ragResponse = await client.rag("What was Uber's profit in 2020?");
query
strRequired

The query for RAG.

vector_search_settings
Optional[Union[VectorSearchSettings, dict]]Defaults to None

Optional settings for vector search, either a dictionary, a VectorSearchSettings object, or None may be passed. If a dictionary is used, non-specified fields will use the server-side default.

kg_search_settings
Optional[Union[KGSearchSettings, dict]]Defaults to None

Optional settings for knowledge graph search, either a dictionary, a KGSearchSettings object, or None may be passed. If a dictionary or None is passed, then R2R will use server-side defaults for non-specified fields.

rag_generation_config
Optional[Union[GenerationConfig, dict]]Defaults to None

Optional configuration for LLM to use during RAG generation, including model selection and parameters. Will default to values specified in r2r.toml.

task_prompt_override
Optional[str]Defaults to None

Optional custom prompt to override the default task prompt.

include_title_if_available
Optional[bool]Defaults to True

Augment document chunks with their respective document titles?

RAG with custom search settings

Use hybrid search in RAG:

1const hybridRagResponse = await client.rag({
2  query: "Who is Jon Snow?",
3  use_hybrid_search: true
4});

RAG with custom completion LLM

Use a different LLM model for RAG:

1const customLLMRagResponse = await client.rag({
2  query: "What is R2R?",
3  rag_generation_config: {
4    model: "anthropic/claude-3-opus-20240229"
5  }
6});

Streaming RAG

Stream RAG responses for real-time applications:

1const streamResponse = await client.rag({
2  query: "Who was Aristotle?",
3  rag_generation_config: { stream: true }
4});
5
6if (streamResponse instanceof ReadableStream) {
7  const reader = streamResponse.getReader();
8  while (true) {
9    const { done, value } = await reader.read();
10    if (done) break;
11    console.log(new TextDecoder().decode(value));
12  }
13}

Advanced RAG Techniques

R2R supports advanced Retrieval-Augmented Generation (RAG) techniques that can be easily configured at runtime. These techniques include Hypothetical Document Embeddings (HyDE) and RAG-Fusion, which can significantly enhance the quality and relevance of retrieved information.

To use an advanced RAG technique, you can specify the search_strategy parameter in your vector search settings:

1const client = new R2RClient();
2
3// Using HyDE
4const hydeResponse = await client.rag(
5    "What are the main themes in Shakespeare's plays?",
6    {
7        vector_search_settings: {
8            search_strategy: "hyde",
9            search_limit: 10
10        }
11    }
12);
13
14// Using RAG-Fusion
15const ragFusionResponse = await client.rag(
16    "Explain the theory of relativity",
17    {
18        vector_search_settings: {
19            search_strategy: "rag_fusion",
20            search_limit: 20
21        }
22    }
23);

For a comprehensive guide on implementing and optimizing advanced RAG techniques in R2R, including HyDE and RAG-Fusion, please refer to our Advanced RAG Cookbook.

Customizing RAG

Putting everything together for highly custom RAG functionality:

1const customRagResponse = await client.rag({
2  query: "Who was Aristotle?",
3  use_vector_search: true,
4  use_hybrid_search: true,
5  use_kg_search: true,
6  kg_generation_config: {},
7  rag_generation_config: {
8    model: "anthropic/claude-3-haiku-20240307",
9    temperature: 0.7,
10    stream: true
11  }
12});

Agents

Multi-turn agentic RAG

The R2R application includes agents which come equipped with a search tool, enabling them to perform RAG. Using the R2R Agent for multi-turn conversations:

1const messages = [
2  { role: "user", content: "What was Aristotle's main contribution to philosophy?" },
3  { role: "assistant", content: "Aristotle made numerous significant contributions to philosophy, but one of his main contributions was in the field of logic and reasoning. He developed a system of formal logic, which is considered the first comprehensive system of its kind in Western philosophy. This system, often referred to as Aristotelian logic or term logic, provided a framework for deductive reasoning and laid the groundwork for scientific thinking." },
4  { role: "user", content: "Can you elaborate on how this influenced later thinkers?" }
5];
6
7const agentResponse = await client.agent({
8  messages,
9  use_vector_search: true,
10  use_hybrid_search: true
11});

Note that any of the customization seen in AI powered search and RAG documentation above can be applied here.

messages
Array<Message>Required

The array of messages to pass to the RAG agent.

use_vector_search
booleanDefaults to

Whether to use vector search.

filters
object

Optional filters for the search.

search_limit
numberDefaults to 10

The maximum number of search results to return.

use_hybrid_search
boolean

Whether to perform a hybrid search (combining vector and keyword search).

use_kg_search
boolean

Whether to use knowledge graph search.

generation_config
object

Optional configuration for knowledge graph search generation.

rag_generation_config
GenerationConfig

Optional configuration for RAG generation, including model selection and parameters.

task_prompt_override
string

Optional custom prompt to override the default task prompt.

include_title_if_available
booleanDefaults to

Whether to include document titles in the context if available.

Multi-turn agentic RAG with streaming

The response from the RAG agent may be streamed directly back:

1const messages = [
2  { role: "user", content: "What was Aristotle's main contribution to philosophy?" },
3  { role: "assistant", content: "Aristotle made numerous significant contributions to philosophy, but one of his main contributions was in the field of logic and reasoning. He developed a system of formal logic, which is considered the first comprehensive system of its kind in Western philosophy. This system, often referred to as Aristotelian logic or term logic, provided a framework for deductive reasoning and laid the groundwork for scientific thinking." },
4  { role: "user", content: "Can you elaborate on how this influenced later thinkers?" }
5];
6
7const agentResponse = await client.agent({
8  messages,
9  use_vector_search: true,
10  use_hybrid_search: true,
11  rag_generation_config: { stream: true }
12});
13
14if (agentResponse instanceof ReadableStream) {
15  const reader = agentResponse.getReader();
16  while (true) {
17    const { done, value } = await reader.read();
18    if (done) break;
19    console.log(new TextDecoder().decode(value));
20  }
21}