RAG
Learn how to configure RAG in your R2R deployment
RAG Customization
RAG (Retrieval-Augmented Generation) in R2R can be extensively customized to suit various use cases. The main components for customization are:
- Generation Configuration: Control the language model’s behavior.
- Search Settings: Fine-tune the retrieval process.
- Task Prompt Override: Customize the system prompt for specific tasks.
LLM Provider Configuration
Refer to the LLM configuration page here.
Retrieval Configuration
Refer to the retrieval configuration page here.
Combining LLM and Retrieval Configuration for RAG
The rag_generation_config parameter allows you to customize the language model’s behavior. Default settings are set on the server-side using the r2r.toml, as described in in previous configuraiton guides. These settings can be overridden at runtime as shown below:
# Configure knowledge graph search
vector_search_settings = {
"use_vector_search": True,
"filters": {"metadata.document_type": "article"},
"search_limit": 20,
"use_hybrid_search": True,
"selected_collection_ids": ["c3291abf-8a4e-5d9d-80fd-232ef6fd8526"]
}
# Configure graphRAG search
kg_search_settings = {
"use_kg_search": True,
"kg_search_type": "global",
"kg_search_level": None,
"kg_search_generation_config": {
"model": "gpt-4",
"temperature": 0.1
},
"entity_types": ["Person", "Organization"],
"relationships": ["worksFor", "foundedBy"],
"max_community_description_length": 65536,
"max_llm_queries_for_global_search": 250,
"local_search_limits": {"__Entity__": 20, "__Relationship__": 20, "__Community__": 20}
}
# Configure LLM generation
rag_generation_config = {
"model": "anthropic/claude-3-opus-20240229",
"temperature": 0.7,
"top_p": 0.95,
"max_tokens_to_sample": 1500,
"stream": True,
"functions": None, # For function calling, if supported
"tools": None, # For tool use, if supported
"add_generation_kwargs": {}, # Additional provider-specific parameters
"api_base": None # Custom API endpoint, if needed
}
When performing a RAG query you can combine these vector search, knowledge graph search, and generation settings at runtime:
from r2r import R2RClient
client = R2RClient()
response = client.rag(
"What are the latest advancements in quantum computing?",
rag_generation_config=rag_generation_config
vector_search_settings=vector_search_settings,
kg_search_settings=kg_search_settings,
)
R2R defaults to the specified server-side settings when no runtime overrides are specified.
RAG Prompt Override
For specialized tasks, you can override the default RAG task prompt at runtime:
task_prompt_override = """You are an AI assistant specializing in quantum computing.
Your task is to provide a concise summary of the latest advancements in the field,
focusing on practical applications and breakthroughs from the past year."""
response = client.rag(
"What are the latest advancements in quantum computing?",
rag_generation_config=rag_generation_config,
task_prompt_override=task_prompt_override
)
This prompt can also be set statically on as part of the server configuration process.
Agent-based Interaction
R2R supports multi-turn conversations and complex query processing through its agent endpoint:
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "What are the key differences between quantum and classical computing?"}
]
response = client.agent(
messages=messages,
vector_search_settings=vector_search_settings,
kg_search_settings=kg_search_settings,
rag_generation_config=rag_generation_config,
)
The agent can break down complex queries into sub-tasks, leveraging both retrieval and generation capabilities to provide comprehensive responses. The settings specified in the example above will propagate to the agent and it’s tools.
By leveraging these configuration options, you can fine-tune R2R’s retrieval and generation process to best suit your specific use case and requirements.
Next Steps
For more detailed information on configuring specific components of R2R, please refer to the following pages: