Introduction

This guide extends the R2R Quickstart by demonstrating how R2R supports multiple large language models (LLMs). Multi-LLM support in R2R allows for a diverse and comprehensive approach to search and retrieval tasks.

LLMs are selected at runtime for maximum flexibility and ease of use. R2R supports any provider supported by LiteLLM along with several other dedicated internal implementations.

Using Different LLM Providers

If you haven’t completed the quickstart or if your target database is empty, start by ingesting sample files:

# export OPENAI_API_KEY=...
python -m r2r.examples.quickstart ingest_as_files  --no-media=true

Now we are ready to test RAG with different LLM providers.

Summary

This guide demonstrates R2R’s flexibility in using multiple LLMs. By leveraging different models from providers like OpenAI, Anthropic, and local options like Ollama, you have full control over how to serve user responses. This allows you to optimize for performance, cost, or specific use case requirements in your RAG applications.

For more information on R2R’s capabilities, visit the R2R GitHub repository or join the R2R Discord community.