LLMs

R2R supports multiple Language Model (LLM) providers, allowing users to easily switch between different models based on their requirements. This guide explains how to configure and use various LLM providers within the R2R framework.

​

Supported Providers

R2R currently supports the following LLM providers:

​

Configuring LLM Providers

To switch between LLM providers, update the completions section in your config.json file:

"completions": {
    "provider": "litellm"
}

Change the provider value to "openai" or "litellm" to switch providers.

​

Provider Details

​

LiteLLM Provider (Default)

The LiteLLM class is the default implementation that integrates with various LLM providers through the LiteLLM library.

To use LiteLLM with a specific provider, set the appropriate environment variables:

export OPENAI_API_KEY=your_openai_api_key

export ANTHROPIC_API_KEY=your_anthropic_api_key

export TOGETHERAI_API_KEY=your_togetherai_api_key

For detailed setup instructions for each provider, refer to the LiteLLM documentation.

​

OpenAI Provider

The OpenAILLM class integrates directly with the OpenAI API.

To use the OpenAI provider:

export OPENAI_API_KEY=your_openai_api_key

​

Configuring Remote LLM Providers

To configure a specific LLM provider:

1. Set the appropriate environment variables as described above.
2. Update the completions section in your config.json file.

Example configuration for OpenAI:

"completions": {
    "provider": "openai",
    "model": "gpt-4"
}

​

Conclusion

By following this guide, you can easily configure and switch between different LLM providers in R2R. This flexibility allows you to choose the best LLM for your specific use case while maintaining a consistent interface within your application.

For more information on customizing R2R, refer to the Customizing R2R documentation.