Embedding

Configuring Embeddings

Configure the embedding provider in your TOML file:

LiteLLM

OpenAI

Ollama

1 [embedding]
2 provider = "litellm"
3 base_model = "openai/text-embedding-3-small"
4 base_dimension = 512
5 concurrent_request_limit = 256
6 max_retries = 3
7 initial_backoff = 1.0
8 max_backoff = 64.0
9 
10 [completion_embedding]
11 provider = "litellm"
12 base_model = "openai/text-embedding-3-small"
13 base_dimension = 512
14 concurrent_request_limit = 256

provider

stringRequired

The embedding provider to use. Either litellm, openai, or ollama.

base_model

stringRequired

The model to request embeddings from.

base_dimension

numberRequired

The number of dimensions of your embedding. For models that do not support passing a dimension parameter, set nan.

batch_size

number

Controls how many text inputs are processed together in a single API call to the Ollama embedding service.

concurrent_request_limit

number

The number of concurrent requests to be made at a time.

max_retries

number

The number of times to retry getting an embedding, should a request fail.

initial_backoff

number

The initial backoff time applied to a failed embedding.

max_backoff

number

The largest possible backoff time that can be applied to a failed embedding.

R2R uses embeddings as the foundation for semantic search and similarity matching capabilities. The embedding system is responsible for converting text into high-dimensional vectors that capture semantic meaning, enabling powerful search and retrieval operations.

Embedding Configuration

The embedding settings are configurable through the embedding section in your TOML configuration file. Learn more about working with R2R config files.

Environment Variables

Provider dependent environment variables must be set. These may include:

1 OPENAI_API_KEY=…
2 AZURE_API_KEY=…
3 GEMINI_API_KEY=…

LiteLLM

LiteLLM offers a Python SDK to call 100+ LLM APIs in OpenAI format

OpenAI

The OpenAI embedding provider makes direct use of the OpenAI Python SDK.

Ollama

The Ollama embedding provider makes direct use of the Ollama Python SDK.