Structured Output

Enable JSON Outputs

Overview

Structured outputs allow users to ensure that the retrieval response generated by the LLM follows a user-defined structure. This provides reliable type-safety, making it easier to generate high-quality, production-ready applications.

R2R supports passing Pydantic models via our Python SDK.

With this, you can:

  • Define the exact structure you expect for responses
  • Automatically validate that responses match your schema
  • Access response fields with proper typing and autocompletion
  • Handle errors gracefully when responses don’t match expectations

Using Structured Outputs with R2R

The example below demonstrates how to define a simple Pydantic model that specifies the expected structure for responses to a query about Hopfield Networks. The model includes fields for the main answer, a confidence score, additional comments, and even a related joke.

1from r2r import R2RClient, GenerationConfig
2from pydantic import BaseModel
3import json
4
5# Define a response model
6class ResponseModel(BaseModel):
7 answer: str
8 confidence: float
9 comments: str
10 related_joke: str
11
12rag_response = client.retrieval.rag(
13 query="What is a Hopfield Network?",
14 rag_generation_config=GenerationConfig(
15 response_format=ResponseModel
16 )
17)

Processing the Response

Once you’ve received a response, you can parse it as JSON and validate it against your Pydantic model. This ensures that the response contains all required fields with the correct data types.

1content = json.loads(rag_response.results.completion)
2print(json.dumps(content, indent=2))
3
4response_obj = ResponseModel.model_validate(content)
5print("\nAs a Pydantic object:")
6print(f"Confidence: {response_obj.confidence}")
7print(f"Comments: {response_obj.comments}")
8print(f"Related Joke: {response_obj.related_joke}")
9print("\nDetailed Answer:")
10print(response_obj.answer)

Example Output

Here’s what the output looks like when running the code above:

1{
2 "answer": "A Hopfield Network is a type of recurrent neural network introduced by John Hopfield in 1982, designed to function as an associative memory system. It consists of binary nodes with symmetric weights, and its dynamics are governed by an energy function that decreases over time, leading the network to stable states that represent stored memories [1], [2].",
3 "confidence": 0.95,
4 "comments": "The Hopfield Network is a foundational concept in neural networks, and its principles are widely studied in computational neuroscience and machine learning.",
5 "related_joke": "Why did the neural network go to therapy? It had too many weights to carry!"
6}
7
8As a Pydantic object:
9Confidence: 0.95
10Comments: The Hopfield Network is a foundational concept in neural networks, and its principles are widely studied in computational neuroscience and machine learning.
11Related Joke: Why did the neural network go to therapy? It had too many weights to carry!
12
13Detailed Answer:
14A Hopfield Network is a type of recurrent neural network introduced by John Hopfield in 1982, designed to function as an associative memory system. It consists of binary nodes with symmetric weights, and its dynamics are governed by an energy function that decreases over time, leading the network to stable states that represent stored memories [1], [2].