Knowledge Graph

​

Introduction

R2R’s KGProvider handles the creation, management, and querying of knowledge graphs in your applications. This guide offers an in-depth look at the system’s architecture, configuration options, and best practices for implementation.

For a practical, step-by-step guide on implementing knowledge graphs in R2R, including code examples and common use cases, see our GraphRAG Cookbook.

​

Providers

The default Neo4jKGProvider offers a complete knowledge graph solution.

Key features:

• Neo4j-based graph database
• Entity and relationship extraction from text
• Graph querying and traversal
• Integration with vector search

​

Configuration

​

Knowledge Graph Configuration

These are located in the r2r.toml file, under the [kg] section.

[kg]
provider = "neo4j"
batch_size = 256
kg_extraction_prompt = "graphrag_triplet_extraction_zero_shot"
user = "your_neo4j_user"
password = "your_neo4j_password"
url = "your_neo4j_url"
database = "your_neo4j_database"

  [kg.kg_creation_settings]
    entity_types = ["Person", "Organization", "Location"] # if empty, all entities are extracted
    relation_types = ["works at", "founded by", "invested in"] # if empty, all relations are extracted
    max_knowledge_triples = 100
    fragment_merge_count = 4 # number of fragments to merge into a single extraction
    generation_config = { model = "gpt-4o-mini" } # and other params, model used for triplet extraction

  [kg.kg_enrichment_settings]
    max_description_input_length = 65536 # increase if you want more comprehensive descriptions
    max_summary_input_length = 65536 # increase if you want more comprehensive summaries
    generation_config = { model = "gpt-4o-mini" } # and other params, model used for node description and graph clustering
    leiden_params = { max_levels = 10 } # more params here: https://neo4j.com/docs/graph-data-science/current/algorithms/leiden/

Alternatively, instead of setting user, password, url, and database in your r2r.toml, you can set these values using environment variables:

export NEO4J_USER=your_username
export NEO4J_PASSWORD=your_password
export NEO4J_URL=your_neo4j_url
export NEO4J_DATABASE=neo4j

Environment variables take precedence over the config settings in case of conflicts. The R2R Docker includes configuration options that facilitate integration with a combined Postgres+pgvector database setup.

​

Implementation Guide

​

File Ingestion and Graph Construction

from r2r import R2RClient

client = R2RClient("http://localhost:7272")

result = client.ingest_files(["path/to/your/file.txt"])

# following will create a graph on all ingested files
document_ids = [] # add document ids that you want to create a graph on
creation_result = client.create_graph(document_ids)
print(f"Creation Result: {creation_result}")
# wait for the creation to complete

enrichment_result = client.enrich_graph() # enrichment will run on all nodes in the graph
print(f"Enrichment Result: {enrichment_result}")
# wait for the enrichment to complete

​

Graph-based Search

There are two types of graph-based search: local and global.

• local search is faster and more accurate, but it is not as comprehensive as global search.
• global search is slower and more comprehensive, but it will give you the most relevant results. Note that global search may perform a large number of LLM calls.

search_result = client.search(
    query="Find founders who worked at Google",
    kg_search_settings={"use_kg_search":True, "kg_search_type": "local"}
)
print(f"Search Result: {search_result}")

​

Retrieval-Augmented Generation

rag_result = client.rag(
    query="Summarize the achievements of founders who worked at Google",
    kg_search_settings={"use_kg_search":True, "kg_search_type": "local"}
)
print(f"RAG Result: {rag_result}")

​

Best Practices

1. Optimize Chunk Size: Adjust the chunk_size based on your data and model capabilities.
2. Use Domain-Specific Entity Types and Relations: Customize these for more accurate graph construction.
3. Balance Batch Size: Adjust batch_size for optimal performance and resource usage.
4. Implement Caching: Cache frequently accessed graph data for improved performance.
5. Regular Graph Maintenance: Periodically clean and optimize your knowledge graph.

​

Advanced Topics

​

Custom Knowledge Graph Providers

Extend the KGProvider class to implement custom knowledge graph providers:

from r2r.base import KGProvider, KGConfig

class CustomKGProvider(KGProvider):
    def __init__(self, config: KGConfig):
        super().__init__(config)
        # Custom initialization...

    def ingest_files(self, file_paths: List[str]):
        # Custom implementation...

    def search(self, query: str, use_kg_search: bool = True):
        # Custom implementation...

    # Implement other required methods...

​

Integrating External Graph Databases

To integrate with external graph databases:

1. Implement a custom KGProvider.
2. Handle data synchronization between R2R and the external database.
3. Implement custom querying methods to leverage the external database’s features.

​

Scaling Knowledge Graphs

For large-scale applications:

1. Implement graph partitioning for distributed storage and processing.
2. Use graph-specific indexing techniques for faster querying.
3. Consider using a graph computing framework for complex analytics.

​

Troubleshooting

Common issues and solutions:

1. Ingestion Errors: Check file formats and encoding.
2. Query Performance: Optimize graph structure and use appropriate indexes.
3. Memory Issues: Adjust batch sizes and implement pagination for large graphs.

​

Performance Considerations

1. Query Optimization: Use efficient Cypher queries for Neo4j.
2. Embedding Caching: Cache embeddings for frequently accessed entities.
3. Asynchronous Processing: Use async methods for non-blocking operations.

​

Conclusion

R2R’s Knowledge Graph system provides a powerful foundation for building applications that require structured data representation and complex querying capabilities. By understanding its components, following best practices, and leveraging its flexibility, you can create sophisticated information retrieval and analysis systems tailored to your specific needs.

For further customization and advanced use cases, refer to the R2R API Documentation and the GraphRAG Cookbook.