​
R2R Service Configuration Troubleshooting Guide: Postgres, Neo4j, Hatchet

This guide addresses common configuration problems for Postgres, Neo4j, and Hatchet services in R2R deployments.

​
Postgres Configuration Issues

​
1. Connection Failures

Symptom: R2R cannot connect to Postgres database.

Possible Causes and Solutions:

a) Incorrect connection string:

  • Verify the DATABASE_URL environment variable.
  • Ensure it follows the format: postgres://user:password@host:port/dbname

b) Network issues:

  • Check if Postgres is running and accessible from the R2R container.
  • Verify network settings in Docker Compose file.

c) Authentication problems:

  • Confirm that the username and password in the connection string are correct.
  • Check Postgres logs for failed authentication attempts.

​
2. pgvector Extension Missing

Symptom: Vector operations fail or R2R reports missing pgvector functionality.

Solution:

  • Ensure you’re using the pgvector/pgvector image instead of the standard Postgres image.
  • Verify that the pgvector extension is created in your database: 
    CREATE EXTENSION IF NOT EXISTS vector;

​
3. Insufficient Resources

Symptom: Postgres performance issues or crashes under load.

Solution:

  • Adjust resource limits in Docker Compose: 
    postgres:
  deploy:
    resources:
      limits:
        cpus: '2'
        memory: 4G
  • Tune Postgres configuration parameters like shared_buffers, effective_cache_size, etc.

​
Neo4j Configuration Issues

​
1. Authentication Failures

Symptom: R2R cannot authenticate with Neo4j.

Solution:

  • Verify the NEO4J_AUTH environment variable: 
    environment:
  - NEO4J_AUTH=neo4j/yourpassword
  • Ensure R2R is using the correct credentials in its configuration.

​
2. Memory Allocation Issues

Symptom: Neo4j fails to start or crashes with out-of-memory errors.

Solution:

  • Adjust memory settings: 
    environment:
  - NEO4J_server_memory_pagecache_size=1G
  - NEO4J_server_memory_heap_initial__size=1G
  - NEO4J_server_memory_heap_max__size=1G
  • Ensure your host system has enough available memory.

​
3. Plugin Issues

Symptom: Required Neo4j plugins are not loading or functioning.

Solution:

  • Verify plugin configuration: 
    environment:
  - NEO4J_PLUGINS=["apoc", "graph-data-science"]
  - NEO4J_dbms_security_procedures_unrestricted=apoc.*,gds.*
  • Check Neo4j logs for plugin-related errors.
  • Ensure plugin compatibility with your Neo4j version.

​
Hatchet Configuration Issues

​
1. RabbitMQ Connection Problems

Symptom: Hatchet cannot connect to RabbitMQ.

Solution:

  • Verify RabbitMQ connection string: 
    environment:
  - SERVER_TASKQUEUE_RABBITMQ_URL=amqp://user:password@hatchet-rabbitmq:5672/
  • Ensure RabbitMQ service is healthy and accessible.

​
2. Hatchet API Key Issues

Symptom: R2R cannot authenticate with Hatchet.

Solution:

  • Check the Hatchet API key generation process: 
    setup-token:
  command: >
    sh -c "
      TOKEN=$(/hatchet/hatchet-admin token create --config /hatchet/config --tenant-id your-tenant-id)
      echo $TOKEN > /hatchet_api_key/api_key.txt
    "
  • Verify that R2R is correctly reading the API key: 
    r2r:
  environment:
    - HATCHET_CLIENT_TOKEN=${HATCHET_CLIENT_TOKEN}
  command: >
    sh -c '
      export HATCHET_CLIENT_TOKEN=$(cat /hatchet_api_key/api_key.txt)
      exec your_r2r_command
    '

​
3. Hatchet Engine Health Check Failures

Symptom: Hatchet Engine service fails health checks.

Solution:

  • Verify Hatchet Engine configuration: 
    hatchet-engine:
  environment:
    - SERVER_GRPC_BROADCAST_ADDRESS=host.docker.internal:7077
    - SERVER_GRPC_BIND_ADDRESS=0.0.0.0
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:7077/health"]
    interval: 10s
    timeout: 5s
    retries: 5
  • Check Hatchet Engine logs for startup errors.
  • Ensure all required environment variables are set correctly.

​
General Troubleshooting Tips

  1. Check Logs: Always start by examining the logs of the problematic service:

    docker-compose logs postgres
docker-compose logs neo4j
docker-compose logs hatchet-engine

  2. Verify Network Connectivity: Ensure services can communicate:

    docker-compose exec r2r ping postgres
docker-compose exec r2r ping neo4j
docker-compose exec r2r ping hatchet-engine

  3. Check Resource Usage: Monitor CPU and memory usage:

    docker stats

  4. Recreate Containers: Sometimes, recreating containers can resolve issues:

    docker-compose up -d --force-recreate <service-name>

  5. Verify Volumes: Ensure data persistence by checking volume mounts:

    docker volume ls
docker volume inspect <volume-name>

  6. Environment Variables: Double-check all environment variables in your .env file and docker-compose.yml.

By following this guide, you should be able to diagnose and resolve most configuration issues related to Postgres, Neo4j, and Hatchet in your R2R deployment. If problems persist, consider seeking help from the R2R community or support channels.

Was this page helpful?

DependenciesUnstructured