Skip to content

Latest commit

 

History

14 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Graph Database Cloud Benchmark

Reproducible benchmarking of CognoDB Cloud and comparable graph database platforms using a common dataset, equivalent logical workloads, deterministic query selection, and a controlled resource envelope.

Status

Benchmark infrastructure complete.

The current benchmark supports:

  • CognoDB
  • Neo4j
  • Memgraph
  • FalkorDB
  • ArcadeDB

The repository contains the benchmark execution framework, database adapters, workload registry, deterministic query generation, run metadata, result serialization, comparison/scoring infrastructure, and benchmark documentation.


1. Project Objective

The goal of this project is to compare graph database performance under controlled and reproducible conditions.

The benchmark focuses on two primary areas:

  1. Data ingestion performance
  2. Graph query execution performance

The comparison uses the same canonical dataset and equivalent logical workloads across all supported databases.

The results are intended to provide a transparent performance comparison for the documented environment and configuration.

They should not be interpreted as a universal ranking of graph databases.


2. Supported Databases

Database Deployment
CognoDB Cloud
Neo4j Docker
Memgraph Docker
FalkorDB Docker
ArcadeDB Docker

Database versions and deployment details are documented in:

docs/benchmark-methodology.md

3. Benchmark Environment

The benchmark targets a common resource envelope wherever the deployment model permits:

Resource Target
CPU 0.5 vCPU
Memory 512 MB
Storage 1 GB

CognoDB currently uses the available c0 configuration documented by the project methodology.

Self-hosted databases are constrained to the same target resource envelope where supported by their deployment configuration.

Database-specific configuration required to operate within the envelope is documented rather than silently relying on unrestricted defaults.

See:

docs/benchmark-methodology.md

4. Dataset

All databases use the same canonical processed dataset.

Current dataset size:

Nodes:          36,692
Relationships: 183,831

The benchmark consumes:

data/processed/nodes.csv
data/processed/relationships.csv

The benchmark discovers the actual dataset row counts before execution and uses those counts as the run metadata.

The benchmark configuration also validates that the ingested database contains the expected node and relationship counts.


5. Workload Suite

The benchmark currently evaluates seven logical workloads:

Workload Purpose
point_lookup Lookup one user by ID
indexed_lookup Exercise indexed user lookup
relationship_lookup Retrieve direct KNOWS neighbors
traversal_1_hop Traverse exactly one KNOWS hop
traversal_2_hop Traverse exactly two KNOWS hops
traversal_3_hop Traverse exactly three KNOWS hops
aggregation Count all benchmark users

The query syntax may differ between databases, but the logical operation is kept equivalent.

Traversal workloads use a result limit of:

1000

This prevents high-degree graph traversals from producing uncontrolled result sets and keeps the workload operationally comparable.

Detailed workload definitions are documented in:

docs/workload-reference.md

6. Deterministic Query Selection

Parameterized workloads use dynamically generated node IDs.

The benchmark uses:

query_seed = 42

The same seed produces the same node-ID sequence.

This makes query selection reproducible across benchmark runs and databases.

For example, the sequence begins with:

7296
1639
18024
16049
14628
9144
6717
35741
5697
27651

This prevents random query selection from becoming an uncontrolled variable in the comparison.


7. Benchmark Execution

Requirements

The project requires Python and the dependencies used by the benchmark adapters.

Create and activate a virtual environment, then install the required project dependencies appropriate to the current development environment.

Database connection information is provided through environment variables.

Copy:

.env.example

to:

.env

and configure the connection details for the databases you intend to benchmark.

The .env file is ignored by Git.


8. Environment Configuration

The supported environment variable groups are:

CognoDB

COGNODB_URI
COGNODB_USERNAME
COGNODB_PASSWORD

Neo4j

NEO4J_URI
NEO4J_USERNAME
NEO4J_PASSWORD

Memgraph

MEMGRAPH_URI

FalkorDB

FALKORDB_HOST
FALKORDB_PORT

ArcadeDB

ARCADEDB_HOST
ARCADEDB_PORT
ARCADEDB_USERNAME
ARCADEDB_PASSWORD
ARCADEDB_DATABASE

The exact values depend on the deployment being benchmarked.

Do not commit credentials or other secrets.


9. Running a Benchmark

The benchmark CLI is implemented in:

benchmark/cli.py

Benchmark one database

Example:

python -m benchmark.cli --database MEMGRAPH

Specify measurement iterations

python -m benchmark.cli --database MEMGRAPH --iterations 100

Specify warmup iterations

python -m benchmark.cli --database MEMGRAPH --iterations 100 --warmup 10

Benchmark all supported databases

python -m benchmark.cli --all

The CLI does not allow --database and --all to be used together.


10. Benchmark Execution Flow

A benchmark run follows this general flow:

CLI
 |
 v
Load benchmark configuration
 |
 v
Discover dataset counts
 |
 v
Create run metadata
 |
 v
Create results/runs/<run_id>
 |
 v
Select database adapter
 |
 v
Select database workloads
 |
 v
Connect to database
 |
 v
Ingest canonical dataset
 |
 v
Validate node/relationship counts
 |
 v
Execute warmup queries
 |
 v
Execute measured queries
 |
 v
Calculate latency + QPS statistics
 |
 v
Clear benchmark data
 |
 v
Close database connection
 |
 v
Serialize results

This flow is implemented primarily through:

benchmark/cli.py
benchmark/runner.py
benchmark/query.py
benchmark/ingestion.py
benchmark/workload_registry.py
benchmark/database_registry.py

11. Query Measurement

Warmup executions are excluded from measured latency statistics.

Measured executions record individual query latency values.

The benchmark calculates:

  • minimum latency
  • mean latency
  • P50
  • P95
  • P99
  • maximum latency
  • QPS

P95 is the primary latency metric used by the scoring system because it represents tail behavior better than the arithmetic mean.


12. Run Metadata

Each run receives a unique run ID.

Results are stored under:

results/runs/<run_id>/

A run records metadata such as:

run_id
timestamp_utc
git_commit
python_version
platform
processor
node_count
relationship_count
ingestion_batch_size
query_iterations
query_warmup_iterations
query_seed

This allows benchmark results to be associated with the exact source revision and execution configuration that produced them.


13. Result Artifacts

A completed run produces database-specific result files under its run directory.

Typical structure:

results/
└── runs/
    └── <run_id>/
        ├── metadata.json
        ├── neo4j.json
        ├── memgraph.json
        ├── falkordb.json
        ├── arcadedb.json
        └── cognodb.json

Additional aggregate artifacts can include:

results/comparison.json
results/scores.json

Generated benchmark results are treated as artifacts of a benchmark execution rather than source code.


14. Result Metrics

Ingestion

The benchmark records:

elapsed_seconds
nodes_per_second
relationships_per_second

For ingestion:

  • lower elapsed time is better
  • higher nodes/sec is better
  • higher relationships/sec is better

Queries

Each workload records:

min
mean
p50
p95
p99
max
qps

For query performance:

  • lower latency is better
  • higher QPS is better

15. Scoring

The scoring implementation is:

benchmark/scoring.py

The scoring methodology uses best-value normalization.

For lower-is-better metrics:

score = best_value / database_value

For higher-is-better metrics:

score = database_value / best_value

Therefore, the best database for a metric receives:

1.0

Overall weighting

The overall score uses:

20% ingestion
80% query performance

Each query workload contributes equally to the query-performance component.

Each workload is composed of:

50% P95 latency
50% QPS

Conceptually:

workload score
    = 50% normalized P95 score
    + 50% normalized QPS score

query score
    = average of workload scores

overall score
    = 20% ingestion score
    + 80% query score

The complete methodology is documented in:

docs/results-and-scoring.md

16. Running the Scoring Pipeline

After comparison data has been generated, the scoring module can be executed with:

python -m benchmark.scoring

The scorer reads:

results/comparison.json

and writes:

results/scores.json

The console output displays the normalized ranking and per-workload scores.


17. Database Adapter Architecture

All databases implement the common interface:

databases/base.py

The GraphDatabaseAdapter abstraction provides operations for:

connect()
close()
execute()
clear()
health_check()
load_nodes()
load_relationships()
count_nodes()
count_relationships()
name

Database-specific implementations are located under:

databases/
├── arcadedb.py
├── base.py
├── cognodb.py
├── falkordb.py
├── memgraph.py
└── neo4j.py

The benchmark runner operates against this common adapter interface rather than directly depending on an individual database implementation.


18. Workload Architecture

Workloads are represented by:

benchmark/workload.py

Database-specific workload registrations are maintained through:

benchmark/workload_registry.py

The workload implementation files are:

benchmark/
├── workloads.py
├── neo4j_workloads.py
├── memgraph_workloads.py
├── arcadedb_workloads.py
└── cognodb_workloads.py

FalkorDB uses the shared workload definitions in:

benchmark/workloads.py

The registry maps each supported database to its appropriate workload tuple.


19. Project Structure

The important project directories are:

graph-db-cloud-benchmark/
│
├── benchmark/
│   ├── cli.py
│   ├── config.py
│   ├── database_registry.py
│   ├── dataset.py
│   ├── ingestion.py
│   ├── query.py
│   ├── results.py
│   ├── runner.py
│   ├── scoring.py
│   ├── serialization.py
│   ├── workload.py
│   ├── workload_registry.py
│   ├── workloads.py
│   └── *_workloads.py
│
├── databases/
│   ├── base.py
│   ├── neo4j.py
│   ├── memgraph.py
│   ├── falkordb.py
│   ├── arcadedb.py
│   └── cognodb.py
│
├── config/
│   └── settings.py
│
├── data/
│   └── processed/
│       ├── nodes.csv
│       └── relationships.csv
│
├── docs/
│   ├── benchmark-methodology.md
│   ├── architecture.md
│   ├── workload-reference.md
│   └── results-and-scoring.md
│
├── results/
│   └── runs/
│
├── tests/
│
├── .env.example
├── .gitignore
└── README.md

Other project directories contain supporting dataset, infrastructure, script, and development resources.


20. Reproducibility

A benchmark run should preserve the conditions under which it was produced.

Important reproducibility inputs include:

  • database versions
  • deployment model
  • CPU allocation
  • memory allocation
  • storage allocation
  • canonical dataset
  • workload definitions
  • query seed
  • query iterations
  • warmup iterations
  • ingestion batch size
  • benchmark source revision

The benchmark automatically records several of these values in run metadata.

For a controlled comparison, all databases should be executed using the same benchmark configuration and dataset.


21. Benchmark Fairness

The benchmark is designed around controlled comparison rather than unrestricted database defaults.

The comparison attempts to hold constant:

Dataset
Resource envelope
Logical workloads
Query sequence
Warmup methodology
Measurement methodology

Database-specific query syntax and configuration are allowed where necessary to represent equivalent operations or successfully operate within the target resource envelope.

Such differences should be documented rather than hidden.


22. Limitations

This benchmark does not attempt to measure every property of a graph database.

It focuses on the selected:

  • dataset
  • resource envelope
  • database versions
  • ingestion workload
  • query workload suite
  • execution configuration

It does not by itself establish:

  • universal database superiority
  • production reliability
  • operational cost
  • ecosystem quality
  • developer experience
  • feature completeness
  • distributed scaling behavior
  • failure recovery behavior
  • security posture
  • long-running production workload behavior

The final ranking is therefore specific to the documented benchmark conditions.


23. Documentation

Detailed documentation is available under docs/:

Benchmark Methodology

docs/benchmark-methodology.md

Documents:

  • objective
  • resource envelope
  • database versions
  • deployment model
  • dataset
  • fairness
  • reproducibility
  • limitations

Architecture

docs/architecture.md

Documents:

  • benchmark architecture
  • execution flow
  • adapter abstraction
  • workload architecture
  • result pipeline

Workload Reference

docs/workload-reference.md

Documents:

  • workload definitions
  • workload semantics
  • deterministic query selection
  • traversal limits
  • warmup and measurement behavior
  • database-specific query implementations

Results and Scoring

docs/results-and-scoring.md

Documents:

  • result artifacts
  • benchmark metrics
  • normalization
  • scoring
  • weighting
  • ranking
  • interpretation
  • limitations

24. Development Verification

The benchmark source can be syntax-checked with:

python -m compileall benchmark databases config

The database registry can be verified with:

python -c "from benchmark.database_registry import create_adapter; print([create_adapter(name).name for name in ['NEO4J','MEMGRAPH','FALKORDB','ARCADEDB','COGNODB']])"

The workload registry can be verified with:

python -c "from benchmark.workload_registry import get_workloads; [print(name, [w.name for w in get_workloads(name)]) for name in ['NEO4J','MEMGRAPH','FALKORDB','ARCADEDB','COGNODB']]"

These checks verify that the registry and workload modules can be imported and that all supported databases have registered workloads.


25. Git and Secrets

The repository ignores:

.env
.venv/
__pycache__/

Database credentials must remain in local environment configuration and must not be committed.

The example configuration is provided through:

.env.example

The .env.example file contains configuration placeholders and local development defaults where applicable, not production credentials.


26. Benchmark Philosophy

The benchmark follows four principles:

Reproducibility

The same dataset, deterministic query sequence, and documented configuration should produce comparable runs.

Fairness

Databases should be compared under a controlled resource envelope and equivalent logical operations.

Transparency

Database versions, configuration assumptions, workload semantics, measurements, and scoring methodology should be documented.

Separation of concerns

Database-specific behavior belongs in adapters and workload definitions.

Benchmark orchestration, measurement, statistics, serialization, and scoring remain independent of individual database implementations.


27. End-to-End Benchmark Model

The complete system can be summarized as:

Canonical Dataset
       |
       v
Database Adapter
       |
       v
Ingestion
       |
       v
Dataset Validation
       |
       v
Deterministic Workload Selection
       |
       v
Warmup
       |
       v
Measured Query Execution
       |
       v
Latency + QPS Statistics
       |
       v
Run Result Serialization
       |
       v
Cross-Database Comparison
       |
       v
Normalized Scoring
       |
       v
Benchmark Ranking

The resulting benchmark artifacts preserve the measurements and metadata needed to understand how the final comparison was produced.


28. Final Interpretation

The benchmark should be used as a controlled performance study of the selected graph databases under the documented conditions.

A database ranking is meaningful only together with:

  • the tested database version
  • deployment model
  • resource envelope
  • dataset
  • workload suite
  • query configuration
  • scoring methodology

The project therefore treats reproducibility and transparency as part of the benchmark itself, rather than presenting a ranking without its experimental context.

About

Reproducible benchmarking of CognoDB Cloud and comparable graph database engines using identical datasets, workloads, and controlled resources.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages