Data migration and synchronization to SurrealDB from MongoDB, PostgreSQL, MySQL, Neo4j, Kafka, and JSONL. Full and incremental CDC sync. Part of the surreal-skills collection.
Surreal-Sync is a CLI tool for migrating data from various database sources to SurrealDB with full and incremental synchronization via Change Data Capture (CDC).
| Source | Full Sync | Incremental CDC | Method |
|---|---|---|---|
| MongoDB | Yes | Yes | Change streams |
| MySQL | Yes | Yes | Trigger-based CDC + sequence checkpoints |
| PostgreSQL (triggers) | Yes | Yes | Trigger-based CDC + sequence checkpoints |
| PostgreSQL (wal2json) | Yes | Yes | Logical replication with wal2json plugin |
| Neo4j | Yes | Yes | Timestamp-based tracking |
| JSONL Files | Yes | N/A |
| Batch import from JSON Lines |
| Apache Kafka | Yes | Yes | Consumer subscriptions with deduplication |
# Install surreal-sync (Rust binary)
cargo install surreal-sync
# Full sync from PostgreSQL (trigger-based)
surreal-sync from postgres trigger-full \
--connection-string "postgresql://user:pass@localhost/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Incremental CDC from PostgreSQL (wal2json)
surreal-sync from postgres wal2json \
--connection-string "postgresql://user:pass@localhost/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Full sync from MongoDB
surreal-sync from mongo full \
--connection-string "mongodb://localhost:27017/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Batch import from JSONL
surreal-sync from jsonl import \
--file data.jsonl \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Consume from Kafka
surreal-sync from kafka consume \
--bootstrap-servers "localhost:9092" \
--topic my-events \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
surreal-sync from <SOURCE> <COMMAND> \
--connection-string [CONNECTION STRING] \
--surreal-endpoint [SURREAL ENDPOINT] \
--surreal-username [SURREAL USERNAME] \
--surreal-password [SURREAL PASSWORD] \
--to-namespace <NS> \
--to-database <DB>
See the main skill's rule file for complete guidance: