Getting started
This walks you from zero to a running replication in a few minutes, then explains how to point replicare at your own databases.
What replicare does
You give replicare a source database and one or more target databases and a list of tables. It:
- installs lightweight capture (triggers) on the source,
- copies the selected data to the targets (chunked and parallel), then
- streams every insert/update/delete continuously to keep the targets converged.
No WAL access, no REPLICATION role, no wal_level change — just ordinary,
grantable privileges. Architecture explains why.
Try the demo (2 minutes)
The fastest way to see it work end to end:
task docker # build the replicare image
cd examples/demo && docker compose up
This starts a source Postgres (seeded with public.orders), a target Postgres,
and replicare copying then streaming between them. In another shell:
# initial copy landed
docker compose exec target psql -U postgres -d warehouse -c "SELECT count(*) FROM public.orders;"
# make a live change on the source and watch it propagate
docker compose exec source psql -U postgres -d app \
-c "INSERT INTO public.orders VALUES (101,'live'); UPDATE public.orders SET note='changed' WHERE id=1; DELETE FROM public.orders WHERE id=2;"
docker compose exec target psql -U postgres -d warehouse -c "SELECT count(*), max(id) FROM public.orders;"
# operator surface
curl localhost:8080/status | jq
curl localhost:9090/metrics | grep replicare_
Ctrl-C then docker compose down -v to clean up.
Verifying the metrics
/metrics shows many replicare_* series once a sync is streaming, not just
one. To eyeball the series that only populate at runtime:
curl -s localhost:9090/metrics | grep -E \
'replicare_(rows_copied_total|apply_batch_seconds|throughput_rows_per_second|replication_lag_seconds|table_phase_info|target_up)'
replicare_table_phase_info(one line per table,initial_copythenstreaming),replicare_target_upand the row estimatereplicare_initial_copy_rows_targetappear as soon as the sync starts;replicare_rows_copied_totalclimbs as the initial copy commits rows.replicare_apply_batch_secondsandreplicare_replication_lag_secondsrefresh on every streaming drain pass.replicare_throughput_rows_per_secondis only non-zero on a pass that actually applied deltas — make a change on the source (theINSERT/UPDATE/DELETEabove) and re-scrape; a caught-up idle sync correctly reports0here.
The full list of every metric, its type, and its labels is in operations.md.
Point it at your own databases
0. Run the pre-migration checklist
Before pointing replicare at a production source, gather the facts below and confirm the must-haves. The queries for each item, for Postgres and MySQL, are in operations.md → Before a migration: checklist.
Sizing and traffic
- Total database size, and row counts and sizes per table.
- Write volume over 24 hours (inserts, updates and deletes): every change adds one row to replicare's change tracking on the source.
- Daily and weekly low-traffic windows, for catching up and for the final cutover.
Must be true before starting
-
Every table has a primary key or a
UNIQUEconstraint (tables without one are skipped). -
Postgres: every extension the tables use exists on the target. MySQL: every table is InnoDB,
the source is 5.7 or newer (older sources cannot add replicare's triggers next to a table's
own), and the target has
local_infileon. - Materialized views and views are not replicated: plan to create and refresh them on the target. Postgres large objects are not replicated either.
-
Schema changes and migrations are frozen for the whole replication (including online
schema-change tools such as
pt-online-schema-change/gh-oston MySQL). -
Regular vacuum and maintenance jobs are known: plain
VACUUM/ autovacuum is fine; heavy jobs that lock whole tables (VACUUM FULL,CLUSTER,pg_repack, MySQLOPTIMIZE TABLE) are paused during the initial copy and kept out of peak hours afterwards. -
No long-open (
idle in transaction) sessions on the source. - Source and target versions checked (target the same major version or newer), grants in place, and disk room on the source for the change-tracking tables to grow.
-
A plan for sequence / identity counters at cutover (
replicare reseed-sequences).
1. Prepare the target schema
replicare replicates data only — it never creates or migrates target tables. Create/migrate the target tables yourself first, matching the source columns you want to replicate. (Types must be compatible; pre-flight will tell you if not.)
The quickest way to match the source is to dump its structure only (no data) to a file, then load that file on each target.
# Postgres — 1) dump the source structure (no ownership/privilege noise)
pg_dump --schema-only --no-owner --no-privileges -h SRC_HOST -p SRC_PORT -U USER -d DB > schema.sql
# 2) load it into the target
psql -h TARGET_HOST -p TARGET_PORT -U USER -d DB -f schema.sql
# narrow it: -n public (one schema), -t public.orders (one table)
# MySQL — 1) dump (-P is the PORT; lowercase -p is the password prompt)
mysqldump --no-data -h SRC_HOST -P SRC_PORT -u USER -p DB > schema.sql
# 2) load
mysql -h TARGET_HOST -P TARGET_PORT -u USER -p DB < schema.sql
Credentials. These tools ask for a password when the server requires one: Postgres
prompts interactively (and mysqldump -p means "prompt me"). Don't put the password in
the command — it leaks into your shell history and ps. Instead store it in ~/.pgpass
(Postgres, chmod 600) or ~/.my.cnf (MySQL). If you prefer a one-step pipe
(pg_dump … | psql …), note it cannot prompt — the password step and the data stream
collide — so set up ~/.pgpass / PGPASSWORD first, or stick with the two-step form above.
For active-active, run this against every node before starting (and apply any later schema change to every node first — see active-active.md).
2. Grant least-privilege roles
Run the grant scripts (edit the role/schema/db values, or pass them with -v):
psql -v ON_ERROR_STOP=1 -v role=replicare -v db=app -v app_schema=public \
-f deploy/grants-source.sql "host=SOURCE dbname=app user=admin"
psql -v ON_ERROR_STOP=1 -v role=replicare -v db=warehouse -v app_schema=public \
-f deploy/grants-target.sql "host=TARGET dbname=warehouse user=admin"
The source role needs only CREATE-on-db (or a pre-created owned replicare
schema) plus USAGE/TRIGGER/SELECT; the target role needs only DML. See
configuration.md and
../deploy/ for details.
For MySQL, use ../deploy/grants-source-mysql.sql
and ../deploy/grants-target-mysql.sql
instead: the source role needs SELECT+TRIGGER on the replicated tables and
ownership of the replicare capture database; the target role needs the four DML
verbs plus CREATE TEMPORARY TABLES. See the MySQL engine page.
For Redis, use the ACL presets
../deploy/acl-source-redis.txt and
../deploy/acl-target-redis.txt: the source role
is read-only (+scan +dump +pttl +exists +type +memory|usage); the target role
needs +restore +del +scan +exists +module|list. Note the foot-gun — RESTORE is
@dangerous and must be granted explicitly with +restore. See
the Redis engine page.
3. Write a config
Copy ../examples/replicare.yml and edit the
connection blocks and the sync's table selection. Minimal example:
state_store:
engine: postgres
postgres: { host: db, port: 5432, database: replicare_state, user: replicare, password: "${PW}", sslmode: require }
sources:
app:
engine: postgres
postgres: { host: app-db, port: 5432, database: app, user: replicare, password: "${PW}", sslmode: verify-full }
targets:
warehouse:
engine: postgres
postgres: { host: wh-db, port: 5432, database: warehouse, user: replicare, password: "${PW}", sslmode: require }
syncs:
- name: app-to-warehouse
source: app
targets: [warehouse]
include: ["public.*"]
exclude: ["*_audit"]
Choosing tables. include and exclude are lists of schema.table glob
patterns, and exclude is applied on top of include. To drop one specific table,
name it in full under exclude:
include: ["public.*"] # everything in public…
exclude: ["public.audit_log"] # …except this one table
Patterns work too — *_audit excludes every table whose name ends in _audit. A few
notes: selection is read at daemon start, so change it and restart; excluding a table
only stops replicating it (rows already on the target are left in place, not deleted); and
if another replicated table has a foreign key pointing at the excluded one, replicare warns
about the dangling reference (the target must already satisfy that parent). For Redis,
selection uses key patterns instead of schema.table.
Secrets like ${PW} are read from the environment. Full field reference:
configuration.md.
For a MySQL sync, set engine: mysql with mysql: blocks (see the
MySQL connection block); the state
store stays on Postgres. A ready-to-run example is in
../examples/demo-mysql/config.yml.
For a Redis sync, set engine: redis with redis: blocks (see the
Redis connection block); the state store
still stays on Postgres (Redis writes nothing to the source, so all its durable
state lives there). Selection uses key-pattern globs (include: ["app:*"])
rather than schema.table. See the Redis engine page.
4. Validate, then run
export PW=... # the replicare role's password
replicare validate config.yml # pre-flight: no changes, reports problems
replicare run config.yml # start replicating (Ctrl-C / SIGTERM to stop cleanly)
validate connects read-only and classifies every column pair — fix any BLOCK
findings (incompatible/missing types) before running. Once running, check
progress any time with replicare status config.yml.
A few more commands you'll reach for later: replicare capture install|remove to
pre-provision (or tear down) the source-side trigger capture out of band,
replicare reseed --sync <s> --target <t> to force a laggard or diverged target
to re-copy its data, and — when you promote a one-way (DR) target —
replicare reseed-sequences --sync <s> to advance its identity/sequence counters to
max(id)+1 before it takes writes (otherwise nextval() collides with the copied rows;
passive/DR only — it refuses an active-active mesh). See the CLI reference.
5. Deploy
Install the binary and the sample systemd unit, run the Docker image, or deploy on Kubernetes with the Helm chart — see Kubernetes (Helm). See operations.md for tuning, monitoring, retention/reseed, and troubleshooting.
Next
- Configuration reference — every config field.
- CLI reference — every command.
- Operations — tuning, health signals, retention, restarts.
- Troubleshooting — common problems and fixes.
- Architecture — how the pipeline works end to end (capture → copy → stream → apply → checkpoint).
- Postgres engine — the reference engine: trigger CDC, faithful
COPY, FK components, least-privilege grants. - MySQL engine — MySQL→MySQL specifics, plus a runnable MySQL demo.
- Redis engine — Redis→Redis specifics: SCAN reconciliation, the delete sweep, TTL, cluster, and ACLs.