Getting started

This walks you from zero to a running replication in a few minutes, then explains how to point replicare at your own databases.

What replicare does

You give replicare a source database and one or more target databases and a list of tables. It:

  1. installs lightweight capture (triggers) on the source,
  2. copies the selected data to the targets (chunked and parallel), then
  3. streams every insert/update/delete continuously to keep the targets converged.

No WAL access, no REPLICATION role, no wal_level change — just ordinary, grantable privileges. Architecture explains why.

Try the demo (2 minutes)

The fastest way to see it work end to end:

task docker                        # build the replicare image
cd examples/demo && docker compose up

This starts a source Postgres (seeded with public.orders), a target Postgres, and replicare copying then streaming between them. In another shell:

# initial copy landed
docker compose exec target psql -U postgres -d warehouse -c "SELECT count(*) FROM public.orders;"

# make a live change on the source and watch it propagate
docker compose exec source psql -U postgres -d app \
  -c "INSERT INTO public.orders VALUES (101,'live'); UPDATE public.orders SET note='changed' WHERE id=1; DELETE FROM public.orders WHERE id=2;"
docker compose exec target psql -U postgres -d warehouse -c "SELECT count(*), max(id) FROM public.orders;"

# operator surface
curl localhost:8080/status | jq
curl localhost:9090/metrics | grep replicare_

Ctrl-C then docker compose down -v to clean up.

Verifying the metrics

/metrics shows many replicare_* series once a sync is streaming, not just one. To eyeball the series that only populate at runtime:

curl -s localhost:9090/metrics | grep -E \
  'replicare_(rows_copied_total|apply_batch_seconds|throughput_rows_per_second|replication_lag_seconds|table_phase_info|target_up)'
  • replicare_table_phase_info (one line per table, initial_copy then streaming), replicare_target_up and the row estimate replicare_initial_copy_rows_target appear as soon as the sync starts; replicare_rows_copied_total climbs as the initial copy commits rows.
  • replicare_apply_batch_seconds and replicare_replication_lag_seconds refresh on every streaming drain pass.
  • replicare_throughput_rows_per_second is only non-zero on a pass that actually applied deltas — make a change on the source (the INSERT/UPDATE/DELETE above) and re-scrape; a caught-up idle sync correctly reports 0 here.

The full list of every metric, its type, and its labels is in operations.md.

Point it at your own databases

0. Run the pre-migration checklist

Before pointing replicare at a production source, gather the facts below and confirm the must-haves. The queries for each item, for Postgres and MySQL, are in operations.md → Before a migration: checklist.

Sizing and traffic

  • Total database size, and row counts and sizes per table.
  • Write volume over 24 hours (inserts, updates and deletes): every change adds one row to replicare's change tracking on the source.
  • Daily and weekly low-traffic windows, for catching up and for the final cutover.

Must be true before starting

  • Every table has a primary key or a UNIQUE constraint (tables without one are skipped).
  • Postgres: every extension the tables use exists on the target. MySQL: every table is InnoDB, the source is 5.7 or newer (older sources cannot add replicare's triggers next to a table's own), and the target has local_infile on.
  • Materialized views and views are not replicated: plan to create and refresh them on the target. Postgres large objects are not replicated either.
  • Schema changes and migrations are frozen for the whole replication (including online schema-change tools such as pt-online-schema-change / gh-ost on MySQL).
  • Regular vacuum and maintenance jobs are known: plain VACUUM / autovacuum is fine; heavy jobs that lock whole tables (VACUUM FULL, CLUSTER, pg_repack, MySQL OPTIMIZE TABLE) are paused during the initial copy and kept out of peak hours afterwards.
  • No long-open (idle in transaction) sessions on the source.
  • Source and target versions checked (target the same major version or newer), grants in place, and disk room on the source for the change-tracking tables to grow.
  • A plan for sequence / identity counters at cutover (replicare reseed-sequences).

1. Prepare the target schema

replicare replicates data only — it never creates or migrates target tables. Create/migrate the target tables yourself first, matching the source columns you want to replicate. (Types must be compatible; pre-flight will tell you if not.)

The quickest way to match the source is to dump its structure only (no data) to a file, then load that file on each target.

# Postgres — 1) dump the source structure (no ownership/privilege noise)
pg_dump --schema-only --no-owner --no-privileges -h SRC_HOST -p SRC_PORT -U USER -d DB > schema.sql
# 2) load it into the target
psql -h TARGET_HOST -p TARGET_PORT -U USER -d DB -f schema.sql

# narrow it: -n public (one schema), -t public.orders (one table)
# MySQL — 1) dump  (-P is the PORT; lowercase -p is the password prompt)
mysqldump --no-data -h SRC_HOST -P SRC_PORT -u USER -p DB > schema.sql
# 2) load
mysql -h TARGET_HOST -P TARGET_PORT -u USER -p DB < schema.sql

Credentials. These tools ask for a password when the server requires one: Postgres prompts interactively (and mysqldump -p means "prompt me"). Don't put the password in the command — it leaks into your shell history and ps. Instead store it in ~/.pgpass (Postgres, chmod 600) or ~/.my.cnf (MySQL). If you prefer a one-step pipe (pg_dump … | psql …), note it cannot prompt — the password step and the data stream collide — so set up ~/.pgpass / PGPASSWORD first, or stick with the two-step form above.

For active-active, run this against every node before starting (and apply any later schema change to every node first — see active-active.md).

2. Grant least-privilege roles

Run the grant scripts (edit the role/schema/db values, or pass them with -v):

psql -v ON_ERROR_STOP=1 -v role=replicare -v db=app -v app_schema=public \
  -f deploy/grants-source.sql "host=SOURCE dbname=app user=admin"
psql -v ON_ERROR_STOP=1 -v role=replicare -v db=warehouse -v app_schema=public \
  -f deploy/grants-target.sql "host=TARGET dbname=warehouse user=admin"

The source role needs only CREATE-on-db (or a pre-created owned replicare schema) plus USAGE/TRIGGER/SELECT; the target role needs only DML. See configuration.md and ../deploy/ for details.

For MySQL, use ../deploy/grants-source-mysql.sql and ../deploy/grants-target-mysql.sql instead: the source role needs SELECT+TRIGGER on the replicated tables and ownership of the replicare capture database; the target role needs the four DML verbs plus CREATE TEMPORARY TABLES. See the MySQL engine page.

For Redis, use the ACL presets ../deploy/acl-source-redis.txt and ../deploy/acl-target-redis.txt: the source role is read-only (+scan +dump +pttl +exists +type +memory|usage); the target role needs +restore +del +scan +exists +module|list. Note the foot-gun — RESTORE is @dangerous and must be granted explicitly with +restore. See the Redis engine page.

3. Write a config

Copy ../examples/replicare.yml and edit the connection blocks and the sync's table selection. Minimal example:

state_store:
  engine: postgres
  postgres: { host: db, port: 5432, database: replicare_state, user: replicare, password: "${PW}", sslmode: require }
sources:
  app:
    engine: postgres
    postgres: { host: app-db, port: 5432, database: app, user: replicare, password: "${PW}", sslmode: verify-full }
targets:
  warehouse:
    engine: postgres
    postgres: { host: wh-db, port: 5432, database: warehouse, user: replicare, password: "${PW}", sslmode: require }
syncs:
  - name: app-to-warehouse
    source: app
    targets: [warehouse]
    include: ["public.*"]
    exclude: ["*_audit"]

Choosing tables. include and exclude are lists of schema.table glob patterns, and exclude is applied on top of include. To drop one specific table, name it in full under exclude:

    include: ["public.*"]            # everything in public…
    exclude: ["public.audit_log"]    # …except this one table

Patterns work too — *_audit excludes every table whose name ends in _audit. A few notes: selection is read at daemon start, so change it and restart; excluding a table only stops replicating it (rows already on the target are left in place, not deleted); and if another replicated table has a foreign key pointing at the excluded one, replicare warns about the dangling reference (the target must already satisfy that parent). For Redis, selection uses key patterns instead of schema.table.

Secrets like ${PW} are read from the environment. Full field reference: configuration.md.

For a MySQL sync, set engine: mysql with mysql: blocks (see the MySQL connection block); the state store stays on Postgres. A ready-to-run example is in ../examples/demo-mysql/config.yml.

For a Redis sync, set engine: redis with redis: blocks (see the Redis connection block); the state store still stays on Postgres (Redis writes nothing to the source, so all its durable state lives there). Selection uses key-pattern globs (include: ["app:*"]) rather than schema.table. See the Redis engine page.

4. Validate, then run

export PW=...                          # the replicare role's password
replicare validate config.yml          # pre-flight: no changes, reports problems
replicare run config.yml               # start replicating (Ctrl-C / SIGTERM to stop cleanly)

validate connects read-only and classifies every column pair — fix any BLOCK findings (incompatible/missing types) before running. Once running, check progress any time with replicare status config.yml.

A few more commands you'll reach for later: replicare capture install|remove to pre-provision (or tear down) the source-side trigger capture out of band, replicare reseed --sync <s> --target <t> to force a laggard or diverged target to re-copy its data, and — when you promote a one-way (DR) target — replicare reseed-sequences --sync <s> to advance its identity/sequence counters to max(id)+1 before it takes writes (otherwise nextval() collides with the copied rows; passive/DR only — it refuses an active-active mesh). See the CLI reference.

5. Deploy

Install the binary and the sample systemd unit, run the Docker image, or deploy on Kubernetes with the Helm chart — see Kubernetes (Helm). See operations.md for tuning, monitoring, retention/reseed, and troubleshooting.

Next