Replication

Streaming replication

A standby server is a physical (block-level) copy of the primary that stays current by receiving the primary's WAL over a network connection and replaying it, just like continuous crash recovery.

In plain words

A standby (replica) starts life as a copy of the primary's data directory. From then on it keeps up by doing what crash recovery does, but forever: it receives every WAL record the primary writes and replays it onto its own copy of the data.

Two processes do the transport. On the primary a WAL sender reads new WAL and pushes it down a network connection; on the standby a WAL receiver writes it to the standby's own pg_wal. The standby's startup process then replays it. Usually the delay is a fraction of a second.

If the primary fails, you promote the standby and it becomes the new primary. Standbys can also answer read-only queries (hot standby) and can feed further standbys (cascading).

For DBAs and operators

Setting one up

-- on the primary
CREATE ROLE replicator WITH REPLICATION LOGIN PASSWORD '…';
-- pg_hba.conf: host replication replicator 10.0.0.0/24 scram-sha-256

$ pg_basebackup -h primary -U replicator -D /var/lib/postgresql/18/main \
      -R -X stream -C -S standby1 -P

-R creates the file standby.signal and writes primary_conninfo (plus primary_slot_name with -S) into postgresql.auto.conf. -C -S creates a replication slot so the primary keeps WAL the standby still needs.

Real start-up log of a new standby (PostgreSQL 18, excerpt)

LOG: entering standby modeLOG: redo starts at 0/3000028LOG: consistent recovery state reached at 0/3000120LOG: database system is ready to accept read-only connectionsLOG: started streaming WAL from primary at 0/4000000 on timeline 1

Monitoring

-- on the primary
SELECT application_name, state, sync_state,
       sent_lsn, write_lsn, flush_lsn, replay_lsn,
       write_lag, flush_lag, replay_lag,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn)) AS replay_gap
  FROM pg_stat_replication;
-- on the standby
SELECT status, flushed_lsn, sender_host, slot_name FROM pg_stat_wal_receiver;
SELECT pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn(),
       now() - pg_last_xact_replay_timestamp() AS replay_delay;

sent → write → flush → replay: a gap between sent and flush points to network or standby disk; flush far ahead of replay means replay is stuck, typically on a recovery conflict wait or a single-threaded replay bottleneck.
The *_lag columns are times measured from the standby's feedback messages (sent after writes and flushes, and at least every wal_receiver_status_interval); they become NULL on an idle system.
now() - pg_last_xact_replay_timestamp() grows on an idle primary even with no real lag.

If the standby falls too far behind without a slot, the primary may have recycled the WAL it needs:

FATAL: could not receive data from WAL stream: ERROR: requested WAL segment 000000010000000000000041 has already been removed

It can then only continue with restore_command from an archive, or must be rebuilt.

Promotion and failover

SELECT pg_promote(); (or pg_ctl promote) stops the walreceiver, finishes replaying what was received, selects a new timeline and opens read-write:

LOG: received promote requestLOG: redo done at 0/13000120 system usage: …LOG: selected new timeline ID: 2LOG: database system is ready to accept connections

PostgreSQL itself does not detect failure or fence the old primary; tools such as Patroni do. The old primary can rejoin as a standby after pg_rewind (needs wal_log_hints or data checksums) or a fresh base backup.

For developers: inside the source

Where it lives

primary: WalSndLoop(XLogSendPhysical)        replication/walsender.c
  WALRead() up to GetFlushRecPtr()  → CopyData to standby
  ProcessRepliesIfAny() → ProcessStandbyReplyMessage()
standby: WalReceiverMain()                    replication/walreceiver.c
  XLogWalRcvProcessMsg() → XLogWalRcvWrite() → XLogWalRcvFlush()
  WakeupRecovery()  → startup process replays
  XLogWalRcvSendReply()  (write/flush/apply positions)

walsender.c: exec_replication_command() parses the replication protocol (grammar in repl_gram.y / repl_scanner.l); START_REPLICATION runs StartReplication(), which enters WalSndLoop(XLogSendPhysical). A cascading standby sends up to GetStandbyFlushRecPtr() instead of GetFlushRecPtr(). ProcessStandbyReplyMessage() stores the reported positions, computes the *_lag times with LagTrackerRead() from samples that XLogSendPhysical() recorded with LagTrackerWrite(), releases synchronous-commit waiters and advances a physical slot via PhysicalConfirmReceivedLocation().
walreceiver.c: WalReceiverMain(). After each flush it publishes WalRcv->flushedUpto, calls WakeupRecovery() and, if cascading is allowed, WalSndWakeup(). XLogWalRcvSendHSFeedback() sends the hot_standby_feedback xmin.
libpqwalreceiver.c: the libpq transport (libpqrcv_connect(), libpqrcv_startstreaming(), libpqrcv_receive()), loaded as a module so the server binary does not link libpq; see README.
xlogrecovery.c: WaitForWALToBecomeAvailable(), the startup process's state machine that cycles between XLOG_FROM_ARCHIVE, XLOG_FROM_PG_WAL and XLOG_FROM_STREAM, and calls RequestXLogStreaming() (walreceiverfuncs.c) to have the postmaster start a walreceiver. CheckForStandbyTrigger() notices promotion: pg_promote() in xlogfuncs.c writes the promote signal file and sends SIGUSR1 to the postmaster.

Shared state

WalSnd (include/replication/walsender_private.h), one per walsender in WalSndCtl->walsnds[], holds sentPtr, write/flush/apply, writeLag/flushLag/applyLag and sync_standby_priority under a per-entry spinlock; pg_stat_get_wal_senders() copies it into pg_stat_replication. On the standby, WalRcvData (walreceiver.h: walRcvState, receiveStart, flushedUpto, latestWalEnd) backs pg_stat_wal_receiver. Walsenders do not poll: XLogFlush() and XLogBackgroundFlush() call WalSndWakeupProcessRequests(), which (through WalSndWakeup()) broadcasts the condition variable WalSndCtl->wal_flush_cv.

Experiment

TAP tests in src/test/recovery/t/: 001_stream_rep.pl, 004_timeline_switch.pl, 055_cascade_reconnect.pl. PostgreSQL::Test::Cluster gives you init_from_backup(…, has_streaming => 1), wait_for_catchup() and promote() to build your own scenarios.
log_replication_commands = on logs every command a walsender receives (IDENTIFY_SYSTEM, START_REPLICATION …).
Protocol reference: doc/src/sgml/protocol.sgml (streaming replication protocol section), including the reply and hot-standby-feedback message layouts.

Parameters that shape this

primary_conninfo primary_slot_name max_wal_senders wal_level hot_standby wal_receiver_status_interval wal_sender_timeout wal_receiver_timeout wal_keep_size recovery_min_apply_delay wal_retrieve_retry_interval