DB Internals AtlasPostgreSQL 18.6 Home About

WAL and recovery

Archiving and point-in-time recovery

Copy completed WAL to safe storage and keep a base backup, and PostgreSQL can replay forward to a chosen recovery target after that backup.

In plain words

A base backup is a full copy of the data directory, taken while the server keeps running. On its own it is a snapshot of one moment. If you also keep every WAL file written after it (that is continuous archiving), you can start from the snapshot and replay the log forward to any later moment you choose.

That is point-in-time recovery (PITR). Someone dropped a table at 14:05? Restore a suitable base backup and replay archived WAL to a recovery target before the unwanted transaction. A timestamp target such as 14:04:59 can be useful when transaction timing is known, while a named restore point or LSN target can be more precise.

Each time a recovered server is opened for writing, it starts a new timeline: a new branch of history with its own WAL file names, so the "old future" (with the dropped table) and the "new future" never get mixed up.

For DBAs and operators

Archiving

With archive_mode = on, the archiver runs archive_command (or an archive_library module) for each completed segment; %p is the path, %f the file name. A non-zero exit leaves the .ready file in pg_wal/archive_status and it is retried, so a broken archive command makes pg_wal grow until the disk is full:

LOG: archive command failed with exit code 1DETAIL: The failed archive command was: cp pg_wal/000000010000000000000015 /archive/000000010000000000000015
SELECT archived_count, last_archived_wal, failed_count, last_failed_wal, last_failed_time
  FROM pg_stat_archiver;

archive_timeout can force a segment switch on quiet systems so otherwise-partial WAL becomes eligible for archiving after roughly the configured interval. It does not guarantee that the archive itself is no more than that many seconds behind: a slow or failing archiver can still build a backlog. The command must not overwrite existing files and should make the copy durable before reporting success; tools such as pgBackRest, Barman or WAL-G handle this.

Restoring to a point

  1. Restore the base backup into an empty data directory (and remove stale pg_wal contents).
  2. Set restore_command and one target: recovery_target_time, _xid, _lsn, _name (from pg_create_restore_point()) or recovery_target = 'immediate' (stop at the first consistent point).
  3. Create recovery.signal and start. (standby.signal instead means "stay in recovery and keep following WAL": a standby.)

recovery_target_inclusive (default on) decides whether a matching xid/time/LSN is replayed or stopped just before. recovery_target_action: pause (default; inspect read-only, then pg_wal_replay_resume() to promote), promote, or shutdown. recovery_target_timeline defaults to latest.

Real log of a PITR run (PostgreSQL 18)

LOG: restored log file "000000010000000000000014" from archiveLOG: starting point-in-time recovery to "before_drop"LOG: redo starts at 0/14000028LOG: restored log file "000000010000000000000015" from archiveLOG: consistent recovery state reached at 0/14000120LOG: recovery stopping at restore point "before_drop", time 2026-10-22 15:37:40.58803+00LOG: redo done at 0/1501E318 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.03 sLOG: selected new timeline ID: 2LOG: archive recovery complete

"No such file or directory" messages from cp for .history files or the next segment are expected: that is how recovery probes the archive.

Timelines

The new timeline's history file records where it forked: 00000002.history contains 1 0/1501E380 at restore point "before_drop". WAL on the new branch is named 00000002…. Standbys and later restores follow timeline switches through these files, so always archive them too.

Incremental backups (PostgreSQL 17+)

With summarize_wal = on the WAL summarizer records which blocks changed. pg_basebackup --incremental=/backups/full/backup_manifest then copies only changed blocks, and pg_combinebackup full incr1 incr2 -o restored reconstructs a full data directory before you start recovery.

For developers: inside the source

Archiving side

  • xlogarchive.c: XLogArchiveNotify() / XLogArchiveNotifySeg() create archive_status/<segment>.ready (called from XLogWrite() when a segment is finished); XLogArchiveCheckDone() keeps RemoveOldXlogFiles() from recycling segments not yet archived.
  • pgarch.c: PgArchiverMain() → pgarch_ArchiverCopyLoop() → pgarch_readyXlog() (scans for .ready files, up to NUM_FILES_PER_DIRECTORY_SCAN (64) per scan, oldest first, timeline history files before segments) → pgarch_archiveXlog(), which calls the module's archive_file_cb. On success pgarch_archiveDone() renames .ready to .done; after NUM_ARCHIVE_RETRIES (3) failures in a row the archiver sleeps before trying again.
  • shell_archive.c: the built-in module behind archive_command (wait event ArchiveCommand).

Archive modules (extension API)

A library named in archive_library exports _PG_archive_module_init(), returning an ArchiveModuleCallbacks struct (archive_module.h):

typedef struct ArchiveModuleCallbacks
{
    ArchiveStartupCB startup_cb;
    ArchiveCheckConfiguredCB check_configured_cb;
    ArchiveFileCB archive_file_cb;     /* the only required one */
    ArchiveShutdownCB shutdown_cb;
} ArchiveModuleCallbacks;

basic_archive.c is the reference implementation: it copies to a temporary file, fsyncs, then renames, and treats an identical existing file as success.

Recovery side

  • xlogrecovery.c: readRecoverySignalFile() (recovery.signal / standby.signal), validateRecoveryParameters(), read_backup_label(); WaitForWALToBecomeAvailable() switches between the sources XLOG_FROM_ARCHIVE, XLOG_FROM_PG_WAL and XLOG_FROM_STREAM; recoveryStopsBefore()/recoveryStopsAfter() test every record against the RECOVERY_TARGET_* type; getRecoveryStopReason() produces the text written into the history file.
  • RestoreArchivedFile() in xlogarchive.c expands restore_command with BuildRestoreCommand() and runs it with system() (wait event RestoreCommand), restoring into the temporary name RECOVERYXLOG.
  • timeline.c: readTimeLineHistory(), findNewestTimeLine(), writeTimeLineHistory(). At the end of archive recovery StartupXLOG() picks findNewestTimeLine(recoveryTargetTLI) + 1 and writes the new .history file.
  • Backups: src/backend/backup/ (basebackup.c, basebackup_incremental.c), walsummarizer.c and pg_combinebackup.

Observe and experiment

  • TAP tests: src/test/recovery/t/002_archiving.pl, 003_recovery_targets.pl, 004_timeline_switch.pl, 020_archive_status.pl, 028_pitr_timelines.pl; src/bin/pg_combinebackup/t/ for incremental backups. The test helper Cluster.pm sets up archiving with has_archiving => 1 and restores with has_restoring => 1.
  • pg_stat_archiver, the .ready/.done files, and pg_walfile_name() to map an LSN to a segment.

Parameters that shape this

archive_mode archive_command archive_library archive_timeout restore_command recovery_target recovery_target_time recovery_target_xid recovery_target_lsn recovery_target_name recovery_target_inclusive recovery_target_timeline recovery_target_action summarize_wal wal_level

Related topics

Write-ahead log (WAL) Archiver Crash recovery Startup process WAL summarizer Streaming replication Checkpoints