Write-ahead log (WAL)

For developers: inside the source

Where it lives

• src/backend/access/transam/xloginsert.c (linked in the source trail): the record-building API XLogBeginInsert(), XLogRegisterData(), XLogRegisterBuffer(), XLogInsert(), and XLogRecordAssemble(), which chains the pieces together and decides per block whether a full-page image is needed.
• xlog.c: XLogInsertRecord() and the shared state (XLogCtlData, XLogCtlInsert), plus XLogFlush(), XLogWrite(), XLogBackgroundFlush() (the WAL writer's work) and segment recycling.
• xlogrecord.h: the on-disk format, an XLogRecord header (xl_tot_len, xl_xid, xl_prev, xl_info, xl_rmid, xl_crc) followed by XLogRecordBlockHeader entries and the payload.
• rmgrlist.h: the resource managers (Heap, Btree, Transaction, XLOG …) with their redo and desc callbacks; the desc functions in rmgrdesc/ produce what pg_waldump prints.

Writing a record

Every WAL-logged change follows the pattern in the "Write-Ahead Log Coding" section of access/transam/README (simplified from heap_insert()):

START_CRIT_SECTION();
/* ... change the page ... */
MarkBufferDirty(buffer);
XLogBeginInsert();
XLogRegisterData(&xlrec, SizeOfHeapInsert);
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
recptr = XLogInsert(RM_HEAP_ID, info);
PageSetLSN(page, recptr);
END_CRIT_SECTION();

Insertion has two phases. ReserveXLogInsertLocation() advances Insert->CurrBytePos under the insertpos_lck spinlock (the only fully serialized step), then CopyXLogRecordToWAL() copies the bytes into the WAL buffers while holding one of NUM_XLOGINSERT_LOCKS (8) WAL insertion locks, so several backends copy at once. If a checkpoint moved RedoRecPtr after the record was assembled, XLogInsertRecord() returns InvalidXLogRecPtr and XLogInsert() loops to reassemble it with the now-required page images.

Commit and flush

xact.c: RecordTransactionCommit() → XactLogCommitRecord() → XLogFlush(XactLastRecEnd) when synchronous_commit is not off (or the commit drops relations); otherwise it only calls XLogSetAsyncXactLSN(). XLogFlush() calls WaitXLogInsertionsToFinish(), then takes WALWriteLock with LWLockAcquireOrWait(): a backend that had to wait and finds its LSN already flushed by someone else returns without writing. That is group commit. commit_delay is a pg_usleep() before taking the lock, only when at least commit_siblings other transactions are active. XLogWrite() writes and syncs (issue_xlog_fsync()), marks finished segments for the archiver (XLogArchiveNotifySeg()) and requests a checkpoint when XLogCheckpointNeeded() says CheckPointSegments was exceeded.

Observe and experiment

• pg_walinspect: pg_get_wal_records_info(), pg_get_wal_block_info(), pg_get_wal_stats() from SQL; pg_waldump offline.
• wal_consistency_checking = 'heap,btree' adds a page image to every such record and compares it after replay (verifyBackupPageConsistency()), the standard way to test a new redo routine on a standby.
• wal_debug exists only when the server is compiled with WAL_DEBUG defined (see pg_config_manual.h).
• TAP tests: src/test/recovery/t/039_end_of_wal.pl, 018_wal_optimize.pl (the wal_level = minimal skip), src/bin/pg_waldump/t/.

Extension hooks

Extensions can log their own pages with generic WAL (generic_xlog.c: GenericXLogStart(), GenericXLogRegisterBuffer(), GenericXLogFinish(), which logs page deltas), or register a custom resource manager with RegisterCustomRmgr(), which only works from a library in shared_preload_libraries; test_custom_rmgrs is a minimal example.