Start here
The big picture
PostgreSQL is a family of processes that share one block of memory and write to two kinds of files: data files and the write-ahead log.
In plain words
Think of PostgreSQL as a small office. The postmaster is the receptionist: it greets every new connection and hands it a dedicated assistant, the backend process. All assistants share one big whiteboard, the shared memory, where recently used table pages live.
When you change data, your assistant edits the page on the whiteboard and writes a short note of the change in the write-ahead log (WAL). Only the note has to reach the disk before PostgreSQL says "committed". The edited pages are copied to the data files later, in the background, by the checkpointer and background writer.
Other helpers keep things healthy: autovacuum cleans up old row versions, the WAL writer flushes log notes, the archiver copies finished WAL files to backup storage, and WAL senders stream changes to replicas.
For DBAs and operators
Processes, not threads
Every client connection is served by its own operating-system process forked from the postmaster. That gives strong isolation (a crashing backend cannot corrupt another's private memory) at the cost of per-connection overhead, which is why connection poolers such as PgBouncer are common.
Shared memory
Allocated once at start-up. It holds shared_buffers (the page cache), WAL buffers, the lock table, the commit log (CLOG/pg_xact) cache, the proc array used to build snapshots, and since PostgreSQL 15 the cumulative statistics.
Two write paths
- WAL is written sequentially and flushed at commit (unless
synchronous_commitis relaxed). It is what makes a commit durable. - Data pages are written lazily by the background writer and checkpointer (and by a backend itself when it must evict a dirty page). After a crash, recovery replays WAL from the last checkpoint to rebuild any page that had not reached disk.
See the processes yourself
SELECT pid, backend_type, state, query
FROM pg_stat_activity
ORDER BY backend_type;On Linux, ps -ef | grep postgres shows the same list with titles such as postgres: checkpointer and postgres: app appdb 10.0.0.5(51234) idle.
For developers: inside the source
Where it lives
postmaster.c:PostmasterMain()andServerLoop(); the supervisor that forks every other process.postgres.c:PostgresMain(), the message loop each backend runs, andexec_simple_query().ipci.c:CalculateShmemSize()andCreateSharedMemoryAndSemaphores()size and create every shared area (buffers, WAL buffers, lock table, proc array, SLRUs, statistics).bufmgr.candfreelist.c: the buffer pool and its clock sweep.xloginsert.cbuilds WAL records,xlog.cwrites and flushes them,xact.cruns commit.
The UPDATE in the animation, as calls
exec_simple_query() tcop/postgres.c
pg_parse_query() → pg_analyze_and_rewrite_fixedparams() → pg_plan_queries()
PortalRun() → ExecutorRun() → ExecModifyTable() → ExecUpdate()
table_tuple_update() → heap_update() access/heap/heapam.c
log_heap_update() → XLogInsert(RM_HEAP_ID, …)
finish_xact_command() → CommitTransactionCommand() → CommitTransaction()
RecordTransactionCommit(): XactLogCommitRecord() → XLogFlush()
→ TransactionIdCommitTree() (pg_xact)The WAL-before-data rule is enforced when a page is written out: FlushBuffer() calls XLogFlush() up to the page LSN before handing the page to the storage manager. The background side: CheckpointerMain() → CreateCheckPoint() → CheckPointGuts() → CheckPointBuffers() → BufferSync(); BackgroundWriterMain() → BgBufferSync(); WalWriterMain() → XLogBackgroundFlush().
Concurrency in brief
Shared structures are protected by LWLocks (src/backend/storage/lmgr/lwlock.c): each buffer has a content lock and the buffer mapping table is split into partitions; WAL insertion is spread over NUM_XLOGINSERT_LOCKS (8) WAL insertion locks so several backends can copy records into WAL buffers at once. Heavyweight locks (tables, rows waiting on XIDs) live in the lock table described in src/backend/storage/lmgr/README.
Read and experiment
- READMEs:
access/transam/README(WAL and transaction rules),storage/buffer/README(pins, content locks, clock sweep),executor/README,optimizer/README. - Observe:
pg_buffercache(what is in shared buffers),pg_walinspectorpg_waldump(WAL records),pg_stat_io(I/O by process type). - Tests:
make checkrunssrc/test/regress; TAP suites such assrc/test/recoverystart real clusters. - Extensions plug in via
shared_preload_libraries:shmem_request_hook/shmem_startup_hookfor shared memory,post_parse_analyze_hook,planner_hook,ExecutorStart_hookand friends.pg_stat_statements.cis the classic example that uses all of them.
Parameters that shape this
max_connections shared_buffers wal_buffers work_mem synchronous_commit checkpoint_timeout autovacuum
Related topics
Process model Memory architecture Write-ahead log (WAL) Checkpoints MVCC: row versions Life of a query