DB Internals AtlasPostgreSQL 18.6 Home About

Query processing

Asynchronous I/O

PostgreSQL 18 adds an asynchronous-I/O subsystem. Read streams can queue multiple upcoming reads, adjacent reads can be combined, and io_method chooses whether worker processes, io_uring, or synchronous execution performs an eligible I/O.

In plain words

A scan often knows which blocks it will need next. Instead of waiting for one storage read to finish before preparing the next one, PostgreSQL can keep several eligible reads in progress and consume pages as they become ready.

Read streams provide the look-ahead pattern; PostgreSQL 18's AIO subsystem provides the execution machinery. io_method = worker sends work to dedicated I/O workers, io_uring submits it through Linux io_uring when supported, and sync uses the same API but performs eligible I/O synchronously.

effective_io_concurrency influences how many storage operations a session tries to initiate in parallel. Adjacent work can also be combined up to io_combine_limit. The actual benefit depends on the query, cache state and storage; this animation shows the request lifecycle rather than inventing a universal speed-up.

For DBAs and operators

Read stream → buffer manager → AIO

A read stream asks for upcoming blocks. Blocks already in shared buffers need no storage read. For a miss, buffer-manager code reserves/pins destination buffers and lower storage layers define the I/O. PostgreSQL 18's central AIO abstraction uses PgAioHandle objects: a handle is acquired, associated with an operation, submitted, and later completed. Completion callbacks let different layers validate results and update state such as the BufferDesc.

Methods

  • worker (default): asynchronous from the issuing backend's point of view; I/O worker processes execute the underlying I/O.
  • io_uring: Linux 5.1+ and a PostgreSQL build with liburing; the issuing process dispatches through io_uring without the worker hand-off.
  • sync: no true asynchronous execution; the AIO API is used while the operation is carried out synchronously.

Concurrency controls

effective_io_concurrency defaults to 16 in PostgreSQL 18 and controls the number of concurrent storage I/O operations PostgreSQL expects and attempts to initiate for applicable work. io_max_concurrency is a server-start ceiling per process. io_combine_limit, bounded by io_max_combine_limit, caps combined I/O size.

Observe it

SHOW io_method;
SELECT * FROM pg_aios;  -- I/O handles currently visible to the AIO subsystem
SELECT backend_type, context, object, reads, read_bytes, read_time
  FROM pg_stat_io WHERE reads > 0;

Source trail

For developers: inside the source

Where it lives

src/backend/storage/aio/ (design doc README.md):

  • aio.c: handle life cycle: pgaio_io_acquire(), pgaio_io_stage(), pgaio_submit_staged(), pgaio_io_process_completion(), pgaio_io_wait(), pgaio_io_reclaim(), batch mode (pgaio_enter_batchmode() / pgaio_exit_batchmode()).
  • aio_io.c (pgaio_io_start_readv(), pgaio_io_perform_synchronously()), aio_callback.c (callback registry, PGAIO_HCB_* ids), aio_target.c, aio_init.c, aio_funcs.c (pg_get_aios() behind pg_aios).
  • Methods, each an IoMethodOps table: method_worker.c (pgaio_worker_submit(), IoWorkerMain()), method_io_uring.c (pgaio_uring_submit(), pgaio_uring_drain_locked()), method_sync.c.
  • read_stream.c (look-ahead, combining) and StartReadBuffers() / WaitReadBuffers() in bufmgr.c.

Key structures

  • PgAioHandle with PgAioHandleState (aio_internal.h): IDLE → HANDED_OUT → DEFINED → STAGED → SUBMITTED → COMPLETED_IO → COMPLETED_SHARED → COMPLETED_LOCAL, then back to IDLE.
  • PgAioWaitRef: handle id plus generation, because handles are recycled as soon as they complete. PgAioReturn / PgAioResult (aio_types.h): the result for the issuer. Shared completion callbacks run in a critical section, possibly in another process, so they never raise an ERROR; the issuer calls pgaio_result_report().
  • ReadStream (private to read_stream.c): adaptive distance, max_ios (from effective_io_concurrency or maintenance_io_concurrency, per tablespace), the pending read being formed, a circular queue of pinned buffers and an array of in-progress ReadBuffersOperations.

Call flow of a seq scan read

  1. heap_fetch_next_buffer() → read_stream_next_buffer() → read_stream_look_ahead(): the callback returns block numbers, consecutive ones join the pending read up to io_combine_limit, then read_stream_start_pending_read() → StartReadBuffers().
  2. PinBufferForBlock() pins each buffer (hits are done); for misses AsyncReadBuffers() acquires a handle, registers PGAIO_HCB_SHARED_BUFFER_READV, and calls smgrstartreadv() → mdstartreadv() (adds PGAIO_HCB_MD_READV) → FileStartReadV() → pgaio_io_start_readv().
  3. Worker mode: pgaio_worker_submit() queues the handle under AioWorkerSubmissionQueueLock and sets an idle worker's latch; the worker reopens the file and runs pgaio_io_perform_synchronously(), so the shared callbacks (md_readv_complete(), then buffer_readv_complete(): page verification, TerminateBufferIO()) run in the I/O worker.
  4. When the consumer reaches the block, WaitReadBuffers() waits on the wait reference (AioIoCompletion wait event if it actually blocks).

With io_method = sync, StartReadBuffers() starts no I/O: the read happens in WaitReadBuffers(), and the stream can issue posix_fadvise advice instead (not for streams flagged READ_STREAM_SEQUENTIAL, not with direct I/O). effective_io_concurrency = 0 makes streams read synchronously.

Observe it

  • pg_aios (state, operation, target description) and pg_stat_io; wait events AioIoCompletion, AioIoUringSubmit, AioIoUringExecution, IoWorkerMain.
  • Handle transitions are logged at DEBUG2–DEBUG5 by pgaio_debug_io() (log_min_messages = debug3 on a test server); debug_io_direct (developer option) turns on direct I/O.
  • Tests: src/test/modules/test_aio (TAP 001_aio.pl, 002_io_workers.pl).

Parameters that shape this

io_method io_workers effective_io_concurrency maintenance_io_concurrency io_combine_limit io_max_combine_limit io_max_concurrency track_io_timing

Related topics

I/O workers Scan methods Shared buffers Buffer manager internals VACUUM Life of a query Parallel query