Write Ahead Log

Write ahead log (WAL) used to provide database data fault tolerance in the case of system/hardware failures. WAL recovery logs used in automatic recovery procedure get the most actual and consistent database state before a crash.

Savepoint

Flushing of WAL file to filesystem at consistent database state.

In the case of failure it can be rolled forward to restore database state to the latest savepoint time stamp.

Checkpoint

All pending WAL persisted database changes are applied to the database file.

WAL file data truncated to zero following by savepoint sync.

All durty pages flushed to disk.

Database WAL mode activated by the following IWKV_OPTS.wal settings:

IWKV_OPTS opts = {
    ...
    .wal = {
      .enabled = true,  // WAL Enabled/Disabled

      // Time period between WAL file sync at consistent database state. Defaut: 10 seconds
      .savepoint_timeout_sec = 10, 

      // Time period between WAL checkpoints. Default: 5 min.
      .checkpoint_timeout_sec = 300, 

      // Size of internal memory buffer for WAL operations. Default 8Mb
      .wal_buffer_sz = 8 * 1024 * 1024, 
      
       // Maximum amount of memory used by duty pages. Overflow causes WAL checkpoint. 
       // Default: 1Gb
      .checkpoint_buffer_sz = 1ULL * 1024 * 1024 * 1024 
    }
  };

How WAL works

• Database file mmaped using private copy-on-write mapping MAP_PRIVATE
• All ongoing changes saved into WAL file then applied to privately mmaped database file region
• Checkpoint performed on:
  • Amount of private mmap changes data exceeds a specified limit: .checkpoint_buffer_sz
  • .checkpoint_timeout_sec timeout occuried
• Checkpoint steps:
  1. Perform savepoint
  2. Remap database file in shared mmap mode MAP_SHARED
  3. Apply all write WAL operations to the database file
  4. Restore copy-on-write mapping MAP_PRIVATE for database file.
  5. Truncate WAL to zero

Recovering

If database opened with WAL enabled option and found not empty WAL log on file system it will be applied to the database file until last successful savepoint.