Receiving the following error when trying to run a BART backup
pg_basebackup: could not get write-ahead log end position from server: ERROR: requested WAL segment 000000160000035F00000083 has already been removed pg_basebackup: removing data directory "/opt/edb/backup/epas10/1532010220112"
Here are the settings from the custom conf file on my DB server:
wal_buffers = 16MB wal_log_hints = on # wal_level min_wal_size = 1GB max_wal_size = 10GB #wal_level = replica max_wal_senders = 3 wal_keep_segments = 15
Any advice on what I can change?
The above issue regarding the WAL segment removed during the backup method can be avoided using the streaming method (setting xlog-method to stream) of the transaction log in parallel with the creation of the backup for that database server
Please find the possible way's to avoid the WAL segment removed issue.
Includes the required transaction log files (WAL files) in the backup. This will include all transaction logs generated during the backup. If this option is specified, it is possible to start a postmaster directly in the extracted directory without the need to consult the log archive, thus making this a completely standalone backup.
The following methods for collecting the transaction logs are supported:
Stream the transaction log while the backup is created. This will open a second connection to the server and start streaming the transaction log in parallel while running the backup. Therefore, it will use up two connections configured by the max_wal_senders parameter. As long as the client can keep up with transaction log received, using this mode requires no extra transaction logs to be saved on the master.
Note: With stream method Plain format can be used only not compress with -z option.
For the error regarding the missing WAL segment, it could happen if Postgres completes two checkpoints when taking the backup. One thing that you could use is a replication slot that gives the guarantee that segments are retained. Those have been Integrated in pg_basebackup with 9.6.