cancel
Showing results for 
Search instead for 
Did you mean: 

FATAL:Invalid check point HINT: Regarding removing of backup_lable

If the below messages are encountered while starting the database server 

 

2019-04-30 09:00:26 EDT FATAL: the database system is starting up 
2019-04-30 09:00:27 EDT FATAL: the database system is starting up 
2019-04-30 09:00:27 EDT LOG: invalid checkpoint record 
2019-04-30 09:00:27 EDT FATAL: could not locate required checkpoint record 
2019-04-30 09:00:27 EDT HINT: If you are not restoring from a backup, try removing the file "/xtra200/edb/postgres_mode/as9.6/data/backup_label". 
2019-04-30 09:00:27 EDT LOG: startup process (PID 13039) exited with exit code 1 
2019-04-30 09:00:27 EDT LOG: aborting startup due to startup process failure 
2019-04-30 09:00:27 EDT LOG: database system is shut down

First step, Don't attempt to delete the backup_lable. Doing so, without checking few things may corrupt your database.

 

If you are restoring from a backup then make sure that you have all the WAL files that are generated from the time

the backup is started, we can get this information from the backup_lable's 'START WAL LOCATION' wal file and

try to start the database which should fix the issues.

 

What happens if we remove backup_lable file?

 

If you are starting the database by restoring a backup and performing PITR and if you remove the 

backup_lable file then database server might think it is coming from the crash recovery and starts

to restore from the recent checkpoint instead of replaying from the time when the backup started

that can cause the inconsistency.

 

Removing the backup_lable can work in below scenario:

 

In case, the database server is kept in backup mode using the pg_start_backup and fails to execute the pg_stop_backup

it leads to a backup_lable pesence in the $PGDATA directory and in future/later when user attempts to restart the database

it end up looking into backup_lable file which contains old checkpoint related information and fails to start. 

 

You need to check the timestamp of the backup_lable and checkpoint information from pg_controldata

to understand whether the backup_lable is old 

 

Now check the  $PGBIN/pg_controldata -D <data directory> 

 

Latest checkpoint location:           0/11000028
Prior checkpoint location:            0/10068700
Latest checkpoint's REDO location:    0/11000028
Latest checkpoint's REDO WAL file:    000000010000000000000011

If the checkpoint present in the backup_lable is representing old WAL file and timestamp of the backup_lable 

is old, if and only if you are certain that you are not starting the database from a backup, then you can move

the backup_lable to backup_lable_bkp file and try to start the database server.

 

Note: It is recommended to take the filesystem copy of the database server before taking such actions.

Version history
Revision #:
1 of 1
Last update:
‎05-09-2019 01:42 PM
Updated by:
 
Contributors