Standby is in promoting state after autofailover using EFM

SOLVED
Highlighted
Level 3 Adventurer

Standby is in promoting state after autofailover using EFM

Hey guys,

 

I have been working on EFM to test and perform an autofailover scenario. Everything was configured scuccessfully and the output of efm cluster-status was perfect.

 

Here is my settings:

3 nodes (One master and two slaves). Efm agents running on all 3 nodes.auto.failover is set to true in efm.properties.

 

Below is the content of recovery.conf on the two slaves:

standby_mode = 'on'
primary_conninfo = 'user=enterprisedb passfile=''/opt/edb/as10/.pgpass'' host='master_db_ip' port='portnum' sslmode=prefer sslcompression=1 krbsrvname=postgres target_session_attrs=any'
trigger_file='/tmp/trigger_slave1'
recovery_target_timeline = 'latest'
restore_command='scp enterprisedb@master_db_ip:/opt/edb/archives/%f %p'

 

To test failover, I stopped the master database. After which i ran the efm cluster-status command and found that standby was not promoted to master and it was in "Promoting" state. I waited for a long time to see if it becomes a new master but it never did.Please let me know what should i check for to resolve this ? Has anyone faced this issue before?

 

Thanks and Regards,

Nikhil

Archived Discussions


Effective March 31st, we will no longer engage on PostgresRocks.


How to engage with us further?


  • Thought Leadership: EDB Blogs

  • Tips and Tricks: Postgres Tutorials

  • Customer Support: Create a Case Please note: Only customers with an active EDB support subscription and support portal authorization can create support ticket

  • Engage on Stackoverflow While engaging on Stackoverflow tag the question with EDB or EnterpriseDB.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
EDB Team Member

Re: Standby is in promoting state after autofailover using EFM


To test failover, I stopped the master database. After which i ran the efm cluster-status command and found that standby was not promoted to master and it was in "Promoting" state. I waited for a long time to see if it becomes a new master but it never did.Please let me know what should i check for to resolve this ? Has anyone faced this issue before?

Was the trigger file created? If so, then there's something wrong in your database setup so that the standby never came out of recovery -- check the database log. If not, check the efm agent log on that standby node. If there was a problem, though, the efm agent should have sent an email notification to you.

 

You can look for "Writing trigger file as specified in recovery.conf." in the efm agent log to find the place where it tried to promote the standby.

 

Cheers,

Bobby

 

View solution in original post

5 REPLIES 5
Highlighted
EDB Team Member

Re: Standby is in promoting state after autofailover using EFM


To test failover, I stopped the master database. After which i ran the efm cluster-status command and found that standby was not promoted to master and it was in "Promoting" state. I waited for a long time to see if it becomes a new master but it never did.Please let me know what should i check for to resolve this ? Has anyone faced this issue before?

Was the trigger file created? If so, then there's something wrong in your database setup so that the standby never came out of recovery -- check the database log. If not, check the efm agent log on that standby node. If there was a problem, though, the efm agent should have sent an email notification to you.

 

You can look for "Writing trigger file as specified in recovery.conf." in the efm agent log to find the place where it tried to promote the standby.

 

Cheers,

Bobby

 

View solution in original post

Highlighted
Level 3 Adventurer

Re: Standby is in promoting state after autofailover using EFM

Thanks Bobby!

 

Issue resolved. I think it was the location of the trigger file that caused the standby database to be in promoting state and not perform recovery.

 

Location was probably unreachable.

 

Thank you once again.

 

Regards,

Nikhil

Highlighted
EDB Team Member

Re: Standby is in promoting state after autofailover using EFM


Issue resolved. I think it was the location of the trigger file that caused the standby database to be in promoting state and not perform recovery.

One small note of caution here. I only know of this happening once, but that's once too many. When it's time to promote a standby, EFM reads the trigger file location from the recovery.conf file. If someone edits recovery.conf to change the location and forgets to tell the database (i.e. a restart), then efm will happily create the file and the database won't care at all and nothing good happens.

 

Not that you would ever do that, but it's possible. :)

 

Cheers,

Bobby

 

Highlighted
Level 3 Adventurer

Re: Standby is in promoting state after autofailover using EFM

Hi Bobby,

 

Thank you for your valuable inputs!

Extra knowledge surely helps!Smiley Wink

 

Regards,

Nikhil

Highlighted
Adventurer

Re: Standby is in promoting state after autofailover using EFM

 

same thing happened with us.we had a script to rebuild standby database using basebackup but missed edititing recovery.conf file for trigger file.we used to manually add trigger file location after standby starts and before starting efm service.So effectively db was unaware of file location whenever efm initiated promotion  and it took long time to get to this point.Smiley Happy

© 2019 EnterpriseDB Corporation. All rights reserved.   |   Privacy Policy   |  Terms of Use   |   Trademarks