EFM does not Failover

[EduardR]

hi 

have cluster of master standby and witness

Agent Type  Address              Agent  DB       VIP

        -----------------------------------------------------------------------

        Master      172.23.238.66        UP     UP       172.23.238.71*

        Standby     172.23.238.67        UP     UP       172.23.238.71

        Witness     172.23.238.72        UP     N/A      172.23.238.71

Allowed node host list:

        172.23.238.72

Membership coordinator: 172.23.238.72

Standby priority host list:

        172.23.238.67

Promote Status:

        DB Type     Address              WAL LSN          Info

        --------------------------------------------------------------

        Master      172.23.238.66        0/1F0792A8

        Standby     172.23.238.67        0/1F0792A8

        Standby database(s) in sync with master. It is safe to promote.

than  i shuted down the DB on master node vip didn't switch to standy and standby didnt become master .Don't have any master basicly 

what is wrong here ? 

Cluster Status: efm

Agent Type Address Agent DB VIP
-----------------------------------------------------------------------
Idle 172.23.238.66 UP UNKNOWN 172.23.238.71
Standby 172.23.238.67 UP UP 172.23.238.71
Witness 172.23.238.72 UP N/A 172.23.238.71

Allowed node host list:
172.23.238.66 172.23.238.67 172.23.238.72

Membership coordinator: 172.23.238.66

Standby priority host list:
172.23.238.67

Promote Status:

DB Type Address WAL LSN Info
--------------------------------------------------------------
Standby 172.23.238.67 0/23000098

No master database was found.

Idle Node Status (idle nodes ignored in WAL LSN comparisons):

Address WAL LSN Info
--------------------------------------------------------------
172.23.238.66 UNKNOWN Connection to 172.23.238.66:5444 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

[bbissett]

There's no way to help you without more information. I don't even understand how you got into that state or what you did after. Please send reproducible steps and agent logs to your support contact.

Bobby

[EduardR]

the efm cluster was running as in first status

than i shuted down the db on master 

failover didnt happen 

[bbissett]

Well, it's worked for me a few thousand times. Without more info I can't help you.

[EduardR]

what info is needed ? 

[bbissett]

Steps to reproduce and logs. Please send to your support contact.

[pcpg]

Hello,

1. What is the EFM version on your side?

2. Is the EFM service running on all 3 hosts?

3. Have a look at EFM logs - it will say master not reachable and then some reason why is it not attempting failover.

3. Instead of shutting down the master DB can you kill the host and see if it VIP moves to the standly and if promotion takes place?

HTH

[EduardR]

hi 

the efm version is 3.5

I dont see errors in log it is just promoting the stadby endlessy 

logs :

From: efm@localhost
To: [aduardr@amdocs.com]
Subject: [INFO] EFM Assigning VIP to node 172.23.238.66
Body:
EFM node: 172.23.238.66
Cluster name: efm
Database name: postgres
VIP: 172.23.238.71 (Active)

Assigning VIP 172.23.238.71 to node 172.23.238.66

Results:
exit status: 0
2019-09-13 15:03:05 com.enterprisedb.efm.utils.Notifications sendMail INFO: Sending notification:
From: efm@localhost
To: [uardr@amdocs.com]
Subject: [WARNING] EFM Promotion has started on cluster efm
Body:
EFM node: 172.23.238.66
Cluster name: efm
Database name: postgres
VIP: 172.23.238.71 (Active)

Promotion of standby has started on cluster efm.
2019-09-13 15:03:24 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:03:29 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:03:33 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:03:38 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:08:30 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:10:56 com.enterprisedb.efm.admin.AdminServerThread processRequest INFO: Admin server received status request.
2019-09-13 15:10:56 com.enterprisedb.efm.admin.AdminServerThread processRequest INFO: Received param: efm
2019-09-13 15:10:56 com.enterprisedb.efm.nodes.EfmNode checkClusterStatus INFO: Sending status message to all nodes: node_status
2019-09-13 15:10:56 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:12:14 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:12:17 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true
2019-09-13 15:13:19 com.enterprisedb.efm.admin.AdminServerThread processRequest INFO: Admin server received status request.
2019-09-13 15:13:19 com.enterprisedb.efm.admin.AdminServerThread processRequest INFO: Received param: efm
2019-09-13 15:13:19 com.enterprisedb.efm.nodes.EfmNode checkClusterStatus INFO: Sending status message to all nodes: node_status
2019-09-13 15:13:19 com.enterprisedb.efm.nodes.EfmAgent handleStatusCall INFO: responding to db status request with status: Promoting%%172.23.238.66%%true%%true%%172.23.238.71%%true

[EduardR]

yes,3 nodes

master ,witness ,stadnby 

[bbissett]

Not sure how to put this in a different way than I have already -- we need the logs (not just a portion of one log that shows some status requests) to help.

Please send to your support contact.

Bobby

[EduardR]

Dear Bobby 

I posted the logs  for   pcpg . 

Unfortunately I don't have  support from EDB yet.

If I had  I would not bother and post it here .

This is POC .

Based on results we will decide either to go ahead with EFM or  implement our own solution .

Thanks 

[bbissett]

---

@EduardR wrote:

Dear Bobby 

I posted the logs  for   pcpg . 

I don't see any logs there. Do you have a link to them directly. If you want help, you have to give *something* for us to use to help.

You had a cluster. You did *something* and expected failover. What exactly did you do?

After you did that, short of sharing the logs to give us any information, what notifications did you receive?

Bobby

[jwatt]

Hi @EduardR , have you engaged with our presales team as of yet?  We often encourage people who are aiming for a PoC to have a line into EDB--whether a paid POC or otherwise, it's not our intent to leave you out in the cold.  Feel free to PM me and we can get that connection established.