cancel
Showing results for 
Search instead for 
Did you mean: 

Error while configuring Two replication nodes with separate hosts using EDB replication server 7.0

SOLVED
Highlighted
Level 3 Adventurer

Error while configuring Two replication nodes with separate hosts using EDB replication server 7.0

Hi,

 

I have tried and tested EDB replication server 7.0 using One host.

But when I am trying to configure Two replication nodes with separate hosts, while I am able to add database and create publication on the first node, I am getting error while running below command to join the second node on the leader service:

       

       ./runRepCLI.sh -joinnetwork -servername remoteServer -host 192.168.184.194 -port 8082 -user admin
       Service remoteServer has failed to register with the network.

 

I have also seen that one of the Zookeeper service on the first node goes down after running this command.

 

Steps followed:

Node 1 - 192.168.184.192

Node 2 - 192.168.184.194

Step 1:

[root@performance bin]# ./runRepCLI.sh -joinnetwork -servername replserver -host 192.168.184.192 -port 8082
Server is successfully added in the network.

Step 2:

[root@performance bin]# ./runRepCLI.sh -setadminpassword -savepassword
Enter admin password:
Admin password is set successfully.
Step 3:

[root@performance bin]# ./runRepCLI.sh -adddb -servername replserver -dbid 1 -dbtype enterprisedb -dbhost 192.168.184.192 -dbport 5444 -dbuser enterprisedb -dbpassfile encrypted_pass -database edb -user admin
Database is registered successfully.
Step 4:

[root@performance bin]# ./runRepCLI.sh -createpub -pubname pub_all_tables -servername replserver -dbid 1 -tables public.source_tb -user admin
Publication is created successfully.

Step 5:

[root@edbreplication bin]# ./runServer.sh --host 192.168.184.194 --config /usr/edb-rs-7.0/server/etc

Step 6:

./runRepCLI.sh -joinnetwork -servername remoteServer -host 192.168.184.194 -port 8082 -user admin

 

 

 

Error snapshot from logfile on leader service:

2019-01-21 09:44:10,075 [qtp1790421142-25] INFO [c.e.n.c.z.ZooKeeperHelper] - verifyIfZooKeeperStarted: ZooKeeper server running at port 2182 having status LEADER
2019-01-21 09:44:10,076 [qtp1790421142-25] INFO [c.e.n.c.z.QuorumPeerMainExtended] - Shutting down ZooKeeper...
2019-01-21 09:44:10,076 [qtp1790421142-25] INFO [o.a.z.s.q.Leader] - Shutting down
2019-01-21 09:44:10,076 [qtp1790421142-25] INFO [o.a.z.s.q.Leader] - Shutdown called
java.lang.Exception: shutdown Leader! reason: quorum Peer shutdown
at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:650)
at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:1284)
at com.edb.ngen.comm.zookeeper.QuorumPeerMainExtended.stopQuorumPeer(QuorumPeerMainExtended.java:36)
at com.edb.ngen.comm.zookeeper.ZooKeeperHelper.stopZookeeperInstance(ZooKeeperHelper.java:951)
at com.edb.repserver.commands.CommandHandler.joinNetwork(CommandHandler.java:2950)
at com.edb.ngen.rest.resource.extension.commands.CommandsPlugin.joinNetwork(CommandsPlugin.java:684)
at com.edb.ngen.rest.resource.extension.commands.CommandsPlugin.doMethod(CommandsPlugin.java:85)
at com.edb.ngen.rest.resource.RemoteCallService.doCalledMethod(RemoteCallService.java:122)
at com.edb.ngen.rest.resource.RemoteCallService.joinNetwork(RemoteCallService.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)

at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)
2019-01-21 09:44:10,077 [qtp1790421142-25] INFO [o.a.z.j.MBeanRegistry] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id2,name1=replica.2,name2=Leader,name3=Connections,name4=192.168.184.192,name5=0x2000f0d06350002]
2019-01-21 09:44:10,077 [qtp1790421142-25] INFO [o.a.z.s.NIOServerCnxn] - Closed socket connection for client /192.168.184.192:58502 which had sessionid 0x2000f0d06350002
2019-01-21 09:44:10,077 [LearnerCnxAcceptor-/192.168.184.192:2882] INFO [o.a.z.s.q.Leader] - exception while shutting down acceptor: java.net.SocketException: Socket closed

 

 

 

 

 

More Warnings:

2019-01-21 09:43:57,992 [/192.168.184.192:3881] INFO [o.a.z.s.q.QuorumCnxManager] - Received connection reques t /192.168.184.194:45500
112 2019-01-21 09:43:58,019 [RecvWorker:3] WARN [o.a.z.s.q.QuorumCnxManager] - Connection broken for id 3, my id = 1, error =
113 java.io.EOFException: null
114 at java.io.DataInputStream.readInt(DataInputStream.java:392)
115 at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:1133)
116 2019-01-21 09:43:58,019 [RecvWorker:3] WARN [o.a.z.s.q.QuorumCnxManager] - Interrupting SendWorker
117 2019-01-21 09:43:58,021 [SendWorker:3] WARN [o.a.z.s.q.QuorumCnxManager] - Interrupted while waiting for messa ge on queue
118 java.lang.InterruptedException: null
119 at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(Abstr actQueuedSynchronizer.java:2014)
120 at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynch ronizer.java:2088)
121 at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
122 at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1215)
123 at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:75)
124 at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:1052)
125 2019-01-21 09:43:58,021 [SendWorker:3] WARN [o.a.z.s.q.QuorumCnxManager] - Send worker leaving thread id 3 my id = 1
126 2019-01-21 09:43:58,210 [/192.168.184.192:3881] INFO [o.a.z.s.q.QuorumCnxManager] - Received connection reques t /192.168.184.194:45502
127 2019-01-21 09:43:58,231 [RecvWorker:3] WARN [o.a.z.s.q.QuorumCnxManager] - Connection broken for id 3, my id = 1, error =
128 java.net.SocketException: Connection reset
129 at java.net.SocketInputStream.read(SocketInputStream.java:210)
130 at java.net.SocketInputStream.read(SocketInputStream.java:141)
131 at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
132 at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
133 at java.io.DataInputStream.readInt(DataInputStream.java:387)
134 at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:1133)
135 2019-01-21 09:43:58,231 [RecvWorker:3] WARN [o.a.z.s.q.QuorumCnxManager] - Interrupting SendWorker

Thanks! 

1 ACCEPTED SOLUTION

Accepted Solutions
EDB Team Member

Re: Error while configuring Two replication nodes with separate hosts using EDB replication server 7

Hi Nikhil,

 

We have checked the issue you were facing with xdb 7.0 beta version.

Initially we also received similar error as the one you were receiving for remote host:

 

[root@localhost ~]#  $EPRS_HOME/client/bin/runRepCLI.sh -joinnetwork -servername remoteService -host 172.16.106.137 -port 8082

Exception in thread "main" javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused (Connection refused)

at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287)

at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)

at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)

 

[root@localhost etc]# $EPRS_HOME/client/bin/runRepCLI.sh -joinnetwork -servername Server -host 172.16.106.137 -port 8082

Service Server has failed to register with the network.

 

 

After doing some changes such as 

  1. Modified the firewall setting, allowed the client ip.
  2. Modified the client.properties modified leader.host from localhost to server_ip   (this file is present at client machine /usr/edb-rs-7.0/client/etc)

From:

[root@localhost etc]# cat client.properties 

# represents IP address of the leader service

leader.host=localhost

# represents port of leader service

leader.port=8082

 

To: 

[root@localhost etc]# cat client.properties 

# represents IP address of the leader service

leader.host=172.16.106.137

# represents port of leader service

leader.port=8082

 

Finally the error got resolved:

 

[root@localhost etc]#  $EPRS_HOME/client/bin/runRepCLI.sh -joinnetwork -servername remoteService -host 172.16.106.137 -port 8082

Server is successfully added in the network.

 

 

==

Hope this resolves the issue you were facing. 

Please let us know if the issue still exists.

3 REPLIES 3
EDB Team Member

Re: Error while configuring Two replication nodes with separate hosts using EDB replication server 7

We will update you on this.

EDB Team Member

Re: Error while configuring Two replication nodes with separate hosts using EDB replication server 7

Hi Nikhil,

 

We have checked the issue you were facing with xdb 7.0 beta version.

Initially we also received similar error as the one you were receiving for remote host:

 

[root@localhost ~]#  $EPRS_HOME/client/bin/runRepCLI.sh -joinnetwork -servername remoteService -host 172.16.106.137 -port 8082

Exception in thread "main" javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused (Connection refused)

at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287)

at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)

at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)

 

[root@localhost etc]# $EPRS_HOME/client/bin/runRepCLI.sh -joinnetwork -servername Server -host 172.16.106.137 -port 8082

Service Server has failed to register with the network.

 

 

After doing some changes such as 

  1. Modified the firewall setting, allowed the client ip.
  2. Modified the client.properties modified leader.host from localhost to server_ip   (this file is present at client machine /usr/edb-rs-7.0/client/etc)

From:

[root@localhost etc]# cat client.properties 

# represents IP address of the leader service

leader.host=localhost

# represents port of leader service

leader.port=8082

 

To: 

[root@localhost etc]# cat client.properties 

# represents IP address of the leader service

leader.host=172.16.106.137

# represents port of leader service

leader.port=8082

 

Finally the error got resolved:

 

[root@localhost etc]#  $EPRS_HOME/client/bin/runRepCLI.sh -joinnetwork -servername remoteService -host 172.16.106.137 -port 8082

Server is successfully added in the network.

 

 

==

Hope this resolves the issue you were facing. 

Please let us know if the issue still exists.

Level 3 Adventurer

Re: Error while configuring Two replication nodes with separate hosts using EDB replication server 7

@ajinkya 

 

Thank you!