cancel
Showing results for 
Search instead for 
Did you mean: 

Multi-Master Replication Between PPCD/On-Prem And Cloud Database Service Cluster

Migration from PPCD/On-Prem Over CDS Using EPRS - MMR : Offline Snapshot

 

Most of us know EnterpriseDB has a software package for cross-database Multi-Master Replication(MMR) or Single-Master Replication(SMR) servers that supports an asynchronous replication system to replicate data to EDB Postgres Advanced Server(EPAS) and PostgreSQL. The initial software package name was xDB Replication Server (5.x) and subsequently converted into EDB xDB Replication Server(6.x) and today its called as EDB Postgres Replication Server(7.beta).

 

New  EDB Postgres Replication Server 7.Beta(EPRS), has been re-architected in the incorporation of message streaming using Apache Kafka for replication changed data between database. Apache Kafka is a distributed data streaming platform that can publish, subscribe to, store, and process streams of records in real-time. It is designed to handle data streams from multiple sources and deliver them to multiple consumers. Apache Kafka as the data streaming backbone in new EPRS to improve performance and provide enhanced failover capabilities.

 

We can use MMR capability for bi-directional replication from PPCD cluster to Cloud Database Services(CDS) and vise versa. By using MMR we don’t have to worry about moving incremental changes from CDS to PPCD in case of backout plan.



Prerequisites For Master/Source Database

 

  • Servers (CentOS 7.x)
    • EPRS-node1 
    • EPRS-node2 
    •  
  • Dependency softwares
    • Java 1.8 or later
    • Setup Bi-directional passwordless ssh for 'enterprisedb' user between 3 Nodes
    • Open firewall ports for 2181,2881,3881,8081,8082(tcp)

 

  • EDB Software Packages
    • EDB Postgres Advanced Server 10.x
    • EDB Postgres Replication Server 7.BETA

 

Source DB : Version: EPAS-10 on Centos 7(On PPCD or On Premesis)

 

Target DB: Version: EPAS-10 on Centos 7(On Cloud Database Services)



  1. “wal _level” should be logical.
  2. “track_commit_timestamp” should be on.
  3. “max_wal_sender” should be set with the appropriate value (Ex: 10).
  4. “Max_replication_slots” should be set with the appropriate value (Ex: 10).
  5. Table should have Primary Key.
  6. EPRS Port Should be open in Security Group. 

(i.e 2881,8082,3881,2182,3882,2181,2882,9092,8081)

  1. Database Port should be open. ( i.e 5432, 5444)
  2. Use only Public DNS name of Instance while adding the publication and Target clusters.
  3. Make appropriate pg_hba.conf entries. As, in case of Pgpool it might give the connections timeout error in terms of Idle connections of EPRS setup. MMR-Updated.png

     

 

Installation and Configuration Steps

 

  • Start EPRS Server.

cd /usr/edb/rs-7.0/server/bin

./runServer.sh --host ec2-34-255-48-133.eu-west-1.compute.amazonaws.com --config /usr/edb/rs-7.0/server/etc &

 

 

  • Execute “joinnetwork”.

cd /usr/edb/rs-7.0/client/bin

./runRepCLI.sh -joinnetwork -servername localService -host ec2-34-255-48-133.eu-west-1.compute.amazonaws.com -port 8082

 

 

  • Set administrator password

/usr/edb/rs-7.0/client/bin/runRepCLI.sh -setadminpassword -savepassword

 

 

  • Encrypt Database password

/usr/edb/rs-7.0/client/bin/runRepCLI.sh -encrypt -input /tmp/input -output /tmp/output -user admin 

 

 

  • Add Source Database to the Replication.

./runRepCLI.sh -adddb -servername localService -dbid db1 -dbtype enterprisedb -dbhost ec2-34-254-213-11.eu-west-1.compute.amazonaws.com -dbport 5444 -dbuser repuser -dbpassword wjOCadkp7oMLvKejd+xm6g== -database migration_test -user admin

 

 

  • Create Publication.

./runRepCLI.sh -createpub -pubname deptemppub -servername localService -dbid db1 -nodetype RW -alltables public -user admin

 

 

  • Start EPRS Server on Node-2.

cd /usr/edb/rs-7.0/server/bin

./runServer.sh --host ec2-18-200-225-14.eu-west-1.compute.amazonaws.com --config /usr/edb/rs-7.0/server/etc &

 

 

  • Execute “joinnetwork” command to join the Node-2. Make sure to execute this command from Node-1

./runRepCLI.sh -joinnetwork -servername remoteService -host ec2-18-200-225-14.eu-west-1.compute.amazonaws.com  -port 8082 -user admin

 

  • Make sure to execute this specific command on Node-1.

 

 

  • Add Target Database to the Replication.

./runRepCLI.sh -adddb -servername remoteService -dbid db2 -dbtype enterprisedb -dbhost ec2-18-200-225-14.eu-west-1.compute.amazonaws.com -dbport 5444 -dbuser repuser -dbpassword wjOCadkp7oMLvKejd+xm6g== -database migration_test -user admin

 

 

  • Join Target Database to the Publication.

./runRepCLI.sh -joinpub -servername remoteService -dbid db2 -pubname deptemppub -nodetype RW -user admin

 

 

  • Start Offline Snapshot.

./runRepCLI.sh -startsnapshot -pubname deptemppub -dbid db2 -offline -user admin

./runRepCLI.sh -checksnapshot -pubname deptemppub -dbid db2 -user admin

 

  • Once Offline snapshot is done, follow below process to configure Node-2.
  • Take a backup of the old cluster conf files and start Pg_basebackup.
  • Replace the cluster once the backup is done. Place the backed-up conf files and start the cluster. 

 

 

  • Start Streaming.

/usr/edb/rs-7.0/client/bin/runRepCLI.sh -startstreaming -pubname deptpub -user admin



I hope this article is useful for migrating data from your existing PPCD or on-prem cluster using EPRS-7 Multi-Master Replication. If you have any questions please feel free to comment. 








 

Version history
Revision #:
1 of 1
Last update:
3 weeks ago
Updated by:
 
Labels (1)
Contributors