Cross DataCenter Replication Setup Script
This is an alternative way to create a CDCR if there is ssh connectivity from one datacenter to second one.
Also it needs that the ssh key from one server is allowed on all servers.
The CDCR script performs the following operations:
CDCR Setup for VeridiumID 3.3.0+
Layered (2W+3P) Deployments
Custom Layered Deployments
ElasticSearch multi-DC Configuration
Zookeeper Synchronization
Other Operations:
CDCR Setup for older VeridiumID 3.0.0
Layered (2W+3P) Deployments
Custom Layered Deployments
CDCR Setup for VeridiumID 3.2.4
Layered (2W+3P) Deployments
Custom Layered Deployments
1. Prerequisites
The CDCR Script can be found within the VeridiumID Installer Archive, for version 3.3.0 or newer.
The cdcr-setup.py script uses the Veridium Python 3.6.8 package.
The Ansible playbook uses the “current“ user’s SSH key. You can also use a custom id_rsa key. Follow the Script’s help Extras section.
2. CDCR Script
Script Archive Contents
cdcr-setup/
├── ansible
│ ├── inventory <-- Inventory File
│ └── tasks
│ ├── cdcr-files
│ │ └── placeholder.txt <-- Dummy File
│ ├── main.yaml
│ └── test.yaml <-- Custom/Test Installations Only
├── ansible.cfg
└── cdcr-setup.py
3. CDCR Script Help
To access the script’s help run:
python3 cdcr-setup.py
For larger Infrastructures, we recommend configuring the Ansible Inventory File 7 by using the --inventory
option.
Command Options:
-h, --help -> Get the argument definitions
-i, --inventory -> Use pre-configured inventory file -> Inventory File Location: ../cdcr-setup/ansible/inventory
-t, --test-connection -> Check Network connectivity to the hosts from DC1 & DC2
-o, --option -> CDCR Configuration options
CDCR Configuration Options:
veridium-kafka -> Start CDCR Procedure for VeridiumID < 3.3.0
veridium-elk -> Start CDCR Procedure for VeridiumID > 3.3.0 & multi-dc Elasticsearch configuration
elastic -> Start multi-dc Elasticsearch configuration only
zookeeper -> Synchronize Zookeeper directories between DataCenters
Command Examples:
- For checking network connectivity, run:
Option 1, using pre-configured inventory file: python3 cdcr-setup.py -t --inventory
Option 2: python3 cdcr-setup.py -t --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3
- Start CDCR Procedure for VeridiumID < 3.3.0, run:
Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o veridium-kafka --inventory
Option 2: python3 cdcr-setup.py -o veridium-kafka --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3
- Start CDCR Procedure for VeridiumID > 3.3.0, run:
Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o veridium-elk --inventory
Option 2: python3 cdcr-setup.py -o veridium-elk --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3
- To start the multi-DC Elasticsearch configuration, run:
Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o elastic --inventory
Option 2: python3 cdcr-setup.py -o elastic --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3
- To Synchronize Zookeeper directories between DataCenters, run:
Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o zookeeper --inventory
Option 2: python3 cdcr-setup.py -o zookeeper --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3
Extras
- Using a different SSH Key and not the default one from /home/<USER>/.ssh/id_rsa
If you want to use another SSH Key, you will have to simply copy it next to cdcr-setup.py and rename it as cdcr_id_rsa.
You can simply execute the cdcr-setup.py script as shown in the Command Examples section and the script will automatically check to see if there is any key named cdcr_id_rsa found at the aforementioned location.
4. CDCR Script Examples
4.1 CDCR
After configuring the Ansible Inventory File, you can start the installation using the following command:
python3 cdcr-setup.py -o veridium-elk --inventory
For triggering the installation using the complete command, run:
python3 cdcr-setup.py -o veridium-elk --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3
4.2 ElasticSearch
The CDCR default Installation also includes the configurations for a multi-dc Elastic Cluster.
For starting a separate installation, run:
python3 cdcr-setup.py -o elastic --inventory
4.3 Zookeeper Sync
For Syncing the Zookeeper Configuration between Data Centers, run:
python3 cdcr-setup.py -o zookeeper --inventory
5. Misc Operations
5.1 Installation Guide - VID 6.5.30
For VeridiumID 6.5.30 version & older, you will need the CDCR Setup Archive.
Official Download URL: https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/VeridiumID-CDCR/VeridiumID_CDCR.zip
The archive needs to be placed inside the VeridiumID Installer directory. To extract the files, please use the following command, on first machine in first datacenter (please login to all webapps and see from where it was installed the application → see logs folder). Unzip to /home/verdiumid:
- CODE
unzip VeridiumID_CDCR.zip