Cross DataCenter Replication Setup Script

This is an alternative way to create a CDCR if there is ssh connectivity from one datacenter to second one.

Also it needs that the ssh key from one server is allowed on all servers.

The CDCR script performs the following operations:

CDCR Setup for VeridiumID 3.3.0+
- Layered (2W+3P) Deployments
- Custom Layered Deployments
ElasticSearch multi-DC Configuration
Zookeeper Synchronization

Other Operations:

CDCR Setup for older VeridiumID 3.0.0
- Layered (2W+3P) Deployments
- Custom Layered Deployments
CDCR Setup for VeridiumID 3.2.4
- Layered (2W+3P) Deployments
- Custom Layered Deployments

1. Prerequisites

The CDCR Script can be found within the VeridiumID Installer Archive, for version 3.3.0 or newer.

The cdcr-setup.py script uses the Veridium Python 3.6.8 package.

The Ansible playbook uses the “current“ user’s SSH key. You can also use a custom id_rsa key. Follow the Script’s help Extras section.

2. CDCR Script

Script Archive Contents

CODE

cdcr-setup/
├── ansible
│   ├── inventory <-- Inventory File
│   └── tasks
│       ├── cdcr-files
│       │   └── placeholder.txt <-- Dummy File
│       ├── main.yaml
│       └── test.yaml <-- Custom/Test Installations Only
├── ansible.cfg
└── cdcr-setup.py

3. CDCR Script Help

To access the script’s help run:

CODE

python3 cdcr-setup.py

For larger Infrastructures, we recommend configuring the Ansible Inventory File 7 by using the --inventory option.

CODE

Command Options:
        -h, --help -> Get the argument definitions

        -i, --inventory -> Use pre-configured inventory file -> Inventory File Location: ../cdcr-setup/ansible/inventory

        -t, --test-connection -> Check Network connectivity to the hosts from DC1 & DC2

        -o, --option -> CDCR Configuration options

CDCR Configuration Options:
        veridium-kafka -> Start CDCR Procedure for VeridiumID < 3.3.0

        veridium-elk -> Start CDCR Procedure for VeridiumID > 3.3.0 & multi-dc Elasticsearch configuration

        elastic -> Start multi-dc Elasticsearch configuration only

        zookeeper -> Synchronize Zookeeper directories between DataCenters

Command Examples:
- For checking network connectivity, run:
        Option 1, using pre-configured inventory file: python3 cdcr-setup.py -t --inventory
        Option 2: python3 cdcr-setup.py -t --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3

- Start CDCR Procedure for VeridiumID < 3.3.0, run:
        Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o veridium-kafka --inventory
        Option 2: python3 cdcr-setup.py -o veridium-kafka --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3

- Start CDCR Procedure for VeridiumID > 3.3.0, run:
        Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o veridium-elk --inventory
        Option 2: python3 cdcr-setup.py -o veridium-elk --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3

- To start the multi-DC Elasticsearch configuration, run:
        Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o elastic --inventory
        Option 2: python3 cdcr-setup.py -o elastic --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3

- To Synchronize Zookeeper directories between DataCenters, run:
        Option 1, using pre-configured inventory file: python3 cdcr-setup.py -o zookeeper --inventory
        Option 2: python3 cdcr-setup.py -o zookeeper --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3

Extras
- Using a different SSH Key and not the default one from /home/<USER>/.ssh/id_rsa
        If you want to use another SSH Key, you will have to simply copy it next to cdcr-setup.py and rename it as cdcr_id_rsa.
        You can simply execute the cdcr-setup.py script as shown in the Command Examples section and the script will automatically check to see if there is any key named cdcr_id_rsa found at the aforementioned location.

4. CDCR Script Examples

4.1 CDCR

After configuring the Ansible Inventory File, you can start the installation using the following command:

CODE

python3 cdcr-setup.py -o veridium-elk --inventory

For triggering the installation using the complete command, run:

CODE

python3 cdcr-setup.py -o veridium-elk --dc1-name DC1_NAME --dc2-name DC2_NAME --dc1-webapp DC1_WEBAPP_IP1,2 --dc2-webapp DC2_WEBAPP_IP1,2 --dc1-persistence DC1_PERSISTENCE_IP1,2,3 --dc2-persistence DC2_PERSISTENCE_IP1,2,3

4.2 ElasticSearch

The CDCR default Installation also includes the configurations for a multi-dc Elastic Cluster.

For starting a separate installation, run:

CODE

python3 cdcr-setup.py -o elastic --inventory

4.3 Zookeeper Sync

For Syncing the Zookeeper Configuration between Data Centers, run:

CODE

python3 cdcr-setup.py -o zookeeper --inventory

5. Misc Operations

5.1 Installation Guide - VID 6.5.30

For VeridiumID 6.5.30 version & older, you will need the CDCR Setup Archive.

Official Download URL: https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/VeridiumID-CDCR/VeridiumID_CDCR.zip
The archive needs to be placed inside the VeridiumID Installer directory. To extract the files, please use the following command, on first machine in first datacenter (please login to all webapps and see from where it was installed the application → see logs folder). Unzip to /home/verdiumid:
- CODE
```
unzip VeridiumID_CDCR.zip
```