Remove datacenter from CDCR

This article will provide a step by step guide to removing a datacenter from a Cross Datacenter deployment.

During this article we will use the names ‘DC1’ and ‘DC2’ for the two datacenters and the datacenter we will remove will be ‘DC2’.

Variable	Value
Script name	remove_datacenter.sh
Script location	/etc/veridiumid/scripts (On Persistence nodes)
Root user required	YES

1. Prepare CDCR configuration

In this step we will create the configuration file that will be used to remove the second datacenter.

Run the following command on a persistence node in DC1:

CODE

bash /etc/veridiumid/scripts/remove_datacenter.sh -g

The user will then be asked the following:

Example configuration file:

CODE

DC1_PERSIST=10.0.10.1,10.0.10.2,10.0.10.3
DC1_NAME=dc1
DC2_PERSIST=10.0.20.1,10.0.20.2,10.0.20.3
DC2_NAME=dc2
REMOVE_DC=dc2

After generating this file, copy it to all other persistence nodes in DC1, or run the same command on all nodes and generate the file.

2. Stop services in second datacenter

Connect to all nodes in the second datacenter and stop all services by running the following command as root:

CODE

bash /etc/veridiumid/scripts/veridium_services.sh stop
# Or by using the alias
ver_stop

3. Remove datacenter from ElasticSearch

Connect to each persistence node in DC1 and run the following command, one node at a time:

CODE

bash /etc/veridiumid/scripts/remove_datacenter.sh -e -f CONFIG_PATH
# Where CONFIG_PATH is the full path of the configuration file generated during the first step

To check the status of the cluster run the following command:

CODE

bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -x=GET -p=/_cat/health?v

4. Remove datacenter from Cassandra configuration

Connect to a single persistence node in DC1 and run the following command in order to remove nodes in DC2 and alter the keyspaces:

CODE

bash /etc/veridiumid/scripts/remove_datacenter.sh -c -m -f CONFIG_PATH
# Where CONFIG_PATH is the full path of the configuration file generated during the first step

During this step a full repair will be triggered. This will take some time depending on the size of the keyspace.

Connect to each persistence node in DC1 and run the following command in order to modify configuration and restart Cassandra service, one node at a time:

CODE

bash /etc/veridiumid/scripts/remove_datacenter.sh -c -f CONFIG_PATH
# Where CONFIG_PATH is the full path of the configuration file generated during the first step

To check the status of the cluster after running the command on all nodes, run the following:

CODE

/opt/veridiumid/cassandra/bin/nodetool status