-
How to export/import Cassandra data:
##From where we take the backup: ## run repair on one node nodetool clearsnapshot --all nodetool repair veridium ## run compact on that node nodetool compact veridium ## run backup bash /opt/veridiumid/backup/cassandra/cassandra_backup.sh -c=/opt/veridiumid/backup/cassandra/cassandra_backup.conf ## take it and move it to new datacenter: cd /opt/veridiumid/backup/cassandra tar -czf backup.tar.gz 2024-01-06_04-00 mv backup.tar.gz /tmp/ ##On target, uncompress the backup and go to specific location: tar -xvf backup.tar.gz ## go to veridium subfolder cd 2024-01-06_04-00/dc1_10.70.60.248/ ## do truncate of all tables: ARRAY=( `/opt/veridiumid/cassandra/bin/cqlsh --cqlshrc=/opt/veridiumid/cassandra/conf/veridiumid_cqlshrc --ssl -e "use veridium; desc tables;"` ) TABLES="${ARRAY[@]}" for table in $TABLES; do /opt/veridiumid/cassandra/bin/cqlsh --cqlshrc=/opt/veridiumid/cassandra/conf/veridiumid_cqlshrc --ssl -e "truncate table veridium.${table};"; done ## move the DB files to proper location ## (*it might be that some folders don't have db files, so it it ok if no file is to be copied) for var in `ls -1 veridium/`; do table=`echo ${var%-*}`; mv veridium/${var}/snapshots/*/*db veridium/${var}/; done ##load files in cassandra veridium: for var in `ls -1 veridium/`; do table=`echo ${var%-*}`; /opt/veridiumid/cassandra/bin/sstableloader -f /opt/veridiumid/cassandra/conf/cassandra.yaml -d 10.69.50.27 -p 9042 ./veridium/${table}-*/; done ##if necessary, modify MAX_HEAP_SIZE in /opt/veridiumid/cassandra/bin/sstableloader ## for newer cassandra: for var in `ls -1 veridium/`; do table=`echo ${var%-*}` while true; do echo "=== Importing table '$table'... ===" nodetool import veridium ${table} /tmp/2024-06-03_09-51/dc1_10.64.0.196/veridium/${table}-*/ if [ $? -eq 0 ]; then echo "=== Table '$table' imported successfully! ===" break fi echo "=== Sleeping for 30 seconds, then retrying table '$table' ===" sleep 30 done done -
How to export/import Elasticsearch data:
##It can be taken the last backup from ##Or it can be created a new one, with current date (even with events from today) bash /opt/veridiumid/migration/bin/elk_ops.sh --backup --page-size=1000 --mode=DATA --request-timeout=5 --connect-timeout=1 --parallel-tasks=2 --date-to=`date -d '+1 day' +%Y-%m-%d` --dir=/tmp/exportElastic ##Copy the files /tmp/exportElastic to new server and import them: bash /opt/veridiumid/migration/bin/elk_ops.sh --restore --dir=/tmp/elasticDevelop -
How to export/import Zookeeper data:
##export data /opt/veridiumid/migration/bin/migration.sh -d zkDownload ##change the IP NEW_IP="10.69.50.27";IP="10.70.60.248"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done NEW_IP="10.69.51.162";IP="10.70.61.181"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done NEW_IP="10.69.52.9";IP="10.70.62.122"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done ##change also other specific configuration files: NEW_IP="dev6.veridium-dev.com";IP="develop.veridium-dev.com"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done elasticsearch.json config.json -> cassandra Connector, friendCertificateSuffixByDataCeneter, and other parameters ## upload data to the new machine /opt/veridiumid/migration/bin/migration.sh -u zkDownload -
Other configuration files:
## set the same client-ca: /opt/veridiumid/haproxy/conf/client-ca.pem ##regenerate OPA certificate and replace it in the secrets. ##change the proxy secred in the following files with the value from onprem /opt/veridiumid/haproxy/conf/haproxy.cfg /opt/veridiumid/tomcat/conf/context.xml ##load idp credentials /opt/veridiumid/shibboleth-idp/credentials/ sealer.jks idp-signing.crt ##if needed, truststore_root.ks and keystore_root.ks, used in /opt/veridiumid/tomcat/conf/server.xml.