Skip to main content
Skip table of contents

Migrate data from onprem to container, using the same CA

  1. How to export/import Cassandra data:

    CODE
    ##From where we take the backup:
    ## run repair on one node
    nodetool clearsnapshot --all
    nodetool repair veridium 
    ## run compact on that node
    nodetool compact veridium 
    ## run backup
    bash /opt/veridiumid/backup/cassandra/cassandra_backup.sh -c=/opt/veridiumid/backup/cassandra/cassandra_backup.conf
    ## take it and move it to new datacenter:
    cd /opt/veridiumid/backup/cassandra
    tar -czf backup.tar.gz 2024-01-06_04-00
    mv backup.tar.gz /tmp/
    
    ##On target, uncompress the backup and go to specific location:
    tar -xvf backup.tar.gz
    ## go to veridium subfolder
    cd 2024-01-06_04-00/dc1_10.70.60.248/
    
    ## do truncate of all tables:
    ARRAY=( `/opt/veridiumid/cassandra/bin/cqlsh --cqlshrc=/opt/veridiumid/cassandra/conf/veridiumid_cqlshrc --ssl -e "use veridium; desc tables;"` )
    TABLES="${ARRAY[@]}"
    for table in $TABLES; do /opt/veridiumid/cassandra/bin/cqlsh --cqlshrc=/opt/veridiumid/cassandra/conf/veridiumid_cqlshrc --ssl -e "truncate table veridium.${table};"; done
    
    ## move the DB files to proper location
    ## (*it might be that some folders don't have db files, so it it ok if no file is to be copied)
    for var in `ls -1 veridium/`; do table=`echo ${var%-*}`; mv veridium/${var}/snapshots/*/*db veridium/${var}/; done
    
    ##load files in cassandra veridium:
    for var in `ls -1 veridium/`; do table=`echo ${var%-*}`; /opt/veridiumid/cassandra/bin/sstableloader -f /opt/veridiumid/cassandra/conf/cassandra.yaml -d 10.69.50.27 -p 9042 ./veridium/${table}-*/; done
    
    ##if necessary, modify MAX_HEAP_SIZE in /opt/veridiumid/cassandra/bin/sstableloader
    
    ## for newer cassandra:
    for var in `ls -1 veridium/`; do 
      table=`echo ${var%-*}`
      while true; do
        echo "=== Importing table '$table'... ==="
    	nodetool import veridium ${table} /tmp/2024-06-03_09-51/dc1_10.64.0.196/veridium/${table}-*/
        if [ $? -eq 0 ]; then
          echo "=== Table '$table' imported successfully! ===" 
          break
        fi
        echo "=== Sleeping for 30 seconds, then retrying table '$table' ==="
        sleep 30    
      done
    done
    
  2. How to export/import Elasticsearch data:

    CODE
    ##It can be taken the last backup from
    ##Or it can be created a new one, with current date (even with events from today)
    bash /opt/veridiumid/migration/bin/elk_ops.sh --backup --page-size=1000 --mode=DATA --request-timeout=5 --connect-timeout=1 --parallel-tasks=2 --date-to=`date -d '+1 day' +%Y-%m-%d` --dir=/tmp/exportElastic
    ##Copy the files /tmp/exportElastic to new server and import them:
    bash /opt/veridiumid/migration/bin/elk_ops.sh --restore --dir=/tmp/elasticDevelop
    
  3. How to export/import Zookeeper data:

    CODE
    ##export data
    /opt/veridiumid/migration/bin/migration.sh -d zkDownload
    
    ##change the IP
    NEW_IP="10.69.50.27";IP="10.70.60.248"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done
    NEW_IP="10.69.51.162";IP="10.70.61.181"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done
    NEW_IP="10.69.52.9";IP="10.70.62.122"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done
    
    ##change also other specific configuration files:
    NEW_IP="dev6.veridium-dev.com";IP="develop.veridium-dev.com"; LIST_OF_FILE=( `grep -FlR "${IP}" *` ); for file in ${LIST_OF_FILE[@]}; do sed -i "s|${IP}|${NEW_IP}|g" ${file}; done
    elasticsearch.json
    config.json -> cassandra Connector, friendCertificateSuffixByDataCeneter, and other parameters
    
    ## upload data to the new machine
    /opt/veridiumid/migration/bin/migration.sh -u zkDownload
  4. Other configuration files:

    CODE
    ## set the same client-ca:
    /opt/veridiumid/haproxy/conf/client-ca.pem
    
    ##regenerate OPA certificate and replace it in the secrets.
    
    ##change the proxy secred in the following files with the value from onprem
    /opt/veridiumid/haproxy/conf/haproxy.cfg
    /opt/veridiumid/tomcat/conf/context.xml
    
    ##load idp credentials
    /opt/veridiumid/shibboleth-idp/credentials/
    sealer.jks
    idp-signing.crt
    
    ##if needed, truststore_root.ks and keystore_root.ks, used in /opt/veridiumid/tomcat/conf/server.xml.
    
JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.