🔧 Pre-Upgrade Preparation
-
Backup & Snapshot
-
Take VM snapshots or backups of all nodes before starting.
-
Identify node roles:
-
WEBAPP node: is the node where webservices are running
-
PERSISTENCE node: is the node where Cassandra is running.
-
-
Based on the OS version, please download the necessary packages.
cat /etc/redhat-release
Packages
|
Package URL |
MD5 |
SHA1 |
Description |
|
|
|
VeridiumID Update packages archive containing all RPMs, for local update procedure RHEL8 |
|
|
|
|
VeridiumID Update packages archive containing all RPMs, for local update procedure RHEL9 |
Upgrade Methods
You can use either use:
A. Local Packages
-
Download and extract the package
export https_proxy=PROXY_IP:PROXY_PORT ## RHEL8, Rocky8 export https_proxy=PROXY_IP:PROXY_PORT wget --user NEXUS_USER --password NEXUS_PASSWORD \ https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/Veridium-3.9-update/veridiumid-update-packages-rhel8-13.0.73.zip TMP_DEST="/home/veridiumid/update390" unzip veridiumid-update-packages-rhel8-13.0.73.zip -d ${TMP_DEST} ## RHEL9, Rocky9 export https_proxy=PROXY_IP:PROXY_PORT wget --user NEXUS_USER --password NEXUS_PASSWORD \ https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/Veridium-3.9-update/veridiumid-update-packages-rhel9-13.0.73.zip TMP_DEST="/home/veridiumid/update390" unzip veridiumid-update-packages-rhel9-13.0.73.zip -d ${TMP_DEST} -
Run update commands (on WEBAPP first, then PERSISTENCE nodes):
TMP_DEST="/home/veridiumid/update390" sudo yum localinstall -y --disablerepo="*" ${TMP_DEST}/packages/veridiumid_update_procedure-13.0.73-20260818.x86_64.rpm sudo python3 /etc/veridiumid/update-procedure/current/updateVeridium.py --version 13.0.73 --rpm-path ${TMP_DEST}/packages/ sudo bash /etc/veridiumid/scripts/check_services.sh
B. YUM Repository
-
Ensure the repository has the package:
sudo yum list available veridiumid_update_procedure-13.0.73-20260818 -
Run update (one node at a time):
sudo yum clean metadata sudo yum install -y veridiumid_update_procedure-13.0.73 sudo python3 /etc/veridiumid/update-procedure/current/updateVeridium.py --version 13.0.73 --use-repo sudo bash /etc/veridiumid/scripts/check_services.sh
Post-Upgrade Steps
1. Data Migration to ELK
Run once on a Persistence node, only if updating from versions older then 3.8.1:
sudo bash /opt/veridiumid/migration/bin/migrate_to_elk.sh
2. Cassandra Upgrade (if 4.0.9 or v4.1.4) → 5.0.2
Check version on a Persistence node:
/opt/veridiumid/cassandra/bin/nodetool describecluster | grep -A1 "Database versions"
## if the version is 4.0.9 or 4.1.4, than update should be executed; the proper version is 5.0.2
Then upgrade all Persistences node (one node at a time):
If update is done with local packages:
##check status - all nodes should be up - the status "UN" should be for everynode
/opt/veridiumid/cassandra/bin/nodetool describecluster
/opt/veridiumid/cassandra/bin/nodetool status
TMP_DEST="/home/veridiumid/update390"
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/372/update_cassandra.sh ${TMP_DEST}/packages/
##check status - all nodes should be up again, in the cluster - the status "UN" should be for everynode
sudo /opt/veridiumid/cassandra/bin/nodetool status
sudo /opt/veridiumid/cassandra/bin/nodetool describecluster
If update is done with YUM repository:
##check status - all nodes should be up - the status "UN" should be for everynode
/opt/veridiumid/cassandra/bin/nodetool describecluster
/opt/veridiumid/cassandra/bin/nodetool status
## run on everynode
/opt/veridiumid/cassandra/bin/nodetool describecluster
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/372/update_cassandra.sh
##check status - all nodes should be up again, in the cluster - the status "UN" should be for everynode
sudo /opt/veridiumid/cassandra/bin/nodetool status
sudo /opt/veridiumid/cassandra/bin/nodetool describecluster
3. Create one Zookeeper Cluster and enable Read Only mode (Optional)
In Single DC implementation on Persistence nodes, only if updating from versions 3.7.X :
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/372/update_zookeeper_configuration.sh
In case of CDCR, if not already done, run the following procedure, to create one big cluster, with nodes from both datacenters. Previous command should not be executed in case of CDCR.
Also, in order to check if is already created cdcr, please run check_services command. If in zookepeer area are all zookeeper nodes, from both datacenters, it means that this procedure was already applied.
Before starting this configuration, make sure that you have connectivity on ports 2888 and 3888 between ALL persistence nodes.
To test the connectivity run the following commands:
on DC1:
nc -zv IPNODEDC2 2888
nc -zv IPNODEDC2 3888
It is critical important to run this command only if the zookeeper cluster is not already created. Also parameters -c and -e are very important, to skip cassandra and elasticsearch cluster creation.
## In second datacenter stop all webapp services and zookeeper services
ver_stop_webapp
service ver_zookeeper stop
## If you are unsure of the secondary datacenter name, run the following command on any node in the Secondary Datacenter:
## sudo grep DATACENTER_TAG /etc/default/veridiumid/ver_migration | awk -F'=' '{print $2}'
## run this command on primary datacenter on one node in webapp. This will add in zookeeper the proper configuration data
bash /etc/veridiumid/scripts/veridiumid_cdcr.sh -i -w IPWEB1,IPWEB2 -s IPPER1,IPPER2,IPPER3 -n DATACENTER_TO_BE_ADDED
## WHERE DATACENTER_TO_BE_ADDED is the secondary datacenter and IPWEB1,IPWEB2 and IPPER1,IPPER2,IPPER3 are the IPs from the secondary datacenter
## this comand will generate a line on the screen to which you need to add -c and -e parameters
## run this command on all persistence nodes in both datacenters starting with the secondary one -
## the script will create a large cluster containing the Zookeeper nodes in both datacenters and remove data from second DC
bash /etc/veridiumid/scripts/veridiumid_cdcr.sh -a -n DATACENTER_TO_BE_ADDED -z "ZOOKEEPER_PROPERTIES_BASE64" -c -e
## run this command on all webapp nodes in second datacenter
bash /etc/veridiumid/scripts/veridiumid_cdcr.sh -a -n DATACENTER_TO_BE_ADDED -z "ZOOKEEPER_PROPERTIES_BASE64" -c -e
4. ELK Stack Upgrade to 8.17.3 (Required)
Upgrade Elasticsearch, Kibana, and Filebeat (first persistence, then webapp), in case that it was not already updated.
Check version on Persistence nodes:
## check if version is now 8.17.3; if not, run the update procedure; if it is version 8.17.3, do nothing
sudo /opt/veridiumid/elasticsearch/bin/elasticsearch --version
If update is done using local packages
Run on all nodes, first persistence then webapp nodes:
## run below command on all nodes, first on persistance and then on webapp, one by one.
TMP_DEST="/home/veridiumid/update390"
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/380/update_elk_stack.sh ${TMP_DEST}/packages/
eops -l
## it is ok if the cluster is in red status
After all persistance nodes are updated to version 8.17.3, run the following command on all nodes (persistence + webapp) starting with persistence nodes
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/380/update_elk_stack.sh post
##check elasticsearch cluster status that is in green status.
check_services
Run only on one node, on webapp:
##run on one node:
/opt/veridiumid/migration/bin/elk_ops.sh --update-settings
If update is done using YUM repository
Run on all nodes, first persistence then webapp nodes:
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/380/update_elk_stack.sh
## here the elasticsearch cluster might be red
After all persistance nodes are updated to version 8.17.3, run the following command on all nodes (persistence + webapp) starting with persistence nodes
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/380/update_elk_stack.sh post
##check elasticsearch cluster status that is in green status.
Run only on one node, on webapp:
##run on one node:
/opt/veridiumid/migration/bin/elk_ops.sh --update-settings
🌐 Useful Repositories
-
RHEL8: https://veridium-repo.veridium-dev.com/repository/VeridiumRPM8/packages/
-
RHEL9: https://veridium-repo.veridium-dev.com/repository/VeridiumRPM9/packages/
RHEL8 MD5 of each package:
|
Package URL |
MD5 |
SHA1 |
Description |
|---|---|---|---|
|
9dbad333f98815200e893370e756f823 |
307c6b9fc4035134010806c60afb773e33330820 |
VeridiumID Admin Dashboard |
|
|
c9504e54e1e5cfc30a582d5198bf9fb0 |
8fd6bd4e971d6df748281377671f91f559466c48 |
VeridiumID migration tool |
|
|
607cc3f4da285d51b67f5ae60a1e0f12 |
9bc3a61263d2869aaa3e02f812b7ab7705aeb7dd |
VeridiumID Websec |
|
|
1428506f6ab1525490e1479aaa412c05 |
b360b8a2dc44a97f8e37da6bc20706b910185def |
VeridiumID Directory Service component |
|
|
15333e34fd4eb08b68ea8cac0340dd86 |
c45fa5acbe6855b8265f39e7a56c278df1095690 |
VeridiumID DMZ service |
|
|
dbeb67b4be8fe4a14290af3f9fff2008 |
6219e75a7dc1ae39187c43bad079b08c0e548424 |
VeridiumID Fido service |
|
|
a2a3daa05ed491ecc4775a846a1f21ed |
b7f226dbc606183fb092b9bdd33bc5d7739ae54e |
VeridiumID Open Policy Agent |
|
|
e0011435f6d7da7a60828e018f7094b9 |
3bd43586a0f42f9d27f6973949b106b6fb42773d |
VeridiumID Elasticsearch |
|
|
d6b658dfd3f543d96cc2380c547ba123 |
12d8517a5a4f128b1d91ed18690c966c2f540a21 |
VeridiumID Kibana |
|
|
40dd717ef03c7cb0a044af5a2370309c |
c879971ade6695062f349de36aec7a83bc0ead2a |
VeridiumID Zookeeper |
|
|
76ce8297fdac97082ca9d1550728fe90 |
08ae3e798fd01ba99a440e5f99e2a2868a42376a |
VeridiumID Cassandra |
|
|
c770dff1ba8b4f14d21bb6967a6e3162 |
b195005064e55301813f81f851ab84299a8a21ad |
VeridiumID Haproxy |
|
|
34eb4ba9664181f999268ade4e0a5b43 |
ecece74baaafc88ed9783abfaae1a2a67d53f653 |
VeridiumID Self Service Portal |
|
|
2cd3250ab3fb31c72f8ec2b3e04c3c1b |
d751bb35ede6d4b1450da7873481c869e703e687 |
VeridiumID Shibboleth Identity Provider |
|
|
8040837208e46fd4cac18d528589afa1 |
78b6c428f8838e554700299a7e813e3f72f98b15 |
VeridiumID Tomcat |
|
|
cc2440d006167b0280923c3dbbd96582 |
44982a4c85e3632d5f6b366ec645637a3258a135 |
VeridiumID Setupagent |
|
|
610f02864002a6ef9fd54518d3bb4ee3 |
68f1f1c48fc4aa132f6d89e8417810af158dc60f |
VeridiumID FreeRadius |
|
|
3915d1f9e1854d5478bb14525ffcec53 |
c70360ae65b9f187182328fa3fd296b04eca1680 |
VeridiumID 4F biometric library |
|
|
73deb14daff776d3691f5a9ad983491e |
e5f947b8d63ef30197a79cbf4f35feec0fb62790 |
VeridiumID VFace biometric library |
|
|
3768b92c77ad1f7ca21a856fd75f62ae |
9056ec4fb969bf922aaf72cb8224da6c7baaf686 |
Update scripts |
RHEL9 MD5 of each package:
|
Package URL |
MD5 |
SHA1 |
Description |
|---|---|---|---|
|
e9ed1719c0ba903b2c83df35cb562912 |
2b61f683050eb007e370da67e0bdcfe24b1e0eb9 |
VeridiumID Admin Dashboard |
|
|
8518aa544af5da7ad3c3c09ce5115bb4 |
f2ab54828b32187d085455f7f14affcb7f4eb791 |
VeridiumID migration tool |
|
|
d2acb9524824185dd0dc4638dcc3bfbe |
0c5426612da0fab7eb537a9aed66c184919cda83 |
VeridiumID Websec |
|
|
aab48ce29dfd47d6d2ac7b21ed3c8298 |
706a3862360c050c68ba1fc3dde3e48e19700994 |
VeridiumID Directory Service component |
|
|
5b8f7db60963ca2b5768d577f920974c |
282fdb482f5aaf4d02f7560bbf0df8ad7f4f89a6 |
VeridiumID DMZ service |
|
|
97a91c21225bb627e97e96037c54e779 |
e44494a94aaa384b951dcf980abd8eb62b77134a |
VeridiumID Fido service |
|
|
42772d29e0cabf24a5be4a3e1cbae3f8 |
4f0eea558b6614a4a45ed0138ab89e3f562cc183 |
VeridiumID Open Policy Agent |
|
|
de332884919f23b025419b047917b73b |
626e01d54ca73e714942082fbcd682561703f8f6 |
VeridiumID Elasticsearch |
|
|
92db1ada93eee4c6184b64b70a5ba770 |
7a84a01b6eb70b1625701019bf4162abc78eabfe |
VeridiumID Kibana |
|
|
3e9c6c289609e5fac6d7057b66dff859 |
bd0ff31e1776c8eb232c19b6fa7a1cf0aff51751 |
VeridiumID Zookeeper |
|
|
4e5d5e297752d71d55e26a5536775dde |
9acf7d2572f75a49131049d4bce6b6cbe9257efa |
VeridiumID Cassandra |
|
|
771319f3f4f357dfe64bfb0364e4ad13 |
113fc25ff2a16f0a192c55d12fff123b32209bdc |
VeridiumID Haproxy |
|
|
e1508f4a4110fd8c4c30c2376b78b0c2 |
13643a4e3db20702a8beece1cf331b605079f1a3 |
VeridiumID Self Service Portal |
|
|
dff505e94fb855a139df69be13de2a90 |
a377720d7d40fdd90d6d297ba6911df35359a597 |
VeridiumID Shibboleth Identity Provider |
|
|
2503233b46288d817c4bdbd18823b538 |
9a0427e04d4c79be3e72f71bb90832d7a8454064 |
VeridiumID Tomcat |
|
|
9a42269d50a86c939fca8d569d0bdf8d |
f0bebfefe84b2dd902783443a63bcda69766f045 |
VeridiumID Setupagent |
|
|
16c787859dcd24ab0cebfe9a5f26ac7c |
de2e9314479e1ce9152aa5bd775543cd9c6bb982 |
VeridiumID FreeRadius |
|
|
e8a1e46415f6e93743d636f6038ab5a1 |
58c42af29d806c4806a9e98cf6931e4ce9f61671 |
VeridiumID 4F biometric library |
|
|
06957d4414c89c851c0335b4831cf964 |
211092be977f640d38796d713e8ebfc50daf8f9b |
VeridiumID VFace biometric library |
|
|
3b535358b89216a4e4f2ffd978de96fe |
f5bea898efc13e220d4516e786542d080c64b45e |
Update scripts |