Upgrade VeridiumID from 3.5.4/3.6.x to v3.7.0
This document will provide a step by step procedure to upgrade to VeridiumID 3.7.0.
It is recommended to take a snapshot for the servers before update.
The procedure will provide information regarding both update methods:
using a configured YUM repository
using local packages
The update is done without downtime, one node at a time. Please start with WEBAPP nodes and after that do the installation on PERSISTENCE nodes. In order to have no impact, please take out from the balancer the node on which install is running.
WEBAPP node is a server where websecadmin is installed, PERSISTENCE node is a server where Cassandra is installed.
The application 3.5.4/3.6.x is compatible with 3.7.0 database schema.
Summary:
1) Download packages
2) Pre-requirements
3) Start Update
4) Post update steps
5) Other references
1) Download packages
Package URL | MD5 | SHA1 | Description |
---|---|---|---|
bdb3f30aca2ef2b86ec8c5a575b268b4 | b4abab59befbca068ad3fb4a087ad7b2ce221e0d | VeridiumID Update packages archive containing all RPMs, for local update procedure RHEL8 | |
1cab317f4d23ce5d94d7cee0c7d63675 | 27e3fdedb3f885d7936d36e8c405bd849b563cb4 | VeridiumID Update packages archive containing all RPMs, for local update procedure RHEL9 |
Download the package on the server and unzip it.
## download the package on each server; the below command can be used. Please fill in the proxy IP and username and password provided by Veridium.
## it is recommanded to execute these commands with the user that is going to do the installation.
## based on OS version, you have download the necessary package:
## check OS version, by running
cat /etc/redhat-release
## RHEL8, Rocky8
wget --user NEXUS_USER --password NEXUS_PASSWORD -e use_proxy=yes -e http_proxy=PROXY_IP:PROXY_PORT https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/Veridium-3.7.0-update/veridiumid-update-packages-rhel8-11.0.39.zip
## RHEL9, Rocky9
wget --user NEXUS_USER --password NEXUS_PASSWORD -e use_proxy=yes -e http_proxy=PROXY_IP:PROXY_PORT https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/Veridium-3.7.0-update/veridiumid-update-packages-rhel9-11.0.39.zip
Other option is to upload the update package to local repository, based on the OS the client is using - RHEL7,8 or 9.
2) Pre-requirements
2.1) (MANDATORY) Outbound proxy update information
Starting with version 3.7, in order for the push notification to work on Android, the following URL should be whitelisted in proxy for outbound connections.
oauth2.googleapis.com
Until now the following 2 URLs were necessary to be whitelisted.
api.push.apple.com
fcm.googleapis.com
How to test:
export http_proxy=IP:PORT
export https_proxy=IP:PORT
curl https://api.push.apple.com:443
curl https://fcm.googleapis.com:443
curl https://oauth2.googleapis.com:443
##response 404 is fine, if it is received.
2.2) (MANDATORY) User requirements
We recommend using any user with sudo rights or root directly.
Python 3 must be installed. To check if you have a working Python 3 version run the following command:
python3 --version
If Python 3 is not installed, please see section 5.1 - How to install python 3
3) Start Update
Please execute all commands as root or with a user that has sudo privileges.
3.1) Update using local packages
Execute below commands on all nodes, first on WEBAPP and later on PERSITENCE nodes. Please execute the update one by one servers, not in parallel.
TMP_DEST="/home/veridiumid/update370"
#### please choose the one that apply, based on your OS:
##RHEL8
unzip veridiumid-update-packages-rhel8-11.0.39.zip -d ${TMP_DEST}
##RHEL9
unzip veridiumid-update-packages-rhel9-11.0.39.zip -d ${TMP_DEST}
Starting with version 3.7.0, JAVA 11 version is used.
In case of updating from version 3.5.4 to 3.7 please run the following commands to install JAVA 11, otherwise ignore the JAVA installation steps:
## please check JAVA version
java --version
## PLEASE INSTALL JAVA 11 from local repositories, if not already installed; it should be OPENJDK distribution. Without this step the update will not be possible
sudo yum install java-11-openjdk -y
## if JAVA 8 is not the current java, please change it using below command, and select option for JAVA8
sudo update-alternatives --config java
After this, update application:
TMP_DEST="/home/veridiumid/update370"
sudo yum localinstall -y ${TMP_DEST}/packages/veridiumid_update_procedure-11.0.39-20240918.x86_64.rpm
sudo python3 /etc/veridiumid/update-procedure/current/preUpdateSteps.py --version 11.0.39 --rpm-path ${TMP_DEST}/packages/
sudo python3 /etc/veridiumid/update-procedure/current/startUpdate.py --version 11.0.39 --rpm-path ${TMP_DEST}/packages/
sudo bash /etc/veridiumid/scripts/check_services.sh
3.2) Update using a YUM repository
Starting with version 3.7.0, it is used JAVA 11 version. Please install this package before the update:
## please check JAVA version
java --version
## PLEASE INSTALL JAVA 11 from local repositories, if not already installed; it should be OPENJDK distribution. Without this step the update will not be possible
sudo yum install java-11-openjdk -y
## if JAVA 8 is not the current java, please change it using below command, and select option for JAVA8
sudo update-alternatives --config java
Check if packages are visible in the repository. If the packages are not visible, please upload them into your repository, based on the OS you are using.
## check installed package
sudo yum list installed veridiumid_update_procedure
## check availability of the new package; if this package is not available, please fix the issue with the repository
sudo yum list available veridiumid_update_procedure-11.0.39-20240918
If the package is available, please execute below commands on all nodes, first on WEBAPP and later on PERSITENCE nodes. Please execute the update one by one servers, not in parallel.
sudo yum clean metadata
sudo yum install -y veridiumid_update_procedure-11.0.39
sudo python3 /etc/veridiumid/update-procedure/current/preUpdateSteps.py --version 11.0.39 --use-repo
sudo python3 /etc/veridiumid/update-procedure/current/startUpdate.py --version 11.0.39 --use-repo
sudo bash /etc/veridiumid/scripts/check_services.sh
4) Post update steps
4.1) This procedure will migrate all the data to Elasticsearch (devices, accounts) in order to have better reports.
##please run it on a PERSISTENCE node
sudo bash /opt/veridiumid/migration/bin/migrate_to_elk.sh
##the revert option is:
#sudo bash /opt/veridiumid/migration/bin/migrate_to_elk.sh -x
4.2) After updating all nodes, please update Cassandra from 4.0.9 to 4.1.4 on persistence nodes. Please execute the update one by one servers, not in parallel. This procedure might be with a downtime until executed on all nodes. If Cassandra was updated in a previous version, than no update is needed.
If update is done with local packages:
/opt/veridiumid/cassandra/bin/nodetool describecluster
## if the version is 4.0.9, than update should be executed; the proper version is 4.1.4
TMP_DEST="/home/veridiumid/update370"
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/370/update_cassandra.sh ${TMP_DEST}/packages/
##check status
sudo /opt/veridiumid/cassandra/bin/nodetool status
sudo /opt/veridiumid/cassandra/bin/nodetool describecluster
## if number of devices are not seen in websecadmin, please recreate lucene indices, only on one node, after the full udpate
bash /opt/veridiumid/cassandra/conf/recreateCassandraLuceneIndexes.sh -c /opt/veridiumid/cassandra/conf/maintenance.conf
If update is done withing repo:
/opt/veridiumid/cassandra/bin/nodetool describecluster
## if the version is 4.0.9, than update should be executed; the proper version is 4.1.4
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/360/update_cassandra.sh
##check status and wait till it starts before going to next node
sudo /opt/veridiumid/cassandra/bin/nodetool status
sudo /opt/veridiumid/cassandra/bin/nodetool describecluster
## if number of devices are not seen in websecadmin, please recreate lucene indices
sudo bash /opt/veridiumid/cassandra/conf/recreateCassandraLuceneIndexes.sh -c /opt/veridiumid/cassandra/conf/maintenance.conf
4.3) If “Error message: [es/index] failed: [mapper_parsing_exception] failed to parse field [authenticationDeviceOsPatch] of type [date] in document with id“ error appears in bops.log, the bellow procedure should be applied
## this procedure needs to be applied, only if the following error appears in bops.log
## Error message: [es/index] failed: [mapper_parsing_exception] failed to parse field [authenticationDeviceOsPatch] of type [date] in document with id
############################################
index=veridium.sessions-$(date '+%Y-%m')
## copy data from current index to a new index
bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -x=PUT -p=/${index}/_block/write
bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -x=POST -p=/_reindex?wait_for_completion=false -d='{"source":{"index":"'${index}'"},"dest":{"index":"'${index}'_00001"}}'
##use the TASKIS returned by this command, to run the next command and wait to have competion=true
bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -x=GET -p=/_tasks/TASKID_RETURNED_FROM_PREVIOUS_COMMAND
## flush indices and check if the number of entries are the same; if there is the same number of events, remove the current index
bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -x=POST -p=/veridium.*/_flush
bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -l | grep ${index}
## remove current index
bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -x=DELETE -p=/${index}
## just in case of issues, to unblock an index:
###bash /opt/veridiumid/elasticsearch/bin/elasticsearch_ops.sh -x=PUT -p=/${index}_00001/_settings -d='{"index":{"blocks.write":false}}'
4.4) If “Error Forbiden operation appears when accessing SSP”, the following workaround should be applied, and accessed once ok the admin console.
bash /etc/veridiumid/update-procedure/current/resources/scripts/370/reinitApp370.sh
4.5) If fido is used, and some devices can not be enrolled (below error appears), please go to websecadmin -> FIDO Relying Parties and Enable Allow unknown extensions.
java.lang.NullPointerException at com.veridiumid.webauthn.model.AuthenticatorData.verifyExtensions(AuthenticatorData.java:113)
5) Other references.
5.1) How to install python 3
In order to run the update procedure all nodes must have Python 3 installed .
To check if the VeridiumID Python 3 package (this is optional) is present use the following command as root:
## on RHEL7/Centos7 it should be used python 3.7
python3 --version
##Python 3.7.8
yum -y install python3.7
## on RHEL8/RHEL9 it should be used python3.9
sudo yum -y install python39 python39-pip
##Python 3.9.18
Veridium REPO LINKS:
Package URL | Description |
---|---|
https://veridium-repo.veridium-dev.com/repository/VeridiumRPM8/packages/ | Repository with RHEL8 packages |
https://veridium-repo.veridium-dev.com/repository/VeridiumRPM9/packages/ | Repository with RHEL9 packages |
RHEL8 MD5 of each package:
Package URL | MD5 | SHA1 | Description |
---|---|---|---|
7a180a53844578f0c9f3a9bcd8291f79 | 72333291827a2367ca925417530ef81498c6169f | VeridiumID Admin Dashboard | |
24e450c91b1c3c547c421f67fda50ea5 | c23c66460fb358dc50475f04313e080fb6d04bcc | VeridiumID migration tool | |
0d926e3c456ab5d26abe851be83390da | bfc6313549ae48c90a2143fffe6056926d9dd865 | VeridiumID Websec | |
b033ecb68b3fbeae5a2262f8967791a7 | 1bc2dfc0aa220b30e095d43b96a3391bf6a87f43 | VeridiumID Directory Service component | |
1b0fa0c57591a340d31caef3e461b8d9 | 5e2e6e108158f0d14abba591d38fd7a6ff66db36 | VeridiumID DMZ service | |
a838a442272c536e30679d212c7f6ff1 | b8887683480b4bc9b80f5b8218b7ed9be3ef1c7b | VeridiumID Data Retention service | |
28b872a9bcc0204777659abf7cb16aa2 | 782d6536ef7abb948acd6464a7ec2df1c9a8b9e3 | VeridiumID Fido service | |
a079cf478e44ddcb28925b9a5ed74ba8 | c48969802502d592c90af9fca93c7ca3bd86ff71 | VeridiumID Open Policy Agent | |
b7664bd729bc3560253b803e95b81cc5 | 511597e1efcda1a2add4523faf5d37c828f2b7aa | VeridiumID Elasticsearch | |
046237cf41bf2f7701377ba71d6ce429 | 5757b16eac6b6fd7b97bff7f94daaa0d9988c0e9 | VeridiumID Kibana | |
d64174b11eec1043bacf36e8309e0b6a | 621632f23a0e151ebb2ff6567a324056c41bb2cc | VeridiumID Zookeeper | |
0a4dd8b9ec12444c234ec6ef7c54b557 | c7e3f0b5cdc5984d20ef18c6e634c5d1fcf54881 | VeridiumID Cassandra | |
366afb00c40c5320ec6f0aab31c86173 | de6eea76556ca8869abe6ee516b64d3fa1940f6e | VeridiumID Haproxy | |
d3d7249b35f7550c52e5fb6a8da2f4ee | 98c175de6d1c95e834ccf112c650bda1bc6f7b19 | VeridiumID Self Service Portal | |
4578e23987583c39107b84a51354ab90 | 7b4e985455c234fd578d0a3a4ae7c10102dd68b0 | VeridiumID Shibboleth Identity Provider | |
47392c6907f447ae02eecf92d50138da | 9b06f325f377f71e0dafd52afe032edd3d21472c | VeridiumID Tomcat | |
f4dd65fd473993381ac43d1359791b96 | 773b71af57516facf6d3fb0c5857b61b65edeb27 | VeridiumID FreeRadius | |
ae397bf4d3b4f1f32bc527eb4a30632b | c30bab73ee27d01f13d246bfed995b55a4f4b466 | VeridiumID 4F biometric library | |
9e69cf7a46ce3468354fa37c9a5511d7 | a318e4a80f06b509308bb021eeb36509412319e6 | VeridiumID VFace biometric library | |
03b9cf76e73ac9a5b78a68b8d9fb4da6 | 9d320517ac8f9a97ed6dcddf38cf94c55b487157 | Update scripts |
RHEL9 MD5 of each package:
Package URL | MD5 | SHA1 | Description |
---|---|---|---|
0990d826c90c510db83345b663d41f3e | 9161a5602e56337bfa73a0174c2b1c1fb66a68d7 | VeridiumID Admin Dashboard | |
77d66dbf4229abdee9230387ac1fefbf | 7eeed7104f41b6a24015c0813ebe8dc16b81a7e1 | VeridiumID migration tool | |
321b55d49fe84c82f11df56f63e46d5f | fbd99451db05c2ac995c5b6ff25d7fcecb93d874 | VeridiumID Websec | |
9c60d356cf4192646669cb18246199fc | b26a8e42875b6c5588d20ba2b519ce99b0f52ef6 | VeridiumID Directory Service component | |
a47b147a34c3e8df20108832b3fe900f | f12b87969cbb6e919ab40e77a3075123be3b9909 | VeridiumID DMZ service | |
921b9c366039f19c21bf74c47d38c31d | c86d162b992f1de0e4e04ef0b5d884c98c1d36ea | VeridiumID Data Retention service | |
5a0ea5f4d0cf9c1ee59d77254300e336 | b196f8fbe9caf87121d813af832f2483b01642e7 | VeridiumID Fido service | |
95a898132bc3ad480c8963529f00f660 | ccd9b821e66e88a9fa632c213645ba68d1adf4ec | VeridiumID Open Policy Agent | |
2cac46d0600b6cdf110f61fb2d698b90 | f19cf5b3920ff81b5117d7668464b028e5e6f6a5 | VeridiumID Elasticsearch | |
1e3a4a757bc0469aff610f62c58d08aa | 8122a60f6409ed29ad726c921c722dd871ab0dea | VeridiumID Kibana | |
610e31232d985e81a0670c5d4bd704f6 | a3f13b24d9bb053b41aff0be16e374c4943158f5 | VeridiumID Zookeeper | |
2adff03da82f15fad5d15a658bf39542 | adfd54dd84e3b50a39e6225284f631f81eddbeb7 | VeridiumID Cassandra | |
f863c98b817fa680513e7c96de0a05fb | 094254f6ce79a29d88a14cc96ee194459431eebc | VeridiumID Haproxy | |
097b67e0cab7c064cb5ede40194b1e56 | 4057ec664c5519a9e809abac36d562cbec426dbb | VeridiumID Self Service Portal | |
03843f3828f68cb2a2c98db5d6a09356 | 449bdaded6bb1d52988d37824b2fb523ca0a6144 | VeridiumID Shibboleth Identity Provider | |
b735bc43201417e0b87bc078d045b2e2 | 55c76ae66f6953eca97d6c45620d433b39cfd41e | VeridiumID Tomcat | |
1cd977543049a7f17b16fd237d6e0750 | 3a67c5c3d0805434c0acea4d55695beec53ef613 | VeridiumID FreeRadius | |
57e348d481bf663ee5cf14d92cd0448b | 5c1257d6de074bad439b07ac640248a0e451bf0d | VeridiumID 4F biometric library | |
525b82bbdba4bb3c76f2a0b52d5b5739 | 871a249e4c07e3f83ebf0b817ba811262534ba4a | VeridiumID VFace biometric library | |
448a1276f8c7777a11a775b2671ce5ad | 48bdc5af147170f05d5e337d35b6ca449fbd19c1 | Update scripts |