Upgrade VeridiumID from 3.5.4/3.6.x to v3.7.1

This document will provide a step by step procedure to upgrade to VeridiumID 3.7.1.

It is recommended to take a snapshot for the servers before update.

The procedure will provide information regarding both update methods:

using a configured YUM repository
using local packages

The update is done without downtime, one node at a time. Please start with WEBAPP nodes and after that do the installation on PERSISTENCE nodes. In order to have no impact, please take out from the balancer the node on which install is running.
WEBAPP node is a server where websecadmin is installed, PERSISTENCE node is a server where Cassandra is installed.

The application 3.x is compatible with 3.7.0 database schema.

Summary:

1) Download packages

2) Pre-requirements

3) Start Update

4) Post update steps

5) Other references

1) Download packages

Package URL	MD5	SHA1	Description
Update Packages Archive RHEL8	156c48527d0904b3e67096366883983c	7f114091c22f4d395a4fd3a3b88c453f3e82ba5c	VeridiumID Update packages archive containing all RPMs, for local update procedure RHEL8
Update Packages Archive RHEL9	9307644c9e7aa94f98576078664ffd7d	971d8f81dead4b0f868324e9bcfb139886b918d6	VeridiumID Update packages archive containing all RPMs, for local update procedure RHEL9

Download the package on the server and unzip it.

CODE

## download the package on each server; the below command can be used. Please fill in the proxy IP and username and password provided by Veridium.
## it is recommanded to execute these commands with the user that is going to do the installation.
## based on OS version, you have download the necessary package:
## check OS version, by running 
cat /etc/redhat-release
## RHEL8, Rocky8
wget --user NEXUS_USER --password NEXUS_PASSWORD -e use_proxy=yes -e http_proxy=PROXY_IP:PROXY_PORT https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/Veridium-3.7.1-update/veridiumid-update-packages-rhel8-11.1.10.zip
## RHEL9, Rocky9
wget --user NEXUS_USER --password NEXUS_PASSWORD -e use_proxy=yes -e http_proxy=PROXY_IP:PROXY_PORT https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/Veridium-3.7.1-update/veridiumid-update-packages-rhel9-11.1.10.zip

Other option is to upload the update package to local repository, based on the OS the client is using - RHEL7,8 or 9.

2) Pre-requirements

2.1) (MANDATORY) Outbound proxy update information

Starting with version 3.7, in order for the push notification to work on Android, the following URL should be whitelisted in proxy for outbound connections.

CODE

oauth2.googleapis.com

Until now the following 2 URLs were necessary to be whitelisted.

CODE

api.push.apple.com
fcm.googleapis.com

How to test:

CODE

export http_proxy=IP:PORT
export https_proxy=IP:PORT

curl https://api.push.apple.com:443
curl https://fcm.googleapis.com:443
curl https://oauth2.googleapis.com:443

##response 404 is fine, if it is received.

2.2) (MANDATORY) User requirements

We recommend using any user with sudo rights or root directly.

Python 3 must be installed. To check if you have a working Python 3 version run the following command:

CODE

python3 --version

If Python 3 is not installed, please see section 5.1 - How to install python 3

3) Start Update

Please execute all commands as root or with a user that has sudo privileges.

3.1) Update using local packages

Execute below commands on all nodes, first on WEBAPP and later on PERSITENCE nodes. Please execute the update one by one servers, not in parallel.

CODE

TMP_DEST="/home/veridiumid/update371"
#### please choose the one that apply, based on your OS:
##RHEL8
unzip veridiumid-update-packages-rhel8-11.1.10.zip -d ${TMP_DEST}
##RHEL9
unzip veridiumid-update-packages-rhel9-11.1.10.zip -d ${TMP_DEST}

Starting with version 3.6.0, it is used JAVA 11 version. Please install this package before the update:

CODE

## please check JAVA version
java --version
## PLEASE INSTALL JAVA 11 from local repositories, if not already installed; it should be OPENJDK distribution. Without this step the update will not be possible
sudo yum install java-11-openjdk -y
## if JAVA 8 is not the current java, please change it using below command, and select option for JAVA8
sudo update-alternatives --config java

After this, update application:

CODE

TMP_DEST="/home/veridiumid/update371"
sudo yum localinstall -y ${TMP_DEST}/packages/veridiumid_update_procedure-11.1.10-20241014.x86_64.rpm
sudo python3 /etc/veridiumid/update-procedure/current/preUpdateSteps.py --version 11.1.10 --rpm-path ${TMP_DEST}/packages/
sudo python3 /etc/veridiumid/update-procedure/current/startUpdate.py --version 11.1.10 --rpm-path ${TMP_DEST}/packages/
sudo bash /etc/veridiumid/scripts/check_services.sh

3.2) Update using a YUM repository

Starting with version 3.6.0, it is used JAVA 11 version. Please install this package before the update. This is necessary only for updates from a version older than 3.6.0.

CODE

## please check JAVA version
java --version
## PLEASE INSTALL JAVA 11 from local repositories, if not already installed; it should be OPENJDK distribution. Without this step the update will not be possible
sudo yum install java-11-openjdk -y
## if JAVA 8 is not the current java, please change it using below command, and select option for JAVA8
sudo update-alternatives --config java

Check if packages are visible in the repository. If the packages are not visible, please upload them into your repository, based on the OS you are using.

CODE

## check installed package
sudo yum list installed veridiumid_update_procedure
## check availability of the new package; if this package is not available, please fix the issue with the repository
sudo yum list available veridiumid_update_procedure-11.1.10-20241014

If the package is available, please execute below commands on all nodes, first on WEBAPP and later on PERSITENCE nodes. Please execute the update one by one servers, not in parallel.

CODE

sudo yum clean metadata
sudo yum install -y veridiumid_update_procedure-11.1.10
sudo python3 /etc/veridiumid/update-procedure/current/preUpdateSteps.py --version 11.1.10 --use-repo
sudo python3 /etc/veridiumid/update-procedure/current/startUpdate.py --version 11.1.10 --use-repo
sudo bash /etc/veridiumid/scripts/check_services.sh

4) Post update steps

4.1) This procedure will migrate all the data to Elasticsearch (devices, accounts) in order to have better reports. This is necessary only for updates from a version older than 3.7.0.

CODE

##please run it on a PERSISTENCE node
sudo bash /opt/veridiumid/migration/bin/migrate_to_elk.sh
##the revert option is:
#sudo bash /opt/veridiumid/migration/bin/migrate_to_elk.sh -x

4.2) After updating all nodes, please update Cassandra from 4.0.9 to 4.1.4 on persistence nodes. Please execute the update one by one servers, not in parallel. This procedure might be with a downtime until executed on all nodes. If Cassandra was updated in a previous version, than no update is needed.

If update is done with local packages:

CODE

/opt/veridiumid/cassandra/bin/nodetool describecluster
## if the version is 4.0.9, than update should be executed; the proper version is 4.1.4
TMP_DEST="/home/veridiumid/update371"
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/360/update_cassandra.sh ${TMP_DEST}/packages/
##check status
sudo /opt/veridiumid/cassandra/bin/nodetool status
sudo /opt/veridiumid/cassandra/bin/nodetool describecluster
## if number of devices are not seen in websecadmin, please recreate lucene indices, only on one node, after the full udpate
bash /opt/veridiumid/cassandra/conf/recreateCassandraLuceneIndexes.sh -c /opt/veridiumid/cassandra/conf/maintenance.conf

If update is done withing repo:

CODE

/opt/veridiumid/cassandra/bin/nodetool describecluster
## if the version is 4.0.9, than update should be executed; the proper version is 4.1.4
sudo bash /etc/veridiumid/update-procedure/current/resources/scripts/360/update_cassandra.sh
##check status and wait till it starts before going to next node
sudo /opt/veridiumid/cassandra/bin/nodetool status
sudo /opt/veridiumid/cassandra/bin/nodetool describecluster
## if number of devices are not seen in websecadmin, please recreate lucene indices
sudo bash /opt/veridiumid/cassandra/conf/recreateCassandraLuceneIndexes.sh -c /opt/veridiumid/cassandra/conf/maintenance.conf

4.3) If “Error message: [es/index] failed: [mapper_parsing_exception] failed to parse field [authenticationDeviceOsPatch] of type [date] in document with id“ error appears in bops.log, the bellow procedure should be applied

CODE

index=veridium.sessions-$(date '+%Y-%m')
/opt/veridiumid/migration/bin/elk_ops.sh --reindex --index-name=${index} --dest-index=${index}-001

5) Other references.

5.1) How to install python 3

In order to run the update procedure all nodes must have Python 3 installed .

To check if the VeridiumID Python 3 package (this is optional) is present use the following command as root:

CODE

## on RHEL7/Centos7 it should be used python 3.7
python3 --version
##Python 3.7.8
yum -y install python3.7
## on RHEL8/RHEL9 it should be used python3.9
sudo yum -y install python39 python39-pip
##Python 3.9.18

Veridium REPO LINKS:

Package URL	Description
https://veridium-repo.veridium-dev.com/repository/VeridiumRPM8/packages/	Repository with RHEL8 packages
https://veridium-repo.veridium-dev.com/repository/VeridiumRPM9/packages/	Repository with RHEL9 packages

RHEL8 MD5 of each package:

Package URL	MD5	SHA1	Description
WebsecAdmin	89e57d4549197508bae88e13000aaeb6	db2de5afbee10b02dfcb5561fb83ebdf700e6268	VeridiumID Admin Dashboard
Migration	5fa027f8d727783019ed3b8523af4b4e	aaa176bf7104d199fe4a889d5149162b992e05ec	VeridiumID migration tool
Websec	de90cff8baeb8576a745c92047742198	ee3543e33f2e4c04c8433b119efa56a214fe4423	VeridiumID Websec
AdService	3dc9495d54c5ec2ab1d5df533f80f8e3	80bf2f5b3bcdb7ebb450f6a6af89e2b8c38da2c7	VeridiumID Directory Service component
DMZ	e245bd1aab29802d252a4af430005715	9bfc584f607336a6116453dee3482568edc1d8e9	VeridiumID DMZ service
Data Retention	3b19c2c989377c8efb2687d9a63cb1b5	f4e91264239c9d4e8f6d1d5d4c5a23d7cb1d064d	VeridiumID Data Retention service
Fido	db66c3d60073288a02642db0e9425232	81b3fba563ad96b2219fd3c4bfbd7e86a08a693a	VeridiumID Fido service
OPA	35095272f28448f7809ef4504b7f021d	65f9a8355f854c267f7c2b4b1f8789b350223233	VeridiumID Open Policy Agent
Elasticsearch	2b2e574ed554a59efdc74e9d5278a9ea	c0213c4cdfca6bb2973501710702c868cb80a7bb	VeridiumID Elasticsearch
Kibana	2414ec757318bb62d3d1d61ac2cfab44	2e707d95f98c18a8c399a4f6ac017df9eafa9465	VeridiumID Kibana
Zookeeper	e1afb7572348c405f956c0b84d7f888c	5c02d6b2df62e276338c6da98cdbcaf9d4c5f18a	VeridiumID Zookeeper
Cassandra	e2cffcc14dd36a257c64b5415ebfea01	49b1fcd2a052259bab71b00cc651cbe8b80f9ba1	VeridiumID Cassandra
Haproxy	bac10f15ad8bdb1ac3d469693eb21f24	774cd13a82a979731739a061edc53c00f61a86b0	VeridiumID Haproxy
SelfServicePortal	84c218042e1b740268edba1d92c5f614	922388bcdde32b9187663abcf131c2eee816b179	VeridiumID Self Service Portal
Shibboleth	1166e331722a24f92703f288192839d5	77cf96c31ba130f2bd07e4ff0f179a5c00910849	VeridiumID Shibboleth Identity Provider
Tomcat	b784758f69725a3794a62ce4badd3706	58959ef5716229d3c82a8e7c69d248bdfbd507d9	VeridiumID Tomcat
Freeradius	513bd5fb018f42c5cd778faddcee52c0	b97b125ed603c2a60a2ae48bf3227a5697e63f9a	VeridiumID FreeRadius
4F	b73431c146290ce325dc9ab1c58a9169	cff1d54f90c574b2b4e20d9f086e9129315572a3	VeridiumID 4F biometric library
VFace	15d979cf509e19579cb53ee34b83599c	93d86eda5abb1a05284c0267fc578ae127f0c61c	VeridiumID VFace biometric library
Update procedure RPM	6b34c37667de4c422d202f6d4ae24c86	624d10287c1e1cde1b8f568cf1f2fbf3bb55f3bd	Update scripts

RHEL9 MD5 of each package:

Package URL	MD5	SHA1	Description
WebsecAdmin	54417db7151612f5a2debf26cad8fc6a	1ab911988cb86d197279a75e074c81256d2256ed	VeridiumID Admin Dashboard
Migration	90e01b38426504003878aa5117c97f09	01cadc542e35d0309d97e16727ae92bd24c3512c	VeridiumID migration tool
Websec	f8a69d47092ad084f349042110e9247f	d454adcfde5b6d24b005ba9766b528a3cd2ecfce	VeridiumID Websec
AdService	4156d91c7c143628d330cc81d7f98e6d	96b1c4f0ae96d40598f3086e2763bb25b9e8fd60	VeridiumID Directory Service component
DMZ	9330ec334d859882174e3e6f825171a7	0e9c941551d72e9b2be065317b3ebecf7db56069	VeridiumID DMZ service
Data Retention	997578c290483786692cfc28e1118dba	9687ef885886855da1c94d2e63e829ec6e736710	VeridiumID Data Retention service
Fido	957da9589fdadeecee06860fc519358d	a1d998d85f80239ae07dc8bee9494fe3ec4a3cc4	VeridiumID Fido service
OPA	90292c254df1e7f01313f204f8c18a7a	da10fd6b6250ce18a86d1f6bad92a823d0eada77	VeridiumID Open Policy Agent
Elasticsearch	4aadb28bb88f891bbece5617c5eb7e3b	c30b3a1a785ad6aae0becec583ea101300cfac3a	VeridiumID Elasticsearch
Kibana	6ea0c601aebb2cb3aa6fbde81d0a4df8	a28e8b3eb84ffed1cf00bbe675c94ef451eeec68	VeridiumID Kibana
Zookeeper	2860504c10489cab51c4cea6a8fbf987	bb55922d5af00f00cd57b6f4ccefddea8a34c211	VeridiumID Zookeeper
Cassandra	532809b78aff78b040bd231c02d87da9	7d226ea479eb8555d0cbcad441585d76edb916bc	VeridiumID Cassandra
Haproxy	67590364dd85ecfe3f8f3e6f609824a5	26eda0cbef4c4d753b3fc5375e60ebe53581ff5c	VeridiumID Haproxy
SelfServicePortal	44520b8bb824edf2b9562908a832568d	ee7d7b27a29e4f84711a0c803a127a0c4b851b66	VeridiumID Self Service Portal
Shibboleth	dde5ab6978b4d93b701054a5955f5516	b806f0941c264af59e0d86853859dce8b2acde09	VeridiumID Shibboleth Identity Provider
Tomcat	1b8c8d9eff218ca83fe2f9ba21327129	b1da4bd43513f7b5fd9d17ed1d2a68d283fa00f8	VeridiumID Tomcat
Freeradius	b8e698dd1faf5054e870a5ae33ab39c5	2116a9f57b43345b4bbc78c3275c0f4b2cdb201b	VeridiumID FreeRadius
4F	72306d011d5693851691813dd8c8f357	21691fe244adeed6eff949dfe4d8adb555060398	VeridiumID 4F biometric library
VFace	55ef80142ca1635b12f6d5092544d9e9	c6d4d3871d54aa983d101c0e633f82b39544aa71	VeridiumID VFace biometric library
Update procedure RPM	03aeec1048200a738b520776b0f7f330	25f3adf7685da51ac26aa65b0cd45a29d2430135	Update scripts