Performance Tests - on client server

Performance tests address several areas:

User registration
Login in SAML applications (e.g. SSP)
Login with CP

Requirements – short summary

A list of users created in AD for running the tests – recommended: 10,000–50,000 distinct users
A Linux machine with resources:
- For ~50000 users in parallel: 32 CPU, 32 GB RAM, 60 GB disk
- For ~10000 users in parallel: 16 CPU, 8 GB RAM is sufficient
The performance test server must have network access to Veridium Services and RAEP.
- Recommended:
  - Direct network access from Performance testing machine to WEBAPP servers and RAEP (application ports)
  - Network access from Performance testing machine to the load balancer in front of WEBAPPs
- This allows comparison between:
  - Direct application performance
  - Balancer + application performance

1. Veridium environment preparation for load testing

(Optional) In websecadmin, configure the SSP application to use implicitUpn as the NameID attribute.
Remove uid from “search by attributes” in LDAP, if it is defined.
Disable notifications during login/registration to avoid sending unwanted emails/SMS.
(Mandatory) In Veridium Manager (Websecadmin), configure the Enrollment process to have only one step → AD Enrollment, and disable User enrollment via invitation code from AD Enrollment (only for user registration).
Testing was done with the default journey and the following policies:
- TRUE for Native Biometric – Mobile, PIN
- All other policies can be OPTIONAL or FALSE.
Observation (load balancer stickiness):
- Stickiness must be guaranteed by the balancer for Shibboleth either:
  - based on IP, or
  - based on COOKIE.
- If IP-based stickiness is used:
  - Start tests from different servers, or
  - Send traffic directly to a specific WEBAPP server.
In RAEP, verify that caching is enabled:
C:\Program Files\VeridiumID\RAEPServer\RaWebApp\Web.config
CODE
```
<add key="CacheCerts" value="true" />
```
In Veridium Manager (Websecadmin) under LDAP → Search by Attributes, verify that all configured fields are actually used in AD.
- Do not leave unused attributes configured, especially those that are almost always N/A, because they can cause long-running searches and add load on AD.

In websecadmin config.json, verify:

CODE

"replaceDeviceWhenOneDevicePerAccount": false

2. Performance server preparation – RHEL 9

CODE

# Run the commands below as a sudo-capable user (e.g. root).
# 1. Install required packages
# 1.1 Use sudo if you are not root
sudo yum -y install unzip mlocate net-tools wget jq bind-utils java-17-openjdk
# 2. Install Docker, docker-compose, docker-cli
# Recommended Docker version: 24.0.5 (tested with 27.3.1)
sudo yum -y install docker-ce docker-ce-cli containerd.io
sudo systemctl enable --now docker.service
sudo curl -L "https://github.com/docker/compose/releases/download/v2.30.3/docker-compose-$(uname -s)-$(uname -m)" \
  -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# Finish chapter 3. Server requirements – continue with docker user and group permissions
sudo groupadd docker
sudo gpasswd -a $USER docker
sudo usermod -aG docker $USER

3. How to run tests

3.1 Download and unpack test suite

On the server(s) where tests will run, download the archive:

For version 3.8.4:
https://veridium-repo.veridium-dev.com/repository/VeridiumUtils/Veridium_3.8.4/bdd/perfTests.tar.gz

CODE

# Unpack
tar -xvf perfTests.tar.gz
# Load images and start local infrastructure for monitoring tests
bash ./perfTests/scripts/loadDockerImages.sh
# Check that local Docker containers are running – there should be 3 images running
docker ps
# To stop the local infrastructure if needed:
docker-compose -f ./dockerCompose/infra/docker-compose.yaml down

3.2 Configure monitoring (Grafana / Graphite)

Connect to: http://IP:8084
- User: admin
- Password: see ./perfTests/scripts/loadDockerImages.sh (look for reset-admin-password).
In the UI:
- Go to Home → Connections → Data sources → Graphite
- Change the URL to: http://IP:8084/
- Click Save & test.

3.3 Create the user list file

Create a file containing all usernames that will be used in testing and copy it to:
perfTests/automation-testing-runner-1.0.0-SNAPSHOT/bin/

Example content:

CODE

user_loadtest1@dev.local
user_loadtest2@dev.local
user_loadtest3@dev.local

3.4 Configure the environment file

Edit automation-testing-runner-1.0.0-SNAPSHOT/bin/envfile.json with your environment details:

CODE

{
  "enrollmentIntegrationId": "ADv2MultiStepEnrollment",
  "websecBaseUrl": "https://dev68.veridium-dev.com/websec",
  "dmzBaseUrl": "https://dev68.veridium-dev.com:8544/dmzwebsec",
  "adminBaseUrl": "https://dev68.veridium-dev.com:9444/websecadmin",
  "raBaseUrl": "https://dev-dc2.dev.local:446/RaWebApp",
  "epBaseUrl": "https://dev-dc2.dev.local:446/BopsEnroll/BopsEnroll.svc",
  "sp": {
    "login": "https://dev68.veridium-dev.com:9987/ssp/saml2/authenticate",
    "sso": "https://dev68.veridium-dev.com:8945/idp/profile/SAML2/POST/SSO",
    "status": "https://dev68.veridium-dev.com:8945/idp/profile/veridium-session-status"
  },
  "cassandra": {
    "host": "10.202.0.20",
    "port": 9042
  },
  "graphite": {
    "enabled": true,
    "host": "10.202.0.20",
    "port": 2003,
    "stepSec": 10
  },
  "deviceEnrollmentProxy": {
    "certificate": "MMMM=",
    "certificatePassword": "551fb169-c5c9-48a8-9f4c-de54bd069ae7"
  }
}

3.5 Run the tests

Go to the runner directory:

CODE

cd automation-testing-runner-1.0.0-SNAPSHOT/bin

For help, run:

CODE

cat README.txt
##Runner usage
./automation-testing-runner
usage: runner
    --env-file <env-file>             Provides support for specifying environment base path to run the automated tests.
    --periodic <periodic>             Interval in ms; 0 means a single cycle will be executed and then it will stop.
    --service-name <service-name>     Label used to differentiate different executions.
    --tags <tags>                     BDD tags to run:
                                      @enrolment, @saml-login, @cp-enrolment, @cp-login
    --tasks <tasks>                   Number of parallel tasks.
                                      Users from usernames-file will be divided between tasks.
    --username <username>             Single username used in tests.
                                      Use either --username or --usernames-file.
    --usernames-file <usernames-file> File with all usernames used in tests.

Recommended execution order (per-user, for validation)

CODE

# 1. Enroll a mobile device for a single user
./automation-testing-runner --service-name local --env-file envfile.json \
  --tags @mobile-device-enrolment --periodic 0 --username user_lt1000
# 2. Login (SAML, QR) for the same user
./automation-testing-runner --service-name local --env-file envfile.json \
  --tags @saml-login-qr --periodic 0 --username user_lt1000
# 3. Enroll CP device once, to register the CP certificate (equivalent for RAEP)
./automation-testing-runner --service-name local --env-file envfile.json \
  --tags @cp-device-enrolment --periodic 0
# 4. Run CP login tests (users already registered)
./automation-testing-runner --service-name local --env-file envfile.json \
  --tags @p-login-qr --periodic 0

Run tests for all users

CODE

# Enrolment for all users
nohup ./automation-testing-runner --service-name local --env-file envfile.json \
  --tags @mobile-device-enrolment --periodic 0 \
  --usernames-file users.txt --tasks 50 &
# SAML login tests (periodic)
nohup ./automation-testing-runner --service-name local --env-file envfile.json \
  --tags @saml-login-qr --periodic 1 \
  --usernames-file users.txt --tasks 50 &
# CP login tests (periodic)
nohup ./automation-testing-runner --service-name local --env-file envfile.json \
  --tags @cp-login-qr --periodic 1 \
  --usernames-file users.txt --tasks 50 &

To stop all running tests:

CODE

./stopTests.sh

Troubleshooting

Cassandra access

CODE

# List Docker containers
docker ps -a
# Connect to the Cassandra container
docker exec -ti $(docker ps -a | grep cassandra | awk -F' ' '{print $1}') bash
# Inside the container, connect to Cassandra:
cqlsh
# Example queries:
use veridiumbdd;
select * from enrolment_contexts;

Reset local Docker infrastructure

CODE

docker system prune -a
docker-compose -f dockerCompose/infra/docker-compose.yaml down
docker rm -f $(docker ps -a -q)
docker volume rm $(docker volume ls -q)
docker-compose -f dockerCompose/infra/docker-compose.yaml up -d

Java PKIX issues

If you encounter Java PKIX / certificate trust issues, make sure the following line exists in the automation-testing-runner script:

CODE

DEFAULT_JVM_OPTS="-Dlog4j.configurationFile=$(dirname $(realpath $0))/log4j2.xml -Dt