General architecture and design

Context

As a solution, we are proposing a deployment procedure of ILP, as zips directly on the infrastructure used already by VeridiumID server. Key concepts:

the VeridiumID server is already deployed on client’s infrastructure
the ILP will come as an add-on deployment, if the clients wants it (context/motion/context-motion)
the ILP deployment will use the persistence of VeridiumID server (zookeeper/cassandra/elasticsearch)
minimum modifications for client’s infrastructure

General architecture and design

ILP will be deployed in two additional machines on the webapp layer. This machines will require certain based on number of concurrent users resources (CPU and RAM). The machine will be in the same network as the VeridiumID webapp and persistence layer (same firewall rules, same CIDR block). This machines required the following:

to be in number of 1 (for non-HA deployment) or 2 (for HA-deployment)
the image/base will be the same as VeridiumID server (the same OS)
will required 8-16 CPU cores and 16-32 GB RAM
will require Java11 and Python 3..8.18 installed

For smaller companies (1-5 cc users) it is recommended to go for 8CPU,16GB RAM servers, while for medium-big companies (5-15 cc users) it is recommended to go for 16CPU, 32GB RAM servers.

General micro-service design for ILP integration with VID-server

General micro-service design for Motion pipeline

General micro-service design for context pipeline

Deployment design with VeridiumID server

The java8 is required to run java-based micro-services (this is the common-layer for both context and motion):

web-models (GRPC API microservice)
web-ingestion (GRPC API microservice)
web-tenant (GRPC API microservice)
web-users (GRPC API microservice)
cli-server (administrative operations server microservice)
cli(the administrative cli tool, not a fully running microservice) --- only installed not will be started with systemd.
models-cleanup (integration microservice)
processors-users (integration microservice)
processors-errors (integration microservice)
processor-completion (integration microservice)
processor-persistence (integration microservice)
processor-training (integration microservice)

The python3.8 is required to run python-based micro-services (this is separated by context-only and motion-only):

context-only:
- context-features-extraction-processor
- context_models_inference_processor
motion-only
- features_extraction_processor
- motion_preprocess_processor
- user_motion_inference_processor
- user_motion_trainer_processor
- fastvit_features_extraction_processor

General deployment design

Networking and storage

The following will apply for storage:

ILP will use cassandra from VeridiumID server, using a separated keyspace uba (in CDCR if the case), and it will be connected via SSL (instead of plain-text, code changes required here)
ILP will use zookeeper from VeridiumID server, using a specific path /uba/version/* in zookeeper file system, and it will be connected via username/password (code changes required here)
ILP will use kafka , connected on ports 9192/9195 (with ssl connection)
ILP will use elasticsearch from VeridiumID server, using the apiKey/username/password as in VeridiumID server (instead on default ILP certificate-based authentication and authorization, code changes required here)

The following will apply for networking:

ILP uses service-discovery as explained in uba-platform.json file
on each ILP machine (webapp-ilp), it is required to open the following ports: 8100 (web-models), 8220 (web-ingestion), 8120(web-tenant), 8200(web-users), 8110 (cli-server)
one each persistence machine, it is required to open the following ports: 9192/9195 (kafka)
on each webapp veridiumid machine, the following DNS must be solved to any of the ILP-webapps machines:
- CODE
```
ingestion.DOMAIN_SUFFIX
models.DOMAIN_SUFFIX
cli.DOMAIN_SUFFIX
tenant.DOMAIN_SUFFIX
users.DOMAIN_SUFFIX
```
on each ILP-webapp machines, the following DNS must be solved to localhost:
- uba-web-models
- uba-cli-server
- uba-web-tenant
- uba-web-users
- uba-web-ingestion
on each ILP-webapp machines, it should be configured a SSL-terminated on haproxy using the domain certificate using the public domains:
CODE
```
ingestion.DOMAIN_SUFFIX
models.DOMAIN_SUFFIX
cli.DOMAIN_SUFFIX
tenant.DOMAIN_SUFFIX
users.DOMAIN_SUFFIX
```

which will balance traffic to the ILP-webapp machines all the specific ports.

Integration with veridiumid-server

To initialize the plaform, we must use uba-cli to initialize the tenant:

CODE

./uba-cli help
./uba-cli create-index-template-elasticsearch --cdcr=true
./uba-cli tenants register --tenant "{\"tenant_id\":\"79257e79-ae13-4d3d-9be3-5970894ba386\",\"name\":\"Tenant name\",\"description\":\"This is the tenant dedicated to x environment.\",\"enabled\":true,\"timestamp_update\":1658245422000}" --compact-version true

The integration of the veridiumid-server will be made using SSL based on the certificate of ILP (domain certificate which is put in haproxy). On each ILP-webapp machine, we will have a haproxy server, which for

CODE

ingestion.DOMAIN_SUFFIX
models.DOMAIN_SUFFIX
cli.DOMAIN_SUFFIX
tenant.DOMAIN_SUFFIX
users.DOMAIN_SUFFIX

frontends, will do ssl-termination based on domain certificate, and forward them to the backend servers (ILP-webapp1 & ILP-webapp2) on certain ports.

The same configuration must be applied in VeridiumID server (as per SaaS k8s deployment:

use compact inference: true
inference timeout: 4
tenantId: tenantId use to initialize platform
UBA subdomain separator: .
UBA cluster suffix: domain suffix
async: true
enabled: true

Operability and observability

The application logs can be found in the following location: /var/log/veridiumid/uba/ . Here there are logs for each micro-service with a log-rotation policy of 30days. The following scripts can be used for operability and testing:

CODE

#run the following command to see if everything  running (all services without uba-cannary):
uba_check_services
# UBA WebAPP
# to see service status on webapp
uba_check_services
uba_check_kafka
# to stop/start services
uba_stop
uba_start
# VeridiumID Persistence
# to see status on persistence, for UBA services:
uba_check_services
uba_check_kafka
# to stop uba specific services, just kafka
systemctl stop uba-kafka
systemctl start uba-kafka
# to check logs of the platform:
less/vim/cat/tail /var/log/veridiumid/uba/log_file_of_service.log # where log_file_of_service is the file you want to see.

For tracing you can use jaeger UI which can be accessed on ILP_WEBAPP_IP1:16686 or ILP_WEBAPP_IP2:16686 .