Breadcrumbs

General Investigation Guide - General Troubleshooting

Purpose

Use this article as a general troubleshooting flow when an issue occurs unexpectedly and the root cause is unknown.

This guide helps determine whether the issue is caused by:

  • degraded or stopped services

  • infrastructure problems (disk, memory, DNS, network, SSL)

  • new application errors/regressions

Troubleshooting flow

1) Check if services are running properly

Before analyzing logs, confirm that the platform is healthy.

1.1 Confirm service health

Actions

  1. Verify all services are running and stable.

    • bash /etc/veridiumid/scripts/check_services.sh

  2. Ensure no service is in stopped or starting state .

Expected result

  • All components are running.

  • No stopped/starting services.

1.2 Check disk space (Linux)

Disk full situations can break services unexpectedly.

Actions

  1. Check disk usage:

df -h

1.3 Websecadmin monitoring indicators

If available, validate system indicators in Websecadmin.

Actions

  • Websecadmin → Tools → monitoring

    • ensure indicators are green

Also check:

  • Tools → monitoring → websecadmin

    • accounts.commit.log.size, devices.commit.log.size, profiles.commit.log.size, sessions.commit.log.size

    • should be 0 or fluctuate then decrease

2) Check the ERROR logs (Websecadmin → Tools → Application logs)

2.1 Locate relevant ERRORs

Actions

  1. Log in to Websecadmin

  2. Go to:
    Tools → Application logs

  3. Filter by severity:

    • ERROR

  4. Trigger the failing scenario again (if possible).

  5. Identify:

    • first ERROR at failure timestamp

    • repeated errors

    • stack traces

Expected result

  • At least one ERROR log entry correlated with the failure.

3) Check if there are recent errors that were not earlier reported

Many incidents are caused by newly introduced issues.

Actions

  1. Review ERROR logs over a wider period (last hours/days).

  2. Look for:

    • new errors that didn’t exist earlier

    • increasing frequency

    • errors correlated with recent changes

  3. Capture evidence:

    • timestamps

    • full error messages

    • stack traces

Recommended

  • Regularly review errors to detect new issues


4) Collect logs and open incident

4.1 Collect VeridiumID server information

Run below command and attach the result on ticket. This contains veridium version, configurations, service status and ERRORs from the lgos.

Depending on the problem, it is recommended to run this on all nodes and provide the results.

/etc/veridiumid/scripts/gather_env_info.sh

4.2 Collect VeridiumID server logs

Run below command and attach the result on ticket. This contains all veridium logs from the current and last day on the server.

/etc/veridiumid/scripts/getLogs.sh

Last updated: