System metrics with Prometheus

Package Security Manager system performance can be monitored to understand system health, evaluate network traffic, and detect issues. Each of Package Security Manager’s services expose a set of metrics that can be visualized using the built-in Prometheus expression browser. Metrics are provided in OpenMetrics (Prometheus) format.

​

Accessing Prometheus

In your web browser, append /prometheus/ to your Package Security Manager fully qualified domain name (FQDN). For example: https://<FQDN>.com/prometheus/

​

Creating graphs for metrics

Prometheus uses a built-in expression browser for time series visualizations of system metrics.

To create system metric graphs in Prometheus:

1. Enter the name of an expression you want to view in the search box.

2. Select your expression from the list that appears.

   

3. Click Execute.

4. Click the Graph tab.

The graph is populated by the selected metric, and a console readout appears below it.

To isolate a specific resource, select it from the legend below the graph.

​

Popular metrics

​

up

The up time metric tells you if your instance is running.

​

process_open_fds

The process_open_fds metric counts the number of files in the /proc/PID/fd directory. This tells you how many regular files, sockets, pseudo terminals, and other processes you currently have open.

​

process_max_fds

The process_max_fds metric reads /proc/<PID>/limits and uses the Soft Limit from the Max Open Files row.

/limits lists both soft and hard limits. The soft limit is the value the kernel enforces for the corresponding resource, while the hard limit acts as the ceiling for the soft limit.

​

Setting a file limit alert

Using the two metrics above, process_open_fds and process_max_fds, you can write an alert to warn you when a process hits 80% of the limit:

groups:
- name: example
  rules:
  - alert: ProcessNearFDLimits
    expr: process_open_fds / process_max_fds > 0.8
    for: 10m