---
Tag: INFRA, OPS
---
Prometheus Server
Overview
To monitor the health and performance of services and infrastructure, and to alert users when issues are detected. It helps ensure system reliability and aids in debugging and capacity planning.
Covers time-series metrics collection, storage, querying (via PromQL), visualization, and alerting (via the Alertmanager).
DevOps Engineers, OPS, INFRA (Sysadmins) for instrumenting code and monitoring application-specific metrics.
Requironments
| RAM | CPU | Storage (Disk) | Network |
|---|---|---|---|
| 16 GB or more. | 4–8 cores (modern multi-core processor). | SSD Drive (mandatory). | 1 Gbps or 10 Gbps. |
Linux Operating System Requirement: Any modern Linux distribution (e.g., Ubuntu, Debian, CentOS, RHEL).
Docker Engine (28.4.0 or newer)
Docker Compose (v2.39.2 or newer)
Prometheus Server
Prometheus
Purpose: The heart of the system. It scrapes (pulls) metrics from targets, stores them in its Time Series Database (TSDB), and evaluates alerting rules. It requires mounting a configuration file (prometheus.yml).
Node Exporter:
Purpose: Collects essential host OS metrics (CPU, memory, disk I/O, network) from the Linux machine. This is crucial for infrastructure monitoring. You must typically run this with host network access or mount specific filesystem paths to get accurate host metrics.
Alertmanager (for Notifications)
Purpose: Receives alerts from Prometheus, groups them, dedupes them, and routes them to final notification receivers
Grafana (for Visualization)
Purpose: Creates dashboards and visualizes the data collected by Prometheus. Grafana connects to Prometheus as a data source and uses PromQL to query metrics.
Architecture
Gitlab Prometheus-Grafana project
https://qcgit-qcint.quintessence.de/net.quintessence.monitoring/prometheus-grafana/
Configuration Details
gitlab-project - prometheus.yml configs
| Parameter Type | Parameter Name | Description | Default Value | Example Value (from Snippet) |
|---|---|---|---|---|
| Ports | ports: 9090:9090 (Docker) |
Maps the container port 9090 (Prometheus UI/API) to the host port 9090 for external access. | None (Docker Compose requires explicit port mapping). | 9090:9090 |
| IP Ranges / Targets | targets (Config) |
IP addresses or DNS hostnames and ports of the services to scrape metrics from. | None (Must be configured per job). | qcmon-qcint-exporter:9100, alertmanager:9093 |
| API Port | static_configs targets (Internal) |
The internal port where the service exposes its metrics. | None (Typically 9100 for Node Exporter, 9090 for Prometheus). | 9105, 9093, 8085, 9115 |
| External URL | --web.external-url (CLI) |
The base URL for links in the UI and alerts; used when Prometheus is behind a reverse proxy/path prefix. | Empty string (/). |
http://qcmon-qcint.quintessence.de/prometheus/ |
| Alert Manager | alertmanagers targets (Config) |
The address and port of the Alertmanager service to send alerts to. | None. | alertmanager:9093 |
| Parameter | Default Value | Impact if Default is Used |
|---|---|---|
global.scrape_interval |
1 minute (1m) | All jobs would scrape every 1 minute. |
global.scrape_timeout |
30 seconds (30s) | All jobs would time out after 30 seconds. . |
rule_files |
Empty list | rules.yml, node_exporter_rules.yml, blackbox_alerts.yml . |
alerting.alertmanagers |
Empty list | alertmanager:9093. |
metrics_path (Job) |
/metrics |
The job would look for the metrics endpoint at /metrics. This is overridden in several jobs (e.g., /probe for blackbox jobs, /alertmanager/metrics for alertmanager). |
Step-by-Step Setup
- mkdir /home/prometheus-grafana (as root)
- cd /home/prometheus-grafana
- git clone https://qcgit-qcint.quintessence.de/net.quintessence.monitoring/prometheus-grafana.git
- docker compose up -d
Validation & Testing
Docker
linux command: docker ps -a
logs:
linux command: "docker logs prometheus"
Webinterface
local checking: curl -v localhost:9090
external (example): https://qcmon-qcint.quintessence.de/prometheus/
Troubleshooting
- docker compose restart
- docker compose down && docker compose up -d
Security Considerations
Under development
Maintenance
The auto-deployment of the Prometheus+Grafana project is based on the qcmon-qcint virtual machine.
References & Links
Web interface - https://qcmon-qcint.quintessence.de/prometheus/
Git-repo - https://qcgit-qcint.quintessence.de/net.quintessence.monitoring/prometheus-grafana
ssh connection to environment: ssh youruser:qcmon-qcint@qcbas-qcint.quintessence.de -p 2222
Change Log
Changelog
| Date | Author | Message |
|---|---|---|
| 2026-02-25 | aresnikowa | Merge remote-tracking branch 'origin/master' |