The Prometheus monitoring system.
Questions tagged [prometheus]
171 questions
29
votes
6 answers
How to find out the number of time series stored in Prometheus LevelDB
i'm responsible for maintaining the Prometheus servers in our company. The metrics however are provided by the teams.
Is there a way to find out the number of time series stored in the Prometheus datadase? We are using the default LevelDB data…
15
votes
2 answers
How to split out Prometheus config files?
Right now we're using Prometheus for our monitoring and we have a lot of config (our prometheus.yml main config file is 1400+ lines long).
I would like to split this out into logical groupings (maybe DEV/TEST/PROD?) but I can't seem to find any…
srkiNZ84
- 581
11
votes
1 answer
Prometheus: no metric node_cpu
I've setup Prometheus and node_exporter. All example graphs I've found utilize node_cpu but I don't have this metric present. The closest I've come is node_cpu_seconds_total. Am I missing something or is my version of Prometheus incompatible with…
Chris
- 115
9
votes
2 answers
Prometheus Alert Rule for Absent Discovered Target
I'm trying to write a general rule to fire alert when a discovered target goes missing. In particular kubernetes pods annotated for scraping and auto-discovered using kubernetes_sd_configs.
Expressions of the form:…
Budric
- 233
6
votes
1 answer
Prometheus - Use case of service discovery by role endpoints and role pod in Kubernetes
While reading Prometheus Configuration documentation and some sample scrape configurations, I found some kubernetes_sd_configs with role service & role endpoints & role pod
- job_name: kube-state-metrics
kubernetes_sd_configs:
- role: endpoints
…
jiangwei
- 61
- 1
- 4
5
votes
1 answer
Monitor Gitea with Prometheus
I have deployed Gitea. How can I want to monitor it with Prometheus ?
I have tried to scape the /metrics but I get no metrics (HTTP/404)
Franklin Piat
- 878
5
votes
2 answers
Prometheus: scrape interval is 1m, but resolution is still 15s
tl;dr: My scrape interval is 1m, yet I have a 15s resolution. Why?
My prometheus configuration includes a job to scrape kong metrics:
- job_name: kong_blue
honor_timestamps: true
scrape_interval: 1m
scrape_timeout: 10s
metrics_path:…
aspyct
- 380
5
votes
2 answers
Smoothing Grafana graph
I'm setting up montinoring on my company infrastructure. I installed Grafana with the Prometheus Datasource. I built my dashboards and have rather "Spiky" graphs. I was wondering if there was any way to smooth it up a bit so it would be more…
Ror
- 383
5
votes
4 answers
Netdata, prometheus and grafana - why recommended in this constellation?
Currently I do setup monitoring-service for our Server.
I've found some articles, which recommend to use: Netdata, prometheus and grafana
So I wonder what are the benefits to use all three of them.
Can anybody explain it to me, why prometheus and…
suther
- 173
5
votes
1 answer
Prometheus node_exporter metrics - huge spikes in Grafana
Our setup includes:
a few Debian 9.12 nodes with Prometheus node_exporter v. 0.18.1 installed as service
Prometheus server v. 2.14.0 (on Windows Server 2016) scraping the metrics from nodes
Grafana visualizing metrics
Basically, our load could be…
Arseny V.
- 95
5
votes
1 answer
Do prometheus metrics have some sort of "freshness" lifetime?
I'm scraping some metrics (openstack cinder volume sizes) every 15 minutes, and the results produce a discontinuous graph, like this:
(That's the result of the simple query cinder_volume_size_gb).
The metrics "exist" for about five minutes, but…
larsks
- 47,453
4
votes
2 answers
Debugging Prometheus OOMkilled despite 6Gi limits
I'm at the end of my patience with a prometheus setup leveraging kube-prometheus-stack 44.3.0 (latest being 45).
I have two environments, staging and prod. In staging, my prometheus runs smoothly. In prod it has started crashing with OOMKilled…
Liquid
- 151
4
votes
1 answer
Why does my prometheus dashboard only show 2 weeks of data when I have longer term data available?
I have Prometheus configured with Thanos. I'm using the Node Exporter Full dashboard to view data from the node exporter. Regardless of what time range I select (e.g., now-20d), the dashboard only shows 2 weeks of data:
If I go to the "Explore" UI…
larsks
- 47,453
4
votes
3 answers
How do I configure the log level of Prometheus' Alertmanager?
I've set up Prometheus' Alertmanager to, well, manage alerts from Prometheus. I've got the alerts coming in from Prometheus to Alertmanager, but there the story ends. The alerts never get sent on by Alertmanager to my email endpoint.
In order to…
Rovanion
- 659
- 3
- 7
- 22
3
votes
2 answers
Disk usage metrics missing from Prometheus node exporter
I used Prometheus and node exporter a while ago and had access to node_filesystem_* metrics to monitor disk usage but I've recently fired it up on some other servers (Ubuntu Linux) and those metrics seem to be missing.
According to this…
Sam
- 183