i'm responsible for maintaining the Prometheus servers in our company. The metrics however are provided by the teams.
Is there a way to find out the number of time series stored in the Prometheus datadase? We are using the default LevelDB data storage. I need these values to find out if i need to tweak the local storage settings of our Prometheus instances.
Thanks for your help.
How about count({__name__=~".+"})?
It does return number of time series in the database. I compared with amount of metrics currently exposed by each target by manual scrapping, and it matches it +/- 10%. I guess the difference is due to some targets that I had in the past and now they are offline.
prometheus_tsdb_head_series
Just because I always forget and have to Google this, and this question is at the top. As per this answer there will likely be a variance between prometheus_tsdb_head_series & count({__name__=~".+"}) because of variations in what they consider "active", but unless you have a large variability in metric count, I'd recommend prometheus_tsdb_head_series because if you have a lot of metrics, it's a lot faster to query.
https://prometheusui.com:9090/status ---> Head Stats --> this will give you the entire status.
After some more research and thanks to the comments of David.B i found a solution that 'works for me' ™
To find the number of time series stored by Prometheus i use this command in the storage.local.path folder :
ls -l {{0..9},{a..f}}{{0..9},{a..f}} | grep -E "*.db$" | wc -l
In addition in found some metrics in the prometheus documentation that might help when dealing with memory problems/optimization.
This might not be the most sophisticated way but it gave me the numbers i was looking for.
Go to this URL:
https://YOUR-PROMETHEUS-UI:9090/tsdb-status
OR Query this:
prometheus_tsdb_head_series
OR query this:
count({__name__=~".+"}
OR you can also count files in storage.local.path with ls -l {{0..9},{a..f}}{{0..9},{a..f}} | grep -E "*.db$" | wc -l I've not tested this one.
