CI environments like GitLab (self-hostet and Cloud), GitHub, CodeCov, Codacy, ... collect statistics over time, so a developer or team lead can see the evolution of the project over time:
- number of merge requests, commits, contributors, ...
- number of passed/failed tests
- code coverage
- used runtime for e.g. unit tests on a server
- ...
Unfortunately, these statistics are: decentralized (multipe cloud services are needed), specific to the services who offer them, and they are not general purpose.
I'm seeking for a solution to collect data points over time per repository or repository group. My background is hardware development with e.g. FPGAs and also embedded software.
- used hardware resources like gates, memory, multiplier units, ...
- timing errors (how many wires do not meet timing constraints)
- achievable (max) frequency
- number of critical errors, warnings and info messages
Other more software-like parameters could be:
- performance / per test-case runtime
- executable size
All these metrics are essential to detect improvements / optimizations over time or to notice degradation before a hardware designs stops working (get unreliable).
What I know so far:
- Such data is ideally stored in a time series database with either unlimited time span (if you want to compare even years back when the project started) or limited like last 12 month.
- Prometheus is used widely in cloud and network setups e.g. to collect CPU/RAM usage, network traffic, temperatures and other arbitrary data points over time.
- Prometheus is part of a self-hosted GitLab installation.
- Visualization can be done via Grafana.
- Users can defined new diagrams and panels.
- Grafana is part of a self-hosted GitLab installation.
What's missing from my point of view - and here I'm seeking help or ideas:
- How to connect new time series in Prometheus with a Git repository?
- How to define access rights based on who can access a Git repository?
- How to add new views to Grafana if a repository pushes such statistics?
- How to get rid of old data if the repository gets deleted.
- At best configure it with a YAML file in the repository itself.
- ...
Of cause I could set it up if it's just a single repository pushing data points, but I have > 100 repositories and currently 1-3 are added per week.
Is such a service / add-on already available?
