I work as SRE at a company which runs Ruby on Rails application deployed with Unicorn. Not so long time ago, we started to migrate from StatsD (https://github.com/statsd/statsd) based metrics to Prometheus. We wrote wrapper library which uses https://gitlab.com/gitlab-org/prometheus-client-mmap (which also is fork of https://github.com/prometheus/client_ruby) and so far so good, everything went as planned during PoC.
But recently we found some limitation which we are banging head on how to solve. Many people are used to StatD and in application code they used just to increase gauges values (from multiple locations) and aggregating such values gives you "current" value. Now with Prometheus, we have multiple muti-thread workers which updates their own gauge values (workers are labeled e.g. with id). So let's say one process stores 10 in gauge and after some minutes another worker stores 5. How we shall know what value is the latest? Prometheus client_ruby added most_recent aggregations https://github.com/prometheus/client_ruby/pull/172, which helps on single Unicorn, but let's say we want to know what is the latest value between two multiworker Unicorns? Few proposed solutions: 1. Use Pushgateway https://prometheus.io/docs/practices/pushing/ - we really want to avoid running it, if we can, as it will become single point of failure, and probably we also will have some performance issues due to high rate of pushes. 2. Use https://github.com/discourse/prometheus_exporter - that would run as separate instance and seems to be option. But sill how to find what is the latest value between Unicorns? Maybe someone had similar issues and come up with some different solutions? Any feedback is very welcome. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/47ab2eb3-6ee4-4f2e-9da1-f60412ada1d3n%40googlegroups.com.
