I have red the documentation about prometheus and the pushgateway but I still dont understand how I am supposed to "properly" setup the prometheus stack to get the metrics of my jobs.
Use case: We have a couple of jobs which run every day for about 30-90 seconds. So the following instances are run every day: - JobA for customerX - JobA for customerY - JobB for customerX - JobC for customerZ So for JobA there are 2 "instances" running every day, each instance needs to collect some data for one of our customers. It seems the pushgateway is the way to go here as the jobs are not bound to any machine (k8s cronjob scheduled on some node in the pool) and the average runtime is about 1 minute each. Two things still confuse me a lot: 1. what "job-name" and "instance-id" (if any) should I use when pushing the metrics, 2. when and how should I delete the metrics from the pushgateway About 1) For the "job-name" I was going to use "JobA", "JobB" etc For "instance-id" I was thinking about using "customerX-$timestamp" About 2) I am really clueless here. I guess if I dont use an "instance-id" I can delete all metrics by "job-name" once I dont need the job anymore but this could be years in the future.. If I use an "instance-id", the only way to delete the metrics (without keeping track of what instance-ids where used) is to delete all metrics via the admin api. So it seems here I either need to keep state or delete everything, both doesnt sound right.. I would really appreciate some help here. Br David -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/fb41b909-774c-490f-ae17-f53ba473da73n%40googlegroups.com.
