More generally you're asking "how do I write an application which accepts updates to an inventory and makes the same updated data available to multiple containers?"
There are many possible solutions, but ultimately you need a single source of truth which the other containers refer to or receive a copy of. Having a single file that you share using NFS is one way. Exposing it using HTTP and having the other containers copy it is another way. Pushing out the file using rsync or scp is another way. Putting the data in a shared SQL database is another. Posting the target list via a publish-subscribe server like NATS or RabbitMQ is another. You could publish the list using Kubernetes native mechanisms like ConfigMap <https://kubernetes.io/docs/concepts/configuration/configmap/#motivation> (but note that has a maximum size of 1 MiB) Given that this is Prometheus we're talking about, I suggest you look at the http_sd_config instead of file_sd_config. - Make one instance of your container which accepts the target updates using HTTP POST (you already have this) - Modify it so that it exposes the current full list of targets via HTTP GET - Configure all the Prometheus servers with http_sd_config to fetch this list of targets periodically In other words, you have one "inventory" service which all the other "Prometheus" services talk to. This is reasonably robust: if the inventory service goes down, the Prometheus servers will continue to scrape using the last known set of targets. Aside: if you were using Netbox as your central inventory, then you could use netbox-plugin-prometheus-sd to get this functionality. On Wednesday 13 December 2023 at 05:37:01 UTC akshay sharma wrote: > Hi, > > I have two instances of Prometheus running with one service. > with both instances, one sidecar is running which is updating the config > file(file_sd_config) with the target information based on the request(POST, > CURL). > so while doing multiple post requests, the request is reaching through > service to any of the instances in a round-robin fashion. > so both the instance has config file but with different target > information.. so is there any way we can sync the config file between the > two instances? > > as it is giving issues while removing and updating the existing target. > using curl request. > > so please help, Is there any way we can sync the config file? > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/bba005b2-fc34-47d3-9dcc-95321ff7f1dbn%40googlegroups.com.
