On Sun, Nov 29, 2020 at 3:10 PM Ben Kochie <sup...@gmail.com> wrote: > On Sun, Nov 29, 2020 at 11:51 AM Aliaksandr Valialkin <valy...@gmail.com> > wrote: > >> >> >> On Fri, Nov 27, 2020 at 11:11 AM Ben Kochie <sup...@gmail.com> wrote: >> >>> >>>> >>>>> Or else is there any other ways by which we can solve this issue. >>>>> >>>> >>>> Using something other than federation. remote_write is able to buffer >>>> up data locally if the endpoint is down. >>>> >>>> Prometheus itself can't accept remote_write requests, so you'd have to >>>> write to some other system >>>> <https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage> >>>> which can. I suggest VictoriaMetrics, as it's simple to run and has a very >>>> prometheus-like API, which can be queried as if it were a prometheus >>>> instance. >>>> >>> >>> I recommend Thanos, as it scales better and with less effort than >>> VictoriaMetrics. It also uses PromQL code directly, so you will get the >>> same results as Prometheus, not an emulation of PromQL. >>> >>> >> Could you share more details on why you think that VictoriaMetrics has >> scalability issues and is harder to set up and operate than Thanos? >> VictoriaMetrics users have quite the opposite opinion. See >> https://victoriametrics.github.io/CaseStudies.html and >> https://medium.com/faun/comparing-thanos-to-victoriametrics-cluster-b193bea1683 >> . >> > > Thanos uses object storage, which avoids the need for manual sharding of > TSDB storage. Today I have 100TiB of data stored in object storage buckets. > I make no changes to scale up or down these buckets. > > VictoriaMetrics stores data on persistent disks. Every replicated durable persistent disk in GCP <https://cloud.google.com/persistent-disk> can scale up to 64TB <https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd> without the need to stop VictoriaMetrics, i.e. without downtime. Given that VictoriaMetrics compresses real-world data much better than Prometheus <https://valyala.medium.com/prometheus-vs-victoriametrics-benchmark-on-node-exporter-metrics-4ca29c75590f>, a single-node VictoriaMetrics can substitute the whole Thanos cluster for your workload (in theory of course - just give it a try in order to verify this statement :) ). Cluster version of VictoriaMetrics <https://victoriametrics.github.io/Cluster-VictoriaMetrics.html> can scale to petabytes. For example, a cluster with one terabyte capacity can be built with 16 vmstorage nodes with 64TB persistent disk per each node. That's why VictoriaMetrics in production usually has lower infrastructure costs than Thanos.
-- Best Regards, Aliaksandr Valialkin, CTO VictoriaMetrics -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAPbKnmCfjWu60ggYpXbPt%3DeZr9KNi1rTseEyiOd9-opL%3DiuNLQ%40mail.gmail.com.