Re: [prometheus-users] Use remote-write instead of federation

[Brian Candler]

> Federation endpoint is just another scrapping target - in case of network 
failure (or any other failure) I will get an alert that federation endpoint 
is down

This is true.  However the flip side is that remote_write buffers metrics 
while the network is down, whereas federation will not back-fill any 
historical data when the network comes back up.

You can alert on a remote_write endpoint going away, as described here:
https://groups.google.com/g/prometheus-users/c/ur9Tu1kRu6w/m/Q81qPxqQAAAJ

I think you can make a generic alert against loss of *any* remote write 
sender - something like this (untested):
*up{prometheus_agent="true"} offset 1h unless up*

(i.e. "alert if the given metric/timeseries was present one hour ago but 
isn't present now")

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/bd7a1300-05f0-4377-ae0c-050c80571acan%40googlegroups.com.

Re: [prometheus-users] Use remote-write instead of federation

[Oleg Gumbar]

Have similar case. I would like to use remote-write to collect metrics from 
multiple namespaces/clusters, however federation seems me much more 
reliable. Federation endpoint is just another scrapping target - in case of 
network failure (or any other failure) I will get an alert that federation 
endpoint is down. In case of remote write I have risks to stay blind. I see 
no clear mechanism to be sure I'm getting the metrics =/

What are the possible solutions in this case?
On Wednesday, 20 July 2022 at 18:16:29 UTC+2 vteja...@gmail.com wrote:

> @Stuart: I agree with most of the ideas you say :-) I see remote-write as 
> the most appropriate metrics forwarding for my deployment use case.
> Using federation is not good in terms of interface 
> standardization, HA of monitoring stack, and feature support. 
> For the above case, I have functions and a dedicated set 
> of engineers who own such workload to query individual instances, and the 
> global instance is used as centralized monitoring.
> I was looking at this 
> closed bug, raised 
> on Prometheus in the 2019 Summer. To my understanding, there are 
> performance issues with remote-write but most of them are resolved and the 
> community sees remote-write to perform better when compared to the 
> federation. Am I thinking correctly? 
> Could you clarify the performance comparison between 
> remote-write and federation?
>
> /Teja
>
> On Tuesday, July 19, 2022 at 5:02:11 PM UTC+2 Stuart Clark wrote:
>
>> On 19/07/2022 13:24, tejaswini vadlamudi wrote: 
>> > @Ben: Makes a point, but getting Thanos or Cortex into the picture 
>> > could be a way forward after some time. For now, do you think it is 
>> > good enough to use remote-write instead of federation?  From a 
>> > performance and resource consumption POV, do you see remote-write as 
>> > the way-forward? 
>> > 
>> With remote write you could use agent mode, so you don't have to have 
>> local storage other than for the destination instance. 
>>
>> However again it depends what you are trying to achieve and why you have 
>> suggested having four instances. Are you wanting to query all four 
>> instances or only the "global" one? Are you wanting to copy all data to 
>> the "global" instance or only some metrics? Every data point, or only at 
>> a lower frequency? 
>>
>> If you are intending to copy all data (both metrics & data points) that 
>> leans towards remote write as federation works differently. But in that 
>> case there doesn't seem to be any advantage in having the extra three 
>> instances at all (unless you are intending on doing local querying, 
>> alerting or recording rules) - so I'd just have a single instance that 
>> scrapes all namespaces. 
>>
>> Alternatively if you are needing to have separate instances with local 
>> storage/querying then I'd probably not look to copy all the data to the 
>> "global" instance (which just doubles storage and memory usage) and 
>> either use remote write for a much smaller subset of metrics, federation 
>> with a slower scrape rate/reduced set of metrics, or as Ben suggested 
>> something like Thanos (other options exist as well) to do away with the 
>> fourth instance entirely and distribute the queries to the individual 
>> instances instead. 
>>
>> Maybe if you could explain a bit about what the design is hoping to 
>> achieve it would help us advise better? 
>>
>> -- 
>> Stuart Clark 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/344c19de-664e-44a2-b389-56145585d47cn%40googlegroups.com.

Re: [prometheus-users] Use remote-write instead of federation

[tejaswini vadlamudi]

@Stuart: I agree with most of the ideas you say :-) I see remote-write as 
the most appropriate metrics forwarding for my deployment use case.
Using federation is not good in terms of interface 
standardization, HA of monitoring stack, and feature support. 
For the above case, I have functions and a dedicated set of 
engineers who own such workload to query individual instances, and the 
global instance is used as centralized monitoring.
I was looking at this 
closed bug, raised on 
Prometheus in the 2019 Summer. To my understanding, there are performance 
issues with remote-write but most of them are resolved and the community 
sees remote-write to perform better when compared to the federation. Am I 
thinking correctly? 
Could you clarify the performance comparison between 
remote-write and federation?

/Teja

On Tuesday, July 19, 2022 at 5:02:11 PM UTC+2 Stuart Clark wrote:

> On 19/07/2022 13:24, tejaswini vadlamudi wrote:
> > @Ben: Makes a point, but getting Thanos or Cortex into the picture 
> > could be a way forward after some time. For now, do you think it is 
> > good enough to use remote-write instead of federation?  From a 
> > performance and resource consumption POV, do you see remote-write as 
> > the way-forward?
> >
> With remote write you could use agent mode, so you don't have to have 
> local storage other than for the destination instance.
>
> However again it depends what you are trying to achieve and why you have 
> suggested having four instances. Are you wanting to query all four 
> instances or only the "global" one? Are you wanting to copy all data to 
> the "global" instance or only some metrics? Every data point, or only at 
> a lower frequency?
>
> If you are intending to copy all data (both metrics & data points) that 
> leans towards remote write as federation works differently. But in that 
> case there doesn't seem to be any advantage in having the extra three 
> instances at all (unless you are intending on doing local querying, 
> alerting or recording rules) - so I'd just have a single instance that 
> scrapes all namespaces.
>
> Alternatively if you are needing to have separate instances with local 
> storage/querying then I'd probably not look to copy all the data to the 
> "global" instance (which just doubles storage and memory usage) and 
> either use remote write for a much smaller subset of metrics, federation 
> with a slower scrape rate/reduced set of metrics, or as Ben suggested 
> something like Thanos (other options exist as well) to do away with the 
> fourth instance entirely and distribute the queries to the individual 
> instances instead.
>
> Maybe if you could explain a bit about what the design is hoping to 
> achieve it would help us advise better?
>
> -- 
> Stuart Clark
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/e58b5287-609d-48ab-8f20-4a6744d29bedn%40googlegroups.com.

Re: [prometheus-users] Use remote-write instead of federation

[Stuart Clark]

On 19/07/2022 13:24, tejaswini vadlamudi wrote:
@Ben: Makes a point, but getting Thanos or Cortex into the picture 
could be a way forward after some time. For now, do you think it is 
good enough to use remote-write instead of federation?  From a 
performance and resource consumption POV, do you see remote-write as 
the way-forward?


With remote write you could use agent mode, so you don't have to have 
local storage other than for the destination instance.


However again it depends what you are trying to achieve and why you have 
suggested having four instances. Are you wanting to query all four 
instances or only the "global" one? Are you wanting to copy all data to 
the "global" instance or only some metrics? Every data point, or only at 
a lower frequency?


If you are intending to copy all data (both metrics & data points) that 
leans towards remote write as federation works differently. But in that 
case there doesn't seem to be any advantage in having the extra three 
instances at all (unless you are intending on doing local querying, 
alerting or recording rules) - so I'd just have a single instance that 
scrapes all namespaces.


Alternatively if you are needing to have separate instances with local 
storage/querying then I'd probably not look to copy all the data to the 
"global" instance (which just doubles storage and memory usage) and 
either use remote write for a much smaller subset of metrics, federation 
with a slower scrape rate/reduced set of metrics, or as Ben suggested 
something like Thanos (other options exist as well) to do away with the 
fourth instance entirely and distribute the queries to the individual 
instances instead.


Maybe if you could explain a bit about what the design is hoping to 
achieve it would help us advise better?


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/a98eb9c0-21ce-ecac-7bb6-100b28d50986%40Jahingo.com.

Re: [prometheus-users] Use remote-write instead of federation

[tejaswini vadlamudi]

@Ben: Makes a point, but getting Thanos or Cortex into the picture could be 
a way forward after some time. For now, do you think it is good enough to 
use remote-write instead of federation?  From a performance and resource 
consumption POV, do you see remote-write as the way-forward?

Thanks, Teja

On Monday, July 18, 2022 at 11:02:57 PM UTC+2 sup...@gmail.com wrote:

> Yes, Thanos will eliminate the need for instance-4. At the same time it's 
> more efficient because it doesn't use remote write or federation. It can 
> query data from all your Prometheus instances.
>
> On Mon, Jul 18, 2022 at 10:53 PM tejaswini vadlamudi  
> wrote:
>
>> @Ben: Thanks for the suggestion! I heard that remote-write consumes more 
>> system resources like CPU utilization when compared to the federation. I 
>> can test and cross-check it myself but I would like to hear feedback from 
>> the Prometheus experts.
>> @Stuart: Ideally, it is possible to manage the complete stack with 
>> instance-1 but the current case is about deploying and monitoring multiple 
>> workloads/software owned by different vendors.
>>
>> /Teja
>> On Monday, July 18, 2022 at 8:52:59 PM UTC+2 Stuart Clark wrote:
>>
>>> On 18/07/2022 18:00, tejaswini vadlamudi wrote:
>>>
>>> Hello Stuart,  
>>>
>>> I have the 4 Prometheus instances in the same cluster.   
>>>
>>>- Instance-1, monitoring k8s & cadvisor 
>>>- Instance-2, monitoring workload-1 in namespace-1 
>>>- Instance-3, monitoring workload-2 in namespace-2 
>>>- Instance-4 is the central one collecting metrics from all 3 
>>>instances (for global querying and alerting). not sure if the federation 
>>> is 
>>>a good fit for this sort of deployment pattern. 
>>>
>>> What's the reason for having all the different instances? Are these all 
>>> full instances of Prometheus (with local storage) or using agent mode?
>>>
>>> If you are just going to copy everything to the "central" instance on 
>>> the same cluster, why not just do without the extra three clusters and have 
>>> just the one instance that monitors everything?
>>>
>>> -- 
>>> Stuart Clark
>>>
>>> -- 
>>
> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to prometheus-use...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-users/cd955a3f-fb4c-43f8-87e2-4eb006bc7df8n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/e3a7da84-afec-41cd-a8e8-be56261afd53n%40googlegroups.com.

Re: [prometheus-users] Use remote-write instead of federation

[Ben Kochie]

Yes, Thanos will eliminate the need for instance-4. At the same time it's
more efficient because it doesn't use remote write or federation. It can
query data from all your Prometheus instances.

On Mon, Jul 18, 2022 at 10:53 PM tejaswini vadlamudi 
wrote:

> @Ben: Thanks for the suggestion! I heard that remote-write consumes more
> system resources like CPU utilization when compared to the federation. I
> can test and cross-check it myself but I would like to hear feedback from
> the Prometheus experts.
> @Stuart: Ideally, it is possible to manage the complete stack with
> instance-1 but the current case is about deploying and monitoring multiple
> workloads/software owned by different vendors.
>
> /Teja
> On Monday, July 18, 2022 at 8:52:59 PM UTC+2 Stuart Clark wrote:
>
>> On 18/07/2022 18:00, tejaswini vadlamudi wrote:
>>
>> Hello Stuart,
>>
>> I have the 4 Prometheus instances in the same cluster.
>>
>>- Instance-1, monitoring k8s & cadvisor
>>- Instance-2, monitoring workload-1 in namespace-1
>>- Instance-3, monitoring workload-2 in namespace-2
>>- Instance-4 is the central one collecting metrics from all 3
>>instances (for global querying and alerting). not sure if the federation 
>> is
>>a good fit for this sort of deployment pattern.
>>
>> What's the reason for having all the different instances? Are these all
>> full instances of Prometheus (with local storage) or using agent mode?
>>
>> If you are just going to copy everything to the "central" instance on the
>> same cluster, why not just do without the extra three clusters and have
>> just the one instance that monitors everything?
>>
>> --
>> Stuart Clark
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/cd955a3f-fb4c-43f8-87e2-4eb006bc7df8n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CABbyFmrOjfax9YvqQioCJUfr8BB_b-wDsu5mWvuEKPXg_vDayA%40mail.gmail.com.

Re: [prometheus-users] Use remote-write instead of federation

[tejaswini vadlamudi]

@Ben: Thanks for the suggestion! I heard that remote-write consumes more 
system resources like CPU utilization when compared to the federation. I 
can test and cross-check it myself but I would like to hear feedback from 
the Prometheus experts.
@Stuart: Ideally, it is possible to manage the complete stack with 
instance-1 but the current case is about deploying and monitoring multiple 
workloads/software owned by different vendors.

/Teja
On Monday, July 18, 2022 at 8:52:59 PM UTC+2 Stuart Clark wrote:

> On 18/07/2022 18:00, tejaswini vadlamudi wrote:
>
> Hello Stuart,  
>
> I have the 4 Prometheus instances in the same cluster.   
>
>- Instance-1, monitoring k8s & cadvisor 
>- Instance-2, monitoring workload-1 in namespace-1 
>- Instance-3, monitoring workload-2 in namespace-2 
>- Instance-4 is the central one collecting metrics from all 3 
>instances (for global querying and alerting). not sure if the federation 
> is 
>a good fit for this sort of deployment pattern. 
>
> What's the reason for having all the different instances? Are these all 
> full instances of Prometheus (with local storage) or using agent mode?
>
> If you are just going to copy everything to the "central" instance on the 
> same cluster, why not just do without the extra three clusters and have 
> just the one instance that monitors everything?
>
> -- 
> Stuart Clark
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/cd955a3f-fb4c-43f8-87e2-4eb006bc7df8n%40googlegroups.com.

Re: [prometheus-users] Use remote-write instead of federation

[Stuart Clark]

On 18/07/2022 18:00, tejaswini vadlamudi wrote:

Hello Stuart,

I have the 4 Prometheus instances in the same cluster.

  * Instance-1, monitoring k8s & cadvisor
  * Instance-2, monitoring workload-1 in namespace-1
  * Instance-3, monitoring workload-2 in namespace-2
  * Instance-4 is the central one collecting metrics from all 3
instances (for global querying and alerting). not sure if the
federation is a good fit for this sort of deployment pattern.

What's the reason for having all the different instances? Are these all 
full instances of Prometheus (with local storage) or using agent mode?


If you are just going to copy everything to the "central" instance on 
the same cluster, why not just do without the extra three clusters and 
have just the one instance that monitors everything?


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/d047d5f6-ad6b-a334-699d-8d7a4399e26a%40Jahingo.com.

Re: [prometheus-users] Use remote-write instead of federation

[Ben Kochie]

I would probably skip federation and remote write with that setup and use
Thanos to create a single pane view of all of them.

On Mon, Jul 18, 2022, 7:00 PM tejaswini vadlamudi 
wrote:

> Hello Stuart,
>
> I have the 4 Prometheus instances in the same cluster.
>
>- Instance-1, monitoring k8s & cadvisor
>- Instance-2, monitoring workload-1 in namespace-1
>- Instance-3, monitoring workload-2 in namespace-2
>- Instance-4 is the central one collecting metrics from all 3
>instances (for global querying and alerting). not sure if the federation is
>a good fit for this sort of deployment pattern.
>
>
> Thanks, Teja
>
>
> On Monday, July 18, 2022 at 6:49:45 PM UTC+2 Stuart Clark wrote:
>
>> On 18/07/2022 17:21, tejaswini vadlamudi wrote:
>> > Can someone point me to the advantages of using remote-write over
>> > federation?
>> > I understand that remote-write is more of a standard interface in the
>> > monitoring domain.
>> > Are there any handy performance measurements that were
>> observed/recorded?
>> >
>> They are really quite different.
>>
>> Federation is a way of pulling data from a remote Prometheus into
>> (generally) a local one. The puller gets to choose how often to pull
>> data and what data to fetch. If the puller can't fetch the data for any
>> reason (local/remote outage, network issues, etc.) there will be gaps.
>>
>> Remote write is a way of pushing data from a Prometheus server to
>> "something else", which could be another Prometheus or one of the many
>> things which implement the API (e.g. various databases, Thanos, custom
>> analytics tools, etc.). For these you get all the data (basically as
>> soon as it has been scraped) with the ability to do filtering via
>> relabling. If there is an outage/disconnect data will be queued for a
>> while (too long and things will get lost) so small issues can be handled
>> transparently.
>>
>> So you have a difference in what data you get - either all (filtered)
>> data or data on a schedule (so in effect a form of built-in
>> downsampling), and who controls that - either the data source Prometheus
>> or the destination.
>>
>> Which is "better" depends on what you are trying to achieve and the
>> constraints you might have (for example difficulties with accepting
>> network connections or data storage/transfer limits). Don't forget the
>> organisation differences too - for remote write adding/changing a
>> destination (or filter rules) needs changes to every data source
>> Prometheus where federation is purely controlled at the other end, which
>> might be a good or bad thing depending on team responsibilities/timings.
>>
>> --
>> Stuart Clark
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/a174584b-8f1d-4ab8-bcda-bfae9401af0en%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CABbyFmorLg2%3DOGzKP8ydVBy715FC%3D4ZAZ-1DJ4U3EL2EQnGk5g%40mail.gmail.com.

Re: [prometheus-users] Use remote-write instead of federation

[tejaswini vadlamudi]

Hello Stuart, 

I have the 4 Prometheus instances in the same cluster.  

   - Instance-1, monitoring k8s & cadvisor
   - Instance-2, monitoring workload-1 in namespace-1
   - Instance-3, monitoring workload-2 in namespace-2
   - Instance-4 is the central one collecting metrics from all 3 instances 
   (for global querying and alerting). not sure if the federation is a good 
   fit for this sort of deployment pattern.


Thanks, Teja


On Monday, July 18, 2022 at 6:49:45 PM UTC+2 Stuart Clark wrote:

> On 18/07/2022 17:21, tejaswini vadlamudi wrote:
> > Can someone point me to the advantages of using remote-write over 
> > federation?
> > I understand that remote-write is more of a standard interface in the 
> > monitoring domain.
> > Are there any handy performance measurements that were observed/recorded?
> >
> They are really quite different.
>
> Federation is a way of pulling data from a remote Prometheus into 
> (generally) a local one. The puller gets to choose how often to pull 
> data and what data to fetch. If the puller can't fetch the data for any 
> reason (local/remote outage, network issues, etc.) there will be gaps.
>
> Remote write is a way of pushing data from a Prometheus server to 
> "something else", which could be another Prometheus or one of the many 
> things which implement the API (e.g. various databases, Thanos, custom 
> analytics tools, etc.). For these you get all the data (basically as 
> soon as it has been scraped) with the ability to do filtering via 
> relabling. If there is an outage/disconnect data will be queued for a 
> while (too long and things will get lost) so small issues can be handled 
> transparently.
>
> So you have a difference in what data you get - either all (filtered) 
> data or data on a schedule (so in effect a form of built-in 
> downsampling), and who controls that - either the data source Prometheus 
> or the destination.
>
> Which is "better" depends on what you are trying to achieve and the 
> constraints you might have (for example difficulties with accepting 
> network connections or data storage/transfer limits). Don't forget the 
> organisation differences too - for remote write adding/changing a 
> destination (or filter rules) needs changes to every data source 
> Prometheus where federation is purely controlled at the other end, which 
> might be a good or bad thing depending on team responsibilities/timings.
>
> -- 
> Stuart Clark
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/a174584b-8f1d-4ab8-bcda-bfae9401af0en%40googlegroups.com.

Re: [prometheus-users] Use remote-write instead of federation

[Stuart Clark]

On 18/07/2022 17:21, tejaswini vadlamudi wrote:
Can someone point me to the advantages of using remote-write over 
federation?
I understand that remote-write is more of a standard interface in the 
monitoring domain.

Are there any handy performance measurements that were observed/recorded?


They are really quite different.

Federation is a way of pulling data from a remote Prometheus into 
(generally) a local one. The puller gets to choose how often to pull 
data and what data to fetch. If the puller can't fetch the data for any 
reason (local/remote outage, network issues, etc.) there will be gaps.


Remote write is a way of pushing data from a Prometheus server to 
"something else", which could be another Prometheus or one of the many 
things which implement the API (e.g. various databases, Thanos, custom 
analytics tools, etc.). For these you get all the data (basically as 
soon as it has been scraped) with the ability to do filtering via 
relabling. If there is an outage/disconnect data will be queued for a 
while (too long and things will get lost) so small issues can be handled 
transparently.


So you have a difference in what data you get - either all (filtered) 
data or data on a schedule (so in effect a form of built-in 
downsampling), and who controls that - either the data source Prometheus 
or the destination.


Which is "better" depends on what you are trying to achieve and the 
constraints you might have (for example difficulties with accepting 
network connections or data storage/transfer limits). Don't forget the 
organisation differences too - for remote write adding/changing a 
destination (or filter rules) needs changes to every data source 
Prometheus where federation is purely controlled at the other end, which 
might be a good or bad thing depending on team responsibilities/timings.


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/0a925bd8-4cbb-99ff-c372-311488751943%40Jahingo.com.

[prometheus-users] Use remote-write instead of federation

[tejaswini vadlamudi]

Can someone point me to the advantages of using remote-write over 
federation?
I understand that remote-write is more of a standard interface in the 
monitoring domain.
Are there any handy performance measurements that were observed/recorded?

Thanks, Teja

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/31d14713-fe98-416f-9984-c185ce969363n%40googlegroups.com.