Re: [prometheus-developers] Evolving remote APIs

2022-04-13 Thread 'Pengfei Zhang' via Prometheus Developers
Hi, I make a short proposal of structured remote write at https://github.com/prometheus/prometheus/issues/10539. There must be lots of use case/concerns of not moving to structured protocol, but after weighing the pros and cons, I still think have such a proposal with discussion will help Pro

Re: [prometheus-developers] Evolving remote APIs

2021-12-01 Thread Bjoern Rabenstein
On 25.11.21 10:35, Fabian Reinartz wrote: > > The point on TSDB becoming more structured is interesting – how firm are > these plans at this point? Any rough timelines? I hope there will be a PoC for histograms in two or three months. It's hard to estimate how long it will take after that to get

Re: [prometheus-developers] Evolving remote APIs

2021-11-27 Thread Rob Skillington
There's a.now out of date but working proof of concept PR from August last year that added TYPE, HELP and UNIT to the WAL and also to Prometheus Remote Write payloads (on a per TimeSeries samples basis): https://github.com/prometheus/prometheus/pull/7771 Once it's added to the WAL there's no reaso

Re: [prometheus-developers] Evolving remote APIs

2021-11-26 Thread 'Fabian Reinartz' via Prometheus Developers
> As maintainer of Prometheus server, in general, I am worried that > getting a wal that'd be more "able" than the actual Prometheus TSDB > would weaken the Prometheus server use case in favor of SaaS platforms. > > It does not sound great for the users who rely on Prometheus > alone, which I think

Re: [prometheus-developers] Evolving remote APIs

2021-11-26 Thread Bartłomiej Płotka
> directly stream OpenMetrics (or a proto-equivalent) from there, On another front, from an efficiency standpoint, don't we want to batch samples from exact same ts in many cases (e.g network partition)? Kind Regards, Bartek Płotka (@bwplotka) On Thu, Nov 25, 2021 at 10:25 PM Julien Pivotto w

Re: [prometheus-developers] Evolving remote APIs

2021-11-25 Thread Julien Pivotto
On 25 Nov 10:35, Fabian Reinartz wrote: > The point on TSDB becoming more structured is interesting – how firm are > these plans at this point? Any rough timelines? > My first hunch would've been to explore integrating directly at the > scraping layer to directly stream OpenMetrics (or a proto-eq

Re: [prometheus-developers] Evolving remote APIs

2021-11-25 Thread Fabian Reinartz
Thanks for the responses everyone. Sounds like the existing remote-write will become stable as is, which is certainly a logical consequence after all this time. I can certainly imagine some features being added to it, as the transactional RW design shows. Another semi-low-hanging fruit may be to

Re: [prometheus-developers] Evolving remote APIs

2021-11-22 Thread Bjoern Rabenstein
On 18.11.21 16:36, 'Fabian Reinartz' via Prometheus Developers wrote: > > A central issue is that the remote APIs expose the Prometheus storage data > model. It is notably different from the Prometheus/OpenMetrics > instrumentation model and discards most of the structure known at scrape > time. >

Re: [prometheus-developers] Evolving remote APIs

2021-11-18 Thread Tom Wilkie
Sounds good! Should we start a regular working group to focus on this? I know many people have been interested. In the meantime I'll try and get that v1 spec published on the prometheus site so we can officially call it "done". One question I'm interested in discussing is to what extent the exis

Re: [prometheus-developers] Evolving remote APIs

2021-11-18 Thread Bartłomiej Płotka
Hi Fabian! As Julian said, it sounds like we have to then talk about Remote Write v2. We can totally start designing one. Looking forward to proposals in this space! (: Kind Regards, Bartek On Thursday, November 18, 2021 at 6:31:40 PM UTC+1 Julien Pivotto wrote: > On 18 Nov 16:36, 'Fabian Re

Re: [prometheus-developers] Evolving remote APIs

2021-11-18 Thread Julien Pivotto
On 18 Nov 16:36, 'Fabian Reinartz' via Prometheus Developers wrote: > Hi developers, > > We recently launched Google Cloud’s Prometheus metric backend based on > Monarch. We encountered some obstacles regarding the remote APIs, which we > believe to be common for backends that were not built for P