Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

['Owen Williams' via Prometheus Developers]

the committed code in /common includes support for 3 escaping formats:

* all legacy-invalid characters to underscores (cannot round-trip)
* dots are escaped to _dot_ and underscores are converted to double
underscores, all other legacy-invalid characters become single underscores.
(partial round trip)
* a new U__ encoding for unicode code points so names can be fully
round-tripped

https://github.com/prometheus/common/blob/1c9da3533702ae8a54609c7b6f5bdc1ff9b754c9/model/metric.go#L60-L77



On Mon, Jun 10, 2024 at 6:07 AM Bartłomiej Płotka 
wrote:

> First of all, while it can be painful, I think it's ok to double check the 
> already
> approved proposal
> 
>  for
> the big blockers. I don't think I see any big blocker, given this
> discussion.
>
> Just 2c, I wonder if it would be helpful to split metric/label naming
> discussion into:
>
> A) What protocol, storage and PromQL supports (more flexibility for future)
> B) What makes sense to use for everyone (recommendation or mandated for
> some systems)
>
> Essentially we can (and probably should) add versatile escaping
> mechanisms. Does it mean people should call their metrics
> "❤️123易_total"? I don't think it's useful to accept those. It might be
> useful to accept (escaped) dots though given Otel users want (for example)
> without translation, use the same matchers for traces, logs, profiles and
> metrics and they cannot really change that decision.
>
> We could keep (recommend or ensure) the conversion schema for Otel metrics
> (move to _) AND move with generic UTF-8 support. Both could be done in
> parallel, as long as it's officially agreed and documented (:
>
> Kind Regards,
> Bartek Płotka (@bwplotka)
>
>
> On Wed, Jun 5, 2024 at 5:55 PM Bjoern Rabenstein 
> wrote:
>
>> On 05.06.24 18:07, 'Fabian Stäber' via Prometheus Developers wrote:
>> >
>> > So, is the prefered solution to keep things as they are, i.e. keep
>> > replacing dots with underscores?
>>
>> I don't think the purpose of the survey was to find a "preferred
>> solution". First of all, this is a technical decision, not a
>> democratic one. And even if it were, an online survey is inherently
>> biased.
>>
>> The idea behind the survey was (I hope) to get a broad idea what
>> people find surprising or annoying, what they expect, what they like,
>> ... and then we can use those inputs in a responsible fashion to
>> inform decisions.
>>
>> > > why allow two different separator characters if they have no
>> > > semantic difference (no true namespacing).
>> >
>> > This argument seems to resonate with the Prometheus team. If this is the
>> > main concern, we don't solve it by allowing dots in quotes. We solve
>> this
>> > by replacing dots with underscores.
>>
>> As discussed before, this solution has issues because you might run
>> into name collisions, and it is hard to match a name from one side of
>> the conversion wall to the corresponding name on the other side.
>>
>> The previous discussion lead to the conclusion that we want allow all
>> of UTF-8, because OTel does, but that everything that is not a
>> valid conventional Prometheus name will require quoting.
>>
>> We kept open the option of later allowing more characters in the
>> unquoted names, after we have seen how the quoting goes.
>>
>> > >From the survey it looks like most users prefer the current naming
>> scheme
>> > as well:
>> >
>> > [image: screenshot_2024-06-05_18:04:05_908234003.png]
>> > [image: screenshot_2024-06-05_18:04:14_304430186.png]
>>
>> The people in the survey got confronted with the various quoting
>> schemas without providing any context. This can only give us some idea
>> about people's gut feeling, but not much more.
>>
>> > Shall we just drop the idea of adding UTF-8 support?
>>
>> I don't understand the jump to this conclusion. OTel stil supports all
>> of UTF-8 in names. If somebody names a metric in Chinese or Cyrillic,
>> we cannot convert it to "__". That's the whole point. We
>> need UTF-8 support _anyway_. So let's do it and see how it goes before
>> running the umptieth reiteration of "can we just allow dots in metric
>> names".
>>
>> --
>> Björn Rabenstein
>> [PGP-ID] 0x851C3DA17D748D03
>> [email] bjo...@rabenste.in
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/ZmCYjf9yXTLMqNbW%40mail.rabenste.in
>> .
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Prometheus Developers" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/prometheus-developers/ftnfizjXOmk/unsubscribe
> .
> To unsubscribe from this group and all 

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Bartłomiej Płotka]

First of all, while it can be painful, I think it's ok to double check
the already
approved proposal

for
the big blockers. I don't think I see any big blocker, given this
discussion.

Just 2c, I wonder if it would be helpful to split metric/label naming
discussion into:

A) What protocol, storage and PromQL supports (more flexibility for future)
B) What makes sense to use for everyone (recommendation or mandated for
some systems)

Essentially we can (and probably should) add versatile escaping mechanisms.
Does it mean people should call their metrics "❤️123易_total"? I don't
think it's useful to accept those. It might be useful to accept (escaped)
dots though given Otel users want (for example) without translation, use
the same matchers for traces, logs, profiles and metrics and they cannot
really change that decision.

We could keep (recommend or ensure) the conversion schema for Otel metrics
(move to _) AND move with generic UTF-8 support. Both could be done in
parallel, as long as it's officially agreed and documented (:

Kind Regards,
Bartek Płotka (@bwplotka)


On Wed, Jun 5, 2024 at 5:55 PM Bjoern Rabenstein  wrote:

> On 05.06.24 18:07, 'Fabian Stäber' via Prometheus Developers wrote:
> >
> > So, is the prefered solution to keep things as they are, i.e. keep
> > replacing dots with underscores?
>
> I don't think the purpose of the survey was to find a "preferred
> solution". First of all, this is a technical decision, not a
> democratic one. And even if it were, an online survey is inherently
> biased.
>
> The idea behind the survey was (I hope) to get a broad idea what
> people find surprising or annoying, what they expect, what they like,
> ... and then we can use those inputs in a responsible fashion to
> inform decisions.
>
> > > why allow two different separator characters if they have no
> > > semantic difference (no true namespacing).
> >
> > This argument seems to resonate with the Prometheus team. If this is the
> > main concern, we don't solve it by allowing dots in quotes. We solve this
> > by replacing dots with underscores.
>
> As discussed before, this solution has issues because you might run
> into name collisions, and it is hard to match a name from one side of
> the conversion wall to the corresponding name on the other side.
>
> The previous discussion lead to the conclusion that we want allow all
> of UTF-8, because OTel does, but that everything that is not a
> valid conventional Prometheus name will require quoting.
>
> We kept open the option of later allowing more characters in the
> unquoted names, after we have seen how the quoting goes.
>
> > >From the survey it looks like most users prefer the current naming
> scheme
> > as well:
> >
> > [image: screenshot_2024-06-05_18:04:05_908234003.png]
> > [image: screenshot_2024-06-05_18:04:14_304430186.png]
>
> The people in the survey got confronted with the various quoting
> schemas without providing any context. This can only give us some idea
> about people's gut feeling, but not much more.
>
> > Shall we just drop the idea of adding UTF-8 support?
>
> I don't understand the jump to this conclusion. OTel stil supports all
> of UTF-8 in names. If somebody names a metric in Chinese or Cyrillic,
> we cannot convert it to "__". That's the whole point. We
> need UTF-8 support _anyway_. So let's do it and see how it goes before
> running the umptieth reiteration of "can we just allow dots in metric
> names".
>
> --
> Björn Rabenstein
> [PGP-ID] 0x851C3DA17D748D03
> [email] bjo...@rabenste.in
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/ZmCYjf9yXTLMqNbW%40mail.rabenste.in
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CAMssQwa87r9%2B3tAKRSwG6L8YAAu6QQwk4mdt7E%2BGgYR7%3DvOjfQ%40mail.gmail.com.

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Bjoern Rabenstein]

On 05.06.24 18:07, 'Fabian Stäber' via Prometheus Developers wrote:
> 
> So, is the prefered solution to keep things as they are, i.e. keep
> replacing dots with underscores?

I don't think the purpose of the survey was to find a "preferred
solution". First of all, this is a technical decision, not a
democratic one. And even if it were, an online survey is inherently
biased.

The idea behind the survey was (I hope) to get a broad idea what
people find surprising or annoying, what they expect, what they like,
... and then we can use those inputs in a responsible fashion to
inform decisions.

> > why allow two different separator characters if they have no
> > semantic difference (no true namespacing).
> 
> This argument seems to resonate with the Prometheus team. If this is the
> main concern, we don't solve it by allowing dots in quotes. We solve this
> by replacing dots with underscores.

As discussed before, this solution has issues because you might run
into name collisions, and it is hard to match a name from one side of
the conversion wall to the corresponding name on the other side.

The previous discussion lead to the conclusion that we want allow all
of UTF-8, because OTel does, but that everything that is not a
valid conventional Prometheus name will require quoting.

We kept open the option of later allowing more characters in the
unquoted names, after we have seen how the quoting goes.

> >From the survey it looks like most users prefer the current naming scheme
> as well:
> 
> [image: screenshot_2024-06-05_18:04:05_908234003.png]
> [image: screenshot_2024-06-05_18:04:14_304430186.png]

The people in the survey got confronted with the various quoting
schemas without providing any context. This can only give us some idea
about people's gut feeling, but not much more.

> Shall we just drop the idea of adding UTF-8 support?

I don't understand the jump to this conclusion. OTel stil supports all
of UTF-8 in names. If somebody names a metric in Chinese or Cyrillic,
we cannot convert it to "__". That's the whole point. We
need UTF-8 support _anyway_. So let's do it and see how it goes before
running the umptieth reiteration of "can we just allow dots in metric
names".

-- 
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/ZmCYjf9yXTLMqNbW%40mail.rabenste.in.

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

['Owen Williams' via Prometheus Developers]

I would love to understand how to come to a decision as a project in a way 
that will preclude regular second-guessing. We went through the proposal 
process 
(https://github.com/prometheus/proposals/blob/main/proposals/2023-08-21-utf8.md)
 
so I though the question of both "if" and "how" were already closed.

On Wednesday, June 5, 2024 at 12:07:48 PM UTC-4 Fabian Stäber wrote:

> Thanks a lot for sharing!
>
> So, is the prefered solution to keep things as they are, i.e. keep 
> replacing dots with underscores?
>
> > why allow two different separator characters if they have no 
> semantic difference (no true namespacing).
>
> This argument seems to resonate with the Prometheus team. If this is the 
> main concern, we don't solve it by allowing dots in quotes. We solve this 
> by replacing dots with underscores.
>
> From the survey it looks like most users prefer the current naming scheme 
> as well:
>
> [image: screenshot_2024-06-05_18:04:05_908234003.png]
> [image: screenshot_2024-06-05_18:04:14_304430186.png]
> Shall we just drop the idea of adding UTF-8 support?
>
> Fabian
>
>
>
>
> On Wed, Jun 5, 2024 at 5:31 PM Arthur Silva Sens  
> wrote:
>
>> The results of Otel-Prometheus interoperability are out, which is 
>> probably relevant to the discussion here (specially the questions regarding 
>> UTF-8)
>>
>>
>> https://github.com/open-telemetry/sig-end-user/tree/main/end-user-surveys/otel-prom-interoperability
>>
>> On Wed, May 29, 2024 at 12:41 PM Julius Volz  wrote:
>>
>>> Hah, I knew it would be a good idea to check with Björn :D Thanks Björn, 
>>> that's a great write-up!
>>>
>>> Yes, that also convinced me about not allowing the dot as a normal 
>>> character for now. Lots of good arguments, but number 3 actually resonates 
>>> the most with me - why allow two different separator characters if they 
>>> have no semantic difference (no true namespacing).
>>>
>>> On Wed, May 29, 2024 at 3:26 PM 'George Robinson' via Prometheus 
>>> Developers  wrote:
>>>
 Thanks Björn for writing this up, and also writing up Collected 
 reasons why Prometheus doesn't allow dot as a regular character in metric 
 and label names 
 . I 
 think it adds a huge amount of value for people looking to participate in 
 discussion! Having read through it all I withdraw my original support for 
 just adding dot to metric names. There are a lot of considerations I did 
 not know about that I agree with thanks to your document.

 On Tuesday, May 28, 2024 at 11:17:10 PM UTC+1 Bjoern Rabenstein wrote:

> I'm trying to keep things short, as all of this had been discussed 
> at length before. 
>
> WRT "how to explain UTF-8 support to users": I actually don't think 
> this is a huge problem. I would frame it "this is like file 
> names". You can use blanks and slashes in Unix file names, and if you 
> do, it requires weird quoting or escaping, but that's not a huge 
> problem in practice. People just don't use them if they care. And if 
> they have to interact with other file sources, where blanks are 
> common, they cope. And yes, that means that names from OTel semantic 
> conventions will always be considered weird, but that's a problem of 
> OTel, not all the other languages where a dot has a special 
> meaning. Segue to the next paragraph... 
>
> WRT the dot in OTel semantic conventions: Personally, I'm more 
> convinced than ever that it was a grave mistake to use dots in the 
> semantic conventions. I understand the history thereof, but the moment 
> that OTel self-declared as the overarching standard for all kind of 
> telemetry, they should have realized that using a character that has a 
> special meaning or is even an operator in so many languages is a 
> really really bad idea. This is not just PromQL specific. Originally, 
> I thought it's infeasible to change the semantic conventions at this 
> point, but by now, that's exactly what I think OTel should do. If the 
> dot were an actual operator in OTel (let's say a separator of actual 
> 1st class namespaces) rather than just a convention within a 
> technically opaque string, I could see some merit. But as it is not, 
> it's just annoying and has no benefits whatsoever. 
>
> Despite having said all of that, I don't realistically expect that 
> OTel is going to change the semantic conventions. So next question is 
> how to deal with it. There are many reasons why it's a bad idea to 
> allow the dot in Prometheus metric names, most of them weren't 
> mentioned in this thread. I won't enumerate them all again. We can do 
> that if we really want to open that can of worms again. Segue to the 
> next paragraph... 
>
> In all the discussions we had before, my impression was that the 
> consensus (in the spirit of RFC 

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Arthur Silva Sens]

The results of Otel-Prometheus interoperability are out, which is probably
relevant to the discussion here (specially the questions regarding UTF-8)

https://github.com/open-telemetry/sig-end-user/tree/main/end-user-surveys/otel-prom-interoperability

On Wed, May 29, 2024 at 12:41 PM Julius Volz  wrote:

> Hah, I knew it would be a good idea to check with Björn :D Thanks Björn,
> that's a great write-up!
>
> Yes, that also convinced me about not allowing the dot as a normal
> character for now. Lots of good arguments, but number 3 actually resonates
> the most with me - why allow two different separator characters if they
> have no semantic difference (no true namespacing).
>
> On Wed, May 29, 2024 at 3:26 PM 'George Robinson' via Prometheus
> Developers  wrote:
>
>> Thanks Björn for writing this up, and also writing up Collected reasons
>> why Prometheus doesn't allow dot as a regular character in metric and label
>> names .
>> I think it adds a huge amount of value for people looking to participate in
>> discussion! Having read through it all I withdraw my original support for
>> just adding dot to metric names. There are a lot of considerations I did
>> not know about that I agree with thanks to your document.
>>
>> On Tuesday, May 28, 2024 at 11:17:10 PM UTC+1 Bjoern Rabenstein wrote:
>>
>>> I'm trying to keep things short, as all of this had been discussed
>>> at length before.
>>>
>>> WRT "how to explain UTF-8 support to users": I actually don't think
>>> this is a huge problem. I would frame it "this is like file
>>> names". You can use blanks and slashes in Unix file names, and if you
>>> do, it requires weird quoting or escaping, but that's not a huge
>>> problem in practice. People just don't use them if they care. And if
>>> they have to interact with other file sources, where blanks are
>>> common, they cope. And yes, that means that names from OTel semantic
>>> conventions will always be considered weird, but that's a problem of
>>> OTel, not all the other languages where a dot has a special
>>> meaning. Segue to the next paragraph...
>>>
>>> WRT the dot in OTel semantic conventions: Personally, I'm more
>>> convinced than ever that it was a grave mistake to use dots in the
>>> semantic conventions. I understand the history thereof, but the moment
>>> that OTel self-declared as the overarching standard for all kind of
>>> telemetry, they should have realized that using a character that has a
>>> special meaning or is even an operator in so many languages is a
>>> really really bad idea. This is not just PromQL specific. Originally,
>>> I thought it's infeasible to change the semantic conventions at this
>>> point, but by now, that's exactly what I think OTel should do. If the
>>> dot were an actual operator in OTel (let's say a separator of actual
>>> 1st class namespaces) rather than just a convention within a
>>> technically opaque string, I could see some merit. But as it is not,
>>> it's just annoying and has no benefits whatsoever.
>>>
>>> Despite having said all of that, I don't realistically expect that
>>> OTel is going to change the semantic conventions. So next question is
>>> how to deal with it. There are many reasons why it's a bad idea to
>>> allow the dot in Prometheus metric names, most of them weren't
>>> mentioned in this thread. I won't enumerate them all again. We can do
>>> that if we really want to open that can of worms again. Segue to the
>>> next paragraph...
>>>
>>> In all the discussions we had before, my impression was that the
>>> consensus (in the spirit of RFC 7282) was to not add the dot to the
>>> characters that don't require quoting. As the saying goes, in OSS, a
>>> "no" is temporary and a "yes" is forever. So we can re-open this
>>> debate as often as anyone wishes. If the result is different at some
>>> point in the future, so be it. It's unlikely that I will change my
>>> mind (in fact, as alluded to above, I'm more convinced than ever that
>>> Prometheus should resist the urge). But that doesn't necessarily
>>> prevent an RFC-7282-style consensus. (Or we could also just have a
>>> vote, like in the old days, although that should be a last resort.)
>>> Despite the opinions expressed so far, I would doubt that I'm the only
>>> one who will be opposed.
>>>
>>> Julius has previously described quite nicely how OTel conventions and
>>> practices creep into the Prometheus ecosystem, undermining original
>>> properties of Prometheus as "simple, light-weight, and
>>> opinionated". The whole quoting syntax that opened this thread is for
>>> me a way of allowing what OTel needs but also of containing the damage
>>> and keep things in spirit for normal Prometheus users. Maybe another
>>> thing to include when explaining the syntax to normal Prometheus
>>> users.
>>>
>>> --
>>> Björn Rabenstein
>>> [PGP-ID] 0x851C3DA17D748D03
>>> [email] bjo...@rabenste.in
>>>
>> --
>> You received this message 

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Julius Volz]

Hah, I knew it would be a good idea to check with Björn :D Thanks Björn,
that's a great write-up!

Yes, that also convinced me about not allowing the dot as a normal
character for now. Lots of good arguments, but number 3 actually resonates
the most with me - why allow two different separator characters if they
have no semantic difference (no true namespacing).

On Wed, May 29, 2024 at 3:26 PM 'George Robinson' via Prometheus Developers
 wrote:

> Thanks Björn for writing this up, and also writing up Collected reasons
> why Prometheus doesn't allow dot as a regular character in metric and label
> names .
> I think it adds a huge amount of value for people looking to participate in
> discussion! Having read through it all I withdraw my original support for
> just adding dot to metric names. There are a lot of considerations I did
> not know about that I agree with thanks to your document.
>
> On Tuesday, May 28, 2024 at 11:17:10 PM UTC+1 Bjoern Rabenstein wrote:
>
>> I'm trying to keep things short, as all of this had been discussed
>> at length before.
>>
>> WRT "how to explain UTF-8 support to users": I actually don't think
>> this is a huge problem. I would frame it "this is like file
>> names". You can use blanks and slashes in Unix file names, and if you
>> do, it requires weird quoting or escaping, but that's not a huge
>> problem in practice. People just don't use them if they care. And if
>> they have to interact with other file sources, where blanks are
>> common, they cope. And yes, that means that names from OTel semantic
>> conventions will always be considered weird, but that's a problem of
>> OTel, not all the other languages where a dot has a special
>> meaning. Segue to the next paragraph...
>>
>> WRT the dot in OTel semantic conventions: Personally, I'm more
>> convinced than ever that it was a grave mistake to use dots in the
>> semantic conventions. I understand the history thereof, but the moment
>> that OTel self-declared as the overarching standard for all kind of
>> telemetry, they should have realized that using a character that has a
>> special meaning or is even an operator in so many languages is a
>> really really bad idea. This is not just PromQL specific. Originally,
>> I thought it's infeasible to change the semantic conventions at this
>> point, but by now, that's exactly what I think OTel should do. If the
>> dot were an actual operator in OTel (let's say a separator of actual
>> 1st class namespaces) rather than just a convention within a
>> technically opaque string, I could see some merit. But as it is not,
>> it's just annoying and has no benefits whatsoever.
>>
>> Despite having said all of that, I don't realistically expect that
>> OTel is going to change the semantic conventions. So next question is
>> how to deal with it. There are many reasons why it's a bad idea to
>> allow the dot in Prometheus metric names, most of them weren't
>> mentioned in this thread. I won't enumerate them all again. We can do
>> that if we really want to open that can of worms again. Segue to the
>> next paragraph...
>>
>> In all the discussions we had before, my impression was that the
>> consensus (in the spirit of RFC 7282) was to not add the dot to the
>> characters that don't require quoting. As the saying goes, in OSS, a
>> "no" is temporary and a "yes" is forever. So we can re-open this
>> debate as often as anyone wishes. If the result is different at some
>> point in the future, so be it. It's unlikely that I will change my
>> mind (in fact, as alluded to above, I'm more convinced than ever that
>> Prometheus should resist the urge). But that doesn't necessarily
>> prevent an RFC-7282-style consensus. (Or we could also just have a
>> vote, like in the old days, although that should be a last resort.)
>> Despite the opinions expressed so far, I would doubt that I'm the only
>> one who will be opposed.
>>
>> Julius has previously described quite nicely how OTel conventions and
>> practices creep into the Prometheus ecosystem, undermining original
>> properties of Prometheus as "simple, light-weight, and
>> opinionated". The whole quoting syntax that opened this thread is for
>> me a way of allowing what OTel needs but also of containing the damage
>> and keep things in spirit for normal Prometheus users. Maybe another
>> thing to include when explaining the syntax to normal Prometheus
>> users.
>>
>> --
>> Björn Rabenstein
>> [PGP-ID] 0x851C3DA17D748D03
>> [email] bjo...@rabenste.in
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/d14d59cf-204b-4215-afb8-3b5adee96be4n%40googlegroups.com
> 

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

['George Robinson' via Prometheus Developers]

Thanks Björn for writing this up, and also writing up Collected reasons why 
Prometheus doesn't allow dot as a regular character in metric and label 
names . I 
think it adds a huge amount of value for people looking to participate in 
discussion! Having read through it all I withdraw my original support for 
just adding dot to metric names. There are a lot of considerations I did 
not know about that I agree with thanks to your document.

On Tuesday, May 28, 2024 at 11:17:10 PM UTC+1 Bjoern Rabenstein wrote:

> I'm trying to keep things short, as all of this had been discussed
> at length before.
>
> WRT "how to explain UTF-8 support to users": I actually don't think
> this is a huge problem. I would frame it "this is like file
> names". You can use blanks and slashes in Unix file names, and if you
> do, it requires weird quoting or escaping, but that's not a huge
> problem in practice. People just don't use them if they care. And if
> they have to interact with other file sources, where blanks are
> common, they cope. And yes, that means that names from OTel semantic
> conventions will always be considered weird, but that's a problem of
> OTel, not all the other languages where a dot has a special
> meaning. Segue to the next paragraph...
>
> WRT the dot in OTel semantic conventions: Personally, I'm more
> convinced than ever that it was a grave mistake to use dots in the
> semantic conventions. I understand the history thereof, but the moment
> that OTel self-declared as the overarching standard for all kind of
> telemetry, they should have realized that using a character that has a
> special meaning or is even an operator in so many languages is a
> really really bad idea. This is not just PromQL specific. Originally,
> I thought it's infeasible to change the semantic conventions at this
> point, but by now, that's exactly what I think OTel should do. If the
> dot were an actual operator in OTel (let's say a separator of actual
> 1st class namespaces) rather than just a convention within a
> technically opaque string, I could see some merit. But as it is not,
> it's just annoying and has no benefits whatsoever.
>
> Despite having said all of that, I don't realistically expect that
> OTel is going to change the semantic conventions. So next question is
> how to deal with it. There are many reasons why it's a bad idea to
> allow the dot in Prometheus metric names, most of them weren't
> mentioned in this thread. I won't enumerate them all again. We can do
> that if we really want to open that can of worms again. Segue to the
> next paragraph...
>
> In all the discussions we had before, my impression was that the
> consensus (in the spirit of RFC 7282) was to not add the dot to the
> characters that don't require quoting. As the saying goes, in OSS, a
> "no" is temporary and a "yes" is forever. So we can re-open this
> debate as often as anyone wishes. If the result is different at some
> point in the future, so be it. It's unlikely that I will change my
> mind (in fact, as alluded to above, I'm more convinced than ever that
> Prometheus should resist the urge). But that doesn't necessarily
> prevent an RFC-7282-style consensus. (Or we could also just have a
> vote, like in the old days, although that should be a last resort.)
> Despite the opinions expressed so far, I would doubt that I'm the only
> one who will be opposed.
>
> Julius has previously described quite nicely how OTel conventions and
> practices creep into the Prometheus ecosystem, undermining original
> properties of Prometheus as "simple, light-weight, and
> opinionated". The whole quoting syntax that opened this thread is for
> me a way of allowing what OTel needs but also of containing the damage
> and keep things in spirit for normal Prometheus users. Maybe another
> thing to include when explaining the syntax to normal Prometheus
> users.
>
> -- 
> Björn Rabenstein
> [PGP-ID] 0x851C3DA17D748D03
> [email] bjo...@rabenste.in
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/d14d59cf-204b-4215-afb8-3b5adee96be4n%40googlegroups.com.

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Bjoern Rabenstein]

I'm trying to keep things short, as all of this had been discussed
at length before.

WRT "how to explain UTF-8 support to users": I actually don't think
this is a huge problem. I would frame it "this is like file
names". You can use blanks and slashes in Unix file names, and if you
do, it requires weird quoting or escaping, but that's not a huge
problem in practice. People just don't use them if they care. And if
they have to interact with other file sources, where blanks are
common, they cope. And yes, that means that names from OTel semantic
conventions will always be considered weird, but that's a problem of
OTel, not all the other languages where a dot has a special
meaning. Segue to the next paragraph...

WRT the dot in OTel semantic conventions: Personally, I'm more
convinced than ever that it was a grave mistake to use dots in the
semantic conventions. I understand the history thereof, but the moment
that OTel self-declared as the overarching standard for all kind of
telemetry, they should have realized that using a character that has a
special meaning or is even an operator in so many languages is a
really really bad idea. This is not just PromQL specific. Originally,
I thought it's infeasible to change the semantic conventions at this
point, but by now, that's exactly what I think OTel should do. If the
dot were an actual operator in OTel (let's say a separator of actual
1st class namespaces) rather than just a convention within a
technically opaque string, I could see some merit. But as it is not,
it's just annoying and has no benefits whatsoever.

Despite having said all of that, I don't realistically expect that
OTel is going to change the semantic conventions. So next question is
how to deal with it. There are many reasons why it's a bad idea to
allow the dot in Prometheus metric names, most of them weren't
mentioned in this thread. I won't enumerate them all again. We can do
that if we really want to open that can of worms again. Segue to the
next paragraph...

In all the discussions we had before, my impression was that the
consensus (in the spirit of RFC 7282) was to not add the dot to the
characters that don't require quoting. As the saying goes, in OSS, a
"no" is temporary and a "yes" is forever. So we can re-open this
debate as often as anyone wishes. If the result is different at some
point in the future, so be it. It's unlikely that I will change my
mind (in fact, as alluded to above, I'm more convinced than ever that
Prometheus should resist the urge). But that doesn't necessarily
prevent an RFC-7282-style consensus. (Or we could also just have a
vote, like in the old days, although that should be a last resort.)
Despite the opinions expressed so far, I would doubt that I'm the only
one who will be opposed.

Julius has previously described quite nicely how OTel conventions and
practices creep into the Prometheus ecosystem, undermining original
properties of Prometheus as "simple, light-weight, and
opinionated". The whole quoting syntax that opened this thread is for
me a way of allowing what OTel needs but also of containing the damage
and keep things in spirit for normal Prometheus users. Maybe another
thing to include when explaining the syntax to normal Prometheus
users.

-- 
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/ZlZX4SJpb/ajSLkd%40mail.rabenste.in.

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Julius Volz]

On Mon, May 27, 2024 at 10:07 AM 'George Robinson' via Prometheus
Developers  wrote:

> Hi! I worked with Björn and Josh to add support for UTF-8 in Alertmanager,
> and I thought I would share my opinion on this matter as we had to solve
> similar problems there.
>
> I'm with Bryan and Fabian on the dot operator. If dot is all that's needed
> to support the OTel character set for metric names in Prometheus, and we
> don't expect further relaxations of this character set, then I think we
> should add it to Prometheus. We can then defer the problem of supporting
> UTF-8 in metrics names and how it would work with the existing PromQL
> grammar.
>
> I also wouldn't worry about future use of dot as a syntactic sugar for
> label matchers or native histograms. I personally don't think it adds much
> to PromQL over the more verbose foo{job="bar"} or
> histogram_sum(sum(request_duration_seconds)), and I would argue it makes
> PromQL more difficult to learn.
>

>From what I've heard so far, I would tend to agree with all that. However,
I'd like to make sure that Björn also weighs in on the dot character
question, since he was the main person raising concerns about it in the
past. CC-ing him here :)

On Björn's suggestion of __name__, I don't share Julius' concerns. In fact,
> I think it's quite nice because:
>
> 1. You can and sometimes have to use __name__ in Prometheus if the metric
> name conflicts with a reserved word. It is also a documented feature in
> Prometheus and extending it for metric names with UTF-8 seems very sensible
> to me.
>

While this is technically true, I've never seen such a reserved-word
collision in practice, and the problem would only occur for a few reserved
words, like "on"/"ignoring", but e.g. not for just "rate" or "by" or "sum"
etc. when they are grammatically used as a metric name. Other than that,
you only ever need to know about "__name__" if your metrics are structured
weirdly and you need to do regexes against them or if you're doing
admin-level queries.


> 2. In TSDB, the metric name is just a label anyway, is it not? If
> anything, once you understand how Prometheus stores metrics, querying a
> metric using __name__ feels more natural than the current PromQL grammar
> where it is separate from all other label matchers.
>

Yeah, I think it's good for people to know that it's just a special label,
but you should almost never need to access "__name__" unless your metrics
are structured badly (encoding things in the metric name that should really
be in a label) or if you are doing admin-level Prometheus stuff (e.g.
figuring out which dimensional groupings produce how many time series in
your Prometheus server). So I think requiring power users to use "__name__"
for those exceptional queries is acceptable, but requiring it for just
every-day normal user queries would make the query language less
approachable.

However, I'm not convinced that we should adopt the short {="metric_name"}
> or optional = as in {"metric_name"}. In this case I think consistency and
> uniformity of the grammar is more important than verbosity.
>

Yeah, I'm split on that. On one hand, I don't want people to have to type
the internal "__name__" label, on the other hand I like things to be
explicit as well. Maybe one could have a special string syntax (e.g.
prefixed somehow like in Python) for metric names that doesn't collide with
our existing strings, so you could just write something like:

m"some.metric.name.with.dots"{foo="bar"}

I'm not sure if I like that much more though 

However, this is all irrelevant if we choose to accept dot anyway :)
>

As long as people stick to OTel conventions at least, yeah :)

Cheers,
Julius


>
> George
>
>
>
>
> On Friday, May 24, 2024 at 2:44:17 PM UTC+1 Julius Volz wrote:
>
>> Yeah, that's true, that would help a lot already. Maybe it's time to open
>> up the dot after all. I know we've kept it reserved for "reasons" (first as
>> a potential "job" label separator, then later for other potential future
>> ideas like ".sum" / ".count" for native histograms), but maybe it's worth
>> finally using it for this. I know Björn has the strongest opinions on the
>> dot :)
>>
>> On Fri, May 24, 2024 at 1:31 PM Fabian Stäber  wrote:
>>
>>> What Bryan said. OpenTelemetry's semantic conventions don't use all of
>>> UTF-8. They are restricted to the same character set as Prometheus plus the
>>> dot.
>>> If we simply allow the dot, we cover all of OTel's semantic conventions
>>> for metric names and label names.
>>> I don't care if custom metrics that use arbitrary UTF-8 characters
>>> require a weird escaping syntax.
>>> The goal should be to make OpenTelemetry's semantic conventions work
>>> nicely with Prometheus.
>>>
>>> Fabian
>>>
>>> On Fri, May 24, 2024 at 11:52 AM Bryan Boreham 
>>> wrote:
>>>
 I think we should allow dot (.) in names without quoting.

 It gives up a possible operator, but makes the syntax nicer for the
 vast majority of people who need 

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

['George Robinson' via Prometheus Developers]

Hi! I worked with Björn and Josh to add support for UTF-8 in Alertmanager, 
and I thought I would share my opinion on this matter as we had to solve 
similar problems there.

I'm with Bryan and Fabian on the dot operator. If dot is all that's needed 
to support the OTel character set for metric names in Prometheus, and we 
don't expect further relaxations of this character set, then I think we 
should add it to Prometheus. We can then defer the problem of supporting 
UTF-8 in metrics names and how it would work with the existing PromQL 
grammar.

I also wouldn't worry about future use of dot as a syntactic sugar for 
label matchers or native histograms. I personally don't think it adds much 
to PromQL over the more verbose foo{job="bar"} or 
histogram_sum(sum(request_duration_seconds)), and I would argue it makes 
PromQL more difficult to learn.

On Björn's suggestion of __name__, I don't share Julius' concerns. In fact, 
I think it's quite nice because:

1. You can and sometimes have to use __name__ in Prometheus if the metric 
name conflicts with a reserved word. It is also a documented feature in 
Prometheus and extending it for metric names with UTF-8 seems very sensible 
to me.
2. In TSDB, the metric name is just a label anyway, is it not? If anything, 
once you understand how Prometheus stores metrics, querying a metric using 
__name__ feels more natural than the current PromQL grammar where it is 
separate from all other label matchers.

However, I'm not convinced that we should adopt the short {="metric_name"} 
or optional = as in {"metric_name"}. In this case I think consistency and 
uniformity of the grammar is more important than verbosity.

However, this is all irrelevant if we choose to accept dot anyway :)

George

  


On Friday, May 24, 2024 at 2:44:17 PM UTC+1 Julius Volz wrote:

> Yeah, that's true, that would help a lot already. Maybe it's time to open 
> up the dot after all. I know we've kept it reserved for "reasons" (first as 
> a potential "job" label separator, then later for other potential future 
> ideas like ".sum" / ".count" for native histograms), but maybe it's worth 
> finally using it for this. I know Björn has the strongest opinions on the 
> dot :)
>
> On Fri, May 24, 2024 at 1:31 PM Fabian Stäber  wrote:
>
>> What Bryan said. OpenTelemetry's semantic conventions don't use all of 
>> UTF-8. They are restricted to the same character set as Prometheus plus the 
>> dot.
>> If we simply allow the dot, we cover all of OTel's semantic conventions 
>> for metric names and label names.
>> I don't care if custom metrics that use arbitrary UTF-8 characters 
>> require a weird escaping syntax.
>> The goal should be to make OpenTelemetry's semantic conventions work 
>> nicely with Prometheus.
>>
>> Fabian
>>
>> On Fri, May 24, 2024 at 11:52 AM Bryan Boreham  wrote:
>>
>>> I think we should allow dot (.) in names without quoting. 
>>>
>>> It gives up a possible operator, but makes the syntax nicer for the vast 
>>> majority of people who need a change. 
>>>
>>> Bryan
>>>
>>> On 24 May 2024, at 10:00, Julius Volz  wrote:
>>>
>>> 
>>> Hi,
>>>
>>> While others are figuring out how to add UTF-8 support and other OTel 
>>> compatibility functionality to Prometheus, my brain has been trying to 
>>> figure out what all of this will mean for the normal Prometheus user, how 
>>> we should explain this new optionality in Prometheus, and what we should 
>>> recommend with regards to UTF-8 usage outside of the OTel compatibility 
>>> scenario.
>>>
>>> What (literally) keeps me up at night the most is the new selector 
>>> syntax that metric names with previously illegal characters will now 
>>> require: 
>>> https://docs.google.com/document/d/1yFj5QSd1AgCYecZ9EJ8f2t4OgF2KBZgJYVde-uzVEtI/edit#heading=h.yxv3hzhog2l2
>>> .
>>>
>>> In this proposal, Björn did the best he could given the difficult 
>>> situation that OTel compatibility support push puts us in, and I also don't 
>>> have better ideas for nicer syntaxes. However, it looks like all current 
>>> selector syntax suggestions require putting the metric name not only in 
>>> quotes, but also within the curly braces, and possibly even introducing new 
>>> (IMO) very odd short-hand syntaxes for it.
>>>
>>> Given that I think this new selector syntax is very unelegant, inferior, 
>>> and confusing (harder to type, harder to read and understand intuitively) 
>>> compared to Prometheus' really clean and nice existing selector syntax that 
>>> we've had for over 10 years, how should we approach UTF-8 support when 
>>> explaining it to users? Should we say, "Yes, the Prometheus data model now 
>>> allows any UTF-8 characters in metric names, but you should really ignore 
>>> this fact completely and stick to the old character set, unless you have to 
>>> deal with UTF-8 for the purpose of OTel metrics because otherwise your 
>>> PromQL will start looking really weird"? Or do we not give any 
>>> recommendations around it at all?
>>>
>>> I just 

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Julius Volz]

Yeah, that's true, that would help a lot already. Maybe it's time to open
up the dot after all. I know we've kept it reserved for "reasons" (first as
a potential "job" label separator, then later for other potential future
ideas like ".sum" / ".count" for native histograms), but maybe it's worth
finally using it for this. I know Björn has the strongest opinions on the
dot :)

On Fri, May 24, 2024 at 1:31 PM Fabian Stäber  wrote:

> What Bryan said. OpenTelemetry's semantic conventions don't use all of
> UTF-8. They are restricted to the same character set as Prometheus plus the
> dot.
> If we simply allow the dot, we cover all of OTel's semantic conventions
> for metric names and label names.
> I don't care if custom metrics that use arbitrary UTF-8 characters require
> a weird escaping syntax.
> The goal should be to make OpenTelemetry's semantic conventions work
> nicely with Prometheus.
>
> Fabian
>
> On Fri, May 24, 2024 at 11:52 AM Bryan Boreham 
> wrote:
>
>> I think we should allow dot (.) in names without quoting.
>>
>> It gives up a possible operator, but makes the syntax nicer for the vast
>> majority of people who need a change.
>>
>> Bryan
>>
>> On 24 May 2024, at 10:00, Julius Volz  wrote:
>>
>> 
>> Hi,
>>
>> While others are figuring out how to add UTF-8 support and other OTel
>> compatibility functionality to Prometheus, my brain has been trying to
>> figure out what all of this will mean for the normal Prometheus user, how
>> we should explain this new optionality in Prometheus, and what we should
>> recommend with regards to UTF-8 usage outside of the OTel compatibility
>> scenario.
>>
>> What (literally) keeps me up at night the most is the new selector syntax
>> that metric names with previously illegal characters will now require:
>> https://docs.google.com/document/d/1yFj5QSd1AgCYecZ9EJ8f2t4OgF2KBZgJYVde-uzVEtI/edit#heading=h.yxv3hzhog2l2
>> .
>>
>> In this proposal, Björn did the best he could given the difficult
>> situation that OTel compatibility support push puts us in, and I also don't
>> have better ideas for nicer syntaxes. However, it looks like all current
>> selector syntax suggestions require putting the metric name not only in
>> quotes, but also within the curly braces, and possibly even introducing new
>> (IMO) very odd short-hand syntaxes for it.
>>
>> Given that I think this new selector syntax is very unelegant, inferior,
>> and confusing (harder to type, harder to read and understand intuitively)
>> compared to Prometheus' really clean and nice existing selector syntax that
>> we've had for over 10 years, how should we approach UTF-8 support when
>> explaining it to users? Should we say, "Yes, the Prometheus data model now
>> allows any UTF-8 characters in metric names, but you should really ignore
>> this fact completely and stick to the old character set, unless you have to
>> deal with UTF-8 for the purpose of OTel metrics because otherwise your
>> PromQL will start looking really weird"? Or do we not give any
>> recommendations around it at all?
>>
>> I just fear that UTF-8 characters will start creeping in everywhere
>> because people see that it's supported. And then all users will eventually
>> just have to start using the new syntax for everything, because "that's the
>> one that always works", and because they didn't even know or think about
>> that fact when they were creating their metrics. In any case, all this will
>> require a whole lot of new explanations for every new Prometheus user, and
>> the same is true for some of the other OTel features.
>>
>> There's other things that worry me about UTF-8 support and OTel support
>> in general, but the selector syntax and what this will ultimately do to
>> Prometheus users down the road is the main one. Any opinions on how we
>> should advertise this to the regular (non-OTel) Prometheus user going
>> forward?
>>
>> Cheers,
>> Julius
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6YowuTe6tLp1N0jEQc5zhAQE0RDpdME4tP4m-uXEmRKqwPA%40mail.gmail.com
>> 
>> .
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/C96DD934-9A56-4A1E-B1CA-E45AAD1B1E41%40gmail.com
>> 

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

['Fabian Stäber' via Prometheus Developers]

What Bryan said. OpenTelemetry's semantic conventions don't use all of
UTF-8. They are restricted to the same character set as Prometheus plus the
dot.
If we simply allow the dot, we cover all of OTel's semantic conventions for
metric names and label names.
I don't care if custom metrics that use arbitrary UTF-8 characters require
a weird escaping syntax.
The goal should be to make OpenTelemetry's semantic conventions work nicely
with Prometheus.

Fabian

On Fri, May 24, 2024 at 11:52 AM Bryan Boreham  wrote:

> I think we should allow dot (.) in names without quoting.
>
> It gives up a possible operator, but makes the syntax nicer for the vast
> majority of people who need a change.
>
> Bryan
>
> On 24 May 2024, at 10:00, Julius Volz  wrote:
>
> 
> Hi,
>
> While others are figuring out how to add UTF-8 support and other OTel
> compatibility functionality to Prometheus, my brain has been trying to
> figure out what all of this will mean for the normal Prometheus user, how
> we should explain this new optionality in Prometheus, and what we should
> recommend with regards to UTF-8 usage outside of the OTel compatibility
> scenario.
>
> What (literally) keeps me up at night the most is the new selector syntax
> that metric names with previously illegal characters will now require:
> https://docs.google.com/document/d/1yFj5QSd1AgCYecZ9EJ8f2t4OgF2KBZgJYVde-uzVEtI/edit#heading=h.yxv3hzhog2l2
> .
>
> In this proposal, Björn did the best he could given the difficult
> situation that OTel compatibility support push puts us in, and I also don't
> have better ideas for nicer syntaxes. However, it looks like all current
> selector syntax suggestions require putting the metric name not only in
> quotes, but also within the curly braces, and possibly even introducing new
> (IMO) very odd short-hand syntaxes for it.
>
> Given that I think this new selector syntax is very unelegant, inferior,
> and confusing (harder to type, harder to read and understand intuitively)
> compared to Prometheus' really clean and nice existing selector syntax that
> we've had for over 10 years, how should we approach UTF-8 support when
> explaining it to users? Should we say, "Yes, the Prometheus data model now
> allows any UTF-8 characters in metric names, but you should really ignore
> this fact completely and stick to the old character set, unless you have to
> deal with UTF-8 for the purpose of OTel metrics because otherwise your
> PromQL will start looking really weird"? Or do we not give any
> recommendations around it at all?
>
> I just fear that UTF-8 characters will start creeping in everywhere
> because people see that it's supported. And then all users will eventually
> just have to start using the new syntax for everything, because "that's the
> one that always works", and because they didn't even know or think about
> that fact when they were creating their metrics. In any case, all this will
> require a whole lot of new explanations for every new Prometheus user, and
> the same is true for some of the other OTel features.
>
> There's other things that worry me about UTF-8 support and OTel support in
> general, but the selector syntax and what this will ultimately do to
> Prometheus users down the road is the main one. Any opinions on how we
> should advertise this to the regular (non-OTel) Prometheus user going
> forward?
>
> Cheers,
> Julius
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6YowuTe6tLp1N0jEQc5zhAQE0RDpdME4tP4m-uXEmRKqwPA%40mail.gmail.com
> 
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/C96DD934-9A56-4A1E-B1CA-E45AAD1B1E41%40gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CAPX310iyemf6SE8TO6OJ_9_sS3Zr9R%3DOizCOpynsYwN9uZisng%40mail.gmail.com.

Re: [prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Bryan Boreham]

I think we should allow dot (.) in names without quoting. It gives up a possible operator, but makes the syntax nicer for the vast majority of people who need a change. BryanOn 24 May 2024, at 10:00, Julius Volz  wrote:Hi,While others are figuring out how to add UTF-8 support and other OTel compatibility functionality to Prometheus, my brain has been trying to figure out what all of this will mean for the normal Prometheus user, how we should explain this new optionality in Prometheus, and what we should recommend with regards to UTF-8 usage outside of the OTel compatibility scenario.What (literally) keeps me up at night the most is the new selector syntax that metric names with previously illegal characters will now require: https://docs.google.com/document/d/1yFj5QSd1AgCYecZ9EJ8f2t4OgF2KBZgJYVde-uzVEtI/edit#heading=h.yxv3hzhog2l2.In this proposal, Björn did the best he could given the difficult situation that OTel compatibility support push puts us in, and I also don't have better ideas for nicer syntaxes. However, it looks like all current selector syntax suggestions require putting the metric name not only in quotes, but also within the curly braces, and possibly even introducing new (IMO) very odd short-hand syntaxes for it.Given that I think this new selector syntax is very unelegant, inferior, and confusing (harder to type, harder to read and understand intuitively) compared to Prometheus' really clean and nice existing selector syntax that we've had for over 10 years, how should we approach UTF-8 support when explaining it to users? Should we say, "Yes, the Prometheus data model now allows any UTF-8 characters in metric names, but you should really ignore this fact completely and stick to the old character set, unless you have to deal with UTF-8 for the purpose of OTel metrics because otherwise your PromQL will start looking really weird"? Or do we not give any recommendations around it at all?I just fear that UTF-8 characters will start creeping in everywhere because people see that it's supported. And then all users will eventually just have to start using the new syntax for everything, because "that's the one that always works", and because they didn't even know or think about that fact when they were creating their metrics. In any case, all this will require a whole lot of new explanations for every new Prometheus user, and the same is true for some of the other OTel features.There's other things that worry me about UTF-8 support and OTel support in general, but the selector syntax and what this will ultimately do to Prometheus users down the road is the main one. Any opinions on how we should advertise this to the regular (non-OTel) Prometheus user going forward?Cheers,Julius



-- 
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6YowuTe6tLp1N0jEQc5zhAQE0RDpdME4tP4m-uXEmRKqwPA%40mail.gmail.com.




-- 
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/C96DD934-9A56-4A1E-B1CA-E45AAD1B1E41%40gmail.com.

[prometheus-developers] Limiting the blast radius of OTel / UTF-8 support for normal Prometheus users?

[Julius Volz]

Hi,

While others are figuring out how to add UTF-8 support and other OTel
compatibility functionality to Prometheus, my brain has been trying to
figure out what all of this will mean for the normal Prometheus user, how
we should explain this new optionality in Prometheus, and what we should
recommend with regards to UTF-8 usage outside of the OTel compatibility
scenario.

What (literally) keeps me up at night the most is the new selector syntax
that metric names with previously illegal characters will now require:
https://docs.google.com/document/d/1yFj5QSd1AgCYecZ9EJ8f2t4OgF2KBZgJYVde-uzVEtI/edit#heading=h.yxv3hzhog2l2
.

In this proposal, Björn did the best he could given the difficult situation
that OTel compatibility support push puts us in, and I also don't have
better ideas for nicer syntaxes. However, it looks like all current
selector syntax suggestions require putting the metric name not only in
quotes, but also within the curly braces, and possibly even introducing new
(IMO) very odd short-hand syntaxes for it.

Given that I think this new selector syntax is very unelegant, inferior,
and confusing (harder to type, harder to read and understand intuitively)
compared to Prometheus' really clean and nice existing selector syntax that
we've had for over 10 years, how should we approach UTF-8 support when
explaining it to users? Should we say, "Yes, the Prometheus data model now
allows any UTF-8 characters in metric names, but you should really ignore
this fact completely and stick to the old character set, unless you have to
deal with UTF-8 for the purpose of OTel metrics because otherwise your
PromQL will start looking really weird"? Or do we not give any
recommendations around it at all?

I just fear that UTF-8 characters will start creeping in everywhere because
people see that it's supported. And then all users will eventually just
have to start using the new syntax for everything, because "that's the one
that always works", and because they didn't even know or think about that
fact when they were creating their metrics. In any case, all this will
require a whole lot of new explanations for every new Prometheus user, and
the same is true for some of the other OTel features.

There's other things that worry me about UTF-8 support and OTel support in
general, but the selector syntax and what this will ultimately do to
Prometheus users down the road is the main one. Any opinions on how we
should advertise this to the regular (non-OTel) Prometheus user going
forward?

Cheers,
Julius

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6YowuTe6tLp1N0jEQc5zhAQE0RDpdME4tP4m-uXEmRKqwPA%40mail.gmail.com.