Re: [prometheus-users] Tracking Metrics Per Session/Request

[Stuart Clark]

On 03/07/2023 19:59, KW wrote:

Hello,

I work with an ASP.NET Core API that uses sessions to store the user's 
state. The API calls out to many other different microservices to run 
logic based on the user's request, we update and save the state on the 
server, and return the relevant information in the response. I'm 
looking to be able to troubleshoot performance issues by looking at 
metrics for an entire session. I roughly understand how I can add 
timing metrics, but what is the mechanism I'd use to differentiate 
sessions, and even similar requests within the session? For example, 
I'd like to know that the third identical request that was made on a 
given session took 43 seconds while the others were only 2 seconds. If 
I use a label for session id, I won't see the individual request 
timings... would I need to create an additional "request id" label 
that I can use? Perhaps pull the Activity.Current trace id that is 
used for exemplars? Not sure what the best practices are here.


This sounds like you might actually want traces rather than just 
metrics, which can be added using Tempo or other tools.


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/1211d451-51ff-440b-6938-3df8dbb30d94%40Jahingo.com.

[prometheus-users] Tracking Metrics Per Session/Request

[KW]

Hello,

I work with an ASP.NET Core API that uses sessions to store the user's 
state. The API calls out to many other different microservices to run logic 
based on the user's request, we update and save the state on the server, 
and return the relevant information in the response. I'm looking to be able 
to troubleshoot performance issues by looking at metrics for an entire 
session. I roughly understand how I can add timing metrics, but what is the 
mechanism I'd use to differentiate sessions, and even similar requests 
within the session? For example, I'd like to know that the third identical 
request that was made on a given session took 43 seconds while the others 
were only 2 seconds. If I use a label for session id, I won't see the 
individual request timings... would I need to create an additional "request 
id" label that I can use? Perhaps pull the Activity.Current trace id that 
is used for exemplars? Not sure what the best practices are here.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/603b72ce-071c-4c1b-b23c-2ccc92243db0n%40googlegroups.com.

Re: [prometheus-users] How to override severity of a built in alert?

[Julius Volz]

Hi David,

Prometheus does not have built-in alerts, but I assume you are using some
framework around Prometheus that configures a "KubeJobFailed" alert? Is it
kube-prometheus with the kubernetes-mixin (since that includes an example
alert by that name:
https://github.com/kubernetes-monitoring/kubernetes-mixin/blob/003ba5eadfbd69817d1215952133d3ecf99fbd92/alerts/apps_alerts.libsonnet#L268-L281)?
If you share more about your kube-prometheus or kubernetes-mixin jsonnet
pipeline, I'm sure someone could help. Although at that point it's probably
more of a jsonnet question than a Prometheus one.

Kind regards,
Julius

On Mon, Jul 3, 2023 at 9:11 AM David Dean  wrote:

> Hi -
>
> How can I modify the severity of built in alert KubeJobFailed from warning
> to info?
>
> I don't want to change anything else about the built in alert, only the
> severity.
>
> And I only want to change the one alert, KubeJobFailed, not any other.
>
> I need to change the severity to info so PagerDuty will not generate
> alerts outside working hours. In our scenario, Kubernetes Jobs don't need
> an immediate callout.
>
> I've spent ages Googling and checking the docs but cannot find out how to
> do it anywhere!
>
> Thanks, Dave
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/2c6284f5-35a6-43d9-b892-41803d8bd4d9n%40googlegroups.com
> 
> .
>


-- 
Julius Volz
PromLabs - promlabs.com

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CAObpH5zTY_ZKngy6Pf2n768bNT1H%2B%2BW2jgKydj8cJGg63dNXLQ%40mail.gmail.com.

[prometheus-users] How to override severity of a built in alert?

[David Dean]

Hi -

How can I modify the severity of built in alert KubeJobFailed from warning 
to info?

I don't want to change anything else about the built in alert, only the 
severity.

And I only want to change the one alert, KubeJobFailed, not any other.

I need to change the severity to info so PagerDuty will not generate alerts 
outside working hours. In our scenario, Kubernetes Jobs don't need an 
immediate callout.

I've spent ages Googling and checking the docs but cannot find out how to 
do it anywhere!

Thanks, Dave

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/2c6284f5-35a6-43d9-b892-41803d8bd4d9n%40googlegroups.com.