Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-13 Thread Gyula Fóra
> > is > > > > > > not > > > > > > > a hard guarantee. > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > Xintong > > > > > > >

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-13 Thread Venkatakrishnan Sowrirajan
> is > > > > > not > > > > > > a hard guarantee. > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > Xintong > > > > > > > > > > > > > > > > > > > > > > > > On Wed

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-13 Thread Gyula Fóra
> > > > > > > > > > I think the currently exposed info is all that is available > through > > > > > > GarbageCollectorMXBean. This FLIP does not aim to introduce a new > > > more > > > > > > granular way of reporting th

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
; > > new mechanism and may be a breaking change. > > > > > > > > > > We basically want to simply extend the current reporting here with > > the > > > > rate > > > > > metrics and the total metrics. > > > > > > > > > > Gyula > > >

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Gyula Fóra
rics. > > > > > > > > Gyula > > > > > > > > On Wed, Sep 6, 2023 at 9:24 AM Matt Wang wrote: > > > > > > > > > Hi Gyula, > > > > > > > > > > +1 for this proposal. > > > > > > > > &

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
23 at 9:24 AM Matt Wang wrote: > > > > > > > Hi Gyula, > > > > > > > > +1 for this proposal. > > > > > > > > Do we need to add a metric to record the count of different > > > > collectors? Now there is only a total count. For example, > &g

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Gyula Fóra
this proposal. > > > > > > Do we need to add a metric to record the count of different > > > collectors? Now there is only a total count. For example, > > > for G1, there is no way to distinguish whether it is the > > > young generation or the old generation. > > > > > >

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
, > > for G1, there is no way to distinguish whether it is the > > young generation or the old generation. > > > > > > > > -- > > > > Best, > > Matt Wang > > > > > > Replied Message > > | From | Gyula Fóra | >

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Gyula Fóra
count. For example, > for G1, there is no way to distinguish whether it is the > young generation or the old generation. > > > > -- > > Best, > Matt Wang > > > Replied Message > | From | Gyula Fóra | > | Date | 09/6/2023 15:03 | > | To | | >

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread liu ron
> | Date | 09/6/2023 15:03 | > | To | | > | Subject | Re: [DISCUSS] FLIP-361: Improve GC Metrics | > Thanks Xintong! > > Just so I understand correctly, do you suggest adding a metric for > delta(Time) / delta(Count) since the last reporting ? > .TimePerGc or .AverageTime would

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Matt Wang
| From | Gyula Fóra | | Date | 09/6/2023 15:03 | | To | | | Subject | Re: [DISCUSS] FLIP-361: Improve GC Metrics | Thanks Xintong! Just so I understand correctly, do you suggest adding a metric for delta(Time) / delta(Count) since the last reporting ? .TimePerGc or .AverageTime would make

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Gyula Fóra
Thanks Xintong! Just so I understand correctly, do you suggest adding a metric for delta(Time) / delta(Count) since the last reporting ? .TimePerGc or .AverageTime would make sense. AverageTime may be a bit nicer :) My only concern is how useful this will be in reality. If there are only (or

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
Thanks for bringing this up, Gyula. The proposed changes make sense to me. +1 for them. In addition to the proposed changes, I wonder if we should also add something like timePerGc? This would help understand whether there are long pauses, due to GC STW, that may lead to rpc unresponsiveness and

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Rui Fan
Thanks for the clarification! By default the meterview measures for 1 minute sounds good to me! +1 for this proposal. Best, Rui On Wed, Sep 6, 2023 at 1:27 PM Gyula Fóra wrote: > Thanks for the feedback Rui, > > The rates would be computed using the MeterView class (like for any other > rate

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-05 Thread Gyula Fóra
Thanks for the feedback Rui, The rates would be computed using the MeterView class (like for any other rate metric), just because we report the value per second it doesn't mean that we measure in a second granularity. By default the meterview measures for 1 minute and then we calculate the per

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-05 Thread Rui Fan
Hi Gyula, +1 for this proposal. The current GC metric is really unfriendly. I have a concern with your proposed rate metric: the rate is perSecond instead of per minute. I'm unsure whether it's suitable for GC metric. There are two reasons why I suspect perSecond may not be well compatible with

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-05 Thread Maximilian Michels
Hi Gyula, +1 The proposed changes make sense and are in line with what is available for other metrics, e.g. number of records processed. -Max On Tue, Sep 5, 2023 at 2:43 PM Gyula Fóra wrote: > > Hi Devs, > > I would like to start a discussion on FLIP-361: Improve GC Metrics [1]. > > The

[DISCUSS] FLIP-361: Improve GC Metrics

2023-09-05 Thread Gyula Fóra
Hi Devs, I would like to start a discussion on FLIP-361: Improve GC Metrics [1]. The current Flink GC metrics [2] are not very useful for monitoring purposes as they require post processing logic that is also dependent on the current runtime environment. Problems: - Total time is not very