2. Metrics
> > >
> > > I agree the new metrics should be compatible with the rest of the Flink
> > > metric reporting mechanism. I will update the FLIP and propose names
> for
> > > the metrics.
> > >
> > > Kind regards,
> > > Emre
>
> > extensions to the proposal that are not captured in the FLIP.
> >
> >
> >
> >
> > 1. Configurability - what kind of configuration would you propose to
> > maintain for this feature? Would On/off switch and/or aggregated period
> > length be configura
o not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > Hi Emre,
> >
> >
> > Thank you for driving this proposal. I've got two questions a
r...@amazon.co.uk.inva> kar...@amazon.co.uk.inva <mailto:kar...@amazon.co.uk.inva>>LID>
> Sent: Monday, January 15, 2024 4:59 PM
> To: dev@flink.apache.org <mailto:dev@flink.apache.org>
> <mailto:dev@flink.apache.org <mailto:dev@flink.apache.org>> <
>
skew metric via metric reporters
> mechanism. Should we capture proposed metric schema in the FLIP ?
>
>
> Kind regards,
> Krzysztof
>
>
> ________________
> From: Kartoglu, Emre kar...@amazon.co.uk.inva>LID>
> Sent: Monday, January 15, 2024 4:5
2024 4:59 PM
To: dev@flink.apache.org <mailto:dev@flink.apache.org> mailto:dev@flink.apache.org>>
Subject: [DISCUSS] FLIP-418: Show data skew score on Flink Dashboard
Hello,
I’m opening this thread to discuss a FLIP[1] to make data skew more visible on
Flink Dashboard.
Data skew i
onday, January 15, 2024 4:59 PM
To: dev@flink.apache.org
Subject: [DISCUSS] FLIP-418: Show data skew score on Flink Dashboard
Hello,
I’m opening this thread to discuss a FLIP[1] to make data skew more visible on
Flink Dashboard.
Data skew is currently not as visible as it should be. Users ha
Hi Xuyang,
Thanks for the feedback! Please find my response below.
> 1. How will the colors of vertics with high data skew scores be unified with
> existing backpressure and high busyness
colors on the UI? Users should be able to distinguish at a glance which vertics
in the entire job graph is
Hi Rui,
Thanks for the feedback. Please find my response below:
> The number_of_records_received_by_each_subtask is the total received records,
> right?
No it's not the total. I understand why this is confusing. I had initially
wanted to name it "the list of number of records received by each
Thanks Emre for driving this proposal!
It's very useful for troubleshooting.
I have a question:
The number_of_records_received_by_each_subtask is the
total received records, right?
I'm not sure whether we should check data skew based on
the latest duration period.
In the production, I found th
Hi, Emre.
In large-scale production jobs, the phenomenon of data skew often occurs.
Having an metric on the UI that
reflects data skew without the need for manual inspection of each vertex by
clicking on them would be quite cool.
This could help users quickly identify problematic nodes, simplif
Hello,
I’m opening this thread to discuss a FLIP[1] to make data skew more visible on
Flink Dashboard.
Data skew is currently not as visible as it should be. Users have to click each
operator and check how much data each sub-task is processing and compare the
sub-tasks against each other. This
12 matches
Mail list logo
