I am happy to announce that “FLIP-418: Show data skew score on Flink Dashboard”
has been accepted with Consensus.
FLIP:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-418%3A+Show+data+skew+score+on+Flink+Dashboard
Votes:
* Aleksandr Pilipenko +1 (non-binding)
* Danny Cranmer +
Thanks all, this vote is now closed. I will announce the results on a separate
thread.
On 2024/01/29 10:09:10 "Kartoglu, Emre" wrote:
> Hello,
>
> I'd like to call votes on FLIP-418: Show data skew score on Flink Dashboard.
>
> FLIP:
> https://cwiki.apache.org/
r learning a series
of backgrounds.
For example, as you mentioned before, flink has a metric:
numRecordsInPerSecond.
I believe users know what numRecordsInPerSecond means even if they
didn't read any documentation.
Of course, I'm opening for it. I may have missed something. I'd
ceived record numbers are
[10,10, 10, 100, 10].
data skew score = max / mean = 100 / (140/5) = 100/ 28 = 3.57.
The data skew score is between 1 and infinity. 1 is the best, and
the bigger the worse.
Looking forward to your opinions.
Best,
Rui
On Tue, Jan 23, 2024 at 6:41 PM Kartoglu, Emre mailt
Hello,
I'd like to call votes on FLIP-418: Show data skew score on Flink Dashboard.
FLIP:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-418%3A+Show+data+skew+score+on+Flink+Dashboard
Discussion: https://lists.apache.org/thread/m5ockoork0h2zr78h77dcrn71rbt35ql
Kind regards,
Emre
the FLIP ?
2. Metrics - are we planning to emit the skew metric via metric reporters
mechanism. Should we capture proposed metric schema in the FLIP ?
Kind regards,
Krzysztof
____
From: Kartoglu, Emre mailto:kar...@amazon.co.uk.inva>LID>
Sent: Monday, January 15,
ire job graph is skewed.
2. Can you tell me that you prefer to unify Data Skew Score and Exception tab?
In my opinion, Data Skew Score is in
the same category as the existing Backpressured and Busy metrics.
Looking forward to your reply.
--
Best！
Xuyang
At 2024-01-16 00:59:
2. Can you tell me that you prefer to unify Data Skew Score and Exception
> tab? In my opinion, Data Skew Score is in
> the same category as the existing Backpressured and Busy metrics.
>
>
> Looking forward to your reply.
>
>
>
> --
>
> Best！
> Xuyang
>
>
&
Hello,
I’m opening this thread to discuss a FLIP[1] to make data skew more visible on
Flink Dashboard.
Data skew is currently not as visible as it should be. Users have to click each
operator and check how much data each sub-task is processing and compare the
sub-tasks against each other. This
Hello,
Is there a reason why a type of data skew score (probably just a percentage) is
not shown on the Flink Dashboard / UI?
Currently users have to click on each operator and check how much data each
subtask is processing to tell if there is skew. This is not efficient,
especially cumbersome
features wrt best practices so that we
could understand how the plugin will help us.
Afaik, FLIP is recommended for improvement ideas that will change public
APIs. I am not sure if a new maven plugin belongs to it.
Best regards,
Jing
On Tue, May 16, 2023 at 11:29 AM Kartoglu, Emre mailto:kar.
Hi David,
This looks awesome. I am no expert on UI/UX, but still have opinions 😊
I normally use the Overview tab for monitoring Flink jobs, and having control
inputs there breaks my assumption that Overview is “read-only” and for
“watching”.
Having said that for “educational purposes” that migh
Hello all,
Myself and 2 colleagues developed a Maven plugin (no support for Gradle or
other build tools yet) that we use internally to detect potential issues in
Flink apps at compilation/packaging stage:
* Known connector version incompatibilities – so far covering Kafka and
Kinesis
*
13 matches
Mail list logo
