Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi all Thanks to everyone involved in the discussion I have started the voting thread for all the sub FLIPs (including FLIP-98, FLIP-99, FLIP-100, FLIP-101, FLIP-102, FLIP-103, and FLIP-104) Please take some time to vote on the FLIP you are interested in, thanks lining jing 于2020年2月19日周三 下午3:

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

I think we can create the PR for the GC status later if we could find an easy way to obtain it, before that the users could get GC logs from the FLIP-103. By the way, there is a similar topic 'FlameGraph In Job Vertex' in FLIP-75 in the early discussion stage https://docs.google.com/document/d/1tI

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi, A quick question/comment about FLIP-102. Have you thought about adding GC stats? I’m not sure what’s easily do-able, but something that would allow user to see GC issues (long/frequent pauses, lots of CPU time spent in the GC) would be quite useful for analysing performance/stability issues

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi all I have drafted the docs of top-level FLIPs for the individual changes proposed in FLIP-75. will update it to the cwiki page and start the voting stage soon if there is no objection. - FLIP-98: Better Back Pressure Detection

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Till I got your point, will create sub FLIPs and votings according to the FLIP-75 and previous discussion soon. Till Rohrmann 于2020年2月9日周日 下午5:27写道： > Hi Yadong, > > I think it would be fine to simply link to this discussion thread to keep > the discussion history. Maybe an easier way would b

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Yadong, I think it would be fine to simply link to this discussion thread to keep the discussion history. Maybe an easier way would be to create top-level FLIPs for the individual changes proposed in FLIP-75. The reason I'm proposing this is that it would be easier to vote on it and to implemen

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Till FLIP-75 has been open since September, and the design doc has been iterated over 3 versions and more than 20 patches. I had a try, but it is hard to split the design docs into sub FLIP and keep all the discussion history at the same time. Maybe it is better to start another discussion to

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

I think there is no such description because we never did it before. I just figured that FLIP-75 could actually be a good candidate to start this practice. We would need a community discussion first, though. Cheers, Till On Mon, Feb 3, 2020 at 10:28 AM Yadong Xie wrote: > Hi Till > I didn’t fin

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Till I didn’t find how to create of sub flip at cwiki.apache.org do you mean to create 9 more FLIPS instead of FLIP-75? Till Rohrmann 于2020年1月30日周四 下午11:12写道： > Would it be easier if FLIP-75 would be the umbrella FLIP and we would vote > on the individual improvements as sub FLIPs? Decreasing

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Would it be easier if FLIP-75 would be the umbrella FLIP and we would vote on the individual improvements as sub FLIPs? Decreasing the scope should make things easier. Cheers, Till On Thu, Jan 30, 2020 at 2:35 PM Robert Metzger wrote: > Thanks a lot for this work! I believe the web UI is very i

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Thanks a lot for this work! I believe the web UI is very important, in particular to new users. I'm very happy to see that you are putting effort into improving the visibility into Flink through the proposed changes. I can not judge if all the changes make total sense, but the discussion has been

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Thanks for the update Yadong. Big +1 for the proposed improvements for Flink's web UI. I think they will be super helpful for our users. Cheers, Till On Tue, Jan 7, 2020 at 10:00 AM Yadong Xie wrote: > Hi everyone > > We have spent some time updating the documentation since the last > discussio

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi everyone We have spent some time updating the documentation since the last discussion. In short, the latest FLIP-75 contains the following proposal(including both frontend and RestAPI) 1. Job Level - better job backpressure detection - load more feature in job exception -

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi all, I have updated the backend design in FLIP-75 . Here are some brief introductions: - Add metric for manage memory FLINK-14406 . - E

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

2. Sounds good to me. 4. If that is the case, I would suggest to make a large default page size, so incase of huge log data we have less large pages rather than lots of small pages. Thank you~ Xintong Song On Tue, Oct 8, 2019 at 11:03 AM Yadong Xie wrote: > Hi Xintong Song > > 2. We could s

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Xintong Song 2. We could switch between the detailed mode(including cpu, task heap, task off-heap, shuffle, on-heap managed, off-heap managed) and the summary mode(only including cpu and mem), which is very easy to do in UI design. 4. I think the key point is not pagination in Web UI but the R

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

@Yadong 2. I agree that we can update the task executor ui after flip-56 is done. But I would suggest keep it on discussion to come up with a proper ui design for task executor resources. I don't think the mentioned image from flip-56 is a good choice. That image is a simplified figure with cpu an

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

For 3. At the moment the log and stdout file serving requires the TaskExecutor to be running. But in some scenarios when having a NFS, it should be enough to know where the file is located. However, this assumption does not hold in the general case. Cheers, Till On Mon, Sep 30, 2019 at 11:43 AM Y

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Xintong Song Thanks for your comments! 1. I think it is a good idea that to align CPU and memory usage with FLIP-49 if it will release in version 1.10 2. We can update the task executor UI design after FLIP-56 merged into master. Actually, the image

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Thanks for drafting the FLIP and starting this discussion, Yadong. I have some comments: - I can see that the proposed memory and cpu usage to be displayed (in section 1.1) are aligned with the current ResourceProfile fields. However, we are working on changing the memory fields in 1.1

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Filed a jira to track this[1]. Thanks a lot. [1] https://issues.apache.org/jira/browse/FLINK-14242 Best, Paul Lam > 在 2019年9月27日，14:34，Yadong Xie 写道： > > Hi Paul > Thanks for your suggestion. > I think it is easy to implement, could you cre

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Paul Thanks for your suggestion. I think it is easy to implement, could you create a JIRA for me? Paul Lam 于2019年9月27日周五 上午11:11写道： > Hi Yadong, > > Thanks a lot for summing up the Web UI efforts. > > I have a minor suggestion: can we provide a collapse button for the task > names in job grap

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi Yadong, Thanks a lot for summing up the Web UI efforts. I have a minor suggestion: can we provide a collapse button for the task names in job graph visualization? For some complex jobs, especially SQL jobs, the task names are quite long which makes the job graph hard to read. Best, Paul L

[DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi all Flink Web UI is the main platform for most users to monitor their jobs and clusters. We have reconstructed Flink web in 1.9.0 version, but there are still some shortcomings. This discussion thread aims to provide a better experience for Flink UI users. Here is the design doc I drafted: h