For 3. At the moment the log and stdout file serving requires the TaskExecutor to be running. But in some scenarios when having a NFS, it should be enough to know where the file is located. However, this assumption does not hold in the general case.
Cheers, Till On Mon, Sep 30, 2019 at 11:43 AM Yadong Xie <vthink...@gmail.com> wrote: > Hi Xintong Song > > Thanks for your comments! > > 1. I think it is a good idea that to align CPU and memory usage with > FLIP-49 if it will release in version 1.10 > 2. We can update the task executor UI design after FLIP-56 merged into > master. Actually, the image > < > https://cwiki.apache.org/confluence/download/attachments/125309297/BlinkResourceTM.png?version=1&modificationDate=1566223821000&api=v2 > > > in FLIP-56 is a good UI design, we can follow it in the Flink web. > 3. No idea about it, maybe anyone famailar with the runtime part could > answer it? but it would be great to add it to the web UI in my opinion. > 4. I'm not sure will keyword searching across all the pages may cost too > many resources in job manager, but I think it would be very useful if the > REST API could support it. > > Best, > Yadong > > Xintong Song <tonysong...@gmail.com> 于2019年9月29日周日 下午8:11写道： > > > Thanks for drafting the FLIP and starting this discussion, Yadong. > > > > > > I have some comments: > > > > > > - I can see that the proposed memory and cpu usage to be displayed (in > > section 1.1) are aligned with the current ResourceProfile fields. > > However, > > we are working on changing the memory fields in 1.10 with FLIP-49 > [1]. I > > suggest we align the UI design with the new FLIP-49 memory fields. > > - The task executor overview design (in section 1.2) is based on the > > current slot model. The coming FLIP-56 [2] which is also planned for > > 1.10 > > is changing the model so that task executors no longer have fixed > > number of > > slots, but allocated slots (may have different resources) and > available > > resources. > > - I can see that there's discussions in the google doc about using > > different color for available resources. However, the resource > > availability > > for different fields can be different, and may not be simply > > displayed by a > > different color. E.g., a task executor may have two slot, while > slot > > 1 > > takes (20% cpu, 10% heap mem, 50% managed mem, etc.), slot 2 takes > > (10% > > cpu, 35% heap mem, 0% managed mem etc.), and the remaining > > resources in > > the task executor are (70% cpu, 55% heap mem, 50% managed mem, > > etc.). How > > do you plan to display that? > > - I would suggest to have multiple bars for each task executor, > while > > each bar represents one of the resource fields. In addition, we > > may have a > > number (or some other figures) showing how many slots are allocated > > from > > the task executor. > > - Is there any way we provide access to logs of terminated task > > executors? It occurs to us a lot that a job failed due to a task > > executor > > fail/lost. And we have to find the logs of failed task executors by > > manually accessing the file system. I think it would be helpful if we > > can > > find the logs of failed task executors directly in flink webui. > > - Regarding log pagination, is there any way to provide keyword > > searching across all the pages? > > > > > > Thank you~ > > > > Xintong Song > > > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors > > [2] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation > > > > On Fri, Sep 27, 2019 at 3:57 PM Paul Lam <paullin3...@gmail.com> wrote: > > > > > Filed a jira to track this[1]. Thanks a lot. > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-14242 < > > > https://issues.apache.org/jira/browse/FLINK-14242> > > > > > > Best, > > > Paul Lam > > > > > > > 在 2019年9月27日，14:34，Yadong Xie <vthink...@gmail.com> 写道： > > > > > > > > Hi Paul > > > > Thanks for your suggestion. > > > > I think it is easy to implement, could you create a JIRA for me? > > > > > > > > Paul Lam <paullin3...@gmail.com> 于2019年9月27日周五 上午11:11写道： > > > > > > > >> Hi Yadong, > > > >> > > > >> Thanks a lot for summing up the Web UI efforts. > > > >> > > > >> I have a minor suggestion: can we provide a collapse button for the > > task > > > >> names in job graph visualization? For some complex jobs, especially > > SQL > > > >> jobs, the task names are quite long which makes the job graph hard > to > > > read. > > > >> > > > >> Best, > > > >> Paul Lam > > > >> > > > >>> 在 2019年9月27日，10:13，Yadong Xie <vthink...@gmail.com> 写道： > > > >>> > > > >>> Hi all > > > >>> > > > >>> Flink Web UI is the main platform for most users to monitor their > > jobs > > > >> and > > > >>> clusters. We have reconstructed Flink web in 1.9.0 version, but > there > > > are > > > >>> still some shortcomings. > > > >>> > > > >>> This discussion thread aims to provide a better experience for > Flink > > UI > > > >>> users. > > > >>> > > > >>> Here is the design doc I drafted: > > > >>> > > > >>> > > > >> > > > > > > https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit?usp=sharing > > > >>> > > > >>> > > > >>> The FLIP can be found at [2]. > > > >>> > > > >>> Please keep the discussion here, in the mailing list. > > > >>> > > > >>> Looking forward to your opinions, any feedbacks are welcome. > > > >>> > > > >>> [1]: > > > >>> > > > >> > > > > > > https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit?usp=sharing > > > >>> < > > > >> > > > > > > https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit# > > > >>> > > > >>> [2]: > > > >>> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-75%3A+Flink+Web+UI+Improvement+Proposal > > > >> > > > >> > > > > > > > > >
