Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-16 Thread Rui Fan
Thanks Yun for the update, LGTM. Best, Rui On Mon, Oct 16, 2023 at 1:34 PM Yu Chen wrote: > Hi David. > > Thanks for your detailed comments. > The async-profiler has a lot of features, but there are some requirements > to use it (some cases encountered in production are listed below): > 1.

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-15 Thread Yu Chen
Hi David. Thanks for your detailed comments. The async-profiler has a lot of features, but there are some requirements to use it (some cases encountered in production are listed below): 1. Prior to JDK 11, alloc mode requires HotSpot debug symbols, and for OpenJDK, you need to install them

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-15 Thread Yun Tang
Christle Sent: Saturday, October 14, 2023 4:11 To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers In the Wiki, this FLIP is motivated by: - That the current flamegraph functionality can only see operator-level stack traces, while

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread David Christle
In the Wiki, this FLIP is motivated by: - That the current flamegraph functionality can only see operator-level stack traces, while async-profiler provides CPU/allocation/locks information, along with deeper Java & system call stack information. - Low configurability (e.g. cannot set the sampling

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread Rui Fan
One minor comment: In general, the generic java profiler includes memory analysis, cpu, thread, deadlock, etc. The FLIP title is java profiler, but the FLIP just supports flamegraph at process level. So the `powerful java profiler` title may not be suitable. Would you mind updating the FLIP

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread Yu Chen
Hi all. If there are no further questions, we will start a vote on FLIP-375 next week. Best regards, Yu Chen Yu Chen 于2023年10月9日周一 17:24写道: > Hi all, > > Yun Tang and I are opening this thread to discuss our proposal to > integrate async-profiler's capabilities for profiling taskmananger

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-10 Thread Jing Ge
> > [1] https://github.com/async-profiler/async-profiler/issues/14 > > Best > Yun Tang > > > From: Jing Ge > Sent: Tuesday, October 10, 2023 12:05 > To: dev@flink.apache.org > Subject: Re: [DISCUSS] FLIP-375: Built-in cross-plat

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-10 Thread Yun Tang
er side. They flamegraph-files will be cleanup > automatically once reached to rest.profiling.history-size. > > Best > Yun Tang > > > > ____ > From: Rui Fan <1996fan...@gmail.com> > Sent: Tuesday, October 10, 2023 10:10 > T

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Jing Ge
aph-files will be cleanup > automatically once reached to rest.profiling.history-size. > > Best > Yun Tang > > > > ____ > From: Rui Fan <1996fan...@gmail.com> > Sent: Tuesday, October 10, 2023 10:10 > To: dev@flink.apache.org > Subject: Re: [DI

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Yun Tang
as > > > well. In the OLAP scenario and sometimes in the streaming scenario > (when > > > there're some heavy operations during execution plan generation or in > > > operator coordinators), the JM can have bottleneck as well. > > > > > > Best, > >

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
o > (when > > > there're some heavy operations during execution plan generation or in > > > operator coordinators), the JM can have bottleneck as well. > > > > > > Best, > > > Zhanghao Chen > > > > > > From: Yu

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Jing Ge
> Best, > > Zhanghao Chen > > ________ > > From: Yu Chen > > Sent: Monday, October 9, 2023 17:24 > > To: dev@flink.apache.org > > Subject: [DISCUSS] FLIP-375: Built-in cross-platform powerful java > > profiler on task

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Yu Chen
heavy operations during execution plan generation or in > operator coordinators), the JM can have bottleneck as well. > > Best, > Zhanghao Chen > > From: Yu Chen > Sent: Monday, October 9, 2023 17:24 > To: dev@flink.apache.org > Subject: [DISCU

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
> there're some heavy operations during execution plan generation or in > operator coordinators), the JM can have bottleneck as well. > > Best, > Zhanghao Chen > > From: Yu Chen > Sent: Monday, October 9, 2023 17:24 > To: dev@flink.apache.or

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Zhanghao Chen
, October 9, 2023 17:24 To: dev@flink.apache.org Subject: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers Hi all, Yun Tang and I are opening this thread to discuss our proposal to integrate async-profiler's capabilities for profiling taskmananger (e.g

[DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Yu Chen
Hi all, Yun Tang and I are opening this thread to discuss our proposal to integrate async-profiler's capabilities for profiling taskmananger (e.g., generating flame graphs) in the Flink Web [1]. Currently, Flink provides ThreadDump and Operator-Level Flame Graphs by sampling task threads. The