RE: [EXTERNAL] Re: FW: [Gluten] Feedback collection for Gluten Native Diagnose Experience Enhancement
Hey Hongze, Sorry for reply late, currently not, will share once we have a draft design for that, but basically, we want to have a whole GLUTEN SQL Data frame Tab as main entry page to do same things as how Photon does with additional functionality support as async flame graphs. Thanks, Yangyang Gao -Original Message- From: Zhang Hongze Sent: Tuesday, April 2, 2024 5:05 PM To: dev@gluten.apache.org Subject: [EXTERNAL] Re: FW: [Gluten] Feedback collection for Gluten Native Diagnose Experience Enhancement [You don't often get email from hon...@apache.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] +1 Can't wait to see this. So far profiling Gluten's code is non-trivial and the features in proposal will ease that a lot. About showing flame graphs on UI (which seems to be super helpful, indeed): Do you have some initial designs for that? Like it's turned on/off by configuration or an UI button or something? Thanks, Hongze - To unsubscribe, e-mail: dev-unsubscr...@gluten.apache.org For additional commands, e-mail: dev-h...@gluten.apache.org
Re: FW: [Gluten] Feedback collection for Gluten Native Diagnose Experience Enhancement
+1 Can't wait to see this. So far profiling Gluten's code is non-trivial and the features in proposal will ease that a lot. About showing flame graphs on UI (which seems to be super helpful, indeed): Do you have some initial designs for that? Like it's turned on/off by configuration or an UI button or something? Thanks, Hongze
Re: FW: [Gluten] Feedback collection for Gluten Native Diagnose Experience Enhancement
hey , I just put the content and image into the google doc. This is the link https://docs.google.com/document/d/1sUfJfqG2iW7ocfptzKB1PHduD4weDofUvwXV4q979Js/edit XiDuo You 于2024年4月1日周一 11:12写道: > Hi yangyang, > > Thank you for the proposal. +1 for the idea. > > BTW, I can not see the images. Can we move the details to a google docs ? > You can reference a url instead. > > > Yangyang Gao 于2024年3月30日周六 14:11写道: > > > > > > > Hey there 😊, > > > > > > > > It’s Yangyang from Microsoft working on Gluten Project. The purpose of > > this email is to collect ideas on improving the gluten native diagnose > > experience. > > > > > > > > Gluten is indeed impressive, particularly in its significant contribution > > to boosting the performance of Spark 👍🎉 . but unfortunately, the > > current online diagnose experience doesn't quite match up. > > > > Right now, we're teaming up with the Intel Gluten team to plan some > > enhancements for the gluten native diagnose experience, the main goal is > to > > make it easier for users to quickly locate issues when errors or any > > performance regression occur online. > > > > > > > > Our primary idea now is to utilize the gluten UI page as the main entry > > point and incorporate the necessary enhanced features directly onto this > > page, referred to as *gluten UI enhancement*, so that much enhancement > > work can directly go into OSS. > > > > > > > > Some ideas currently under consideration include: > > > > 1) Onboarding colorized DGA graph in Gluten UI like how Photon does to > > help users quickly identify which nodes are fallback or not and the > context > > detail. > > > > 2) Onboarding execution time summary table in Gluten UI like How Photon > > does to help users quickly identify which stage/op is time-consuming. > > > > [image: A white paper with black text Description automatically > generated] > > > > 3) Onboarding an async profile Flame graph in Gluten UI including > > CPU_FLAME_GRAPHS, CPU_STACK_TRACES, and WALL_CLOCK_STACK_TRACE to assist > > users pinpointing which native function calls are time-consuming. > > > > > > > > > > > > Our main goal with this email is to gather feedback and requirements from > > our community members. We value your input! Among the options mentioned, > > which one do you think is most essential right now? > > > > > > > > Additionally, *we're open to hearing your fresh ideas and any new > > requirements you might have for improving the gluten native diagnose > > experience. Your contributions to this discussion are highly valued and > > appreciated!* > > > > > > > > We look forward to your feedback. > > > > > > > > Thank you. > > > > gayan...@microsoft.com > > >
Re: FW: [Gluten] Feedback collection for Gluten Native Diagnose Experience Enhancement
Hi yangyang, Thank you for the proposal. +1 for the idea. BTW, I can not see the images. Can we move the details to a google docs ? You can reference a url instead. Yangyang Gao 于2024年3月30日周六 14:11写道: > > > Hey there 😊, > > > > It’s Yangyang from Microsoft working on Gluten Project. The purpose of > this email is to collect ideas on improving the gluten native diagnose > experience. > > > > Gluten is indeed impressive, particularly in its significant contribution > to boosting the performance of Spark 👍🎉 . but unfortunately, the > current online diagnose experience doesn't quite match up. > > Right now, we're teaming up with the Intel Gluten team to plan some > enhancements for the gluten native diagnose experience, the main goal is to > make it easier for users to quickly locate issues when errors or any > performance regression occur online. > > > > Our primary idea now is to utilize the gluten UI page as the main entry > point and incorporate the necessary enhanced features directly onto this > page, referred to as *gluten UI enhancement*, so that much enhancement > work can directly go into OSS. > > > > Some ideas currently under consideration include: > > 1) Onboarding colorized DGA graph in Gluten UI like how Photon does to > help users quickly identify which nodes are fallback or not and the context > detail. > > 2) Onboarding execution time summary table in Gluten UI like How Photon > does to help users quickly identify which stage/op is time-consuming. > > [image: A white paper with black text Description automatically generated] > > 3) Onboarding an async profile Flame graph in Gluten UI including > CPU_FLAME_GRAPHS, CPU_STACK_TRACES, and WALL_CLOCK_STACK_TRACE to assist > users pinpointing which native function calls are time-consuming. > > > > > > Our main goal with this email is to gather feedback and requirements from > our community members. We value your input! Among the options mentioned, > which one do you think is most essential right now? > > > > Additionally, *we're open to hearing your fresh ideas and any new > requirements you might have for improving the gluten native diagnose > experience. Your contributions to this discussion are highly valued and > appreciated!* > > > > We look forward to your feedback. > > > > Thank you. > > gayan...@microsoft.com >