Maybe a legend somewhere near the top would help eliminate a bunch of "what is this questions"
-----Original Message----- From: Gopal V [mailto:[email protected]] Sent: Tuesday, August 12, 2014 3:10 PM To: Jianfeng (Jeff) Zhang; [email protected] Subject: Re: Feedback for tez-swimlanes On 8/6/14, 4:32 PM, Jianfeng (Jeff) Zhang wrote: (Closing feedback loop and implementing asks from Prasanth/Bikas) Replies inline to the questions asked. > 1. Does each column represent dag ? My first intuition is vertex. May > I know the progress of each vertex from this UI ? This is not a live UI, this is a post-hoc analysis tool to find out scheduling issues and to take apart a "query is slow" bug report. So this will not update as the query progresses, this only works after a session ends has completed & the task logs have been aggregated by YARN. > 2. Follow question 1, if each column represent dag. Does it > necessary to put all the dags in one session in one unified UI ? IMO, > users are more concerned for each specified dag. All the dags should > be independent. We could represent one page as one dag, and each > column as vertex (topological order ), and inside each vertex, draw the > task and task attempt progress. This was originally built to understand the container re-use behaviour across DAGs in a single session & to understand the gaps between tasks of different vertices in the scheduling operation (like reducer slow-start). That is the purpose of this particular visualization, which is different from the Ambari/Tez WebUI implementations. Since this is not a web application, there is no scope for pagination or interactivity. > 3. What does green line in the first column represent ? The green line is the "CONTAINER_LAUNCHED" event (which represents the start of YARN container). The gap between that green line and the actual task is the amount of time spent in class loading, starting up the Processor, communicating to the AM back asking for tasks. Cheers, Gopal > On Thu, Aug 7, 2014 at 5:06 AM, Prasanth Jayachandran < > [email protected]> wrote: > >> Hi >> >> The swimlane view looks very cool. I have couple of suggestions >> 1) It will be good to have headings for left and right most columns. >> (container_id and hostname) >> 2) The dag id has execution time at the end. Will be better to add >> the unit for the same (14.2 s). >> >> Thanks >> Prasanth Jayachandran >> >> On Aug 6, 2014, at 1:23 PM, Gopal V <[email protected]> wrote: >> >> > Hi, >> > >> > I just moved tez-swimlanes from my github into master (TEZ-1332). >> > >> > This is written in python at the moment and works off the tez logs >> > to >> make diagrams which look like. >> > >> > http://people.apache.org/~gopalv/query27.svg >> > >> > This isn't the final version of the visualization, so I'd like some >> feedback on making it useful, like what needs documenting or >> explaining in there. >> > >> > Cheers, >> > Gopal >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or >> entity to which it is addressed and may contain information that is >> confidential, privileged and exempt from disclosure under applicable >> law. If the reader of this message is not the intended recipient, you >> are hereby notified that any printing, copying, dissemination, >> distribution, disclosure or forwarding of this communication is >> strictly prohibited. If you have received this communication in >> error, please contact the sender immediately and delete it from your >> system. Thank You. >> > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or > entity to which it is addressed and may contain information that is > confidential, privileged and exempt from disclosure under applicable > law. If the reader of this message is not the intended recipient, you > are hereby notified that any printing, copying, dissemination, > distribution, disclosure or forwarding of this communication is > strictly prohibited. If you have received this communication in error, > please contact the sender immediately and delete it from your system. > Thank You. > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
