On 8/6/14, 4:32 PM, Jianfeng (Jeff) Zhang wrote:
(Closing feedback loop and implementing asks from Prasanth/Bikas)
Replies inline to the questions asked.
1. Does each column represent dag ? My first intuition is vertex. May I
know the progress of each vertex from this UI ?
This is not a live UI, this is a post-hoc analysis tool to find out
scheduling issues and to take apart a "query is slow" bug report.
So this will not update as the query progresses, this only works after a
session ends has completed & the task logs have been aggregated by YARN.
2. Follow question 1, if each column represent dag. Does it necessary to
put all the dags in one session in one unified UI ? IMO, users are more
concerned for each specified dag. All the dags should be independent. We
could represent one page as one dag, and each column as vertex (topological
order ), and inside each vertex, draw the task and task attempt progress.
This was originally built to understand the container re-use behaviour
across DAGs in a single session & to understand the gaps between tasks
of different vertices in the scheduling operation (like reducer slow-start).
That is the purpose of this particular visualization, which is different
from the Ambari/Tez WebUI implementations.
Since this is not a web application, there is no scope for pagination or
interactivity.
3. What does green line in the first column represent ?
The green line is the "CONTAINER_LAUNCHED" event (which represents the
start of YARN container).
The gap between that green line and the actual task is the amount of
time spent in class loading, starting up the Processor, communicating to
the AM back asking for tasks.
Cheers,
Gopal
On Thu, Aug 7, 2014 at 5:06 AM, Prasanth Jayachandran <
[email protected]> wrote:
Hi
The swimlane view looks very cool. I have couple of suggestions
1) It will be good to have headings for left and right most columns.
(container_id and hostname)
2) The dag id has execution time at the end. Will be better to add the
unit for the same (14.2 s).
Thanks
Prasanth Jayachandran
On Aug 6, 2014, at 1:23 PM, Gopal V <[email protected]> wrote:
> Hi,
>
> I just moved tez-swimlanes from my github into master (TEZ-1332).
>
> This is written in python at the moment and works off the tez logs to
make diagrams which look like.
>
> http://people.apache.org/~gopalv/query27.svg
>
> This isn't the final version of the visualization, so I'd like some
feedback on making it useful, like what needs documenting or explaining in
there.
>
> Cheers,
> Gopal
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
