[
https://issues.apache.org/jira/browse/TEZ-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4143:
------------------------------
Attachment: (was: TEZ-4145.03.patch)
> Provide an option to disable DAG graph (.dot) generation for latency
> sensitive jobs
> -----------------------------------------------------------------------------------
>
> Key: TEZ-4143
> URL: https://issues.apache.org/jira/browse/TEZ-4143
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: László Bodor
> Priority: Minor
> Attachments: TEZ-4143.01.patch
>
>
> In latency sensitive jobs, sometimes it takes time to submit the DAG due to
> ".dot" file creation in busy cluster. It would be good to have
> {{tez.generate.debug.artifacts}} cover for
> {{DAGAppMaster::generateDAGVizFile}}
> Pasting some log snippets for later ref:
> {noformat}
> syslog_dag_1581571406290_3831_10007_post:2020-04-11 06:36:41,911 [INFO] [IPC
> Server handler 0 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file,
> dagId=dag_1581571406290_3831_10008,
> filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10008.dot
> syslog_dag_1581571406290_3831_10007_post:2020-04-11 06:36:42,744 [INFO] [IPC
> Server handler 0 on 38609] |common.TezUtilsInternal|: Redirecting log file
> based on addend: dag_1581571406290_3831_10008
> syslog_dag_1581571406290_3831_10008_post:2020-04-11 06:36:45,377 [INFO] [IPC
> Server handler 1 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file,
> dagId=dag_1581571406290_3831_10009,
> filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10009.dot
> syslog_dag_1581571406290_3831_10008_post:2020-04-11 06:36:45,690 [INFO] [IPC
> Server handler 1 on 38609] |common.TezUtilsInternal|: Redirecting log file
> based on addend: dag_1581571406290_3831_10009
> syslog_dag_1581571406290_3831_10012_post:2020-04-11 06:36:50,451 [INFO] [IPC
> Server handler 0 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file,
> dagId=dag_1581571406290_3831_10013,
> filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10013.dot
> syslog_dag_1581571406290_3831_10012_post:2020-04-11 06:36:51,574 [INFO] [IPC
> Server handler 0 on 38609] |common.TezUtilsInternal|: Redirecting log file
> based on addend: dag_1581571406290_3831_10013
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
