[
https://issues.apache.org/jira/browse/TEZ-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bikas Saha updated TEZ-714:
---------------------------
Description:
Follow up jira from TEZ-41.
1) If there's multiple OutputCommitters on a Vertex, they can be run in
parallel.
2) Running an OutputCommitter in the main thread blocks all other event
handling, w.r.t the DAG, and causes the event queue to back up.
3) This should also cover shared commits that happen in the DAG.
was:
Follow up jira from TEZ-41.
1) If there's multiple OutputCommitters on a Vertex, they can be run in
parallel.
2) Running an OutputCommitter in the main thread blocks all other event
handling, w.r.t the DAG, and causes the event queue to back up.
> OutputCommitters should not run in the main AM dispatcher thread
> ----------------------------------------------------------------
>
> Key: TEZ-714
> URL: https://issues.apache.org/jira/browse/TEZ-714
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Priority: Critical
>
> Follow up jira from TEZ-41.
> 1) If there's multiple OutputCommitters on a Vertex, they can be run in
> parallel.
> 2) Running an OutputCommitter in the main thread blocks all other event
> handling, w.r.t the DAG, and causes the event queue to back up.
> 3) This should also cover shared commits that happen in the DAG.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
