[
https://issues.apache.org/jira/browse/TEZ-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bikas Saha updated TEZ-1612:
----------------------------
Attachment: TEZ-1612.1.patch
Changed to get source vertex num tasks from the context instead of hard coding
it in the config.
Theoretically, the parallelism determination can happen in any order because
its mostly data size dependent and prediction of output size. So the
parallelism of the source should not be hard code during determining one's own
parallelism.
However, scheduling of downstream tasks should happen only after the upstream
tasks have been scheduled.
> Pig on tez unit test intermittent hang
> --------------------------------------
>
> Key: TEZ-1612
> URL: https://issues.apache.org/jira/browse/TEZ-1612
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Daniel Dai
> Assignee: Bikas Saha
> Attachments: DAG1.png, TEZ-1612.1.patch, runwithmaster.tar.gz,
> syslog_dag_1411413615885_0001_1, testfail1.log.tar.gz
>
>
> Several Pig unit tests hang intermittently. For example,
> TestNewPlanImplicitSplit.testImplicitSplitInCoGroup, which is a DAG of 4
> nodes:
> !DAG1.png!
> It uses auto-parallelism, vertex 106 change parallelism from 2->1, and vertex
> 107 from 21->1.
> Log attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
