[
https://issues.apache.org/jira/browse/TEZ-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189534#comment-14189534
]
Jeff Zhang commented on TEZ-1711:
---------------------------------
[~bikassaha] Attach a new patch,
bq. Given this change, should we remove inputSpecList and outputSpecList as
member vars of VertexImpl?
Still use them as member of VertexImpl, but new them in constructor and clear
them in getInputSpecList and getOutputSpectList. This could avoid creating new
List especially for large job. Does it make sense to you ?
bq. Why is this change making the test DAG fail?
The affected test case is for TEZ-1689 ( Exception handling for
EdgeManagerPlugin ). Without this patch, only the first task attempt is failed
in AM side, the following task attempts wouldn't not been affected in AM side (
because we cache the outputSpecList ), but will throw exception in TezChild
since we don't get the correct outputSpecList, (but that can not been simulated
in unit test case , the unit test case can only simulate behavior in AM side).
So without this patch, AM would think the dag is still running. While with this
patch, all the task attempts would fail in AM side, and finally cause the DAG
fail.
> Don't cache outputSpecList in VertexImpl.getOutputSpecList(taskIndex)
> ---------------------------------------------------------------------
>
> Key: TEZ-1711
> URL: https://issues.apache.org/jira/browse/TEZ-1711
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.5.1
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Attachments: TEZ-1711-2.patch, TEZ-1711.patch
>
>
> It would cache the outputSpecList in its
> VertexImpl.getOutputSepcList(taskIndex), but I don't think we should cache it
> as it depends on the taskIndex, although in all the EdgeManagerPlugin
> Implementations, the value is the same no matter what the taskIndex is. But
> it has risk that if we have a new EdgeManagerPlugin that has different
> behavior. Or if this case would never happens, then just remove the taskIndex
> from the method parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
