[
https://issues.apache.org/jira/browse/TEZ-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061264#comment-14061264
]
Siddharth Seth commented on TEZ-1272:
-------------------------------------
bq. The new method is not calling the following and this actually does
something regardless of predecessorConf. So the new code may be missing that. I
am guessing you are doing that deliberately. Mind describing the reasoning? Is
it because that was only handling the input/output duplication which is not
needed now?
The old method used to set the Intermediate-Input / Intermediate-Output keys as
required based on the predecessor. This was primarily because this config may
be shared between the Input/Output on the same task. That's no longer the
expectation - and instead, edges need to specify a configuration with the
correct key set.
bq. How did this ever work?
I'm guessing it did not.
bq. This one and others like it, there is probably no need to INTERMEDIATE in
the name, right?
Will remove them. Had INTERMEDIATE there because of prior confusion with
map-output.compress vs output.compress in the MR world. To be clear, these
properties are not used by MROutput.
bq. Orthogonally, does OrderedPartitionedKVEdgeConfigurator sound better than
OrderedPartitionedKVEdgeConfiguration.
The EdgeConfiguration paris could be renamed. Something like
OrderedPartitionedKVEdgeConfigurer. Separate jira though.
bq. Does this unblock TEZ-696?
Yes it should.
Thanks for the review. Will upload an updated patch and commit.
> Change YARNRunner to make use of EdgeConfigurations
> ---------------------------------------------------
>
> Key: TEZ-1272
> URL: https://issues.apache.org/jira/browse/TEZ-1272
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-1272.1.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
