[
https://issues.apache.org/jira/browse/TEZ-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059562#comment-14059562
]
Bikas Saha edited comment on TEZ-1272 at 7/12/14 1:06 AM:
----------------------------------------------------------
For the following replacement.
{code}-
MultiStageMRConfToTezTranslator.translateVertexConfToTez(finalReduceConf,
- iReduceStageConf);
+ MRHelpers.translateVertexConfToTez(finalReduceConf);{code}
The new method is not calling the following and this actually does something
regardless of predecessorConf. So the new code may be missing that. I am
guessing you are doing that deliberately. Mind describing the reasoning? Is it
because that was only handling the input/output duplication which is not needed
now?
{code} translateMultiStageWithPredecessor(vertexConf, predecessorConf) {code}
How did this ever work?
{code}- registerMRToRuntimeKeyTranslation(TezJobConfig.TEZ_CREDENTIALS_PATH,
MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY);
+
registerMRToRuntimeKeyTranslation(MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY,
TezJobConfig.TEZ_CREDENTIALS_PATH);
{code}
This one and others like it, there is probably no need to INTERMEDIATE in the
name, right? e.g. OnFileUnorderedOuput could write final output if so desired?
{code}TEZ_RUNTIME_INTERMEDIATE_INPUT_COMPRESS_CODEC{code}
Orthogonally, does OrderedPartitionedKVEdgeConfigurator sound better than
OrderedPartitionedKVEdgeConfiguration. Reasons, its the entity that configures
instead of being the configuration itself. It actually does more like create
edge properties. And finally, it disconnects the name from Configuration.
Does this unblock TEZ-696?
Looks good.
was (Author: bikassaha):
For the following replacement.
{code}-
MultiStageMRConfToTezTranslator.translateVertexConfToTez(finalReduceConf,
- iReduceStageConf);
+ MRHelpers.translateVertexConfToTez(finalReduceConf);{code}
The new method is not calling the following and this actually does something
regardless of predecessorConf. So the new code may be missing that. I am
guessing you are doing that deliberately. Mind describing the reasoning?
{code} translateMultiStageWithPredecessor(vertexConf, predecessorConf) {code}
How did this ever work?
{code}- registerMRToRuntimeKeyTranslation(TezJobConfig.TEZ_CREDENTIALS_PATH,
MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY);
+
registerMRToRuntimeKeyTranslation(MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY,
TezJobConfig.TEZ_CREDENTIALS_PATH);
{code}
This one and others like it, there is probably no need to INTERMEDIATE in the
name, right? e.g. OnFileUnorderedOuput could write final output if so desired?
{code}TEZ_RUNTIME_INTERMEDIATE_INPUT_COMPRESS_CODEC{code}
Orthogonally, does OrderedPartitionedKVEdgeConfigurator sound better than
OrderedPartitionedKVEdgeConfiguration. Reasons, its the entity that configures
instead of being the configuration itself. It actually does more like create
edge properties. And finally, it disconnects the name from Configuration.
Looks good.
> Change YARNRunner to make use of EdgeConfigurations
> ---------------------------------------------------
>
> Key: TEZ-1272
> URL: https://issues.apache.org/jira/browse/TEZ-1272
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-1272.1.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
