[
https://issues.apache.org/jira/browse/PIG-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-1219:
----------------------------
Attachment: PIG-1219-3.patch
The test failure is because the way we test it, not the core code. We now
require the quantile file to be created before we run JobControlCompiler. In
our testcase, we invoke the methods of JobControlCompiler directly without
actually running the job, so we do not have quantile file when we get into
JobControlCompiler. Change testcase to force create the quantile file.
> Extra listStatus call to the namenode in WeightedRangePartitioner
> -----------------------------------------------------------------
>
> Key: PIG-1219
> URL: https://issues.apache.org/jira/browse/PIG-1219
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1219-1.patch, PIG-1219-2.patch, PIG-1219-3.patch
>
>
> We call FileLocalizer.openDFSFile in WeightedRangePartitioner to open
> quantile file. openDFSFile internally will check the existence of the
> quantile file, which adds burden to hdfs namenode. We shall remove this extra
> check.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
