[ https://issues.apache.org/jira/browse/PIG-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-1290: -------------------------------- Status: Patch Available (was: Open) Again there seem to be transient unrelated test failures - am resubmitting one more time - will also kick off a unit test run on my machine. > WeightedRangePartitioner should not check if input is empty if quantile file > is empty > ------------------------------------------------------------------------------------- > > Key: PIG-1290 > URL: https://issues.apache.org/jira/browse/PIG-1290 > Project: Pig > Issue Type: Bug > Affects Versions: 0.6.0, 0.7.0 > Reporter: Pradeep Kamath > Assignee: Pradeep Kamath > Fix For: 0.7.0 > > Attachments: PIG-1290.patch > > > Currently WeightedRangePartitioner checks if the input is also empty if the > quantile file is empty. For this it tries to read the input (which under the > covers will result in creating splits for the input etc). If the input is a > directory with many files, this could result in many calls to the namenode > from each task - this can be avoided. > If the input is non empty and quantile file is empty, then we would error out > anyway (this should be confirmed). Also while fixing this jira we should > ensure that pig can still do order by on empty input. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.