[
https://issues.apache.org/jira/browse/PIG-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789015#action_12789015
]
Thejas M Nair commented on PIG-1143:
------------------------------------
To summarize my above comment, the approach in load-store redesign of not using
the file-size at all is better .
> Poisson Sample Loader should compute the number of samples required only once
> -----------------------------------------------------------------------------
>
> Key: PIG-1143
> URL: https://issues.apache.org/jira/browse/PIG-1143
> Project: Pig
> Issue Type: Bug
> Reporter: Sriranjan Manjunath
> Assignee: Sriranjan Manjunath
>
> The current poisson sampler forces each of the maps to compute the sample
> number. This is redundant and causes issues when a large directory is
> specified in the join. The sampler should be changed to calculate the sample
> count only once and this information should be shared with the remaining
> mappers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
