[
https://issues.apache.org/jira/browse/PIG-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893428#action_12893428
]
Thejas M Nair commented on PIG-1524:
------------------------------------
It will be useful to have a measure of number of records being spilled as well
as the number of bags that spilled.
ie have two counters - 'proactive spill count' and 'proactively spilled record
count' .
> 'Proactive spill count' is misleading
> -------------------------------------
>
> Key: PIG-1524
> URL: https://issues.apache.org/jira/browse/PIG-1524
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
>
> InternalCacheBag, InternalSortedBag, InternalDistinctBag increment this
> counter for every record that it writes to disk, once it exceeds the memory
> limit. This number is misleading.
> Instead, this counter should be increment it by 1 for each instance of these
> bags that has spilled to disk.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
