[jira] Commented: (PIG-1307) when we spill the DefaultDataBag we are not setting the sized changed flag to be true.
[ https://issues.apache.org/jira/browse/PIG-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847636#action_12847636 ] Daniel Dai commented on PIG-1307: - +1. Will commit it shortly. > when we spill the DefaultDataBag we are not setting the sized changed flag to > be true. > -- > > Key: PIG-1307 > URL: https://issues.apache.org/jira/browse/PIG-1307 > Project: Pig > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed > Fix For: 0.7.0 > > Attachments: PIG-1307.patch > > > pig uses a size changed flag to indicate when we should recalculate the > memory footprint of the bag. the setting of this flag is sprinkled throughout > the code. unfortunately, it is missing in DefaultDataBag.spill(). there may > be other cases as well. the problem with this case is that when the low > memory threshold kicks in, bags are spilled until the desired amount of > memory is "freed". since the flag is not being reset subsequent calls to the > threshold events will retrigger the spill() and think more memory was freed > even though nothing was actually spilled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1307) when we spill the DefaultDataBag we are not setting the sized changed flag to be true.
[ https://issues.apache.org/jira/browse/PIG-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847631#action_12847631 ] Benjamin Reed commented on PIG-1307: i don't have test cases since the test cases would be orders of magnitude higher more difficult to write than the patch and may not reproduce the problem across different machine configurations. > when we spill the DefaultDataBag we are not setting the sized changed flag to > be true. > -- > > Key: PIG-1307 > URL: https://issues.apache.org/jira/browse/PIG-1307 > Project: Pig > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed > Fix For: 0.7.0 > > Attachments: PIG-1307.patch > > > pig uses a size changed flag to indicate when we should recalculate the > memory footprint of the bag. the setting of this flag is sprinkled throughout > the code. unfortunately, it is missing in DefaultDataBag.spill(). there may > be other cases as well. the problem with this case is that when the low > memory threshold kicks in, bags are spilled until the desired amount of > memory is "freed". since the flag is not being reset subsequent calls to the > threshold events will retrigger the spill() and think more memory was freed > even though nothing was actually spilled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1307) when we spill the DefaultDataBag we are not setting the sized changed flag to be true.
[ https://issues.apache.org/jira/browse/PIG-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847587#action_12847587 ] Daniel Dai commented on PIG-1307: - Hi, Ben, Is the patch ready? Do you need help to add some test cases? > when we spill the DefaultDataBag we are not setting the sized changed flag to > be true. > -- > > Key: PIG-1307 > URL: https://issues.apache.org/jira/browse/PIG-1307 > Project: Pig > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed > Fix For: 0.7.0 > > Attachments: PIG-1307.patch > > > pig uses a size changed flag to indicate when we should recalculate the > memory footprint of the bag. the setting of this flag is sprinkled throughout > the code. unfortunately, it is missing in DefaultDataBag.spill(). there may > be other cases as well. the problem with this case is that when the low > memory threshold kicks in, bags are spilled until the desired amount of > memory is "freed". since the flag is not being reset subsequent calls to the > threshold events will retrigger the spill() and think more memory was freed > even though nothing was actually spilled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1307) when we spill the DefaultDataBag we are not setting the sized changed flag to be true.
[ https://issues.apache.org/jira/browse/PIG-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847208#action_12847208 ] Hadoop QA commented on PIG-1307: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12439206/PIG-1307.patch against trunk revision 924558. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/243/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/243/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/243/console This message is automatically generated. > when we spill the DefaultDataBag we are not setting the sized changed flag to > be true. > -- > > Key: PIG-1307 > URL: https://issues.apache.org/jira/browse/PIG-1307 > Project: Pig > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed > Fix For: 0.7.0 > > Attachments: PIG-1307.patch > > > pig uses a size changed flag to indicate when we should recalculate the > memory footprint of the bag. the setting of this flag is sprinkled throughout > the code. unfortunately, it is missing in DefaultDataBag.spill(). there may > be other cases as well. the problem with this case is that when the low > memory threshold kicks in, bags are spilled until the desired amount of > memory is "freed". since the flag is not being reset subsequent calls to the > threshold events will retrigger the spill() and think more memory was freed > even though nothing was actually spilled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
