[ https://issues.apache.org/jira/browse/PIG-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565705#comment-13565705 ]
Koji Noguchi commented on PIG-2923: ----------------------------------- Hi Dmitriy, I'm seeing a weird error when pig 0.11 tries to spill. Can this change be related? Opened PIG-3147. > Lazily register bags with SpillableMemoryManager > ------------------------------------------------ > > Key: PIG-2923 > URL: https://issues.apache.org/jira/browse/PIG-2923 > Project: Pig > Issue Type: Improvement > Reporter: Dmitriy V. Ryaboy > Assignee: Dmitriy V. Ryaboy > Fix For: 0.11 > > Attachments: bagspill_delayed_register.patch, bagspill_delay.patch > > > Currently, all Spillable DataBags get registered by the BagFactory at the > moment of creation. In practice, a lot of these bags will not get large > enough to be worth spilling; we can avoid a lot of memory overhead and > cheapen the process of finding a bag to spill when we do need it, by allowing > Bags themselves to register when they grow to some respectable threshold. > Related JIRAs: PIG-2917, PIG-2918 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira