[ https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13685881#comment-13685881 ]
Mark Wagner commented on PIG-3325: ---------------------------------- Thanks for taking a look, Dmitriy. I agree that doing work in add() is the wrong way to go. I don't think there's a way to get the time back down to 400 ns while still having lazy registration, but that may be okay if it prevents bad behavior elsewhere. I'll try out caching the memory sizes during sorting and see how things improve. That should improve performance no matter how 'spillables' gets populated. > Adding a tuple to a bag is slow > ------------------------------- > > Key: PIG-3325 > URL: https://issues.apache.org/jira/browse/PIG-3325 > Project: Pig > Issue Type: Bug > Affects Versions: 0.11, 0.11.1, 0.11.2 > Reporter: Mark Wagner > Assignee: Mark Wagner > Priority: Critical > Attachments: PIG-3325.demo.patch, PIG-3325.optimize.1.patch > > > The time it takes to add a tuple to a bag has increased significantly, > causing some jobs to take about 50x longer compared to 0.10.1. I've tracked > this down to PIG-2923, which has made adding a tuple heavier weight (it now > includes some memory estimation). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira