[
https://issues.apache.org/jira/browse/PIG-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12547514
]
Sam Pullara commented on PIG-40:
--------------------------------
I've tested the attached program in a few VMs/OSs. It appears to do something
more like what you want and it will increase performance as well because memory
will only be checked after automatic garbage collections instead of before
every add to the bag. I haven't yet integrated it in to PIG as I would like to
get any feedback about it.
> Memory management in BigDataBag is probably wrong
> -------------------------------------------------
>
> Key: PIG-40
> URL: https://issues.apache.org/jira/browse/PIG-40
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Sam Pullara
> Attachments: MemoryUsage.java
>
>
> src/org/apache/pig/data/BigDataBag.java
> 1) You should not use finalizers for things other than external resources --
> using them here is very dangerous and could inadvertantly lead to deadlocks
> and object resurrection and just decreases performance without any advantage.
> 2) Using .freeMemory() the way it is used in this class is broken.
> freeMemory() is going to return a mostly random number between 0 and the real
> amount. Adding gc() in here is a terrible performance burden. If you really
> want to do something like this you should using softreferences and
> finalization queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
