The spillable objects are data bags right? We don't want bags
themselves to be garbage collected. Its the tuples inside them that
are the problem.
I was thinking more along the lines of
class MemoryManager {
List<Spillable> spillables;
public static MemoryManager getMemoryManager();
public void register(Spillable spillable){
spillables.add(spillable);
}
public boolean isMemoryLow(){
//Sam's stuff
}
private reclaimMemory(){
//traverse the list and somehow ask the biggest one to spill
spillable.spill();
//garbage collect?
}
}
On Dec 12, 2007, at 3:31 PM, Benjamin Reed (JIRA) wrote:
[ https://issues.apache.org/jira/browse/PIG-40?
page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
tabpanel#action_12551159 ]
Benjamin Reed commented on PIG-40:
----------------------------------
How about
class MemoryManager {
static MemoryManager getMemoryManager();
void add(Spillable spillable);
}
add() will put Spillable on a WeakList so that the Spillable object
will get GCed when strong references go away.
Memory management in BigDataBag is probably wrong
-------------------------------------------------
Key: PIG-40
URL: https://issues.apache.org/jira/browse/PIG-40
Project: Pig
Issue Type: Bug
Components: impl
Reporter: Sam Pullara
Assignee: Benjamin Reed
Attachments: BigDataBag.java, MemoryUsage.java
src/org/apache/pig/data/BigDataBag.java
1) You should not use finalizers for things other than external
resources -- using them here is very dangerous and could
inadvertantly lead to deadlocks and object resurrection and just
decreases performance without any advantage.
2) Using .freeMemory() the way it is used in this class is
broken. freeMemory() is going to return a mostly random number
between 0 and the real amount. Adding gc() in here is a terrible
performance burden. If you really want to do something like this
you should using softreferences and finalization queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
