I'd like to note here: I've made a lot of experiments and see no problem here: we're asking the system for *physical* memory availability (at least this is what the patch is trying to do), while there are still enough virtual memory left. For example: on WinXP the threshold is exceeded after 10-20 iterations but OOM really happened (if not call GC) only after 41 iterations. That's why 64M + 95% was enough for me to smoothly run the test.
Thanks, Mikhail On 5/17/07, Leo Li <[EMAIL PROTECTED]> wrote:
Another problem is the concern for the threshold to invoke GC. One problem is that it is hardcoded. But of more importance is that the condition is too tough, for example, 512M on win32 platform as Alexey Varlamov has mentioned on the jira comments . I think one reason for that is when the threshold is met and we invoke System.gc(), but the thread is not blocked until gc completes. So user thread is still allocating native memory. Thus the threshold must be set so demanding in order to prevent out-of-memory before gc takes effect. And it also leads to performance problem, for one thing, gc is invoked too early. For another, several request for gc will be invoked during the gap from the first time memory threshold is hit till the gc takes effect at last.(Of course how many gc is actually invoked is decided by vm.) Although it seems impossible to get direct support from java public api, is it possible for our vm to provide a call that will surely invoke a new gc (or do nothing when a gc is now running )and let the caller blocked till current gc completes? If so, I believe the threshold will be much more acceptable and near the free memory demand for gc of VM. And I can try a self-adaptive strategy, like the working set management of os, to decide the threshold dynamically. Good luck! On 5/17/07, Leo Li <[EMAIL PROTECTED]> wrote: > > Excuse me, actually I have just found there is a bug about the patch on > windows platform. That is why the testcase runs slow. I will modify it now. > Sorry for the inconvenience. > > > On 5/17/07, Mikhail Markov <[EMAIL PROTECTED]> wrote: > > > > On 5/16/07, Andrew Zhang <[EMAIL PROTECTED]> wrote: > > > > > > On 5/15/07, Mikhail Markov <[EMAIL PROTECTED]> wrote: > > > > > > > > Hi, all! > > > > > > > > I'd like to raise the problem with freeing native memory which is > > out of > > > > GC > > > > control again :-) (and > > > https://issues.apache.org/jira/browse/HARMONY-3148as > > > > one of it's demonstration). > > > > (See the previous round at > > > > http://thread.gmane.org/gmane.comp.java.harmony.devel/25768). > > > > > > > > Several people have added comments to the JIRA, but we need a > > general > > > > decision on the following question: > > > > > > > > Do we accept the way which was introduced by Leo's patch in H-3148 ( > > i.e. > > > > check if there are enough native memory available before allocating > > new > > > > one, > > > > and call System.gc() (or System.runFinalization()) if necessary)? > > > > > > > > I'm +1 for this method. > > > > > > > > > Hi, > > > > > > I'm not sure whether I understand the problem correctly, but is it > > > possible > > > that vm onlys invokes gc when it fails to allocate, instead of > > checking > > > native memory every time before allocation which would may cause > > > performance > > > downgrade? > > > > > > Well, i've tried both approaches on the test described in H-3148 (100 > > iterations of allocating 20Mb DirectByteBuffer-s) on my laptop with J9: > > 1) Pro-active memory monitoring (I've changed thresholds to 95% & 64M): > > the > > test works ok, I could do normal work in parallel > > 2) Recovering from OOM: the test works terribly slow, I couldn't do > > anything > > else as there was active disk swapping. On 42nd iteration the OOM was > > got, > > System.gc () was invoked and after that the memory allocation was > > retried. It > > was unsuccessfull and OOM was still got (perhaps there was not enough > > memory > > already even to proper run GC). > > > > So, even if second allocation in 2-nd method was successfull, i'd prefer > > the > > first one :-). > > > > Thanks, > > Mikhail > > > > (Mark mentioned that he'd refactored the patch if he had time:-) - i'm > > ready > > > > to do this if he has no time.) > > > > > > > > Thanks, > > > > Mikhail > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Andrew Zhang > > > > > > http://zhanghuangzhu.blogspot.com/ > > > > > > > > > -- > Leo Li > China Software Development Lab, IBM > -- Leo Li China Software Development Lab, IBM
