[: Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The "-mx" and "-ss" switches.)]
- Forwarded message from -
To: Dalibor Topic <[EMAIL PROTECTED]>
Subject: Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The "-mx"
and "-ss" switches.)
On Fri, Mar 14, 2003 at 05:55:21AM -0800, Dalibor Topic wrote:
> hi Mark,
>
> --- Mark J Roberts <[EMAIL PROTECTED]> wrote:
> > I am sick and tired of manually overriding the heap
> > size limit in
> > order to run Freenet without hitting the arbitrary
> > default 64MB
> > limit.
>
> I think that's due to a bug in kaffe ('s class
> library) as JDK seems to be able to run Freenet in
> 30M, or so, according to what Greg posted here. I'm
> trying to track that one down with Greg, but it would
> be helpful if you joined in, as a Freenet developer.
>
> I've looked at the gcstats outputs provided by Greg,
> and most of the Objects lying around are either
> java.lang.Objects or java.util.HashMap$Entries, with a
> bunch of freenet.fs.dir.* Objects following. Grepping
> throuh the source indicates that java.lang.Objects are
> used by freenet for synchronization, right?
>
> My prime suspect at the moment are the
> HashMap.Entries, so I posted a patch to track down
> who's creating all these HashMap.Entries, hoping that
> would provide some clue where they are used, and why
> they don't go away ;)
>
> The idea is to create a stack trace for every
> constructed instance of HashMap.Entry and map the
> trace to the number of instances created with the same
> stack trace. So the HashMap.Entry constructor
> increases the counter for its stack trace, or inserts
> it into the map.
Well, my immediate suspicion would be that: every once in a while,
freenet writes out the datastore index. In order to minimize the time
the datastore is locked for, we clone the index in a lock, and then
write it out from the clone. HashMap.clone() clones the individual items
as well as the table, so we end up with a whole lot of rather short
lived objects. If some way could be found to make the clone() lazily
copy (refcounts?), this would greatly reduce the GC/allocation stress.
Freenet on Kaffe still suffers from mysterious unexplained pauses in
execution that cause much higher routingTime's (a very bad thing, it
causes the node to assume it is overloaded and reject requests) than
Sun; maybe this is caused by a combination of this and the fact that
Kaffe uses a monolithic GC that locks the whole VM.
>
> The patch has (at least) one problem, it relies on
> Runtime.exit() to print the map of stack traces.
> Apparently it seems to be hard to get freenet to
> exit(). So I'm not sure how to proceed from here: if
> it is possible to tell a freenet node to shut down
> through exit(), that would be helpful, otherwise I
> could add a thread that prints out the map in regular
> intervals.
>
> The other course of action would be to replace kaffe's
> HashMap (if that's the culprit) with another
> implementation, let's say Classpath's, and see if that
> yields any benefits.
>
> > I just don't understand why these options are even
> > there, or why
> > they are not unlimited by default. It is _breaking_
> > applications.
>
> In theory, they are not necessary. In practice, it's
> rather unconvenient to have a maximum memory setting
> higher that the amount of available RAM. In that case,
> the gc might spend a lot of time asking the virtual
> memory manager to shuffle pages around, and that
> degrades performance severely. The gc kicks in when a
> certain percentage of memory is used.
>
> The 64Mb limit is the same that Sun's using on the JDK
> 1.4.1 for example. See
> http://java.sun.com/j2se/1.4/docs/tooldocs/linux/java.html
> under -Xmx . So if you're getting problems because
> your application is running out of memory on kaffe,
> but works fine on Sun's JDK, then that's a bug in
> kaffe, in my opinion. Setting -mx to unlimited by
> default would just mask it ;)
>
> Additionally, here's a blurb on usefullness of the
> java heap setting from the Java Server documentation (
> http://salamander.merit.edu/system/doc/administration/performance.html
> ):
>
> "The heap limit helps determine when a garbage
> collection (GC) will take place. You can set the limit
> high to avoid a GC, but then you could end up paging.
> Generally, we have found that it is better to take the
> hit from a GC fairly often and to avoid paging. The
> other thing to keep in mind about garbage collection
> is that the entire server stops while it is going. So
> if you have an occasional, very long GC, the clients
> will hang for that time. That leads to more
> variability i
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The "-mx" and "-ss" switches.)
On Fri, Mar 14, 2003 at 10:30:14PM -0600, Mark J Roberts wrote: > Mark J Roberts: > > I'll hack something together and reply with that stack trace map as > > soon as I have the time. > > I called System.exit() when Freenet was using over 150MB of memory. > The output is attached. It looks like you have some sort of infinite looping problem. You should check and not create a Throwable if the object being tracked is a Throwable :) -- Matthew Toseland [EMAIL PROTECTED]/[EMAIL PROTECTED] Full time freenet hacker. http://freenetproject.org/ Freenet Distribution Node (temporary) at http://80-192-4-36.cable.ubr09.na.blueyonder.co.uk:8889/0bIANZmR~K8/ ICTHUS. pgp0.pgp Description: PGP signature
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The "-mx" and "-ss" switches.)
Mark J Roberts: > I'll hack something together and reply with that stack trace map as > soon as I have the time. I called System.exit() when Freenet was using over 150MB of memory. The output is attached. log.bz2 Description: Binary data
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The "-mx" and "-ss" switches.)
Dalibor Topic:
> The patch has (at least) one problem, it relies on Runtime.exit()
> to print the map of stack traces. Apparently it seems to be hard
> to get freenet to exit(). So I'm not sure how to proceed from
> here: if it is possible to tell a freenet node to shut down
> through exit(), that would be helpful, otherwise I could add a
> thread that prints out the map in regular intervals.
I'll hack something together and reply with that stack trace map as
soon as I have the time.
I suspected that Sun's implementation also had the memory limit. I
haven't used a proprietary JVM in ages, so I can't say whether Kaffe
is much worse. Freenet is a really bloated program no matter how you
run it, in any case.
It's entirely reasonable that a user would _want_ his OS to page out
data. The working set nearly always constitutes only a small share
of a program's allocated memory.
I don't see why the frequency of garbage collection relates to
this--that blurb explicitly suggests that reducing its frequency
through allowing more "slack" is a false optimization. What's more,
if any JVM were crazy enough to wait until the heap limit is hit
before invoking the GC ("helps determine when a garbage collection
(GC) will take place"), even lightweight programs will consume an
average of half the heap limit in memory.
So I'm not at all persuaded that Sun is doing anything remotely sane
with that limit, or that Kaffe needs to fall into the same trap.
___
kaffe mailing list
[EMAIL PROTECTED]
http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The "-mx" and "-ss" switches.)
hi Mark,
--- Mark J Roberts <[EMAIL PROTECTED]> wrote:
> I am sick and tired of manually overriding the heap
> size limit in
> order to run Freenet without hitting the arbitrary
> default 64MB
> limit.
I think that's due to a bug in kaffe ('s class
library) as JDK seems to be able to run Freenet in
30M, or so, according to what Greg posted here. I'm
trying to track that one down with Greg, but it would
be helpful if you joined in, as a Freenet developer.
I've looked at the gcstats outputs provided by Greg,
and most of the Objects lying around are either
java.lang.Objects or java.util.HashMap$Entries, with a
bunch of freenet.fs.dir.* Objects following. Grepping
throuh the source indicates that java.lang.Objects are
used by freenet for synchronization, right?
My prime suspect at the moment are the
HashMap.Entries, so I posted a patch to track down
who's creating all these HashMap.Entries, hoping that
would provide some clue where they are used, and why
they don't go away ;)
The idea is to create a stack trace for every
constructed instance of HashMap.Entry and map the
trace to the number of instances created with the same
stack trace. So the HashMap.Entry constructor
increases the counter for its stack trace, or inserts
it into the map.
The patch has (at least) one problem, it relies on
Runtime.exit() to print the map of stack traces.
Apparently it seems to be hard to get freenet to
exit(). So I'm not sure how to proceed from here: if
it is possible to tell a freenet node to shut down
through exit(), that would be helpful, otherwise I
could add a thread that prints out the map in regular
intervals.
The other course of action would be to replace kaffe's
HashMap (if that's the culprit) with another
implementation, let's say Classpath's, and see if that
yields any benefits.
> I just don't understand why these options are even
> there, or why
> they are not unlimited by default. It is _breaking_
> applications.
In theory, they are not necessary. In practice, it's
rather unconvenient to have a maximum memory setting
higher that the amount of available RAM. In that case,
the gc might spend a lot of time asking the virtual
memory manager to shuffle pages around, and that
degrades performance severely. The gc kicks in when a
certain percentage of memory is used.
The 64Mb limit is the same that Sun's using on the JDK
1.4.1 for example. See
http://java.sun.com/j2se/1.4/docs/tooldocs/linux/java.html
under -Xmx . So if you're getting problems because
your application is running out of memory on kaffe,
but works fine on Sun's JDK, then that's a bug in
kaffe, in my opinion. Setting -mx to unlimited by
default would just mask it ;)
Additionally, here's a blurb on usefullness of the
java heap setting from the Java Server documentation (
http://salamander.merit.edu/system/doc/administration/performance.html
):
"The heap limit helps determine when a garbage
collection (GC) will take place. You can set the limit
high to avoid a GC, but then you could end up paging.
Generally, we have found that it is better to take the
hit from a GC fairly often and to avoid paging. The
other thing to keep in mind about garbage collection
is that the entire server stops while it is going. So
if you have an occasional, very long GC, the clients
will hang for that time. That leads to more
variability in service quality than more frequent but
smaller GCs."
In short: let's fix the bug ;)
cheers,
dalibor topic
__
Do you Yahoo!?
Yahoo! Web Hosting - establish your business online
http://webhosting.yahoo.com
___
kaffe mailing list
[EMAIL PROTECTED]
http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
