> What kind of management does it need to do? i.e. what would very good
> management look like?
I've already given some thoughts on this to Adam in the Gradle issue tracker
issue about 'runaway daemon'.
But this is a hard question. I don't have the answer, just some random thoughts
:-)
What I definitely don't want:
- multiple daemons parking themselves in memory practically forever.
So one thing it needs to do is limit the number of daemons that are allowed to
exist concurrently.
I think the 'kill' logic should be more and more aggressive based on # of
daemons already in use.
And it would be nice if there was some API controlled way to set a hard limit
on the # of daemons allowed
to exist at the same time.
The 'daemon manager' should be able to manage all Gradle daemons, across gradle
versions
(a common thing in my use has been multiple daemons get started because of
different gradle
versions). Sometimes one of those daemons is 'by accident' because gradle
version set wrong...
and only used once. But it sticks around for what seems forever.
Also it has been a problem that daemons would get 'stuck' or something. And
then sit there
'defunct' while another daemon got started in its place. This way, I've seen
10+ concurrent daemons on my machine.
So daemons should have a robust 'self destruct'. If something is wrong, or if
they haven't been used in some time, they should just commit suicide. And this
mechanism should be robust and not itself fail or get stuck somehow.
Another useful thing would be that I could somehow 'own my daemon' and kill it
when I know I am not
going to be using it again (e.g. if the IDE is shutdown, it's 'associated'
daemon probably should also
go away as it probably won't be used anymore).
> > Another thing to consider: as you say, indeed the JVM never gives
> > memory back to the OS. The only way to really give it back is to
> > terminate the JVM (i.e. daemon). So I'd suggest not having the
> > daemon stick around for too long, unless it is really being used
> > quite frequently…
>
> At the moment, the default idle time is 3 hours. This will be user
> configurable though.
I exaggerate, but 3 minutes would be better :-)
I see we are somewhat far apart on the meaning of 'frequently'.
>
> > My compromise would be a combination of two strategies:
> > - be generous with max memory limits (so things work without much
> > manual tweaking)
> > - be conservative and 'kill' unused daemons of fairly quickly (so
> > unused resources are returned to OS)
>
> This sounds right to me, now we just need to agree on what the idle
> timeout should be :)
Tricky :-). My feeling is... something that can be measured in minutes, not
hours :-)
Maybe 10 minutes? Maybe 5 minutes, maybe 15 minutes (that's long!)...
3 hours in my world... is practically equivalent to 'forever'. You might as
well have no timeout.
Just put in to perspective: how much do you gain from not having to pay the
startup
cost once every 3 hours? Is that worth permanently sacrificing +1Gb of memory?
If it is tweakable somehow via the tooling API it is less important what the
exact value would be, because I can tweak it from the IDE if I don't like
its default setting.
>
> If we are too conservative we almost completely negate the user
> experience benefit of using the daemon.
>
The same is true if you keep the daemon there for too long. Because it degrades
overall performance (memory given to the daemon does hurt performance because
it isn't available for other processes and OS level caches).
I'm more worried about overall performance degradation than speeding up the
occasional gradle command.
This is especially the case *if* you reach the 'swap tipping point', where it
makes the difference between a usable or unusable system. Paying the startup
cost for Gradle degrades the experience, but doesn't really make my machine
unusable.
This problem is worse for me, because as a Tooling API user, I do not have the
option of not using the daemon. A commandline user always has this option if
the daemon doesn't work for them.
> > I believe, it just isn't worth the price to keep a daemon parked in
> > precious memory for extended periods of time, if it is not being
> > used. Rather than counting on paging it out, its much better to
> > shut it down. If it is paged out, it won't give you great
> > performance next time you try to use it anyway.
>
> It would be interesting to test if paging in is cheaper than
> bootstrapping Groovy or not. I'm not suggesting we do this.
Testing a bit to find out doesn't hurt :-)
Also keep in mind, I don't think the JVM is ever really 100% 'idle'. So even
when not doing anything, it is probably still touching some of the memory in
its heap every so often. So try to factor in a form of 'constant drag' on the
system just for having a parked daemon.
>
> Shutting it down is costly though in terms of user experience so
> that's not without consequence. This is a fine balancing act.
Keeping it there may be more costly in certain cases (if it causes paging).
Also... shutting it down, in itself doesn't really hurt user experience.
It is the startup that hurts, not the shutdown.
So the shutdown only hurts assuming that I'm actually going to be using
the same daemon again, soon.
> No problem, no offence taken. I just had to clarify what I was
> saying. Your input is very valuable and I think we are increasing
> the understanding here. Although, personally I don't think I have a
> good idea on these defaults yet.
That's good. Glad to hear that. I think it is good to keep an open mind about
this. And
also it should point to the importance of allowing the user to override
the defaults. You do your best to set sensible defaults, but accept that
they may not be right for every use case.
I'm speaking as a Tooling API client here... and for me, there currently
is no way to tweak any of these defaults.
Thanks for the interesting discussion :-)
Kris
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
