On Sunday 13 April 2008 05:14, Dossy Shiobara wrote: > On 2008.04.13, Don Baccus <[EMAIL PROTECTED]> wrote: > > Or just put back the config parameters as they were, they worked fine > > for everyone for a decade. I don't remember the masses shouting that > > the config approach wasn't sufficient. It suddenly appeared without > > discussion nor documentation. > > Uh, either your memory is poor or you need to review the mailing list > archives again. > > The number of times people have asked "can I change [a particular > setting] without requiring a nsd restart" and have been given the answer > "no, config changes require a server restart" has indirectly suggested > that if we can move something out of the config and into the runtime > environment, it would be preferred as people ask for it repeatedly over > time. >
I remember it well. And most of these 'people' don't read the source code, so they don't understand that the config file settings are read only. Read only is a performance improving strategy. If you can change config settings, you need to go through a specialized API, get a mutex, etc. FOR EVERY SINGLE LOOKUP! What is confusing to me is that you (Dossy) know all this, or at least you should know it. So, before pushing to bring runtime configuration/dynamic tuning to AOLserver, please present details about where you are going to store this information, access it, and somehow avoid the source code complications and performance hit that this would cause. There are so many 'wrongs' in what you are proposing it is hard to know where to begin. Since you reference the thread pools, where this issue last came up, I'll use it as an example. I instrumented the threadpools code so that I could see what was actually happening, per request. Any dynamic tuning would require some kind of monitoring, right? Well, what I found out is pretty surprising. The threadpool code is, first of all, blindingly fast. Each queue is protected by mutex/condition code, but there are some mistakes where a condition signal is used instead of a condition broadcast. What happens in this case is that a single thread hogs the queue. With a low conns-per-thread setting, a thread can serve all of its conns and exit before any other thread gets to run. I have modified the code to test this and with small changes, you can get more sharing. (One way to see the current behavior is to hit AOLserver using ab with a concurrency above the number of threads. The distribution of time to service a request starts to get large, and a few requests get suspended until the traffic dies down.) So, the point of this information is that changing the parameters dynamically in this situation would be hugely intrusive. By the time you made your changes, the situation could be completely different. And, more than anything, you don't know what the situation is going to be in the next second. Burst/lull behavior is unpredictable. Another thing I noticed is that performance is very little affected by changing the parameters. Like I said, nobody even notices that in the current code, one thread can hog all the queued requests (for fast requests). So what is wrong with dynamic config: Engineering: 1. Lack of theory/data on correct config setting for current state 2. Need to massively reorganize all code that uses config 3. Need to provide query for current state/config settings 4. Need to record/save dynamic settings in case of a restart 5. Complications in code to support overriding global settings. Administration/Control: 1. Move from centralized control to distributed control 2. Move from sys-admin control to user control 3. Move from written down settings to unknown settings 3. More difficult to figure out what config params were used when tracking down bugs/performance issues. I'm starting to get the feeling that we are once again in the six-month itch to find a direction. And each time the same seemingly interesting ideas are presented, which seem so reasonable. They seem reasonable until you ask make the same suggestion to some other community: new language (tell that to the RoR community, or Tomcat community) need for speed (what?) dynamic configuation (when we don't even have docs on static config) add feature xyz to the core (because it is harder to do it correctly) just do it (because figuring out what and why takes too much time) The one, very good and serious suggestion is DOCUMENTATION. And documentation is the first step in doing any of the above. You can't impove something unless you know what that something can do. But for documentation we don't need some bs documentation manager, just files and CVS. Start with the API and config settings. I have a tool/module which can print all the existing config settings (actually used by existing code). I've posted a list. Did anyone care? Who knows. Until we get serious about this boring documentation task, moving forward with new projects is pointless. All decisions are based upon the 'sounds good' argument, which only requires someone to say, wait for it: sounds good. You know we could make a rule: want to commit something (non-bugfix) to CVS? Put in x number of hours working on documentation. Want to propose some new feature, same deal: finish documentation on the current situation. And above all, if you want to make changes to the core code, get support from the community first, support, not disinterest. After a while even I get tired of saying no. Seriously, tom jackson > Being able to tweak and tune thread pools at runtime has come up many, > many times. Each time, the answer is: change the config setting and > restart, which usually wasn't desirable because restarts during the day > on a production system weren't something folks could do ad-hoc, as they > were running a single server, not load-balanced ... > > Seriously, this has been a long-standing ask, to make more config > settings modifiable at runtime. Perhaps the way compression has been > implemented wasn't perfect, but it was a start, and I think Jeff's > suggestion of shipping a filter proc that enables compression > server-wide by default is the right answer. Making the setting purely > config-driven is just perpetuating the "sorry, config settings need > restarts" problem. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
