Re: re-read the config file iff it has changed
Vicki Brown wrote: At 17:40 -0800 03/19/2005, jdow wrote: There is a substantial hit, Vicki, on the order of a factor of two on my machines. We are talking about Only when the Config File has Changed_. OK, so you get a factor of two, what, once a week? Sendmail does this (you run newaliases or maketo trigger it). For clarity's sake, sendmail has real-time access to certain db files (like aliases.db which is generated by 'newaliases'). since sendmail has real-time access to these files, re-creating the .db file from the text version is all that is necessary. However; if you make changes to the sendmail.mc file and the run make to create the sendmail.cf file, you still need to restart sendmail for it to read those changes. SpamAssassin reads in all its config files into memory and has no real-time file access for configuration files. I simply do not believe there can be a substantial hit if spamd re-reads the config file Only When The Config File Has Changed in order to read the config file in only when it has been changed you need to store state information somewhere (in memory or a real-time accessed db file, etc) for each config file. Since SA will read in /path/to/configfiles/*.cf there could be any number of files that state needs to be stored for. Also, to be prudent, state would also need to be stored for /usr/share/spamassassin/*.cf since some people will change those config files even against recommendations. when fine-tuning for performance, even a call to stat() on a file or group of files can introduce performance hits. This is because it effectively still has to open and close the file-handle. Then there's the matter of; in what way, and how often do you poll the .cf files to check for changes? that in itself could add a lot of overhead to the program that is unnecessary. [..snip..] alan
Re: re-read the config file iff it has changed
At 13:55 -0500 03/20/2005, Theo Van Dinter wrote: I simply do not believe there can be a substantial hit if spamd re-reads the config file Besides the fact there are tens of config files that would have to be watched ( It's _already_ watching and __reading__ tens of config files. man spamd: ...spamd will check per-user config files for every message, What's one more on rare occasions, really? I'm sorry. I don't buy the arguments. I will remain unconvinced. -- Vicki Brown ZZZ Journeyman Sourceror: zz |\ _,,,---,,_ Code, Docs, Process, Scripts Philtres zz /,`.-'`'-. ;-;;,_ Perl, WWW, Mac OS X http://cfcl.com/vlb |,4- ) )-,_. ,\ ( `'-' SF Bay Area, CA USA ___ '---''(_/--' `-'\_) ___
Re: re-read the config file iff it has changed
At 13:55 -0500 03/20/2005, Theo Van Dinter wrote: Well, that's not sendmail rereading the config. newaliases generates a new DBM/hash file from a flat text file. Sendmail then realizes the file (that it has open) has changed and reopens the new file for access. The DB is a lookup table, not a config (ala sendmail.cf). Duh. Sendmail then realizes the file (that it has open) has changed and reopens the new file for access. This is what we programmers call an implementation detail. if spamd/spamc already _had_ the code to do what I want I wouldn't be asking for it, now would I? At 11:24 +0900 03/21/2005, alan premselaar wrote: For clarity's sake, sendmail has real-time access to certain db files (like aliases.db which is generated by 'newaliases'). since sendmail has real-time access to these files, re-creating the .db file from the text version is all that is necessary. Uhuh. in order to read the config file in only when it has been changed you need to store state information somewhere uhuh. when fine-tuning for performance, even a call to stat() on a file or group of files can introduce performance hits. This is because it effectively still has to open and close the file-handle. options. Recall that I did say an option to...'. I will accept the hit (which I personally think wouldn't be big enough to notice). Theo Van Dinter This is a very standard method of having a daemon notice a config change. But this is a daemon that notices changes in user prefs files in real time so the performance issue is spurious. It's _already_ taking a performances hit _every single time_ for every single user. -- Vicki Brown ZZZ Journeyman Sourceror: zz |\ _,,,---,,_ Code, Docs, Process, Scripts Philtres zz /,`.-'`'-. ;-;;,_ Perl, WWW, Mac OS X http://cfcl.com/vlb |,4- ) )-,_. ,\ ( `'-' SF Bay Area, CA USA ___ '---''(_/--' `-'\_) ___
Re: re-read the config file iff it has changed
On Sun, Mar 20, 2005 at 07:06:10PM -0800, Vicki Brown wrote: What's one more on rare occasions, really? Exactly, rare occasions. Just send a SIGHUP. I'm sorry. I don't buy the arguments. I will remain unconvinced. Ditto. :) -- Randomly Generated Tagline: It was nice of you to let me reattach your arm. --Zoidber pgpwEOtGfMJ15.pgp Description: PGP signature
Re: re-read the config file iff it has changed
But this is a daemon that notices changes in user prefs files in real time so the performance issue is spurious. It's _already_ taking a performances hit _every single time_ for every single user. No. For several reasons. 1) Usually user rules are disallowed. So all SA has to do is open one file and parse a realtively few lines, which don't include rules. It then overlays scores on the existing pre-parsed rules. 2) Rules live in many files, like 50s or hundreds of them. These files do NOT get reread for every user. These are the files that have to be reread to rebuild the rules after a change. 3) The user rules 'files' in many cases are actually database entries, so there are no files to open in the first place. Going out and checking the timestamps on 100 or 200 rules files for every user would be considerably more overhead than checking the timestamp on one file, and probably more overhead than opening and reading that one small file. Now, checking *occasionally* might not be a bad idea. Where occasionally was maybe once every few minutes or every few hundred mail messages, depending on the traffic level at a site. Event this could of course be the wrong thing to do, so there would probably need to be an option to enable this mode. I'm not at all sure what the appropriate default setting for this option should be. Loren
Re: re-read the config file iff it has changed
On Sun, Mar 20, 2005 at 10:39:26AM -0800, Vicki Brown wrote: Sendmail does this (you run newaliases or maketo trigger it). Well, that's not sendmail rereading the config. newaliases generates a new DBM/hash file from a flat text file. Sendmail then realizes the file (that it has open) has changed and reopens the new file for access. The DB is a lookup table, not a config (ala sendmail.cf). I simply do not believe there can be a substantial hit if spamd re-reads the config file Besides the fact there are tens of config files that would have to be watched (versus the handful that most daemons have), there's a ton of processing involved with loading in the config. There's no way to remove a previously loaded config, so spamd would have to reread all of the config files anytime there's a change to any of the them. At that point, it's more efficient to simply send a SIGHUP to spamd when you've made a change. It will then reload the config and you'll be off and running. This is a very standard method of having a daemon notice a config change. -- Randomly Generated Tagline: A duel of wits? To the DEATH? pgpjGs49fl8Qq.pgp Description: PGP signature
