Giuseppe,
> Hi Mark, while I was trying to figure out and measure the performance
> of a amavisd+postfix+spamassassin system, to see whether the
> TimeElapsedSpamCheck was reliable, I was coming out with this patch
> (see attach) to know which plugin is effectively loaded by amavisd. In
> fact I haven't found such information in logs, and for testing purpose
> one have to put at least in the same conditions as of the amavisd
> user, and need to know all the plugins effectively loaded
> (often the ones from the amavis user are not the same as the
> system-wide SA). Can you include (or maybe a better one) it in the
> next release?
Ok, I'll add the:
do_log(2, "SpamAssassin loaded plugins: %s", join(', ', sort
map {my($n)=ref $_; $n=~s/^Mail::SpamAssassin::Plugin:://; $n} @plugins));
> Apart this I found that on a typical Dual Quad Xeon 5420 2.5Ghz
> (indeed the extra 7 cores aren't used very much unless you increase
> the concurrency level), a typical run with:
>
> "spamassassin -t -D < mailspam.msg"
>
> takes around 10s, so the 7-8s shown for TimeElapsedSpamCheck by
> amavisd-agent should be reliable, considering also that it should run
> in daemon mode so the loading time shouldn't be counted.
Right.
> Of these 10s, 7-8 seconds are usually taken by DNSBL SA checking;
That's about reasonable, perhaps a bit on the high side.
I use the settings:
rbl_timeout 5.5 2.2
rbl_timeout 7.7 3.3 open-whois.org
Note that DNS-based checks only add to latency and have
little effect on throughput. Extra latency can easily be
compensated for by running more concurrent processes
(at the expense of more memory consumed).
> disabling all the network checking (i.e. using spamassassin -t -D -L
> <mailspam.msg), and redoing the tests, this 10s usually drops around
> 3.9s, while further using the compile (Rule2XS) feature, these 3.9s
> drops to 2.6s (40% better).
Btw, using SpamAssassin 3.3 with amavisd adds a SA timing report
to the amavisd log (at level 2 or above), e.g.:
(80089-08) TIMING-SA total 1348 ms - parse: 2 (0.2%),
extract_message_metadata: 8 (0.6%), get_uri_detail_list: 0.62 (0.0%),
tests_pri_-1000: 10 (0.7%), tests_pri_-950: 0.90 (0.1%),
tests_pri_-900: 1.08 (0.1%), tests_pri_-400: 20 (1.5%),
check_bayes: 18 (1.3%), tests_pri_0: 1097 (81.3%),
check_dkim_adsp: 62 (4.6%), check_spf: 217 (16.1%),
poll_dns_idle: 188 (14.0%), check_razor2: 541 (40.1%),
check_dcc: 166 (12.3%), check_pyzor: 0.03 (0.0%),
tests_pri_500: 10 (0.7%), tests_pri_899: 98 (7.3%),
check_crm114: 97 (7.2%), tests_pri_1000: 34 (2.5%),
total_awl: 33 (2.4%), check_awl: 17 (1.3%), update_awl: 10 (0.7%),
learn: 30 (2.2%), get_report: 2 (0.2%)
> Now I'd like to do the same evaluation for
> amavisd, i.e. with all the network testing disabled. Is there a way to
> disable for instance the DNSBL and all the remote testing in one shot
> from within amavisd.conf (more or less like the -L option does for the
> spamassassin standalone command)?
$sa_local_tests_only = 1;
> I tried to add sort of
> "skip_rbl_checks 1" to user_prefs but isn't effective.
It'd better be in local.cf, but it should be effective.
> In this way
> should be possible to evaluate the theoretical peak limit (in
> msgs/day) that a server could sustain per CPU/core and tune up the
> performance accordingly under the best conditions. BTW, what other
> values people reach under latest spamassassin (3.2.5) and amavisd-new
> (2.6.1) in these subject?
Your figures look about right. It depends much on what additional
rules you have, and what additional plugins are in use.
> At this point I would like to ask you whether the following feature
> could be added and if it's worthwhile: it seems that most of the spam
> is sent during night and the weekend.
I'd rather say that an absolute rate of spam doesn't change much,
but as there is less regular mail traffic at night, the ratio
of ham/spam does change.
> Would be possible to add
> amavisd-new a feature to have it's behaviour selective according to
> the time: e.g. since 75% of SA time is spent over DNS|RBL checks,
DNS/RBL checks are insignificant as far as aggregate mail throughput
is concerned.
> and often a server need to be much more responsive in sending and
> receiving mails during daylight, would be fine to have a sort of two
> profiles in amavisd-new: one for day (with maybe all network tests
> disabled) and one for night and weekends under which the network
> testing are disabled. In this way, when during daylight much server
> power and responsiveness is needed for delivering mails it won't spent
> much time in DNS|RBLs (ok, it will pass more spam), while during night
> it could use all the tests available.
It would be possible, but I don't think it is worth it.
If a server can't cope with mail rate (with some reserve),
one needs a bigger machine, and not sacrifice quality of
filtering.
> Such approach theoretically could be made with some sed|perl "search &
> replace" scripts for user_prefs, commenting or uncommenting on the fly
> some test, and then restarts the amavisd process, but this sound a
> sort of ugly and awkward, and would be much more elegant inside amavisd.
Changing the $sa_local_tests_only is a bit tricky, as it is given
to SpamAssassin during its initialization. Some other settings
may be easier to customize through policy banks, loading a policy
bank could be done by a custom huuk based on hour-of-the-day
if needed.
Mark
------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/
