HI Mark, Thanks for your very insightful reply
Spot on - CPU has been relatively high when the delay happens. As per your guess, the swap is hardly used based on the following vmstat output: [EMAIL PROTECTED] /etc/postfix # vmstat 5 procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 2 1 10880 78580 289760 630456 0 0 13 2 12 14 23 8 51 18 0 1 10880 76012 289776 630576 0 0 22 958 392 583 48 11 17 25 1 0 10880 77816 289784 630772 0 0 19 1003 398 586 52 11 8 28 3 1 10880 76888 289788 630700 0 0 10 687 336 453 28 6 39 27 4 0 10880 77152 289804 630752 0 0 10 858 382 613 60 14 4 22 We are still using amavisd-new 2.3.3-3 (from Ubuntu 6.0.6 LTS package), I noticed TimeElapsedDecoding below doesn't exist in our amavisd-agent. I have attached an output from amavisd-agent, could you please help with parsing the report? I tried to look for some manual about the parameters but wasn't able to find one on the net. Our bayes DB is a berkeley db on a filesystem: [EMAIL PROTECTED] ~amavis/.spamassassin # ls -la bayes_* -rw------- 1 amavis amavis 8376 2008-06-24 14:05 bayes_journal -rw------- 1 amavis amavis 670367744 2008-06-24 14:05 bayes_seen -rw------- 1 amavis amavis 20455424 2008-06-24 14:05 bayes_toks is our bayes_seen too large? we haven't looked at the option of using the bayes into database (i.e. mysql) in the past as this would add a complexity to the mail system; we prefer to keep it simple as much as possible but if this is a time for us to change, I would be able to propose a project to investigate the use of database for the bayes db. I would appreciate any feedback you have Many thanks CP Subject: Re: [AMaViS-user] Amavis bottleneck? Date: Mon, Jun 23, 2008 at 08:37:00PM +0200 Quoting Mark Martinec ([EMAIL PROTECTED]): : Bernd Probst wrote: : : > Increasing the number of $max_servers should increase throughput as long as : > enough memory is. : : Tha usual bottleneck when SpamAssassin is in play is the available CPU. : Increasing $max_servers only increases througput up to the point : where CPU is saturated. : : > Since you already set $max_servers = 25 I think you're : > system is permanently in a state of swapping. : : I very much doubt so. The 1.5GB of memory should suffice : for 25 server processes and possibly slightly more : (assuming certain outdated huge SARE rules are not in use). : : : Christian, : : > I can confirm the swap space wasn't continuously used during this busy : > period. I was monitoring all server aspect at the time and swap was : > relatively unused during this period. : : It is the vmstat 'pages paged in' count that would show a memory : shortage (unlikely here). Used swap space is not bad in itself, : it may just show that some unused sections of code or data : are not wasting physical memory. : : > how can I build a amavis processing stat? i.e. the time it takes to scan : > a message? the log file isn't very friendly. : : amavisd-agent gives a quick-glance overview of main timing : sections averages, e.g.: : : TimeElapsedDecoding 15290 s 0.063 s/msg (InMsgs) : TimeElapsedPenPals 190 s 0.001 s/msg (InMsgs) : TimeElapsedReceiving 26297 s 0.108 s/msg (InMsgs) : TimeElapsedSending 7842 s 0.032 s/msg (InMsgs) : TimeElapsedSpamCheck 495648 s 2.040 s/msg (InMsgs) : TimeElapsedTotal 582771 s 2.398 s/msg (InMsgs) : TimeElapsedVirusCheck 5807 s 0.024 s/msg (InMsgs) : : It is most likely that the majority of processing time is spent : in SpamAssassin. To get a SpamAssassin timing breakdown you need : SpamAssassin 3.3 (from CVS) and amavisd-new-2.6.0 or later: : : amavis[78061]: (78061-16) TIMING-SA total 1857 ms - parse: 2 (0.1%), : extract_message_metadata: 53 (2.9%), get_uri_detail_list: 3 (0.2%), : tests_pri_-1000: 10 (0.5%), tests_pri_-950: 0.63 (0.0%), : tests_pri_-900: 0.83 (0.0%), tests_pri_-400: 26 (1.4%), : check_bayes: 24 (1.3%), tests_pri_0: 1627 (87.6%), : check_spf: 749 (40.3%), poll_dns_idle: 725 (39.0%), : check_dkim_signature: 1.72 (0.1%), check_razor2: 559 (30.1%), : check_dcc: 167 (9.0%), check_pyzor: 0.02 (0.0%), : tests_pri_500: 7 (0.4%), tests_pri_1000: 13 (0.7%), : total_awl: 11 (0.6%), check_awl: 4 (0.2%), : update_awl: 1.96 (0.1%), learn: 94 (5.1%), get_report: 2 (0.1%) : : I believe the most recent amavis-logwatch is able to parse and : report a summary on the TIMING-SA sections too (see recent topic: : "get-file-type2 - about 1/3 of total processing time"). : : Is your bayes db on MySQL (good) or somewhere else (not so good) ? : : As a last resort, enabling graylisting would help to shed : some load off the system. : : Mark ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ AMaViS-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
