Hi,
At 20:11 06.09.04 +0530, you wrote: >Dear Erwin, > >Sorry for question not really related to Vpopmail. > >It seems that I am hit by "Silly Qmail (Queue) Syndrome". > >I am using the Spamcontrol Patch v2.2.12 along with vpopmail-5.4.6, but >have not used the experimental "bigtodo". > >Wished to apply the bigtodo. I would like to get clarified that whether you >bigtodo is based on ext_todo patch or big-todo patch or both. I had not >initially compiled the bigtodo thinking that it is experimental. > >What do you suggest.
Well. At first you have to tell why you think you are hit by the "Silly Qmail Syndrom". Any hints ?
Second. Apart from the big-todo enhencement, my implementation of Andre Oppermann's performance enhancements dont work well. After investigation a look of time and testing I didn't find any significant performance improvement. Note: The code in SPAMCONTROL is not the ext-big-todo; however it is based of Andre's first suggestion to influence qmail's scheduler for mail processing; which was buggy by itself.
Third. The best thing is to avoid bounces to non-existing accounts. Use my RECIPIENTS extension as part of Qmail or perhaps the "real-rcptto" patch.
The forthcoming SPAMCONTROL version will include verion 0.42 of the RECIPIENTS extension; check my Qmail page (http://www.fehcom.de/qmail.html).
regards. --eh.
Hi Erwin,
Thanks for nice reply.
I am attaching "Queue Size" graph (5 Minute Average) updated Tuesday, 7 September 2004 at 0:50 (EDT).
You can notice between 0400 - 1000 hrs (EDT) a quite high Mail Queue. During that time period the smtpd is running to the tune of 100/100. But the send is running to the tune of local 3/15 remote 5/40. The "messages in queue but not yet preprocessed" goes on increasing in wild. When the smtpd runs to the tune of 85/100 its all okay. This has started happening on almost every start of the week, when huge volume of genuine + virus infected customers mails start pouring in.
I am not using "RECIPIENTS extension", but using "badrcptto" for whitelisting mechanism, which works very well (might be a bit slow due to the reason that lookup is being done into txt database).
I am also using http://linux.voyager.hr/ucspi-tcp/tcpserver-limits-2004-07-25.diff patch to limit concurrent connection from single IP. This helps identifying Virus trodden computers and denying them connection (it's a boon).
I also have Caching-DNS on this Server (djbdns).
About the todo patches the comments of Dave Sill (of Qmail Handbook fame) are interesting to note in the thread:
"Outbound email rate slows when inbound rate is high" http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&c2coff=1&threadm=e6c47de7.0310091325.147cade4%40posting.google.com&rnum=2&prev=/groups%3Fq%3Dext-todo%26hl%3Den%26lr%3D%26ie%3DUTF-8%26c2coff%3D1%26selm%3De6c47de7.0310091325.147cade4%2540posting.google.com%26rnum%3D2
Also one can have a look at the thread "ext-todo and big-todo patches" http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&c2coff=1&threadm=wx0lm56pfo0.fsf%40sws5.ctd.ornl.gov&rnum=1&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DUTF-8%26c2coff%3D1%26q%3Ddave%2Bsill%2Bext-todo%2Band%2Bbig-todo
I tried to apply the patch http://www.nrg4u.com/qmail/ext_todo-20030105.patch over and above spamcontrol but it failed at:
patch p1 < ext_todo-20030105.patch patching file p1 patching file EXTTODO-INFO patching file FILES patching file Makefile Hunk #2 succeeded at 713 (offset 8 lines). Hunk #4 succeeded at 818 (offset 8 lines). Hunk #5 succeeded at 1598 (offset 87 lines). Hunk #6 succeeded at 1585 (offset 9 lines). Hunk #7 succeeded at 1694 (offset 87 lines). patching file TARGETS Hunk #1 succeeded at 405 (offset 20 lines). patching file hier.c Hunk #1 succeeded at 115 with fuzz 1 (offset 7 lines). patching file install-big.c patching file qmail-send.c Hunk #1 FAILED at 1215. Hunk #2 succeeded at 1527 (offset 88 lines). Hunk #3 succeeded at 1662 (offset 20 lines). Hunk #4 succeeded at 1797 (offset 88 lines). 1 out of 4 hunks FAILED -- saving rejects to file qmail-send.c.rej patching file qmail-start.c patching file qmail-todo.c
It also might be possible that my disk speed be contributing a bit to the bottleneck. Current iostat figures are (not taken at the silly qmail syndrome time, I would take that figure to when I am hit again).
iostat -d -x /dev/hda2 2 Linux 2.4.18-18.7.x (slsp-da4p21) 09/07/2004
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
/dev/hda2 0.03 2.27 1.85 0.37 14.98 21.73 7.49 10.86 16.57 0.13 208.66 171.08 3.79
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
/dev/hda2 0.00 0.00 2.00 0.00 16.00 0.00 8.00 0.00 8.00 0.15 75.00 50.00 1.00
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
/dev/hda2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
/dev/hda2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
I am getting more and more convinced that ext-todo might be a possible solution in this situation.
What do you feel !
Devendra Singh
<<attachment: queue-size-day.png>>
