Re: [Mailman-Users] Mailman / Sendmail horriblyslowperformance..suggestions please?!?
I have not modified the default values in the SMTPDirect module or Defaults.py. If I could get qrunner to tell me anything about what message(s) it is working I could tell you how many recipients were on that list. Each list in the current system should be no more than 2-5 people. We moved over a small subset of mailing lists from the older majordomo server. However, there are some lists with several hundres recipients on it that will be moving in the future when these issues get worked out. Orphaned locks? (ocean) root $ ls -lt (~mailman/locks) total 16 -rw-rw-r--2 mailman mailman55 Aug 25 2003 qrunner.lock -rw-rw-r--2 mailman mailman55 Aug 25 2003 qrunner.lock.ocean.otr.usm.edu.16302 -rw-rw-r--2 mailman mailman62 Aug 25 2003 gifted.studies.lock -rw-rw-r--2 mailman mailman62 Aug 25 2003 gifted.studies.lock.ocean.otr.usm.edu.16302 smtp log Aug 25 10:16:47 2003 (16302) smtp for 1 recips, completed in 1.400 seconds Aug 25 10:16:47 2003 (16302) smtp for 1 recips, completed in 3.799 seconds Aug 25 10:16:49 2003 (16302) smtp for 1 recips, completed in 1.968 seconds Aug 25 10:16:49 2003 (16302) smtp for 1 recips, completed in 1.990 seconds I'm guessing this would mean that there are not any orphaned locks. When I restart qrunner by hand, I kill the proces and remove all the locks from the locks directory. I thought the 3.799 seconds in the smtp log was qrunner telling you how long it took to send the mail through the MTA. - Original Message - From: "Richard Barrett" <[EMAIL PROTECTED]> To: "Andrew Falgout" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Monday, August 25, 2003 9:52 AM Subject: Re: [Mailman-Users] Mailman / Sendmail horribly slowperformance..suggestions please?!? > > On Monday, August 25, 2003, at 02:28 pm, Andrew Falgout wrote: > > > The error log is at 0 bytes. I have not overridden the Defaults.py > > which > > shows Defaults.py:MTA_ALIASES_STYLE = 'sendmail'. So I assume that > > means > > that sendmail is the MTA. > You will be using SMTP to the local MTA (which you say is Sendmail) if > the following MM config variables are set as follows in either > mm_cfg.py or the Defaults.py file. The following are th standard MM > 2.0.13 default values: > > DELIVERY_MODULE = 'SMTPDirect' > SMTPHOST = 'localhost' > > Also are the following default values still in place: > > SMTP_MAX_RCPTS = 500 > MAX_DELIVERY_THREADS = 0 > > > > > Aug 25 08:08:31 2003 (27798) smtp for 2 recips, completed in 1.045 > > seconds > > Aug 25 08:08:31 2003 (27798) smtp for 2 recips, completed in 1.149 > > seconds > > Aug 25 08:12:47 2003 (27798) smtp for 1 recips, completed in 0.208 > > seconds > > Aug 25 08:12:47 2003 (27798) smtp for 1 recips, completed in 0.241 > > seconds > > Aug 25 08:14:25 2003 (31184) smtp for 1 recips, completed in 0.701 > > seconds > > Aug 25 08:14:25 2003 (31184) smtp for 1 recips, completed in 0.716 > > seconds > > Aug 25 08:17:21 2003 (31184) smtp for 1 recips, completed in 0.167 > > seconds > > Aug 25 08:17:21 2003 (31184) smtp for 1 recips, completed in 0.179 > > seconds > > Aug 25 08:19:34 2003 (31184) smtp for 1 recips, completed in 3.962 > > seconds > > Aug 25 08:19:34 2003 (31184) smtp for 1 recips, completed in 4.234 > > seconds > > > > This is just a few of the lines in my /var/log/mailman/smtp file. > > There are > > a few bounces from time to time, but not on great frequency. I did > > send an > > email to a test list, and attempted to time when I would see my email > > address in the post log, but after 10 minutes I have not seen it. This > > really confuses me, because the mail system on this server delivers > > mail > > directly to a mailbox in seconds. Why the delay? > > > I am going to assume that you had not made the edit to > $prefix/Mailman/Handlers/SMTPDirect.py I suggested in one of my > response and that these log entries are the regular ones? > > Bear in mind that the number in brackets in these log entries - (27798) > for instance - is the process id of the qurnner handling the message > and the same qrunner is handling multiple messages. > > The smtp log entries are interesting if you have a large backlog of > messages in the qfiles directory because: > > 1. only a few recipients are being delivered for each log entry. Are > your mailing list that sparely populated for subscribers? Do these smtp > log entries tie in with post entries for lists with few members and > /var/log/mail entries for outgoing mail through the MTA? > > 2. quite long periods - 2, 3, 4 minutes in the sample log entries you > give
Re: [Mailman-Users] Mailman / Sendmail horriblyslowperformance..suggestions please?!?
I've included some of the diagnostics that was requested. As of this moment there are 7423 messages in the qfiles directory for mailman. And it continues to grow. Please let me know if my thinking is flawed on how mails flow through the system. 1. Message to a list hits sendmail initially and goes into the mqueue directory for sendmail for processing. 2. Sendmail delivers the message to the mailman wrapper which places the message in the qfiles directory for qrunner to process. 3. qrunner picks up these messages one by one, and delivers the message to each of the recpts listed for the list. How does qrunner decide which email to process? Is there a way to have it process a certain message in the qfiles directory? Are there any utilities for determining what is in the queue or why it was queued by qrunner. I really do appreciate all the help. Yes.. I am using sendmail as my MTA. Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 ocean.otr.usm.edu ESMTP Sendmail 8.11.6/8.11.6; Mon, 25 Aug 2003 09:21:56 -0500 (ocean) root $ time host python.org DNS lookups are almost instantaneous. python.org has address 194.109.137.226 host python.org 0.04s user 0.01s system 97% cpu 0.051 total (ocean) root $ dig -t python.org (/var/spool/mailman/qfiles) ;; Warning, ignoring invalid type python.org ; <<>> DiG 9.2.1 <<>> -t python.org ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19709 ;; flags: qr rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 13 ;; QUESTION SECTION: ;. IN NS ;; ANSWER SECTION: . 5402IN NS G.ROOT-SERVERS.NET. . 5402IN NS H.ROOT-SERVERS.NET. . 5402IN NS I.ROOT-SERVERS.NET. . 5402IN NS J.ROOT-SERVERS.NET. . 5402IN NS K.ROOT-SERVERS.NET. . 5402IN NS L.ROOT-SERVERS.NET. . 5402IN NS M.ROOT-SERVERS.NET. . 5402IN NS A.ROOT-SERVERS.NET. . 5402IN NS B.ROOT-SERVERS.NET. . 5402IN NS C.ROOT-SERVERS.NET. . 5402IN NS D.ROOT-SERVERS.NET. . 5402IN NS E.ROOT-SERVERS.NET. . 5402IN NS F.ROOT-SERVERS.NET. ;; ADDITIONAL SECTION: A.ROOT-SERVERS.NET. 600537 IN A 198.41.0.4 B.ROOT-SERVERS.NET. 600537 IN A 128.9.0.107 C.ROOT-SERVERS.NET. 510409 IN A 192.33.4.12 D.ROOT-SERVERS.NET. 510409 IN A 128.8.10.90 E.ROOT-SERVERS.NET. 602151 IN A 192.203.230.10 F.ROOT-SERVERS.NET. 510409 IN A 192.5.5.241 G.ROOT-SERVERS.NET. 602151 IN A 192.112.36.4 H.ROOT-SERVERS.NET. 602151 IN A 128.63.2.53 I.ROOT-SERVERS.NET. 510408 IN A 192.36.148.17 J.ROOT-SERVERS.NET. 602151 IN A 192.58.128.30 K.ROOT-SERVERS.NET. 510408 IN A 193.0.14.129 L.ROOT-SERVERS.NET. 510408 IN A 198.32.64.12 M.ROOT-SERVERS.NET. 510409 IN A 202.12.27.33 ;; Query time: 3 msec ;; SERVER: 131.95.84.2#53(131.95.84.2) ;; WHEN: Mon Aug 25 09:24:48 2003 ;; MSG SIZE rcvd: 436 - Original Message - From: "Jon Carnes" <[EMAIL PROTECTED]> To: "Andrew Falgout" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]>; "Richard Barrett" <[EMAIL PROTECTED]> Sent: Monday, August 25, 2003 9:09 AM Subject: Re: [Mailman-Users] Mailman / Sendmail horribly slowperformance..suggestions please?!? > The description of the problem fits right in with the primary DNS > resolver not working and the server having to fall back to the secondary > for each look up. > > That might not be your problem, but in anycase, you should run a > local-caching Nameserver on your mailserver. Then check to see how long > it takes to do dns lookups: > host -t mx some.host.com > dig -t mx some.host.com > > The "dig" command will tell you how long it took to get the dns > information. The "host" command will simply give you the information > you requested - which in this case is a listing of the Mailservers for > the domain some.host.com > > You might also want to tweak your sendmail configuration: > http://www.trilug.org/~jonc/mailserver/PartIII.html > > BTW: the many MTA's use the "sendmail aliases style", the fact that > Mailman is using it to communicate with your MTA does not insure that > you are using Sendmail as your MTA. > > To find out which MTA you are using: > telnet localhost 25 > > Read the header i
Re: [Mailman-Users] Mailman / Sendmail horribly slowperformance..suggestions please?!?
The error log is at 0 bytes. I have not overridden the Defaults.py which shows Defaults.py:MTA_ALIASES_STYLE = 'sendmail'. So I assume that means that sendmail is the MTA. Aug 25 08:08:31 2003 (27798) smtp for 2 recips, completed in 1.045 seconds Aug 25 08:08:31 2003 (27798) smtp for 2 recips, completed in 1.149 seconds Aug 25 08:12:47 2003 (27798) smtp for 1 recips, completed in 0.208 seconds Aug 25 08:12:47 2003 (27798) smtp for 1 recips, completed in 0.241 seconds Aug 25 08:14:25 2003 (31184) smtp for 1 recips, completed in 0.701 seconds Aug 25 08:14:25 2003 (31184) smtp for 1 recips, completed in 0.716 seconds Aug 25 08:17:21 2003 (31184) smtp for 1 recips, completed in 0.167 seconds Aug 25 08:17:21 2003 (31184) smtp for 1 recips, completed in 0.179 seconds Aug 25 08:19:34 2003 (31184) smtp for 1 recips, completed in 3.962 seconds Aug 25 08:19:34 2003 (31184) smtp for 1 recips, completed in 4.234 seconds This is just a few of the lines in my /var/log/mailman/smtp file. There are a few bounces from time to time, but not on great frequency. I did send an email to a test list, and attempted to time when I would see my email address in the post log, but after 10 minutes I have not seen it. This really confuses me, because the mail system on this server delivers mail directly to a mailbox in seconds. Why the delay? - Original Message - From: "Richard Barrett" <[EMAIL PROTECTED]> To: "Andrew Falgout" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Sunday, August 24, 2003 3:36 PM Subject: Re: [Mailman-Users] Mailman / Sendmail horribly slow performance..suggestions please?!? > > On Sunday, August 24, 2003, at 12:51 am, Andrew Falgout wrote: > > > I'm currently running mailman-2.0.13-1 eith sendmail-8.11.6-25.72. > > Posts > > are processed very fast, but delivery is more than slow. It takes 1-2 > > minutes per post to delivery mail. Evaluating what is going on in the > > queue is a little more than cumbersome. Over the course of 3-4 days > > mails > > have been piling up in the qfiles directory to the tune of 4000 mails > > and > > waiting. There are mails 3 days old waiting to go out. I'm looking > > for > > ways to evaluate what is in the queue, why it's in the queue, and how > > to > > tell qrunner to report more information on what it's thinking. Like > > perhaps the msgid of the email it's trying to process at this time. > > > Are you using the local sendmail instance as the outgoing SMTP server? > > Regardless, what do the Mailman post, smtp and smtp-failure logs show > about mail being handled by the system. How does this tie in with the > MTA's mail log. > > If your outgoing mail is being handled by the local sendmail MTA then I > would normally expect Mailman to handoff to the MTA without much delay. > If MM is having problems handing off to sendmail then you might find > some evidence of this in the smtp-failure log. > > -- > > Thank You, > > Andrew > > > > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+- > > +-+-+- > > - Andrew Wilson Falgout (RHCE) Linux, E Pluribus > > UNIX! - > > - Systems Admin II, TIU Office: (601) 266-5603 > > - > > - University of Southern Mississippi > > - > > - Fax: (601) 266-6788 > > - > > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- > > +-+-+-+ > > > -- > Mailman-Users mailing list > [EMAIL PROTECTED] > http://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > > This message was sent to: [EMAIL PROTECTED] > Unsubscribe or change your options at > http://mail.python.org/mailman/options/mailman-users/andrew.falgout%40usm.edu > -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ This message was sent to: [EMAIL PROTECTED] Unsubscribe or change your options at http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman / Sendmail horribly slow performance..suggestions please?!?
Sendmail usually delivers regular mail on an almost instant basis. What I'm getting it a feel that qrunner is not passing the mails from the qfiles directory for sendmail to deliver. The machine in question has around 15k uses on it, and they have almost instant mail delivery through sendmail for regular mail. When a mail comes to the list through sendmail initially, it gets delivered through the mqueue for sendmail and is posted to the qfiles of mm almost instantly. However that is where the holdup is. Qrunner doesn't seem to be dealing with these files well at all. They are sitting in the qfiles directory and are being delivered or sent out at a rate of 1-2 per minute. I don't see an smtp-failure log to check. If there are any features for debugging I can turn on in MM please let me know. Lock logging is the only thing I could find initially. On Sun, 24 Aug 2003, Richard Barrett wrote: > > On Sunday, August 24, 2003, at 12:51 am, Andrew Falgout wrote: > > > I'm currently running mailman-2.0.13-1 eith sendmail-8.11.6-25.72. > > Posts > > are processed very fast, but delivery is more than slow. It takes 1-2 > > minutes per post to delivery mail. Evaluating what is going on in the > > queue is a little more than cumbersome. Over the course of 3-4 days > > mails > > have been piling up in the qfiles directory to the tune of 4000 mails > > and > > waiting. There are mails 3 days old waiting to go out. I'm looking > > for > > ways to evaluate what is in the queue, why it's in the queue, and how > > to > > tell qrunner to report more information on what it's thinking. Like > > perhaps the msgid of the email it's trying to process at this time. > > > Are you using the local sendmail instance as the outgoing SMTP server? > > Regardless, what do the Mailman post, smtp and smtp-failure logs show > about mail being handled by the system. How does this tie in with the > MTA's mail log. > > If your outgoing mail is being handled by the local sendmail MTA then I > would normally expect Mailman to handoff to the MTA without much delay. > If MM is having problems handing off to sendmail then you might find > some evidence of this in the smtp-failure log. > > -- > > Thank You, > > Andrew > > > > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+- > > +-+-+- > > - Andrew Wilson Falgout (RHCE) Linux, E Pluribus > > UNIX! - > > - Systems Admin II, TIU Office: (601) 266-5603 > > - > > - University of Southern Mississippi > > - > > - Fax: (601) 266-6788 > > - > > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- > > +-+-+-+ > -- Thank You, Andrew -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+-+-+-+- - Andrew Wilson Falgout (RHCE) Linux, E Pluribus UNIX! - - Systems Admin II, TIU Office: (601) 266-5603- - University of Southern Mississippi - - Fax: (601) 266-6788- -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ This message was sent to: [EMAIL PROTECTED] Unsubscribe or change your options at http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Mailman / Sendmail horribly slow performance..suggestions please?!?
I'm currently running mailman-2.0.13-1 eith sendmail-8.11.6-25.72. Posts are processed very fast, but delivery is more than slow. It takes 1-2 minutes per post to delivery mail. Evaluating what is going on in the queue is a little more than cumbersome. Over the course of 3-4 days mails have been piling up in the qfiles directory to the tune of 4000 mails and waiting. There are mails 3 days old waiting to go out. I'm looking for ways to evaluate what is in the queue, why it's in the queue, and how to tell qrunner to report more information on what it's thinking. Like perhaps the msgid of the email it's trying to process at this time. -- Thank You, Andrew -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+-+-+-+- - Andrew Wilson Falgout (RHCE) Linux, E Pluribus UNIX! - - Systems Admin II, TIU Office: (601) 266-5603- - University of Southern Mississippi - - Fax: (601) 266-6788- -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ This message was sent to: [EMAIL PROTECTED] Unsubscribe or change your options at http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] mail man list problem
We recently moved to mailman from majordomo, and I'm having serious issues with performance. I've tried subscribing to the users and developers lists, but they have not responded to me within 24 hours. By this I mean, when I tried to subscribe, I never got an email for verifying the subscription. I really need help. I did not realize the performace would be so slow to respond, and my /var/spool/mailman/qfiles is filling up quick. I'm up to 3000 messages in the system, and it appears to only be sending out 1 smtp log entry every minute. How can I boost this performance?!?! Thank You, Andrew Wilson Falgout (RHCE) System Admin II University of Southern Mississippi Office: 601-266-5603 =>There are 10 kinds of people in the world. =>Those that understand binary, and those that don't. -- Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ This message was sent to: [EMAIL PROTECTED] Unsubscribe or change your options at http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
