Re: [exim] Slow down ?
> When several messages are sent to @ orange.fr in a too short period of > time, they are (temporarily) refused: > > Too many connections I use in the transport: serialize_hosts = * You can use: serialize_hosts = smtp-in.orange.fr At reboot: rm /var/spool/exim/db/misc* -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Slow down ?
On Mon, 4 Feb 2019 at 12:53, Jeremy Harris via Exim-users < exim-users@exim.org> wrote: > On 04/02/2019 09:22, Odhiambo Washington via Exim-users wrote: > > Out of curiosity, I went checking for this option, but only > > found remote_max_parallel documented - as a global/main option. > > Is max_parallel as a transport option documented anywhere?? > > I can quickly find it in the "option index". Where did you look? > In the documentation. It seems I did not dig deeper enough. > > PS: please do not copy me on mails sent to the list. I read the > list. I don't need a duplicate. > I am not CCing you. I do reply-all, but that does include your address by default. I keep forgetting to edit. Sorry for that. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-) -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Slow down ?
On 04/02/2019 09:22, Odhiambo Washington via Exim-users wrote: > Out of curiosity, I went checking for this option, but only > found remote_max_parallel documented - as a global/main option. > Is max_parallel as a transport option documented anywhere?? I can quickly find it in the "option index". Where did you look? PS: please do not copy me on mails sent to the list. I read the list. I don't need a duplicate. -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Slow down ?
On Mon, 4 Feb 2019 at 01:16, Jeremy Harris via Exim-users < exim-users@exim.org> wrote: > On 03/02/2019 09:17, Francois Sauterey via Exim-users wrote> I've a > problem with one of main French provider : orange.fr > > When several messages are sent to @ orange.fr in a too short period of > > time, they are (temporarily) refused: > > > >> SMTP error from remote mail server after initial connection: 421 > >> mwinf5c78 ME Trop de connexions, veuillez verifier votre > >> configuration. Too many connections, slow down. OFR004_104 [104] > > The french part of this message say some thing like "Too many > > connections, please verify you configuration" > > One thing you could do would be to deliberately queue this > class of messages (decided during ACL), and run your queue > using "-qq", and not so often. You'd then have single > connections handling multiple messages to a larger extent, > so fewer of them overall. > > Another thing you could do would be to investigate the > transport option "max_parallel". > > Out of curiosity, I went checking for this option, but only found remote_max_parallel documented - as a global/main option. Is max_parallel as a transport option documented anywhere?? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-) -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Slow down ?
On 03/02/2019 09:17, Francois Sauterey via Exim-users wrote> I've a problem with one of main French provider : orange.fr > When several messages are sent to @ orange.fr in a too short period of > time, they are (temporarily) refused: > >> SMTP error from remote mail server after initial connection: 421 >> mwinf5c78 ME Trop de connexions, veuillez verifier votre >> configuration. Too many connections, slow down. OFR004_104 [104] > The french part of this message say some thing like "Too many > connections, please verify you configuration" One thing you could do would be to deliberately queue this class of messages (decided during ACL), and run your queue using "-qq", and not so often. You'd then have single connections handling multiple messages to a larger extent, so fewer of them overall. Another thing you could do would be to investigate the transport option "max_parallel". -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
[exim] Slow down ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi every body, I've a problem with one of main French provider : orange.fr When several messages are sent to @ orange.fr in a too short period of time, they are (temporarily) refused: > SMTP error from remote mail server after initial connection: 421 > mwinf5c78 ME Trop de connexions, veuillez verifier votre > configuration. Too many connections, slow down. OFR004_104 [104] The french part of this message say some thing like "Too many connections, please verify you configuration" Then the mailq takes a long time to empty ... Well, the question in "how to slow down" ? If I found the solution for Postfix, I did not find the one for exim :-( Can you help me ? Regards Francois -BEGIN PGP SIGNATURE- Version: GnuPG v2 iF4EAREIAAYFAlxWsaUACgkQFN3+vrHD9BsobQD/eRNUxEq/QszncsrbNTxHfJOv VU0lcJWHT/Y6eLAtB2wA/2yhK81TrAi7hu5yLp3vBxacnSsDHc8gzrPTZ4hn+8pw =mk2V -END PGP SIGNATURE- -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Does EXIM slow down when it gets fat?
On 09/04/15 14:26, Rob Gunther wrote: > In our normal day-to-day we have maybe 2,500 messages in the exim queue at > any time. We retry delivery every 15 minutes, slowing down the retry as > messages gets older. > > Had a bit of a routing issue, where Exim was accepting mail but could not > deliver to the next server (not a mailbox). > > The queue grew on one server to just over 200,000 messages ( 14 GIGS worth! > ). > > We stopped sending new mail to the server, so it could deal with what it > had. > > I assumed it would blast through those messages very quick. > > It didn't move quick, it seemed to crawl. Exim does have a rep. for behaving that way. The writer (I think) commented that it was traditionally intended for environments where the queue could be kept small. There is some built-in support for spreading the queue over several directories. See split_spool_directory. Some sites implement things like: once an item is old-enough in the queue, move it to a different queue directory, which is serviced by a separate exim instance. See also https://github.com/Exim/exim/wiki/Performance -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Does EXIM slow down when it gets fat?
Hi Rob,
Rob Gunther (Do 09 Apr 2015 15:26:18 CEST):
> In our normal day-to-day we have maybe 2,500 messages in the exim queue at
> any time. We retry delivery every 15 minutes, slowing down the retry as
> messages gets older.
…
> When the queue got down to maybe 90,000 messages I found the speed that
> Exim was processing seemed to be increasing, started to time and see how
> many messages it was doing per minute.
>
> At 85,300 messages in the queue it did about 480 deliveries per minute.
> At 67,775 messages in the queue it did about 785 deliveries per minute.
> At 40,458 messages in the queue it did about 1,227 deliveries per minute.
A single queue runner process isn't very fast. The master process spawns
new queue runners on a regularly basis (15 minutes in your setup?)
Thus I'd assume that in the beginning you had just one queue runner,
after a while 2, then 3 and so on… sending more messages/minute.
> I have read on here that Exim does not do well with a large queue, is my
> observation normal? The bigger the queue the slower the processing?
I'd say even with only 2500 messages in the queue you won't send more
than 480/minute (the 480 isn't a fixed number, I copied it from your
observations, the rate depends on lots of factors: DNS lookup, file
system bottle necks, locking of the hint files, …)
> How could I have handled that massive amount of mail automatically, I want
> to see those messages flying!
You may try to start more queue runners at once, manually… Or try to
use -q5s as a parameter for the daemon, it would ask the daemon to
spawn a new queue runner every 5 seconds.
> Is there a way to configure Exim to stop accepting mail if there is so much
> in the queue? Maybe if there is 75,000 message in the queue we just refuse
> to accept anymore because there is obviously a problem and adding more to
> the queue is just going to make things worse.
Acceptance can be dependend on the load of your system, but I think
there is now number/expansion item with the numbers of queued messages…
Because the listener can't know about the queue. There may be even more
systems putting the messages into the same (shared) queue. Currently
Exim maintains no simply and cheap way to get the number of queued
messages.
But - of course you can do something like this:
acl_check_connect:
defer condition = ${if <{2500}{${run{$exim_path -bpc
But it may waste ressources for counting the messages..
Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de internet & unix support -
Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
gnupg encrypted messages are welcome --- key ID: F69376CE -
! key id 7CBF764A and 972EAC9F are revoked since 2015-01 -
signature.asc
Description: Digital signature
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[exim] Does EXIM slow down when it gets fat?
In our normal day-to-day we have maybe 2,500 messages in the exim queue at any time. We retry delivery every 15 minutes, slowing down the retry as messages gets older. Had a bit of a routing issue, where Exim was accepting mail but could not deliver to the next server (not a mailbox). The queue grew on one server to just over 200,000 messages ( 14 GIGS worth! ). We stopped sending new mail to the server, so it could deal with what it had. I assumed it would blast through those messages very quick. It didn't move quick, it seemed to crawl. Doing a tail -f command on the log I could easily read the log it was moving so slow. I ended up opening a bunch of SSH connections, picking a domain and doing something like this: for eximid in `exiqgrep -i -r @example.com`; do exim -m $eximid; done; That got a higher number of messages processing, still with no load on the server. When the queue got down to maybe 90,000 messages I found the speed that Exim was processing seemed to be increasing, started to time and see how many messages it was doing per minute. At 85,300 messages in the queue it did about 480 deliveries per minute. At 67,775 messages in the queue it did about 785 deliveries per minute. At 40,458 messages in the queue it did about 1,227 deliveries per minute. I have read on here that Exim does not do well with a large queue, is my observation normal? The bigger the queue the slower the processing? How could I have handled that massive amount of mail automatically, I want to see those messages flying! Is there a way to configure Exim to stop accepting mail if there is so much in the queue? Maybe if there is 75,000 message in the queue we just refuse to accept anymore because there is obviously a problem and adding more to the queue is just going to make things worse. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
