>attachment is over 25mb 25 MB is simply too large to be sent using SMTP.
>Could you try your same test but with a 20mb attachment? I'll try it. >Curious to see if >you get the slowdown that I do. I'm sure - I'll not get the slowdown. Let's see. Sep-10-16 07:53:10 [Worker_1] [TLS-in] 209.85.220.169 info: started TLS-SSL session for client 209.85.220.169 - using TLSv1_2 , ECDHE-RSA-AES256-GCM-SHA384 Sep-10-16 07:53:10 M1-86790-10336 [Worker_1] [TLS-in] [TLS-out] 209.85.220.169 <[email protected]> message proxied without processing - message size (27020463) is above 500000 (npSize). Sep-10-16 07:53:11 M1-86790-10336 [Worker_1] [TLS-in] [TLS-out] 209.85.220.169 <[email protected]> to: [email protected] [SMTP Reply] 354 OK, send. Sep-10-16 07:56:07 M1-86790-10336 [Worker_1] [TLS-in] [TLS-out] 209.85.220.169 <[email protected]> to: [email protected] info: message is too large ( > neverQueueSize 12000000 byte) to be queued for further internal processing! Skipping DKMI, Plugins and charset conversion. Sep-10-16 08:01:11 M1-86790-10336 [Worker_1] [TLS-in] [TLS-out] 209.85.220.169 <[email protected]> to: [email protected] info: received all data - all data moved to send queue (1) Sep-10-16 08:01:11 M1-86790-10336 [Worker_1] [TLS-in] [TLS-out] 209.85.220.169 <[email protected]> to: [email protected] info: all DATA written to server - sent [CR][LF].[CR][LF] Sep-10-16 08:01:11 M1-86790-10336 [Worker_1] [TLS-in] [TLS-out] 209.85.220.169 <[email protected]> to: [email protected] [SMTP Reply] 250 Queued (480.768 seconds) Sep-10-16 08:01:12 [Worker_1] Finished message - received DATA size: 25.77 MByte - sent DATA size: 25.77 MByte Sep-10-16 08:01:12 [Worker_1] Disconnected: session:A6E6D1C 209.85.220.169 - command list was 'EHLO,STARTTLS,EHLO,MAIL FROM,RCPT TO,DATA,QUIT' - used 19359 SocketCalls - processing time 483 seconds - damped 0 seconds data receive speed 55KB/s frame size ~ 1400 Byte no renegotiation NOTICE: (this is not new - it has the same age like the forest) The perl module Convert::Scalar has to be installed ! I use version 1.11. This module is required to pre-allocate memory for internal data structures. For example, if the SIZE extension is used by a sender, assp allocates the full memory required to receive, analayze and to send this mail (after MAIL FROM:) - in my example 26MB several times. If the SIZE extension is not used, assp grows the memory allocation in big steps while receiving the mail, before perl will do it (using small steps). If Convert::Scalar is not installed, Perl will grow the required memory space as it is required. This is done by copying the old memory to a larger memory location. example: length $mail-receive-buffer = 5MB - the memory allocated by perl is 5MB - additionally 1KB is received by assp. Now Perl tries to expand the memory allocation of $mail-receive-buffer by 1KB - but if this is not possible, it copies the 5MB to a new location which is 5MB + 1KB long and frees up the old memory location. Now think about a currently 20MB received mail buffer. For each junk of additionally 1KB data, Perl will copy 20MB. How ever this problem description is not related to gmail.com - if Convert::Scalar is missing or not working like expected (Convert::Scalar::grow), receiving large mails will take general very very long - if the mail is too large, Perl will crash. Thomas Von: K Post <[email protected]> An: For Users of ASSP <[email protected]> Datum: 09.09.2016 22:11 Betreff: Re: [Assp-user] Extreme slow on bigger emails >>Now tell me - what should I look for and why I should change any code related to SSL? Beats me! This is way beyond my limited skill set, which is why I've been begging for help here. I just know that with 25mb attachments from gmail with TLS on in ASSP, it takes a crazy long time and if the attachment is over 25mb it'll ultimately timeout over and over until gmail gives up and sends a NDR. I've only (so far) seen this with gmail. I thought it was just me, but now there's another report. Could you try your same test but with a 20mb attachment? Curious to see if you get the slowdown that I do. It's fine for small files but rapidly crawls to a crawl after about 3 to 5 mb. Thanks On Fri, Sep 9, 2016 at 5:37 AM, Thomas Eckardt <[email protected]> wrote: > Just made a test - I've set ConnectionLog to diagnostic. > > Sep-09-16 10:36:16 [Worker_1] Info: Worker_1 got connection from > MainThread > Sep-09-16 10:36:16 [Worker_1] Info: try to connect to server at > 127.0.0.1:325 > Sep-09-16 10:36:16 [Worker_1] Info: connected to server at 127.0.0.1:325 > Sep-09-16 10:36:16 [Worker_1] Connected: session:B43AAAC > 209.85.216.174:35180 > 10.1.1.12:25 > 127.0.0.1:50969 > 127.0.0.1:325 , > 34-35 > Sep-09-16 10:36:16 [Worker_1] 209.85.216.174 [SMTP Reply] 220 > mail.thockar.com is ready - using ASSP 2.5.2(16250) > ... > Sep-09-16 10:36:17 [Worker_1] [TLS-in] 209.85.216.174 info: started > TLS-SSL session for client 209.85.216.174 - using TLSv1_2 , > ECDHE-RSA-AES256-GCM-SHA384 > Sep-09-16 10:36:17 [Worker_1] [TLS-in] [TLS-out] info: started TLS-SSL > session for server 127.0.0.1 - using TLSv1_2 , ECDHE-RSA-AES256-GCM-SHA384 > Sep-09-16 10:36:17 [Worker_1] [TLS-in] [TLS-out] 209.85.216.174 [SMTP > Reply] 250 HELP > Sep-09-16 10:36:17 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> info: found message size announcement: > 1.13 MByte > Sep-09-16 10:36:17 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> message proxied without processing - > message size (1188241) is above 500000 (npSize). > ... > Sep-09-16 10:36:18 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> to: [email protected] > DKIM-Signature found > ... > Sep-09-16 10:36:27 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> to: [email protected] [Plugin] > calling plugin ASSP_AFC > ... > Sep-09-16 10:36:27 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > [MessageOK] 209.85.216.174 <[email protected]> to: [email protected] > message ok - (noProcessing - message size (1188241) is above 500000 > (npSize)) - ..... > Sep-09-16 10:36:27 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> to: [email protected] info: > received all data - all data moved to send queue (8) > Sep-09-16 10:36:29 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> to: [email protected] info: all > DATA written to server - sent [CR][LF].[CR][LF]... > ... > Sep-09-16 10:36:29 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> to: [email protected] [SMTP > Reply] 250 Queued (11.008 seconds) > Sep-09-16 10:36:29 M1-10177-06112 [Worker_1] [TLS-in] [TLS-out] > 209.85.216.174 <[email protected]> to: [email protected] info: no > (more) data readable from 209.85.216.174 (connection closed by peer) - > last command was 'QUIT' > Sep-09-16 10:36:29 [Worker_1] Finished message - received DATA size: 1.13 > MByte - sent DATA size: 1.13 MByte > Sep-09-16 10:36:29 [Worker_1] Disconnected: session:B43AAAC 209.85.216.174 > - command list was 'EHLO,STARTTLS,EHLO,MAIL FROM,RCPT TO,DATA,QUIT' - used > 859 SocketCalls - processing time 13 seconds - damped 0 seconds > > > The frame size used by google is 1400. > As you can see - there is not a single SSL renegotiation like > > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read - renegotiation > in > > > progress - SSL_WANT_READ > > The connection to the local MTA uses also STARTTLS. > The overall transmission time is 13 seconds, including all attachment > checks. The receive time for the data is 9 seconds. > > versions: > > ASSP 2.5.2 build 16250 > Perl 5.16.3 > IO::Socket::SSL - 2.031 > Net-SSLeay: version 1.74 - build information > openssl : OpenSSL 1.0.2g 1 Mar 2016 > compiler: cl /MD /Ox /O2 /Ob2 /W3 /Gs0 /GF /Gy /nologo > -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -DDSO_WIN32 > -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE > -DOPENSSL_USE_APPLINK -I. -DOPENSSL_NO_RC5 -DOPENSSL_NO_MD2 > -DOPENSSL_NO_SSL2 -DOPENSSL_NO_KRB5 -DOPENSSL_NO_JPAKE > -DOPENSSL_NO_WEAK_SSL_CIPHERS -DOPENSSL_NO_STATIC_ENGINE > built on: reproducible build, date unspecified > platform: platform: VC-WIN32 > > 'SSLSMTPConfigure' is not configured > > The used certificate is a Class 2 from StartSSL. The hostname in the 220 > greeting and the one in the MX-record (both are equal) are valid for this > certificate and the certificate itself is also valid! > > The strongest possible encryption was negotiated: TLSv1_2 , > ECDHE-RSA-AES256-GCM-SHA384 > > You can see - it is working perfect - even with gmail.com. > > Now tell me - what should I look for and why I should change any code > related to SSL? > > Thomas > > > > > > > Von: K Post <[email protected]> > An: For Users of ASSP <[email protected]> > Datum: 08.09.2016 19:55 > Betreff: Re: [Assp-user] Extreme slow on bigger emails > > > > Fair enough, but that means we've got to monitor that. My bigger concern > is that we're making an exception to reduce security on connections from > one of the biggest players in the industry. Wouldn't it be preferable to > figure out what's going on? Is Google doing something completely > non-standard? > > On Thu, Sep 8, 2016 at 2:37 AM, Thomas Eckardt > <[email protected]> > wrote: > > > >But what happens when they start sending > > from another IP? > > > > Than, the SPF record is changed. > > > > Thomas > > > > > > > > > > > > Von: K Post <[email protected]> > > An: For Users of ASSP <[email protected]> > > Datum: 07.09.2016 20:38 > > Betreff: Re: [Assp-user] Extreme slow on bigger emails > > > > > > > > This sounds similar to what I'm seeing with gmail mails when TLS is on. > > > > It's hard for me to believe that Google would be doing something > > completely > > non-standard. I can't see there being a chance that they'd have some > > setting that makes outgoing mail slow for them, though I suppose it's > > possible. > > > > I feel like there must be some setting or code error in ASSP that > doesn't > > agree with the way that google's doing things. I hate to say it, but > > Google's email presence is just too big to ignore this. Sure we can > turn > > off TLS or restrict based on IP. But what happens when they start > sending > > from another IP? Remember they'll stop the transmission if it's been > > alive > > too long. And what happens when they start flagging sent items in > > people's > > mailbox as insecure, as they've begun doing with those in the inbox? > > > > Now, I've got no clue whatsoever how to figure out what the problem but > > I'm > > happy to help test in any way I can. > > > > On Tue, Sep 6, 2016 at 2:03 AM, Thomas Eckardt > > <[email protected]> > > wrote: > > > > > As a workaround you may add the following IPv4 ranges to 'noTLSIP' > > > > > > 64.18.0.0/20 > > > 64.233.160.0/19 > > > 66.102.0.0/20 > > > 66.249.80.0/20 > > > 72.14.192.0/18 > > > 74.125.0.0/16 > > > 108.177.8.0/21 > > > 173.194.0.0/16 > > > 207.126.144.0/20 > > > 209.85.128.0/17 > > > 216.58.192.0/19 > > > 216.239.32.0/19 > > > 172.217.0.0/19 > > > > > > They are the IPv4 ranges shown in the SFP record of gmail.com > > > > > > Thomas > > > > > > > > > Von: Christian Leicht <[email protected]> > > > An: [email protected] > > > Datum: 04.09.2016 22:43 > > > Betreff: Re: [Assp-user] Extreme slow on bigger emails > > > > > > > > > > > > Some more infos > > > > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read renegotiation > > > finished - recovered from - SSL_WANT_READ > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read - renegotiation > in > > > progress - SSL_WANT_READ > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read renegotiation > > > finished - recovered from - SSL_WANT_READ > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read - renegotiation > in > > > progress - SSL_WANT_READ > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read renegotiation > > > finished - recovered from - SSL_WANT_READ > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read - renegotiation > in > > > progress - SSL_WANT_READ > > > Sep 4 22:40:02 mail assp[2628]: wwl8-21214-10692 209.85.220.172 > > > <[email protected]> to: [email protected] info: ssl-read renegotiation > > > finished - recovered from - SSL_WANT_READ > > > > > > > > > Am 04.09.2016 um 22:25 schrieb Christian Leicht: > > > > I have some more informations. > > > > I enable all debug logs in assp and see that the peer is disconnect > > the > > > > transmission. > > > > I watched 3 transfers more accurately, and no transmission is break > up > > > > at the same size. And the transmission is very slow. 1MB = 1000sek. > > > > > > > > What could that be? > > > > > > > > Sep 4 21:29:14 mail assp[2628]: Info: will remove file > > > > '/usr/share/assp/notspam/ab--1358091.eml' , because mail delivery > was > > > > incomplete for a good mail > > > > Sep 4 21:29:14 mail assp[2628]: Finished message - received DATA > > size: > > > > 15.12 MByte - sent DATA size: 84 Byte > > > > Sep 4 21:29:14 mail assp[2628]: Disconnected: session:7FD8EC8F0E98 > > > > 209.85.220.172 - processing time 1169 seconds > > > > Sep 4 21:29:14 mail assp[2628]: wwl8-16186-00032 209.85.220.172 > > > > <[email protected]> to: [email protected] info: file > > > > /usr/share/assp/notspam/ab--1358091.eml was deleted - reason: > > incomplete > > > > good mail > > > > > > > > > > > > Sep 4 21:29:15 mail assp[2628]: Info: will remove file > > > > '/usr/share/assp/notspam/test--1358085.eml' , because mail delivery > > was > > > > incomplete for a good mail > > > > Sep 4 21:29:15 mail assp[2628]: Finished message - received DATA > > size: > > > > 21.99 MByte - sent DATA size: 168 Byte > > > > Sep 4 21:29:15 mail assp[2628]: Disconnected: session:EFC56A8 > > > > 212.227.15.4 - processing time 2225 seconds > > > > Sep 4 21:29:15 mail assp[2628]: wwl8-15130-05665 212.227.15.4 > > > > <[email protected]> to: [email protected] info: file > > > > /usr/share/assp/notspam/test--1358085.eml was deleted - reason: > > > > incomplete good mail > > > > > > > > > > > > > > > > Sep 4 22:11:24 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:25 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:25 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:25 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:25 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:25 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:25 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:26 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:27 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 1400 > > byte > > > > of DATA > > > > Sep 4 22:11:27 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: read and processed 920 > > byte > > > > of DATA > > > > Sep 4 22:11:27 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: no (more) data readable > > from > > > > 209.85.220.172 (connection closed by peer) - last command was 'DATA' > > > > Sep 4 22:11:27 mail assp[2628]: wwl8-18717-12751 209.85.220.172 > > > > <[email protected]> to: [email protected] info: file > > > > /usr/share/assp/notspam/ab--1358118.eml was deleted - reason: > > incomplete > > > > good mail > > > > > > > > > > > > > > > > Am 03.09.2016 um 23:06 schrieb Christian Leicht: > > > >> Hello, i use the latest ASSP v2 on a new server. > > > >> All works fine but larger Emails (20MB) are extreme slow and some > are > > > >> lost connection. > > > >> This addresses are whitelisted an no ClamAV is involved. TLS is > > working > > > >> on this connections. Send a 20 MB Message from web.de to my Server > > take > > > >> 20 minutes. Emails are going thru Postfix without Amavis. > > > >> > > > >> What can i do? > > > >> > > > >> Thanks for help > > > >> Christian > > > >> > > > >> > > > ------------------------------------------------------------ > > > ------------------ > > > >> _______________________________________________ > > > >> Assp-user mailing list > > > >> [email protected] > > > >> https://lists.sourceforge.net/lists/listinfo/assp-user > > > >> > > > > > > > > > > > ------------------------------------------------------------ > > > ------------------ > > > > _______________________________________________ > > > > Assp-user mailing list > > > > [email protected] > > > > https://lists.sourceforge.net/lists/listinfo/assp-user > > > > > > > > > > ------------------------------------------------------------ > > > ------------------ > > > _______________________________________________ > > > Assp-user mailing list > > > [email protected] > > > https://lists.sourceforge.net/lists/listinfo/assp-user > > > > > > > > > > > > > > > > > > > > > DISCLAIMER: > > > ******************************************************* > > > This email and any files transmitted with it may be confidential, > > legally > > > privileged and protected in law and are intended solely for the use of > > the > > > > > > individual to whom it is addressed. > > > This email was multiple times scanned for viruses. There should be no > > > known virus in this email! > > > ******************************************************* > > > > > > > > > ------------------------------------------------------------ > > > ------------------ > > > > > > _______________________________________________ > > > Assp-user mailing list > > > [email protected] > > > https://lists.sourceforge.net/lists/listinfo/assp-user > > > > > > > > ------------------------------------------------------------ > > ------------------ > > _______________________________________________ > > Assp-user mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/assp-user > > > > > > > > > > DISCLAIMER: > > ******************************************************* > > This email and any files transmitted with it may be confidential, > legally > > privileged and protected in law and are intended solely for the use of > the > > > > individual to whom it is addressed. > > This email was multiple times scanned for viruses. There should be no > > known virus in this email! > > ******************************************************* > > > > > > ------------------------------------------------------------ > > ------------------ > > > > _______________________________________________ > > Assp-user mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/assp-user > > > > > ------------------------------------------------------------ > ------------------ > _______________________________________________ > Assp-user mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/assp-user > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > Assp-user mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/assp-user > > ------------------------------------------------------------------------------ _______________________________________________ Assp-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/assp-user DISCLAIMER: ******************************************************* This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! *******************************************************
------------------------------------------------------------------------------
_______________________________________________ Assp-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/assp-user
