It's been about a full day now with 75. I see nothing but greatness.
On Sun, Oct 2, 2016 at 2:59 AM, Thomas Eckardt <thomas.ecka...@thockar.com>
wrote:
> >then I see 9 seconds of idle/damping,
>
> If the complete mail is queued, this time is required to do the post
> checks and conversions (charset conversion, plugins level 2, DKIM) and to
> send the mail data to your MTA.
>
>
> Thomas
>
>
>
>
>
> Von: K Post <nntp.p...@gmail.com>
> An: ASSP development mailing list <assp-test@lists.sourceforge.net>
> Datum: 01.10.2016 22:01
> Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses /
> servers
>
>
>
> I've been testing 16275 for a couple hours now. FAST FAST FAST and stable
> so far. I see no errors in the maillog.
>
> Some notes / questions
>
> 1) 11MB attachment (14 MB email after encoding) transfers from google,
> total time 25-29 seconds. It seems that the whole email transfers in
> about
> 20 seconds, then I see 9 seconds of idle/damping, I've never before seen
> (or noticed I suppose) the idle column grow under normal circumstances.
> Did you change the way messages are processed, in terms of doing
> scans/analysis post transfer? This idle counter doesn't matter to me,
> speed's great, but I want to insure there's nothing wrong - and I'm
> curious
> if something changed on this front.
>
> 2) I see no notable difference in speed when I disable TLS for google now.
> This is TERRIFIC. There's certainly processing overhead for SSL, but your
> code is so fast now that it's not noticeable.
>
> 3) Any point in adding a column to shutdown_list saying what the task at
> hand is? AFC? ClamAV? Etc?
>
> For reference - we resumed this discussion August 1 (there were previous
> attempts at getting this resolved, but they didn't go anywhere). About 2
> months later, 70 or so messages in this thread, a ton of your time,
> dedication, and brainpower, we went from a gmail TLS message totaling 14
> MB
> taking upwards of 13 minutes and being unreliable now take only 25 seconds
> and be rock solid! I don't know how you figured all of this out, put up
> with my persistence, or had the time and brainpower to spare, but you seem
> to have conquered this terrible problem!
>
> Tausend dank! Ich bin Ihnen sehr dankbar für.
>
>
>
>
> On Sat, Oct 1, 2016 at 11:23 AM, Thomas Eckardt
> <thomas.ecka...@thockar.com>
> wrote:
>
> > Ken, 2.5.2 build 16275 is in CVS /test
> >
> > Switching on the 'SSL_read_ahead' flag was a nice idea, but this caused
> > some sockets - some times - at any state - to become unreadable.
> > I've done extensive test on this - but I was not able to find a
> > reproduceable reason for the behavior.
> > I removed this code in build 16275. The read ahead is still available
> for
> > TLS/SSL and it is used per default by assp. The mechanism is a bit
> > different like in build 16273/74.
> > Every TLS/SSL socket gets additionally 50 milliseconds to read and
> decode
> > a much as available data from the underlying BIO
> >
> > On my nice old slow system, I was able to receive a 20MB mail from gmail
> > in 110 seconds.
> > Even for yahoo (they use 4096 byte SSL frames) it makes a big
> difference.
> >
> > Thomas
> >
> >
> >
> >
> > Von: K Post <nntp.p...@gmail.com>
> > An: ASSP development mailing list <assp-test@lists.sourceforge.net>
> > Datum: 30.09.2016 19:21
> > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses /
> > servers
> >
> >
> >
> > 70 and 71 is fine here (Windows). 73 was SUPER fast with SSL messages
> > from gmail, but then we got the idle / delay issues and had to revert to
> > 71. Haven't had a chance to try 74 (need to wait for after hours)
> >
> >
> > On Fri, Sep 30, 2016 at 1:04 PM, Colin Waring <co...@dolphinict.co.uk>
> > wrote:
> >
> > > 16256 works acceptably but shuts down once or twice a day. 16270 or
> > > 16274_1 gave me problems with delays.
> > >
> > > I suspect the shutting down is a symptom of a different problem as it
> > has
> > > happened for a while.
> > >
> > > On 30 Sep 2016 17:57, Thomas Eckardt <thomas.ecka...@thockar.com>
> wrote:
> > > Hmm ... not OK.
> > >
> > > for my records:
> > >
> > > build 16256 is running fine
> > > builds 16270 and higher make problems
> > >
> > > right?
> > >
> > > Thomas
> > >
> > >
> > >
> > >
> > >
> > > Von: cw <colin.war...@gmail.com>
> > > An: ASSP development mailing list
> <assp-test@lists.sourceforge.net>
> > > Datum: 30.09.2016 17:19
> > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses /
> > > servers
> > >
> > >
> > >
> > > I've had to roll back now unfortunately as I'm getting email problems
> > > again
> > > :(
> > >
> > > On Fri, Sep 30, 2016 at 3:50 PM, cw <colin.war...@gmail.com> wrote:
> > >
> > > > Mixed results on this. So far no problems with running workers being
> > > > logged but the GUI has become incredibly unresponsive. By
> unresponsive
> > I
> > > > mean I waited a good couple of minutes for the shutdown_list page to
> > > load.
> > > > The dot on the main page is red yet the workers page is all green.
> > > > Scratch that, it has refreshed again and I have a worker stuck:
> > > > Worker 3, loop age 252s, action: header (Content-Disposition -attr)
> :
> > :
> > > > filename name (stuck)
> > > > 30s later and it is healthy again..
> > > >
> > > > On the server I haven't upgraded the shutdown_list page comes up
> > within
> > > > seconds. I'm not sure whether to leave it running or whether this is
> > > > evidence of the same kind of unresponsiveness that cause me to have
> to
> > > roll
> > > > back earlier this week.
> > > >
> > > > On Fri, Sep 30, 2016 at 3:29 PM, cw <colin.war...@gmail.com> wrote:
> > > >
> > > >> I wish I'd spotted this before writing out the other message. I'll
> > give
> > > >> it a test now for you.
> > > >>
> > > >> On Fri, Sep 30, 2016 at 2:17 PM, Thomas Eckardt <
> > > >> thomas.ecka...@thockar.com> wrote:
> > > >>
> > > >>> Collin, this should no longer happen using the updated 2.5.2
> 16274_1
> > > at
> > > >>> CVS /test
> > > >>>
> > > >>> Thomas
> > > >>>
> > > >>>
> > > >>>
> > > >>> Von: cw <colin.war...@gmail.com>
> > > >>> An: ASSP development mailing list
> > > <assp-test@lists.sourceforge.net>
> > > >>> Datum: 29.09.2016 16:40
> > > >>> Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> addresses
> > /
> > > >>> servers
> > > >>>
> > > >>>
> > > >>>
> > > >>> Hi Thomas,
> > > >>> I moved up to 16270 following this thread of discussion but then
> had
> > a
> > > >>> day
> > > >>> working away. I've come back to huge issues with delays, mails not
> > > going
> > > >>> through and many, many of these in the logs:
> > > >>>
> > > >>> Info: unable to detect any running worker for a new connection -
> > wait
> > > >>> (max
> > > >>> 30 seconds)
> > > >>>
> > > >>> When I say many, I have over 21,000 lines in today's log file. I
> > also
> > > >>> found
> > > >>> the GUI unresponsive or not connecting at all and ASSP restarting
> > > quite
> > > >>> regularly.
> > > >>>
> > > >>> I've dropped back to 16256 and things are instantly better. Do you
> > > think
> > > >>> going up to 16273 might improve things over 16270 or am I better
> > > holding
> > > >>> off for now?
> > > >>> All the best,
> > > >>> Colin.
> > > >>>
> > > >>> On Thu, Sep 29, 2016 at 3:15 PM, Thomas Eckardt
> > > >>> <thomas.ecka...@thockar.com>
> > > >>> wrote:
> > > >>>
> > > >>> > I just released 2.5.2 build 16273 at CVS test folder
> > > >>> >
> > > >>> > http://assp.cvs.sourceforge.net/viewvc/assp/assp2/test/
> > > >>> >
> > > >>> > This release should make a very large difference for SSL/TLS
> mails
> > > sent
> > > >>> by
> > > >>> > hosts that uses small SSL-frame size.
> > > >>> >
> > > >>> > Tell me your test results.
> > > >>> >
> > > >>> >
> > > >>> > Thomas
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > Von: K Post <nntp.p...@gmail.com>
> > > >>> > An: ASSP development mailing list
> > > <assp-test@lists.sourceforge.net
> > > >>> >
> > > >>> > Datum: 28.09.2016 19:42
> > > >>> > Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> > addresses
> > > /
> > > >>> > servers
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > But I want a postman driving a Ferarri with monster truck tires
> > that
> > > >>> can
> > > >>> > roll over the traffic (and if wishes are being granted, I'd
> prefer
> > > the
> > > >>> car
> > > >>> > in a deep blue instead of classic red).
> > > >>> >
> > > >>> > We regularly see people attaching large files or a bunch of
> > smaller
> > > >>> ones
> > > >>> > that add up to a big email, I'm talking lots and lots of
> different
> > > >>> people
> > > >>> > from outside the organization sending to us, and this happens on
> a
> > > >>> daily
> > > >>> > basis. It's especially popular with photos and huge scans
> > > multi-page
> > > >>> > 600dpi (which people don't understand can be done at low
> > > resolution).
> > > >>> > Often it's people sending in scanned official documents for us
> to
> > > >>> review
> > > >>> > an
> > > >>> > help them. They're not our staff, they're the people we help.
> > They
> > > >>> have
> > > >>> > a
> > > >>> > tendency of not following any instructions, and ignore the fact
> > that
> > > we
> > > >>> > have a web based system for the process. We can't control it
> and
> > > the
> > > >>> > powers that be don't want us lowering the 30 MB threshold across
> > the
> > > >>> > board. Lot of these people use gmail.com addresses and google
> > > allows
> > > >>> for
> > > >>> > up to 25 MB - https://support.google.com/mail/answer/6584
> > > >>> >
> > > >>> > I think it's really interesting that google seems to use this
> > > >>> inefficient
> > > >>> > small packet size for SSL, allows for 25MB emails, is a big
> > > proponent
> > > >>> of
> > > >>> > SSL, and at the same time doesn't allow mails to take more than
> 15
> > > >>> minutes
> > > >>> > to transfer. Now that you've made things >much< more efficient
> on
> > > the
> > > >>> > ASSP
> > > >>> > side, I'm hoping that all will be okay. I just get annoyed by
> > > >>> > inefficiency.
> > > >>> >
> > > >>> >
> > > >>> > I'm going to tryrunning with npSize of zero, the no queuing size
> > set
> > > >>> very
> > > >>> > high and see how that goes. I want to insure that even the
> > biggest
> > > >>> emails
> > > >>> > are scanned for malware before hitting the inbox.
> > > >>> >
> > > >>> >
> > > >>> > I've never said this before, but it seems like google's doing it
> > > wrong.
> > > >>> I
> > > >>> > think we've exhausted our options here. If anyone has a Google
> > > >>> > engineering
> > > >>> > friend, or a friend of a friend, this might be a good time to
> have
> > a
> > > >>> > little
> > > >>> > chat in an effort to reach someone who can either explain why
> > > they're
> > > >>> use
> > > >>> > a
> > > >>> > 1.4 kB frame size for ssl packets, but a normal much bigger size
> > for
> > > >>> > non-encrypted traffic or to get them to see an error in their
> ways
> > > and
> > > >>> fix
> > > >>> > this!
> > > >>> >
> > > >>> > Thank you again for all of the discussion and explanation and of
> > > course
> > > >>> > the
> > > >>> > code changes which have made a huge difference.
> > > >>> >
> > > >>> > On Wed, Sep 28, 2016 at 12:59 PM, Thomas Eckardt
> > > >>> > <thomas.ecka...@thockar.com
> > > >>> > > wrote:
> > > >>> >
> > > >>> > > >I'm still afraid of running into frequent problems with 25mb
> > > >>> > attachments
> > > >>> > > though.
> > > >>> > >
> > > >>> > > 25MB - I don't know anyone who allows this. Sending a link to
> a
> > > >>> download
> > > >>> > > is much smarter.
> > > >>> > > ASSP_AFC has an option to do this for your outgoing mails! -
> > > >>> > > ASSP_AFCWebScript
> > > >>> > >
> > > >>> > > >Can someone else run a debug test with Gmail and TLS to see
> if
> > > >>> you're
> > > >>> > > seeing these tiny packets too?
> > > >>> > >
> > > >>> > > I've done it - same behavior - 1400 byte per SSL frame from
> > gmail.
> > > >>> > >
> > > >>> > >
> > > >>> > > >Would it be an easy modification to show the negotiated
> cipher
> > in
> > > >>> the
> > > >>> > >
> > > >>> > > ASSP does this for years now - simply look in to the received
> > > header
> > > >>> > line
> > > >>> > > added by assp.
> > > >>> > >
> > > >>> > > -- -- -- Received: from vs10241.internet1.de ([158.181.49.77]
> > > >>> > > helo=vs10241.internet1.de)
> > > >>> > > by xxxxxx with SMTPS(TLSv1_2
> > ECDHE-RSA-AES256-GCM-SHA384)
> > > >>> > (2.5.2);
> > > >>> > > 24 Jul 2016 05:34:12 +0200
> > > >>> > >
> > > >>> > > if you set 'ConnectionLog' at least to verbose, you'll see the
> > > >>> > > SSL-negotiation results in maillog.txt.
> > > >>> > >
> > > >>> > > info: started TLS-SSL session for client $cliIP - using
> > > >>> > > $Con{$ssls}->{sslversion} , $Con{$ssls}->{sslcipher}
> > > >>> > >
> > > >>> > > >what other options are there?
> > > >>> > >
> > > >>> > > Set 'EnableHighPerformance' to very high.
> > > >>> > >
> > > >>> > > >Are you sure that negotiating a lesser cipher with Gmail
> > wouldn't
> > > >>> have
> > > >>> > > them switch to sending larger SSL packets?
> > > >>> > >
> > > >>> > > Yes, I'm 100% sure.
> > > >>> > >
> > > >>> > > An SSL peer is free to use any SSL frame size between 1 byte
> and
> > > >>> 16kB.
> > > >>> > > There is no SSL-negotiation parameter for min or max frame
> size.
> > > >>> > > If gmail.com sends 1.4kB frames, they are free to do it this
> > way.
> > > >>> There
> > > >>> > is
> > > >>> > > no technical way to force them to send larger frames.
> > > >>> > >
> > > >>> > > >If neverQueue is set to 12MB, am I correct in saying that
> we're
> > > open
> > > >>> to
> > > >>> > a
> > > >>> > > targeted exploit, say an .exe in a .zip as long as the email
> is
> > > more
> > > >>> > than
> > > >>> > > 12MB?
> > > >>> > >
> > > >>> > > Yes, you are correct.
> > > >>> > >
> > > >>> > > >or is that just after the whole message is received?
> > > >>> > >
> > > >>> > > Yes, after the complete mail is received.
> > > >>> > >
> > > >>> > > It is like it is - if the postman with his Ferrari is in a
> > traffic
> > > >>> jam,
> > > >>> > > you'll have to wait longer for your letter or you'll get it
> the
> > > next
> > > >>> > day.
> > > >>> > >
> > > >>> > >
> > > >>> > > Thomas
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > Von: K Post <nntp.p...@gmail.com>
> > > >>> > > An: ASSP development mailing list
> > > >>> <assp-test@lists.sourceforge.net>
> > > >>> > > Datum: 28.09.2016 17:03
> > > >>> > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> > > >>> addresses /
> > > >>> > > servers
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > Thanks again for the detailed explanation.
> > > >>> > >
> > > >>> > > 10% faster is terrific, and it's more than 50% faster since
> > before
> > > >>> > 16270.
> > > >>> > > I'm still afraid of running into frequent problems with 25mb
> > > >>> attachments
> > > >>> > > though. Yes, that's too big for email in our opinion, but
> Gmail
> > > >>> allows
> > > >>> > it
> > > >>> > > as do most ISP's these days. It's not terribly unusual for us
> > to
> > > >>> have
> > > >>> a
> > > >>> > a
> > > >>> > > person try to send and email with four or five 4 MB photos -
> and
> > > we
> > > >>> > can't
> > > >>> > > stop that trend. That'll translate into a message somewhere
> in
> > > the
> > > >>> > range
> > > >>> > > of 22-25 MB. I'm not sure if we're fast enough to
> > >consistently<
> > > >>> > receive
> > > >>> > > these. And on top of that, if it's an outside big wig,
> emailing
> > > an
> > > >>> > inside
> > > >>> > > big wig, they don't have 15 minutes to waste waiting for the
> big
> > > >>> email
> > > >>> > to
> > > >>> > > arrive. Then they yell at me, which is why I keep asking more
> > of
> > > >>> these
> > > >>> > > questions -- and there's a lot of them here (sorry)
> > > >>> > >
> > > >>> > >
> > > >>> > > *Are you sure that negotiating a lesser cipher with Gmail
> > wouldn't
> > > >>> have
> > > >>> > > them switch to sending larger SSL packets? * I have a solid
> high
> > > >>> level
> > > >>> > > understanding of encryption, but once you get down the the
> > packet
> > > >>> level,
> > > >>> > > I'm a bit out of my depth, despite what I've learned from your
> > > >>> excellent
> > > >>> > > explanations.
> > > >>> > >
> > > >>> > >
> > > >>> > > Can someone else run a debug test with Gmail and TLS to see if
> > > you're
> > > >>> > > seeing these tiny packets too? I keep going back to *not
> being
> > > able
> > > >>> to
> > > >>> > > believe that Google doing something that was any slower than
> > > >>> necessary.
> > > >>> > *
> > > >>> > > They
> > > >>> > > try to optimize everything and are huge proponents of SSL
> > > everywhere.
> > > >>> If
> > > >>> > > they're sending tons of tiny un-optimized packets, there's got
> > to
> > > be
> > > >>> a
> > > >>> > > reason or cause. I can't imagine that it's a default and
> > > deliberate
> > > >>> > > setting. I tried calling US support, but this is a network
> > > >>> engineering
> > > >>> > > type of issue - no chance of me getting through to someone
> there
> > > who
> > > >>> > even
> > > >>> > > understands the question.
> > > >>> > >
> > > >>> > > The npSize option and neverQueueSize, only affects things
> after
> > > the
> > > >>> > whole
> > > >>> > > message is received right? * It's not like it's doing
> something
> > > >>> > > significantly extra at each SSL cylce is it except using up
> RAM
> > > >>> right??*
> > > >>> > > My unscientific tests are shower marginally slower speeds
> when
> > > >>> > receiving
> > > >>> > > large email from Google with TLS on and neverQueueSize set
> crazy
> > > >>> high,
> > > >>> > so
> > > >>> > > maybe I'm wrong. Being that this same message through
> > outlook.com
> > > >>> only
> > > >>> > > takes 30 seconds, even with npSize set to 0 (unlimited) and
> > > >>> > neverQueueSize
> > > >>> > > = 999,000,000, I'm wondering if the processing power and ram
> > > >>> > availability
> > > >>> > > (24 GB) on my assp box combined with low usage (only averaging
> > > 5000
> > > >>> > > messages a day) might be enough to have neverQueueSize really
> > high
> > > to
> > > >>> > > always scan everything. I'll play around there, but if
> > > >>> neverQueueSize
> > > >>> > is
> > > >>> > > 99MB, will things like ASSP_AFCDetectSpamAttachRe be slow
> WHILE
> > > the
> > > >>> > > message
> > > >>> > > is transferring, or is that just after the whole message is
> > > received?
> > > >>> > >
> > > >>> > > If neverQueue is set to 12MB, am I correct in saying that
> we're
> > > open
> > > >>> to
> > > >>> > a
> > > >>> > > targeted exploit, say an .exe in a .zip as long as the email
> is
> > > more
> > > >>> > than
> > > >>> > > 12MB? If so, that seems like a big hole, and something that I
> > > >>> > personally
> > > >>> > > would be willing to sacrifice speed/processing power to close.
> > > >>> > >
> > > >>> > > Suggestion, note in the gui under npSize that 0 really sets
> this
> > > to a
> > > >>> > > maximum of 12,000,000 unless neverQueueSize is set higher in
> > > >>> > > CorrectASSPcfg.pm. I've got that correct, right?)
> > > >>> > >
> > > >>> > > I forgot that DKIM signing can be done over the body too! We
> > > don't
> > > >>> do
> > > >>> > > that, but I forgot that others might. Maybe we have npSize
> only
> > > >>> ignore
> > > >>> > > DKIM if it's done over the body? DKIM should be fast if it's
> > just
> > > >>> the
> > > >>> > > typical headers that are used right? We have npSize set to
> > zero
> > > >>> > though,
> > > >>> > > so this doesn't really impact me.
> > > >>> > >
> > > >>> > > Would it be an easy modification to show the negotiated cipher
> > in
> > > the
> > > >>> > > maillog or in the email body, if nothing else, just to have
> more
> > > >>> info?
> > > >>> > > X-ASSP-Cipher: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384
> > > >>> > > for example or in the received line like Microsoft does:
> > > >>> > >
> > > >>> > > Received: from __________ (x.x.x.x) by ______
> > > >>> > > (y.y.y.y) with Microsoft SMTP Server (version=TLS1_2,
> > > >>> > > cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384)
> > > >>> > >
> > > >>> > > Or like Gmail does
> > > >>> > > Received: from smtp.ourcharity.org (smtp.ourcharity.org.
> > > [x.x.x.x])
> > > >>> > > by mx.google.com with ESMTPS id
> > > >>> > > l1si54144332.11.2016.09.28.07.51.21
> > > >>> > > for <nntp.p...@gmail.com>
> > > >>> > > (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256
> > > >>> > bits=128/128);
> > > >>> > > Wed, 28 Sep 2016 07:24:22 -0700 (PDT)
> > > >>> > >
> > > >>> > > It's a little more data (and certainly your time to code
> this),
> > > but
> > > >>> > could
> > > >>> > > be useful if we need it in the future.
> > > >>> > >
> > > >>> > > and::
> > > >>> > >
> > > >>> > > Again Ken, the SSL parameters are NOT the problem on your
> > system.
> > > >>> Your
> > > >>> > >
> > > >>> > > debug log shows a socket read time of max ~ 0.5 milliseconds
> > > (typical
> > > >>> > ~0.3
> > > >>> > >
> > > >>> > > ms) for SSL. This read operation includes the time required
> for
> > > the
> > > >>> > >
> > > >>> > > decryption of the data. This is very very fast!
> > > >>> > >
> > > >>> > > It is simply the amazing count (10.400) of read and process
> > > >>> operations
> > > >>> > >
> > > >>> > > (cycles) required by assp for such a mail, that causes the
> > overall
> > > >>> slow
> > > >>> > >
> > > >>> > > mail receive.
> > > >>> > >
> > > >>> > > in summary, if there's not an ASSP setting that I can tweak to
> > > have
> > > >>> > google
> > > >>> > > start sending larger SSL packets like Outlook.com and everyone
> > > else
> > > >>> I've
> > > >>> > > seen does, what other options are there? Are you suggesting
> > that
> > > ALL
> > > >>> > ASSP
> > > >>> > > installations are incapable of receiving large TLS emails from
> > > Google
> > > >>> > > regardless of the installation's processing power, RAM, and
> > > >>> bandwidth?
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > On Wed, Sep 28, 2016 at 4:26 AM, Thomas Eckardt
> > > >>> > > <thomas.ecka...@thockar.com>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > >The same email now took only 269 seconds.
> > > >>> > > >
> > > >>> > > > This is OK for googles behavior - the 1440 byte SSL frame.
> > > Around
> > > >>> 300s
> > > >>> > > was
> > > >>> > > > expected by me for this mail - hmm.. it is 10% better -
> nice!
> > > >>> > > >
> > > >>> > > > Take the following math.
> > > >>> > > >
> > > >>> > > > mail size = 15000 kB
> > > >>> > > > google frame size = 1.44 kB
> > > >>> > > >
> > > >>> > > > required assp loop cycles = mail size / frame size = 10.400
> > > >>> > > >
> > > >>> > > > mail size = 15000 kB
> > > >>> > > > outlook.com frame size = 16 kB
> > > >>> > > >
> > > >>> > > > required assp loop cycles = mail size / frame size = 940
> > > >>> > > >
> > > >>> > > > >I really wonder if I need to tweak my cipher_list.
> > > >>> > > >
> > > >>> > > > No!
> > > >>> > > > Again Ken, the SSL parameters are NOT the problem on your
> > > system.
> > > >>> Your
> > > >>> > > > debug log shows a socket read time of max ~ 0.5 milliseconds
> > > >>> (typical
> > > >>> > > ~0.3
> > > >>> > > > ms) for SSL. This read operation includes the time required
> > for
> > > the
> > > >>> > > > decryption of the data. This is very very fast!
> > > >>> > > > It is simply the amazing count (10.400) of read and process
> > > >>> operations
> > > >>> > > > (cycles) required by assp for such a mail, that causes the
> > > overall
> > > >>> > slow
> > > >>> > > > mail receive.
> > > >>> > > >
> > > >>> > > > >Is ClamAV scanning skipped too?
> > > >>> > > > Yes.
> > > >>> > > >
> > > >>> > > > >Could the plugins be run on the full mail after receipt,
> > > >>> regardless
> > > >>> > of
> > > >>> > > > size?
> > > >>> > > >
> > > >>> > > > Override the config parameters. Keep in mind that 'npSize'
> may
> > > also
> > > >>> > > > involved in skipping or processing some mail body checks.
> > > >>> > > >
> > > >>> > > > >Isn't DKIM checking just a one time thing and not
> intensive?
> > > >>> > > >
> > > >>> > > > The full DKIM check is very intensive. It requires to
> > calculate
> > > an
> > > >>> > > RSA/SHA
> > > >>> > > > hash over the complete mail.
> > > >>> > > > DCC and Razor are doing something similar.
> > > >>> > > > ASSP_AFC would make all checks (ClamAV, FileScan, content
> > checks
> > > >>> with
> > > >>> > > > several regular expression, decompression of attachments
> ....)
> > > for
> > > >>> the
> > > >>> > > > complete mail. It parses the complete mail at once with
> > > >>> Email::MIME.
> > > >>> > > This
> > > >>> > > > requires a huge amount of memory. Not a big deal on a 64bit
> OS
> > > with
> > > >>> > 8GB
> > > >>> > > > RAM and several CPU cores - who can !?
> > > >>> > > > All these can be done for any mail size, if the system is
> able
> > > to
> > > >>> > > process
> > > >>> > > > the amount of data fast enough.
> > > >>> > > > On most havy load systems, the 12.000.000 will be to large
> and
> > > may
> > > >>> > lead
> > > >>> > > in
> > > >>> > > > to stucking workers.
> > > >>> > > >
> > > >>> > > > ASSP has this set of config parameters - change them to your
> > > need -
> > > >>> > try
> > > >>> > > -
> > > >>> > > > and if does not work switch back - nothing more easy.
> > > >>> > > >
> > > >>> > > > Thomas
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > Von: K Post <nntp.p...@gmail.com>
> > > >>> > > > An: ASSP development mailing list
> > > >>> > <assp-test@lists.sourceforge.net>
> > > >>> > > > Datum: 27.09.2016 21:08
> > > >>> > > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> > > >>> addresses
> > > >>> /
> > > >>> > > > servers
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > Our primary internet connection went down again (nothing to
> do
> > > with
> > > >>> > > ASSP)
> > > >>> > > > which gave me the opportunity to replace 16270 with 16271.
> > > Nothing
> > > >>> > like
> > > >>> > > > making lemonade out of lemons...
> > > >>> > > >
> > > >>> > > > The same email now took only 269 seconds. That's about 15x
> > > longer
> > > >>> > than
> > > >>> > > > with TLS off, but WOW that's way better than it was before.
> > > >>> > > > I also tried with the blank cipher list, no notable
> difference
> > > >>> > > > And with the SSL buffers set to 0 (64 MB), again without a
> > speed
> > > >>> > > > difference.
> > > >>> > > >
> > > >>> > > > SO- you've made a real difference here!! Is there more
> > > >>> optimization
> > > >>> > to
> > > >>> > > > be
> > > >>> > > > made?
> > > >>> > > >
> > > >>> > > > The rub is that the exact same message sent through
> > Outlook.com
> > > to
> > > >>> us,
> > > >>> > > > took
> > > >>> > > > exactly 30 seconds, just a 50% overhead when compared to the
> > 19
> > > >>> > seconds
> > > >>> > > > for
> > > >>> > > > a non-TLS message of the same size, instead of a 1500%
> > overhead
> > > for
> > > >>> > > > encryption when receiving from google.
> > > >>> > > >
> > > >>> > > > *Is there some magical debug switch that I could turn on to
> > see
> > > >>> what
> > > >>> > > > encryption Outlook.com is using and compare that to what
> > > Google's
> > > >>> > > > connection to us with?* I think prohibiting whatever the
> slow
> > > >>> cipher
> > > >>> > > that
> > > >>> > > > google's using (probably a really strong one) might make the
> > > final
> > > >>> bit
> > > >>> > > of
> > > >>> > > > difference.
> > > >>> > > >
> > > >>> > > > I'm breathing so much easier now!! Thank you.
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > On Tue, Sep 27, 2016 at 2:08 PM, K Post
> <nntp.p...@gmail.com>
> > > >>> wrote:
> > > >>> > > >
> > > >>> > > > > Despite all the problems we have with personalities and
> > > policies
> > > >>> in
> > > >>> > > our
> > > >>> > > > > organization, the infrastructure is pretty solid. MTU's
> are
> > > set
> > > >>> > > > correctly,
> > > >>> > > > > no fragmentation, no jitter. There's low latency across
> > the
> > > >>> board,
> > > >>> > > and
> > > >>> > > > > really low bandwidth usage. If we sent 1000 mails a day,
> > it's
> > > a
> > > >>> > lot.
> > > >>> > > > >
> > > >>> > > > > ...and yes, they even pay their bills, though not me very
> > well
> > > :)
> > > >>> > > > >
> > > >>> > > > > I think it's which SSL algorithm is being used that's at
> > least
> > > >>> > > partially
> > > >>> > > > > to blame. I have:
> > > >>> > > > > SSL_Version: SSLv23:!SSLv3:!SSLv2
> > > >>> > > > > SSL_Cipher_list:
> kEECDH+ECDSA:kEECDH:kEDH:HIGH:+SHA:+RC4:RC4
> > > >>> > > > >
> :!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!DSS:!PSK:!SRP:!kECDH:!
> > > >>> > > > > CAMELLIA128:!IDEA:!SEED
> > > >>> > > > >
> > > >>> > > > > I tried the default wih SSL_Cipher_List blank before, I
> > don't
> > > >>> think
> > > >>> > > > there
> > > >>> > > > > was a difference (but I've played with so many settings, I
> > > really
> > > >>> > > don't
> > > >>> > > > > remember)
> > > >>> > > > >
> > > >>> > > > > And last, on the SSL buffer size. If set to zero in the
> > gui,
> > > on
> > > >>> > > windows
> > > >>> > > > > 2012, it says in green that it's set to 64 MB. I follow
> > what
> > > >>> you're
> > > >>> > > > saying
> > > >>> > > > > about it readying 4x 16 Kb without a loop cycle. Is that
> a
> > > good
> > > >>> or
> > > >>> > > bad
> > > >>> > > > > thing though?
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > ------------------------------------------------------------
> > > >>> > > > ------------------
> > > >>> > > > _______________________________________________
> > > >>> > > > Assp-test mailing list
> > > >>> > > > Assp-test@lists.sourceforge.net
> > > >>> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > DISCLAIMER:
> > > >>> > > > *******************************************************
> > > >>> > > > This email and any files transmitted with it may be
> > > confidential,
> > > >>> > > legally
> > > >>> > > > privileged and protected in law and are intended solely for
> > the
> > > use
> > > >>> of
> > > >>> > > the
> > > >>> > > >
> > > >>> > > > individual to whom it is addressed.
> > > >>> > > > This email was multiple times scanned for viruses. There
> > should
> > > be
> > > >>> no
> > > >>> > > > known virus in this email!
> > > >>> > > > *******************************************************
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > ------------------------------------------------------------
> > > >>> > > > ------------------
> > > >>> > > >
> > > >>> > > > _______________________________________________
> > > >>> > > > Assp-test mailing list
> > > >>> > > > Assp-test@lists.sourceforge.net
> > > >>> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>> > > >
> > > >>> > > >
> > > >>> > > ------------------------------------------------------------
> > > >>> > > ------------------
> > > >>> > > _______________________________________________
> > > >>> > > Assp-test mailing list
> > > >>> > > Assp-test@lists.sourceforge.net
> > > >>> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > DISCLAIMER:
> > > >>> > > *******************************************************
> > > >>> > > This email and any files transmitted with it may be
> > confidential,
> > > >>> > legally
> > > >>> > > privileged and protected in law and are intended solely for
> the
> > > use
> > > >>> of
> > > >>> > the
> > > >>> > >
> > > >>> > > individual to whom it is addressed.
> > > >>> > > This email was multiple times scanned for viruses. There
> should
> > be
> > > no
> > > >>> > > known virus in this email!
> > > >>> > > *******************************************************
> > > >>> > >
> > > >>> > >
> > > >>> > > ------------------------------------------------------------
> > > >>> > > ------------------
> > > >>> > >
> > > >>> > > _______________________________________________
> > > >>> > > Assp-test mailing list
> > > >>> > > Assp-test@lists.sourceforge.net
> > > >>> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>> > >
> > > >>> > >
> > > >>> > ------------------------------------------------------------
> > > >>> > ------------------
> > > >>> > _______________________________________________
> > > >>> > Assp-test mailing list
> > > >>> > Assp-test@lists.sourceforge.net
> > > >>> > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > DISCLAIMER:
> > > >>> > *******************************************************
> > > >>> > This email and any files transmitted with it may be
> confidential,
> > > >>> legally
> > > >>> > privileged and protected in law and are intended solely for the
> > use
> > > of
> > > >>> the
> > > >>> >
> > > >>> > individual to whom it is addressed.
> > > >>> > This email was multiple times scanned for viruses. There should
> be
> > > no
> > > >>> > known virus in this email!
> > > >>> > *******************************************************
> > > >>> >
> > > >>> >
> > > >>> > ------------------------------------------------------------
> > > >>> > ------------------
> > > >>> >
> > > >>> > _______________________________________________
> > > >>> > Assp-test mailing list
> > > >>> > Assp-test@lists.sourceforge.net
> > > >>> > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>> >
> > > >>> >
> > > >>> ------------------------------------------------------------
> > > >>> ------------------
> > > >>> _______________________________________________
> > > >>> Assp-test mailing list
> > > >>> Assp-test@lists.sourceforge.net
> > > >>> https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> DISCLAIMER:
> > > >>> *******************************************************
> > > >>> This email and any files transmitted with it may be confidential,
> > > legally
> > > >>> privileged and protected in law and are intended solely for the
> use
> > of
> > > >>> the
> > > >>>
> > > >>> individual to whom it is addressed.
> > > >>> This email was multiple times scanned for viruses. There should be
> > no
> > > >>> known virus in this email!
> > > >>> *******************************************************
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > > >>> ------------------
> > > >>>
> > > >>> _______________________________________________
> > > >>> Assp-test mailing list
> > > >>> Assp-test@lists.sourceforge.net
> > > >>> https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >>>
> > > >>>
> > > >>
> > > >
> > > ------------------------------------------------------------
> > > ------------------
> > > _______________________________________________
> > > Assp-test mailing list
> > > Assp-test@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > >
> > >
> > >
> > >
> > > DISCLAIMER:
> > > *******************************************************
> > > This email and any files transmitted with it may be confidential,
> > legally
> > > privileged and protected in law and are intended solely for the use of
> > the
> > >
> > > individual to whom it is addressed.
> > > This email was multiple times scanned for viruses. There should be no
> > > known virus in this email!
> > > *******************************************************
> > >
> > >
> > >
> > > ------------------------------------------------------------
> > > ------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > > _______________________________________________
> > > Assp-test mailing list
> > > Assp-test@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > >
> > >
> > ------------------------------------------------------------
> > ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Assp-test mailing list
> > Assp-test@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/assp-test
> >
> >
> >
> >
> > DISCLAIMER:
> > *******************************************************
> > This email and any files transmitted with it may be confidential,
> legally
> > privileged and protected in law and are intended solely for the use of
> the
> >
> > individual to whom it is addressed.
> > This email was multiple times scanned for viruses. There should be no
> > known virus in this email!
> > *******************************************************
> >
> >
> > ------------------------------------------------------------
> > ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Assp-test mailing list
> > Assp-test@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/assp-test
> >
> >
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
>
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
