Hi Thomas,
I've put 16275 on one of my servers this morning. I'm seeing the no running
workers error every 10-20 minutes causing ASSP to shut down. I've sat there
with the worker status page open and I'm not seeing any workers getting
stuck that coincide with this problem. In fact when the error is scrolling
past in the logs the workers are happily carrying on with whatever emails
they are processing.
Whilst typing, worker 5 has just gone to Bayes(OK_Run) Stuck with a timer
of around 240 before it cleared. This didn't trigger the fault just like
the other stuck message didn't really coincide and cleared itself.
I've upgraded the second server and so far am not seeing the delays that I
did with 16270. I'll keep an eye on it. Sure wish I could figure out what
is causing the no running workers error though.
All the best,
Colin
On Sun, Oct 2, 2016 at 7:44 PM, K Post <nntp.p...@gmail.com> wrote:
> It's been about a full day now with 75. I see nothing but greatness.
>
> On Sun, Oct 2, 2016 at 2:59 AM, Thomas Eckardt <thomas.ecka...@thockar.com
> >
> wrote:
>
> > >then I see 9 seconds of idle/damping,
> >
> > If the complete mail is queued, this time is required to do the post
> > checks and conversions (charset conversion, plugins level 2, DKIM) and to
> > send the mail data to your MTA.
> >
> >
> > Thomas
> >
> >
> >
> >
> >
> > Von: K Post <nntp.p...@gmail.com>
> > An: ASSP development mailing list <assp-test@lists.sourceforge.net>
> > Datum: 01.10.2016 22:01
> > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses /
> > servers
> >
> >
> >
> > I've been testing 16275 for a couple hours now. FAST FAST FAST and
> stable
> > so far. I see no errors in the maillog.
> >
> > Some notes / questions
> >
> > 1) 11MB attachment (14 MB email after encoding) transfers from google,
> > total time 25-29 seconds. It seems that the whole email transfers in
> > about
> > 20 seconds, then I see 9 seconds of idle/damping, I've never before
> seen
> > (or noticed I suppose) the idle column grow under normal circumstances.
> > Did you change the way messages are processed, in terms of doing
> > scans/analysis post transfer? This idle counter doesn't matter to me,
> > speed's great, but I want to insure there's nothing wrong - and I'm
> > curious
> > if something changed on this front.
> >
> > 2) I see no notable difference in speed when I disable TLS for google
> now.
> > This is TERRIFIC. There's certainly processing overhead for SSL, but
> your
> > code is so fast now that it's not noticeable.
> >
> > 3) Any point in adding a column to shutdown_list saying what the task at
> > hand is? AFC? ClamAV? Etc?
> >
> > For reference - we resumed this discussion August 1 (there were previous
> > attempts at getting this resolved, but they didn't go anywhere). About 2
> > months later, 70 or so messages in this thread, a ton of your time,
> > dedication, and brainpower, we went from a gmail TLS message totaling 14
> > MB
> > taking upwards of 13 minutes and being unreliable now take only 25
> seconds
> > and be rock solid! I don't know how you figured all of this out, put up
> > with my persistence, or had the time and brainpower to spare, but you
> seem
> > to have conquered this terrible problem!
> >
> > Tausend dank! Ich bin Ihnen sehr dankbar für.
> >
> >
> >
> >
> > On Sat, Oct 1, 2016 at 11:23 AM, Thomas Eckardt
> > <thomas.ecka...@thockar.com>
> > wrote:
> >
> > > Ken, 2.5.2 build 16275 is in CVS /test
> > >
> > > Switching on the 'SSL_read_ahead' flag was a nice idea, but this caused
> > > some sockets - some times - at any state - to become unreadable.
> > > I've done extensive test on this - but I was not able to find a
> > > reproduceable reason for the behavior.
> > > I removed this code in build 16275. The read ahead is still available
> > for
> > > TLS/SSL and it is used per default by assp. The mechanism is a bit
> > > different like in build 16273/74.
> > > Every TLS/SSL socket gets additionally 50 milliseconds to read and
> > decode
> > > a much as available data from the underlying BIO
> > >
> > > On my nice old slow system, I was able to receive a 20MB mail from
> gmail
> > > in 110 seconds.
> > > Even for yahoo (they use 4096 byte SSL frames) it makes a big
> > difference.
> > >
> > > Thomas
> > >
> > >
> > >
> > >
> > > Von: K Post <nntp.p...@gmail.com>
> > > An: ASSP development mailing list <assp-test@lists.sourceforge.net
> >
> > > Datum: 30.09.2016 19:21
> > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses /
> > > servers
> > >
> > >
> > >
> > > 70 and 71 is fine here (Windows). 73 was SUPER fast with SSL
> messages
> > > from gmail, but then we got the idle / delay issues and had to revert
> to
> > > 71. Haven't had a chance to try 74 (need to wait for after hours)
> > >
> > >
> > > On Fri, Sep 30, 2016 at 1:04 PM, Colin Waring <co...@dolphinict.co.uk>
> > > wrote:
> > >
> > > > 16256 works acceptably but shuts down once or twice a day. 16270 or
> > > > 16274_1 gave me problems with delays.
> > > >
> > > > I suspect the shutting down is a symptom of a different problem as it
> > > has
> > > > happened for a while.
> > > >
> > > > On 30 Sep 2016 17:57, Thomas Eckardt <thomas.ecka...@thockar.com>
> > wrote:
> > > > Hmm ... not OK.
> > > >
> > > > for my records:
> > > >
> > > > build 16256 is running fine
> > > > builds 16270 and higher make problems
> > > >
> > > > right?
> > > >
> > > > Thomas
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Von: cw <colin.war...@gmail.com>
> > > > An: ASSP development mailing list
> > <assp-test@lists.sourceforge.net>
> > > > Datum: 30.09.2016 17:19
> > > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> addresses /
> > > > servers
> > > >
> > > >
> > > >
> > > > I've had to roll back now unfortunately as I'm getting email problems
> > > > again
> > > > :(
> > > >
> > > > On Fri, Sep 30, 2016 at 3:50 PM, cw <colin.war...@gmail.com> wrote:
> > > >
> > > > > Mixed results on this. So far no problems with running workers
> being
> > > > > logged but the GUI has become incredibly unresponsive. By
> > unresponsive
> > > I
> > > > > mean I waited a good couple of minutes for the shutdown_list page
> to
> > > > load.
> > > > > The dot on the main page is red yet the workers page is all green.
> > > > > Scratch that, it has refreshed again and I have a worker stuck:
> > > > > Worker 3, loop age 252s, action: header (Content-Disposition -attr)
> > :
> > > :
> > > > > filename name (stuck)
> > > > > 30s later and it is healthy again..
> > > > >
> > > > > On the server I haven't upgraded the shutdown_list page comes up
> > > within
> > > > > seconds. I'm not sure whether to leave it running or whether this
> is
> > > > > evidence of the same kind of unresponsiveness that cause me to have
> > to
> > > > roll
> > > > > back earlier this week.
> > > > >
> > > > > On Fri, Sep 30, 2016 at 3:29 PM, cw <colin.war...@gmail.com>
> wrote:
> > > > >
> > > > >> I wish I'd spotted this before writing out the other message. I'll
> > > give
> > > > >> it a test now for you.
> > > > >>
> > > > >> On Fri, Sep 30, 2016 at 2:17 PM, Thomas Eckardt <
> > > > >> thomas.ecka...@thockar.com> wrote:
> > > > >>
> > > > >>> Collin, this should no longer happen using the updated 2.5.2
> > 16274_1
> > > > at
> > > > >>> CVS /test
> > > > >>>
> > > > >>> Thomas
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Von: cw <colin.war...@gmail.com>
> > > > >>> An: ASSP development mailing list
> > > > <assp-test@lists.sourceforge.net>
> > > > >>> Datum: 29.09.2016 16:40
> > > > >>> Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> > addresses
> > > /
> > > > >>> servers
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Hi Thomas,
> > > > >>> I moved up to 16270 following this thread of discussion but then
> > had
> > > a
> > > > >>> day
> > > > >>> working away. I've come back to huge issues with delays, mails
> not
> > > > going
> > > > >>> through and many, many of these in the logs:
> > > > >>>
> > > > >>> Info: unable to detect any running worker for a new connection -
> > > wait
> > > > >>> (max
> > > > >>> 30 seconds)
> > > > >>>
> > > > >>> When I say many, I have over 21,000 lines in today's log file. I
> > > also
> > > > >>> found
> > > > >>> the GUI unresponsive or not connecting at all and ASSP restarting
> > > > quite
> > > > >>> regularly.
> > > > >>>
> > > > >>> I've dropped back to 16256 and things are instantly better. Do
> you
> > > > think
> > > > >>> going up to 16273 might improve things over 16270 or am I better
> > > > holding
> > > > >>> off for now?
> > > > >>> All the best,
> > > > >>> Colin.
> > > > >>>
> > > > >>> On Thu, Sep 29, 2016 at 3:15 PM, Thomas Eckardt
> > > > >>> <thomas.ecka...@thockar.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > I just released 2.5.2 build 16273 at CVS test folder
> > > > >>> >
> > > > >>> > http://assp.cvs.sourceforge.net/viewvc/assp/assp2/test/
> > > > >>> >
> > > > >>> > This release should make a very large difference for SSL/TLS
> > mails
> > > > sent
> > > > >>> by
> > > > >>> > hosts that uses small SSL-frame size.
> > > > >>> >
> > > > >>> > Tell me your test results.
> > > > >>> >
> > > > >>> >
> > > > >>> > Thomas
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > Von: K Post <nntp.p...@gmail.com>
> > > > >>> > An: ASSP development mailing list
> > > > <assp-test@lists.sourceforge.net
> > > > >>> >
> > > > >>> > Datum: 28.09.2016 19:42
> > > > >>> > Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> > > addresses
> > > > /
> > > > >>> > servers
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > But I want a postman driving a Ferarri with monster truck tires
> > > that
> > > > >>> can
> > > > >>> > roll over the traffic (and if wishes are being granted, I'd
> > prefer
> > > > the
> > > > >>> car
> > > > >>> > in a deep blue instead of classic red).
> > > > >>> >
> > > > >>> > We regularly see people attaching large files or a bunch of
> > > smaller
> > > > >>> ones
> > > > >>> > that add up to a big email, I'm talking lots and lots of
> > different
> > > > >>> people
> > > > >>> > from outside the organization sending to us, and this happens
> on
> > a
> > > > >>> daily
> > > > >>> > basis. It's especially popular with photos and huge scans
> > > > multi-page
> > > > >>> > 600dpi (which people don't understand can be done at low
> > > > resolution).
> > > > >>> > Often it's people sending in scanned official documents for us
> > to
> > > > >>> review
> > > > >>> > an
> > > > >>> > help them. They're not our staff, they're the people we help.
> > > They
> > > > >>> have
> > > > >>> > a
> > > > >>> > tendency of not following any instructions, and ignore the fact
> > > that
> > > > we
> > > > >>> > have a web based system for the process. We can't control it
> > and
> > > > the
> > > > >>> > powers that be don't want us lowering the 30 MB threshold
> across
> > > the
> > > > >>> > board. Lot of these people use gmail.com addresses and google
> > > > allows
> > > > >>> for
> > > > >>> > up to 25 MB - https://support.google.com/mail/answer/6584
> > > > >>> >
> > > > >>> > I think it's really interesting that google seems to use this
> > > > >>> inefficient
> > > > >>> > small packet size for SSL, allows for 25MB emails, is a big
> > > > proponent
> > > > >>> of
> > > > >>> > SSL, and at the same time doesn't allow mails to take more than
> > 15
> > > > >>> minutes
> > > > >>> > to transfer. Now that you've made things >much< more efficient
> > on
> > > > the
> > > > >>> > ASSP
> > > > >>> > side, I'm hoping that all will be okay. I just get annoyed by
> > > > >>> > inefficiency.
> > > > >>> >
> > > > >>> >
> > > > >>> > I'm going to tryrunning with npSize of zero, the no queuing
> size
> > > set
> > > > >>> very
> > > > >>> > high and see how that goes. I want to insure that even the
> > > biggest
> > > > >>> emails
> > > > >>> > are scanned for malware before hitting the inbox.
> > > > >>> >
> > > > >>> >
> > > > >>> > I've never said this before, but it seems like google's doing
> it
> > > > wrong.
> > > > >>> I
> > > > >>> > think we've exhausted our options here. If anyone has a Google
> > > > >>> > engineering
> > > > >>> > friend, or a friend of a friend, this might be a good time to
> > have
> > > a
> > > > >>> > little
> > > > >>> > chat in an effort to reach someone who can either explain why
> > > > they're
> > > > >>> use
> > > > >>> > a
> > > > >>> > 1.4 kB frame size for ssl packets, but a normal much bigger
> size
> > > for
> > > > >>> > non-encrypted traffic or to get them to see an error in their
> > ways
> > > > and
> > > > >>> fix
> > > > >>> > this!
> > > > >>> >
> > > > >>> > Thank you again for all of the discussion and explanation and
> of
> > > > course
> > > > >>> > the
> > > > >>> > code changes which have made a huge difference.
> > > > >>> >
> > > > >>> > On Wed, Sep 28, 2016 at 12:59 PM, Thomas Eckardt
> > > > >>> > <thomas.ecka...@thockar.com
> > > > >>> > > wrote:
> > > > >>> >
> > > > >>> > > >I'm still afraid of running into frequent problems with 25mb
> > > > >>> > attachments
> > > > >>> > > though.
> > > > >>> > >
> > > > >>> > > 25MB - I don't know anyone who allows this. Sending a link to
> > a
> > > > >>> download
> > > > >>> > > is much smarter.
> > > > >>> > > ASSP_AFC has an option to do this for your outgoing mails! -
> > > > >>> > > ASSP_AFCWebScript
> > > > >>> > >
> > > > >>> > > >Can someone else run a debug test with Gmail and TLS to see
> > if
> > > > >>> you're
> > > > >>> > > seeing these tiny packets too?
> > > > >>> > >
> > > > >>> > > I've done it - same behavior - 1400 byte per SSL frame from
> > > gmail.
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > >Would it be an easy modification to show the negotiated
> > cipher
> > > in
> > > > >>> the
> > > > >>> > >
> > > > >>> > > ASSP does this for years now - simply look in to the received
> > > > header
> > > > >>> > line
> > > > >>> > > added by assp.
> > > > >>> > >
> > > > >>> > > -- -- -- Received: from vs10241.internet1.de
> ([158.181.49.77]
> > > > >>> > > helo=vs10241.internet1.de)
> > > > >>> > > by xxxxxx with SMTPS(TLSv1_2
> > > ECDHE-RSA-AES256-GCM-SHA384)
> > > > >>> > (2.5.2);
> > > > >>> > > 24 Jul 2016 05:34:12 +0200
> > > > >>> > >
> > > > >>> > > if you set 'ConnectionLog' at least to verbose, you'll see
> the
> > > > >>> > > SSL-negotiation results in maillog.txt.
> > > > >>> > >
> > > > >>> > > info: started TLS-SSL session for client $cliIP - using
> > > > >>> > > $Con{$ssls}->{sslversion} , $Con{$ssls}->{sslcipher}
> > > > >>> > >
> > > > >>> > > >what other options are there?
> > > > >>> > >
> > > > >>> > > Set 'EnableHighPerformance' to very high.
> > > > >>> > >
> > > > >>> > > >Are you sure that negotiating a lesser cipher with Gmail
> > > wouldn't
> > > > >>> have
> > > > >>> > > them switch to sending larger SSL packets?
> > > > >>> > >
> > > > >>> > > Yes, I'm 100% sure.
> > > > >>> > >
> > > > >>> > > An SSL peer is free to use any SSL frame size between 1 byte
> > and
> > > > >>> 16kB.
> > > > >>> > > There is no SSL-negotiation parameter for min or max frame
> > size.
> > > > >>> > > If gmail.com sends 1.4kB frames, they are free to do it this
> > > way.
> > > > >>> There
> > > > >>> > is
> > > > >>> > > no technical way to force them to send larger frames.
> > > > >>> > >
> > > > >>> > > >If neverQueue is set to 12MB, am I correct in saying that
> > we're
> > > > open
> > > > >>> to
> > > > >>> > a
> > > > >>> > > targeted exploit, say an .exe in a .zip as long as the email
> > is
> > > > more
> > > > >>> > than
> > > > >>> > > 12MB?
> > > > >>> > >
> > > > >>> > > Yes, you are correct.
> > > > >>> > >
> > > > >>> > > >or is that just after the whole message is received?
> > > > >>> > >
> > > > >>> > > Yes, after the complete mail is received.
> > > > >>> > >
> > > > >>> > > It is like it is - if the postman with his Ferrari is in a
> > > traffic
> > > > >>> jam,
> > > > >>> > > you'll have to wait longer for your letter or you'll get it
> > the
> > > > next
> > > > >>> > day.
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > Thomas
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > Von: K Post <nntp.p...@gmail.com>
> > > > >>> > > An: ASSP development mailing list
> > > > >>> <assp-test@lists.sourceforge.net>
> > > > >>> > > Datum: 28.09.2016 17:03
> > > > >>> > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> > > > >>> addresses /
> > > > >>> > > servers
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > Thanks again for the detailed explanation.
> > > > >>> > >
> > > > >>> > > 10% faster is terrific, and it's more than 50% faster since
> > > before
> > > > >>> > 16270.
> > > > >>> > > I'm still afraid of running into frequent problems with 25mb
> > > > >>> attachments
> > > > >>> > > though. Yes, that's too big for email in our opinion, but
> > Gmail
> > > > >>> allows
> > > > >>> > it
> > > > >>> > > as do most ISP's these days. It's not terribly unusual for
> us
> > > to
> > > > >>> have
> > > > >>> a
> > > > >>> > a
> > > > >>> > > person try to send and email with four or five 4 MB photos -
> > and
> > > > we
> > > > >>> > can't
> > > > >>> > > stop that trend. That'll translate into a message somewhere
> > in
> > > > the
> > > > >>> > range
> > > > >>> > > of 22-25 MB. I'm not sure if we're fast enough to
> > > >consistently<
> > > > >>> > receive
> > > > >>> > > these. And on top of that, if it's an outside big wig,
> > emailing
> > > > an
> > > > >>> > inside
> > > > >>> > > big wig, they don't have 15 minutes to waste waiting for the
> > big
> > > > >>> email
> > > > >>> > to
> > > > >>> > > arrive. Then they yell at me, which is why I keep asking
> more
> > > of
> > > > >>> these
> > > > >>> > > questions -- and there's a lot of them here (sorry)
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > *Are you sure that negotiating a lesser cipher with Gmail
> > > wouldn't
> > > > >>> have
> > > > >>> > > them switch to sending larger SSL packets? * I have a solid
> > high
> > > > >>> level
> > > > >>> > > understanding of encryption, but once you get down the the
> > > packet
> > > > >>> level,
> > > > >>> > > I'm a bit out of my depth, despite what I've learned from
> your
> > > > >>> excellent
> > > > >>> > > explanations.
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > Can someone else run a debug test with Gmail and TLS to see
> if
> > > > you're
> > > > >>> > > seeing these tiny packets too? I keep going back to *not
> > being
> > > > able
> > > > >>> to
> > > > >>> > > believe that Google doing something that was any slower than
> > > > >>> necessary.
> > > > >>> > *
> > > > >>> > > They
> > > > >>> > > try to optimize everything and are huge proponents of SSL
> > > > everywhere.
> > > > >>> If
> > > > >>> > > they're sending tons of tiny un-optimized packets, there's
> got
> > > to
> > > > be
> > > > >>> a
> > > > >>> > > reason or cause. I can't imagine that it's a default and
> > > > deliberate
> > > > >>> > > setting. I tried calling US support, but this is a network
> > > > >>> engineering
> > > > >>> > > type of issue - no chance of me getting through to someone
> > there
> > > > who
> > > > >>> > even
> > > > >>> > > understands the question.
> > > > >>> > >
> > > > >>> > > The npSize option and neverQueueSize, only affects things
> > after
> > > > the
> > > > >>> > whole
> > > > >>> > > message is received right? * It's not like it's doing
> > something
> > > > >>> > > significantly extra at each SSL cylce is it except using up
> > RAM
> > > > >>> right??*
> > > > >>> > > My unscientific tests are shower marginally slower speeds
> > when
> > > > >>> > receiving
> > > > >>> > > large email from Google with TLS on and neverQueueSize set
> > crazy
> > > > >>> high,
> > > > >>> > so
> > > > >>> > > maybe I'm wrong. Being that this same message through
> > > outlook.com
> > > > >>> only
> > > > >>> > > takes 30 seconds, even with npSize set to 0 (unlimited) and
> > > > >>> > neverQueueSize
> > > > >>> > > = 999,000,000, I'm wondering if the processing power and ram
> > > > >>> > availability
> > > > >>> > > (24 GB) on my assp box combined with low usage (only
> averaging
> > > > 5000
> > > > >>> > > messages a day) might be enough to have neverQueueSize really
> > > high
> > > > to
> > > > >>> > > always scan everything. I'll play around there, but if
> > > > >>> neverQueueSize
> > > > >>> > is
> > > > >>> > > 99MB, will things like ASSP_AFCDetectSpamAttachRe be slow
> > WHILE
> > > > the
> > > > >>> > > message
> > > > >>> > > is transferring, or is that just after the whole message is
> > > > received?
> > > > >>> > >
> > > > >>> > > If neverQueue is set to 12MB, am I correct in saying that
> > we're
> > > > open
> > > > >>> to
> > > > >>> > a
> > > > >>> > > targeted exploit, say an .exe in a .zip as long as the email
> > is
> > > > more
> > > > >>> > than
> > > > >>> > > 12MB? If so, that seems like a big hole, and something that
> I
> > > > >>> > personally
> > > > >>> > > would be willing to sacrifice speed/processing power to
> close.
> > > > >>> > >
> > > > >>> > > Suggestion, note in the gui under npSize that 0 really sets
> > this
> > > > to a
> > > > >>> > > maximum of 12,000,000 unless neverQueueSize is set higher in
> > > > >>> > > CorrectASSPcfg.pm. I've got that correct, right?)
> > > > >>> > >
> > > > >>> > > I forgot that DKIM signing can be done over the body too! We
> > > > don't
> > > > >>> do
> > > > >>> > > that, but I forgot that others might. Maybe we have npSize
> > only
> > > > >>> ignore
> > > > >>> > > DKIM if it's done over the body? DKIM should be fast if it's
> > > just
> > > > >>> the
> > > > >>> > > typical headers that are used right? We have npSize set to
> > > zero
> > > > >>> > though,
> > > > >>> > > so this doesn't really impact me.
> > > > >>> > >
> > > > >>> > > Would it be an easy modification to show the negotiated
> cipher
> > > in
> > > > the
> > > > >>> > > maillog or in the email body, if nothing else, just to have
> > more
> > > > >>> info?
> > > > >>> > > X-ASSP-Cipher: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384
> > > > >>> > > for example or in the received line like Microsoft does:
> > > > >>> > >
> > > > >>> > > Received: from __________ (x.x.x.x) by ______
> > > > >>> > > (y.y.y.y) with Microsoft SMTP Server (version=TLS1_2,
> > > > >>> > > cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384)
> > > > >>> > >
> > > > >>> > > Or like Gmail does
> > > > >>> > > Received: from smtp.ourcharity.org (smtp.ourcharity.org.
> > > > [x.x.x.x])
> > > > >>> > > by mx.google.com with ESMTPS id
> > > > >>> > > l1si54144332.11.2016.09.28.07.51.21
> > > > >>> > > for <nntp.p...@gmail.com>
> > > > >>> > > (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256
> > > > >>> > bits=128/128);
> > > > >>> > > Wed, 28 Sep 2016 07:24:22 -0700 (PDT)
> > > > >>> > >
> > > > >>> > > It's a little more data (and certainly your time to code
> > this),
> > > > but
> > > > >>> > could
> > > > >>> > > be useful if we need it in the future.
> > > > >>> > >
> > > > >>> > > and::
> > > > >>> > >
> > > > >>> > > Again Ken, the SSL parameters are NOT the problem on your
> > > system.
> > > > >>> Your
> > > > >>> > >
> > > > >>> > > debug log shows a socket read time of max ~ 0.5 milliseconds
> > > > (typical
> > > > >>> > ~0.3
> > > > >>> > >
> > > > >>> > > ms) for SSL. This read operation includes the time required
> > for
> > > > the
> > > > >>> > >
> > > > >>> > > decryption of the data. This is very very fast!
> > > > >>> > >
> > > > >>> > > It is simply the amazing count (10.400) of read and process
> > > > >>> operations
> > > > >>> > >
> > > > >>> > > (cycles) required by assp for such a mail, that causes the
> > > overall
> > > > >>> slow
> > > > >>> > >
> > > > >>> > > mail receive.
> > > > >>> > >
> > > > >>> > > in summary, if there's not an ASSP setting that I can tweak
> to
> > > > have
> > > > >>> > google
> > > > >>> > > start sending larger SSL packets like Outlook.com and
> everyone
> > > > else
> > > > >>> I've
> > > > >>> > > seen does, what other options are there? Are you suggesting
> > > that
> > > > ALL
> > > > >>> > ASSP
> > > > >>> > > installations are incapable of receiving large TLS emails
> from
> > > > Google
> > > > >>> > > regardless of the installation's processing power, RAM, and
> > > > >>> bandwidth?
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Wed, Sep 28, 2016 at 4:26 AM, Thomas Eckardt
> > > > >>> > > <thomas.ecka...@thockar.com>
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > > > >The same email now took only 269 seconds.
> > > > >>> > > >
> > > > >>> > > > This is OK for googles behavior - the 1440 byte SSL frame.
> > > > Around
> > > > >>> 300s
> > > > >>> > > was
> > > > >>> > > > expected by me for this mail - hmm.. it is 10% better -
> > nice!
> > > > >>> > > >
> > > > >>> > > > Take the following math.
> > > > >>> > > >
> > > > >>> > > > mail size = 15000 kB
> > > > >>> > > > google frame size = 1.44 kB
> > > > >>> > > >
> > > > >>> > > > required assp loop cycles = mail size / frame size = 10.400
> > > > >>> > > >
> > > > >>> > > > mail size = 15000 kB
> > > > >>> > > > outlook.com frame size = 16 kB
> > > > >>> > > >
> > > > >>> > > > required assp loop cycles = mail size / frame size = 940
> > > > >>> > > >
> > > > >>> > > > >I really wonder if I need to tweak my cipher_list.
> > > > >>> > > >
> > > > >>> > > > No!
> > > > >>> > > > Again Ken, the SSL parameters are NOT the problem on your
> > > > system.
> > > > >>> Your
> > > > >>> > > > debug log shows a socket read time of max ~ 0.5
> milliseconds
> > > > >>> (typical
> > > > >>> > > ~0.3
> > > > >>> > > > ms) for SSL. This read operation includes the time required
> > > for
> > > > the
> > > > >>> > > > decryption of the data. This is very very fast!
> > > > >>> > > > It is simply the amazing count (10.400) of read and process
> > > > >>> operations
> > > > >>> > > > (cycles) required by assp for such a mail, that causes the
> > > > overall
> > > > >>> > slow
> > > > >>> > > > mail receive.
> > > > >>> > > >
> > > > >>> > > > >Is ClamAV scanning skipped too?
> > > > >>> > > > Yes.
> > > > >>> > > >
> > > > >>> > > > >Could the plugins be run on the full mail after receipt,
> > > > >>> regardless
> > > > >>> > of
> > > > >>> > > > size?
> > > > >>> > > >
> > > > >>> > > > Override the config parameters. Keep in mind that 'npSize'
> > may
> > > > also
> > > > >>> > > > involved in skipping or processing some mail body checks.
> > > > >>> > > >
> > > > >>> > > > >Isn't DKIM checking just a one time thing and not
> > intensive?
> > > > >>> > > >
> > > > >>> > > > The full DKIM check is very intensive. It requires to
> > > calculate
> > > > an
> > > > >>> > > RSA/SHA
> > > > >>> > > > hash over the complete mail.
> > > > >>> > > > DCC and Razor are doing something similar.
> > > > >>> > > > ASSP_AFC would make all checks (ClamAV, FileScan, content
> > > checks
> > > > >>> with
> > > > >>> > > > several regular expression, decompression of attachments
> > ....)
> > > > for
> > > > >>> the
> > > > >>> > > > complete mail. It parses the complete mail at once with
> > > > >>> Email::MIME.
> > > > >>> > > This
> > > > >>> > > > requires a huge amount of memory. Not a big deal on a 64bit
> > OS
> > > > with
> > > > >>> > 8GB
> > > > >>> > > > RAM and several CPU cores - who can !?
> > > > >>> > > > All these can be done for any mail size, if the system is
> > able
> > > > to
> > > > >>> > > process
> > > > >>> > > > the amount of data fast enough.
> > > > >>> > > > On most havy load systems, the 12.000.000 will be to large
> > and
> > > > may
> > > > >>> > lead
> > > > >>> > > in
> > > > >>> > > > to stucking workers.
> > > > >>> > > >
> > > > >>> > > > ASSP has this set of config parameters - change them to
> your
> > > > need -
> > > > >>> > try
> > > > >>> > > -
> > > > >>> > > > and if does not work switch back - nothing more easy.
> > > > >>> > > >
> > > > >>> > > > Thomas
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > Von: K Post <nntp.p...@gmail.com>
> > > > >>> > > > An: ASSP development mailing list
> > > > >>> > <assp-test@lists.sourceforge.net>
> > > > >>> > > > Datum: 27.09.2016 21:08
> > > > >>> > > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com
> > > > >>> addresses
> > > > >>> /
> > > > >>> > > > servers
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > Our primary internet connection went down again (nothing to
> > do
> > > > with
> > > > >>> > > ASSP)
> > > > >>> > > > which gave me the opportunity to replace 16270 with 16271.
> > > > Nothing
> > > > >>> > like
> > > > >>> > > > making lemonade out of lemons...
> > > > >>> > > >
> > > > >>> > > > The same email now took only 269 seconds. That's about 15x
> > > > longer
> > > > >>> > than
> > > > >>> > > > with TLS off, but WOW that's way better than it was before.
> > > > >>> > > > I also tried with the blank cipher list, no notable
> > difference
> > > > >>> > > > And with the SSL buffers set to 0 (64 MB), again without a
> > > speed
> > > > >>> > > > difference.
> > > > >>> > > >
> > > > >>> > > > SO- you've made a real difference here!! Is there more
> > > > >>> optimization
> > > > >>> > to
> > > > >>> > > > be
> > > > >>> > > > made?
> > > > >>> > > >
> > > > >>> > > > The rub is that the exact same message sent through
> > > Outlook.com
> > > > to
> > > > >>> us,
> > > > >>> > > > took
> > > > >>> > > > exactly 30 seconds, just a 50% overhead when compared to
> the
> > > 19
> > > > >>> > seconds
> > > > >>> > > > for
> > > > >>> > > > a non-TLS message of the same size, instead of a 1500%
> > > overhead
> > > > for
> > > > >>> > > > encryption when receiving from google.
> > > > >>> > > >
> > > > >>> > > > *Is there some magical debug switch that I could turn on to
> > > see
> > > > >>> what
> > > > >>> > > > encryption Outlook.com is using and compare that to what
> > > > Google's
> > > > >>> > > > connection to us with?* I think prohibiting whatever the
> > slow
> > > > >>> cipher
> > > > >>> > > that
> > > > >>> > > > google's using (probably a really strong one) might make
> the
> > > > final
> > > > >>> bit
> > > > >>> > > of
> > > > >>> > > > difference.
> > > > >>> > > >
> > > > >>> > > > I'm breathing so much easier now!! Thank you.
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > On Tue, Sep 27, 2016 at 2:08 PM, K Post
> > <nntp.p...@gmail.com>
> > > > >>> wrote:
> > > > >>> > > >
> > > > >>> > > > > Despite all the problems we have with personalities and
> > > > policies
> > > > >>> in
> > > > >>> > > our
> > > > >>> > > > > organization, the infrastructure is pretty solid. MTU's
> > are
> > > > set
> > > > >>> > > > correctly,
> > > > >>> > > > > no fragmentation, no jitter. There's low latency across
> > > the
> > > > >>> board,
> > > > >>> > > and
> > > > >>> > > > > really low bandwidth usage. If we sent 1000 mails a day,
> > > it's
> > > > a
> > > > >>> > lot.
> > > > >>> > > > >
> > > > >>> > > > > ...and yes, they even pay their bills, though not me very
> > > well
> > > > :)
> > > > >>> > > > >
> > > > >>> > > > > I think it's which SSL algorithm is being used that's at
> > > least
> > > > >>> > > partially
> > > > >>> > > > > to blame. I have:
> > > > >>> > > > > SSL_Version: SSLv23:!SSLv3:!SSLv2
> > > > >>> > > > > SSL_Cipher_list:
> > kEECDH+ECDSA:kEECDH:kEDH:HIGH:+SHA:+RC4:RC4
> > > > >>> > > > >
> > :!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!DSS:!PSK:!SRP:!kECDH:!
> > > > >>> > > > > CAMELLIA128:!IDEA:!SEED
> > > > >>> > > > >
> > > > >>> > > > > I tried the default wih SSL_Cipher_List blank before, I
> > > don't
> > > > >>> think
> > > > >>> > > > there
> > > > >>> > > > > was a difference (but I've played with so many settings,
> I
> > > > really
> > > > >>> > > don't
> > > > >>> > > > > remember)
> > > > >>> > > > >
> > > > >>> > > > > And last, on the SSL buffer size. If set to zero in the
> > > gui,
> > > > on
> > > > >>> > > windows
> > > > >>> > > > > 2012, it says in green that it's set to 64 MB. I follow
> > > what
> > > > >>> you're
> > > > >>> > > > saying
> > > > >>> > > > > about it readying 4x 16 Kb without a loop cycle. Is that
> > a
> > > > good
> > > > >>> or
> > > > >>> > > bad
> > > > >>> > > > > thing though?
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > ------------------------------
> ------------------------------
> > > > >>> > > > ------------------
> > > > >>> > > > _______________________________________________
> > > > >>> > > > Assp-test mailing list
> > > > >>> > > > Assp-test@lists.sourceforge.net
> > > > >>> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > DISCLAIMER:
> > > > >>> > > > *******************************************************
> > > > >>> > > > This email and any files transmitted with it may be
> > > > confidential,
> > > > >>> > > legally
> > > > >>> > > > privileged and protected in law and are intended solely for
> > > the
> > > > use
> > > > >>> of
> > > > >>> > > the
> > > > >>> > > >
> > > > >>> > > > individual to whom it is addressed.
> > > > >>> > > > This email was multiple times scanned for viruses. There
> > > should
> > > > be
> > > > >>> no
> > > > >>> > > > known virus in this email!
> > > > >>> > > > *******************************************************
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > ------------------------------
> ------------------------------
> > > > >>> > > > ------------------
> > > > >>> > > >
> > > > >>> > > > _______________________________________________
> > > > >>> > > > Assp-test mailing list
> > > > >>> > > > Assp-test@lists.sourceforge.net
> > > > >>> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > ------------------------------------------------------------
> > > > >>> > > ------------------
> > > > >>> > > _______________________________________________
> > > > >>> > > Assp-test mailing list
> > > > >>> > > Assp-test@lists.sourceforge.net
> > > > >>> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > DISCLAIMER:
> > > > >>> > > *******************************************************
> > > > >>> > > This email and any files transmitted with it may be
> > > confidential,
> > > > >>> > legally
> > > > >>> > > privileged and protected in law and are intended solely for
> > the
> > > > use
> > > > >>> of
> > > > >>> > the
> > > > >>> > >
> > > > >>> > > individual to whom it is addressed.
> > > > >>> > > This email was multiple times scanned for viruses. There
> > should
> > > be
> > > > no
> > > > >>> > > known virus in this email!
> > > > >>> > > *******************************************************
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > ------------------------------------------------------------
> > > > >>> > > ------------------
> > > > >>> > >
> > > > >>> > > _______________________________________________
> > > > >>> > > Assp-test mailing list
> > > > >>> > > Assp-test@lists.sourceforge.net
> > > > >>> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>> > >
> > > > >>> > >
> > > > >>> > ------------------------------------------------------------
> > > > >>> > ------------------
> > > > >>> > _______________________________________________
> > > > >>> > Assp-test mailing list
> > > > >>> > Assp-test@lists.sourceforge.net
> > > > >>> > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > DISCLAIMER:
> > > > >>> > *******************************************************
> > > > >>> > This email and any files transmitted with it may be
> > confidential,
> > > > >>> legally
> > > > >>> > privileged and protected in law and are intended solely for the
> > > use
> > > > of
> > > > >>> the
> > > > >>> >
> > > > >>> > individual to whom it is addressed.
> > > > >>> > This email was multiple times scanned for viruses. There should
> > be
> > > > no
> > > > >>> > known virus in this email!
> > > > >>> > *******************************************************
> > > > >>> >
> > > > >>> >
> > > > >>> > ------------------------------------------------------------
> > > > >>> > ------------------
> > > > >>> >
> > > > >>> > _______________________________________________
> > > > >>> > Assp-test mailing list
> > > > >>> > Assp-test@lists.sourceforge.net
> > > > >>> > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>> >
> > > > >>> >
> > > > >>> ------------------------------------------------------------
> > > > >>> ------------------
> > > > >>> _______________________________________________
> > > > >>> Assp-test mailing list
> > > > >>> Assp-test@lists.sourceforge.net
> > > > >>> https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> DISCLAIMER:
> > > > >>> *******************************************************
> > > > >>> This email and any files transmitted with it may be confidential,
> > > > legally
> > > > >>> privileged and protected in law and are intended solely for the
> > use
> > > of
> > > > >>> the
> > > > >>>
> > > > >>> individual to whom it is addressed.
> > > > >>> This email was multiple times scanned for viruses. There should
> be
> > > no
> > > > >>> known virus in this email!
> > > > >>> *******************************************************
> > > > >>>
> > > > >>>
> > > > >>> ------------------------------------------------------------
> > > > >>> ------------------
> > > > >>>
> > > > >>> _______________________________________________
> > > > >>> Assp-test mailing list
> > > > >>> Assp-test@lists.sourceforge.net
> > > > >>> https://lists.sourceforge.net/lists/listinfo/assp-test
> > > > >>>
> > > > >>>
> > > > >>
> > > > >
> > > > ------------------------------------------------------------
> > > > ------------------
> > > > _______________________________________________
> > > > Assp-test mailing list
> > > > Assp-test@lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >
> > > >
> > > >
> > > >
> > > > DISCLAIMER:
> > > > *******************************************************
> > > > This email and any files transmitted with it may be confidential,
> > > legally
> > > > privileged and protected in law and are intended solely for the use
> of
> > > the
> > > >
> > > > individual to whom it is addressed.
> > > > This email was multiple times scanned for viruses. There should be no
> > > > known virus in this email!
> > > > *******************************************************
> > > >
> > > >
> > > >
> > > > ------------------------------------------------------------
> > > > ------------------
> > > > Check out the vibrant tech community on one of the world's most
> > > > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > > > _______________________________________________
> > > > Assp-test mailing list
> > > > Assp-test@lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > > >
> > > >
> > > ------------------------------------------------------------
> > > ------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > > _______________________________________________
> > > Assp-test mailing list
> > > Assp-test@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > >
> > >
> > >
> > >
> > > DISCLAIMER:
> > > *******************************************************
> > > This email and any files transmitted with it may be confidential,
> > legally
> > > privileged and protected in law and are intended solely for the use of
> > the
> > >
> > > individual to whom it is addressed.
> > > This email was multiple times scanned for viruses. There should be no
> > > known virus in this email!
> > > *******************************************************
> > >
> > >
> > > ------------------------------------------------------------
> > > ------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > > _______________________________________________
> > > Assp-test mailing list
> > > Assp-test@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/assp-test
> > >
> > >
> > ------------------------------------------------------------
> > ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Assp-test mailing list
> > Assp-test@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/assp-test
> >
> >
> >
> >
> > DISCLAIMER:
> > *******************************************************
> > This email and any files transmitted with it may be confidential, legally
> > privileged and protected in law and are intended solely for the use of
> the
> >
> > individual to whom it is addressed.
> > This email was multiple times scanned for viruses. There should be no
> > known virus in this email!
> > *******************************************************
> >
> >
> > ------------------------------------------------------------
> > ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Assp-test mailing list
> > Assp-test@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/assp-test
> >
> >
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
