Thanks again for the detailed explanation.
10% faster is terrific, and it's more than 50% faster since before 16270.
I'm still afraid of running into frequent problems with 25mb attachments
though. Yes, that's too big for email in our opinion, but Gmail allows it
as do most ISP's these days. It's not terribly unusual for us to have a a
person try to send and email with four or five 4 MB photos - and we can't
stop that trend. That'll translate into a message somewhere in the range
of 22-25 MB. I'm not sure if we're fast enough to >consistently< receive
these. And on top of that, if it's an outside big wig, emailing an inside
big wig, they don't have 15 minutes to waste waiting for the big email to
arrive. Then they yell at me, which is why I keep asking more of these
questions -- and there's a lot of them here (sorry)
*Are you sure that negotiating a lesser cipher with Gmail wouldn't have
them switch to sending larger SSL packets? * I have a solid high level
understanding of encryption, but once you get down the the packet level,
I'm a bit out of my depth, despite what I've learned from your excellent
explanations.
Can someone else run a debug test with Gmail and TLS to see if you're
seeing these tiny packets too? I keep going back to *not being able to
believe that Google doing something that was any slower than necessary. * They
try to optimize everything and are huge proponents of SSL everywhere. If
they're sending tons of tiny un-optimized packets, there's got to be a
reason or cause. I can't imagine that it's a default and deliberate
setting. I tried calling US support, but this is a network engineering
type of issue - no chance of me getting through to someone there who even
understands the question.
The npSize option and neverQueueSize, only affects things after the whole
message is received right? * It's not like it's doing something
significantly extra at each SSL cylce is it except using up RAM right??*
My unscientific tests are shower marginally slower speeds when receiving
large email from Google with TLS on and neverQueueSize set crazy high, so
maybe I'm wrong. Being that this same message through outlook.com only
takes 30 seconds, even with npSize set to 0 (unlimited) and neverQueueSize
= 999,000,000, I'm wondering if the processing power and ram availability
(24 GB) on my assp box combined with low usage (only averaging 5000
messages a day) might be enough to have neverQueueSize really high to
always scan everything. I'll play around there, but if neverQueueSize is
99MB, will things like ASSP_AFCDetectSpamAttachRe be slow WHILE the message
is transferring, or is that just after the whole message is received?
If neverQueue is set to 12MB, am I correct in saying that we're open to a
targeted exploit, say an .exe in a .zip as long as the email is more than
12MB? If so, that seems like a big hole, and something that I personally
would be willing to sacrifice speed/processing power to close.
Suggestion, note in the gui under npSize that 0 really sets this to a
maximum of 12,000,000 unless neverQueueSize is set higher in
CorrectASSPcfg.pm. I've got that correct, right?)
I forgot that DKIM signing can be done over the body too! We don't do
that, but I forgot that others might. Maybe we have npSize only ignore
DKIM if it's done over the body? DKIM should be fast if it's just the
typical headers that are used right? We have npSize set to zero though,
so this doesn't really impact me.
Would it be an easy modification to show the negotiated cipher in the
maillog or in the email body, if nothing else, just to have more info?
X-ASSP-Cipher: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384
for example or in the received line like Microsoft does:
Received: from __________ (x.x.x.x) by ______
(y.y.y.y) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384)
Or like Gmail does
Received: from smtp.ourcharity.org (smtp.ourcharity.org. [x.x.x.x])
by mx.google.com with ESMTPS id l1si54144332.11.2016.09.28.07.51.21
for <nntp.p...@gmail.com>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Wed, 28 Sep 2016 07:24:22 -0700 (PDT)
It's a little more data (and certainly your time to code this), but could
be useful if we need it in the future.
and::
Again Ken, the SSL parameters are NOT the problem on your system. Your
debug log shows a socket read time of max ~ 0.5 milliseconds (typical ~0.3
ms) for SSL. This read operation includes the time required for the
decryption of the data. This is very very fast!
It is simply the amazing count (10.400) of read and process operations
(cycles) required by assp for such a mail, that causes the overall slow
mail receive.
in summary, if there's not an ASSP setting that I can tweak to have google
start sending larger SSL packets like Outlook.com and everyone else I've
seen does, what other options are there? Are you suggesting that ALL ASSP
installations are incapable of receiving large TLS emails from Google
regardless of the installation's processing power, RAM, and bandwidth?
On Wed, Sep 28, 2016 at 4:26 AM, Thomas Eckardt <thomas.ecka...@thockar.com>
wrote:
> >The same email now took only 269 seconds.
>
> This is OK for googles behavior - the 1440 byte SSL frame. Around 300s was
> expected by me for this mail - hmm.. it is 10% better - nice!
>
> Take the following math.
>
> mail size = 15000 kB
> google frame size = 1.44 kB
>
> required assp loop cycles = mail size / frame size = 10.400
>
> mail size = 15000 kB
> outlook.com frame size = 16 kB
>
> required assp loop cycles = mail size / frame size = 940
>
> >I really wonder if I need to tweak my cipher_list.
>
> No!
> Again Ken, the SSL parameters are NOT the problem on your system. Your
> debug log shows a socket read time of max ~ 0.5 milliseconds (typical ~0.3
> ms) for SSL. This read operation includes the time required for the
> decryption of the data. This is very very fast!
> It is simply the amazing count (10.400) of read and process operations
> (cycles) required by assp for such a mail, that causes the overall slow
> mail receive.
>
> >Is ClamAV scanning skipped too?
> Yes.
>
> >Could the plugins be run on the full mail after receipt, regardless of
> size?
>
> Override the config parameters. Keep in mind that 'npSize' may also
> involved in skipping or processing some mail body checks.
>
> >Isn't DKIM checking just a one time thing and not intensive?
>
> The full DKIM check is very intensive. It requires to calculate an RSA/SHA
> hash over the complete mail.
> DCC and Razor are doing something similar.
> ASSP_AFC would make all checks (ClamAV, FileScan, content checks with
> several regular expression, decompression of attachments ....) for the
> complete mail. It parses the complete mail at once with Email::MIME. This
> requires a huge amount of memory. Not a big deal on a 64bit OS with 8GB
> RAM and several CPU cores - who can !?
> All these can be done for any mail size, if the system is able to process
> the amount of data fast enough.
> On most havy load systems, the 12.000.000 will be to large and may lead in
> to stucking workers.
>
> ASSP has this set of config parameters - change them to your need - try -
> and if does not work switch back - nothing more easy.
>
> Thomas
>
>
>
> Von: K Post <nntp.p...@gmail.com>
> An: ASSP development mailing list <assp-test@lists.sourceforge.net>
> Datum: 27.09.2016 21:08
> Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses /
> servers
>
>
>
> Our primary internet connection went down again (nothing to do with ASSP)
> which gave me the opportunity to replace 16270 with 16271. Nothing like
> making lemonade out of lemons...
>
> The same email now took only 269 seconds. That's about 15x longer than
> with TLS off, but WOW that's way better than it was before.
> I also tried with the blank cipher list, no notable difference
> And with the SSL buffers set to 0 (64 MB), again without a speed
> difference.
>
> SO- you've made a real difference here!! Is there more optimization to
> be
> made?
>
> The rub is that the exact same message sent through Outlook.com to us,
> took
> exactly 30 seconds, just a 50% overhead when compared to the 19 seconds
> for
> a non-TLS message of the same size, instead of a 1500% overhead for
> encryption when receiving from google.
>
> *Is there some magical debug switch that I could turn on to see what
> encryption Outlook.com is using and compare that to what Google's
> connection to us with?* I think prohibiting whatever the slow cipher that
> google's using (probably a really strong one) might make the final bit of
> difference.
>
> I'm breathing so much easier now!! Thank you.
>
>
> On Tue, Sep 27, 2016 at 2:08 PM, K Post <nntp.p...@gmail.com> wrote:
>
> > Despite all the problems we have with personalities and policies in our
> > organization, the infrastructure is pretty solid. MTU's are set
> correctly,
> > no fragmentation, no jitter. There's low latency across the board, and
> > really low bandwidth usage. If we sent 1000 mails a day, it's a lot.
> >
> > ...and yes, they even pay their bills, though not me very well :)
> >
> > I think it's which SSL algorithm is being used that's at least partially
> > to blame. I have:
> > SSL_Version: SSLv23:!SSLv3:!SSLv2
> > SSL_Cipher_list: kEECDH+ECDSA:kEECDH:kEDH:HIGH:+SHA:+RC4:RC4
> > :!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!DSS:!PSK:!SRP:!kECDH:!
> > CAMELLIA128:!IDEA:!SEED
> >
> > I tried the default wih SSL_Cipher_List blank before, I don't think
> there
> > was a difference (but I've played with so many settings, I really don't
> > remember)
> >
> > And last, on the SSL buffer size. If set to zero in the gui, on windows
> > 2012, it says in green that it's set to 64 MB. I follow what you're
> saying
> > about it readying 4x 16 Kb without a loop cycle. Is that a good or bad
> > thing though?
> >
> >
> >
> ------------------------------------------------------------
> ------------------
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
>
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
> ------------------------------------------------------------
> ------------------
>
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
------------------------------------------------------------------------------
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
