I've had to roll back now unfortunately as I'm getting email problems again :(
On Fri, Sep 30, 2016 at 3:50 PM, cw <colin.war...@gmail.com> wrote: > Mixed results on this. So far no problems with running workers being > logged but the GUI has become incredibly unresponsive. By unresponsive I > mean I waited a good couple of minutes for the shutdown_list page to load. > The dot on the main page is red yet the workers page is all green. > Scratch that, it has refreshed again and I have a worker stuck: > Worker 3, loop age 252s, action: header (Content-Disposition -attr) : : > filename name (stuck) > 30s later and it is healthy again.. > > On the server I haven't upgraded the shutdown_list page comes up within > seconds. I'm not sure whether to leave it running or whether this is > evidence of the same kind of unresponsiveness that cause me to have to roll > back earlier this week. > > On Fri, Sep 30, 2016 at 3:29 PM, cw <colin.war...@gmail.com> wrote: > >> I wish I'd spotted this before writing out the other message. I'll give >> it a test now for you. >> >> On Fri, Sep 30, 2016 at 2:17 PM, Thomas Eckardt < >> thomas.ecka...@thockar.com> wrote: >> >>> Collin, this should no longer happen using the updated 2.5.2 16274_1 at >>> CVS /test >>> >>> Thomas >>> >>> >>> >>> Von: cw <colin.war...@gmail.com> >>> An: ASSP development mailing list <assp-test@lists.sourceforge.net> >>> Datum: 29.09.2016 16:40 >>> Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses / >>> servers >>> >>> >>> >>> Hi Thomas, >>> I moved up to 16270 following this thread of discussion but then had a >>> day >>> working away. I've come back to huge issues with delays, mails not going >>> through and many, many of these in the logs: >>> >>> Info: unable to detect any running worker for a new connection - wait >>> (max >>> 30 seconds) >>> >>> When I say many, I have over 21,000 lines in today's log file. I also >>> found >>> the GUI unresponsive or not connecting at all and ASSP restarting quite >>> regularly. >>> >>> I've dropped back to 16256 and things are instantly better. Do you think >>> going up to 16273 might improve things over 16270 or am I better holding >>> off for now? >>> All the best, >>> Colin. >>> >>> On Thu, Sep 29, 2016 at 3:15 PM, Thomas Eckardt >>> <thomas.ecka...@thockar.com> >>> wrote: >>> >>> > I just released 2.5.2 build 16273 at CVS test folder >>> > >>> > http://assp.cvs.sourceforge.net/viewvc/assp/assp2/test/ >>> > >>> > This release should make a very large difference for SSL/TLS mails sent >>> by >>> > hosts that uses small SSL-frame size. >>> > >>> > Tell me your test results. >>> > >>> > >>> > Thomas >>> > >>> > >>> > >>> > >>> > >>> > Von: K Post <nntp.p...@gmail.com> >>> > An: ASSP development mailing list <assp-test@lists.sourceforge.net >>> > >>> > Datum: 28.09.2016 19:42 >>> > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses / >>> > servers >>> > >>> > >>> > >>> > But I want a postman driving a Ferarri with monster truck tires that >>> can >>> > roll over the traffic (and if wishes are being granted, I'd prefer the >>> car >>> > in a deep blue instead of classic red). >>> > >>> > We regularly see people attaching large files or a bunch of smaller >>> ones >>> > that add up to a big email, I'm talking lots and lots of different >>> people >>> > from outside the organization sending to us, and this happens on a >>> daily >>> > basis. It's especially popular with photos and huge scans multi-page >>> > 600dpi (which people don't understand can be done at low resolution). >>> > Often it's people sending in scanned official documents for us to >>> review >>> > an >>> > help them. They're not our staff, they're the people we help. They >>> have >>> > a >>> > tendency of not following any instructions, and ignore the fact that we >>> > have a web based system for the process. We can't control it and the >>> > powers that be don't want us lowering the 30 MB threshold across the >>> > board. Lot of these people use gmail.com addresses and google allows >>> for >>> > up to 25 MB - https://support.google.com/mail/answer/6584 >>> > >>> > I think it's really interesting that google seems to use this >>> inefficient >>> > small packet size for SSL, allows for 25MB emails, is a big proponent >>> of >>> > SSL, and at the same time doesn't allow mails to take more than 15 >>> minutes >>> > to transfer. Now that you've made things >much< more efficient on the >>> > ASSP >>> > side, I'm hoping that all will be okay. I just get annoyed by >>> > inefficiency. >>> > >>> > >>> > I'm going to tryrunning with npSize of zero, the no queuing size set >>> very >>> > high and see how that goes. I want to insure that even the biggest >>> emails >>> > are scanned for malware before hitting the inbox. >>> > >>> > >>> > I've never said this before, but it seems like google's doing it wrong. >>> I >>> > think we've exhausted our options here. If anyone has a Google >>> > engineering >>> > friend, or a friend of a friend, this might be a good time to have a >>> > little >>> > chat in an effort to reach someone who can either explain why they're >>> use >>> > a >>> > 1.4 kB frame size for ssl packets, but a normal much bigger size for >>> > non-encrypted traffic or to get them to see an error in their ways and >>> fix >>> > this! >>> > >>> > Thank you again for all of the discussion and explanation and of course >>> > the >>> > code changes which have made a huge difference. >>> > >>> > On Wed, Sep 28, 2016 at 12:59 PM, Thomas Eckardt >>> > <thomas.ecka...@thockar.com >>> > > wrote: >>> > >>> > > >I'm still afraid of running into frequent problems with 25mb >>> > attachments >>> > > though. >>> > > >>> > > 25MB - I don't know anyone who allows this. Sending a link to a >>> download >>> > > is much smarter. >>> > > ASSP_AFC has an option to do this for your outgoing mails! - >>> > > ASSP_AFCWebScript >>> > > >>> > > >Can someone else run a debug test with Gmail and TLS to see if >>> you're >>> > > seeing these tiny packets too? >>> > > >>> > > I've done it - same behavior - 1400 byte per SSL frame from gmail. >>> > > >>> > > >>> > > >Would it be an easy modification to show the negotiated cipher in >>> the >>> > > >>> > > ASSP does this for years now - simply look in to the received header >>> > line >>> > > added by assp. >>> > > >>> > > -- -- -- Received: from vs10241.internet1.de ([158.181.49.77] >>> > > helo=vs10241.internet1.de) >>> > > by xxxxxx with SMTPS(TLSv1_2 ECDHE-RSA-AES256-GCM-SHA384) >>> > (2.5.2); >>> > > 24 Jul 2016 05:34:12 +0200 >>> > > >>> > > if you set 'ConnectionLog' at least to verbose, you'll see the >>> > > SSL-negotiation results in maillog.txt. >>> > > >>> > > info: started TLS-SSL session for client $cliIP - using >>> > > $Con{$ssls}->{sslversion} , $Con{$ssls}->{sslcipher} >>> > > >>> > > >what other options are there? >>> > > >>> > > Set 'EnableHighPerformance' to very high. >>> > > >>> > > >Are you sure that negotiating a lesser cipher with Gmail wouldn't >>> have >>> > > them switch to sending larger SSL packets? >>> > > >>> > > Yes, I'm 100% sure. >>> > > >>> > > An SSL peer is free to use any SSL frame size between 1 byte and >>> 16kB. >>> > > There is no SSL-negotiation parameter for min or max frame size. >>> > > If gmail.com sends 1.4kB frames, they are free to do it this way. >>> There >>> > is >>> > > no technical way to force them to send larger frames. >>> > > >>> > > >If neverQueue is set to 12MB, am I correct in saying that we're open >>> to >>> > a >>> > > targeted exploit, say an .exe in a .zip as long as the email is more >>> > than >>> > > 12MB? >>> > > >>> > > Yes, you are correct. >>> > > >>> > > >or is that just after the whole message is received? >>> > > >>> > > Yes, after the complete mail is received. >>> > > >>> > > It is like it is - if the postman with his Ferrari is in a traffic >>> jam, >>> > > you'll have to wait longer for your letter or you'll get it the next >>> > day. >>> > > >>> > > >>> > > Thomas >>> > > >>> > > >>> > > >>> > > Von: K Post <nntp.p...@gmail.com> >>> > > An: ASSP development mailing list >>> <assp-test@lists.sourceforge.net> >>> > > Datum: 28.09.2016 17:03 >>> > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com >>> addresses / >>> > > servers >>> > > >>> > > >>> > > >>> > > Thanks again for the detailed explanation. >>> > > >>> > > 10% faster is terrific, and it's more than 50% faster since before >>> > 16270. >>> > > I'm still afraid of running into frequent problems with 25mb >>> attachments >>> > > though. Yes, that's too big for email in our opinion, but Gmail >>> allows >>> > it >>> > > as do most ISP's these days. It's not terribly unusual for us to >>> have >>> a >>> > a >>> > > person try to send and email with four or five 4 MB photos - and we >>> > can't >>> > > stop that trend. That'll translate into a message somewhere in the >>> > range >>> > > of 22-25 MB. I'm not sure if we're fast enough to >consistently< >>> > receive >>> > > these. And on top of that, if it's an outside big wig, emailing an >>> > inside >>> > > big wig, they don't have 15 minutes to waste waiting for the big >>> email >>> > to >>> > > arrive. Then they yell at me, which is why I keep asking more of >>> these >>> > > questions -- and there's a lot of them here (sorry) >>> > > >>> > > >>> > > *Are you sure that negotiating a lesser cipher with Gmail wouldn't >>> have >>> > > them switch to sending larger SSL packets? * I have a solid high >>> level >>> > > understanding of encryption, but once you get down the the packet >>> level, >>> > > I'm a bit out of my depth, despite what I've learned from your >>> excellent >>> > > explanations. >>> > > >>> > > >>> > > Can someone else run a debug test with Gmail and TLS to see if you're >>> > > seeing these tiny packets too? I keep going back to *not being able >>> to >>> > > believe that Google doing something that was any slower than >>> necessary. >>> > * >>> > > They >>> > > try to optimize everything and are huge proponents of SSL everywhere. >>> If >>> > > they're sending tons of tiny un-optimized packets, there's got to be >>> a >>> > > reason or cause. I can't imagine that it's a default and deliberate >>> > > setting. I tried calling US support, but this is a network >>> engineering >>> > > type of issue - no chance of me getting through to someone there who >>> > even >>> > > understands the question. >>> > > >>> > > The npSize option and neverQueueSize, only affects things after the >>> > whole >>> > > message is received right? * It's not like it's doing something >>> > > significantly extra at each SSL cylce is it except using up RAM >>> right??* >>> > > My unscientific tests are shower marginally slower speeds when >>> > receiving >>> > > large email from Google with TLS on and neverQueueSize set crazy >>> high, >>> > so >>> > > maybe I'm wrong. Being that this same message through outlook.com >>> only >>> > > takes 30 seconds, even with npSize set to 0 (unlimited) and >>> > neverQueueSize >>> > > = 999,000,000, I'm wondering if the processing power and ram >>> > availability >>> > > (24 GB) on my assp box combined with low usage (only averaging 5000 >>> > > messages a day) might be enough to have neverQueueSize really high to >>> > > always scan everything. I'll play around there, but if >>> neverQueueSize >>> > is >>> > > 99MB, will things like ASSP_AFCDetectSpamAttachRe be slow WHILE the >>> > > message >>> > > is transferring, or is that just after the whole message is received? >>> > > >>> > > If neverQueue is set to 12MB, am I correct in saying that we're open >>> to >>> > a >>> > > targeted exploit, say an .exe in a .zip as long as the email is more >>> > than >>> > > 12MB? If so, that seems like a big hole, and something that I >>> > personally >>> > > would be willing to sacrifice speed/processing power to close. >>> > > >>> > > Suggestion, note in the gui under npSize that 0 really sets this to a >>> > > maximum of 12,000,000 unless neverQueueSize is set higher in >>> > > CorrectASSPcfg.pm. I've got that correct, right?) >>> > > >>> > > I forgot that DKIM signing can be done over the body too! We don't >>> do >>> > > that, but I forgot that others might. Maybe we have npSize only >>> ignore >>> > > DKIM if it's done over the body? DKIM should be fast if it's just >>> the >>> > > typical headers that are used right? We have npSize set to zero >>> > though, >>> > > so this doesn't really impact me. >>> > > >>> > > Would it be an easy modification to show the negotiated cipher in the >>> > > maillog or in the email body, if nothing else, just to have more >>> info? >>> > > X-ASSP-Cipher: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384 >>> > > for example or in the received line like Microsoft does: >>> > > >>> > > Received: from __________ (x.x.x.x) by ______ >>> > > (y.y.y.y) with Microsoft SMTP Server (version=TLS1_2, >>> > > cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) >>> > > >>> > > Or like Gmail does >>> > > Received: from smtp.ourcharity.org (smtp.ourcharity.org. [x.x.x.x]) >>> > > by mx.google.com with ESMTPS id >>> > > l1si54144332.11.2016.09.28.07.51.21 >>> > > for <nntp.p...@gmail.com> >>> > > (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 >>> > bits=128/128); >>> > > Wed, 28 Sep 2016 07:24:22 -0700 (PDT) >>> > > >>> > > It's a little more data (and certainly your time to code this), but >>> > could >>> > > be useful if we need it in the future. >>> > > >>> > > and:: >>> > > >>> > > Again Ken, the SSL parameters are NOT the problem on your system. >>> Your >>> > > >>> > > debug log shows a socket read time of max ~ 0.5 milliseconds (typical >>> > ~0.3 >>> > > >>> > > ms) for SSL. This read operation includes the time required for the >>> > > >>> > > decryption of the data. This is very very fast! >>> > > >>> > > It is simply the amazing count (10.400) of read and process >>> operations >>> > > >>> > > (cycles) required by assp for such a mail, that causes the overall >>> slow >>> > > >>> > > mail receive. >>> > > >>> > > in summary, if there's not an ASSP setting that I can tweak to have >>> > google >>> > > start sending larger SSL packets like Outlook.com and everyone else >>> I've >>> > > seen does, what other options are there? Are you suggesting that ALL >>> > ASSP >>> > > installations are incapable of receiving large TLS emails from Google >>> > > regardless of the installation's processing power, RAM, and >>> bandwidth? >>> > > >>> > > >>> > > >>> > > On Wed, Sep 28, 2016 at 4:26 AM, Thomas Eckardt >>> > > <thomas.ecka...@thockar.com> >>> > > wrote: >>> > > >>> > > > >The same email now took only 269 seconds. >>> > > > >>> > > > This is OK for googles behavior - the 1440 byte SSL frame. Around >>> 300s >>> > > was >>> > > > expected by me for this mail - hmm.. it is 10% better - nice! >>> > > > >>> > > > Take the following math. >>> > > > >>> > > > mail size = 15000 kB >>> > > > google frame size = 1.44 kB >>> > > > >>> > > > required assp loop cycles = mail size / frame size = 10.400 >>> > > > >>> > > > mail size = 15000 kB >>> > > > outlook.com frame size = 16 kB >>> > > > >>> > > > required assp loop cycles = mail size / frame size = 940 >>> > > > >>> > > > >I really wonder if I need to tweak my cipher_list. >>> > > > >>> > > > No! >>> > > > Again Ken, the SSL parameters are NOT the problem on your system. >>> Your >>> > > > debug log shows a socket read time of max ~ 0.5 milliseconds >>> (typical >>> > > ~0.3 >>> > > > ms) for SSL. This read operation includes the time required for the >>> > > > decryption of the data. This is very very fast! >>> > > > It is simply the amazing count (10.400) of read and process >>> operations >>> > > > (cycles) required by assp for such a mail, that causes the overall >>> > slow >>> > > > mail receive. >>> > > > >>> > > > >Is ClamAV scanning skipped too? >>> > > > Yes. >>> > > > >>> > > > >Could the plugins be run on the full mail after receipt, >>> regardless >>> > of >>> > > > size? >>> > > > >>> > > > Override the config parameters. Keep in mind that 'npSize' may also >>> > > > involved in skipping or processing some mail body checks. >>> > > > >>> > > > >Isn't DKIM checking just a one time thing and not intensive? >>> > > > >>> > > > The full DKIM check is very intensive. It requires to calculate an >>> > > RSA/SHA >>> > > > hash over the complete mail. >>> > > > DCC and Razor are doing something similar. >>> > > > ASSP_AFC would make all checks (ClamAV, FileScan, content checks >>> with >>> > > > several regular expression, decompression of attachments ....) for >>> the >>> > > > complete mail. It parses the complete mail at once with >>> Email::MIME. >>> > > This >>> > > > requires a huge amount of memory. Not a big deal on a 64bit OS with >>> > 8GB >>> > > > RAM and several CPU cores - who can !? >>> > > > All these can be done for any mail size, if the system is able to >>> > > process >>> > > > the amount of data fast enough. >>> > > > On most havy load systems, the 12.000.000 will be to large and may >>> > lead >>> > > in >>> > > > to stucking workers. >>> > > > >>> > > > ASSP has this set of config parameters - change them to your need - >>> > try >>> > > - >>> > > > and if does not work switch back - nothing more easy. >>> > > > >>> > > > Thomas >>> > > > >>> > > > >>> > > > >>> > > > Von: K Post <nntp.p...@gmail.com> >>> > > > An: ASSP development mailing list >>> > <assp-test@lists.sourceforge.net> >>> > > > Datum: 27.09.2016 21:08 >>> > > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com >>> addresses >>> / >>> > > > servers >>> > > > >>> > > > >>> > > > >>> > > > Our primary internet connection went down again (nothing to do with >>> > > ASSP) >>> > > > which gave me the opportunity to replace 16270 with 16271. Nothing >>> > like >>> > > > making lemonade out of lemons... >>> > > > >>> > > > The same email now took only 269 seconds. That's about 15x longer >>> > than >>> > > > with TLS off, but WOW that's way better than it was before. >>> > > > I also tried with the blank cipher list, no notable difference >>> > > > And with the SSL buffers set to 0 (64 MB), again without a speed >>> > > > difference. >>> > > > >>> > > > SO- you've made a real difference here!! Is there more >>> optimization >>> > to >>> > > > be >>> > > > made? >>> > > > >>> > > > The rub is that the exact same message sent through Outlook.com to >>> us, >>> > > > took >>> > > > exactly 30 seconds, just a 50% overhead when compared to the 19 >>> > seconds >>> > > > for >>> > > > a non-TLS message of the same size, instead of a 1500% overhead for >>> > > > encryption when receiving from google. >>> > > > >>> > > > *Is there some magical debug switch that I could turn on to see >>> what >>> > > > encryption Outlook.com is using and compare that to what Google's >>> > > > connection to us with?* I think prohibiting whatever the slow >>> cipher >>> > > that >>> > > > google's using (probably a really strong one) might make the final >>> bit >>> > > of >>> > > > difference. >>> > > > >>> > > > I'm breathing so much easier now!! Thank you. >>> > > > >>> > > > >>> > > > On Tue, Sep 27, 2016 at 2:08 PM, K Post <nntp.p...@gmail.com> >>> wrote: >>> > > > >>> > > > > Despite all the problems we have with personalities and policies >>> in >>> > > our >>> > > > > organization, the infrastructure is pretty solid. MTU's are set >>> > > > correctly, >>> > > > > no fragmentation, no jitter. There's low latency across the >>> board, >>> > > and >>> > > > > really low bandwidth usage. If we sent 1000 mails a day, it's a >>> > lot. >>> > > > > >>> > > > > ...and yes, they even pay their bills, though not me very well :) >>> > > > > >>> > > > > I think it's which SSL algorithm is being used that's at least >>> > > partially >>> > > > > to blame. I have: >>> > > > > SSL_Version: SSLv23:!SSLv3:!SSLv2 >>> > > > > SSL_Cipher_list: kEECDH+ECDSA:kEECDH:kEDH:HIGH:+SHA:+RC4:RC4 >>> > > > > :!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!DSS:!PSK:!SRP:!kECDH:! >>> > > > > CAMELLIA128:!IDEA:!SEED >>> > > > > >>> > > > > I tried the default wih SSL_Cipher_List blank before, I don't >>> think >>> > > > there >>> > > > > was a difference (but I've played with so many settings, I really >>> > > don't >>> > > > > remember) >>> > > > > >>> > > > > And last, on the SSL buffer size. If set to zero in the gui, on >>> > > windows >>> > > > > 2012, it says in green that it's set to 64 MB. I follow what >>> you're >>> > > > saying >>> > > > > about it readying 4x 16 Kb without a loop cycle. Is that a good >>> or >>> > > bad >>> > > > > thing though? >>> > > > > >>> > > > > >>> > > > > >>> > > > ------------------------------------------------------------ >>> > > > ------------------ >>> > > > _______________________________________________ >>> > > > Assp-test mailing list >>> > > > Assp-test@lists.sourceforge.net >>> > > > https://lists.sourceforge.net/lists/listinfo/assp-test >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > DISCLAIMER: >>> > > > ******************************************************* >>> > > > This email and any files transmitted with it may be confidential, >>> > > legally >>> > > > privileged and protected in law and are intended solely for the use >>> of >>> > > the >>> > > > >>> > > > individual to whom it is addressed. >>> > > > This email was multiple times scanned for viruses. There should be >>> no >>> > > > known virus in this email! >>> > > > ******************************************************* >>> > > > >>> > > > >>> > > > ------------------------------------------------------------ >>> > > > ------------------ >>> > > > >>> > > > _______________________________________________ >>> > > > Assp-test mailing list >>> > > > Assp-test@lists.sourceforge.net >>> > > > https://lists.sourceforge.net/lists/listinfo/assp-test >>> > > > >>> > > > >>> > > ------------------------------------------------------------ >>> > > ------------------ >>> > > _______________________________________________ >>> > > Assp-test mailing list >>> > > Assp-test@lists.sourceforge.net >>> > > https://lists.sourceforge.net/lists/listinfo/assp-test >>> > > >>> > > >>> > > >>> > > >>> > > DISCLAIMER: >>> > > ******************************************************* >>> > > This email and any files transmitted with it may be confidential, >>> > legally >>> > > privileged and protected in law and are intended solely for the use >>> of >>> > the >>> > > >>> > > individual to whom it is addressed. >>> > > This email was multiple times scanned for viruses. There should be no >>> > > known virus in this email! >>> > > ******************************************************* >>> > > >>> > > >>> > > ------------------------------------------------------------ >>> > > ------------------ >>> > > >>> > > _______________________________________________ >>> > > Assp-test mailing list >>> > > Assp-test@lists.sourceforge.net >>> > > https://lists.sourceforge.net/lists/listinfo/assp-test >>> > > >>> > > >>> > ------------------------------------------------------------ >>> > ------------------ >>> > _______________________________________________ >>> > Assp-test mailing list >>> > Assp-test@lists.sourceforge.net >>> > https://lists.sourceforge.net/lists/listinfo/assp-test >>> > >>> > >>> > >>> > >>> > DISCLAIMER: >>> > ******************************************************* >>> > This email and any files transmitted with it may be confidential, >>> legally >>> > privileged and protected in law and are intended solely for the use of >>> the >>> > >>> > individual to whom it is addressed. >>> > This email was multiple times scanned for viruses. There should be no >>> > known virus in this email! >>> > ******************************************************* >>> > >>> > >>> > ------------------------------------------------------------ >>> > ------------------ >>> > >>> > _______________________________________________ >>> > Assp-test mailing list >>> > Assp-test@lists.sourceforge.net >>> > https://lists.sourceforge.net/lists/listinfo/assp-test >>> > >>> > >>> ------------------------------------------------------------ >>> ------------------ >>> _______________________________________________ >>> Assp-test mailing list >>> Assp-test@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/assp-test >>> >>> >>> >>> >>> DISCLAIMER: >>> ******************************************************* >>> This email and any files transmitted with it may be confidential, legally >>> privileged and protected in law and are intended solely for the use of >>> the >>> >>> individual to whom it is addressed. >>> This email was multiple times scanned for viruses. There should be no >>> known virus in this email! >>> ******************************************************* >>> >>> >>> ------------------------------------------------------------ >>> ------------------ >>> >>> _______________________________________________ >>> Assp-test mailing list >>> Assp-test@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/assp-test >>> >>> >> >
------------------------------------------------------------------------------
_______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
