Hi Thomas, thanks for explaining this behaviour. Let's see if I get this right...
Let's assume an IP reveals constant misbehaviour adding a PB-IP-Score of 60 every hour. It started off at 0 so after 6 hours (default PenaltyExpiration) the score would be 360 and rising, but *surprise* after PeneltyExpiration time the IP gets a complete amnesty and is allowed to restart clean at 0 again. That does not reflect that IP's behaviour. I think the amnesty can be justified as soon as the offending IP sends one good message. But if it does not? Wouldn't it be better to have a "gliding" score, i.e. with every recalculation all entries which are older than PenaltyExpiration minutes get substracted from the overall score, but the rest is kept? (Obvious problem: codewise this could be complicated, because you would have to keep track of every single increment of the score and its timestamp.) Best regards Dirk -----Ursprüngliche Nachricht----- Von: Thomas Eckardt [mailto:thomas.ecka...@thockar.com] Gesendet: Montag, 3. Oktober 2016 11:50 An: ASSP development mailing list <assp-test@lists.sourceforge.net> Betreff: Re: [Assp-test] PB-IP-Score suddenly dropping The PBBlack record is removed after 'PenaltyExpiration' minutes of the record creation (NOT the last update). Thomas Von: "Dirk Kulmsee" <d.kulm...@netgroup.de> An: "'ASSP development mailing list'" <assp-test@lists.sourceforge.net> Datum: 03.10.2016 10:19 Betreff: Re: [Assp-test] PB-IP-Score suddenly dropping Hi Thomas, if there was a good message causing this, then I should see the IP in question in my log before the drop. But there is not a single line.The score is high, nothing happens, the score is low. This happened again today and I grep'ed the log for e.g. 118.71.251 (leaving out the last byte of the IP to see everything from a /24 area around it): 2016-10-03 02:52:52 [Worker_1] 118.71.251.67 info: PB-IP-Score for '118.71.251.67' is 480, added 60 in this session 2016-10-03 02:52:52 [Worker_1] 118.71.251.67 disconnected: session:7F11A94F5860 118.71.251.67 - processing time 24 seconds 2016-10-03 06:37:38 [Worker_1] Connected: session:7F11A94EBAB0 118.71.251.67:20540 > 192.168.12.242:25 > 127.0.0.1:125 2016-10-03 06:37:38 [Worker_1] 118.71.251.67 Disabled SMTP AUTH for External IPs 2016-10-03 06:37:39 [Worker_1] [unsupported_AUTH] 118.71.251.67 AUTH not allowed 2016-10-03 06:37:39 [Worker_1] 118.71.251.67 Message-Score: added 60 (autValencePB) for too many AUTH errors from 118.71.251.0, total score for this message is now 60 2016-10-03 06:37:39 [Worker_1] 118.71.251.67 PB-IP-Score for '118.71.251.67' is 60, added 60 for AUTHErrors Nobody from 118.71.251 shows up between 02:52 and 06:37. Still the score drops from 480 to 0. There is however some background work being done during that time, e.g.: 2016-10-03 03:31:11 [Worker_10000] PenaltyBox: cleaning BlackBox (PBBlack) finished: IP's before=81, deleted=19 2016-10-03 06:31:15 [Worker_10000] PenaltyBox: cleaning BlackBox (PBBlack) finished: IP's before=76, deleted=12 Can you give me a hint what to look at to better understand this? I have set PenaltyDuration = 60 and PenaltyExpiration=720. What else can be of influence here? Thanks a lot Dirk -----Ursprüngliche Nachricht----- Von: Thomas Eckardt [mailto:thomas.ecka...@thockar.com] Gesendet: Montag, 3. Oktober 2016 09:12 An: ASSP development mailing list <assp-test@lists.sourceforge.net> Betreff: Re: [Assp-test] PB-IP-Score suddenly dropping There are dozend of reasons why this can happen. Most common is 'PenaltyExpiration'. If there is a good mail transfered by an IP, the IP score is deleted to prevent false positives. Where good means - no doubed, like 'contentOnly', RWL, SPF, DKIM .... Thomas. Von: "Dirk Kulmsee" <d.kulm...@netgroup.de> An: "'ASSP development mailing list'" <assp-test@lists.sourceforge.net> Datum: 02.10.2016 20:04 Betreff: [Assp-test] PB-IP-Score suddenly dropping Hi all, I just tracked some IPs through my logfiles just to see how they build up their score. Something strange is happening: Case 1: between 09:51:13 and 12:49:10 PB-IP-Score drops from 600 to 0 without any visible reason 2016-10-02 06:13:54 [Worker_1] Connected: session:7F11F4A35FA0 118.71.251.67:53467 > 192.168.12.242:25 > 127.0.0.1:125 2016-10-02 06:13:54 [Worker_1] 118.71.251.67 Disabled SMTP AUTH for External IPs 2016-10-02 06:13:55 [Worker_1] [unsupported_AUTH] 118.71.251.67 AUTH not allowed 2016-10-02 06:13:55 [Worker_1] 118.71.251.67 Message-Score: added 60 (autValencePB) for too many AUTH errors from 118.71.251.0, total score for this message is now 60 2016-10-02 06:13:55 [Worker_1] 118.71.251.67 PB-IP-Score for '118.71.251.67' is 540, added 60 for AUTHErrors 2016-10-02 06:13:55 [Worker_1] 118.71.251.67 [SMTP Error] 502 AUTH not supported 2016-10-02 06:13:55 [Worker_1] 118.71.251.67 info: start damping (12 s) 2016-10-02 06:14:07 [Worker_1] 118.71.251.67 info: PB-IP-Score for '118.71.251.67' is 540, added 60 in this session 2016-10-02 06:14:07 [Worker_1] 118.71.251.67 disconnected: session:7F11F4A35FA0 118.71.251.67 - processing time 13 seconds 2016-10-02 06:22:56 [Worker_1] Delayed ip 118.71.251.67, because PBBlack(540) is higher than DelayIP(500)- last penalty reason was: AUTHErrors 2016-10-02 07:07:29 [Worker_1] Connected: session:7F11F4C41160 118.71.251.67:54518 > 192.168.12.242:25 > 127.0.0.1:125 2016-10-02 07:07:29 [Worker_1] 118.71.251.67 Disabled SMTP AUTH for External IPs 2016-10-02 07:07:29 [Worker_1] [unsupported_AUTH] 118.71.251.67 AUTH not allowed 2016-10-02 07:07:29 [Worker_1] 118.71.251.67 Message-Score: added 60 (autValencePB) for too many AUTH errors from 118.71.251.0, total score for this message is now 60 2016-10-02 07:07:29 [Worker_1] 118.71.251.67 PB-IP-Score for '118.71.251.67' is 600, added 60 for AUTHErrors 2016-10-02 07:07:29 [Worker_1] 118.71.251.67 [SMTP Error] 502 AUTH not supported 2016-10-02 07:07:29 [Worker_1] 118.71.251.67 info: start damping (12 s) 2016-10-02 07:07:53 [Worker_1] 118.71.251.67 info: PB-IP-Score for '118.71.251.67' is 600, added 60 in this session 2016-10-02 07:07:53 [Worker_1] 118.71.251.67 disconnected: session:7F11F4C41160 118.71.251.67 - processing time 24 seconds 2016-10-02 09:51:13 [Worker_1] Delayed ip 118.71.251.67, because PBBlack(600) is higher than DelayIP(500)- last penalty reason was: AUTHErrors 2016-10-02 12:49:10 [Worker_1] Connected: session:7F11F573EEF0 118.71.251.67:2425 > 192.168.12.242:25 > 127.0.0.1:125 2016-10-02 12:49:10 [Worker_1] 118.71.251.67 Disabled SMTP AUTH for External IPs 2016-10-02 12:49:10 [Worker_1] [unsupported_AUTH] 118.71.251.67 AUTH not allowed 2016-10-02 12:49:10 [Worker_1] 118.71.251.67 Message-Score: added 60 (autValencePB) for too many AUTH errors from 118.71.251.0, total score for this message is now 60 2016-10-02 12:49:10 [Worker_1] 118.71.251.67 PB-IP-Score for '118.71.251.67' is 60, added 60 for AUTHErrors 2016-10-02 12:49:10 [Worker_1] 118.71.251.67 [SMTP Error] 502 AUTH not supported 2016-10-02 12:49:11 [Worker_1] 118.71.251.67 info: start damping (12 s) 2016-10-02 12:49:34 [Worker_1] 118.71.251.67 info: PB-IP-Score for '118.71.251.67' is 60, added 60 in this session 2016-10-02 12:49:34 [Worker_1] 118.71.251.67 disconnected: session:7F11F573EEF0 118.71.251.67 - processing time 24 seconds Case 2: between 15:02:57 and 15:41:09 PB-IP-Score drops from 600 to 0 without any visible reason 2016-10-02 11:49:40 [Worker_1] Connected: session:7F11F65EC988 46.32.239.160:64727 > 192.168.12.242:25 > 127.0.0.1:125 2016-10-02 11:49:40 [Worker_1] 46.32.239.160 Disabled SMTP AUTH for External IPs 2016-10-02 11:49:41 [Worker_1] [unsupported_AUTH] 46.32.239.160 AUTH not allowed 2016-10-02 11:49:41 [Worker_1] 46.32.239.160 Message-Score: added 60 (autValencePB) for too many AUTH errors from 46.32.239.0, total score for this message is now 60 2016-10-02 11:49:41 [Worker_1] 46.32.239.160 PB-IP-Score for '46.32.239.160' is 540, added 60 for AUTHErrors 2016-10-02 11:49:41 [Worker_1] 46.32.239.160 [SMTP Error] 502 AUTH not supported 2016-10-02 11:49:41 [Worker_1] 46.32.239.160 info: start damping (12 s) 2016-10-02 11:50:05 [Worker_1] 46.32.239.160 info: PB-IP-Score for '46.32.239.160' is 540, added 60 in this session 2016-10-02 11:50:05 [Worker_1] 46.32.239.160 disconnected: session:7F11F65EC988 46.32.239.160 - processing time 25 seconds 2016-10-02 14:43:24 [Worker_1] Delayed ip 46.32.239.160, because PBBlack(540) is higher than DelayIP(500)- last penalty reason was: AUTHErrors 2016-10-02 15:02:32 [Worker_1] Connected: session:7F11F6667F10 46.32.239.160:64548 > 192.168.12.242:25 > 127.0.0.1:125 2016-10-02 15:02:32 [Worker_1] 46.32.239.160 Disabled SMTP AUTH for External IPs 2016-10-02 15:02:33 [Worker_1] [unsupported_AUTH] 46.32.239.160 AUTH not allowed 2016-10-02 15:02:33 [Worker_1] 46.32.239.160 Message-Score: added 60 (autValencePB) for too many AUTH errors from 46.32.239.0, total score for this message is now 60 2016-10-02 15:02:33 [Worker_1] 46.32.239.160 PB-IP-Score for '46.32.239.160' is 600, added 60 for AUTHErrors 2016-10-02 15:02:33 [Worker_1] 46.32.239.160 [SMTP Error] 502 AUTH not supported 2016-10-02 15:02:33 [Worker_1] 46.32.239.160 info: start damping (12 s) 2016-10-02 15:02:57 [Worker_1] 46.32.239.160 info: PB-IP-Score for '46.32.239.160' is 600, added 60 in this session 2016-10-02 15:02:57 [Worker_1] 46.32.239.160 disconnected: session:7F11F6667F10 46.32.239.160 - processing time 25 seconds 2016-10-02 15:41:08 [Worker_1] Connected: session:7F11F70B3378 46.32.239.160:56422 > 192.168.12.242:25 > 127.0.0.1:125 2016-10-02 15:41:08 [Worker_1] 46.32.239.160 Disabled SMTP AUTH for External IPs 2016-10-02 15:41:09 [Worker_1] [unsupported_AUTH] 46.32.239.160 AUTH not allowed 2016-10-02 15:41:09 [Worker_1] 46.32.239.160 Message-Score: added 60 (autValencePB) for too many AUTH errors from 46.32.239.0, total score for this message is now 60 2016-10-02 15:41:09 [Worker_1] 46.32.239.160 PB-IP-Score for '46.32.239.160' is 60, added 60 for AUTHErrors 2016-10-02 15:41:09 [Worker_1] 46.32.239.160 [SMTP Error] 502 AUTH not supported 2016-10-02 15:41:09 [Worker_1] 46.32.239.160 info: start damping (12 s) 2016-10-02 15:41:33 [Worker_1] 46.32.239.160 info: PB-IP-Score for '46.32.239.160' is 60, added 60 in this session 2016-10-02 15:41:33 [Worker_1] 46.32.239.160 disconnected: session:7F11F70B3378 46.32.239.160 - processing time 25 seconds I'm currently running ASSP version 2.5.2(16270) on Debian with perl 5.22. There have been no restarts or resets of ASSP for 6 days. Any ideas what has happened here? Best regards Dirk ---------------------------------------------------------------------------- -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test DISCLAIMER: ******************************************************* This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! ******************************************************* ---------------------------------------------------------------------------- -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test DISCLAIMER: ******************************************************* This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! ******************************************************* ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
