Re: Restore pf tables metadata after a reboot

[Brian Brombacher]

No reason to expire ssh brute force.  They will never stop.

Manual flush if someone accidentally locked themselves out.

Just my two cents :)

> On Jun 4, 2020, at 12:48 AM, Anatoli  wrote:
> 
> 
>> 
>> Even then it seems that some of them turn up again pretty much
>> instantly after expiry.
> 
> You could update the expire time on each new connection/port scan
> attempt. This way you could put say 4 days expire time and block these
> IPs on all ports on all your systems and new connection attempts would
> update the expire for all the systems.
> 
> 4 days is because 5 days is a typical timeout for a temporary error for
> SMTP. It may happen that someone used for 24hs a cloud instance and
> then got banned by the cloud provider, the IP used for
> spam/scans/attacks could be reused for another client for a legit
> activity. So if that new client for the old IP sends to your client some
> important mail, it's not lost and doesn't generate an undeliverable mail
> report, it just takes some days to reach the destination (with retries
> by the origin server).
> 
> 4 weeks looks excessive for cloud shared IPs.
> 
> 
>> On 30/5/20 07:25, Peter Nicolai Mathias Hansteen wrote:
>> 
>> 
 30. mai 2020 kl. 11:54 skrev Walter Alejandro Iglesias :
>>> 
>>> The problem is most system administrators out there do very little.  If
>>> you were getting spam or attacks from some IP, even if you report the
>>> issue to the respective whois abuse@ address, chances are attacks from
>>> that IP won't stop next week, nor even next month.
>>> 
>>> So, in general terms, I would refrain as much as possible from hurry to
>>> expiring addresses.  Just my opinion.
>> 
>> Yes, there are a lot of systems out there that seem to be not really 
>> maintained at all. After years of advocating 24 hour expiry some time back I 
>> went to four weeks on the ssh brutes blacklist. Even then it seems that some 
>> of them turn up again pretty much instantly after expiry.
>> 
>> All the best,
>> 
>> —
>> Peter N. M. Hansteen, member of the first RFC 1149 implementation team
>> http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
>> "Remember to set the evil bit on all malicious network traffic"
>> delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
>> 
>> 
>> 
>> 
> 

Re: Restore pf tables metadata after a reboot

[Anatoli]

> Even then it seems that some of them turn up again pretty much
> instantly after expiry.

You could update the expire time on each new connection/port scan
attempt. This way you could put say 4 days expire time and block these
IPs on all ports on all your systems and new connection attempts would
update the expire for all the systems.

4 days is because 5 days is a typical timeout for a temporary error for
SMTP. It may happen that someone used for 24hs a cloud instance and
then got banned by the cloud provider, the IP used for
spam/scans/attacks could be reused for another client for a legit
activity. So if that new client for the old IP sends to your client some
important mail, it's not lost and doesn't generate an undeliverable mail
report, it just takes some days to reach the destination (with retries
by the origin server).

4 weeks looks excessive for cloud shared IPs.


On 30/5/20 07:25, Peter Nicolai Mathias Hansteen wrote:
> 
> 
>> 30. mai 2020 kl. 11:54 skrev Walter Alejandro Iglesias :
>>
>> The problem is most system administrators out there do very little.  If
>> you were getting spam or attacks from some IP, even if you report the
>> issue to the respective whois abuse@ address, chances are attacks from
>> that IP won't stop next week, nor even next month.
>>
>> So, in general terms, I would refrain as much as possible from hurry to
>> expiring addresses.  Just my opinion.
> 
> Yes, there are a lot of systems out there that seem to be not really 
> maintained at all. After years of advocating 24 hour expiry some time back I 
> went to four weeks on the ssh brutes blacklist. Even then it seems that some 
> of them turn up again pretty much instantly after expiry.
> 
> All the best,
> 
> —
> Peter N. M. Hansteen, member of the first RFC 1149 implementation team
> http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
> "Remember to set the evil bit on all malicious network traffic"
> delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
> 
> 
> 
> 

Re: Restore pf tables metadata after a reboot

[Peter Nicolai Mathias Hansteen]

> 30. mai 2020 kl. 11:54 skrev Walter Alejandro Iglesias :
> 
> The problem is most system administrators out there do very little.  If
> you were getting spam or attacks from some IP, even if you report the
> issue to the respective whois abuse@ address, chances are attacks from
> that IP won't stop next week, nor even next month.
> 
> So, in general terms, I would refrain as much as possible from hurry to
> expiring addresses.  Just my opinion.

Yes, there are a lot of systems out there that seem to be not really maintained 
at all. After years of advocating 24 hour expiry some time back I went to four 
weeks on the ssh brutes blacklist. Even then it seems that some of them turn up 
again pretty much instantly after expiry.

All the best,

—
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.






signature.asc
Description: Message signed with OpenPGP

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

In article  Peter Nicolai 
Mathias Hansteen  wrote:
> It is a possibly desirable feature, but I an not aware whether any of the 
> currently capable developers are considering putting in the work to implement 
> it.
> 

Let me finish the idea, not with the intention to pressure developers
asking for features but to share my experience and thoughts about the
issue.

I've also been publishing (long) blacklists in my website as you do.  As
an experiment I didn't expire any until recently when, as I explained,
they reached the hard limit in memory (200).  And, as I suggested,
right before I expired addresses old spam, recognizable by the format,
appeared again.

The problem is most system administrators out there do very little.  If
you were getting spam or attacks from some IP, even if you report the
issue to the respective whois abuse@ address, chances are attacks from
that IP won't stop next week, nor even next month.

So, in general terms, I would refrain as much as possible from hurry to
expiring addresses.  Just my opinion.

Re: Restore pf tables metadata after a reboot

[Peter Nicolai Mathias Hansteen]

> 29. mai 2020 kl. 19:23 skrev Walter Alejandro Iglesias :

> Could you summarize here which part of these articles of yours answer my
> original question, please?
> 
> For example, this list you share (linked in your article):
> 
>  https://home.nuug.no/~peter/pop3gropers_full.txt
> 
> It would be great to be able to do the following before and after a
> reboot respectivelly:
> 
>  # pfctl -t smtp -vT show > file   # (notice the verbose option)
>  # pfctl -t smtp -T replace -f file
> 
> But we know that doesn't work.

True, pfctl does not have the ability to parse back that metadata.

It is a possibly desirable feature, but I an not aware whether any of the 
currently capable developers are considering putting in the work to implement 
it.

All the best,


—
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.






signature.asc
Description: Message signed with OpenPGP

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

Hello Peter,

In article  Peter Nicolai 
Mathias Hansteen  wrote:
> > 28. mai 2020 kl. 19:09 skrev Bruno Flueckiger :
> > 
> > 
> > You can save the list of IPs in a table and reload it after a reboot as
> > described here: https://www.bsdhowto.ch/savepftables.html
> 
> 
> I have a similar setup at bsdly.net , only I dump the 
> tables to file and run expiry via a cron job that runs twice an hour - the 
> writeup at 
> https://bsdly.blogspot.com/2017/04/forcing-password-gropers-through.html 
>  
> has most of the useful info and some related wrinkles.
> 

Could you summarize here which part of these articles of yours answer my
original question, please?

For example, this list you share (linked in your article):

  https://home.nuug.no/~peter/pop3gropers_full.txt

It would be great to be able to do the following before and after a
reboot respectivelly:

  # pfctl -t smtp -vT show > file   # (notice the verbose option)
  # pfctl -t smtp -T replace -f file

But we know that doesn't work.


> All the best,
> 
> 
> —
> Peter N. M. Hansteen, member of the first RFC 1149 implementation team
> http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
> "Remember to set the evil bit on all malicious network traffic"
> delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
> 
> 

Re: Restore pf tables metadata after a reboot

[Peter Nicolai Mathias Hansteen]

> 28. mai 2020 kl. 19:09 skrev Bruno Flueckiger :
> 
> 
> You can save the list of IPs in a table and reload it after a reboot as
> described here: https://www.bsdhowto.ch/savepftables.html


I have a similar setup at bsdly.net , only I dump the tables 
to file and run expiry via a cron job that runs twice an hour - the writeup at 
https://bsdly.blogspot.com/2017/04/forcing-password-gropers-through.html 
 has 
most of the useful info and some related wrinkles.

All the best,


—
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.






signature.asc
Description: Message signed with OpenPGP

Re: Restore pf tables metadata after a reboot

[Bruno Flueckiger]

On 29.05., Walter Alejandro Iglesias wrote:
> In article <20200528165448.ga22...@flueckiger.lan> Bruno Flueckiger 
>  wrote:
> > On 26.05., Walter Alejandro Iglesias wrote:
> > > I understand that this command:
> > >
> > >   # pfctl -t spam -T expire 
> > >
> > > Takes in care the "Cleared" date:
> > >
> > >   # pfctl -t spam -vT show
> > >  ___.___.22.65
> > >   Cleared: Mon May 25 16:10:22 2020
> > >  ___.___.167.62
> > >   Cleared: Mon May 25 16:10:22 2020
> > >   [...]
> > >
> > > Is there a way to save and restore tables metadata after a reboot
> > > preserving those dates?
> > >
> >
> > You can save the list of IPs in a table and reload it after a reboot as
> > described here: https://www.bsdhowto.ch/savepftables.html
>
> Nice website. ;-)
>

Thanks :-)

> >
> > As there is no way to save the dates the date for each IP will be set to
> > the current date and time when load happens.
>
> The interesting point and the reason of my concern is to choose a
> convenient "expire time."  With mail is problematic but with ssh, since
> I know exactly whom I want to allow external access (just me,) I let
> them accumulate.  I block ssh attackers in the ssh port only, people
> sharing those addresses are not affected.  So, I thought, the only
> concern in the ssh case was how much a big number of entries could
> affect pf performance, till at some point my tables reached the memory
> hard limit and I had to remove IPs arbitrarily. :-)
>

Well, I use my system in production. Therefore I prefer to be on the
safe side and remove old entries from my block tables rather than
risking instabilities or performance penalties.

> In summary, pfctl expire command does nothing after a reboot.  Then you
> have two options:
>
>   - To use a (cron) expire time significantly lower than the desirable.
>
>   - To expire entries when your tables are about to reach the memory
> hard limit.
>
> In both cases you'll probably suffer spam again from IPs that were
> already blocked.
>

What is a desirable expire time for blocked IPs in your view?

For SSH I don't care how many times an attacker tries it. As soon as the
IP is in the blocking table I don't even get log entries for it.

In case of SMTP I don't rely solely on IP blocking to fight spam. The
blocking only kicks in if there are too many simultaneous connections
comming from the same IP.

Cheers,
Bruno

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

In article <20200528165448.ga22...@flueckiger.lan> Bruno Flueckiger 
 wrote:
> On 26.05., Walter Alejandro Iglesias wrote:
> > I understand that this command:
> >
> >   # pfctl -t spam -T expire 
> >
> > Takes in care the "Cleared" date:
> >
> >   # pfctl -t spam -vT show
> >  ___.___.22.65
> >   Cleared: Mon May 25 16:10:22 2020
> >  ___.___.167.62
> >   Cleared: Mon May 25 16:10:22 2020
> >   [...]
> >
> > Is there a way to save and restore tables metadata after a reboot
> > preserving those dates?
> >
> 
> You can save the list of IPs in a table and reload it after a reboot as
> described here: https://www.bsdhowto.ch/savepftables.html

Nice website. ;-)

> 
> As there is no way to save the dates the date for each IP will be set to
> the current date and time when load happens.

The interesting point and the reason of my concern is to choose a
convenient "expire time."  With mail is problematic but with ssh, since
I know exactly whom I want to allow external access (just me,) I let
them accumulate.  I block ssh attackers in the ssh port only, people
sharing those addresses are not affected.  So, I thought, the only
concern in the ssh case was how much a big number of entries could
affect pf performance, till at some point my tables reached the memory
hard limit and I had to remove IPs arbitrarily. :-)

In summary, pfctl expire command does nothing after a reboot.  Then you
have two options:

  - To use a (cron) expire time significantly lower than the desirable.

  - To expire entries when your tables are about to reach the memory
hard limit.

In both cases you'll probably suffer spam again from IPs that were
already blocked.


> 
> Cheers,
> Bruno
> 
> 

Walter

Re: Restore pf tables metadata after a reboot

[Bruno Flueckiger]

On 26.05., Walter Alejandro Iglesias wrote:
> I understand that this command:
>
>   # pfctl -t spam -T expire 
>
> Takes in care the "Cleared" date:
>
>   # pfctl -t spam -vT show
>  ___.___.22.65
>   Cleared: Mon May 25 16:10:22 2020
>  ___.___.167.62
>   Cleared: Mon May 25 16:10:22 2020
>   [...]
>
> Is there a way to save and restore tables metadata after a reboot
> preserving those dates?
>

You can save the list of IPs in a table and reload it after a reboot as
described here: https://www.bsdhowto.ch/savepftables.html

As there is no way to save the dates the date for each IP will be set to
the current date and time when load happens.

Cheers,
Bruno

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

On Tue, May 26, 2020 at 11:25:21PM +0200, Anders Andersson wrote:
> On Tue, May 26, 2020 at 2:14 PM Walter Alejandro Iglesias
>  wrote:
> >
> > I understand that this command:
> >
> >   # pfctl -t spam -T expire 
> >
> > Takes in care the "Cleared" date:
> >
> >   # pfctl -t spam -vT show
> >  ___.___.22.65
> >   Cleared: Mon May 25 16:10:22 2020
> >  ___.___.167.62
> >   Cleared: Mon May 25 16:10:22 2020
> >   [...]
> >
> > Is there a way to save and restore tables metadata after a reboot
> > preserving those dates?
> 
> Isn't this what pfctl -S and -L does?

I *guess* what pfctrl -S does is to save in a file the same you see in
'pfctl -s states' output but in binary format.

Re: Restore pf tables metadata after a reboot

[Anders Andersson]

On Tue, May 26, 2020 at 2:14 PM Walter Alejandro Iglesias
 wrote:
>
> I understand that this command:
>
>   # pfctl -t spam -T expire 
>
> Takes in care the "Cleared" date:
>
>   # pfctl -t spam -vT show
>  ___.___.22.65
>   Cleared: Mon May 25 16:10:22 2020
>  ___.___.167.62
>   Cleared: Mon May 25 16:10:22 2020
>   [...]
>
> Is there a way to save and restore tables metadata after a reboot
> preserving those dates?

Isn't this what pfctl -S and -L does?

Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

I understand that this command:

  # pfctl -t spam -T expire 

Takes in care the "Cleared" date:

  # pfctl -t spam -vT show
 ___.___.22.65
  Cleared: Mon May 25 16:10:22 2020
 ___.___.167.62
  Cleared: Mon May 25 16:10:22 2020
  [...]

Is there a way to save and restore tables metadata after a reboot
preserving those dates?