Re: Is there an upper limit to PF's tables?
On Mon, 18 Jun 2018 12:08:33 +0200 "Kristof Provost" said On 18 Jun 2018, at 0:19, Chris H wrote: > Sorry. Looks like I might be coming to the party a little late. But > I'm > currently running a 9.3 box that runs as a IP (service) filter for > much > of a network. While I've patched the box well enough to keep it safe > to > continue running. I am reluctant to up(grade|date) it to 11, or > CURRENT, > based on some of the information related to topics like this thread. > Currently, the 9.3 box maintains some 18 million entries *just* within > the SPAM related table. The other tables contain no less that 1 > million. > As it stands I have *no* trouble loading pf(4) with all of the tables > totaling some 20+ million entries, *even* when the BOX is working with > as little 4Gb ram. > Has something in pf(4) changed, since 9.3 that would now prevent me > from continuing to use my current setup, and tables? > No. There are no new limits in 11, and the only thing that *might* be an issue is validation improvements in 12. Still, anything that worked on 9 is expected to work on 12 (if not, report a bug). Thank you very much for the informative reply, Kristof! Please don’t keep running unsupported versions. You're reply leaves me little reason to think I need, or want to. :-) Thanks, again! --Chris Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On Mon, 18 Jun 2018 12:21:47 +0200 "Kurt Jaeger" said Hi! > > So loading all entries in to empty table works fine, but reloading > > didn't work. > Sorry. Looks like I might be coming to the party a little late. But I'm > currently running a 9.3 box that runs as a IP (service) filter for much > of a network. While I've patched the box well enough to keep it safe to > continue running. I am reluctant to up(grade|date) it to 11, or CURRENT, > based on some of the information related to topics like this thread. > Currently, the 9.3 box maintains some 18 million entries *just* within > the SPAM related table. The other tables contain no less that 1 million. > As it stands I have *no* trouble loading pf(4) with all of the tables > totaling some 20+ million entries, *even* when the BOX is working with > as little 4Gb ram. > Has something in pf(4) changed, since 9.3 that would now prevent me > from continuing to use my current setup, and tables? Well, if you plan to upgrade, I'd suggest you do some tests, like dumping those tables and loading them on a new box. At all our installations we did use PF in 9.x times and had no problems to move to 11.x. Thanks for the reply, Kurt. That's good advice, indeed. As that was pretty much my "game plan". But recently I've seen a few entries on the list, and a few pr(1)'s regarding the inability to start pf(1), because the tables were too large. Whereas I hadn't heard anyone mention it in the past. So it seemed prudent to ask. :-) Thanks again, Kurt! --Chris -- p...@opsec.eu+49 171 31013722 years to go ! ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
Hi! > > So loading all entries in to empty table works fine, but reloading > > didn't work. > Sorry. Looks like I might be coming to the party a little late. But I'm > currently running a 9.3 box that runs as a IP (service) filter for much > of a network. While I've patched the box well enough to keep it safe to > continue running. I am reluctant to up(grade|date) it to 11, or CURRENT, > based on some of the information related to topics like this thread. > Currently, the 9.3 box maintains some 18 million entries *just* within > the SPAM related table. The other tables contain no less that 1 million. > As it stands I have *no* trouble loading pf(4) with all of the tables > totaling some 20+ million entries, *even* when the BOX is working with > as little 4Gb ram. > Has something in pf(4) changed, since 9.3 that would now prevent me > from continuing to use my current setup, and tables? Well, if you plan to upgrade, I'd suggest you do some tests, like dumping those tables and loading them on a new box. At all our installations we did use PF in 9.x times and had no problems to move to 11.x. -- p...@opsec.eu+49 171 31013722 years to go ! ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On 18 Jun 2018, at 0:19, Chris H wrote: Sorry. Looks like I might be coming to the party a little late. But I'm currently running a 9.3 box that runs as a IP (service) filter for much of a network. While I've patched the box well enough to keep it safe to continue running. I am reluctant to up(grade|date) it to 11, or CURRENT, based on some of the information related to topics like this thread. Currently, the 9.3 box maintains some 18 million entries *just* within the SPAM related table. The other tables contain no less that 1 million. As it stands I have *no* trouble loading pf(4) with all of the tables totaling some 20+ million entries, *even* when the BOX is working with as little 4Gb ram. Has something in pf(4) changed, since 9.3 that would now prevent me from continuing to use my current setup, and tables? No. There are no new limits in 11, and the only thing that *might* be an issue is validation improvements in 12. Still, anything that worked on 9 is expected to work on 12 (if not, report a bug). Please don’t keep running unsupported versions. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On Thu, 14 Jun 2018 21:44:08 +0200 "Miroslav Lachman" <000.f...@quip.cz> said Dave Horsfall wrote on 2018/06/14 19:40: > I can't get access to kernel sauce right now, but I'm hitting over 1,000 > entries from woodpeckers[*] etc; is there some upper limit, or is it > just purely dynamic? > > aneurin% freebsd-version > 10.4-RELEASE-p9 One of our customers have machine with 10.4 too. They are blocking all Tor IP addresses. The table has 272574 entries now. There were/(are) some problems with reload of PF: # service pf reload Reloading pf rules. /etc/pf.conf:37: cannot define table reserved: Cannot allocate memory /etc/pf.conf:38: cannot define table czech_net: Cannot allocate memory /etc/pf.conf:39: cannot define table goodguys: Cannot allocate memory /etc/pf.conf:40: cannot define table badguys: Cannot allocate memory /etc/pf.conf:41: cannot define table tor_net: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded Even if there is "set limit table-entries 30" I do not understand PF internals but I think PF needs twice the memory for reload (if there are already a lot of entries). Because workaround for this was simple as reload PF with empty table and then load table entries: # mv /etc/pf.tor_net.table /etc/pf.tor_net.table.BaK # touch /etc/pf.tor_net.table # pfctl -t tor_net -T flush 201703 addresses deleted. # pfctl -vf /etc/pf.conf # pfctl -t tor_net -T replace -f /etc/pf.tor_net.table.BaK So loading all entries in to empty table works fine, but reloading didn't work. Sorry. Looks like I might be coming to the party a little late. But I'm currently running a 9.3 box that runs as a IP (service) filter for much of a network. While I've patched the box well enough to keep it safe to continue running. I am reluctant to up(grade|date) it to 11, or CURRENT, based on some of the information related to topics like this thread. Currently, the 9.3 box maintains some 18 million entries *just* within the SPAM related table. The other tables contain no less that 1 million. As it stands I have *no* trouble loading pf(4) with all of the tables totaling some 20+ million entries, *even* when the BOX is working with as little 4Gb ram. Has something in pf(4) changed, since 9.3 that would now prevent me from continuing to use my current setup, and tables? Thanks! --Chris Miroslav Lachman ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org" ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
Thanks, all, for your suggestions; I suspect that this ancient server with but 512MB memory will need upgrading soon :-) Thankfully, all it does at the moment is act as my mail/web server, and an internal firewall to the Mac and Penguin boxes; I do my development work on the Mac[*], and test it out on those in turn (and usually ending up cursing Penguin/OS for egregiously breaking something). [*] Except for devices with a serial port, because I simply don't trust serial/USB adaptor cables and their shoddy drivers; my next FreeBSD server will still have genuine serial/parallel ports (it will also be a GPS NTP server, and FreeBSD supports the all-important PPS signal on the serial port). -- Dave ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
Ian FREISLICH wrote on 2018/06/14 22:03: On 06/14/2018 03:44 PM, Miroslav Lachman wrote: # service pf reload Reloading pf rules. /etc/pf.conf:37: cannot define table reserved: Cannot allocate memory /etc/pf.conf:38: cannot define table czech_net: Cannot allocate memory /etc/pf.conf:39: cannot define table goodguys: Cannot allocate memory /etc/pf.conf:40: cannot define table badguys: Cannot allocate memory /etc/pf.conf:41: cannot define table tor_net: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded Even if there is "set limit table-entries 30" I do not understand PF internals but I think PF needs twice the memory for reload (if there are already a lot of entries). Because workaround for this was simple as reload PF with empty table and then load table entries: Did you try setting the table limit to 50? I believe that PF does a copyin from pfctl essentially building the new inactive ruleset and switching to it at commit. This would result in the twice memory requirement you're seeing. It has been a long long time for me so I've probably not explained correctly. No I didn't tried anything above 30 but I will try it next time. (maybe 60) Miroslav Lachman ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On 06/14/2018 03:44 PM, Miroslav Lachman wrote: Dave Horsfall wrote on 2018/06/14 19:40: I can't get access to kernel sauce right now, but I'm hitting over 1,000 entries from woodpeckers[*] etc; is there some upper limit, or is it just purely dynamic? aneurin% freebsd-version 10.4-RELEASE-p9 One of our customers have machine with 10.4 too. They are blocking all Tor IP addresses. The table has 272574 entries now. There were/(are) some problems with reload of PF: # service pf reload Reloading pf rules. /etc/pf.conf:37: cannot define table reserved: Cannot allocate memory /etc/pf.conf:38: cannot define table czech_net: Cannot allocate memory /etc/pf.conf:39: cannot define table goodguys: Cannot allocate memory /etc/pf.conf:40: cannot define table badguys: Cannot allocate memory /etc/pf.conf:41: cannot define table tor_net: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded Even if there is "set limit table-entries 30" I do not understand PF internals but I think PF needs twice the memory for reload (if there are already a lot of entries). Because workaround for this was simple as reload PF with empty table and then load table entries: Did you try setting the table limit to 50? I believe that PF does a copyin from pfctl essentially building the new inactive ruleset and switching to it at commit. This would result in the twice memory requirement you're seeing. It has been a long long time for me so I've probably not explained correctly. Ian -- ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
Dave Horsfall wrote on 2018/06/14 19:40: I can't get access to kernel sauce right now, but I'm hitting over 1,000 entries from woodpeckers[*] etc; is there some upper limit, or is it just purely dynamic? aneurin% freebsd-version 10.4-RELEASE-p9 One of our customers have machine with 10.4 too. They are blocking all Tor IP addresses. The table has 272574 entries now. There were/(are) some problems with reload of PF: # service pf reload Reloading pf rules. /etc/pf.conf:37: cannot define table reserved: Cannot allocate memory /etc/pf.conf:38: cannot define table czech_net: Cannot allocate memory /etc/pf.conf:39: cannot define table goodguys: Cannot allocate memory /etc/pf.conf:40: cannot define table badguys: Cannot allocate memory /etc/pf.conf:41: cannot define table tor_net: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded Even if there is "set limit table-entries 30" I do not understand PF internals but I think PF needs twice the memory for reload (if there are already a lot of entries). Because workaround for this was simple as reload PF with empty table and then load table entries: # mv /etc/pf.tor_net.table /etc/pf.tor_net.table.BaK # touch /etc/pf.tor_net.table # pfctl -t tor_net -T flush 201703 addresses deleted. # pfctl -vf /etc/pf.conf # pfctl -t tor_net -T replace -f /etc/pf.tor_net.table.BaK So loading all entries in to empty table works fine, but reloading didn't work. Miroslav Lachman ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On 14 Jun 2018, at 19:40, Dave Horsfall wrote: I can't get access to kernel sauce right now, but I'm hitting over 1,000 entries from woodpeckers[*] etc; is there some upper limit, or is it just purely dynamic? aneurin% freebsd-version 10.4-RELEASE-p9 Ian already gave some good information, but it’s important to note that there are a number of different limits, and the maximum number of states is different from the limit on table sizes. There’s no immediate limit to the number of addresses in a table. It mostly depends on having enough memory. On 12 you may start to run into issues loading it in one go once you have more than 65k entries. If you do run into that, that particular limit can be tuned using `sysctl net.pf.request_maxcount` Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On 06/14/2018 01:40 PM, Dave Horsfall wrote:
I can't get access to kernel sauce right now, but I'm hitting over
1,000 entries from woodpeckers[*] etc; is there some upper limit, or
is it just purely dynamic?
aneurin% freebsd-version
10.4-RELEASE-p9
You're ultimately physically bound by memory, however there are
configurable limits, see pf.conf(5):
set timeout { \
adaptive.start X, \
adaptive.end Y \
}
set limit states AA
set limit frags BB
set limit src-nodes CC
I've run pf with over 1.5M states, but the limits do have to be tuned.
Ian
[*]
A fairly loose definition in the anti-spammer community, but it
includes attempts every few *seconds* when they encounter my
RFC-compliant banner, when I make 'em wait a bit for my 220, and those
who regard 5xx as a challenge.
Perhaps I should consider an external firewall; at the moment the
(consumer-grade) router allows only certain services to certain
servers (and doesn't bother logging the rejects, much to my disgust)
and its "IP blocking" simply doesn't work, so the mail server blocks
the spammer IPs instead (entire countries where necessary).
-- Dave, who has been accused of being an "anti-spam nazi"
___
freebsd-pf@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.or
--
___
freebsd-pf@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Is there an upper limit to PF's tables?
I can't get access to kernel sauce right now, but I'm hitting over 1,000 entries from woodpeckers[*] etc; is there some upper limit, or is it just purely dynamic? aneurin% freebsd-version 10.4-RELEASE-p9 [*] A fairly loose definition in the anti-spammer community, but it includes attempts every few *seconds* when they encounter my RFC-compliant banner, when I make 'em wait a bit for my 220, and those who regard 5xx as a challenge. Perhaps I should consider an external firewall; at the moment the (consumer-grade) router allows only certain services to certain servers (and doesn't bother logging the rejects, much to my disgust) and its "IP blocking" simply doesn't work, so the mail server blocks the spammer IPs instead (entire countries where necessary). -- Dave, who has been accused of being an "anti-spam nazi" ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
