On Thursday 06 April 2006 16.48, Daniel Hartmeier wrote: > On Thu, Apr 06, 2006 at 09:52:34AM -0400, Peter wrote: > > > Do you know if there is something going on to make this possible? > > > And today the only way is a rule for each customer IP in pf.conf > > > then....? > > > Or are there maybe other tools except labels in PF to make this > > > statistics to > > > work in an easy way? > > > > I'm just a poor user like yourself. To me, it doesn't sound like a big > > change. Maybe Daniel can let us know. > > Obviously, if you want per-IP counters, the kernel needs to allocate > memory for each counter per IP. It should be clear that if you want to > have individual counters for 100,000 addresses, you need to allocate a > memory for those 100,000 counters. Since we're talking about a > non-trivial amount of memory there, there's no way pf will automatically > keep such counters by default, on the off-chance that some users will > actually query some of them.
Yes I understand. But it could be nice if it could be expanded by a special flag in PF. Like "set expansion". Then you enable it manually instead of that it just work and maybe by mistake eat a lot of memory if you have some huge netblocks specified. Maybe it also can be specified within the label statement per rule. Like... pass in quick on $EXTERNAL_INT inet from any to $COLOC_IPS_1 label expand "COLOC_SERVERS:$dstaddr" keep state (note the "expand" above) > > There's two ways to get such counters already. Adding individual rules > per IP is the first one. I know. But that is the last choice.... > This requires more memory (which any solution > will), but also makes ruleset evaluation slower. I agree that it's not > an elegant solution for thousands of IP addresses. I agree... > > The other is address tables. If you change your ruleset to > > table <coloc_ips_1> const { 65.45.128.128, 65.45.128.129, \ > 65.45.128.130, ..., 65.45.128.254, 65.45.128.255 } > > pass in quick on $EXTERNAL_INT inet from any to <COLOC_IPS_1> \ > keep state > > you get counters allocated and updated for every address in the table, > and you can query them with > > # pfctl -t COLOC_IPS_1 -vTs > 65.45.128.128/32 > Cleared: Tue Mar 14 14:22:32 2006 > In/Block: [ Packets: 0 Bytes: 0 ] > In/Pass: [ Packets: 0 Bytes: 0 ] > Out/Block: [ Packets: 0 Bytes: 0 ] > Out/Pass: [ Packets: 0 Bytes: 0 ] > 65.45.128.129/32 > ... This sounds like the best workaround for now. even if I have to add all addresses to a table. Hope this eat less memory than one rule per IP address in PF. > If you want multiple counters for each IP address, like > per-IP-and-protocol counters, you'll need to duplicate the table for > each protocol, i.e. multiple tables containing the same addresses, for > the sake of allocating multiple counters per address: > > table <coloc_ips_1_http> const { 65.45.128.128, 65.45.128.129, \ > 65.45.128.130, ..., 65.45.128.254, 65.45.128.255 } > table <coloc_ips_1_smtp> const { 65.45.128.128, 65.45.128.129, \ > 65.45.128.130, ..., 65.45.128.254, 65.45.128.255 } > ... As this is customer servers we only want totals at the moment... So this is not a problem. > > That's pretty optimal with regards to memory usage. The address itself > is small compared to all the counters associated with it. You mean the above table workaround doesn't slow down the ruleset and also eat less memory? > If you want > separate counters per protocol, you HAVE to allocate the counters per > protocol and address. Allocating the address itself twice is a > relatively small waste. > > So, I consider this a sufficiently elegant (existing!) solution, the > only annoying thing is that you have to manually enumerate all IPs > within the netblock. > > This could be improved by adding a little syntactic sugar to pfctl, > introducing some optional syntax for table additions, like > > table ... { 10.1.1.1, 10.2.2/24*, 10.2.3/24 } > > where the '*' means 'don't add the netblock itself, but instead generate > and insert all individual address within that netblock), i.e. the above > table would then contain the entries > > 10.1.1.1 > 10.2.2.0 > 10.2.2.1 > 10.2.2.2 > ... > 10.2.2.255 > 10.2.3/24 when can we have that syntactic sugar in the code ;-) > > This doesn't HAVE to be done by pfctl itself, you can generate the list > automatically with jot(1) or similar, > but I guess it might be nice. Correct. It would be really nice. Can it be a future feature request? > > Anything beyond this, like 'I want to track a whole /8, but I don't have > the memory to pre-allocate 2^24 counters, I want counters allocated on > demand for those addresses actually seen', is NOT a simple change. > > Daniel Thanks in advance /Per-Olov