Hi Darren,
Per your questions below, I should probably give a clearer
objective statement.
I'm trying to implement blocking of some empirically derived lists
of ip address/subnets known to generate malicious and/or otherwise
undesireable traffic. These lists can be rather large, depending
on which categories of addresses are being blocked - on the order of
10,000 - 100,000 subnet ranges.
Based on my experience with implementing these lists as one
filter rule per address/range, there appeared to be a measurable
connection lag when packets had to be matched against more than about
5,000 rules. This led me to a scheme where I presorted the ranges
and used head/group statements to limit checking to x.x.x.x/8 subnet
ranges. This scheme improved efficiency considerably. These sets of
rules are generated by way of the raw lists of addresses and a
perl script.
The simplest scheme for me to implement is to just replace
the large groups of rules with single group members
blocking x/8 pools, but the groupmap seems like it might
allow me further simplify my scheme. My questions were just
to try to determine how radically I want to try to modify my scheme.
A couple of more questions:
5) Why are pools either "in" or "out"? Is there some way to use a
single pool of addresses to match both in and out addresses?
6) You answered part of my question on 1) below... but
Just to clarify,
when a packet matches and is assigned a group in a groupmap,
this behaves as an "entry point" to a group just like matching
a rule with a head statement to that group?
7) If I have a filter rule like:
block in quick on pppoe0 from pool/333 to any
If I have a very long list of subnets in pool 333, does first match
win or does it traverse the full list before returning?
Because there can be "exceptions" in the pool, I would suspect
it has to traverse the whole thing each time. Again, this leads
me to try to presort the really large lists to reduce redundant
traversals.
Thank you for your patience!
gene
>> Ok, I just sat down and tried to map out an ippool strategy to my
>> problem and have come up with a few follow on questions...
>>
>> 1) It seems from the man page example that grpmap may act
>> like either a second "head" statement or a substitute for a
>> head statement. Without more than a "fragment" of a complete
>> ipf.conf and ippool.conf, it's hard to tell...
>>
>> from the man page...
>>
>> ipf.conf: call now fr_srcgrpmap/1010 in all
>> pass in all group 1020
>> block in all group 1030
>>
>> ippool.conf: group-map in role = ipf number = 1010
>> { 1.1.1.1/32, group = 1020; 3.3.0.0/16, group = 1030; };
>>
>> Does there also need to be statements - head 1020 and
>> head 1030 in the above ipf.conf? Or must there not be?
>
> There is no requirement for "head 1020" or "head 1030" in ipf.conf.
>
>> 2) This starts to make ipf look programmable.
>> Is there a kernel function similar to fr_srcgrpmap
>> that instead maps to a pool?
>
> No.
>> table role = ipf type = tree number = 1255
[snip]
>> 255.129.225.165/32; 255.129.248.128/26; };
>
> I'm not sure I understand what you're trying to do here.
>
>> 3) Is my syntax for empty pools and multi-line input correct above?
>
> Sure, there are no rules about how much needs to be on one line
> or how many lines you use.
>
>> 4) I would take it that ippool records have considerably less
>> kernel memory overhead than filter rules?
>
> I don't know - I haven't measured it.
>
>> What I'm kinda looking for is a universal and efficient way
>> to filter out random addresses/subnets from the entire ipv4 space.
>> I can perform pre-sorting to prevent the entire list from being
>> traversed for each input.
>
> I think you want to define a single pool as having all of these
> addresses and just refernce that?
