Darren Reed wrote:
> What I am looking for is a new packet event,
> not the reuse of an existing one, that has different
> semantics to those used for packet filtering (firewall.)
> A load balancr needs to interact with the packet in a
> different way to a firewall/NAT, so I see no reason why
> hooks designed to support those specific features should
> be reused for load balancing or as the template from
> which to create a load balancing packet event.
My understanding is that the pfhooks project provided a rather general
approach for packet *filter* hooks, and they seem to work quite well for
that purpose. You seem to think of them as "generic IP packet hooks". I
don't think they were designed for that purpose in mind; I certainly
didn't review them as such. And such generality might not be useful.
>> There were concerns raised that the lack of a general-purpose hook
>> mechanism (despite the claims that there is already such a mechanism, it
>> doesn't appear to be the case) is leading to exploding complexity in the
>> IP implementation. Erik noted that the existence of a general-purpose
>> hook mechanism would not necessarily address the complexity problem due
>> to the sensitive nature of the interactions between the different hook
>> consumers.
That was only one of my points.
We have several cases when we use hard-code code paths to plug in
different functionality in IP. IPsec is a good example of that. One
could in principle argue that IPsec should have used a "generic IP
packet hooks" interface instead of hard-coding callouts from IP to
IPsec. But I think we did the right thing there, since IPsec is quite an
integral part of the IP layer and we need a tight and evolving coupling
between the two.
I also pointed out in the meeting that we might implement future things
that would likewise benefit from hard-coded callouts. Mobile IP and
shim6 are things that it wouldn't make sense to implement using some
generic "IP packet hooks" framework, for the same reasons that IPsec
doesn't fit.
I think L3/L4 LB fits in the same category even though its interactions
with IP is currently simpler than e.g., IPsec or MIP. If we look at the
micro-architecture of ip_input() we see that one of the things it needs
to do is to determine the next-hop of the packet. This is determined
from a source route option or the IP destination field.
An L3/L4 LB affects this. In particular, when doing DSR i.e. the packet
is not modified, this is the only effect the LB has; the nexthop is
determined from the LB decision. But that micro-architecture has to be
such that we get predictable behavior when combined with a 1) source
route option and/or 2) a pfhooks user which NATs the packet.
Hence the order is critical.
Jim pointed out that this could be done using pfhooks plus the net
routeto/inject. But that would make L3/L4 LB be (yet another) bolt on
which doesn't fit well which seems counter to our need to reduce and
manage the implementation complexity in the IP stack.
> If the general purpose mechanism isn't general
> purpose enough then it behooves us to document
> what its failings are and try to improve it.
I do think we should document the issues of how to use pfhooks for
packet filter and NAT. But I don't think we should try to incrementally
turn it into something it wasn't intended to be ("IP hooks").
FWIW I've told you of some of the issues a while back.
1. It isn't clear which hook entrypoints the hook user can modify the
packet and have IP take notice (e.g., to route it based on a changed
destination.) I think we should just document that this can only be done
for "physical_in_event".
2. It would be good of the result of the hook was different when the
packet was modified as opposed to passed through unmodified, since that
would help detect incorrect cases of #1 and also allow optimizations in
ip_input.
3. Having multiple consumers on the same hook where more than one of
them modify the packet seems problematic. I don't know if we have any
cases where folks want to do this,. If we do at least the order is
critical. But it might also be that user2 needs to be aware that user1
modified the packet. [My, perhaps incorrect, understanding of Linux is
that they manage such complexity by having iptables be the only user of
the hook and which allows them to extend and modify iptables to do that
ever they need. Hence they can avoid ships-in-the-night modifications of
the same packet.]
> Additionally, we should make a better attempt to
> use existing interfaces and improve them as required
> (especially when the idea is to promote them for
> external use) rather than baking new, private, ones
> every time we have a slightly different programming
> task.
You make a very general statement above, which might read like
motherhood and apply pie.
In any particular case I think it requires some careful engineering
judgment to determine what can be reused and the trade offs. Reusing the
round hole for the square peg is known to be problematic.
Erik
