Re: PFIL_HOOKS should be made default in 5.0
Darren Reed wrote: If you make them non-optional, which is what started this thread, then you *are* talking about having to add an option in to get rid of them. Seriously, Terry, how many NO_foo options exist, today ? Any non-zero number of them is too many. Personally, I also dislike the foo options, too... anything that requires me to recompile something to use it annoys me. 8-). I understand that people all want their pet software to run out of the box without modification. I'm not the one who wants that, it's people who USE FreeBSD. You remember users, don't you Terry ? :) Yeah... those are the people we've been writing the new installer for, for the last 6 years, but never finishing it, right? Seriously, the problem is that not enough work has been done to allow segmentation of the fastpath, with accessor/mutator functions, with zero or low cost. The closest you can really get to doing the right thing in the current FreeBSD codebase is to use existing hooks for things that are already there, and eat the overhead you were going to eat anyway. That works out to no new overhead, abuse existing overhead, work to eliminate existing overhead in future releases. A good example of terrible overhead that should not be there is that the IPv4 code bloats the per connection data horribly, when IPSEC is enabled, but not being used. Another good example is in the ip_input code calling a function that does a pcb hash lookup for divert/fast bridging, and then calls it again in tcp_input, rather than passing the lookup, that was already completed, in as part of the context for the tcp_input. Ideally, there would be a combined has space that ipfilter, the bridging code, DSR, TCP splicing, the SYN cache, and everybody else who wanted to do connection state based decisions would use. The only reason I even commented was that it seems to me that there was an attempt at an end-run to get more overhead in for some excuse other than the real one. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
On Fri, Dec 20, 2002 at 11:29:19PM +0200, Vallo Kallaste vallo wrote: Yes, and this undefined symbols message will make no sense from user perspective. Then fix it. The fix is trivial: [description of possible fix snipped] As I've stated several times and as you most certainly know I'm not developer. What are you trying to accomplish by the phrase then fix it? Put me down, eh? I have encountered this problem several times and for the first time the message about unresolved symbol(s) made no sense and forced me to do time consuming searches over the 'Net to get a clue what's going on. Will we want to get possible users using FreeBSD or will we want argue about it to death? The users get bored and move on, that's it. Uh, sorry Terry. I was lightly drunk (just got back from party) yesterday when I wrote this. Althought the writing has some right points (from my side of view), the overall tone is rude. I'm so sorry. -- Vallo Kallaste [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
Sergey Mokryshev wrote: I'm really not a fan of NO_PFIL_HOOKS as an option. I'm not talking about NO_PFIL_HOOKS but options PFIL_HOOKS in GENERIC. Too many people may foot shoot themselves trying to upgrade from 4-STABLE to 5.0. If you make them non-optional, which is what started this thread, then you *are* talking about having to add an option in to get rid of them. I understand that people all want their pet software to run out of the box without modification. Probably the correct thing to do is to wire in ipfilter as a Netgraph module. AFAIK Solaris, HP-UX and others lack Netgraph support, but support pfil. They support Streams, instead. Same ecological niche. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
On Sat, 21 Dec 2002, Terry Lambert wrote: Sergey Mokryshev wrote: I'm really not a fan of NO_PFIL_HOOKS as an option. I'm not talking about NO_PFIL_HOOKS but options PFIL_HOOKS in GENERIC. Too many people may foot shoot themselves trying to upgrade from 4-STABLE to 5.0. If you make them non-optional, which is what started this thread, then you *are* talking about having to add an option in to get rid of them. I understand that people all want their pet software to run out of the box without modification. I did not start this thread :-) I've filled a PR a while ago. Since PFIL code is optional - let it be. IMHO it is good to keep the same behaviour of the default installations between versions, but entries in UPDATING, RELEASE NOTES and, probably later, FAQ will ease the transition. Probably the correct thing to do is to wire in ipfilter as a Netgraph module. AFAIK Solaris, HP-UX and others lack Netgraph support, but support pfil. They support Streams, instead. Same ecological niche. Never get a chance to dig in. Perhaps in the future. Darren states that PFIL code was derived from NetBSD so there are no licensing issues. Sergey Mokryshev. -- Sergey S. Mokryshev [EMAIL PROTECTED] SMP453, MOKR-RIPN To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
Darren Reed wrote:
This is a reasonable argument... if it's possible to tune it so
that it's fast. Hacking in the IP Filter hooks unonditionally
for code that can't really be distributed as part of the system
because of its license, and thus making things slower, with no
chance to make them faster later, is not my idea of A Really
Good Thing(tm).
I don't understand this paragraph at all.
The original posting in this thread gave a patch to unconditionalize
the PFIL_HOOKS thing, so that the ipfilter module could load on a
default kernel, without having to do a reasonable amount of work.
The initial reason claimed for doing this was to avoid the error
message about unresolved symbols, when trying to load the ipfilter
module. In other words, it was a complaint that the messages were
not specific to the problem, but were rather the generic messages.
My response to that was that you could create accessor/mutator
functions which were always defined, but not always functional.
By using these functions, instead of trying to access the pointers
directly, there are no undefined symbols, and you get the specific
error message, rather than the generic: any message you want it to
print.
But the complaint was just an ecuse. The real reason is that the
people complaining don't want to have to recompile the kernel to
use the ipfilter module: the complaint was because they wanted it
to be resolved in a particular way, so that they got a result that
they weren't willing to ask for directly. Solving the first one
left them unhappy.
That's fine; now that the truth has come out, instead of being
coyly hidden in a side issue, it's obvious what's wanted.
But compiling the kernel with the hooks in place is not the only
way to solve the problem.
Things would be a lot easier if people would ask for what they
want, instead of trying to out-strategize the people they expect
to say no, if they were to ask for what they really wanted. 8-(.
The purpose of pfil(9) is not to facilitate ipfilter but to act
as a mechanism for anything to filter packets to use it as the
way to receive packets. Ideally ipfw* should also use pfil(9)
and not have those large chunks of code in ip_{in,out}put.c.
Yeah, that's the reason that BPF and netgraph and streams and ...
were invented, too. Wouldn't it be great if everyone adopted the
API I like best? 8-) 8-).
Probably the correct thing to do is to wire in ipfilter as a
Netgraph module.
If/when the joining between layer 2 and layer 3 in the kernel
uses netgraph rather than the current mechanisms, then it would
be appropriate to use netgraph for ipfilter.
That's not a good demand; here's why: There are two types of
data paths: (1) the fast path, and (2) the path for research and
for things that are going to be slow anyway, by their nature.
The ipfilter code is in the second category.
It's really bogus to insist that everything take the slow path
because something slow has to take the slow path.
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
Sergey Mokryshev wrote: Darren states that PFIL code was derived from NetBSD so there are no licensing issues. This is Darren Reed's ipfilter.c code, which he will not allow to be distributed modified, and so Theo got all upset and diked it out of OpenBSD , and then wrote a clone of it, right? There *are* licensing issues; it's an issue of interpretation; I can't believe you missed the explosion on the mailing list over the clarification of interpretation Darren posted... -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
In some email I received from Terry Lambert, sie wrote: Sergey Mokryshev wrote: Darren states that PFIL code was derived from NetBSD so there are no licensing issues. This is Darren Reed's ipfilter.c code, which he will not allow to be distributed modified, and so Theo got all upset and diked it out of OpenBSD , and then wrote a clone of it, right? There *are* licensing issues; it's an issue of interpretation; I can't believe you missed the explosion on the mailing list over the clarification of interpretation Darren posted... I can't believe you missed it all being sorted out in the end :-) (i.e. there is no longer any issue) Darren To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
In some email I received from Terry Lambert, sie wrote: Sergey Mokryshev wrote: I'm really not a fan of NO_PFIL_HOOKS as an option. I'm not talking about NO_PFIL_HOOKS but options PFIL_HOOKS in GENERIC. Too many people may foot shoot themselves trying to upgrade from 4-STABLE to 5.0. If you make them non-optional, which is what started this thread, then you *are* talking about having to add an option in to get rid of them. Seriously, Terry, how many NO_foo options exist, today ? I understand that people all want their pet software to run out of the box without modification. I'm not the one who wants that, it's people who USE FreeBSD. You remember users, don't you Terry ? :) Probably the correct thing to do is to wire in ipfilter as a Netgraph module. AFAIK Solaris, HP-UX and others lack Netgraph support, but support pfil. They support Streams, instead. Same ecological niche. That's STREAMS thank you very much! I'll talk more on that point in another email. Darren To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
In some email I received from Terry Lambert, sie wrote:
[...]
The original posting in this thread gave a patch to unconditionalize
the PFIL_HOOKS thing, so that the ipfilter module could load on a
default kernel, without having to do a reasonable amount of work.
ipfilter being the only code that currently uses that mechanism.
At least I think it's a better approach than using the LKM hooks
present in FreeBSD prior to 5.x
My response to that was that you could create accessor/mutator
functions which were always defined, but not always functional.
By using these functions, instead of trying to access the pointers
directly, there are no undefined symbols, and you get the specific
error message, rather than the generic: any message you want it to
print.
That doesn't work when ipfilter is compiled outside of the kernel
build directory (as often happens), where it is going to be unaware
of whether or not PFIL_HOOKS has been defined for the kernel.
The real reason is that the
people complaining don't want to have to recompile the kernel to
use the ipfilter module:
That or any other module that uses the pfil(9) interface.
the complaint was because they wanted it
to be resolved in a particular way, so that they got a result that
they weren't willing to ask for directly. Solving the first one
left them unhappy.
But compiling the kernel with the hooks in place is not the only
way to solve the problem.
It is a better step forward than the proposal I've seen you put
forward about accessor hooks and having them prevent the obscure
modload error message.
The purpose of pfil(9) is not to facilitate ipfilter but to act
as a mechanism for anything to filter packets to use it as the
way to receive packets. Ideally ipfw* should also use pfil(9)
and not have those large chunks of code in ip_{in,out}put.c.
Yeah, that's the reason that BPF and netgraph and streams and ...
were invented, too. Wouldn't it be great if everyone adopted the
API I like best? 8-) 8-).
Terry, did you have any hand in writing netgraph ? I've seen hardly
anyone else put their hand up in favour of it so I'm suspecting a
certain amount of bias here :-) :)
Well, for BPF it works. Lots of devices have bpf_*tap()s in them
and nobody bothers about the overhead from that.
For STREAMS you don't need to do anything.
I suppose I'm trying to push pfil(9) into the same kind of role as
BPF.
Probably the correct thing to do is to wire in ipfilter as a
Netgraph module.
If/when the joining between layer 2 and layer 3 in the kernel
uses netgraph rather than the current mechanisms, then it would
be appropriate to use netgraph for ipfilter.
That's not a good demand; here's why: There are two types of
data paths: (1) the fast path, and (2) the path for research and
for things that are going to be slow anyway, by their nature.
Well, let me describe how ipfilter works in a STREAMS environment,
or rather, how version 4.0 of ipfilter works.
To separate the STREAMS gunk out of ipfilter (or as much as possible)
I wrote an equivalent of pfil(9) for HP-UX 11/Solaris. That is a
STREAMS module that gets push'd onto the network device before the
IP module does so all IP traffic goes through pfil(9). IPFilter
registers with pfil(9) to intercept the packets that would otherwise
be going straight through it to the IP routines.
For it to be worthwhile to use netgraph on FreeBSD, FreeBSD would need
to be re-engineered such that the linkage between layer 2 devices (all
of the ethernet cards, etc) and layer 3 code (ip, ipv6, etc) was done
through netgraph - or at least that's how I see it. From a security
perspective that's definately how I'd want it to be - the filtering
as the middle man, not some inspector hanging off the side somewhere.
The ipfilter code is in the second category.
It's really bogus to insist that everything take the slow path
because something slow has to take the slow path.
I think you're missing the point about why people want everything to
take the slow path. People are using it for security and when you're
doing that, there is no fast path or slow path, there is only a
secure path.
btw, I did make an effort to read how to use netgraph and I'm shocked.
It looks like even more effort to use than STREAMS and not in a good
way either.
Darren
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
On Thu, Dec 19, 2002 at 08:46:44PM -0800, Sam Leffler [EMAIL PROTECTED] wrote: #ifndef PFIL_HOOKS #error You must specify PFIL_HOOKS when using ipfilter #endif Unfortunately there's no way that I know to express this if ipfilter is loaded as a module. Duh, there'll probably be unresolved symbols if you try to load ipl.ko w/o PFIL_HOOKS defined in the kernel. Yes, and this undefined symbols message will make no sense from user perspective. -- Vallo Kallaste [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
Vallo Kallaste wrote:
On Thu, Dec 19, 2002 at 08:46:44PM -0800, Sam Leffler [EMAIL PROTECTED] wrote:
#ifndef PFIL_HOOKS
#error You must specify PFIL_HOOKS when using ipfilter
#endif
Unfortunately there's no way that I know to express this if ipfilter is
loaded as a module.
Duh, there'll probably be unresolved symbols if you try to load ipl.ko w/o
PFIL_HOOKS defined in the kernel.
Yes, and this undefined symbols message will make no sense from
user perspective.
Then fix it. The fix is trivial:
1) Create accessor/mutator functions which are *not*
protected by #ifndef PFIL_HOOKS.
2) Use the accessor/mutator functions from within the
ipfilter code to register and deregister use of the
hooks.
3) In the case of the #ifndef PFIL_HOOKS, cause the
accessor/mutator functions to return errors.
Like so:
#ifdef PFIL_HOOKS
pfil_hook_t *pfil_hook_func;
#endif
int
pfil_set_hook( pfil_hook_t *func)
{
#ifdef PFIL_HOOKS
pfil_hook_func = func;
return( 0);
#else /* !PFIL_HOOKS */
return( -1);
#endif /* !PFIL_HOOKS */
}
...the (in this example) mutator function will always be defined,
and the error in a kernel without PFIL_HOOKS defined can be
trapped by the caller during the module registration phase, at
which point a console error can be printed.
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
On Fri, Dec 20, 2002 at 08:30:42AM -0800, Terry Lambert [EMAIL PROTECTED] wrote: Vallo Kallaste wrote: On Thu, Dec 19, 2002 at 08:46:44PM -0800, Sam Leffler [EMAIL PROTECTED] wrote: #ifndef PFIL_HOOKS #error You must specify PFIL_HOOKS when using ipfilter #endif Unfortunately there's no way that I know to express this if ipfilter is loaded as a module. Duh, there'll probably be unresolved symbols if you try to load ipl.ko w/o PFIL_HOOKS defined in the kernel. Yes, and this undefined symbols message will make no sense from user perspective. Then fix it. The fix is trivial: [description of possible fix snipped] As I've stated several times and as you most certainly know I'm not developer. What are you trying to accomplish by the phrase then fix it? Put me down, eh? I have encountered this problem several times and for the first time the message about unresolved symbol(s) made no sense and forced me to do time consuming searches over the 'Net to get a clue what's going on. Will we want to get possible users using FreeBSD or will we want argue about it to death? The users get bored and move on, that's it. -- Vallo Kallaste [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
On Fri, 20 Dec 2002, Vallo Kallaste wrote: On Fri, Dec 20, 2002 at 08:30:42AM -0800, Terry Lambert [EMAIL PROTECTED] wrote: Vallo Kallaste wrote: On Thu, Dec 19, 2002 at 08:46:44PM -0800, Sam Leffler [EMAIL PROTECTED] wrote: #ifndef PFIL_HOOKS #error You must specify PFIL_HOOKS when using ipfilter #endif Unfortunately there's no way that I know to express this if ipfilter is loaded as a module. Duh, there'll probably be unresolved symbols if you try to load ipl.ko w/o PFIL_HOOKS defined in the kernel. Yes, and this undefined symbols message will make no sense from user perspective. Then fix it. The fix is trivial: [description of possible fix snipped] As I've stated several times and as you most certainly know I'm not developer. What are you trying to accomplish by the phrase then fix it? Put me down, eh? I have encountered this problem several times and for the first time the message about unresolved symbol(s) made no sense and forced me to do time consuming searches over the 'Net to get a clue what's going on. Will we want to get possible users using FreeBSD or will we want argue about it to death? The users get bored and move on, that's it. Unfortunately nobody cares to look into PR database (conf/44576) In case PFIL_HOOKS really slows IP processing I don't mind keeping this out of GENERIC, however it should be noted in UPDATING and release notes. I did not do any time consuming searches the first time I tried to load ipl.ko, but I've spent some time reading NOTES before upgrading to -CURRENT and I am using IP Filter for about three years now on Solaris and FreeBSD (thanks, Darren). IMHO GENERIC is not supposed to be fast, but to be useable out-of-the box. Bringing in PFIL_HOOKS with the appropriate comment about things being more complicated and slow will ease 4.0-5.0 upgrade and do not prevent someone from building custom kernel without it. Sincerely yours, Sergey Mokryshev. -- Sergey S. Mokryshev [EMAIL PROTECTED] SMP453, MOKR-RIPN To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
On Fri, 20 Dec 2002, Terry Lambert wrote: Sergey Mokryshev wrote: Unfortunately nobody cares to look into PR database (conf/44576) In case PFIL_HOOKS really slows IP processing I don't mind keeping this out of GENERIC, however it should be noted in UPDATING and release notes. I did not do any time consuming searches the first time I tried to load ipl.ko, but I've spent some time reading NOTES before upgrading to -CURRENT and I am using IP Filter for about three years now on Solaris and FreeBSD (thanks, Darren). IMHO GENERIC is not supposed to be fast, but to be useable out-of-the box. This is a reasonable argument... if it's possible to tune it so that it's fast. Hacking in the IP Filter hooks unonditionally for code that can't really be distributed as part of the system because of its license, and thus making things slower, with no chance to make them faster later, is not my idea of A Really Good Thing(tm). I'm really not a fan of NO_PFIL_HOOKS as an option. I'm not talking about NO_PFIL_HOOKS but options PFIL_HOOKS in GENERIC. Too many people may foot shoot themselves trying to upgrade from 4-STABLE to 5.0. dd in vi is really quick. We still can remove this options from BOOTMFS kernel to keep it small. Probably the correct thing to do is to wire in ipfilter as a Netgraph module. AFAIK Solaris, HP-UX and others lack Netgraph support, but support pfil. Darren may (or may not) explain it further. IMHO it is almost impossible and pointless (I did not look in the code, just MHO). Sergey Mokryshev -- Sergey S. Mokryshev [EMAIL PROTECTED] SMP453, MOKR-RIPN To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
On Fri, Dec 20, 2002 at 12:27:59PM +1100, Darren Reed wrote the words in effect of:
Well someone has blown my cover from developers and has asked here
what I was trying to be more surrepticious about!
In some email I received from Sam Leffler, sie wrote:
A teeny-weeny issue I would like to discuss, is that we make the pfil(9)
hooks code default in 5.0, and remove the kernel option; this is because
it creates problems when PFIL_HOOKS is not in the (e.g. GENERIC) kernel,
and someone tries to load the ipfilter kernel module (ipl.ko). [1]
I have discussed this with Darren, but would just like to make it
public, so it can be discussed by the release engineers etc. I
apologize but I do not have patches for this.
Enabling PFIL_HOOKS changes various code paths. Doing this so late in the
release cycle is a bad idea. I also recall that there is a performance
penalty (at least in the bridge code) for having this enabled.
There are callouts in both the IPv{4,6} paths for input and output with
PFIL_HOOKS and also bridging.
PFIL_HOOKS is 1 .c file and 1 .h file and a very small amount of code.
Also, given its generic nature, I'd hope that ipfw* could eventually
move to use it for intercepting packets along the above code paths.
The bloat factor from including it in the base kernel should be very
small and perhaps the impact of the code being active in those packet
paths close to immeasurable (I hope.)
Both issues make it seem like it should stay an option for 5.0.
I agree with this.
Maybe we should put in the release notes, that:
PFIL_HOOKS is required for IPFILTER
--
Hiten Pandya ([EMAIL PROTECTED], [EMAIL PROTECTED])
http://www.unixdaemons.com/~hiten/
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
Maybe we should put in the release notes, that: PFIL_HOOKS is required for IPFILTER The right thing is to force the dependency in the code (I don't think there's a way to express it to config). The ipfilter code should probably have something like #ifndef PFIL_HOOKS #error You must specify PFIL_HOOKS when using ipfilter #endif Unfortunately there's no way that I know to express this if ipfilter is loaded as a module. Sam To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: PFIL_HOOKS should be made default in 5.0
Maybe we should put in the release notes, that: PFIL_HOOKS is required for IPFILTER The right thing is to force the dependency in the code (I don't think there's a way to express it to config). The ipfilter code should probably have something like #ifndef PFIL_HOOKS #error You must specify PFIL_HOOKS when using ipfilter #endif Unfortunately there's no way that I know to express this if ipfilter is loaded as a module. Duh, there'll probably be unresolved symbols if you try to load ipl.ko w/o PFIL_HOOKS defined in the kernel. Sam To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
