Re: Brige, Traffic Shaping and FTP
> I believe what is being requested is a kind of state engine that is > smart enough to modify packets on the fly and recognize "related" > packets. This is common in many commercial firewalls and also in > SunScreen and Netfilter/IPTables. Okay. To avoid confusion from here on in, take this as a definition: Def'n: (Kernel Proxy) A kind of state engine, implemented in the kernel that is smart enough to modify packets on the fly and recognize "related" packets. The naming comes from the ipf kernel-proxy for FTP. > I would argue that this is not quite the same as a proxy. However, > now-a-days, the difference between a stateful/NAT firewall and a Proxy is a > fine line and somewhat subjective. No. It's not a proxy. It's a "kernel proxy." We are ALL talking about the same thing here. > > Yes. Since the kernel sees traffic a packet at a time, it essentially > > has to "fake" the statefulness of TCP in order to track these things. > > It's a layer violation. It's also next to impossible, without re- > > implementing TCP. > What I don't get, is if what you are saying is true, how do commercial > firewall products work? They don't have access to the O/S source code and > manage. What am I missing? They implement a kernel proxy. They also implement kernel proxy bugs. (c.f. explots for both ipf and checkpoints' FTP proxies, allowing the malicious user to create arbitrary firewall states) > How is it a layer violation? Can't you link in with a callback or > something? Why do you have to re-implement TCP? Because you have to inspect traffic at the TCP level in order to determine what "related" traffic to allow through the firewall. (i.e. monitor the FTP control connection). Unfortunately, you are looking at packets as they arrive (i.e. the IP layer). If packets all arrived uniquely, un-fragmented, and in-order, this would be straigtforward. They do not. See Ptacek and Newsham's paper for a thousand reasons why this is a bad assumption. > In the end, I think Daniel is right. It sounds like a cool feature, but > someone would have to step up or help fund it. Perhaps some day, I'll learn > enough to be able to contribute... And if they are going to step up, they should think about scrub, bpf, and userland. Not kernel and payload inspection. Now, I've said my peace. Back to slackin' -kj
RE: Brige, Traffic Shaping and FTP
> > In fact, even if it does not really matter to you in fact, I'm not > > talking about a kernel "proxy" here. I'm talking about something smart > > enough to tag packets "related" and so to "pass" them. > > A piece of kernel code smart enough to inspect packets comprising > a partular protocol, and extract enough information to determine if > they are, in fact "related" is exactly what Henning is referring to > as a kernel "proxy" > Having used many different firewalls, I think I see the argument here. Traditionally a proxy is something that would "block" the flow of traffic. The packets would terminate at the proxy, the proxy would inspect them up to the application layer/Layer 7. The proxy would then initiate the conversation to the "outside" on behalf of the client. I believe what is being requested is a kind of state engine that is smart enough to modify packets on the fly and recognize "related" packets. This is common in many commercial firewalls and also in SunScreen and Netfilter/IPTables. I would argue that this is not quite the same as a proxy. However, now-a-days, the difference between a stateful/NAT firewall and a Proxy is a fine line and somewhat subjective. > > If we go on with FTP, a piece of code that attach data connexions to > > the command connexion initiated before. In case of a bridge, I > > clearly do not need (and do not want !) a proxy, nor NAT support. > > Yes. Since the kernel sees traffic a packet at a time, it essentially > has to "fake" the statefulness of TCP in order to track these things. > It's a layer violation. It's also next to impossible, without re- > implementing > TCP. This is EXACTLY why the decision to make these kinds of per-protocol > decision was relegated to userland in the first place: because TCP will > handle the state issues, before userland sees the packets. > This is very interesting to me. Perhaps I just don't know enough to understand what you are saying. I have done some socket and kernel programming, but I'm no Daniel or Henning. What I don't get, is if what you are saying is true, how do commercial firewall products work? They don't have access to the O/S source code and manage. What am I missing? How is it a layer violation? Can't you link in with a callback or something? Why do you have to re-implement TCP? I do think if you were going to do something like this, a framework would have to be agreed upon. Perhaps it could be a userland framework although you'd have to look at performance and scalability. > Think fragment attacks. Think of the attacks against IPF's ftp kernel > code. Think of EVERY ambiguity that scrub was intended to help correct. > > We had this debate long ago. The kernel way is the WRONG way. Writing > a userland proxy for a particular protocol is by definition easier > than writing the kernel equivalent, since you don't have to figure out > what > the TCP will do with ambiguities. > It is definitely true that the more kernel code you have, the higher the risks are. The traditional problem with userland programs (from my understanding) is that they don't scale. That seems to be a big part of why proxies are slow. That's why many people are trying to cache web pages in the kernel--it's faster. Don't get me wrong, I'm not necessarily advocating this position. It's a trade off between performance and security. I'm sure Henning is pulling his hair out reading this... :) In the end, I think Daniel is right. It sounds like a cool feature, but someone would have to step up or help fund it. Perhaps some day, I'll learn enough to be able to contribute... <> Jim
Re: Brige, Traffic Shaping and FTP
On Mon, Mar 01, 2004 at 11:21:55PM +0100, Julien Bordet wrote: > Again, I agree. But that does not resolve the issue. Please consider the > case I'm talking about (Bridge, ...). What we need is a OpenBSD > solution. We do one of the best packet filter available, make bridging > surprisingly easy, but the killer feature is missing. This topic comes up again and again. I think it's clear by now that most pf developers have no personal need for this feature. I guess it's mostly commercial environments with paying customers who can't be inconvenienced with scp that would like to have it. But it seems it's not worth enough to pay a salary writing, even if it might save a couple of dollars on appliance/license costs. If I'm wrong, and a lot of people with more skill and time than money would want it, surely someone would be writing it by now. Especially if it was "the missing killer feature". The reasoning that this could be easily done in pf in the kernel is just wrong, it would be about the same amount of work doing it in userland (think bpf listener and raw socket sender) on a bridge, and that would be more secure and portable. So, don't push the burden on others, start working on it today ;) Daniel
Re: Brige, Traffic Shaping and FTP
On Mon, Mar 01, 2004 at 11:21:55PM +0100, Julien Bordet wrote: > As I said, there may a user land solution. Some kind of global user > space "advisor" daemon, helping packet filter to make complicated > decisions, for example. Having a userland program doing blocking operations on kernel packet flow is not feasible. The way to do this is by completely passing the packets to the userland process, and having the userland process re-inject the packets. No real API change would be needed to do something like this. Configure PF to block and log the packets that you're interested in, and have a userland process that watches pflog and uses BPF to send out the packets that you actually want passed.
Re: Brige, Traffic Shaping and FTP
[EMAIL PROTECTED] wrote: A piece of kernel code smart enough to inspect packets comprising a partular protocol, and extract enough information to determine if they are, in fact "related" is exactly what Henning is referring to as a kernel "proxy" Ok, that is what Henning calls a proxy. But this is not my definition : for me a proxy receive connexion from a client, and initiates another connexion to the server. In that case there are two connexions, instead of one when only firewalling. Anyway, you know that, and that is not the most important. Yes. Since the kernel sees traffic a packet at a time, it essentially has to "fake" the statefulness of TCP in order to track these things. It's a layer violation. It's also next to impossible, without re-implementing TCP. This is EXACTLY why the decision to make these kinds of per-protocol decision was relegated to userland in the first place: because TCP will handle the state issues, before userland sees the packets. Think fragment attacks. Think of the attacks against IPF's ftp kernel code. Think of EVERY ambiguity that scrub was intended to help correct. We had this debate long ago. The kernel way is the WRONG way. Writing a userland proxy for a particular protocol is by definition easier than writing the kernel equivalent, since you don't have to figure out what the TCP will do with ambiguities. Again, I agree. But that does not resolve the issue. Please consider the case I'm talking about (Bridge, ...). What we need is a OpenBSD solution. We do one of the best packet filter available, make bridging surprisingly easy, but the killer feature is missing. In fact, I do not care whether it is in the kernel or not. The only thing I'm sure is that it cannot be a proxy (in my definition). As I said, there may a user land solution. Some kind of global user space "advisor" daemon, helping packet filter to make complicated decisions, for example. henning wrote: you have a solution. it is called ftp-proxy. But that's not bridging any longer... Julien
Re: Brige, Traffic Shaping and FTP
* Ed White <[EMAIL PROTECTED]> [2004-03-02 00:11]: > On Monday 01 March 2004 22:22, Henning Brauer wrote: > > the only place to solve this is obviously writing a proxy. > > wether that is in kernel or not doesn't change a shit. > > well, except for the tiny detail that a security problem in your > > userland proxy doesn't give the attacker remote root... and it easier > > to write too. > Henning, I don't understood if you're talking about the same thing I > proposed... i understand very well, and that "idea" is either horribly broad open and thus insecure, or needs understanding of the application level protocol -> we have a proxy -- Henning Brauer, BS Web Services, http://bsws.de [EMAIL PROTECTED] - [EMAIL PROTECTED] Unix is very simple, but it takes a genius to understand the simplicity. (Dennis Ritchie)
Re: Brige, Traffic Shaping and FTP
* Julien Bordet <[EMAIL PROTECTED]> [2004-03-01 23:31]: > Henning Brauer wrote: > >* Julien Bordet <[EMAIL PROTECTED]> [2004-03-01 21:35]: > >>However, when one does bridge traffic shaping, this is not the same thing > >>at all : proxifying means that your are not bridging any more, using a IP > >>address for the bridge, and so on. I really think it is a very dirty > >>solution. The kernel space solution here is much cleaner, as it is > >>transparent for the firewall administrator. > >you are so wrong. > >doing this kind of proxying in-kernel is just plain wrong, > >and error-prone. > In fact, even if it does not really matter to you in fact, I'm not > talking about a kernel "proxy" here. I'm talking about something smart > enough to tag packets "related" and so to "pass" them. that is a proxy in my eyes. in any case this shares the problems with ipf's in kernel proxy and linux' netfilter gunk. a bugtraq archive near you shows them they are somewhat popular there. > Yet, I'm talking about a feature we need. Bridging with a certain > "understanding" of the FTP protocol is clearly needed. nah > And yes FTP is a > crappy protocol,. I'm not a I-want-everything-in-the-kernel guy, I 'd > like a solution. you have a solution. it is called ftp-proxy. -- Henning Brauer, BS Web Services, http://bsws.de [EMAIL PROTECTED] - [EMAIL PROTECTED] Unix is very simple, but it takes a genius to understand the simplicity. (Dennis Ritchie)
Re: Brige, Traffic Shaping and FTP
On Monday 01 March 2004 22:22, Henning Brauer wrote: > the only place to solve this is obviously writing a proxy. > wether that is in kernel or not doesn't change a shit. > well, except for the tiny detail that a security problem in your > userland proxy doesn't give the attacker remote root... and it easier > to write too. Henning, I don't understood if you're talking about the same thing I proposed... I don't want any proxy or application level software in the kernel. I said that PF could support an extension of keep state and I called it permit state, because it permits traffic in the opposite direction (from server to client) until the state created is in the table. The only security problem is related to application that binds on the client. In fact the server could "talk" with client... However this can be easily solved with the help of tagging. Should I post a step by step example ? Ed
Re: Brige, Traffic Shaping and FTP
> In fact, even if it does not really matter to you in fact, I'm not > talking about a kernel "proxy" here. I'm talking about something smart > enough to tag packets "related" and so to "pass" them. A piece of kernel code smart enough to inspect packets comprising a partular protocol, and extract enough information to determine if they are, in fact "related" is exactly what Henning is referring to as a kernel "proxy" > If we go on with FTP, a piece of code that attach data connexions to > the command connexion initiated before. In case of a bridge, I > clearly do not need (and do not want !) a proxy, nor NAT support. Yes. Since the kernel sees traffic a packet at a time, it essentially has to "fake" the statefulness of TCP in order to track these things. It's a layer violation. It's also next to impossible, without re-implementing TCP. This is EXACTLY why the decision to make these kinds of per-protocol decision was relegated to userland in the first place: because TCP will handle the state issues, before userland sees the packets. Think fragment attacks. Think of the attacks against IPF's ftp kernel code. Think of EVERY ambiguity that scrub was intended to help correct. We had this debate long ago. The kernel way is the WRONG way. Writing a userland proxy for a particular protocol is by definition easier than writing the kernel equivalent, since you don't have to figure out what the TCP will do with ambiguities. -kj
Re: Brige, Traffic Shaping and FTP
On Mon, 1 Mar 2004, Julien Bordet wrote: > In fact, even if it does not really matter to you in fact, I'm not > talking about a kernel "proxy" here. I'm talking about something smart > enough to tag packets "related" and so to "pass" them. If we go on with > FTP, a piece of code that attach data connexions to the command > connexion initiated before. In case of a bridge, I clearly do not need > (and do not want !) a proxy, nor NAT support. You mean like a proxy that inserts states for the data connections? oh, hang on...
Re: Brige, Traffic Shaping and FTP
Henning Brauer wrote: * Julien Bordet <[EMAIL PROTECTED]> [2004-03-01 21:35]: However, when one does bridge traffic shaping, this is not the same thing at all : proxifying means that your are not bridging any more, using a IP address for the bridge, and so on. I really think it is a very dirty solution. The kernel space solution here is much cleaner, as it is transparent for the firewall administrator. you are so wrong. doing this kind of proxying in-kernel is just plain wrong, and error-prone. In fact, even if it does not really matter to you in fact, I'm not talking about a kernel "proxy" here. I'm talking about something smart enough to tag packets "related" and so to "pass" them. If we go on with FTP, a piece of code that attach data connexions to the command connexion initiated before. In case of a bridge, I clearly do not need (and do not want !) a proxy, nor NAT support. don't people read bugtraq? don't people learn from all the security problems ipf and the linux guys had with their in-kernel proxies? I do read bugtraq, and yes I'm aware of security problem of ipf and netfilter. Yet, I'm talking about a feature we need. Bridging with a certain "understanding" of the FTP protocol is clearly needed. And yes FTP is a crappy protocol,. I'm not a I-want-everything-in-the-kernel guy, I 'd like a solution. Julien
Re: Brige, Traffic Shaping and FTP
* Julien Bordet <[EMAIL PROTECTED]> [2004-03-01 21:35]: > However, when one does bridge traffic shaping, this is not the same thing > at all : proxifying means that your are not bridging any more, using a IP > address for the bridge, and so on. I really think it is a very dirty > solution. The kernel space solution here is much cleaner, as it is > transparent for the firewall administrator. you are so wrong. doing this kind of proxying in-kernel is just plain wrong, and error-prone. don't people read bugtraq? don't people learn from all the security problems ipf and the linux guys had with their in-kernel proxies? -- Henning Brauer, BS Web Services, http://bsws.de [EMAIL PROTECTED] - [EMAIL PROTECTED] Unix is very simple, but it takes a genius to understand the simplicity. (Dennis Ritchie)
Re: Brige, Traffic Shaping and FTP
* David Chubb <[EMAIL PROTECTED]> [2004-03-01 21:36]: > As it is I can force Tribes 2 to use a range of ports for players and I have > poked holes for that range...this is by far not the best security method > though and not all applications allow for forcing a static range of ports. so? the only place to solve this is obviously writing a proxy. wether that is in kernel or not doesn't change a shit. well, except for the tiny detail that a security problem in your userland proxy doesn't give the attacker remote root... and it easier to write too. -- Henning Brauer, BS Web Services, http://bsws.de [EMAIL PROTECTED] - [EMAIL PROTECTED] Unix is very simple, but it takes a genius to understand the simplicity. (Dennis Ritchie)
RE: Brige, Traffic Shaping and FTP
I think part of the bigger issue are applications other than FTP. I personally would love to host servers for games that dynamically assign outgoing ports to clients after the client authenticates via a static port (example: Tribes 2 uses port 25000 for initial authentication, but after that the server assigns that player a different transfer port for the actual game communication.). As it is I can force Tribes 2 to use a range of ports for players and I have poked holes for that range...this is by far not the best security method though and not all applications allow for forcing a static range of ports. --David Chubb > -Original Message- > From: Julien Bordet [mailto:[EMAIL PROTECTED] > Sent: Monday, March 01, 2004 2:05 PM > To: Ed White > Cc: [EMAIL PROTECTED] > Subject: Re: Brige, Traffic Shaping and FTP > > > Hi, > > I'm not sure whether Ed's idea would be the best way to do > it, but it raises a very good question that makes pf > sometimes not useful as it should be. > > When one setups a firewall, I agree that it can be globally > the same whether FTP is transparent proxified through user > space proxy or directly managed by the kernel. Both cases can > be efficient, and the former is simpler and probably clearer, > as it reduces the amount of protocole specific code into the kernel. > > However, when one does bridge traffic shaping, this is not > the same thing at all : proxifying means that your are not > bridging any more, using a IP address for the bridge, and so > on. I really think it is a very dirty solution. The kernel > space solution here is much cleaner, as it is transparent for > the firewall administrator. Thus he does not have to take > care of the ports used by the FTP protocol. > > The idea of ftpsesame could be good, but it does not seem to > be on the way to inclusion into the tree... > > I really think the OpenBSD bridge/traffic shaper solution is > the best available (by far). But having to proxify FTP or > managing FTP data by ports is such a pain in the neck ... > > Julien >
Re: Brige, Traffic Shaping and FTP
Hi, I'm not sure whether Ed's idea would be the best way to do it, but it raises a very good question that makes pf sometimes not useful as it should be. When one setups a firewall, I agree that it can be globally the same whether FTP is transparent proxified through user space proxy or directly managed by the kernel. Both cases can be efficient, and the former is simpler and probably clearer, as it reduces the amount of protocole specific code into the kernel. However, when one does bridge traffic shaping, this is not the same thing at all : proxifying means that your are not bridging any more, using a IP address for the bridge, and so on. I really think it is a very dirty solution. The kernel space solution here is much cleaner, as it is transparent for the firewall administrator. Thus he does not have to take care of the ports used by the FTP protocol. The idea of ftpsesame could be good, but it does not seem to be on the way to inclusion into the tree... I really think the OpenBSD bridge/traffic shaper solution is the best available (by far). But having to proxify FTP or managing FTP data by ports is such a pain in the neck ... Julien
