Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
Hi Evgeniy, On Wednesday 03 January 2007 18:23, Evgeniy Polyakov wrote: > Out of curiosity, would you use netchannels [1] if the implementation > will be much broader? Since what you have created works exactly like > netchannels netfilter NAT target (although it does not change ports, > but it can be trivially extended), but without all existing netfilter > overhead and without hacks in core TCP/UDP/IP/route code. Indeed, a netchannels based implementation would be very nice. Combined with a userspace network stack I think this could be a very powerful tool, especially for people doing dirty tricks -- like transparent proxying in our case. However, I think that adopting netchannels now would be an enormous work on our part. Of course, personally I'm really interested in netchannels and the related projects, but I agree with Harald that we still have a long way to go before being able to switch to netchannels. And I definitely _hate_ the previous incarnations of our tproxy patches enough that even this patchset seems acceptable for me. ;) -- Regards, Krisztian Kovacs - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
On Sun, Jan 07, 2007 at 05:11:06PM +0100, Lennert Buytenhek wrote: > On Sun, Jan 07, 2007 at 03:11:34PM +0100, Harald Welte wrote: > > > > So instead of using NAT to dynamically redirect traffic to local > > > addresses, we now rely on "native" non-locally-bound sockets and do > > > early socket lookups for inbound IPv4 packets. > > > > It's good to see a solid implementation of this 'old idea'. > > > > Just as a quick historical note to netdev: This is the way how the > > netfilter project advised the balabit guys to implement fully > > transparent proxy support, after having seen the complexity of the old > > nat-based TPROXY patches. > > Didn't rusty tell the balabit guys to use the NAT approach? that was originally, way back. It turned out to be a bad idea, after all... way too complex. At least that's how I look at it. Too sad :( Rusty and me then had the idea about the routing based approach at some point, if I remember correctly. We talked about it with Krisztian and Balazs at least on one occasion. All that isn't really important. All I wanted to say was: "I (and AFAIR the netfilter core team) believe this is the way to implement good support for transparent proxying. It's already the second completely independent implementation, let's merge it after all." -- - Harald Welte <[EMAIL PROTECTED]> http://netfilter.org/ "Fragmentation is like classful addressing -- an interesting early architectural error that shows how much experimentation was going on while IP was being designed."-- Paul Vixie signature.asc Description: Digital signature
Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
On Sun, Jan 07, 2007 at 03:11:34PM +0100, Harald Welte wrote: > > So instead of using NAT to dynamically redirect traffic to local > > addresses, we now rely on "native" non-locally-bound sockets and do > > early socket lookups for inbound IPv4 packets. > > It's good to see a solid implementation of this 'old idea'. > > Just as a quick historical note to netdev: This is the way how the > netfilter project advised the balabit guys to implement fully > transparent proxy support, after having seen the complexity of the old > nat-based TPROXY patches. Didn't rusty tell the balabit guys to use the NAT approach? - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
Hi Krisztian! On Wed, Jan 03, 2007 at 05:33:57PM +0100, KOVACS Krisztian wrote: > So instead of using NAT to dynamically redirect traffic to local > addresses, we now rely on "native" non-locally-bound sockets and do > early socket lookups for inbound IPv4 packets. It's good to see a solid implementation of this 'old idea'. Just as a quick historical note to netdev: This is the way how the netfilter project advised the balabit guys to implement fully transparent proxy support, after having seen the complexity of the old nat-based TPROXY patches. So I personally support this patchset and vote for it to be included (with whatever modifications netdev deems apropriate) It might be that there now is the experimental netchannels system which might provide an even better way for transparent proxy support. However, ever since ip_tables was merged in the 2.3.x days, we have lacked good support for transparent proxies. Now that the first incarnation of the NAT based TPROXY patch for 2.4.x had to be developed and maintained out-of-tree for many years, I definitely think it's better to merge the new, way less intrusive, patchset. Some interested party can work on a netchannels implementation later on, but that's the next generation... Cheers, -- - Harald Welte <[EMAIL PROTECTED]> http://netfilter.org/ "Fragmentation is like classful addressing -- an interesting early architectural error that shows how much experimentation was going on while IP was being designed."-- Paul Vixie signature.asc Description: Digital signature
Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
On Thu, Jan 04, 2007 at 01:13:27PM +0100, KOVACS Krisztian wrote: > > I'd also love to see the old tproxy API go away entirely. It was > > always a bit of a pain to use. > > It's gone with these patches: all you need is to bind() to foreign > addresses, like in the Linux 2.2 days. That's how I understood it. Great. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
Hi, On Wednesday 03 January 2007 20:33, Lennert Buytenhek wrote: > I'd also love to see the old tproxy API go away entirely. It was > always a bit of a pain to use. It's gone with these patches: all you need is to bind() to foreign addresses, like in the Linux 2.2 days. -- Regards, Krisztian Kovacs - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
On Wed, Jan 03, 2007 at 05:33:57PM +0100, KOVACS Krisztian wrote: > The following set of patches implement transparent proxying support > loosely modeled on the Linux 2.2 transparent proxying functionality. In a transparent http proxy server I wrote a while ago, we used to use tproxy for making outgoing connections appear to be originating from a foreign IP address, but moved to inserting an iptables nat rule from the proxy app every time an outgoing connection needs to be made, due to the pain of having to patch in the tproxy patches every time we needed to do a kernel update. I'd love to see working tproxy functionality merged upstream for that reason alone. I'd also love to see the old tproxy API go away entirely. It was always a bit of a pain to use. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC 00/10] Transparent proxying patches version 4
On Wed, Jan 03, 2007 at 05:33:57PM +0100, KOVACS Krisztian ([EMAIL PROTECTED]) wrote: > The following set of patches implement transparent proxying support > loosely modeled on the Linux 2.2 transparent proxying functionality. > > In the last few years we've been maintaining a set of patches > implementing Netfilter NAT to provide similar functionality. However, > as time passed, more and more bugs surfaced, some of which were not > possible to fix using that approach. Also, those patches required > modification of user-space application code and the "API" provided was > neither clean nor easy to use. > > So instead of using NAT to dynamically redirect traffic to local > addresses, we now rely on "native" non-locally-bound sockets and do > early socket lookups for inbound IPv4 packets. These lookups are done > in a separate Netfilter/iptables module, so there are only negligible > performance implications of building transparent proxying support as a > module and then not loading it. Out of curiosity, would you use netchannels [1] if the implementation will be much broader? Since what you have created works exactly like netchannels netfilter NAT target (although it does not change ports, but it can be trivially extended), but without all existing netfilter overhead and without hacks in core TCP/UDP/IP/route code. Some quote for netfilter maillist on behalf of advertisement :) Network channel is peer-to-peer protocol agnostic communication channel between hardware and userspace. They allow to work directly with network hardware from userspace without any kind of filtering or processing from kernel. Network channels are organized into single multidimensional trie, which allows to perform route, netfilter and other types of lookups in one traversal, since it is completely protocol agnostic. Layering model does not exist in netchannels - layers are the way to design protocols, not implement them, thus actual protocol processing happens on the ends of netchannel - for example in userspace (userspace network stack), which improves cache locality and reduce overhead of unneded layer crossing. Netchannels featureset includes: * multidimensional wildcards support * RCU searching * single multidimensional trie for different kinds of dataflows * dedicated processing threads with possibility to schedule processing on different CPUs for those netchannel types which are not acked with processing context * userspace netchannel backend (allows to receive packets to userspace), which can be used for: o high-performance sniffers o tun/tap device replacement o packet socket replacement (note, that netchannels steal packets from main stack) o userspace network stack implementation [2] o own protocol stack implementaion (from VPN tunnels to TOE) * netfilter netchannel backend (only NAT is supported as the most interesting user, NAT caches appropriate route, so essentially routing becomes part of the netchannel trie) 1. Netchannels homepage. http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel 2. Userspace network stack. http://tservice.net.ru/~s0mbre/old/?section=projects&item=unetstack 3. Netchannel vs. socket benchmarks. http://tservice.net.ru/~s0mbre/blog/2006/10/26#2006_10_26 http://tservice.net.ru/~s0mbre/blog/2006/12/21#2006_12_21 4. Netchannels multidimensional wildcard trie testing. userspace test (scales to millions of end nodes) http://tservice.net.ru/~s0mbre/blog/2006/12/02#2006_12_02 kernelspace test (tens of thousands of netchannels) http://tservice.net.ru/~s0mbre/blog/2006/12/21#2006_12_21_1 -- Evgeniy Polyakov - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH/RFC 00/10] Transparent proxying patches version 4
The following set of patches implement transparent proxying support loosely modeled on the Linux 2.2 transparent proxying functionality. In the last few years we've been maintaining a set of patches implementing Netfilter NAT to provide similar functionality. However, as time passed, more and more bugs surfaced, some of which were not possible to fix using that approach. Also, those patches required modification of user-space application code and the "API" provided was neither clean nor easy to use. So instead of using NAT to dynamically redirect traffic to local addresses, we now rely on "native" non-locally-bound sockets and do early socket lookups for inbound IPv4 packets. These lookups are done in a separate Netfilter/iptables module, so there are only negligible performance implications of building transparent proxying support as a module and then not loading it. Small modifications were also necessary in IP/TCP/UDP core code to support the Netfilter modules. All those have been functionally split out into stand-alone patches among which there are no direct dependencies. Among these changes are ones which I think might be potentially risky, especially the core IPv4 routing code changes. Also please note that at the moment only IPv4 support is implemented, but opposed to the NAT-based approach taken by older TProxy versions IPv6 support is possible this way. Comments welcome... -- Regards, Krisztian Kovacs - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
