Re: [PATCH] tproxy2 patch to the apache 2.2.15
hello paul~ sorry for my late reply. - Original Message - From: Paul Fee p...@talk21.com To: dev@httpd.apache.org Sent: Friday, August 13, 2010 9:18 PM Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15 JeHo Park wrote: snip yes, i see, so i also made tproxy4 apache patch to the version httpd 2.2.9 and tested it in debian linux box successfully!. the software version i tested looks below -- kernel: vanilla 2.6.31 [tproxy4 included as default ] apache: 2.2.9 [tproxy4 patch applied] iptables: 1.4.3 ebtables: 2.0.8 -- i tested the tproxy4 apache successfully in the debian lenny. but i met some strange things that was .. the same tproxy4 software did not operated correctly in the CentOS the main Environment me and our team developed in is not the debian but the CentOS so i had to give up the tproxy4. this is why i made the tproxy2 apache patch... in the kernel 2.6.18 CentOS kernel :-( Can you share your tproxy4 based patches. I think they're more interesting as they'll work across more distributions in the future. here is my tproxy4 patch actually speaking, i modified and fixed a patch file which i downloaded from the google svn. http://211.174.184.69/apache-tproxy4 RHEL6 beta has tproxy4 support, as will CentOS6 in time. Your tproxy4 work will become usable when your main environment upgrades. good news :-) thanks Here's a post showing tproxy history, it recommends against tproxy2: https://lists.balabit.hu/pipermail/tproxy/2008-November/000994.html Bazsi suggests starting with tproxy4 for 2.6.17 and propagate that forward to a 2.6.18 kernel. The tproxy4 API looks easier to use than tproxy2. forex- Unfortunately I didn't find the tproxy4 for 2.6.17 kernel patch. really ? great! i didn't know that ! Hopefully you can locate the tproxy4 for 2.6.17 patch as that would allow Apache to work consistently in both your environment and with 2.6.28+ kernels. but it seems wondering whether Bazsi do backport the tproxy4 kernel patch to the kernel 2.6.17 or 2.6.18 anyway recently, i applied my tproxy2 patch - exactly speaking, i modified or inserted some little bit codes to the existing patch --- to a commercial sites and then i found ..maybe .. tproxy2 is not real transparency.. because i had to insert some route infomations to the box for packet routing problems. However most important is to have future proof Apache changes that will be compatible with distros other than just CentOS5/RHEL5, for example RHEL6. Although you're tied to CentOS5 now, I think Apache trunk would benefit more from tproxy4 patches. The tproxy2 work has a limited future. i see what you mean ~ Incidentally, how are you managing the iptables rules? Is it assumed that these will be setup before Apache httpd is started? Or do you think Apache should own the rules, creating them at startup and removing them on shutdown. yes, i see, both tproxy2 and tproxy4 need some L2 bridge, L3 or route rules by the iptables and etc so i always insert the rules before or after starting apache httpd. and i hope Apache don't own the rules. i call the deletion of the rules from the box as software bypass :-) i think it is not needed the Apache httpd own the rules .. for more easy debugging and other usages .. Handling the iptables rules within Apache would present difficulties. For example if Apache died/crashed, the rules could be left lingering. Perhaps yes it is really disaster it's best not to pollute Apache with operation system networking setup, especially non-portable settings that are unique to Linux. i understand what you said Thanks, Paul Thanks JeHo Park
Re: [PATCH] tproxy2 patch to the apache 2.2.15
Hello Paul ~ - Original Message - From: Paul Fee p...@talk21.com To: dev@httpd.apache.org Sent: Thursday, August 12, 2010 6:59 PM Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15 JeHo Park wrote: hello Daniel thanks your interest. - Original Message - From: Daniel Ruggeri drugg...@primary.net To: dev@httpd.apache.org Sent: Wednesday, August 04, 2010 9:11 AM Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15 On 8/3/2010 9:57 AM, JeHo Park wrote: hello ~ it's my first mail to apache dev .. and i am beginner of the apache. :-) Anyway ... recently, i wrote transparent proxy [tproxy2] patch to the httpd-2.2.15 because i needed web proxy and needed to know the source address of any client who try to connect to my web server and after all, i tested the performance of my patched tproxy with AVALANCHE 2900. if anyone ask me the performance result, i will send it to him [the size of the test result pdf is big size] *- here is the platform infomation this patch applied ---* 1. OS CentOS release 5.2 (Final) 2. KERNEL Linux version 2.6.18-194.el5-tproxy2 (r...@localhost.localdomain mailto:r...@localhost.localdomain) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #10 SMP Wed May 26 17:35:19 KST 2010 3. iptables iptables-1.3.8 + tproxy2 supporting patch *-- here is the usage of tproxy2 patched httpd configuration ---* httpd.conf VirtualHost 192.168.200.1:80 ProxyTproxy On # On/Off flag ProxyTPifaddr 192.168.200.1 # IP address of bridge interface br0. example) br0 = eth0 + eth1 /VirtualHost i attach the kernel tproxy2 patch to the kernel above[2.6.18-194.el5-tproxy2 ], httpd-2.2.15 tproxy2 patch and kernel configuration for tproxy2 above all, i want to know my patch is available or not .. and want feedback from anyone :-) JeHo; Hi, can you help me understand what the usage case is for this patch? as far as i know, there is another modules for IP transparency for example tproxy4 and X-Forwarded-For ...etc. but tproxy4 is only available from kernel version 2.6.24 and above X-Forwarded-For make the L3, L4 security box unavailable, because the main function of the x-Forwarded-for is to make the web server know client IP address, we can't sure whether there are some another security box [L3, L4 ..firewall ] between the proxy and web server, in this point, X-Forwarded-For make the security box unavailable. What service or capability does it provide that is not currently available? i just tested the patch in my local network. it worked right and i did performance test with the avalanche. but i didn't test it in field .. and various network environment. so i hope so many people use, test this patch -- Daniel Ruggeri Hi JeHo, Thank you for sharing your patches. I was unable to use your Apache patches on Fedora 13 (kernel 2.6.33). I didn't use your kernel patch since tproxy4.1 was merged into the Linux kernel at 2.6.28. yes i see, from my memory, since vanilla kernel 2.6.24, tproxy2 could not be applied to the kernel maybe...it seemed that the tproxy2 core was divided into the inet socket + L3 route code by david miller (netdev maintainer of the linux kernel as you already know) and .. they called it as the tproxy4 ...i can't sure You've patched tproxy2 into the CentOS/RHEL 2.6.18 kernel. tproxy2 behaves differently from tproxy4.1 hence it's to be expected that your userspace patches doesn't work with 2.6.28+ kernels. yes, i see, so i also made tproxy4 apache patch to the version httpd 2.2.9 and tested it in debian linux box successfully!. the software version i tested looks below -- kernel: vanilla 2.6.31 [tproxy4 included as default ] apache: 2.2.9 [tproxy4 patch applied] iptables: 1.4.3 ebtables: 2.0.8 -- i tested the tproxy4 apache successfully in the debian lenny. but i met some strange things that was .. the same tproxy4 software did not operated correctly in the CentOS the main Environment me and our team developed in is not the debian but the CentOS so i had to give up the tproxy4. this is why i made the tproxy2 apache patch... in the kernel 2.6.18 CentOS kernel :-( Here's a post showing tproxy history, it recommends against tproxy2: https://lists.balabit.hu/pipermail/tproxy/2008-November/000994.html Bazsi suggests starting with tproxy4 for 2.6.17 and propagate that forward to a 2.6.18 kernel. The tproxy4 API looks easier to use than tproxy2. forex- Unfortunately I didn't find the tproxy4 for 2.6.17 kernel patch. really ? great! i didn't know that ! but it seems wondering whether Bazsi do backport the tproxy4 kernel patch to the kernel 2.6.17 or 2.6.18 anyway recently, i applied my tproxy2 patch - exactly speaking, i modified or inserted some little bit codes to the existing patch --- to a commercial sites and then i found ..maybe .. tproxy2 is not real transparency.. because i had to insert some route infomations to the box for packet
Re: [PATCH] tproxy2 patch to the apache 2.2.15
hello clere - Original Message - From: jean-frederic clere jfcl...@gmail.com To: dev@httpd.apache.org Sent: Wednesday, August 04, 2010 5:32 PM Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15 On 08/03/2010 04:57 PM, JeHo Park wrote: hello ~ it's my first mail to apache dev .. and i am beginner of the apache. :-) Interesting stuff... But: - The machine depend stuff in httpd usually goes to APR. (I would add the logic to APR and have a HAVE_APR_TPROXY*). i absolutely understood what you said - The kernel is nice but was it accepted in the current kernels? If yes since when? no, i just ported tproxy2 kernel patch [refer the linke below] to the CentOS kernel 2.6.18-194.el5 the mainstream of linux kernel applied tproxy4 from the version 2.6.24 but tproxy2 had not been applied .. http://www.balabit.com/downloads/files/tproxy/ - Without the performance results it is hard to see if it is worth the effort. please check the link below http://211.174.184.69/kisa-avalanche2900-20100712 and the test-result.ppt files shows the summarized result of the performance test and etc Cheers Jean-Frederic
Re: [PATCH] tproxy2 patch to the apache 2.2.15
hello Junyong Jiang
client -conn 1- tproxy - conn 2 -- web server
211.x br0 : 10.a
10.b
as you know previously, there are 2 connections,
one is the connection between client and tproxy and the other is between tproxy
and web server
and if you configure bridged's IP address, tproxy server binds with this
address and random ports and
make connection with a client and server with this bridged IP address
but during run time, web server receives some packets whose source address is
real client address (211.x)
how about check out proxy_util.c .. modules/proxy/proxy_util.c
// snip --//
here! /* use bridged IP address as binding address */
sin.sin_family = AF_INET;
sin.sin_addr.s_addr = inet_addr (conf-tproxy_ifaddr);
sin.sin_port = 0;
if(bind(dst-fd, (struct sockaddr *) sin, sizeof(sin)) == -1) {
ap_log_error(APLOG_MARK, APLOG_ERR, 0, s,
%s/%d Z-linuxpark-2: Error bind (): [ifaddr:%s]: %d:%s\n,
__FUNCTION__, __LINE__, conf-tproxy_ifaddr, errno,
strerror (errno));
return -2;
}
here! /* use client address as source address using SOL_IP ipv4 socket
option */
itp.op = TPROXY_ASSIGN;
itp.v.addr.faddr = src-addr.sin_addr;
itp.v.addr.fport = ntohs (src-addr.sin_port);
ret = setsockopt(dst-fd, SOL_IP, IP_TPROXY, itp, sizeof(itp));
if (ret 0 ) {
#if 0
if (errno == EADDRINUSE)
goto exit;
#endif
ap_log_error(APLOG_MARK, APLOG_ERR, 0, s,
%s/%d Z-linuxpark-3: Error setsockopt (): %d:%s:
[fd: %d, src: %u.%u.%u.%u port(%d)] \n,
__FUNCTION__, __LINE__, errno, strerror (errno),
dst-fd,
NIPQUAD (src-addr.sin_addr.s_addr), src-addr.sin_port);
return -3;
}
and then try to connect !
itp.op = TPROXY_FLAGS;
itp.v.flags = ITP_CONNECT;
ret = setsockopt(dst-fd, SOL_IP, IP_TPROXY, itp, sizeof(itp));
if (ret 0 ) {
ap_log_error(APLOG_MARK, APLOG_ERR, 0, s,
%s/%d Z-linuxpark-4: Error setsockopt (): %d:%s\n,
__FUNCTION__, __LINE__, errno, strerror (errno));
return -4;
}
// snip --//
- Original Message -
From: Junyong Jiang
To: JeHo Park
Sent: Thursday, August 05, 2010 11:26 AM
Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15
So I know!
In this case, what's the use of the bridge's IP address?
2010/8/5 JeHo Park jhp...@elim.net
Junyong Jiang, previous my reply included wrong sentence, so i fixed it !
as you know, in transparent proxy mode [or in the tproxy mode], backend
server receives a packet whose source address is real client address
if you set your proxy box as NAT mode, backend server receives a packet
whose source address is the proxy server's bridge IP.
so if you set up my tproxy httpd correctly, backend server will receive a
packet whose source address is real client address
thanks~
- Original Message -
From: Junyong Jiang
To: jhp...@elim.net
Sent: Thursday, August 05, 2010 9:57 AM
Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15
Hello Park,
I want to ask you one more question. In you test of the apache tproxy
mode, on the backend server( that means the real web server), what is the
source IP address? Is it the real client's or the proxy server's bridge IP?
Thanks!
2010/8/5 JeHo Park jhp...@elim.net
hello clere
- Original Message -
From: jean-frederic clere jfcl...@gmail.com
To: dev@httpd.apache.org
Sent: Wednesday, August 04, 2010 5:32 PM
Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15
On 08/03/2010 04:57 PM, JeHo Park wrote:
hello ~
it's my first mail to apache dev .. and i am beginner of the apache.
:-)
Interesting stuff... But:
- The machine depend stuff in httpd usually goes to APR. (I would add
the logic to APR and have a HAVE_APR_TPROXY*).
i absolutely understood what you said
- The kernel is nice but was it accepted in the current kernels? If
yes
since when?
no, i just ported tproxy2 kernel patch [refer the linke below] to the
CentOS kernel 2.6.18-194.el5
the mainstream of linux kernel applied tproxy4 from the version 2.6.24
but tproxy2 had not been applied ..
http://www.balabit.com/downloads/files/tproxy/
- Without the performance results it is hard to see if it is worth the
effort.
please check the link below
http://211.174.184.69/kisa-avalanche2900-20100712
and the test-result.ppt files shows the summarized result
Re: [PATCH] tproxy2 patch to the apache 2.2.15
hello Daniel thanks your interest. - Original Message - From: Daniel Ruggeri drugg...@primary.net To: dev@httpd.apache.org Sent: Wednesday, August 04, 2010 9:11 AM Subject: Re: [PATCH] tproxy2 patch to the apache 2.2.15 On 8/3/2010 9:57 AM, JeHo Park wrote: hello ~ it's my first mail to apache dev .. and i am beginner of the apache. :-) Anyway ... recently, i wrote transparent proxy [tproxy2] patch to the httpd-2.2.15 because i needed web proxy and needed to know the source address of any client who try to connect to my web server and after all, i tested the performance of my patched tproxy with AVALANCHE 2900. if anyone ask me the performance result, i will send it to him [the size of the test result pdf is big size] *- here is the platform infomation this patch applied ---* 1. OS CentOS release 5.2 (Final) 2. KERNEL Linux version 2.6.18-194.el5-tproxy2 (r...@localhost.localdomain mailto:r...@localhost.localdomain) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #10 SMP Wed May 26 17:35:19 KST 2010 3. iptables iptables-1.3.8 + tproxy2 supporting patch *-- here is the usage of tproxy2 patched httpd configuration ---* httpd.conf VirtualHost 192.168.200.1:80 ProxyTproxy On # On/Off flag ProxyTPifaddr 192.168.200.1 # IP address of bridge interface br0. example) br0 = eth0 + eth1 /VirtualHost i attach the kernel tproxy2 patch to the kernel above[2.6.18-194.el5-tproxy2 ], httpd-2.2.15 tproxy2 patch and kernel configuration for tproxy2 above all, i want to know my patch is available or not .. and want feedback from anyone :-) JeHo; Hi, can you help me understand what the usage case is for this patch? as far as i know, there is another modules for IP transparency for example tproxy4 and X-Forwarded-For ...etc. but tproxy4 is only available from kernel version 2.6.24 and above X-Forwarded-For make the L3, L4 security box unavailable, because the main function of the x-Forwarded-for is to make the web server know client IP address, we can't sure whether there are some another security box [L3, L4 ..firewall ] between the proxy and web server, in this point, X-Forwarded-For make the security box unavailable. What service or capability does it provide that is not currently available? i just tested the patch in my local network. it worked right and i did performance test with the avalanche. but i didn't test it in field .. and various network environment. so i hope so many people use, test this patch -- Daniel Ruggeri
