Updated TPROXY patch - squid-2.5-cttproxy-04JES.diff

2004-04-12 Thread James MacLean
Hi folks,

This applies to squid-2.5.STABLE5 and adds support for Squid transparent 
proxy support via http://www.balabit.com/downloads/tproxy/linux-2.4/devel/

It is based on the work of Gianni Tedesco, and changed with the help of 
KOVACS Krisztian.

I have been using it to redirect traffic from two sites successfully, but 
can not comment on how well it will scale :).

The intention of these patches is to proxy squid requests to servers with
the client IP as the source. Normally it is Squid's IP that is the source 
of all requests.

I am not on the dev list.

Enjoy,
JES
--

These are patches based on the work of: 

Gianni Tedesco 

and updated with the help of and many thanks to :

KOVACS Krisztian 

This patch adds a new onoff config "linux_tproxy" (if you configure with
 --enable-linux-tproxy) which, when set, will spoof the source address of
outgoing server connections to be the same as the original client
address. It now also works with persistent sever connections. To take
advantage of this code you need Linux 2.4/2.6 with netfilter enabled and the
TPROXY patches installed from:

http://www.balabit.com/downloads/tproxy/linux-2.4/devel/

There are two small nits with the code:

 1. You must supply a tcp_outgoing_address in your squid.conf, this is
because of some deep magic in the Linux TCP/IP stack.
 2. Squid must run as root in order to do the connection spoofing bits.
For me (JES) this meant setting :
cache_effective_user root root
cache_effective_group root

NOTES:
 . You do not need to add any iptable rules to make this part work in the Linux Kernel.
 . You still need to "-j REDIRECT 3128" to get initial packets to squid. (Or use tproxy
   reidrect method - see examples in cttproxy README)
 . In squid.conf add :
   linux_tproxy on
   tcp_outgoing_address 
 . You do have to run autoconf to add --enable-linux-tproxy to configure.
 . Example configure:
   ./configure \
   --prefix=/usr/local/squid \
   --enable-linux-netfilter \
   --enable-linux-tproxy \
   --enable-async-io \
   --disable-http-violations \
   --enable-underscores \
   --disable-hostname-checks \
   --enable-ntlm-auth-helpers=SMB \
   --enable-auth=ntlm \
   --enable-basic-auth-helpers=NCSA \
   --enable-storeio=ufs,aufs,diskd \
   --enable-err-languages="English French"

TODO:
  o Port all changes to cvs HEAD branch.
  o Attempt to fix connect(2) problem in kernel which requires bind(2) to
local address. (JES - did not investigate if this still stands).
  o Hack kernel space code so squid does not need to run as root.

--
JES 

--- squid-2.5.STABLE5/acconfig.h2002-07-01 05:55:11.0 -0300
+++ squid-2.5.STABLE5-tproxy/acconfig.h 2004-03-15 10:45:59.0 -0400
@@ -331,6 +331,11 @@
  */
 #undef LINUX_NETFILTER
 
+/* 
+ * Enable real Transparent Proxy support for Netfilter TPROXY.
+ */
+#undef LINUX_TPROXY
+
 /*
  * Do we have unix sockets? (required for the winbind ntlm helper
  */
--- squid-2.5.STABLE5/configure.in  2004-02-29 18:30:21.0 -0400
+++ squid-2.5.STABLE5-tproxy/configure.in   2004-03-15 10:45:59.0 -0400
@@ -739,6 +739,17 @@
   fi
 ])
 
+dnl Enable Linux transparent proxy support
+AC_ARG_ENABLE(linux-tproxy,
+[  --enable-linux-tproxy
+  Enable real Transparent Proxy support for Netfilter 
TPROXY.],
+[ if test "$enableval" = "yes" ; then
+   echo "Linux Netfilter/TPROXY enabled"
+   AC_DEFINE(LINUX_TPROXY)
+   LINUX_TPROXY="yes"
+  fi
+])
+
 AM_CONDITIONAL(MAKE_LEAKFINDER, false)
 dnl Enable Leak Finding Functions
 AC_ARG_ENABLE(leakfinder,
@@ -1181,6 +1192,7 @@
libc.h \
limits.h \
linux/netfilter_ipv4.h \
+   linux/netfilter_ipv4/ip_tproxy.h \
malloc.h \
math.h \
memory.h \
@@ -1814,6 +1826,27 @@
 sleep 10
 fi
 
+dnl Linux Netfilter/TPROXY support requires some specific header files
+dnl Shamelessly copied from shamelessly copied from above
+if test "$LINUX_TPROXY" ; then
+AC_MSG_CHECKING(if TPROXY header files are installed)
+# hold on to your hats...
+if test "$ac_cv_header_linux_netfilter_ipv4_ip_tproxy_h" = "yes"; then
+LINUX_TPROXY="yes"
+AC_DEFINE(LINUX_TPROXY, 1)
+else
+LINUX_TPROXY="no"
+AC_DEFINE(LINUX_TPROXY, 0)
+fi
+AC_MSG_RESULT($LINUX_TPROXY)
+fi
+if test "$LINUX_TPROXY" = "no" ; then
+echo "WARNING: Cannot find TPROXY headers, you need to install the"
+echo "tproxy package from:"
+echo " - lynx http://www.balabit.com/downloads/tproxy/linux-2.4/";
+sleep 10
+fi
+
 if test -z "$USE_GNUREGEX" ; then
 case "$host" in
 *-sun-solaris2.[[0-4]])
--- squid-2.5.STABLE5/include/autoconf.h.in 2003-01-17 21:46:33.0 -0400
+++ squid-2.5.STABLE5-tproxy/include/autoconf.h.in  2004-03-15 10:45:59.0 
-0400
@@ -350,6 +350,11 @@
  */
 #undef LINUX_NETFILTER
 
+/* 
+ * Enable real Transparent Proxy support for Netfilter TPROXY.
+ */
+#undef LINUX_TPROXY
+
 /*
  * Do 

Re: ETag support

2004-04-12 Thread Henrik Nordstrom
On Sun, 11 Apr 2004, Mati wrote:

> 2. our whole project is continously being recorded on our site 
> http://rainbow.mimuw.edu.pl/~ms189442/mmsquid we invite everyone 
> interested to visit the site - any suggestions are welcome.

Have been reading your side and patch conclusions, and you seem to have 
identified the problematic spots.

The store management in Squid-3 is significantly different from that of 
Squid-2.5. This part is most likely better rewritten from scratch than 
trying to port the patch.

And regarding Vary... the two go intimately together. There is not very
much value in implementing ETag if not having Vary using it.. If all you 
want it to implement the missing ETag functionality with no respect to 
Vary then the patch is basically not needed. The bulk of the patch 
implements the Vary+ETag combo.

While developing the ETag patch I tried to collect relevant notes on how 
these headers go together at http://devel.squid-cache.org/etag/

Regards
Henrik



Re: ETag support

2004-04-12 Thread Henrik Nordstrom
On Sun, 11 Apr 2004, Mati wrote:

> 1. because we are working on ETags, it would be very useful for us to 
> have our own access to CVS. what should we do to gain it?

See http://devel.squid-cache.org/howto.html and
http://devel.squid-cache.org/CVS.html. We are using the SourceForge
services for the public developer CVS trees and web space, together with a
set of scripts to make source version tracking considerably easier.

> 2. our whole project is continously being recorded on our site
> http://rainbow.mimuw.edu.pl/~ms189442/mmsquid we invite everyone
> interested to visit the site - any suggestions are welcome.

Regarding documentation.. Squid-3.0 is considerably more documented than
Squid-2.5 was as Robert kindly has spent a great deal of time on
documenting the new designs while refactoring and redesigning to C++. This
documentation is found in the (incomplete) programmers guide found in the
doc/Programming-Guide directory.

Yes, there is a lot of difference compared to Squid-2.5, and it is not 
practical to document the difference. Instead we try to focus on 
documenting the current code starting with the new or rewritten parts.

If there is any questions on the code, old or new or patched, please ask. 
Will try to help explaining things.

Regards
Henrik