NP Daniel,
i'm happy that more people are getting into it.
i have done my research now and i have used the next:
fedora| 16 p2p1(eth0) 192.168.15.100 gw 192.168.15.1
router\fw debian squeeze| eth1 192.168.15.1
2.6.32-5-amd64 | eth2 192.168.16.1
| eth0 192.168.10.107 gw 192.168.10.201
same debian as proxy |eth0 192.168.16.100 gw 192.168.16.1
squid 3.2.0.17
the 192.168.10.201 gw is a Gentoo AIO machine connected to DSL
and masquerading.
AIO = (proxy dhcp dns storage testing)
the AIO has a static route to the 192.168.15+16.0/24 nets using the
192.168.10.107
stay with me because it's complicated.
i used iptables marks and custom routing tables
iptables -t mangle -N FROMINT
iptables -t mangle -A FROMINT --jump MARK --set-mark 3
iptables -t mangle -N FROMPXY
iptables -t mangle -A FROMPXY --jump MARK --set-mark 2
iptables -t mangle -N TOPXY
iptables -t mangle -A TOPXY --jump MARK --set-mark 1
iptables -t mangle -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 --jump
TOPXY
iptables -t mangle -A PREROUTING -i eth2 -p tcp -m tcp --dport 80 --jump
FROMPXY
iptables -t mangle -A PREROUTING -i eth0 -p tcp -m tcp --sport 80 --jump
FROMINT
these rules are marking the packets so the routing level can identify them
mark 1 is from client on eth1 and destined to port 80\http
mark 2 is the "next hop" from the proxy destined to the web
mark 3 is the packets that coming from the web server back to the client
but needed to be routed into the proxy back.
then i assigned every mark a routing table to handle it.
i had to create the routing tables in /etc/iproute2/rt_tables
##start
100 topxy
120 frompxy
140 fromint
255 local
254 main
253 default
##end
ip rule add fwmark 1 table topxy
ip rule add fwmark 2 table frompxy
ip rule add fwmark 3 table fromint
now this is a very difficult part to understand
anything that comes and destined to the proxy must go to the proxy and
should not have any other routing options so the topxy table should have
only route to the proxy
so:
ip route add default via 192.168.16.100 dev eth2 table topxy
but if anything comes from the proxy it should be sent to destination so
we copy the original routes using:
ip route show table main | while read ROUTE ; do ip route add table
frompxy $ROUTE; done
i could have used the same table for topxy and fromint because it has
the same rule that redirects the traffic to the proxy but i like it
organized so:
ip route add default via 192.168.16.100 dev eth2 table fromint
this is very good for a real routing environment but if you start nating
it will get messy. why ???
because a packet that was in the kernel before will not be
natted\masqueraded on the output interface.
i dont know if the problem is in the kernel\iptables\routing or what.
so there is a problem with natting.
i'm still wondering on th ipv6 thing
has anyone tried to intercept ipv6 traffic using tproxy??
Eliezer
On 01/06/2012 01:10, Daniel Niasoff wrote:
Hi All,
I'm jumping in the middle here but I have tproxy working with a separate router
as follows;
Here are the rules from my router.
# Don.t mark webcache traffic
$IPTABLES -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s $SQUID
# Internal subnets to exclude
$IPTABLES -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -d 192.168.0.0/16
#Don.t cache internal
$IPTABLES -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s 192.168.0.0/16
#Don.t cache internal
# Now mark our traffic
$IPTABLES -t mangle -A PREROUTING -j MARK --set-mark 5 -p tcp --dport 80
ip rule add fwmark 5 table 5
ip route add default via $SQUID dev bond0.8 table 5
On my squid box I have the following Iptables rules
*mangle
:PREROUTING ACCEPT [47356:2123379]
:INPUT ACCEPT [44233:3551720]
:FORWARD ACCEPT [14057:711976]
:OUTPUT ACCEPT [27932:3507208]
:POSTROUTING ACCEPT [42005:4222687]
:DIVERT - [0:0]
-A PREROUTING -p tcp -m socket -j DIVERT
-A PREROUTING -p tcp -m tcp --dport 80 -j TPROXY --on-port 3129 --on-ip 0.0.0.0
--tproxy-mark 0x1/0x1
-A DIVERT -j MARK --set-xmark 0x1/0xffffffff
-A DIVERT -j ACCEPT
COMMIT
#This makes sure that the traffic comes back to this squid box.
*nat
:PREROUTING ACCEPT [627:29853]
:POSTROUTING ACCEPT [46:2562]
:OUTPUT ACCEPT [93:5582]
-A POSTROUTING -s 10.0.0.0/8 -o eth0 -p tcp -m tcp --dport 80 -j SNAT
--to-source #SQUIDIP
COMMIT
ip -f inet rule add fwmark 1 lookup 100
ip -f inet route add local default dev eth0 table 100
This system works well for me and I have multiple squid proxies is a
transparent load balanced config (using Linux virtual server)
I've had 10 or 15 users testing it and with no complaints so far but I had to
use the latest source code (not build 3.2.0.17)
Hatzlacha
Daniel
-----Original Message-----
From: Eliezer Croitoru [mailto:elie...@ngtech.co.il]
Sent: 31 May 2012 21:17
To: Thomas York
Cc: squid-users@squid-cache.org
Subject: Re: [squid-users] Linux + TPROXY + Remote Squid
the marking is not all the thing.
you should also use routing tables based on the marking so in the prerouting
mangle you mark and then the routing tables are compatible with the routing
table.
i will think of something.
but it's out of the scope of squid and moving to routing.
Eliezer
On 31/05/2012 19:26, Thomas York wrote:
A TPROXY isn't useless just because I'm using NAT. The whole point of
using TPROXY is that it will also work with IPv6 (since iptables lacks
NAT capability with IPv6, which is fine). I'm marking and diverting
connections from eth2, because that's the interface that has clients connected
to it.
I had a separate table and marking for return port 80 data on the
router (eth0), but it didn't make any difference. I had the same issue
that I have now. As of this point, the only working solution I've seen
is to consolidate the proxy and the router. I REALLY hate to do this,
but at this point it looks like my only working solution.
-- Thomas York
-----Original Message-----
From: Eliezer Croitoru [mailto:elie...@ngtech.co.il]
Sent: Thursday, May 31, 2012 12:03 PM
To: Thomas York
Cc: squid-users@squid-cache.org
Subject: Re: [squid-users] Linux + TPROXY + Remote Squid
well as i was suspecting you made the scenario from a movie.
in the real world you will design it a little different.
here a picture on the net:
http://cloud.ngtech.co.il/public.php?service=files&token=d88ff9e412a47
a2842c b8ac7137c6227f196d8f2&path=/squid-net.png
in yout specific case you are natting the internet access anyway so a
tproxy is useless.
i'm still trying to understand why you are marking and diverting the
connections from eth2.
you have one problem in this specific case.
you are trying to do some marking that will direct the clients into
this router in a loop from the clients to router and then from router
to squid and from squid to box using the same source ip of the client
in this part you will get a big loop.
what you should do is a marking of packets coming from each interface
differently and by this mark define 2 routing tables:
one that marked for squidbox because it came from the eth1 1 another
mark is for packets that are comming on interface eth5 and are to port
80 will be marked 2 and will be routed\nated to the gw on eth0 another
mark is on the
eth0 interface when packets are coming from the http server it should
be routed to squidbox.
if you do ask me i would have put squidbox in the eth0 net and do the
nat on squidbox instead of on the router.
Eliezer
text summary:
win7 eth0 10.1.1.253/24
gw 10.1.1.254
FW
eth0 10.1.17.158/24
gw 10.1.17.254
eth1 10.1.1.254/24
eth5 10.0.1.254/24
iptables:
*nat
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
*mangle
:PREROUTING ACCEPT [126:15633]
:INPUT ACCEPT [126:15633]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [67:8420]
:POSTROUTING ACCEPT [67:8420]
:DIVERT - [0:0]
-A PREROUTING -i eth2 -p tcp -m tcp --dport 80 -j DIVERT -A DIVERT -j
MARK --set-xmark 0x1/0xffffffff -A DIVERT -j LOG --log-prefix "DIVERT
:" --log-level 7 -A DIVERT -j ACCEPT COMMIT
ip route list table 100
default via 10.0.1.1 dev eth5
10.0.1.0/24 dev eth5 scope link
ip route list table main
defautlt via 10.1.17.254 dev eth0
squid
eth0 10.0.1.1/24
gw 10.0.1.254
On 31/05/2012 16:27, Thomas York wrote:
The way you have it set up (the server running Squid also being a
router) works fine. I run in to problems when I try to separate them out.
rp filter is disabled for all interfaces on both the proxy and the router.
Here is my current network configuration for this test bed..
Firewall
------------
Fw (iptables router) has three network interfaces.
eth0 connects to the 'internet'.
eth0 has the IP of 10.1.17.158/24 with a default gateway of
10.1.17.254
eth0 is being NAT'ed to allow the Windows 7 client and proxy access
to the 'internet'
eth1 connects to a Windows 7 client.
eth1 has the IP of 10.1.1.254/24.
eth5 connect to the proxy (proxy01)
eth5 has the IP of 10.0.1.254/24.
root@fw:~# ip route list table 100
default via 10.0.1.1 dev eth5
10.0.1.0/24 dev eth5 scope link
root@fw:~# iptables-save
# Generated by iptables-save v1.4.13 on Thu May 31 09:23:53 2012 *raw
:PREROUTING ACCEPT [1866:289737] :OUTPUT ACCEPT [788:89384] COMMIT #
Completed on Thu May 31 09:23:53 2012 # Generated by iptables-save
v1.4.13 on Thu May 31 09:23:53 2012 *nat :PREROUTING ACCEPT [26:3577]
:INPUT ACCEPT [23:3388] :OUTPUT ACCEPT [5:352] :POSTROUTING ACCEPT
[2:120] -A POSTROUTING -o eth0 -j MASQUERADE COMMIT # Completed on
Thu May 31 09:23:53 2012 # Generated by iptables-save v1.4.13 on Thu
May 31 09:23:53 2012 *mangle :PREROUTING ACCEPT [126:15633] :INPUT
ACCEPT [126:15633] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [67:8420]
:POSTROUTING ACCEPT [67:8420] :DIVERT - [0:0] -A PREROUTING -i eth2
-p tcp -m tcp --dport 80 -j DIVERT -A DIVERT -j MARK --set-xmark
0x1/0xffffffff -A DIVERT -j LOG --log-prefix "DIVERT :" --log-level 7
-A DIVERT -j ACCEPT COMMIT # Completed on Thu May 31 09:23:53 2012 #
Generated by iptables-save v1.4.13 on Thu May 31 09:23:53 2012
*filter :INPUT ACCEPT [1313:186460] :FORWARD ACCEPT [12:815] :OUTPUT
ACCEPT [733:82296] COMMIT # Completed on Thu May 31 09:23:53 2012
root@fw:~# iptables --list -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DIVERT tcp -- anywhere anywhere tcp dpt:http
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain DIVERT (1 references)
target prot opt source destination
MARK all -- anywhere anywhere MARK set 0x1
LOG all -- anywhere anywhere LOG level
debug prefix "DIVERT :"
ACCEPT all -- anywhere anywhere
Windows 7 client
--------------
The client has one network interface with the IP address of
10.1.1.253/24 and a default gateway of 10.1.1.254
Proxy
-------------
Proxy01 (Squid proxy) has one network interface.
eth0 connects to the firewall.
eth0 has the IP of 10.0.1.1/24 and a default gateway of 10.0.1.254.
-----Original Message-----
From: Eliezer Croitoru [mailto:elie...@ngtech.co.il]
Sent: Wednesday, May 30, 2012 11:09 PM
To: squid-users@squid-cache.org
Subject: Re: [squid-users] Linux + TPROXY + Remote Squid
i was curios about it because the last time i setup a tproxy on
debian it took me couple minutes.
i am using debian squeeze 6.0.5 with basic 2.6.32-5-amd64 kernel and
squid
3.1.6 from debian repos and tproxy works fine for me!!
debian installed with squid3 ebtables bridge-utils .
(also tested with self built squid3.2.0.17 and squid3.1.19)
the main thing with tproxy is to allow the VM net card promiscuous
mode
and
on the router machine disable reverse path filter using:
sysctl -a |grep rp_filter
should all be with the value 0
i am still trying to understand what you are doing on each of the servers.
what are the networks and what are the machines and what every
machine
does?
what i got until now was:
W7|eth0[what ip?} -(some net)---> ethX[what ip?]--|debian_router|--[what
ip?]ethX--(some net)-->ethX[what ip?]--|squid_debian|ethX[what
ip?]--->{{{ internet}}}
please fill my gap about ethX numbers and on any MASQUERADING that
happens.
notice that if you are doing DNAT there is not point at all in TPROXY
because the client IP was lost already.
output of:
iptables-save
ip route list
#if you are using some routing tables then also ip rotue show table
table_number_or_name_here
the only problem i have seen is that if i have an established session
from the client and i reload the rules i get this squid error page:
##start
ERROR
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the URL:
(null)://www.cnn.com/
Invalid URL
Some aspect of the requested URL is incorrect.
Some possible problems are:
Missing or incorrect access protocol (should be "http://"; or
similar)
Missing hostname
Illegal double-escape in the URL-Path
Illegal character in hostname; underscores are not allowed.
Your cache administrator is webmaster.
Generated Thu, 31 May 2012 01:17:12 GMT by localhost (squid/3.1.6)
##end
i will check with the latest squid version.
i am using this script to load the iptables rules:
#start
#!/bin/sh -x
##!/bin/sh -x
#load modules requierd for the tproxy modprobe ip_tables modprobe
nf_conntrack_ipv4 modprobe xt_tcpudp modprobe nf_tproxy_core modprobe
xt_MARK modprobe xt_TPROXY modprobe xt_socket
sysctl net.netfilter.nf_conntrack_acct sysctl
net.netfilter.nf_conntrack_acct=1 ip route flush table 100 ip rule
del fwmark 1 lookup 100 ip rule add fwmark 1 lookup 100 ip -f inet
route
add
local 0.0.0.0/0 dev lo table 100
echo "flushing any exiting rules"
iptables -t mangle -F
iptables -t mangle -X DIVERT
echo "creating rules"
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle
-A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket
-j
DIVERT
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
--tproxy-mark
0x1/0x1 --on-port 3129 ##end
this one for ebtables:
#start
#!/bin/sh -x
CLIENT_IFACE="eth1"
INET_IFACE="eth0"
ebtables -t broute -F
ebtables -t broute -A BROUTING -i $CLIENT_IFACE -p ipv4 --ip-proto
tcp --ip-dport 80 -j redirect --redirect-target ACCEPT ebtables -t
broute -A BROUTING -i $INET_IFACE -p ipv4 --ip-proto tcp --ip-sport
80 -j redirect --redirect-target ACCEPT
ebtables -t broute -A BROUTING -i eth1 -p ipv4 --ip-proto tcp
--ip-dport
80 -j redirect --redirect-target DROP ebtables -t broute -A BROUTING
-i eth0 -p ipv4 --ip-proto tcp --ip-sport
80 -j redirect --redirect-target DROP
cd /proc/sys/net/bridge/
for i in *
do
echo 0> $i
done
unset i
#end
Eliezer
On 25/05/2012 17:35, Thomas York wrote:
I have a lab environment set up using two Debian Wheezy servers
(Squeeze doesn't have a new enough kernel or iptables to do TPROXY
properly). One of the servers is a router and the other is a proxy
server. There are several clients connected to the router to
simulate a production routing environment. If I have both the TPROXY
redirection and Squid on the same server, Squid handles the requests
and everything works perfectly. However, this isn't how I want the
proxy to be configured in our production environment. I've changed
my iptables rules on the router to redirect all tagged 1 packets to
the proxy server. This is working perfectly fine and I can see the
data being routed to the proxy server using tcpdump on both the
router and the proxy. However, Squid on the proxy server doesn't seem to 'see'
the data being routed and doesn't do anything with it. I have
"http_port 3129 tproxy" set on the proxy server. Is there anything
special
I need to do using iptables on the proxy server?
Both servers are running kernel 3.2.0-2-amd64 and iptables 1.4.13
from Wheezy and the Squid being used on the proxy is 3.1.19. If any
more information is needed, please just let me know and I'd be happy
to supply it. Thanks.
--Thomas York
--
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations eliezer<at> ngtech.co.il
--
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations eliezer<at> ngtech.co.il
--
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer <at> ngtech.co.il
