Re: [squid-users] TPROXY Routing
I'll check that as well... have to grab some IPs and add to the interfaces. 2010/4/2 Henrik Nordström : > fre 2010-04-02 klockan 11:54 -0700 skrev Kurt Sandstrom: >> Thats the thing... if I enable the ebtables rules the bridging of http >> on the local network stops but squid shows no activity even though the >> tproxy counters increase. > > And if you configure a client to use the TPROXY as a router while having > the ebtables ruels disabled? (routing test) > > Regards > Henrik > >
Re: [squid-users] TPROXY Routing
Thats the thing... if I enable the ebtables rules the bridging of http on the local network stops but squid shows no activity even though the tproxy counters increase. If I wget to 0.0.0.0 my squid shows wget connection but returns a gateway error so I know the squid is replying to requests it receives. I'm thinking perhaps the traffic might be being directed to the bridge route instead of lo which would cause it to die. 2010/4/2 Henrik Nordström : > fre 2010-04-02 klockan 09:47 -0700 skrev Kurt Sandstrom: > >> 2 things I may try this evening... grab tcp traffic from eth0 and br0 >> to see if redirected port 3129 is being routed out of the system >> instead of to the localhost. Then try (a shot in the dark) changing: > > Which MAC address is being used on the packets sent out? > > Have a feeling the packets never gets diverted off the bridge.. if so > then the MAC is unchanged when the packet is sent out. > > If the packet did get diverted from the bridge to routing then the > source MAC of the packets when leaving the server will be that of br0. > > other sign to look for is if the IP ttl gets decremented. If the packet > is being bridged then ttl stays the same, if it's being routed then ttl > is decremented by one. > > Regards > Henrik > >
Re: [squid-users] TPROXY Routing
You are correct in that it's a routing issue... I have network -> eth1(no ip bridged)->eth0(no ip bridged)-> gateway(router) the eth1 and eth0 interfaces have a br0 assigned. when I assign the bridge interface I use the following for routing: ifconfig br0 xxx.xxx.xxx.xxx netmask 255.255.0.0 up #routable IP route add default gw xxx.xxx.xxx.xxx dev br0#gateway Then I use: ebtables -t broute -A BROUTING -i eth1 -p ipv4 --ip-proto tcp --ip-dport 80 -j redirect --redirect-target DROP ebtables -t broute -A BROUTING -i eth0 -p ipv4 --ip-proto tcp --ip-sport 80 -j redirect --redirect-target DROP iptables -t mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 cd /proc/sys/net/bridge/ for i in * do echo 0 > $i done unset i and I think this is where the problem resides but may be wrong: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 My iptables are being traversed and I can see the counters increasing in the PREROUTING chain TPROXY target 2 things I may try this evening... grab tcp traffic from eth0 and br0 to see if redirected port 3129 is being routed out of the system instead of to the localhost. Then try (a shot in the dark) changing: ip route add local 0.0.0.0/0 dev lo table 100 to ip route add local 0.0.0.0/0 dev br0 table 100 If you have any other ideas then please let me know... I know I'm close and the help received here has really helped Kurt I did a couple tests on the system last night. If wget 0.0.0.0:3129(tproxy port) then I see traffic in the squid access.log. I recieve a gateway not found error 2010/4/2 johan firdianto : > Have you setup ebtables to drop packet, > ebtables -t broute -A BROUTING -i $CLIENT_IFACE -p ipv4 --ip-proto tcp > --ip-dport 80 -j redirect --redirect-target DROP > ebtables -t broute -A BROUTING -i $INET_IFACE -p ipv4 --ip-proto tcp > --ip-sport 80 -j redirect --redirect-target DROP > > second hint, > route all your network/netmask ip address to dev bridge, > example: > ip route add 192.168.100.0/24 dev br0 > ip route add 10.0.0.0/8 dev br0 > BUT, if you have router again below your bridge, you should define > routing in your bridge. > Because your box actually act as bridge and router. Act as router > because you intercepted trafic to squid. So, when kernel will forward > the traffic to network, they must know which interface to forward. > > > > 2010/4/2 Henrik Nordström : >> tor 2010-04-01 klockan 13:43 -0700 skrev Kurt Sandstrom: >>> The bridging is working just not redirecting to the squid. I can see >>> the counters increment for port 80 but nothing on the squid side. >> >> TPROXY has some quite peculiar requirements, and the combination with >> bridgeing makes those even more complex. And is why I ask that you first >> verify your TPROXY setup in routing mode before trying the same in >> bridge mode. It's simply about isolating why things do not work for you >> instead of trying to guess if it's the bridge-iptables integration, >> ebtables, iptables TPROXY rules, routing, or whatever.. >> >> Regards >> Henrik >> >> >
Re: [squid-users] TPROXY Routing
The bridging is working just not redirecting to the squid. I can see the counters increment for port 80 but nothing on the squid side. 2010/4/1 Henrik Nordström : > tor 2010-04-01 klockan 11:10 -0700 skrev Kurt Sandstrom: >> It is set up with 2 nics as a bridge. The routing I was refering to is >> only internal to the box.. ie through iptables > > bridge... haven't tried TPROXY in bridge mode, only router mode. > > Due to the complexity involved I would recomment you first try TPROXY in > router mode, then move on to extend it to bridge mode. And remember that > you need to divert the return traffic as well in the bridge or it won't > work. > > Regards > Henrik > >
Re: [squid-users] TPROXY Routing
It is set up with 2 nics as a bridge. The routing I was refering to is only internal to the box.. ie through iptables On Thu, Apr 1, 2010 at 5:09 AM, johan firdianto wrote: > Make sure you have setup triangle routing correctly. > your squid act as bridge ? or act as router/gateway with dual > interface ethernet ? > or standalone server with single ethernet. > option 1 and 2, doesn't need routing setup, traffic incoming and > outgoing must hit the squid box. > But for option 3, you should setup your router to make sure outgoing > traffic to port 80 should hit the squid first, and forward to > internet, and the reply traffic from internet should come back to > squid box before forwarded to client. > > 2010/4/1 Kurt Sandstrom : >> I have the following in startup >> >> ip rule add fwmark 1 lookup 100 >> ip route add local 0.0.0.0/0 dev lo table 100 >> >> The ouput of ip route show table 100: local default dev lo scope host >> >> One other thing is strange, my PREROUTING rules in mangle don't load >> in my script. I have to manually add them. Timing issue perhaps? >> >> Startup script loded from rc.local: >> >> echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter >> echo 1 > /proc/sys/net/ipv4/ip_forward >> iptables -t mangle -N DIVERT >> iptables -t mangle -A DIVERT -j MARK --set-mark 1 >> iptables -t mangle -A DIVERT -j ACCEPT >> iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT >> iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY >> --tproxy-mark 0x1/0x1 --on-port 3129 >> ebtables -t broute -A BROUTING -i eth1 -p ipv4 --ip-proto tcp >> --ip-dport 80 -j redirect --redirect-target DROP >> ebtables -t broute -A BROUTING -i eth0 -p ipv4 --ip-proto tcp >> --ip-sport 80 -j redirect --redirect-target DROP >> cd /proc/sys/net/bridge/ >> for i in * >> do >> echo 0 > $i >> done >> unset i >> >> ip rule add fwmark 1 lookup 100 >> ip route add local 0.0.0.0/0 dev lo table 100 >> >> >> 2010/3/31 Henrik Nordström : >>> ons 2010-03-31 klockan 09:47 -0700 skrev Kurt Sandstrom: >>>> I have been unable to get TPROXY working correctly with squid. I have >>>> used the steps in http://wiki.squid-cache.org/Features/Tproxy4 and re >>>> checked everything. >>>> >>> >>> I did not see your routing setup in the data you dumped. Without the >>> routing configured then TPROXY won't intercept, just route like normal.. >>> >>> http://wiki.squid-cache.org/Features/Tproxy4#Routing_configuration >>> >>> Regards >>> Henrik >>> >>> >> >
Re: [squid-users] TPROXY Routing
I have the following in startup ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 The ouput of ip route show table 100: local default dev lo scope host One other thing is strange, my PREROUTING rules in mangle don't load in my script. I have to manually add them. Timing issue perhaps? Startup script loded from rc.local: echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter echo 1 > /proc/sys/net/ipv4/ip_forward iptables -t mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 ebtables -t broute -A BROUTING -i eth1 -p ipv4 --ip-proto tcp --ip-dport 80 -j redirect --redirect-target DROP ebtables -t broute -A BROUTING -i eth0 -p ipv4 --ip-proto tcp --ip-sport 80 -j redirect --redirect-target DROP cd /proc/sys/net/bridge/ for i in * do echo 0 > $i done unset i ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 2010/3/31 Henrik Nordström : > ons 2010-03-31 klockan 09:47 -0700 skrev Kurt Sandstrom: >> I have been unable to get TPROXY working correctly with squid. I have >> used the steps in http://wiki.squid-cache.org/Features/Tproxy4 and re >> checked everything. >> > > I did not see your routing setup in the data you dumped. Without the > routing configured then TPROXY won't intercept, just route like normal.. > > http://wiki.squid-cache.org/Features/Tproxy4#Routing_configuration > > Regards > Henrik > >
[squid-users] TPROXY Routing
I have been unable to get TPROXY working correctly with squid. I have used the steps in http://wiki.squid-cache.org/Features/Tproxy4 and re checked everything. Versions: Kernel 2.6.28-11-server (ubuntu) Squid Cache: Version 3.1.1 configure options: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/include' '--mandir=/share/man' '--infodir=/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=/lib/squid3' '--disable-maintainer-mode' '--disable-dependency-tracking' '--srcdir=.' '--datadir=/usr/share/squid3' '--sysconfdir=/etc/squid3' '--mandir=/usr/share/man' '--with-cppunit-basedir=/usr' '--enable-inline' '--enable-async-io=8' '--enable-storeio=ufs,aufs' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-underscores' '--enable-follow-x-forwarded-for' '--enable-auth=basic' '--enable-external-acl-helpers=ip_user' '--with-filedescriptors=65536' '--with-default-user=proxy' '--enable-epoll' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' --with-squid=/home/mike/squid-3.1.1 --enable-ltdl-convenience iptables v1.4.3 I can see http traffic incrementing through my DIVERT and PREROUTING tables Chain PREROUTING (policy ACCEPT 166K packets, 41M bytes) pkts bytes target prot opt in out source destination 2963 202K DIVERT tcp -- anyany anywhere anywheresocket 1684 85244 TPROXY tcp -- anyany anywhere anywheretcp dpt:www TPROXY redirect 0.0.0.0:3129 mark 0x1/0x1 Chain INPUT (policy ACCEPT 22640 packets, 1278K bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 23918 packets, 3770K bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 23918 packets, 3770K bytes) pkts bytes target prot opt in out source destination Chain DIVERT (1 references) pkts bytes target prot opt in out source destination 2963 202K MARK all -- anyany anywhere anywhereMARK xset 0x1/0x 2963 202K ACCEPT all -- anyany anywhere anywhere When I use -v -v there all the counters for errors are at 0 Squidclient shows: Connection information for squid: Number of clients accessing cache: 2 Number of HTTP requests received: 7 (from squidclient access) And my store isn't growing at all. It seems squid is not getting the traffic from my iptables... any ideas??
[squid-users] Re: ACL All Error
UPDATE! It seems to only happen when the --disable-ipv6 option is used during compile, perhaps the default acl handler for IPV4 needs a bit of love... doesn't seem the same as the one included as default. On Wed, Mar 31, 2010 at 1:00 AM, Kurt Sandstrom wrote: > Ok I have an issue here... Compiled Squid > Squid Cache: Version 3.1.1 > configure options: '--build=x86_64-linux-gnu' '--prefix=/usr' > '--includedir=/include' '--mandir=/share/man' '--infodir=/share/info' > '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=/lib/squid3' > '--disable-maintainer-mode' '--disable-dependency-tracking' > '--srcdir=.' '--datadir=/usr/share/squid3' '--sysconfdir=/etc/squid3' > '--mandir=/usr/share/man' '--with-cppunit-basedir=/usr' > '--enable-inline' '--enable-async-io=8' '--enable-storeio=ufs,aufs' > '--enable-removal-policies=lru,heap' '--enable-delay-pools' > '--enable-cache-digests' '--enable-underscores' > '--enable-follow-x-forwarded-for' '--enable-auth=basic' > '--enable-external-acl-helpers=ip_user' '--with-filedescriptors=65536' > '--with-default-user=proxy' '--enable-epoll' > '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' > --with-squid=/home/myuser/squid-3.1.1 --enable-ltdl-convenience > > and keep getting an error: > > squid -X > > FATAL: Bungled Default Configuration line 8: miss_access allow all > > Using the default squid.conf: > > > acl manager proto cache_object > acl localhost src 127.0.0.1/32 > acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 > acl localnet src 10.0.0.0/8 # RFC1918 possible internal network > acl localnet src 172.16.0.0/12 # RFC1918 possible internal network > acl localnet src 192.168.0.0/16 # RFC1918 possible internal network > # Have Added and removed with the same issue > # acl all src 0.0.0.0/0.0.0.0 > # acl all src 0.0.0.0/0 > acl SSL_ports port 443 > acl Safe_ports port 80 # http > acl Safe_ports port 21 # ftp > acl Safe_ports port 443 # https > acl Safe_ports port 70 # gopher > acl Safe_ports port 210 # wais > acl Safe_ports port 1025-65535 # unregistered ports > acl Safe_ports port 280 # http-mgmt > acl Safe_ports port 488 # gss-http > acl Safe_ports port 591 # filemaker > acl Safe_ports port 777 # multiling http > acl CONNECT method CONNECT > http_access allow manager localhost > http_access deny manager > http_access deny !Safe_ports > http_access deny CONNECT !SSL_ports > http_access allow localnet > http_access allow localhost > http_access deny all > http_port 3128 > hierarchy_stoplist cgi-bin ? > coredump_dir /var/cache > refresh_pattern ^ftp: 1440 20% 10080 > refresh_pattern ^gopher: 1440 0% 1440 > refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 > refresh_pattern . 0 20% 4320 > > > Details of error: > > 2010/03/31 00:51:02.160| ACL::Prototype::Registered: invoked for type src > 2010/03/31 00:51:02.160| ACL::Prototype::Registered: yes > 2010/03/31 00:51:02.160| ACL::FindByName 'all' > 2010/03/31 00:51:02.160| ACL::FindByName found no match > 2010/03/31 00:51:02.160| aclParseAclLine: Creating ACL 'all' > 2010/03/31 00:51:02.160| ACL::Prototype::Factory: cloning an object > for type 'src' > 2010/03/31 00:51:02.160| aclIpParseIpData: all > 2010/03/31 00:51:02.160| aclIpParseIpData: magic 'all' found. > 2010/03/31 00:51:02.160| aclParseAclList: looking for ACL name 'all' > 2010/03/31 00:51:02.160| ACL::FindByName 'all' > 2010/03/31 00:51:02.160| ACL::FindByName found no match > 2010/03/31 00:51:02.160| aclParseAclList: ACL name 'all' not found. > 2010/03/31 00:51:02.160| leave_suid: PID 27212 called > FATAL: Bungled Default Configuration line 8: miss_access allow all > > I have tried squid -X -f /etc/squid3/squid.conf to verify that I'm > using the right config > > > Any Ideas? >
[squid-users] ACL All Error
Ok I have an issue here... Compiled Squid Squid Cache: Version 3.1.1 configure options: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/include' '--mandir=/share/man' '--infodir=/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=/lib/squid3' '--disable-maintainer-mode' '--disable-dependency-tracking' '--srcdir=.' '--datadir=/usr/share/squid3' '--sysconfdir=/etc/squid3' '--mandir=/usr/share/man' '--with-cppunit-basedir=/usr' '--enable-inline' '--enable-async-io=8' '--enable-storeio=ufs,aufs' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-underscores' '--enable-follow-x-forwarded-for' '--enable-auth=basic' '--enable-external-acl-helpers=ip_user' '--with-filedescriptors=65536' '--with-default-user=proxy' '--enable-epoll' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' --with-squid=/home/myuser/squid-3.1.1 --enable-ltdl-convenience and keep getting an error: squid -X FATAL: Bungled Default Configuration line 8: miss_access allow all Using the default squid.conf: acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network # Have Added and removed with the same issue # acl all src 0.0.0.0/0.0.0.0 # acl all src 0.0.0.0/0 acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access allow localhost http_access deny all http_port 3128 hierarchy_stoplist cgi-bin ? coredump_dir /var/cache refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 Details of error: 2010/03/31 00:51:02.160| ACL::Prototype::Registered: invoked for type src 2010/03/31 00:51:02.160| ACL::Prototype::Registered: yes 2010/03/31 00:51:02.160| ACL::FindByName 'all' 2010/03/31 00:51:02.160| ACL::FindByName found no match 2010/03/31 00:51:02.160| aclParseAclLine: Creating ACL 'all' 2010/03/31 00:51:02.160| ACL::Prototype::Factory: cloning an object for type 'src' 2010/03/31 00:51:02.160| aclIpParseIpData: all 2010/03/31 00:51:02.160| aclIpParseIpData: magic 'all' found. 2010/03/31 00:51:02.160| aclParseAclList: looking for ACL name 'all' 2010/03/31 00:51:02.160| ACL::FindByName 'all' 2010/03/31 00:51:02.160| ACL::FindByName found no match 2010/03/31 00:51:02.160| aclParseAclList: ACL name 'all' not found. 2010/03/31 00:51:02.160| leave_suid: PID 27212 called FATAL: Bungled Default Configuration line 8: miss_access allow all I have tried squid -X -f /etc/squid3/squid.conf to verify that I'm using the right config Any Ideas?
