Re: [squid-users] Interception Caching Problems
Thanks all for the help. Amos, the following worked like a charm and thanks so much: Once thats working you can then go on to catch other common web server evasion ports (81, 87, 88, 8000, 8080, 8081, 8181, , 3128, 1080, 2080) Redirectding just port 80 wasnn't accomplishing anything. Are there any other well used ports to cover, or are the ones above what you would stick with? I'm definately hitting now, though. We racked up 3 gigs in the cache in a couple of weeks. Nice. Those ports are the ones I've confirmed as widely used. There are likely others I have not found yet. And people can set their own custom ones completely randomly. The next thing to look at is whether you can have any type of L7 (HTTP) protocol auto-detect at L3 (firewall) then you might want to set a snooper going to locate more ports and divert specific src-IP:dst-port pairs as they are found. I've seen this being experimented on P2P traffic, but nothing yet for HTTP. The only outstanding issue I have is that cache.log reports everyting as a miss. I have the zph kernel patch applied. I can definately tell the difference in speed between hits and misses, but everything in the log says Preserving TOS on miss. This isn't a huge issue now that we are hitting, but it would be beneficial to have the log accurately reflect hits and misses. Version is 3.0 stable 8. From the fact its logging I assume you have the initial and the followup patches applied. Squid-3 should be logging 'ZPH Local hit' on HITS. Mayhap you need to contact ZPH about a bug if it still exists there. They are still maintaining the testing and use of their patch. Amos
Re: [squid-users] Interception caching problems
Jason Cosby wrote: Well, I solved my DNS problems. BIND was in fact killing squid with it's errors (IPC can be a PITA sometimes). pdnsd is working great. I have verified that the 3128 redirect command is working. Accessing the server from a Linux client with lynx shows the hits in cache.log. lynx follows the standards. I still do not have IE7 or Firefox hitting the cache, though. I feel a little odd asking this, as I'm positive that there are thousands out there with these two browsers working just fine on redirect, but apparently I need to sniff out every port that either browser ever uses for regular TCP requests and redirect those. Does anyone know what those ports are? I could eventually find out, but it would take a good chunk of time (that I don't have) with tcpdump. huh? what are you redirecting on? all you need to redirect is web traffic from a network range of IPs going to a port (80) to catch most web traffic. Once thats working you can then go on to catch other common web server evasion ports (81, 87, 88, 8000, 8080, 8081, 8181, , 3128, 1080, 2080) Amos -- Please use Squid 2.7.STABLE4 or 3.0.STABLE8
Re: [squid-users] Interception caching problems
I'm serving in Iraq, where bandwidth is low and DNS servers are thousands of miles away. squid is a great solution for my unit. I set up squid-3.0-STABLE8 behind SNAT to do intercetion caching with the standard: iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j REDIRECT --to-port 3128 and http_port 3128 transparent but squid does not intercept the packets. Setting the proxy in the browsers (IE7 and Forefox3) results in squid caching as expected. After many agonizing days of trying to determine why I was not getting hits when leaving the browsers un-configured, I finally had everyone set their proxy settings to the server and port 3128 (dhcpd takes care of pointing them at the right subnet and gateway). The issues I now face are that other apps don't run right, particularly for the Mac guys (can't have seperate settings in browser and other network apps). I need to run this transparently if at all possible. Am I missing something with the newest browsers? tcpdump did report that IE7 was sending packets to port 137. Is Firefox also sending to non-standard ports? I even tried DNAT'ing everything from eth1 to port 3128 as a test, but no hits. Do I have squid listen on all possible tcp ports used by both browsers? Is iptables 1.4.1 buggy (doubtful)? Do I re-route all possible tcp ports to 3128? If so, does anyone know what all of the ports used by these two browsers are? Are the browsers sending out Don't intercept me headers when in default setup and Intercept me headers when configured for proxy? I'm at a loss. squid is doing a fantastic job of keeping a lot of traffic local, but I fear I will have to cease using it in order to keep everything else working if I can't solve this. IM and VC apps top the list down here since everyone tries to stay in touch with home, so I have to keep them working. Thanks so much for any help, Jason A couple of Qs. - is your squid built with --enable-linux-netfilter ? - is squid running on the NAT box? - are the requests just dying, or being served okay as TCP_MISS? - whats the rest of your config say? To keep explicit config (it is better anyway). Windows people are screwed (way to go MS). For the non-Windows users there is a global environment variable in most OS which applications usually use for proxy settings: http_proxy=http://fubar.example.org:3128/; or a control panel somewhere in the OS for 'proxy settings' which sets it properly for the whole machine. Not in the browser-only settings. Amos
Re: [squid-users] Interception caching problems
Hi, Pls fill below varable with yours. $LAN= Lan ip range. example- 192.168.0.0/24 $INTERFAZ_INT= Interface connects to the Internet $INTERFAZ_LAN= Interface conncects to Lan $LAN_IP of the squid box = Lan ip. example- 192.168.0.1 I use below rules for tranceparent interception on Linux. #Enabling ip forwarding echo 1 /proc/sys/net/ipv4/ip_forward #For squid traffic to Accept iptables -A INPUT -d $LAN_IP -p tcp -s $LAN --dport 3128 -j ACCEPT iptables -A FORWARD -p udp -s $LAN --dport 53 -m state --state NEW -j ACCEPT iptables -A FORWARD -p tcp -s $LAN -m multiport --dports 20,21,22,25,43,53,80,443,110,143 -m state --state NEW -j ACCEPT iptables -A OUTPUT -p udp --dport 53 -j ACCEPT iptables -A OUTPUT -p tcp -m multiport --dports 20,21,22,25,43,53,80,443,110,143 -j ACCEPT iptables -t nat -A POSTROUTING -p udp -o $INTERFAZ_INT -s $LAN --dport 53 -j SNAT --to-source $INT_IP iptables -t nat -A POSTROUTING -p tcp -o $INTERFAZ_INT -s $LAN -m multiport --dports 20,21,22,25,43,53,80,443,110,143 -j SNAT --to-source $INT_IP #Redirecting traffic destined to port 80 to port 3128 iptables -t nat -A PREROUTING -p tcp -i $INTERFAZ_LAN --dport 80 -j REDIRECT --to-port 3128 in addition to that, Pls check you Clients PCs. their gateway, DNS servers
Re: [squid-users] Interception caching problems
Hi, Pls fill below varable with yours. $LAN= Lan ip range. example- 192.168.0.0/24 $INTERFAZ_INT= Interface connects to the Internet $INTERFAZ_LAN= Interface conncects to Lan $LAN_IP of the squid box = Lan ip. example- 192.168.0.1 I use below rules for tranceparent interception on Linux. Most of those rules do not apply to web traffic. They appear to be standard rules for a gateway done manually without a control tool. see: http://wiki.squid-cache.org/ConfigExamples/Intercept/LinuxRedirect the issues do not appear to be with the interception itself. Just with the followup or underlying traffic flows. Amos
