[squid-users] The requested URL could not be retrieved: invalid url
Almost there! Squid3.0.STABLE1 squid.conf: visible_hostname iqBase http_port 3128 transparent acl iqnet src 192.168.60.0/255.255.255.0 cache_dir ufs /usr/local/squid/var/cache 100 16 256 acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 acl CONNECT method CONNECT http_access allow iqnet http_access allow manager localhost http_access deny manager http_access deny CONNECT !SSL_ports icp_access allow iqnet http_access deny all icp_access allow iqnet icp_access deny all htcp_access allow iqnet htcp_access deny all http_port 3128 hierarchy_stoplist cgi-bin ? access_log /usr/local/squid/var/logs/access.log squid acl QUERY urlpath_regex cgi-bin \? cache deny QUERY cache_effective_user nobody refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern . 0 20% 4320 icp_port 3130 coredump_dir /usr/local/squid/var/cache === Contents of the access.log: === $ cat /usr/local/squid/var/logs/access.log 1202382009.744 0 192.168.60.199 NONE/400 1856 GET /firefox?client=firefox-a&rls=org.mozilla:en-GB:official - NONE/- text/html 1202382009.907 0 192.168.60.199 NONE/400 1760 GET /favicon.ico - NONE/- text/html 1202382028.023 0 192.168.60.199 NONE/400 1738 GET / - NONE/- text/html 1202382046.868 0 192.168.60.199 NONE/400 1738 GET / - NONE/- text/html 1202382046.970 0 192.168.60.199 NONE/400 1760 GET /favicon.ico - NONE/- text/html 1202387676.866 0 192.168.60.199 NONE/400 1856 GET /firefox?client=firefox-a&rls=org.mozilla:en-GB:official - NONE/- text/html 1202387677.121 0 192.168.60.199 NONE/400 1760 GET /favicon.ico - NONE/- text/html 1202387691.774 0 192.168.60.199 NONE/400 2082 GET /safebrowsing/update?client=navclient-auto-ffox&appver=2.0.0.11&version=goog-white-domain:1:23,goog-white-url:1:371,goog-black-url:1:18337,goog-black-enchash:1:44096 - NONE/- text/html 1202387700.522 0 192.168.60.199 NONE/400 1856 GET /firefox?client=firefox-a&rls=org.mozilla:en-GB:official - NONE/- text/html === Browser Display: === ERROR The requested URL could not be retrieved While trying to retrieve the URL: /firefox?client=firefox-a&rls=org.mozilla:en-GB:official The following error was encountered: * Invalid URL Some aspect of the requested URL is incorrect. Possible problems: * Missing or incorrect access protocol (should be `http://'' or similar) * Missing hostname * Illegal double-escape in the URL-Path * Illegal character in hostname; underscores are not allowed Your cache administrator is webmaster. Generated Thu, 07 Feb 2008 12:35:00 GMT by iqBase (squid/3.0.STABLE1) === Can anyone see what's wrong?
Re: [squid-users] The requested URL could not be retrieved: invalid url
your squid.conf has two http_port using same port.. try to remove http_port 3128 or change the port number for last http_port On Feb 7, 2008 10:50 AM, Dave Coventry <[EMAIL PROTECTED]> wrote: > Almost there! > > Squid3.0.STABLE1 squid.conf: > > > visible_hostname iqBase > http_port 3128 transparent > acl iqnet src 192.168.60.0/255.255.255.0 > cache_dir ufs /usr/local/squid/var/cache 100 16 256 > acl manager proto cache_object > acl localhost src 127.0.0.1/32 > acl to_localhost dst 127.0.0.0/8 > acl SSL_ports port 443 > acl CONNECT method CONNECT > http_access allow iqnet > http_access allow manager localhost > http_access deny manager > http_access deny CONNECT !SSL_ports > icp_access allow iqnet > http_access deny all > icp_access allow iqnet > icp_access deny all > htcp_access allow iqnet > htcp_access deny all > http_port 3128 > hierarchy_stoplist cgi-bin ? > access_log /usr/local/squid/var/logs/access.log squid > acl QUERY urlpath_regex cgi-bin \? > cache deny QUERY > cache_effective_user nobody > refresh_pattern ^ftp: 144020% 10080 > refresh_pattern ^gopher:14400% 1440 > refresh_pattern . 0 20% 4320 > icp_port 3130 > coredump_dir /usr/local/squid/var/cache > === > > Contents of the access.log: > > === > $ cat /usr/local/squid/var/logs/access.log > 1202382009.744 0 192.168.60.199 NONE/400 1856 GET > /firefox?client=firefox-a&rls=org.mozilla:en-GB:official - NONE/- > text/html > 1202382009.907 0 192.168.60.199 NONE/400 1760 GET /favicon.ico - > NONE/- text/html > 1202382028.023 0 192.168.60.199 NONE/400 1738 GET / - NONE/- text/html > 1202382046.868 0 192.168.60.199 NONE/400 1738 GET / - NONE/- text/html > 1202382046.970 0 192.168.60.199 NONE/400 1760 GET /favicon.ico - > NONE/- text/html > 1202387676.866 0 192.168.60.199 NONE/400 1856 GET > /firefox?client=firefox-a&rls=org.mozilla:en-GB:official - NONE/- > text/html > 1202387677.121 0 192.168.60.199 NONE/400 1760 GET /favicon.ico - > NONE/- text/html > 1202387691.774 0 192.168.60.199 NONE/400 2082 GET > /safebrowsing/update?client=navclient-auto-ffox&appver=2.0.0.11&version=goog-white-domain:1:23,goog-white-url:1:371,goog-black-url:1:18337,goog-black-enchash:1:44096 > - NONE/- text/html > 1202387700.522 0 192.168.60.199 NONE/400 1856 GET > /firefox?client=firefox-a&rls=org.mozilla:en-GB:official - NONE/- > text/html > === > > Browser Display: > > === > ERROR > The requested URL could not be retrieved > > While trying to retrieve the URL: > /firefox?client=firefox-a&rls=org.mozilla:en-GB:official > > The following error was encountered: > > * Invalid URL > > Some aspect of the requested URL is incorrect. Possible problems: > > * Missing or incorrect access protocol (should be `http://'' or similar) > * Missing hostname > * Illegal double-escape in the URL-Path > * Illegal character in hostname; underscores are not allowed > > Your cache administrator is webmaster. > Generated Thu, 07 Feb 2008 12:35:00 GMT by iqBase (squid/3.0.STABLE1) > === > > Can anyone see what's wrong? > -- Sds. Alexandre J. Correa Onda Internet / OPinguim.net http://www.ondainternet.com.br http://www.opinguim.net
Re: [squid-users] The requested URL could not be retrieved: invalid url
On Feb 7, 2008 5:20 PM, Alexandre Correa <[EMAIL PROTECTED]> wrote: > your squid.conf has two http_port using same port.. > > try to remove http_port 3128 or change the port number for last http_port > > Many thanks for your response. Yes, I spotted that after posting. However, removing the second reference did not remedy the situation. It seems that the server name is removed. For example, if the URL requested is "http://news.bbc.co.uk/2/hi/africa/default.stm"; then squid appears to remove the primary DNS request (in this case "http://news.bbc.co.uk";) and then generate the error based on the directory and file name of the requested URL (in this case "2/hi/africa/default.stm"). Additionally (I didn't post this in my original RFH as I hadn't spotted it) my cache.log comprises the following: 2008/02/08 10:08:01| Starting Squid Cache version 3.0.STABLE1 for i686-pc-linux-gnu... 2008/02/08 10:08:01| Process ID 4460 2008/02/08 10:08:01| With 1024 file descriptors available 2008/02/08 10:08:01| DNS Socket created at 0.0.0.0, port 32773, FD 7 2008/02/08 10:08:01| Adding nameserver 192.168.10.213 from /etc/resolv.conf 2008/02/08 10:08:01| Unlinkd pipe opened on FD 12 2008/02/08 10:08:01| Swap maxSize 102400 KB, estimated 7876 objects 2008/02/08 10:08:01| Target number of buckets: 393 2008/02/08 10:08:01| Using 8192 Store buckets 2008/02/08 10:08:01| Max Mem size: 8192 KB 2008/02/08 10:08:01| Max Swap size: 102400 KB 2008/02/08 10:08:02| Version 1 of swap file with LFS support detected... 2008/02/08 10:08:02| Rebuilding storage in /usr/local/squid/var/cache (CLEAN) 2008/02/08 10:08:02| Using Least Load store dir selection 2008/02/08 10:08:02| Set Current Directory to /usr/local/squid/var/cache 2008/02/08 10:08:02| Loaded Icons. 2008/02/08 10:08:02| Accepting transparently proxied HTTP connections at 0.0.0.0, port 3128, FD 14. 2008/02/08 10:08:02| Accepting ICP messages at 0.0.0.0, port 3130, FD 15. 2008/02/08 10:08:02| HTCP Disabled. 2008/02/08 10:08:02| Ready to serve requests. 2008/02/08 10:08:02| Done reading /usr/local/squid/var/cache swaplog (0 entries) 2008/02/08 10:08:02| Finished rebuilding storage from disk. 2008/02/08 10:08:02| 0 Entries scanned 2008/02/08 10:08:02| 0 Invalid entries. 2008/02/08 10:08:02| 0 With invalid flags. 2008/02/08 10:08:02| 0 Objects loaded. 2008/02/08 10:08:02| 0 Objects expired. 2008/02/08 10:08:02| 0 Objects cancelled. 2008/02/08 10:08:02| 0 Duplicate URLs purged. 2008/02/08 10:08:02| 0 Swapfile clashes avoided. 2008/02/08 10:08:02| Took 0.07 seconds ( 0.00 objects/sec). 2008/02/08 10:08:02| Beginning Validation Procedure 2008/02/08 10:08:02| Completed Validation Procedure 2008/02/08 10:08:02| Validated 25 Entries 2008/02/08 10:08:02| store_swap_size = 0 2008/02/08 10:08:02| storeLateRelease: released 0 objects 2008/02/08 10:15:56| WARNING: transparent proxying not supported Does Squid need to be compiled explicitly with transparent proxying enabled?
Re: [squid-users] The requested URL could not be retrieved: invalid url
It shouldn't be this difficult. I mean, I setup ransaprent proxies in a matter of minutes these days. Yes, transparent proxying support needs to be compiled in for your architecture. So. To make this quick and painless: * which os * squid version 3.0, ok * squid config, sans comments * ip firewalling/redirection rules. I'll take what we work through and fix whatever bugs are causing this to not work, and put in some documentation so others aren't caught out. Adrian On Fri, Feb 08, 2008, Dave Coventry wrote: > Well the good news is that I am no longer getting the "The requested > URL could not be retrieved: invalid url" error. > > The bad news is that I am no longer getting any response at all. > > I must admit that this is probably the singlemost frustrating piece of > software I have ever tried to get to work. I have been working on this > since the 7th of January when I promised my client I would have his > proxy server up and running and I am no nearer resolving it. > > I find it very hard to believe that anyone can get it to run at all > and the suspicion is dawning that this whole project is a huge > elaborate hoax designed to generate light relief for bored BOFHs. > > I do not consider myself to be a newbie, having used linux for some > eight or nin years. Nor is my requirement particularly complex: > > I have simply set up a computer, bought especially for the purpose, > with 2 NICs one of which (eth1) is running as a DHCP server and the > other (eth0). -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: [squid-users] The requested URL could not be retrieved: invalid url
Adrian, On Feb 8, 2008 5:08 PM, Adrian Chadd <[EMAIL PROTECTED]> wrote: > It shouldn't be this difficult. I mean, I setup > ransaprent proxies in a matter of minutes > these days. > > Yes, transparent proxying support needs to be > compiled in for your architecture. > > So. To make this quick and painless: > > * which os > * squid version 3.0, ok > * squid config, sans comments > * ip firewalling/redirection rules. I'll try any distro, it's a greenfield machine. I started on Ubuntu, worked through Slackware, among others and am now back with Ubuntu 7.10 Server. The first thing I did was set up the DHCP server. Then I downloaded Squid3.0.STABLE1, extracted, ran './configure --prefix=/usr/local/squid' 'make all' 'make install' I set up my squid.conf according to the 'QUICKSTART', set up iptables according to this script: (script from http://www.cyberciti.biz/tips/linux-setup-transparent-proxy-squid-howto.html) == #!/bin/sh SQUID_SERVER="192.168.60.254″ INTERNET="eth0″ LAN_IN="eth1″ SQUID_PORT="3128″ # Clean old firewall iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X modprobe ip_conntrack modprobe ip_conntrack_ftp echo 1 > /proc/sys/net/ipv4/ip_forward iptables -P INPUT DROP iptables -P OUTPUT ACCEPT iptables -A INPUT -i lo -j ACCEPT iptables -A OUTPUT -o lo -j ACCEPT iptables -A INPUT -i $INTERNET -m state / --stateSTABLISHED,RELATED -j ACCEPT iptables --table nat --append POSTROUTING / --out-interface $INTERNET -j MASQUERADE iptables --append FORWARD --in-interface / $LAN_IN -j ACCEPT iptables -A INPUT -i $LAN_IN -j ACCEPT iptables -A OUTPUT -o $LAN_IN -j ACCEPT iptables -t nat -A PREROUTING -i $LAN_IN / -p tcp --dport 80 -j DNAT --to / $SQUID_SERVER:$SQUID_PORT iptables -t nat -A PREROUTING -i $INTERNET / -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT iptables -A INPUT -j LOG iptables -A INPUT -j DROP == At this point Squid worked, but it hacked off the Domain part of the URL and was unable to resolve the directory/htmlfilename part of the URL. I tried everything I could think of to get it to work, without success. In the end I wiped the Hard Drive (I must've reformatted this machine twenty or thirty times over the last 5 weeks) and started again from scratch using these instructions: http://kuscsik.blogspot.com/2008/01/transparent-proxy-with-squid-3-on.html This is where I am at the moment.
Re: [squid-users] The requested URL could not be retrieved: invalid url
On Fri, Feb 08, 2008, Dave Coventry wrote: > I'll try any distro, it's a greenfield machine. > > I started on Ubuntu, worked through Slackware, among others and am now > back with Ubuntu 7.10 Server. Ok. Ubuntu 7.10 is fine. > The first thing I did was set up the DHCP server. Ok. Not squid related. > > Then I downloaded Squid3.0.STABLE1, extracted, ran './configure > --prefix=/usr/local/squid' > 'make all' > 'make install' Under linux, add --enable-linux-netfilter to the configure line. > I set up my squid.conf according to the 'QUICKSTART', set up iptables > according to this script: > (script from > http://www.cyberciti.biz/tips/linux-setup-transparent-proxy-squid-howto.html) Why didn't you follow http://wiki.squid-cache.org/ConfigExamples/ ? There's half a dozen examples involving Linux + transparent interception. Adrian > == > #!/bin/sh > SQUID_SERVER="192.168.60.254?$B!m > INTERNET="eth0?$B!m > LAN_IN="eth1?$B!m > SQUID_PORT="3128?$B!m > > # Clean old firewall > iptables -F > iptables -X > iptables -t nat -F > iptables -t nat -X > iptables -t mangle -F > iptables -t mangle -X > > modprobe ip_conntrack > modprobe ip_conntrack_ftp > echo 1 > /proc/sys/net/ipv4/ip_forward > > > iptables -P INPUT DROP > iptables -P OUTPUT ACCEPT > > iptables -A INPUT -i lo -j ACCEPT > iptables -A OUTPUT -o lo -j ACCEPT > > iptables -A INPUT -i $INTERNET -m state / > --stateSTABLISHED,RELATED -j ACCEPT > > iptables --table nat --append POSTROUTING / > --out-interface $INTERNET -j MASQUERADE > iptables --append FORWARD --in-interface / > $LAN_IN -j ACCEPT > > iptables -A INPUT -i $LAN_IN -j ACCEPT > iptables -A OUTPUT -o $LAN_IN -j ACCEPT > > > iptables -t nat -A PREROUTING -i $LAN_IN / > -p tcp --dport 80 -j DNAT --to / > $SQUID_SERVER:$SQUID_PORT > > iptables -t nat -A PREROUTING -i $INTERNET / > -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT > > iptables -A INPUT -j LOG > iptables -A INPUT -j DROP > == > > At this point Squid worked, but it hacked off the Domain part of the > URL and was unable to resolve the directory/htmlfilename part of the > URL. > > I tried everything I could think of to get it to work, without success. > > In the end I wiped the Hard Drive (I must've reformatted this machine > twenty or thirty times over the last 5 weeks) and started again from > scratch using these instructions: > http://kuscsik.blogspot.com/2008/01/transparent-proxy-with-squid-3-on.html > > This is where I am at the moment. -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: [squid-users] The requested URL could not be retrieved: invalid url
Well the good news is that I am no longer getting the "The requested URL could not be retrieved: invalid url" error. The bad news is that I am no longer getting any response at all. I must admit that this is probably the singlemost frustrating piece of software I have ever tried to get to work. I have been working on this since the 7th of January when I promised my client I would have his proxy server up and running and I am no nearer resolving it. I find it very hard to believe that anyone can get it to run at all and the suspicion is dawning that this whole project is a huge elaborate hoax designed to generate light relief for bored BOFHs. I do not consider myself to be a newbie, having used linux for some eight or nin years. Nor is my requirement particularly complex: I have simply set up a computer, bought especially for the purpose, with 2 NICs one of which (eth1) is running as a DHCP server and the other (eth0).
Re: [squid-users] The requested URL could not be retrieved: invalid url
Thanks Adrian, I have managed to get Squid running. Now I need to configure it to Authenticate the users...
Re: [squid-users] The requested URL could not be retrieved: invalid url
On Sat, Feb 09, 2008, Dave Coventry wrote: > Thanks Adrian, I have managed to get Squid running. > > Now I need to configure it to Authenticate the users... Ah, that bits more difficult. ;) What authentication scheme are you after? adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: [squid-users] The requested URL could not be retrieved: invalid url
On Feb 9, 2008 4:20 PM, Adrian Chadd wrote: > Ah, that bits more difficult. ;) What authentication scheme are you after? I was hoping to do it through Samba. What Authentication scheme would you suggest...
Re: [squid-users] The requested URL could not be retrieved: invalid url
Dave Coventry wrote: On Feb 9, 2008 4:20 PM, Adrian Chadd wrote: Ah, that bits more difficult. ;) What authentication scheme are you after? I was hoping to do it through Samba. What Authentication scheme would you suggest... The problem arises from the fact that the browser has no knowledge that it is passing through a proxy. It asks the origin web server for a page and is confronted with a request for proxy authentication. What should it do? There are a few suggestions in the mailing list archives, including cookie-based authentication, and IP based authentication (I know that 2.6 has a session helper included in the source that would be a good base for this), but no solutions. Perhaps things are different when utilizing NTLM authentication... Chris
Re: [squid-users] The requested URL could not be retrieved: invalid url
On Fri, Feb 8, 2008 at 7:36 PM, Dave Coventry <[EMAIL PROTECTED]> wrote: > On Feb 8, 2008 7:37 PM, Adrian Chadd wrote: > > Under linux, add --enable-linux-netfilter to the configure line. > > Okay, I'll try that. I've managed to get squid working (without authentication as yet), but I have a really strange error. Whenever I access my apache server, squid removes the domain part of the URL and delivers an error. For example if I access my apache server http://myimaginarysite.dydns.org I get the following error: ~ snip ~~~ ERROR The requested URL could not be retrieved While trying to retrieve the URL: / The following error was encountered: * Invalid URL Some aspect of the requested URL is incorrect. Possible problems: * Missing or incorrect access protocol (should be `http://'' or similar) * Missing hostname * Illegal double-escape in the URL-Path * Illegal character in hostname; underscores are not allowed Your cache administrator is webmaster. Generated Thu, 28 Feb 2008 19:18:17 GMT by iqBase (squid/3.0.STABLE1) ~ snip ~~~ If I try http://myimaginarysite.dyndns.org/records/july.html the same error occurrs, but now it says: ERROR The requested URL could not be retrieved While trying to retrieve the URL: /records/july.html Anyone got any ideas? Previously I was getting the same errors whenever I tried to access any site, now the errors only occur when I try to access my own apache server since I compiled with "./configure --enable-linux-netfilter"
Re: [squid-users] The requested URL could not be retrieved: invalid url
> On Fri, Feb 8, 2008 at 7:36 PM, Dave Coventry <[EMAIL PROTECTED]> wrote: > > On Feb 8, 2008 7:37 PM, Adrian Chadd wrote: > > > Under linux, add --enable-linux-netfilter to the configure line. > > > > Okay, I'll try that. On 28.02.08 21:41, Dave Coventry wrote: > I've managed to get squid working (without authentication as yet), but > I have a really strange error. > > Whenever I access my apache server, squid removes the domain part of > the URL and delivers an error. > > For example if I access my apache server > http://myimaginarysite.dydns.org I get the following error: > > ~ snip ~~~ > ERROR > The requested URL could not be retrieved > > While trying to retrieve the URL: / > > The following error was encountered: > > * Invalid URL I guess you are trying to use squid as intercepting proxy but didn't tell it so. Look at "transparent" option for http_port directive -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. One OS to rule them all, One OS to find them, One OS to bring them all and into darkness bind them
Re: [squid-users] The requested URL could not be retrieved: invalid url
On Thu, Feb 28, 2008 at 11:08 PM, Matus UHLAR - fantomas wrote: > > I guess you are trying to use squid as intercepting proxy but didn't tell it > so. Look at "transparent" option for http_port directive Hi Matus, I have "http_port 3128 transparent" in my squid.conf and this is the only occurance of the http_port directive. It works fine except when I need to access the intranet apache server (which is on the same machine).
Re: [squid-users] The requested URL could not be retrieved: invalid url
Matus UHLAR - fantomas wrote: On Fri, Feb 8, 2008 at 7:36 PM, Dave Coventry <[EMAIL PROTECTED]> wrote: On Feb 8, 2008 7:37 PM, Adrian Chadd wrote: > Under linux, add --enable-linux-netfilter to the configure line. Okay, I'll try that. On 28.02.08 21:41, Dave Coventry wrote: I've managed to get squid working (without authentication as yet), but I have a really strange error. Whenever I access my apache server, squid removes the domain part of the URL and delivers an error. For example if I access my apache server http://myimaginarysite.dydns.org I get the following error: ~ snip ~~~ ERROR The requested URL could not be retrieved While trying to retrieve the URL: / The following error was encountered: * Invalid URL I guess you are trying to use squid as intercepting proxy but didn't tell it so. Look at "transparent" option for http_port directive I think this error message is normal for transparent proxies. They do not natively recevie the domain in the "METHOD-URL-PROTOCOL" tuple. The squid code in transparent mode should be pulling the Host: info from headers, but may not report it in the page even if using it. Is the URL you are asking for actually real and reachable to squid? You could try adding 'vhost' to the options to force squid check Host: header and see if something funky is causing it not to by default. Amos -- Please use Squid 2.6STABLE17+ or 3.0STABLE1+ There are serious security advisories out on all earlier releases.
