[squid-users] Re: Should I see a massive slowdown when chaining squid => privoxy
Amos Jeffries writes: [...] Harry wrote: >> What kind of things should I do to start tracking down what is >> hampering the connections so bad. >> > Amos replied: > This may help... > http://www.extremetech.com/article2/0,2845,1854196,00.asp After creating a user.js file in the mozilla profile default containing: --- 8< snip -- 8< snip -- 8http://www.ford-trucks.com/ Between 17-19 seconds ---- ---=--- - Then trying the connection with no proxy: kill firefox, start FF, clear cache, Type in http://www.ford-trucks.com/ Connection was fully completed in 8 seconds again. ---- ---=--- - Now trying the first experiment (privoxy alone as proxy) but this time with no `user.js' file: Connection is completed in 16 seconds plus. ---- ---=--- - Now with user.js in place but no proxy Connection completes in 8 seconds ---- ---=--- - ---- ---=--- - And since this is a squid group. Proxy set to squid alone (user.js in place) The connection to about 30 seconds for roughly 60% of that ford cite to load. I abandoned the completing the connection after 2 full minutes. ---- ---=--- - Then to be fair, I removed the user.js hacks. Restarted firefox, (with proxy set to squid alone and no user.js). First, it took a goodly while for my home page (goggle.com/ig) to load, cleared the cache and typed in: http://www.ford-trucks.com About 60% loaded in 12 seconds and completed in about 35 seconds. ---- ---=--- - Apparently the `user.js' hack isn't all that helpful. And in fact appears to be something of a hindrance. (Note: I ran these experiments repeatedly.. the results shown above are about the average results obtained) I don't know what to make of it all, I guess my configs are pretty weak being fairly default for both privoxy and squid, but so far it appears that both squid and privoxy cause serious slowdown of page loading. My full configs: , | Squid: |www.jtan.com/~reader/sqcfg/disp.cgi ( squid.conf with all default |comments). | |www.jtan.com/~reader/sqcfg/strp.cgi (squid.conf stripped). | | Privoxy: |www.jtan.com/~reader/prcfg/disp.cgi (privoxy config with all |default comments). | |www.jtan.com/~reader/prcfg/strp.cgi (privoxy config stripped). `
[squid-users] Re: Should I see a massive slowdown when chaining squid => privoxy
Amos Jeffries writes: [...] > This may help... > http://www.extremetech.com/article2/0,2845,1854196,00.asp Thanks for the continuing good information. About that citation above: The author of the hacks on that page has suggested that readers might edit a `user.js' file. Probably something of a dumb question but, is that something I am supposed to create myself, like: `%APPDATA%\Mozilla\Firefox\Profiles\xxx.default\user.js Or should that file already exist? I don't see a file with that name on a windows or linux install of recent firefox.
[squid-users] Re: Should I see a massive slowdown when chaining squid => privoxy
Amos Jeffries writes: [...] > Speed gain/loss/other depends on what you are moving from. > > MORE IMPORTANTLY: how you define "slow"! > OK, getting down to cases here. Here is one test.. First clear the cache just the simple way tools/options/advanced/network `clear now' Close firefox (4.0.1) Start firefox (It starts on google which is struggling to resolve) Clear the cache once more Type this URL in http://www.ford-trucks.com Start stopwatch, then start firefox on that address by hitting enter.. I get 1:32 (one min, 32 sec) with chain of browser => squid => privoxy in place Now with no proxy at all. CLear the cache, kill firefox. Start firefox, (it starts on google very quickly), clear the cache once more for good measure. Type in http://www.ford-trucks.com, Start the stopwatch, then start firefox on that address by hitting enter... I get 0:9 (9 seconds) with no proxy in place That is something on the order of 900% faster... I think. ---- ---=--- - What kind of things should I do to start tracking down what is hampering the connections so bad.
[squid-users] Re: Should I see a massive slowdown when chaining squid => privoxy
Amos Jeffries writes: [...] Harry wrote (summarized -ed hp): Adding squid and privoxy into a proxy setup seems to really really slow down my browsing as compared to browsing with a direct connection. (no proxy) And asks if this is normal. [...] > Speed gain/loss/other depends on what you are moving from. > > MORE IMPORTANTLY: how you define "slow"! > > Keep in mind that you also now have around 2x the processing going on > with 2 proxies. The difference added by Squid can be at least > 10ms. Some people call that noticeable slowdown. Some dont care about > anything less than a second. I'm guessing its more than seconds slower but not really sure how to gage the difference reliably, so as not to be giving flawed information here or have difference due to caching or something. Can you suggest a method to arrive at a fairly good comparison? > * 3.1 is about 10-20% slower than the latest 2.7 on the same > config. With the older versions of 3.1 being on the slower end of that > scale as we work to optimize and fix things throughout the series. Wouldn't 20% be noticeable? So you're saying to back down a few versions for now? > * Moving to Squid from a non-proxy setup can be a major drop down > depending on the browser age. The browsers themselves drop the > parallel fetch rate from hundreds down to under 10. Browser tweaking > is the only way to avoid this. I'm using firefox 4.X on all home lan machines (that have a gui). Can you recommend some documentation that might help with what you called `Browser tweaking', I've never done anything special to a browser other than add or subtract add-on tools. > * Moving from browser->privoxy to a browser->squid->privoxy setup you > should have seen only a small drop. Some possibilities are Squid using > slow disks (maybe RAID), or Squid box is swapping, or the bandwidth is > being routed down the same physical links to/from Squid. No raid, and the hardware of the server is P4 Intel(R) Celeron(R) CPU 3.06GHz and 2GBram running on oldish IDE discs ---- ---=--- - Probably should have included some questions about the squid.conf and privoxy/config in the first post. Maybe there are things in there that are not good left as default. I realize that both tools have several config files. I've left all but the two main ones in default state and have posted my working /etc/squid/squid.conf and /etc/privoxy/config There is a lot of debris in the comments but still I thought it might be useful to leave it all in for jogging memories...But also included a way to prune the comments by changing the name of the cgi script at the end of the URL from `disp.cgi' to `strp.cgi'. Any coaching would be well appreciated. ---- ---=--- - squid.conf WITH comments: www.jtan.com/~reader/sqcfg/disp.cgi squid.conf WITH OUT comments www.jtan.com/~reader/sqcfg/strp.cgi = privoxy's config WITH comments: www.jtan.com/~reader/prcfg/disp.cgi privoxy's config WITH OUT comments: www.jtan.com/~reader/prcfg/strp.cgi
[squid-users] Re: Should I see a massive slowdown when chaining squid => privoxy
Eliezer Croitoru writes: > another question. > are you by any change made a speedtest on the current setup? > if you do get the right speed but wrong speed to get the page > processed it's a known side effect of privoxy. Is there some standard way to do a speed test? I'm not sure I'd know how to do one that was at least semi-scientific. Find a slow loading page and do `time firefox URL'... killing it soon as it loads? Then clean the cache and do it again with squid and privoxy in the loop?
[squid-users] Re: Should I see a massive slowdown when chaining squid => privoxy
sichent writes: >> Setup: Gentoo linux OS on squid and privoxy home lan server >> Squid-3.1.12 >> privoxy-3.0.17 >> >> I'm not running an html server, just trying to use squid and privoxy >> for my own browsing. >> > > Why not to use ICAP or URL rewriter functionality built into Squid to > achieve the same results as privoxy instead of having this "chaining" > setup? Well for starts, I'd be jumping off into stuff I know nothing about and more than likely end up spending way more time than I should on it. I don't know much about squid and privoxy either but at least have tinkered with them several times over the years.
[squid-users] Should I see a massive slowdown when chaining squid => privoxy
Setup: Gentoo linux OS on squid and privoxy home lan server Squid-3.1.12 privoxy-3.0.17 I'm not running an html server, just trying to use squid and privoxy for my own browsing. I'm attempting to get started with squid and privoxy. So far using nearly original config files in both cases. First I tried just using privoxy without squid and that seemed to work OK. I set the privoxy listen-address in /etc/privoxy/config like: listen-address 192.168.0.2:8118 (The address of a gmane box on the lan) I was not able to access gmail thru a firefox gadget but could get gmail directly alright. But other than that, No real noticeable change in browsing speed as against NO proxy at all. I then used this website to help setup squid: https://www.antagonism.org/web/squid-proxy.shtml And did the things suggested there. Chaining in this direction: Browser -> Squid -> Privoxy -> Privoxy's IP -> Public Internet Adding squid into the mix... I Added these two lines to the end of squid.conf: cache_peer 192.168.0.2 parent 8118 0 default no-query no-digest no-netdb-exchange never_direct allow all And uncommented these lines in squid.conf: acl localnet src 192.168.0.0/24 # RFC1918 possible internal network header_access From deny all header_access Referer deny all header_access Server deny all header_access User-Agent deny all header_access WWW-Authenticate deny all header_access Link deny all other wise the defaults were left in place ---- ---=--- - With squid in the mix, things began to crawl... Apparently I've jacked up the config enough that browsing is now painful rather that snappy. Can anyone spot some kind of nastiness I've introduced?
[squid-users] Re: Access denials following Machine name change
Henrik Nordstrom <[EMAIL PROTECTED]> writes: > This is your http_access lines denying access for some reason. But I > could not see anything obvious in the config you posted.. > > Anything in cache.log? > > What did access.log say? Just now, starting firefox I get this screen: ERROR The requested URL could not be retrieved While trying to retrieve the URL: http://www.google.com/ The following error was encountered: * Access Denied. Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect. access.log shows: 1141381871.443 119 192.168.0.4 TCP_REFRESH_HIT/200 1724 GET http://www.google.com/favicon.ico - DIRECT/72.14.203.104 text/html If I click the URL in above output I then get google but the top images is missing. And these lines are added to access.log: 1141382106.544 0 192.168.1.2 TCP_DENIED/403 1395 GET http://www.google.com/ - NONE/- text/html 1141382108.484 48 192.168.0.4 TCP_MISS/200 1588 GET http://www.google.com/ - DIRECT/72.14.203.99 text/html 1141382108.584 0 192.168.1.2 TCP_DENIED/403 1441 GET http://www.google.com/intl/en/images/logo.gif - NONE/- text/html You notice a new host IP is in there (ending 1.2). That is a second nic that talks to a dmz machine. It shouldn't be there right? cache.log has nothing to say so far. Here I've connected to a bank ok and then attempted to log in which is denied: o While trying to retrieve the URL: www.corusbankhb.com:443 o The following error was encountered: o* Access Denied. access.log: 1141382349.763 0 192.168.1.2 TCP_DENIED/403 1397 CONNECT www.corusbankhb.com:443 - NONE/- text/html 1141382355.123 5358 192.168.0.4 TCP_MISS/200 3399 CONNECT www.corusbankhb.com:443 - DIRECT/208.35.186.137 - 1141382355.244 6313 192.168.0.4 TCP_MISS/200 19720 CONNECT www.corusbankhb.com:443 - DIRECT/208.35.186.137 -
[squid-users] Access denials following Machine name change
setup: Gentoo linux (kernel 2.6.15) squid-2.5.12 Following a recent name change to the HOST squid runs on, I continue to get odd access denials after having carefully gone thru the squid config files I found only one place where the old HOST name appeared. `visible_hostname' and of course changed that. I've also rm 'ed the old /var/cache/squid with: rm -rf /var/cache/squid/* And rebuilt it with a restart of squid. Still I get denials at places where I didn't before and example might be loging into an online banking session. I'm able to get to the bank but get denied when I attempt to login: While trying to retrieve the URL: www.corusbankhb.com:443 The following error was encountered: * Access Denied. Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect. Your cache administrator is root. Generated Thu, 02 Mar 2006 12:12:56 GMT by reader.local.lan (squid/2.5.STABLE12) The last line shows the correct host name. Prior to the rename this didn't happen. Squid.conf details: grep -v '^#\|^$' /etc/squid/squid.conf hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? no_cache deny QUERY auth_param basic children 5 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 2 hours auth_param basic casesensitive off refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher: 14400% 1440 refresh_pattern . 0 20% 4320 acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443 563 # https, snews acl Safe_ports port 70# gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl Safe_ports port 901 # SWAT acl purge method PURGE acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !Safe_ports http_access deny CONNECT !SSL_ports acl our_networks src 192.168.0.0/24 http_access allow our_networks http_access allow localhost http_access deny all http_reply_access allow all icp_access allow all visible_hostname reader.local.lan forwarded_for off coredump_dir /var/cache/squid
