[squid-users] Squid Performance Problem

2006-02-22 Thread Mohamed Naji
'm working for an ISP company and I'm using squid as a cache engine
for the first time.
when a dial-up user downloads a cached file originally form a local
server, the download rate is 10 kbps, but if the origin server is
anywhere else on the Internet the download rate of a previously cached
file will be reduced to 5 kbps or even less. (in both cases squid
indicates a successful TCP_HIT).
I've tested that several times on different files from different
sites. and I'm getting always the same results. I've tried to test it
in both online and offline modes. and nothing has changed (except that
squid logs the hits as TCP_OFFLINE_HIT/200 instead of TCP_HIT/200)

Why the two files are not downloaded with the same rate, keeping in
mind that they are both cached and served from the same proxy/cache
server (squid)?
Can anyone help me to solve it?


Re: [squid-users] Squid Performance Problem

2006-02-22 Thread Mark Elsen
 'm working for an ISP company and I'm using squid as a cache engine
 for the first time.
 when a dial-up user downloads a cached file originally form a local
 server, the download rate is 10 kbps, but if the origin server is
 anywhere else on the Internet the download rate of a previously cached
 file will be reduced to 5 kbps or even less. (in both cases squid
 indicates a successful TCP_HIT).
 I've tested that several times on different files from different
 sites. and I'm getting always the same results. I've tried to test it
 in both online and offline modes. and nothing has changed (except that
 squid logs the hits as TCP_OFFLINE_HIT/200 instead of TCP_HIT/200)

 Why the two files are not downloaded with the same rate, keeping in
 mind that they are both cached and served from the same proxy/cache
 server (squid)?
 Can anyone help me to solve it?


Rember that QoS (quality of service) of bandwith is not guaranteed once data
must be received
out of your area-of-control (perimeter infrastructure).

M.


Re: [squid-users] acl req_mime_type

2006-02-22 Thread Mark Elsen
 Hi,

 can anybody tell me what this acl does? does it block downloading or 
 uploading?
 and how can i test it?


It identifies mime types, which can then be used further;
when building access control in squid (http_access rules)

M.


Re: [squid-users] Mirror sites

2006-02-22 Thread Mark Elsen
 Hi,

 I would just like to know.  When downloading a file from a site say for
 example a link like so: http://host1.example.com/test.zip and the file
 has a size say 300kb (and timestamp 2006/01/01 ?) and then downloading
 the same file from another (mirror) site say
 http://host2.example.com/test.zip with the filesize also at 300kb and
 the same timestamp  Will squid redownload the file?  Or will it be
 clever enough to say that I've already downloaded that file, I'll send
 you the cached version?  Do timestamps matter?  Does squid just check
 the filesizes and filenames?


There's no way for SQUID to know, it matches the same object,
SQUID only uses the complete url to decided on that. (MD5 - checksum)

Whether cached copies will be returned, if already accessed,
(from the same 'uniq' server); also depends on freshness info provided
by the webserver ,about the returned object.

What if, in both cases , the webserver issues a 'Cache-control:
no-cache ' header.
Then you question collapes by definition.

M.


Re: [squid-users] Problems with file upload

2006-02-22 Thread Mark Elsen
 Hello Squid-User Group Members,

 The problem is not new, there are many postings about file upload issues with 
 files 1MB. - But I've read the FAQ and searched a few hours in Mailing list 
 archives. The only recommendations I found was to check the Squid Config and 
 php.ini for correct values which I've checked with our hosting provider.

 Symptom: Using Squid Proxy Version 2.5 Stable 9 and Stable 12. Connection is 
 dropped after a while (maybe reaching maximum execution time of apache 
 server) when uploading files  1MB. User sees the typical IE error message, 
 when a site becomes unreachable.

 PHP Settings:
 (simple upload script in php)

 File_uploads: ON
 Max_execution_time: 150
 Memory_limit: 8M
 Safe_mode: OFF
 Post_max_size: 8M
 Upload_max_filesize: 5M

 Apache Settings:
 This is done by the hosting provider, I don't know if they have limits there. 
 If LimitRequestBody is set in apache, the apache answers with Error 413 (to 
 much data on request).

 What I have already done:
 Restarting squid and opening a new session with IE can help for the first 
 upload, but all following uploads fail.

 Uploading the files without proxy server works well with this hosting 
 provider.

 Uploads to perl scripts working well with this hosting provider.

 Testing that issue with a local webserver is already done, but I need to 
 check again, because I am not sure that the proxy was really involved.


 Does anyone saw this problem before? Could it be provider specific? A 
 networking issue?


 - What's in access.log for the failed upload ? (preferably , post extract
with 'strip_query_terms' set to off).

 - Anything further in cache.log ?

 M.


Re: [squid-users] Squid 2.5.STABLE9 and Kernel 2.6.11 SMP

2006-02-22 Thread Mark Elsen
 HI Mark


 I´m sorry vor that. I was asked if the squid is waiting (stopping) or
 crashing.
 The squid isn´t crashing. It waits and waits and waits.


I asked you, to provide, when SQUID is in this condition, to
provide the output of  cache.log entries after a 2 secs DEBUG session,
which can be achieved issuing the following command(s) :

% squid -k debug; sleep 2; squid -k debug

(output is in cache.log)

M.


Re: [squid-users] rebuilding question

2006-02-22 Thread Mark Elsen
 :

 I'm preparing to rebuild squid on a few servers within a production
 cluster to apply the epoll patch and fix a FD issue.  Once everything is
 rebuilt (same configuration options), do I have to run squid -z
 initially?  Or, can squid reuse the existing cache directories after
 being rebuilt?

  You don't have to run 'squid -z'; mind you the epoll patch, is as I believe,
not ready for production use.
There has been a thread about this recently, check the archives.

 I guess my question is, if the config files don't change and the cache
 is still the same, will squid be the wiser?


  Define wiser ?

  M.


Re: [squid-users] Solutions for transparent + proxy_auth?

2006-02-22 Thread Kinkie
On Tue, 2006-02-21 at 10:03 -0600, Steve Brown wrote:
[...]
 In the specific scenario I mentioned, the browser isn't submitting any
 credentials.  The traffic is being intercepted and routed through a
 local proxy which in turns forwards requests to a remote proxy w/
 authentication.  It seems to me that the browser is completely unaware
 that there is any interception taking place.  Isn't that the point?
[...]
 So what is the purpose of the login parameter for the peer_cache config 
 option?

It seems that I misunderstood what you meant. Do you want the PROXY to
authenticate against its parent? Independently from who is the user it
acts in behalf of?


A confused Kinkie


Re: [squid-users] Squid Performance Problem

2006-02-22 Thread Mohamed Naji
On 2/22/06, Mark Elsen [EMAIL PROTECTED] wrote:
  'm working for an ISP company and I'm using squid as a cache engine
  for the first time.
  when a dial-up user downloads a cached file originally form a local
  server, the download rate is 10 kbps, but if the origin server is
  anywhere else on the Internet the download rate of a previously cached
  file will be reduced to 5 kbps or even less. (in both cases squid
  indicates a successful TCP_HIT).
  I've tested that several times on different files from different
  sites. and I'm getting always the same results. I've tried to test it
  in both online and offline modes. and nothing has changed (except that
  squid logs the hits as TCP_OFFLINE_HIT/200 instead of TCP_HIT/200)
 
  Why the two files are not downloaded with the same rate, keeping in
  mind that they are both cached and served from the same proxy/cache
  server (squid)?
  Can anyone help me to solve it?
 

 Rember that QoS (quality of service) of bandwith is not guaranteed once data
 must be received
 out of your area-of-control (perimeter infrastructure).

 M.


Thanks for the tip, but it seams that I am always having the same
results depending on the location of the origin server. I would agree
with you if the transfer rate changes in one of the cases, but the
problem is that a cached file is always transferred with a fixed
download rate (10 to 12 kbps if the origine server is local (on the
same network with squid server), and 4 to 5 kbps if not).
Can you please elaborate?


[squid-users] Problem with Interception Caching/Proxying

2006-02-22 Thread Tony Spencer
Hi

Below is a diagram of part of our network.
We have an L2TP tunnel that gets terminated on our Cisco 7200 for DSL users.
The tunnel comes from our Upstream provider and once terminated we do the
Radius authentication and then redirect all port 80 traffic to our Squid
server.
However the Squid part doesn't seem to be working correctly, although I can
see the request come into Squid by watching the Squid access log, the end
user never gets the web page he requested, unless that web site is on the
10.0.0.x network.

I followed the examples on the Squid site to do the redirection of port 80
traffic, the Cisco has an access list and a route map to redirect port 80 to
the Squid server, the Squid server then forwards port 80 traffic to port
3128. I've even run Squid on port 80 so requests go directly to the Squid
port from the Cisco 7200.

If I configure my web browser on my PC to use the Squid server as a proxy I
can browse web sites fine, however this is when I'm not coming via the
tunnel on the router, just purely over the Internet.

10.0.0.1 (gw on upstream)
|
|
| 10.0.0.4
  -
| cisco 2900 switch ||   Radius   |
- - 
|   |
   10.0.0.2 |   |10.0.0.3
   ---
   |Cisco 7200 |   |   sQUID  |
   -   


The Relevant parts of the Cisco config looks like this:

###
interface FastEthernet3/0
 ip address 10.0.0.2 255.255.255.0
 no ip route-cache cef
 no ip route-cache
 no ip mroute-cache
 duplex full


interface Virtual-Template1
 ip unnumbered FastEthernet3/0
 ip mroute-cache
 ip policy route-map proxy-redirect
 no logging event link-status
 timeout absolute 4320 0
 peer default ip address pool IP-POOL
 no keepalive
 ppp authentication chap pap callin
 ppp multilink

access-list 110 deny   tcp host 10.0.0.3 any
access-list 110 permit tcp any any eq www
access-list 110 deny   tcp any any
!
route-map proxy-redirect permit 10
 match ip address 110
 set ip next-hop 10.0.0.3

##

And the Squid server uses the following iptables rule to forward the web
traffic to port 80.


###
echo 1  /proc/sys/net/ipv4/ip_forward

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to
port 3128
###


It seems that although the Cisco is redirecting port 80 to the Squid server
and the server then forwards the requests to port 3128 and I can see the
requests come in, either the Squid server can't retrieve the website from
the outside world or can't return the site back to the end user.

If I take the route map off the Cisco 7200 and just all traffic through end
users can browse web sites ok.
But as soon as I redirect them to Squid it all goes wrong.

The Cisco 7200 is running IOS 12.3
The Squid server is a Centos 4.2 running squid-2.5.STABLE6-3.4E.11.

I can only think it's a config problem?
If anyone can help I would appreciate it.

Thanks
Tony

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.375 / Virus Database: 267.15.12/266 - Release Date: 21/02/2006
 



[squid-users] Sos transparent proxy problem

2006-02-22 Thread Muhammad Bilal Ahmad

Dear all

 

I am in a panic situation. I have configured squid with Diskd to use as a
transparent proxy following are the lines I used to support it for
transparent proxy.

 

Httpd_accel_host virtual

httpd_accel_port 80

httpd_accel_with_proxy on

httpd_accel_uses_host_header on

 

 

the squid is running on port 3128, and my machine has two network cards.
eth0  has the ip 192.168.0.29 and the eth1 has the live ip. 

 

I have made some of the rules in /etc/rc.local to forward the request to
port 80.

 This is my rc.local

 

touch /var/lock/subsys/local

ulimit -n 8192

echo 1  /proc/sys/net/ipv4/ip_forward

iptables -t nat -A POSTROUTING -p tcp -j MASQUERADE

iptables -t nat -A PREROUTING  -p tcp --dport 80 -i eth0 -j DNAT --to
192.168.0.29:3128

iptables -t nat -A POSTROUTING -p udp -j MASQUERADE

 

 

the dns is also running on the same machine.

 

On the client end I have giving 192.168.0.29 as gateway and 192.168.0.29 as
dns.

 

I am surprised but my browser takes a lot of time around 4 to 5 minutes; l
use IE if I try to open www.hotmail.com it displayes opening page
http://WWW.www.hotmail.com.net; then it redirects to auto search. And then
says internet explorer cannot display the search page.

 

If I turn on the iptables with the above rules. My browser connects if I
give it the proxy server as 192.168.0.29 with port 80.

 

 

I have tried to reconsider every thing which I can.

 

Urgent help is needed.

 

Thanx a million in advance

 

Kind Regards

M Bilal Ahmad

Astt manager Communications

Naveena Group

 

I am using fedora cora2 






AW: [squid-users] Squid 2.5.STABLE9 and Kernel 2.6.11 SMP

2006-02-22 Thread Christian Herzberg
Hi Mark

I will do but I have to wait for the next trouble.

Thanks
Christian

-Ursprüngliche Nachricht-
Von: Mark Elsen [mailto:[EMAIL PROTECTED] 
Gesendet: Mittwoch, 22. Februar 2006 09:53
An: Christian Herzberg
Cc: squid-users@squid-cache.org
Betreff: Re: [squid-users] Squid 2.5.STABLE9 and Kernel 2.6.11 SMP


 HI Mark


 I´m sorry vor that. I was asked if the squid is waiting (stopping) or 
 crashing. The squid isn´t crashing. It waits and waits and waits.


I asked you, to provide, when SQUID is in this condition, to
provide the output of  cache.log entries after a 2 secs DEBUG session,
which can be achieved issuing the following command(s) :

% squid -k debug; sleep 2; squid -k debug

(output is in cache.log)

M.


Re: [squid-users] Sos transparent proxy problem

2006-02-22 Thread Kinkie
On Wed, 2006-02-22 at 15:30 +0500, Muhammad Bilal Ahmad wrote:
 Dear all

[...] 

 touch /var/lock/subsys/local
 
 ulimit -n 8192
 
 echo 1  /proc/sys/net/ipv4/ip_forward
 
 iptables -t nat -A POSTROUTING -p tcp -j MASQUERADE
 
 iptables -t nat -A PREROUTING  -p tcp --dport 80 -i eth0 -j DNAT --to
 192.168.0.29:3128
 
 iptables -t nat -A POSTROUTING -p udp -j MASQUERADE

[...]

Your iptables rules are wrong, especially the DNAT one. Please check the
FAQ at
http://squidwiki.kinkie.it/SquidFaq/InterceptionProxy#head-e59e8be8079565bbfac3f978111ea65b48840ef9

Kinkie


RE: [squid-users] Sos transparent proxy problem

2006-02-22 Thread Muhammad Bilal Ahmad
Thanx for your reply

I have tried all of the instructions listed in the given site but they wont
work.

I think problem is out of the iptables.

Waiting for reply
M Bilal Ahmad

-Original Message-
From: Kinkie [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, February 22, 2006 4:23 PM
To: Muhammad Bilal Ahmad
Cc: 'Steve Brown'; squid-users@squid-cache.org
Subject: Re: [squid-users] Sos transparent proxy problem

On Wed, 2006-02-22 at 15:30 +0500, Muhammad Bilal Ahmad wrote:
 Dear all

[...] 

 touch /var/lock/subsys/local
 
 ulimit -n 8192
 
 echo 1  /proc/sys/net/ipv4/ip_forward
 
 iptables -t nat -A POSTROUTING -p tcp -j MASQUERADE
 
 iptables -t nat -A PREROUTING  -p tcp --dport 80 -i eth0 -j DNAT --to
 192.168.0.29:3128
 
 iptables -t nat -A POSTROUTING -p udp -j MASQUERADE

[...]

Your iptables rules are wrong, especially the DNAT one. Please check the
FAQ at
http://squidwiki.kinkie.it/SquidFaq/InterceptionProxy#head-e59e8be8079565bbf
ac3f978111ea65b48840ef9

Kinkie




[squid-users] Save clients password

2006-02-22 Thread Franco, Battista
Hi
I use squid ldap users authentication.
From my client PCs every time I start IE I need to insert username and
password. 
Is it possible to configure squid user and password popup with a
checkbox to permit to save password?
So next time I'll not retype password.


Re: [squid-users] Problem with intercept squid and boinc

2006-02-22 Thread Oliver Schulze L.

I think my problem could be that I use an IP alias for interception.

Will do some test, thanks for you comments Mark!

Oliver

--
Oliver Schulze L.
[EMAIL PROTECTED]



Re: [squid-users] Passing username from external acl to cache peer

2006-02-22 Thread Henrik Nordstrom
tis 2006-02-14 klockan 13:55 +0800 skrev Russell:
 Hi,
 
 Was hoping to get some help passing usernames from an external acl to a 
 cache peer.  My situation is squid - dansguardian - squid.  First 
 squid for making ident queries and applying some acl's we have in place 
 (quota limits, identification required etc) which then needs to pass the 
 username from the ident query to dansguardian so that users can be put 
 into filter groups.

ident query performed by Squid, or external acl helper returning user
info to Squid?

external acl is not the same thing as ident.

Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel


Re: [squid-users] Help tuning squid

2006-02-22 Thread Henrik Nordstrom
mån 2006-02-13 klockan 14:55 -0200 skrev Carlos Eduardo Gomes Marins:

 I don't have Squidguard nor Dansguardian, only Trendmicro IWSS as mentioned, 
 so all the acls's and handled by Squid itself.
 Is there anything I can do to improve the overall performance?

What kinds of ACLs are you using? No big regex based acls I hope (these
drain a lot of CPU).

Also try the following:

   half_closed_clients off
   quick_abort_min 0 KB
   quick_abort_max 0 KB

Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel


Re: [squid-users] regexp after redirect

2006-02-22 Thread Henrik Nordstrom
tis 2006-02-14 klockan 18:36 -0600 skrev Fernando Rodriguez:
 Hello,
  
 Is there a way to match a regexp after redirect returns??

The rproxy patch available from devel.squid-cache.org adds among a lot
of other things a http_access2 statement executing after redirectors and
can be used for this.


  
 Im using Squid Squidguard, I have done some tests regarding userlists and ip
 lists
 Since both of the matches are done via the same network it usually works but
 since the firs ACL to match is regarding ip if your ip is on the list you
 will have no problems, but if is not then you will be redirected to
 http://jdkalsjd.jjj http://jdkalsjd.jjj/  that obiously dosnt exist but I
 wan to catch that url to ask the user for its login and password using PAM
 so the next time is redirected it will also have a username and password to
 see if this matches a user from squidguard and then contunie with the
 process.

You could send a browser redirect instead of just rewriting the URL..
this way the browser will request the new URL.

Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel


Re: [squid-users] cpu usage increases over time, squid performance declines

2006-02-22 Thread Henrik Nordstrom
tis 2006-02-14 klockan 22:31 -0800 skrev Mike Solomon:
 Hardware:
 DualCore Opeteron 270, 1800MHz

A bit overkill. Squid can not use SMP effectively...

 This would be fantastic, but the machines fall over after several  
 hours. I have 4 machines, each configured identically. They last a  
 few hours - they slowly consume more and more cpu, all in user space,  
 until it starts affecting the median HTTP repsonse time. Then  
 throughput drops precipitously.

Try

 half_closed_clients off

Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel


Re: [squid-users] Quick question about dynamic delay pools - current status

2006-02-22 Thread Henrik Nordstrom
ons 2006-02-15 klockan 16:37 +0200 skrev laurentiu r:
   Hi everyone,
   Just a quick question about the dynamic delay pools. Apologies if it
   has been asked before - it must have been - but I've looked into the
   mail archives and didn't seem to find indication as to what's the
   current status in this matter.

   I saw that at some point there was a fuss on this list about a patch
   for making the delay pools 'dynamic', in the sense that a pool with 
   high a traffic rate could borrow unused bandwidth from other delay
   pools. And some suggestions were made to developers (Henrik Nordstrom)
   to inlcude the patch in the official releases.

   Well, has it been included? Is it in the 2.5Stable12 version? If not,
   what's the way to go for those who need to enable dynamic delay pools
   in Squid?

It has not been included as no patch has been submitted by it's authors
to the Squid developers.

Note: Squid-2.5 is feature frozen, so to get included the patch needs to
be for Squid-3 (what will become Squid-3.0).

The delay pools in Squid-3 has been reworked quite a bit, but I am at
this stage not sure if a functionality similar to the dynamic delay
pools is available as I have not been involved in this part of Squid-3.
But you (or others) are very welcome to start playing with Squid-3 in
your labs to try out all the new cool features. Just don't do it on your
production servers as there still is a bit to go for a production
release of Squid-3.0.

Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel


Re: [squid-users] Redirector Capture

2006-02-22 Thread Henrik Nordstrom
ons 2006-02-15 klockan 11:39 -0600 skrev Fernando Rodriguez:
 Is there any way to capture the return url of a redirector program so i  can
 reprosses the resulting url for password authentication??

Can you explain in more detail what it is you want to do?

Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel


Re: [squid-users] Problem with intercept squid and boinc

2006-02-22 Thread Oliver Schulze L.

Hi Mark,
I have 2 identical servers (CentOS 4.2), with same squid version and
interception iptables settings.

I have the same boinc client behind both squid servers,
and in one that work I see:
1140608197.087   3022 192.168.1.1 TCP_MISS/200 248 POST 
http://setiboincdata.ssl.berkeley.edu/sah_cgi/file_upload_handler - 
DIRECT/66.28.250.125 text/plain


and in the problematic squid server I see:
1140566460.404   2060 192.168.2.90 TCP_MISS/100 123 POST 
http://setiboincdata.ssl.berkeley.edu/sah_cgi/file_upload_handler - 
DIRECT/66.28.250.125 -


What does TCP_MISS/100 mean? As I see, the correct value should be 
TCP_MISS/200


Many thanks
Oliver


Mark Elsen wrote:

  mmm, didn't that interception has all this problems. I have been using
  

it for years in some
client's servers.



It does.

  

Do you know how can I debug even further?



I'd really stress (advise), that you probably found an application
which is broken by using transp. proxying, following the many
hola-cola issues mentioned,
hence tear-down any further thinking and provide none transparant http
access for boinc (when configured to use  http proxy).


M.

  


--
Oliver Schulze L.
[EMAIL PROTECTED]



Re: [squid-users] Squid and WCCP v1 (squid-2.5.STABLE11-3.FC3) on Fedora Core 3 (2.6.9-1.667smp) -- SOS

2006-02-22 Thread Daniel EPEE LEA
Hi,

I have a RHELv4 cache  + Cisco IOS Software, C1700
Software (C1700-K9O3SY7-M), Version 12.3(14)T2,
RELEASE SOFTWARE (fc4).

I have applied your suggestions, but it's still not
working. Please take a lookt at my Router's + Squid
config.

Am I missing something ?
-
!
version 12.3
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Router
!
boot-start-marker
boot-end-marker
!
enable password 
!
no aaa new-model
!
resource policy
!
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
ip subnet-zero
ip wccp version 1
ip wccp web-cache
!
!
no ip dhcp use vrf connected
!
!
ip cef
no ip ips deny-action ips-interface
!
no ftp-server write-enable
!
interface Ethernet0
 ip address x.x.x.x 255.255.255.x
 no ip route-cache cef
 full-duplex
!
interface FastEthernet0
 ip address y.y.y.y 255.255.255.x
 ip wccp web-cache redirect out
 speed auto
 full-duplex
!
interface Serial0
 no ip address
 shutdown
 no fair-queue
!
ip classless
ip route 0.0.0.0 0.0.0.0 y.y.y.5
no ip http server
no ip http secure-server
!
control-plane
!
line con 0
line aux 0
line vty 0 4
 password 
 login
!
end


/etc/sysctl.conf
---
[EMAIL PROTECTED] conf]# cat gre0/rp_filter
1
[EMAIL PROTECTED] conf]# cat bond0/rp_filter
1




Squid.conf

http_port [Server IP]:3128
icp_port 3130
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_mem 256 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size 4096 KB
minimum_object_size 0 KB
maximum_object_size_in_memory 8 KB
cache_dir ufs /usr/local/squid/var/cache 20240 16 256
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
mime_table /usr/local/squid/etc/mime.conf
pid_filename /var/run/squid.pid
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern .   0   20% 4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl Local src [My Local Network]
http_access allow Local
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl our_networks src [my network]
http_access allow our_networks
http_access deny all
http_reply_access allow all
icp_access allow all
icp_access allow all
tcp_outgoing_address [Server IP]
cache_mgr [EMAIL PROTECTED]
cache_effective_user squid
cache_effective_group squid
visible_hostname cache.mydomain.com
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
logfile_rotate 10
forwarded_for on
cachemgr_passwd  shutdown
snmp_port 3401
snmp_access deny all
wccp_router [Router IP]
wccp_outgoing_address [Server IP]
coredump_dir /usr/local/squid/var/cache



Much regards,

Waiting for answers

Daniel

--- Oliver Chato [EMAIL PROTECTED] wrote:

 Hi.
 
 Just for the sake of others who are looking to make 
 Transparent/Interception caching with Squid, WCCP v1
 and Fedora Core 3, 
 this is what we did to get it working:
 
 On the router (IOS 12.3(2)T):
 
 ip cef
 ip wccp version 1
 ip wccp web-cache
 interface interface either directly or
 indirectly connected to the 
 Internet
 ip wccp web-cache redirect out
 end
 
 Also, we did:
 
 conf t
 ip cef  # some systems may already have
 'ip cef global'
 int interface directly or indirectly connected
 to the Squid Server  (or int FastEthernet 0/0
 or other internal interface)
 no ip route-cache cef
 CTRL Z
 
 That's it. For debugging, we used:
 show ip wccp
 show ip wccp web-caches
 show ip wccp web-cache detail
 show ip wccp web-cache view (or: show ip wccp 99
 detail)
 
 On the Linux Server (Fedora Core 3
 (2.6.9-1.667smp)):
 In squid.conf:
 http_port 3128
 httpd_accel_host virtual
 httpd_accel_port 80
 httpd_accel_with_proxy on
 httpd_accel_uses_host_header on
 tcp_outgound_address ip address of your
 interface connected to 
 the WCCP router
 wccp_outgoing_address ip address of your
 interface connected to 
 the 

RE: [squid-users] Save clients password

2006-02-22 Thread Paul Mattingly
If you can configure squid to use NTLM, this authentication process
happens in the background with no user intervention which may be a
better option.

Is your ldap program talking to active directory or some other ldap
directory?

-Original Message-
From: Franco, Battista [mailto:[EMAIL PROTECTED] 
Sent: 22 February 2006 12:12
To: squid-users@squid-cache.org
Subject: [squid-users] Save clients password

Hi
I use squid ldap users authentication.
From my client PCs every time I start IE I need to insert username and
password. 
Is it possible to configure squid user and password popup with a
checkbox to permit to save password?
So next time I'll not retype password.


Re: [squid-users] RHEL v4 + Squid + wccp

2006-02-22 Thread Daniel EPEE LEA
Hi,
My kernel is 2.6.9-22.ELsmp #1 SMP
And I have loaded the ip_gre module.
Please can you point out where I do not get it ?

Regards,

Dan
On 2/21/06, Henrik Nordstrom [EMAIL PROTECTED] wrote:
 mån 2006-02-13 klockan 13:31 -0500 skrev Shoebottom, Bryan:
  Hello,
 
  I have not been able to get the ip_gre module and tunnel to work.  I
  currently use the ip_wccp module
  (http://www.squid-cache.org/WCCP-support/Linux/) and no configured
  tunnel on the linux box.

 ip_gre is the recommended method, but requires a fairly recent kernel to
 work. (Linux 2.6.9 or later I think).

 Regards
 Henrik


 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.2 (GNU/Linux)

 iD8DBQBD+5QG516QwDnMM9sRAiQyAJ9H7jdZEiG0MbFSqp6cNsiSHD9+2QCeMVWe
 F+NR0jyncd5ZXYWdIxacIv4=
 =ASLH
 -END PGP SIGNATURE-






--
--
Daniel Epee Lea


Re: [squid-users] Problem with intercept squid and boinc

2006-02-22 Thread Mark Elsen
On 2/22/06, Oliver Schulze L. [EMAIL PROTECTED] wrote:
 Hi Mark,
 I have 2 identical servers (CentOS 4.2), with same squid version and
 interception iptables settings.

 I have the same boinc client behind both squid servers,
 and in one that work I see:
 1140608197.087   3022 192.168.1.1 TCP_MISS/200 248 POST
 http://setiboincdata.ssl.berkeley.edu/sah_cgi/file_upload_handler -
 DIRECT/66.28.250.125 text/plain

 and in the problematic squid server I see:
 1140566460.404   2060 192.168.2.90 TCP_MISS/100 123 POST
 http://setiboincdata.ssl.berkeley.edu/sah_cgi/file_upload_handler -
 DIRECT/66.28.250.125 -

 What does TCP_MISS/100 mean?

  - Checkout the HTTP status codes in the FAQ :

100Continue

I don´t have a reason of thinking for the moment , how
this relates to the problem you are experiencing with transparant
proxying.

M.


As I see, the correct value should be
 TCP_MISS/200




Re: [squid-users] RHEL v4 + Squid + wccp

2006-02-22 Thread Daniel EPEE LEA
hello,

I have a RHELv4 cache  + Cisco IOS Software, C1700
Software (C1700-K9O3SY7-M), Version 12.3(14)T2,
RELEASE SOFTWARE (fc4).

I have applied your suggestions, but it's still not
working. Please take a lookt at my Router's + Squid
config.

Am I missing something ?
-
!
version 12.3
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Router
!
boot-start-marker
boot-end-marker
!
enable password 
!
no aaa new-model
!
resource policy
!
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
ip subnet-zero
ip wccp version 1
ip wccp web-cache
!
!
no ip dhcp use vrf connected
!
!
ip cef
no ip ips deny-action ips-interface
!
no ftp-server write-enable
!
interface Ethernet0
 ip address x.x.x.x 255.255.255.x
 no ip route-cache cef
 full-duplex
!
interface FastEthernet0
 ip address y.y.y.y 255.255.255.x
 ip wccp web-cache redirect out
 speed auto
 full-duplex
!
interface Serial0
 no ip address
 shutdown
 no fair-queue
!
ip classless
ip route 0.0.0.0 0.0.0.0 y.y.y.5
no ip http server
no ip http secure-server
!
control-plane
!
line con 0
line aux 0
line vty 0 4
 password 
 login
!
end


/etc/sysctl.conf
---
[EMAIL PROTECTED] conf]# cat gre0/rp_filter
1
[EMAIL PROTECTED] conf]# cat bond0/rp_filter
1




Squid.conf

http_port [Server IP]:3128
icp_port 3130
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_mem 256 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size 4096 KB
minimum_object_size 0 KB
maximum_object_size_in_memory 8 KB
cache_dir ufs /usr/local/squid/var/cache 20240 16 256
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
mime_table /usr/local/squid/etc/mime.conf
pid_filename /var/run/squid.pid
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern .   0   20% 4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl Local src [My Local Network]
http_access allow Local
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl our_networks src [my network]
http_access allow our_networks
http_access deny all
http_reply_access allow all
icp_access allow all
icp_access allow all
tcp_outgoing_address [Server IP]
cache_mgr [EMAIL PROTECTED]
cache_effective_user squid
cache_effective_group squid
visible_hostname cache.mydomain.com
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
logfile_rotate 10
forwarded_for on
cachemgr_passwd  shutdown
snmp_port 3401
snmp_access deny all
wccp_router [Router IP]
wccp_outgoing_address [Server IP]
coredump_dir /usr/local/squid/var/cache


Regards,

Waiting for answer

Thanks

Dan

On 2/22/06, Daniel EPEE LEA [EMAIL PROTECTED] wrote:
 Hi,
 My kernel is 2.6.9-22.ELsmp #1 SMP
 And I have loaded the ip_gre module.
 Please can you point out where I do not get it ?

 Regards,




[squid-users] management get MSN 24/7

2006-02-22 Thread Brent Clark

Hi all

I need to setup up a interesting configuration for a different list of users

acl management src /etc/SQUID/management.txt# List of managment users
acl staff src /etc/SQUID/staff.txt  # List of normal users

My MSN controlling time is as so:

acl msntime time M T W H F A 11:59-12:59
acl msntime time M T W H F A 16:59-18:59
acl msnp rep_mime_type ^application/x-msn-messenger$
acl msnq req_mime_type ^application/x-msn-messenger$
http_reply_access allow msnp msntime
http_reply_access allow msnq msntime
http_reply_access deny msnq
http_reply_access deny msnp

I need to make it so that staff get MSN during the time specified, but 
management that get MSN 24/7.

I tried :

http_reply_access deny msnq !management
http_reply_access deny msnp !management

But this doesnt seem to work.

If anyone could assit, I would be most grateful.

Kind Regards
Brent Clark


Re: [squid-users] Sos transparent proxy problem

2006-02-22 Thread Mark Elsen
  Thanx for your reply

 I have tried all of the instructions listed in the given site but they wont
 work.

 I think problem is out of the iptables.



 - For your hotmail issue , please try  (in squid.conf) :

acl hotmail_domains   dstdomain.hotmail.msn.com
header_access   Accept-Encodingdenyhotmail_domains

 (followed by : squid -k reconfigure)

  Afterwards , check cache.log; to make sure that no unwanted errors
  appear;

  Then check the hotmail access, through transparant proxying,
   again.

   M.


Re: [squid-users] R: [squid-users] Save clients password

2006-02-22 Thread Mark Elsen
 Yes my program talks with Windows 2003 AD.


Please ( !- again) , keep discussions into the same original-thread

   - You are friendly-er to the community
   - Archives en search-tools will be able to organize and
 operate, themselves in a more optimal manner;
 which will also benefit you.

M.


RE: [squid-users] rebuilding question

2006-02-22 Thread Gregori Parker
By wiser, I mean: will squid just picked up where it left off with the cache as 
if nothing happened?  Or will items in the cache become alien to squid?

Not a big deal either way, I'll just try it and if I have to wipe the caches, 
so be it.
 

-Original Message-
From: Mark Elsen [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, February 22, 2006 12:56 AM
To: Gregori Parker
Cc: squid-users@squid-cache.org
Subject: Re: [squid-users] rebuilding question

 :

 I'm preparing to rebuild squid on a few servers within a production
 cluster to apply the epoll patch and fix a FD issue.  Once everything is
 rebuilt (same configuration options), do I have to run squid -z
 initially?  Or, can squid reuse the existing cache directories after
 being rebuilt?

  You don't have to run 'squid -z'; mind you the epoll patch, is as I believe,
not ready for production use.
There has been a thread about this recently, check the archives.

 I guess my question is, if the config files don't change and the cache
 is still the same, will squid be the wiser?


  Define wiser ?

  M.



Re: [squid-users] rebuilding question

2006-02-22 Thread Mark Elsen
 By wiser, I mean: will squid just picked up where it left off with the cache 
 as if nothing happened?  Or will items in the cache become alien to squid?

  Certainly not, SQUID is alien-free, and was even designed to
run on :

 http://planetquest.jpl.nasa.gov/TPF/tpf_index.cfm

Some smile of course, seriously ; objects will certainly not
become alien to SQUID.

M.


[squid-users] squid + windows media player

2006-02-22 Thread Terry Dobbs
Has anyone got this working properly? When users access a page that plays a 
.wav/mp3 there is userid/password prompt. If you click cancel it goes away 
until you go to the next page. These pages with the audio have a NSPlayer 
header.


I don't have any rules setup to allow only header X. Surely, someone has 
this working right?


If not, I guess the users will just have to get in the habit of clicking 
cancel!


Thanks for any help 




--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.375 / Virus Database: 267.15.12/266 - Release Date: 2/21/2006



RE: [squid-users] management get MSN 24/7

2006-02-22 Thread Chris Robertson
See lines inserted below...

 -Original Message-
 From: Brent Clark [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, February 22, 2006 5:20 AM
 To: squid-users@squid-cache.org
 Subject: [squid-users] management get MSN 24/7
 
 
 Hi all
 
 I need to setup up a interesting configuration for a 
 different list of users
 
 acl management src /etc/SQUID/management.txt# List 
 of managment users
 acl staff src /etc/SQUID/staff.txt  # List of normal users
 
 My MSN controlling time is as so:
 
 acl msntime time M T W H F A 11:59-12:59
 acl msntime time M T W H F A 16:59-18:59
 acl msnp rep_mime_type ^application/x-msn-messenger$
 acl msnq req_mime_type ^application/x-msn-messenger$

# Allow management MSN
http_reply_access allow msnp management
http_reply_access allow msnq management
# Restrict everyone else based on time

 http_reply_access allow msnp msntime
 http_reply_access allow msnq msntime
 http_reply_access deny msnq
 http_reply_access deny msnp
 
 I need to make it so that staff get MSN during the time 
 specified, but management that get MSN 24/7.
 
 I tried :
 
 http_reply_access deny msnq !management
 http_reply_access deny msnp !management
 
 But this doesnt seem to work.
 
 If anyone could assit, I would be most grateful.
 
 Kind Regards
 Brent Clark
 

Order is critical.

Chris


Re: [squid-users] cpu usage increases over time, squid performance declines

2006-02-22 Thread Mike Solomon
I added this line to the config on two of my hosts, but it did not  
have any effect. The host experienced the same amount of slowdown  
under high load and had to be restarted.


I should note that I changed the config file and did:

sudo squid -k reconfigure

I did not kill the process.

I'm not sure if I understand half_closed_clients exactly, but the  
number of active file descriptors did not change significantly.


As I mentioned before, turning down the keep-alive time and lowering  
the active file descriptors did not seem to have any effect previously.


Thanks,

-Mike

On Feb 21, 2006, at 2:47 PM, Henrik Nordstrom wrote:


tis 2006-02-14 klockan 22:31 -0800 skrev Mike Solomon:

This would be fantastic, but the machines fall over after several
hours. I have 4 machines, each configured identically. They last a
few hours - they slowly consume more and more cpu, all in user space,
until it starts affecting the median HTTP repsonse time. Then
throughput drops precipitously.


Try

 half_closed_clients off

Regards
Henrik




[squid-users] low squid performance?

2006-02-22 Thread Tomasz Kolaj
Hello,

I observed have too low performance. On 2x 64bit Xeon 2,8GHz 2GB DDR2, 2x WD 
RAPTOR Squid 2.5.STABLE12 can answer max for 120 requests/s. 
115 r/s - 97-98% usage of first processor. Second is unusable for squid :/. I 
have two cache_dirs (aufs). One pre disk.

aragorn ~ # squid -v
Squid Cache: Version 2.5.STABLE12
configure options:  --prefix=/usr --bindir=/usr/bin --exec-prefix=/usr 
--sbindir=/usr/sbin --localstatedir=/var --mandir=/usr/share/man 
--sysconfdir=/etc/squid --libexecdir=/usr/lib/squid 
--enable-auth=basic,digest,ntlm --enable-removal-policies=lru,heap 
--enable-digest-auth-helpers=password 
--enable-basic-auth-helpers=SASL,PAM,getpwnam,YP,NCSA,SMB,MSNT,multi-domain-NTLM,winbind
 
--enable-external-acl-helpers=ip_user,unix_group,wbinfo_group,winbind_group 
--enable-ntlm-auth-helpers=SMB,fakeauth,no_check,winbind 
--enable-linux-netfilter --enable-ident-lookups --enable-useragent-log 
--enable-cache-digests --enable-delay-pools --enable-referer-log 
--enable-truncate --enable-arp-acl --with-pthreads --with-large-files 
--enable-htcp --enable-carp --enable-poll --disable-follow-x-forwarded-for 
--host=x86_64-pc-linux-gnu --disable-snmp --enable-ssl --enable-underscores 
--enable-storeio='ufs,diskd,coss,aufs,null' --enable-async-io

from config:
cache_mem  512MB

aragorn ~ # uname -a
Linux aragorn 2.6.15-gentoo-r5 #1 SMP Thu Feb 16 02:03:43 CET 2006 x86_64 
Intel(R) Xeon(TM) CPU 2.80GHz GenuineIntel GNU/Linux

glibc-2.3.90.20060207 with NPTL

Can I do something to improve preformance?
-- 
Tomasz Kolaj


Re: [squid-users] low squid performance?

2006-02-22 Thread Mark Elsen
 Hello,

 I observed have too low performance. On 2x 64bit Xeon 2,8GHz 2GB DDR2, 2x WD
 RAPTOR Squid 2.5.STABLE12 can answer max for 120 requests/s.
 115 r/s - 97-98% usage of first processor. Second is unusable for squid :/. I
 have two cache_dirs (aufs). One pre disk.

 aragorn ~ # squid -v
 Squid Cache: Version 2.5.STABLE12
 configure options:  --prefix=/usr --bindir=/usr/bin --exec-prefix=/usr
 --sbindir=/usr/sbin --localstatedir=/var --mandir=/usr/share/man
 --sysconfdir=/etc/squid --libexecdir=/usr/lib/squid
 --enable-auth=basic,digest,ntlm --enable-removal-policies=lru,heap
 --enable-digest-auth-helpers=password
 --enable-basic-auth-helpers=SASL,PAM,getpwnam,YP,NCSA,SMB,MSNT,multi-domain-NTLM,winbind
 --enable-external-acl-helpers=ip_user,unix_group,wbinfo_group,winbind_group
 --enable-ntlm-auth-helpers=SMB,fakeauth,no_check,winbind
 --enable-linux-netfilter --enable-ident-lookups --enable-useragent-log
 --enable-cache-digests --enable-delay-pools --enable-referer-log
 --enable-truncate --enable-arp-acl --with-pthreads --with-large-files
 --enable-htcp --enable-carp --enable-poll --disable-follow-x-forwarded-for
 --host=x86_64-pc-linux-gnu --disable-snmp --enable-ssl --enable-underscores
 --enable-storeio='ufs,diskd,coss,aufs,null' --enable-async-io

 from config:
 cache_mem  512MB

 aragorn ~ # uname -a
 Linux aragorn 2.6.15-gentoo-r5 #1 SMP Thu Feb 16 02:03:43 CET 2006 x86_64
 Intel(R) Xeon(TM) CPU 2.80GHz GenuineIntel GNU/Linux

 glibc-2.3.90.20060207 with NPTL

 Can I do something to improve preformance?

  - Make your own build and installation of SQUID; configure only those
options which you need.
This may help for performance too.

M.


Re: [squid-users] squid + windows media player

2006-02-22 Thread Mark Elsen
  Has anyone got this working properly? When users access a page that plays a
 .wav/mp3 there is userid/password prompt. If you click cancel it goes away
 until you go to the next page. These pages with the audio have a NSPlayer
 header.

 I don't have any rules setup to allow only header X. Surely, someone has
 this working right?

 If not, I guess the users will just have to get in the habit of clicking
 cancel!

 Thanks for any help


 It seems that the players used, do not support ntlm follow-thru auth. ; hence
 the problems you encounter.

 M.


Re: [squid-users] low squid performance?

2006-02-22 Thread Tomasz Kolaj
Dnia środa, 22 lutego 2006 23:18, Mark Elsen napisał:

   - Make your own build and installation of SQUID; configure only those
 options which you need.
 This may help for performance too.
Ok, I'll rebuild squid without not-needed options. I have top squid's usage at 
18:00-21:00 so I'll check changes tommorow.

What performance I should expect from this hardware?

-- 
Tomasz Kolaj


Re: [squid-users] Need help to improve squid performance

2006-02-22 Thread Raj
After I upgrade the memory to 2gb can I increase the cache_mem value
to 256MB. At the moment it is 64MB.

Thanks

On 2/22/06, Kevin [EMAIL PROTECTED] wrote:
  We are running OpenBSD version 3.6

 I'd recommend going to 3.8.


   Can you define performance issues?
 
  If I access a website it takes 6 to 8 seconds to download the page. We
  have a 10MB internet link and the link utilisation is only 50% on
  average.

 That seems very high.   Something is broken somewhere.

 My home squid is on a minimal OpenBSD machine, about the same as the
 hardware you specify, but on a slow cablemodem.  In this environment,
 it takes about 8 seconds for CNN to fully load, but barely a half
 second for Google, maybe a second for www.undeadly.org

 Of course this is without the two-layer model and without NTLM.

 Kevin



RE: [squid-users] low squid performance?

2006-02-22 Thread Chris Robertson
 -Original Message-
 From: Tomasz Kolaj [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, February 22, 2006 1:30 PM
 To: squid-users@squid-cache.org
 Subject: Re: [squid-users] low squid performance?
 
 
 Dnia środa, 22 lutego 2006 23:18, Mark Elsen napisał:
 
- Make your own build and installation of SQUID; 
 configure only those
  options which you need.
  This may help for performance too.
 Ok, I'll rebuild squid without not-needed options. I have top 
 squid's usage at 
 18:00-21:00 so I'll check changes tommorow.
 
 What performance I should expect from this hardware?
 
 -- 
 Tomasz Kolaj


The answer to that question is dependant on a whole host of variables, such as 
ACLs used, whether it's a proxy or an accelerator, the types of clients 
accessing it (client latency has a dramatic effect on CPU usage), types of 
content retrieved, how your cache_dirs are defined, etc.

Various things that can reduce Squid performance:

* regex based ACLs
* High latency clients
* blocking cache_dir configuration (e.g. using ufs instead of aufs or 
diskd)
* Anti-virus scanning
* Slow authentication back ends

If none of these issues covers your problem, you might look into experimental 
solutions such as the epoll patch 
(http://devel.squid-cache.org/projects.html#epoll). 

Chris


Re: [squid-users] low squid performance?

2006-02-22 Thread Tomasz Kolaj
Dnia środa, 22 lutego 2006 23:57, Chris Robertson napisał:
[cut]

 The answer to that question is dependant on a whole host of variables, such
 as ACLs used, whether it's a proxy or an accelerator, the types of clients
 accessing it (client latency has a dramatic effect on CPU usage), types of
 content retrieved, how your cache_dirs are defined, etc.

 Various things that can reduce Squid performance:

 * regex based ACLs
acl badURL url_regex -i .wmf$
#^ remove wmf after security leaks on ms wmf file format
acl mGG url_regex ^http://adserver.gadu[\-]?gadu.pl/.*$
redirector_access deny !mGG
redirector_bypass on
redirect_program /home/gg_rewrite
#^redirector ro replece banner in popular polish comunicator

acl QUERY urlpath_regex cgi-bin \?

#typical patterns
refresh_pattern -i (.*jpg$|.*gif$|.*png$) 0 50% 28800
refresh_pattern -i (.*html$|.*htm|.*shtml|.*php) 0 20% 1440
refresh_pattern .               0       20%     4320

 * High latency clients

What do you mean high latecy clients?

 * blocking cache_dir configuration (e.g. using ufs instead of aufs or
 diskd) 
cache_dir aufs /var/cache/squid/dysk1 3 32 256
cache_dir aufs /var/cache/squid/dysk2 3 32 256
2x  wd raptor 36GB

 * Anti-virus scanning 

second processor have lot of free time, but first i must tune up squid to 
~130-140 req/s

 * Slow authentication back ends
I don't have authentication backends, ACL from IP (acces filtered by netfilter 
too)

 If none of these issues covers your problem, you might look into
 experimental solutions such as the epoll patch
 (http://devel.squid-cache.org/projects.html#epoll).

I recompiled withoud several options and with patch 
http://devel.squid-cache.org/cgi-bin/diff2/epoll-2_5.patch?s2_5

aragorn squid # squid -v
Squid Cache: Version 2.5.STABLE12
configure options:  --prefix=/usr --bindir=/usr/bin --exec-prefix=/usr 
--sbindir=/usr/sbin --localstatedir=/var --mandir=/usr/share/man 
--sysconfdir=/etc/squid --libexecdir=/usr/lib/squid 
--enable-auth=basic,digest,ntlm --enable-removal-policies=lru,heap 
--enable-linux-netfilter --enable-truncate --with-pthreads --enable-epool 
--disable-follow-x-forwarded-for --host=x86_64-pc-linux-gnu --disable-snmp 
--disable-ssl --enable-underscores --enable-storeio='diskd,coss,aufs,null' 
--enable-async-io


fragmenst of squid.conf:
-- cut -- 
http_port [ip:port]
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_mem 512 MB
maximum_object_size 16384 KB
maximum_object_size_in_memory 16 KB
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF
cache_dir aufs /var/cache/squid/dysk1 3 32 256
cache_dir aufs /var/cache/squid/dysk2 3 32 256
cache_access_log /var/log/squid/access.log
cache_store_log none
mime_table /etc/squid/mime.conf
redirect_children 15
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
request_header_max_size 20 KB
refresh_pattern -i (.*jpg$|.*gif$|.*png$) 0 50% 28800
refresh_pattern -i (.*html$|.*htm|.*shtml|.*php) 0 20% 1440
refresh_pattern .               0       20%     4320
half_closed_clients off
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl administracja src 82.160.43.0/24
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443 563     # https, snews
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl Safe_ports port 901         # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
cache_mgr admin
http_access allow manager localhost
http_access allow manager administracja
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl badURL url_regex -i .wmf$
acl mGG url_regex ^http://adserver.gadu[\-]?gadu.pl/.*$
redirector_access deny !mGG
redirector_bypass on
redirect_program /home/gg_rewrite
acl spywaredomains dstdomain src /etc/squid/spywaredomains.txt
acl our_networks src 82.160.43.0/24 82.160.129.0/24
http_access deny badURL
http_access deny spywaredomains
http_access allow our_networks
http_access allow localhost
http_access deny all
http_reply_access allow all
icp_access allow all
cache_mgr [EMAIL PROTECTED]
visible_hostname w3cache.abp.pl
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
dns_testnames onet.pl wp.pl microsoft.com abp.pl
logfile_rotate 10
append_domain .abp.pl
forwarded_for off
log_icp_queries off
cachemgr_passwd [cut] all
buffered_logs on
coredump_dir /var/cache/squid

RE: [squid-users] low squid performance?

2006-02-22 Thread Chris Robertson
 -Original Message-
 From: Tomasz Kolaj [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, February 22, 2006 2:24 PM
 To: squid-users@squid-cache.org
 Subject: Re: [squid-users] low squid performance?
 
 
 Dnia środa, 22 lutego 2006 23:57, Chris Robertson napisał:
 [cut]
 
  The answer to that question is dependant on a whole host of 
  variables, such
  as ACLs used, whether it's a proxy or an accelerator, the 
  types of clients
  accessing it (client latency has a dramatic effect on CPU 
  usage), types of
  content retrieved, how your cache_dirs are defined, etc.
 
  Various things that can reduce Squid performance:
 
 #^ remove wmf after security leaks on ms wmf file format
 acl mGG url_regex ^http://adserver.gadu[\-]?gadu.pl/.*$

If I'm reading the regex right, you could change this to...

acl mGG dstdomain .adserver.gadugadu.pl .adserver.gadu-gadu.pl

...and you might see a reduction in CPU usage.  I'm not sure how much of one 
though...

 redirector_access deny !mGG
 redirector_bypass on
 redirect_program /home/gg_rewrite
 #^redirector ro replece banner in popular polish comunicator

[cut]

  * High latency clients
 
 What do you mean high latecy clients?
 

The majority of my customers have a network path like:

client-squid-satellite-squid-internet

100 requests/second put my CPU usage in the high 80s (on a 32 bit Intel Xeon 
3.00GHz).

[cut]

 aragorn squid # squid -v
 Squid Cache: Version 2.5.STABLE12
 configure options:  --prefix=/usr --bindir=/usr/bin 
 --exec-prefix=/usr 
 --sbindir=/usr/sbin --localstatedir=/var --mandir=/usr/share/man 
 --sysconfdir=/etc/squid --libexecdir=/usr/lib/squid 
 --enable-auth=basic,digest,ntlm --enable-removal-policies=lru,heap 
 --enable-linux-netfilter --enable-truncate --with-pthreads 
 --enable-epool 

Hopefully that's just a misspelling.  ;o)

 --disable-follow-x-forwarded-for --host=x86_64-pc-linux-gnu 
 --disable-snmp 
 --disable-ssl --enable-underscores 
 --enable-storeio='diskd,coss,aufs,null' 
 --enable-async-io
 
 
 fragmenst of squid.conf:
 -- cut -- 
 http_port [ip:port]
 hierarchy_stoplist cgi-bin ?
 acl QUERY urlpath_regex cgi-bin \?
 no_cache deny QUERY
 cache_mem 512 MB
 maximum_object_size 16384 KB
 maximum_object_size_in_memory 16 KB
 cache_replacement_policy heap GDSF
 memory_replacement_policy heap GDSF
 cache_dir aufs /var/cache/squid/dysk1 3 32 256
 cache_dir aufs /var/cache/squid/dysk2 3 32 256
 cache_access_log /var/log/squid/access.log
 cache_store_log none
 mime_table /etc/squid/mime.conf
 redirect_children 15
 auth_param basic children 5
 auth_param basic realm Squid proxy-caching web server
 auth_param basic credentialsttl 2 hours
 auth_param basic casesensitive off
 request_header_max_size 20 KB
 refresh_pattern -i (.*jpg$|.*gif$|.*png$) 0 50% 28800
 refresh_pattern -i (.*html$|.*htm|.*shtml|.*php) 0 20% 1440
 refresh_pattern .               0       20%     4320
 half_closed_clients off
 acl all src 0.0.0.0/0.0.0.0
 acl manager proto cache_object
 acl localhost src 127.0.0.1/255.255.255.255
 acl administracja src 82.160.43.0/24
 acl to_localhost dst 127.0.0.0/8
 acl SSL_ports port 443 563
 acl Safe_ports port 80          # http
 acl Safe_ports port 21          # ftp
 acl Safe_ports port 443 563     # https, snews
 acl Safe_ports port 70          # gopher
 acl Safe_ports port 210         # wais
 acl Safe_ports port 1025-65535  # unregistered ports
 acl Safe_ports port 280         # http-mgmt
 acl Safe_ports port 488         # gss-http
 acl Safe_ports port 591         # filemaker
 acl Safe_ports port 777         # multiling http
 acl Safe_ports port 901         # SWAT
 acl purge method PURGE
 acl CONNECT method CONNECT
 cache_mgr admin
 http_access allow manager localhost
 http_access allow manager administracja
 http_access deny manager
 http_access allow purge localhost
 http_access deny purge
 http_access deny !Safe_ports
 http_access deny CONNECT !SSL_ports
 acl badURL url_regex -i .wmf$
 acl mGG url_regex ^http://adserver.gadu[\-]?gadu.pl/.*$
 redirector_access deny !mGG
 redirector_bypass on
 redirect_program /home/gg_rewrite
 acl spywaredomains dstdomain src /etc/squid/spywaredomains.txt
 acl our_networks src 82.160.43.0/24 82.160.129.0/24
 http_access deny badURL
 http_access deny spywaredomains
 http_access allow our_networks
 http_access allow localhost
 http_access deny all
 http_reply_access allow all
 icp_access allow all
 cache_mgr [EMAIL PROTECTED]
 visible_hostname w3cache.abp.pl
 httpd_accel_host virtual
 httpd_accel_port 80
 httpd_accel_with_proxy on
 httpd_accel_uses_host_header on
 dns_testnames onet.pl wp.pl microsoft.com abp.pl
 logfile_rotate 10
 append_domain .abp.pl
 forwarded_for off
 log_icp_queries off
 cachemgr_passwd [cut] all
 buffered_logs on
 coredump_dir /var/cache/squid
 store_dir_select_algorithm least-load
 -- cut --
 
 
 Thanks for advice.
 -- 
 Tomasz Kolaj
 

I don't see any other likely problems (not saying there aren't any).

Chris


[squid-users] FILE DESCRIPTORS

2006-02-22 Thread Gregori Parker
Sorry to be pounding the list lately, but I'm about to lose it with
these file descriptors...

I've done everything I have read about to increase file descriptors on
my caching box, and now I just rebuilt a fresh clean squid.  Before I
ran configure, I did ulimit -HSn 8192, and I noticed that while
configuring it said Checking File Descriptors... 8192.  I even
double-checked autoconf.h and saw #define SQUID_MAXFD 8192.  I thought
everything was good, even ran a ulimit -n right before starting squid
and saw 8192!  So I start her up, and in cache.log I see...

2006/02/22 19:05:08| Starting Squid Cache version 2.5.STABLE12 for
x86_64-unknown-linux-gnu...
2006/02/22 19:05:08| Process ID 3657
2006/02/22 19:05:08| With 1024 file descriptors available

Arggghh.

Can anyone help me out?  This is on Fedora Core 4 64-bit

Thanks, sigh - Gregori



Re: [squid-users] management get MSN 24/7

2006-02-22 Thread Brent Clark

Chris Robertson wrote:

See lines inserted below...


acl msntime time M T W H F A 11:59-12:59
acl msntime time M T W H F A 16:59-18:59
acl msnp rep_mime_type ^application/x-msn-messenger$
acl msnq req_mime_type ^application/x-msn-messenger$



# Allow management MSN
http_reply_access allow msnp management
http_reply_access allow msnq management
# Restrict everyone else based on time

Order is critical.

Chris


Hi Chris

Thanks for this.

I appreciate it.

Kind Regards
Brent Clark


Re: [squid-users] Solutions for transparent + proxy_auth?

2006-02-22 Thread Kinkie
On Wed, 2006-02-22 at 11:39 -0600, Steve Brown wrote:
  It seems that I misunderstood what you meant. Do you want the PROXY to
  authenticate against its parent? Independently from who is the user it
  acts in behalf of?
 
 Yes, that's the idea.  All users are restricted to the same ACL, so I
 see no reason to try to auth w/ differnt users, except maybe for
 tracking which computers are going where, but I can figure that out
 later.

Then I don't see why the login option to cache_peer wouldn't work...

Kinkie