Re: [squid-users] Fwd: squid-3.2.0.6 - make issue on OpenBSD 4.8 - 64 bit

2011-04-09 Thread Indunil Jayasooriya
 patch -p0  /PATH/TO/bug3185_mk2.patch

 Exactly correct.

 Being in the base folder of your squid sources when running it
 (/root/software/squid-3.2.0.6/)

DONE. performed below steps.



first,

did cd to /root/software/squid-3.2.0.6/compat/os

backed up as follows.

cp openbsd.h  openbsd.h.orig


then,

did cd to /root/software/squid-3.2.0.6


patch -p0   /tmp/bug3185_mk2.patch

Pls see the output.

Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--
|=== modified file 'compat/os/openbsd.h'
|--- compat/os/openbsd.h2010-11-21 04:40:05 +
|+++ compat/os/openbsd.h2011-04-08 08:10:12 +
--
Patching file compat/os/openbsd.h using Plan A...
Hunk #1 succeeded at 30.
Hmm...  Ignoring the trailing garbage.
done

It worked didn't it? I think yes. your comments are welcome...

then,

# cd /root/software/squid-3.2.0.6/compat/os

# diff openbsd.h openbsd.h.orig

33,40d32
 /* OpenBSD requires netinet/in.h before arpa/inet.h */
 #if HAVE_NETINET_IN_H
 #include netinet/in.h
 #endif
 #if HAVE_ARPA_INET_H
 #include arpa/inet.h
 #endif



anyway , now the full contents of openbsd.h is as follows.

# cat openbsd.h

#ifndef SQUID_OS_OPENBSD_H
#define SQUID_OS_OPENBSD_H

#ifdef _SQUID_OPENBSD_

/
 *--*
 * DO *NOT* MAKE ANY CHANGES below here unless you know what you're doing...*
 *--*
 /

/*
 * Don't allow inclusion of malloc.h
 */
#if HAVE_MALLOC_H
#undef HAVE_MALLOC_H
#endif

/*
 *   This OS has at least one version that defines these as private
 *   kernel macros commented as being 'non-standard'.
 *   We need to use them, much nicer than the OS-provided __u*_*[]
 */
//#define s6_addr8  __u6_addr.__u6_addr8
//#define s6_addr16 __u6_addr.__u6_addr16
#define s6_addr32 __u6_addr.__u6_addr32

/* OpenBSD also hide v6only socket option we need for comm layer. :-( */
#if !defined(IPV6_V6ONLY)
#define IPV6_V6ONLY 27 // from OpenBSD 4.3 headers. (NP:
does not match non-BSD OS values)
#endif

/* OpenBSD requires netinet/in.h before arpa/inet.h */
#if HAVE_NETINET_IN_H
#include netinet/in.h
#endif
#if HAVE_ARPA_INET_H
#include arpa/inet.h
#endif

#endif /* _SQUID_OPENBSD_ */
#endif /* SQUID_OS_OPENBSD_H */



That's all for that patch. I think U r ok.



anyway. for the /dev/pf thing, I will come back with an update



-- 
Thank you
Indunil Jayasooriya


Re: [squid-users] squid 3.2.0.5 smp scaling issues

2011-04-09 Thread Amos Jeffries

On 09/04/11 14:27, da...@lang.hm wrote:

A couple more things about the ACLs used in my test

all of them are allow ACLs (no deny rules to worry about precidence of)
except for a deny-all at the bottom

the ACL line that permits the test source to the test destination has
zero overlap with the rest of the rules

every rule has an IP based restriction (even the ones with url_regex are
source - URL regex)

I moved the ACL that allows my test from the bottom of the ruleset to
the top and the resulting performance numbers were up as if the other
ACLs didn't exist. As such it is very clear that 3.2 is evaluating every
rule.

I changed one of the url_regex rules to just match one line rather than
a file containing 307 lines to see if that made a difference, and it
made no significant difference. So this indicates to me that it's not
having to fully evaluate every rule (it's able to skip doing the regex
if the IP match doesn't work)

I then changed all the acl lines that used hostnames to have IP
addresses in them, and this also made no significant difference

I then changed all subnet matches to single IP address (just nuked /##
throughout the config file) and this also made no significant difference.



Squid has always worked this way. It will *test* every rule from the top 
down to the one that matches. Also testing each line left-to-right until 
one fails or the whole line matches.




so why are the address matches so expensive



3.0 and older IP address is a 32-bit comparison.
3.1 and newer IP address is a 128-bit comparison with memcmp().

If something like a word-wise comparison can be implemented faster than 
memcmp() we would welcome it.




and as noted in the e-mail below, why do these checks not scale nicely
with the number of worker processes? If they did, the fact that one 3.2
process is about 1/3 the speed of a 3.0 process in checking the acls
wouldn't matter nearly as much when it's so easy to get an 8+ core system.



There you have the unknown.



it seems to me that all accept/deny rules in a set should be able to be
combined into a tree to make searching them very fast.

so for example if you have

accept 1
accept 2
deny 3
deny 4
accept 5

you need to create three trees (one with accept 1 and accept 2, one with
deny3 and deny4, and one with accept 5) and then check each tree to see
if you have a match.

the types of match could be done in order of increasing cost, so if you


The config file is specific structure configured by admin under 
guaranteed rules of operation for access lines (top-down, left-to-right, 
first-match-wins) to perform boolean-logic calculations using ACL sets.

 Sorting access line rules is not an option.
 Sorting ACL values and tree-forming them is already done (regex being 
the one exception AFAIK).
 Sorting position-wise on a single access line is also ruled out by 
interactions with deny_info, auth and external ACL types.




have acl entries of type port, src, dst, and url regex, organize the
tree so that you check ports first, then src, then dst, then only if all
that matches do you need to do the regex. This would be very similar to
the shortcut logic that you use today with a single rule where you bail
out when you don't find a match.

you could go with a complex tree structure, but since this only needs to
be changed at boot time,


Um, boot/startup time and arbitrary -k reconfigure times.
With a reverse-configuration display dump on any cache manager request.


it seems to me that a simple array that you can
do a binary search on will work for the port, src, and dst trees. The
url regex is probably easiest to initially create by just doing a list
of regex strings to match and working down that list, but eventually it


This is already how we do these. But with a splay tree instead of binary.


may be best to create a parse tree so that you only have to walk down
the string once to see if you have a match.


That would be nice. Care to implement?
 You just have to get the regex library to adjust its pre-compiled 
patterns with OR into (existing|new) whenever a new pattern string is 
added to an ACL.




you wouldn't quite be able to get this fast as you would have to
actually do two checks, one if you have a match on that level and one
for the rules that don't specify something in the current tree (one
check for if the http_access line specifies a port number and one for if
it doesn't for example)


We get around this problem by using C++ types. ACLChecklist walks the 
tree holding the current location, expected result, and all details 
available about the transaction. Each node in the tree has a match() 
function which gets called at most once per walk. Each ACL data type 
provides its own match() algorithm.


That is why the following config is invalid:
 acl foo src 1.2.3.4
 acl foo port 80



this sort of acl structure would reduce a complex ruleset down to ~O(log
n) instead of the current O(n) (a really complex ruleset would be log n
of each tree added 

Re: [squid-users] Fwd: squid-3.2.0.6 - make issue on OpenBSD 4.8 - 64 bit

2011-04-09 Thread Indunil Jayasooriya
 anyway. for the /dev/pf thing, I will come back with an update


As I said, below two commands NOT good at all.

# chgrp _squid /dev/pf
# chmod g+rw /dev/pf

Now, we have to use

divert-to instead of rdr-to  in pf.conf


Pls read below URL where you get the real thing in regard to it. It
was replied by OpenBSD developer Reyk Floeter.


http://www.mail-archive.com/misc@openbsd.org/msg101469.html


I am home now, I am going to office on monday. then, I will do
accordingly and update you.



-- 
Thank you
Indunil Jayasooriya


Re: [squid-users] Fwd: squid-3.2.0.6 - make issue on OpenBSD 4.8 - 64 bit

2011-04-09 Thread Amos Jeffries

On 09/04/11 21:24, Indunil Jayasooriya wrote:

anyway. for the /dev/pf thing, I will come back with an update



As I said, below two commands NOT good at all.

# chgrp _squid /dev/pf
# chmod g+rw /dev/pf

Now, we have to use

divert-to instead of rdr-to  in pf.conf


Pls read below URL where you get the real thing in regard to it. It
was replied by OpenBSD developer Reyk Floeter.


http://www.mail-archive.com/misc@openbsd.org/msg101469.html



Aha! so PF provides getsockname() now. That means it will require the 
./configure --enable-ipfw-transparent option to Squid.


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.6


Re: [squid-users] Problems with transparancy and pf

2011-04-09 Thread Amos Jeffries

On 07/04/11 08:03, Leslie Jensen wrote:


On 2011-04-06 05:32, Amos Jeffries wrote:



Thank you. I've split the wiki examples we have for PF into separate
OpenBSD and FreeBSD pages and added a new section for the altered
OpenBSD syntax.

Would any of you mind reading through and checking the texts? please?
http://wiki.squid-cache.org/ConfigExamples/Intercept/OpenBsdPf
http://wiki.squid-cache.org/ConfigExamples/Intercept/FreeBsdPf

Amos


For squid31 on Free BSD there are several options already set.

I think it would be helpful to mention a little more in the wiki.

The configure options that are available are:
(From the Makefile)

OPTIONS= SQUID_KERB_AUTH Install Kerberos authentication helpers on \
SQUID_LDAP_AUTH Install LDAP authentication helpers off \
SQUID_NIS_AUTH Install NIS/YP authentication helpers on \
SQUID_SASL_AUTH Install SASL authentication helpers off \
SQUID_IPV6 Enable IPv6 support on \
SQUID_DELAY_POOLS Enable delay pools off \
SQUID_SNMP Enable SNMP support on \
SQUID_SSL Enable SSL support for reverse proxies off \
SQUID_PINGER Install the icmp helper off \
SQUID_DNS_HELPER Use the old 'dnsserver' helper off \
SQUID_HTCP Enable HTCP support on \
SQUID_VIA_DB Enable forward/via database off \
SQUID_CACHE_DIGESTS Enable cache digests off \
SQUID_WCCP Enable Web Cache Coordination Prot. v1 on \
SQUID_WCCPV2 Enable Web Cache Coordination Prot. v2 off \
SQUID_STRICT_HTTP Be strictly HTTP compliant off \
SQUID_IDENT Enable ident (RFC 931) lookups on \
SQUID_REFERER_LOG Enable Referer-header logging off \
SQUID_USERAGENT_LOG Enable User-Agent-header logging off \
SQUID_ARP_ACL Enable ACLs based on ethernet address off \
SQUID_IPFW Enable transparent proxying with IPFW off \
SQUID_PF Enable transparent proxying with PF off \
SQUID_IPFILTER Enable transp. proxying with IPFilter off \
SQUID_FOLLOW_XFF Follow X-Forwarded-For headers off \
SQUID_ECAP En. loadable content adaptation modules off \
SQUID_ICAP Enable ICAP client functionality off \
SQUID_ESI Enable ESI support (experimental) off \
SQUID_AUFS Enable the aufs storage scheme on \
SQUID_COSS Enable COSS (currently not available) off \
SQUID_KQUEUE Use kqueue(2) (experimental) on \
SQUID_LARGEFILE Support log and cache files 2GB off \
SQUID_STACKTRACES Create backtraces on fatal errors off \
SQUID_DEBUG Enable debugging options off



Thank you. I see none of the NAT lookup features are turned on.

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.6


Re: [squid-users] pam_auth pam_end()

2011-04-09 Thread Eugene M. Zheganin

Hi.

On 15.03.2011 16:54, Amos Jeffries wrote:


Start with the -d option.
 Then add/update debug() lines to any place that looks useful. I'm 
interested in making the debug helpful so patches for that are welcome 
upstream.
 debug() operates identical to printf() but sends the result to the 
helper channel for Squid cache.log.


FWIW, I think adding pam_strerror() results into both of the WARNING: 
messages with that text should be enough to point at the actual problem.


Well... I did all of that (and it didn't help). By the way, debug seems 
to be a macro, rather than a squid channel logging function (could it be 
even possible ? main part of squid 3.x is written in C++ and the helper 
part - in C). Anyway, may be it's time to describe my problem, rather 
than to describe the solution as I see it. :)


Okay, the problem description: as I said I have a proxy. That's the 
company main proxy, and the wpad for the network of more than 2K 
machines points at it. So, during the weekdays I have loads of requests 
from all sorts of clients, most of them remains blocked, but all of the 
basic authentication requests are handled by pam_auth. I have 35 
simultaneously running pam_auth processes. During load peaks I ususally 
have 3-5 (sometimes even more) pam_auth processes that eat 100% of the 
both CPUs all together. I used to think that those are the processes 
that squid failed to release. But, when I kill some of it to release the 
CPUs from unnecessary load, squid complains in its log like that:


WARNING: basicauthenticator #8 (FD 93) exited

It's obvious that I'm wrong and this isn't the helper squid cannot 
release, but this is the actually running helper. So the questions are


- why only small parts of basic helpers are affected with such load ?
- why such load even exists ? when I kill affected processes squid 
continues to run without influencing its clients for some time. Then the 
load appears again.

- and, of course, what can be done to solve this.

I had a look at the code of the helper - it seems to be very 
straightforward and simple, so I don't see how such a simple code can 
eat CPU.


The basic helper config is:

auth_param basic program /usr/local/libexec/squid/pam_auth
auth_param basic children 35
auth_param basic realm Squid[Kamtelecom]
auth_param basic credentialsttl 1 minute
auth_param basic casesensitive off

and the pam config for the squid service name is:

authsufficient  pam_unix.so no_warn
authsufficient  /usr/local/lib/pam_winbind.so   
try_first_pass
authsufficient  pam_krb5.so no_warn 
try_first_pass


authrequiredpam_deny.so no_warn


(yup, I use the AD authentication scheme).


Thanks.
Eugene.


Re: [squid-users] pam_auth pam_end()

2011-04-09 Thread Amos Jeffries

On 10/04/11 01:09, Eugene M. Zheganin wrote:

Hi.

On 15.03.2011 16:54, Amos Jeffries wrote:


Start with the -d option.
Then add/update debug() lines to any place that looks useful. I'm
interested in making the debug helpful so patches for that are welcome
upstream.
debug() operates identical to printf() but sends the result to the
helper channel for Squid cache.log.

FWIW, I think adding pam_strerror() results into both of the WARNING:
messages with that text should be enough to point at the actual problem.


Well... I did all of that (and it didn't help). By the way, debug seems
to be a macro, rather than a squid channel logging function (could it be
even possible ? main part of squid 3.x is written in C++ and the helper
part - in C). Anyway, may be it's time to describe my problem, rather
than to describe the solution as I see it. :)

Okay, the problem description: as I said I have a proxy. That's the
company main proxy, and the wpad for the network of more than 2K
machines points at it. So, during the weekdays I have loads of requests
from all sorts of clients, most of them remains blocked, but all of the
basic authentication requests are handled by pam_auth. I have 35
simultaneously running pam_auth processes. During load peaks I ususally
have 3-5 (sometimes even more) pam_auth processes that eat 100% of the
both CPUs all together. I used to think that those are the processes
that squid failed to release. But, when I kill some of it to release the
CPUs from unnecessary load, squid complains in its log like that:

WARNING: basicauthenticator #8 (FD 93) exited

It's obvious that I'm wrong and this isn't the helper squid cannot
release, but this is the actually running helper. So the questions are

- why only small parts of basic helpers are affected with such load ?


credentials tests are requested of the first available helper program.

Under load every time the first helper finished it gets a new request to 
handle, so is constantly in use. So the first few will get all the load 
and later ones will only get load when the earlier one(s) fill up.



- why such load even exists ? when I kill affected processes squid
continues to run without influencing its clients for some time. Then the
load appears again.


That is unclear. It could be anything from that being the actual request 
load, to a config design problem causing unnecessary calls to the auth 
helpers, to a problem in PAM dong a lot of extra work for nothing.



- and, of course, what can be done to solve this.

I had a look at the code of the helper - it seems to be very
straightforward and simple, so I don't see how such a simple code can
eat CPU.


If it is called too often it can east CPU, or if the PAM backend is the 
actual CPU consumer.




The basic helper config is:

auth_param basic program /usr/local/libexec/squid/pam_auth
auth_param basic children 35
auth_param basic realm Squid[Kamtelecom]
auth_param basic credentialsttl 1 minute


60 seconds between checks with the PAM helper will raise load. On small 
networks with few clients this is not a problem, but larger ones it 
could be.



auth_param basic casesensitive off

and the pam config for the squid service name is:

auth sufficient pam_unix.so no_warn
auth sufficient /usr/local/lib/pam_winbind.so try_first_pass
auth sufficient pam_krb5.so no_warn try_first_pass

auth required pam_deny.so no_warn



I don't believe pam_winbind or pam_krb5 will work with this config using 
Basic auth. They are for NTLM and Negotiate auth respectively.


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.6


Re: [squid-users] squid 3.2.0.5 smp scaling issues

2011-04-09 Thread david

On Sat, 9 Apr 2011, Amos Jeffries wrote:


On 09/04/11 14:27, da...@lang.hm wrote:

A couple more things about the ACLs used in my test

all of them are allow ACLs (no deny rules to worry about precidence of)
except for a deny-all at the bottom

the ACL line that permits the test source to the test destination has
zero overlap with the rest of the rules

every rule has an IP based restriction (even the ones with url_regex are
source - URL regex)

I moved the ACL that allows my test from the bottom of the ruleset to
the top and the resulting performance numbers were up as if the other
ACLs didn't exist. As such it is very clear that 3.2 is evaluating every
rule.

I changed one of the url_regex rules to just match one line rather than
a file containing 307 lines to see if that made a difference, and it
made no significant difference. So this indicates to me that it's not
having to fully evaluate every rule (it's able to skip doing the regex
if the IP match doesn't work)

I then changed all the acl lines that used hostnames to have IP
addresses in them, and this also made no significant difference

I then changed all subnet matches to single IP address (just nuked /##
throughout the config file) and this also made no significant difference.



Squid has always worked this way. It will *test* every rule from the top down 
to the one that matches. Also testing each line left-to-right until one fails 
or the whole line matches.




so why are the address matches so expensive



3.0 and older IP address is a 32-bit comparison.
3.1 and newer IP address is a 128-bit comparison with memcmp().

If something like a word-wise comparison can be implemented faster than 
memcmp() we would welcome it.


I wonder if there should be a different version that's used when IPv6 is 
disabled. this is a pretty large hit.


if the data is aligned properly, on a 64 bit system this should still only 
be 2 compares. do you do any alignment on the data now?



and as noted in the e-mail below, why do these checks not scale nicely
with the number of worker processes? If they did, the fact that one 3.2
process is about 1/3 the speed of a 3.0 process in checking the acls
wouldn't matter nearly as much when it's so easy to get an 8+ core system.



There you have the unknown.


I think this is a fairly critical thing to figure out.



it seems to me that all accept/deny rules in a set should be able to be
combined into a tree to make searching them very fast.

so for example if you have

accept 1
accept 2
deny 3
deny 4
accept 5

you need to create three trees (one with accept 1 and accept 2, one with
deny3 and deny4, and one with accept 5) and then check each tree to see
if you have a match.

the types of match could be done in order of increasing cost, so if you


The config file is specific structure configured by admin under guaranteed 
rules of operation for access lines (top-down, left-to-right, 
first-match-wins) to perform boolean-logic calculations using ACL sets.

Sorting access line rules is not an option.
Sorting ACL values and tree-forming them is already done (regex being the 
one exception AFAIK).
Sorting position-wise on a single access line is also ruled out by 
interactions with deny_info, auth and external ACL types.


It would seem that as long as you don't cross boundries between the 
different types, you should be able to optimize within a group.


using my example above, you couldn't combine the 'accept 5' with any of 
the other accepts, but you could combine accept 1 and 2 and combine deny 3 
and 4 togeather.


now, I know that I don't fully understand all the possible ACL types, so 
this may not work for some of them, but I believe that a fairly common use 
case is to have either a lot of allow rules, or a lot of deny rules as a 
block (either a list of sites you are allowed to access, or a list of 
sites that are blocked), so an ability to optimize these use cases may be 
well worth it.



have acl entries of type port, src, dst, and url regex, organize the
tree so that you check ports first, then src, then dst, then only if all
that matches do you need to do the regex. This would be very similar to
the shortcut logic that you use today with a single rule where you bail
out when you don't find a match.

you could go with a complex tree structure, but since this only needs to
be changed at boot time,


Um, boot/startup time and arbitrary -k reconfigure times.
With a reverse-configuration display dump on any cache manager request.


still a pretty rare case, and one where you can build a completely new 
ruleset and swap it out. My point was that this isn't something that you 
have to be able to update dynamically.



it seems to me that a simple array that you can
do a binary search on will work for the port, src, and dst trees. The
url regex is probably easiest to initially create by just doing a list
of regex strings to match and working down that list, but eventually it


This is already how we do these. But 

Re: [squid-users] pam_auth pam_end()

2011-04-09 Thread Eugene M. Zheganin

On 09.04.2011 19:50, Amos Jeffries wrote:

- why such load even exists ? when I kill affected processes squid
continues to run without influencing its clients for some time. Then the
load appears again.


That is unclear. It could be anything from that being the actual 
request load, to a config design problem causing unnecessary calls to 
the auth helpers, to a problem in PAM dong a lot of extra work for 
nothing.
Well, you told earlier that under heavy load first few helpers receive 
the majority of work. Lets assume I have 5 helpers that eat CPU, as it 
really happens sometimes. In the next moment I kill them (I do this 
rather often). Considering the  assumption that CPU load is caused by 
actual needs, such as repeating authentication, not some 'stucking' in 
the PAM framework or helper code, and in the same time - low probability 
of such load to end in the exact same moment when I kill helpers, it has 
to continue, and next bunch of helpers should receive this load and 
start to eat CPU. In reality that doesn't happen, CPU becomes idle.




The basic helper config is:

auth_param basic program /usr/local/libexec/squid/pam_auth
auth_param basic children 35
auth_param basic realm Squid[Kamtelecom]
auth_param basic credentialsttl 1 minute


60 seconds between checks with the PAM helper will raise load. On 
small networks with few clients this is not a problem, but larger ones 
it could be.



auth_param basic casesensitive off

and the pam config for the squid service name is:

auth sufficient pam_unix.so no_warn
auth sufficient /usr/local/lib/pam_winbind.so try_first_pass
auth sufficient pam_krb5.so no_warn try_first_pass

auth required pam_deny.so no_warn



I don't believe pam_winbind or pam_krb5 will work with this config 
using Basic auth. They are for NTLM and Negotiate auth respectively.
So, then the pam_unix.so should work. But I don't have 2K AD users on 
any of these FreeBSD, I have like 30 local users. Actually I'm not that 
sure about pam_winbind.so, but pam_krb5.so definitely can process 
plaintext passwords. As kinit does. I suppose pam_winbind.so is also 
able to handle plaintext passwords, just by the fact that wbinfo can.


Thanks.
Eugene.


Re: [squid-users] Percentege cache

2011-04-09 Thread Eliezer Croitoru

On 09/04/2011 04:22, igor rocha wrote:


I  more thank Amos, but if anyone has any other tips, information, help me

adding to Amos,
the common ISP setup is 30-50 percent but it really depends on the 
knowledge and usage of the ISP cache operators.
caching using helpers instead of the ordinary setup can lead to almost 
like REVERSE proxy percentage.(80-90).


the main problem is that web developers or companies are not that into 
building cache aware sites.
many dynamic content based websites using dynamic headers for a static 
content from unknown reasons.
i do understand some of youtube and other sites concerns about their 
content but the other side is that
they do not want to even start thinking about considering helping ISP's 
and companies to work it around.

(at least not on the table)

i have my own proxy that serves SOHO environment with almost specific 
usage that has a hit ratio of about 80%.

(the refresh patterns are  costumed and im using some helpers i have built).

Eliezer


2011/4/8 Amos Jeffriessqu...@treenet.co.nz:

On 09/04/11 02:49, igor rocha wrote:

?

2011/4/8 igor rochaigorlo...@gmail.com:

Hello,
I know that does not formulate the right question, I am Brazilian and
not mastered English well, but talk about commonly used metrics to
measure the effectiveness of the cache and the amount of bandwidth
saved is hit ratio, defined as the percentage of requests that are
satisfied by the proxy as cache hits.Show me the average percentage of
index pages that are not cached,a paper.

understand?

Ah. I think so.

I'm not aware of any papers on that. It is highly variable between networks.
  We do have two general rule-of-thumbs;
  * that reverse-proxy (CDN) see hit ratios usually around 80%-99% for a
website.
  * that forward-proxy (ISP) see hit rations between 25% and 45%.

with variance outside of those ranges for older Squid versions and poorly
written websites.

This is general-knowledge built up from years of small talks with people
looking at and discussion of their cache ratios. Nothing published exactly.

We have in recent years attempted to collects statistics on these. Which can
be found at http://wiki.squid-cache.org/KnowledgeBase/Benchmarks along with
the methodology used for collection.

Amos


2011/4/8 Amos Jeffries wrote:

On 08/04/11 02:48, igor rocha wrote:

Hello Gentlemen,
could anyone tell me what percentage of sites that are required for
the cache and actually go into the cache, can be an article that
talksabout it, something that helps me to have concrete statistical
data.

Please explain your meaning of required for the cache.

100% of squid cacheable sites get cached. Admin often force100% to be
cached.

I suspect you mean something else though.

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.6



--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.6





Re: [squid-users] question on recompiling

2011-04-09 Thread Amos Jeffries

On 08/04/11 00:35, Leonardo Rodrigues wrote:


Hi,

i have squid 2.7-stable9 compiled and running just fine on a CentOS 5.5
x86_64 box.

because of some bugs on krb5-libs shipped with CentOS, i need to
recompile a single helper (negotiate/squid_kerb_auth) linking with an
updated krb5 lib which was already compiled and stored on /usr/local.


 -L is the library .so location parameter.
 -I is the local library header files location parameter

In 2.7 this should be as simple as:

  LDFLAGS=-L/usr/local/lib64/ \
  CFLAGS=-I/usr/local/include/ \
  ./configure
  cd helpers/negotiate_auth/squid_kerb_auth/
  make

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.6