[squid-users] upgrade but leave earlier version running?

[J Webster]

Is there a way to install the new version of squid and leave 2.6 running 
and then swpa them over once I am sure everything in verison 3 is 
running on the server ok?
I don;t believe CentOS 5.8 has anything in the repos above 2.6 so is 
there a way I can use yum without installing from source and compiling?

Re: [squid-users] Put all port 80, 443 http https rtmp connections from openvpn through squid?

[J Webster]

rtmp can be used on squid with a big BUT.
since rtmp is a tcp protocol you must allow a CONNECT and destination 
ports to be used through the proxy.

but it's not such a safe and good idea to do so.
since the squid box is a router in your case and you will intercept 
the port 80\443 rtmp will not have any trouble if you do use NAT for 
outgoing connections since rtmp works on other ports then 80 and 443.

But the routing will be different somehow won't it?
For example, let's assume youtube uses rtmp.
A user connects via VPN, navigates to www.youtube.com, on the VPN server 
the 80 request is directed through squid, the video server returns the 
80 request and a rtmp request but the rtmp cannot go through squid so 
where does it return, just another port on the VPN server? As long as I 
leave those rtmp ports open then all is okay?
What if there are 50 clients all using rtmp as the same time, how would 
the routing within the 10.8.x.x network happen with squid involved?

Re: [squid-users] block dodgy sites with squidguard

[J Webster]

I'm not even sure how I could block this though.  Anyone who publishes a 
website with illegal pictures is probably not going to list keywords 
that I can search for and prevent.

Would those illegal sites show up in the dest porn blocklist?

On 12/08/12 08:45, Amos Jeffries wrote:

On 11/08/2012 7:46 p.m., J Webster wrote:
I would like to allow access to some 18+ sites on the proxy server 
but prevent anyone looking at dodgy illegal sites.
Is there a way to do that with squidguard because the filter dest 
only seems to be on porn.

Would I have to write my own access control list to prevent this?


That is the best way. You are very unlikely to finad a public list of 
18+ sites that exactly matches your particular set of okay 18+ sites.


Use a public blocklist of porn site if you wish. Add a custom ACL for 
the whitelist to permit them.


You can probably do that is SG, but I would recommend the whitelist be 
used in squid.conf to deny url_rewrite_access. Which makes them not 
even process through SG.


Amos

Re: [squid-users] Put all port 80, 443 http https rtmp connections from openvpn through squid?

[J Webster]

squid is a http proxy and not rtmp.
rtmp use other ports then 80\443 and cannot be used over squid(you can 
if it's tcp and you allow CONNECT and unsafe ports which is not safe.. 
and will make the vpn connection vulnerable and maybe useless)


if you have a solid reason to do so it can be a nice project to try.

a more simple way is to assign dedicated IP for each certificate\client.

Regards,
Eliezer

The reason I asked about rtmp is that many sites you access the video 
via the web browser but it sends it back via rtmp.

So, this is not possible through squid at all?
However, it is possible in a direct connection. So, can you allow 80,443 
to go through squid but accept the return directly if on rtmp? probably not.


So, assign a static IP to a certificate and then have squid log by IP 
address, then have a program match up the ip at the time with the client 
name?

[squid-users] squidguard spyware log

[J Webster]

I see some logs of spyware sites being blocked by squidguard.
I presume these are sites that have cross domain xml or javascript or 
other things built in.
Will squidguard block the whole page even if there is one script in it 
that might be spyware?


2012-08-11 17:10:31 [3630] Request(default/spyware/-) 
http://won.images.streamray.com/images/streamray/won/jpg/m/6/milf36_150.jpg 
93.23.197.116/- user GET $
2012-08-11 17:10:36 [3630] Request(default/spyware/-) 
http://graphics.streamray.com/crossdomain.xml 93.23.197.116/- user GET 
REDIRECT


Is there a way to provide a page saying this site has malware and has 
been blocked rather than just the default block page?

ie 2 different blocking html pages?

Re: [squid-users] Put all port 80, 443 http https rtmp connections from openvpn through squid?

[J Webster]

But once the tunnel reaches the OpenVPN server, you can direct port 80

and 443 traffic from it via the proxy server can't you?
Once it gets to the OpenVPN server (where you would also have the proxy
server), isn't it decrypted?
Lots of companies have VPN tunnels and then route web traffic through a
proxy so it must be possible somehow.

On 11/08/12 13:54, Alex Crow wrote:

On 11/08/12 08:20, J Webster wrote:

Is there a way to push all openvpn connections using http ports
through a transparent squid and how?
Also, can I log which openvpn certificate/client is accessing which
pages in this way?
I assume I would have to use an alternative port or use firewall
rules to only allow squid connections from the network 10.8.x.x

Squid is an HTTP proxy, so no.

You can't really proxy OpenVPN as it's end-to-end encrypted with SSL.
If you issued the certs from your CA it might be possible to MITM it
but that may be illegal in many jurisdictions.

Alex




of course you can.
it's a basic IPTABLES rules and since openvpn uses a tunX interface 
you can intercept all traffic from the tunX interface to the proxy.
but you cant force the clients to use the vpn as gateway to the whole 
word but only to the VPN connection.


Regards,
Eliezer



So, I simply forward port 80 and 443 on network 10.8.00 to a transparent 
squid proxy?
How can I record in the squid logs which OpenVPN client certificate is 
using the proxy?
Also, how do I do this for rtmp connections because port 80 and 443 will 
have to go via the proxy but rtmp will have to bypass it somehow?

Re: [squid-users] Put all port 80, 443 http https rtmp connections from openvpn through squid?

[J Webster]

But once the tunnel reaches the OpenVPN server, you can direct port 80 
and 443 traffic from it via the proxy server can't you?
Once it gets to the OpenVPN server (where you would also have the proxy 
server), isn't it decrypted?
Lots of companies have VPN tunnels and then route web traffic through a 
proxy so it must be possible somehow.


On 11/08/12 13:54, Alex Crow wrote:

On 11/08/12 08:20, J Webster wrote:
Is there a way to push all openvpn connections using http ports 
through a transparent squid and how?
Also, can I log which openvpn certificate/client is accessing which 
pages in this way?
I assume I would have to use an alternative port or use firewall 
rules to only allow squid connections from the network 10.8.x.x

Squid is an HTTP proxy, so no.

You can't really proxy OpenVPN as it's end-to-end encrypted with SSL. 
If you issued the certs from your CA it might be possible to MITM it 
but that may be illegal in many jurisdictions.


Alex

[squid-users] take out something from squidguard.conf without restarting squid

[J Webster]

If I want to remove one of the dest restrictions form squidguard, how 
can I do this without restarting squid?

[squid-users] block dodgy sites with squidguard

[J Webster]

I would like to allow access to some 18+ sites on the proxy server but 
prevent anyone looking at dodgy illegal sites.
Is there a way to do that with squidguard because the filter dest only 
seems to be on porn.

Would I have to write my own access control list to prevent this?

Re: [squid-users] squidguard not blocking

[J Webster]

It wa spermissions on all the conf and db files - I thought I changed 
them but apparently not:

2012-08-11 08:41:22 [1096] init urllist /var/lib/squidguard/weapons/urls
2012-08-11 08:41:22 [1096] loading dbfile 
/var/lib/squidguard/weapons/urls.db

2012-08-11 08:41:22 [1093] squidGuard 1.3 started (1344670882.380)
2012-08-11 08:41:22 [1093] squidGuard ready for requests (1344670882.538)
2012-08-11 08:41:22 [1096] squidGuard 1.3 started (1344670882.382)
2012-08-11 08:41:22 [1096] squidGuard ready for requests (1344670882.538)
2012-08-11 08:41:22 [1094] init urllist /var/lib/squidguard/weapons/urls
2012-08-11 08:41:22 [1094] loading dbfile 
/var/lib/squidguard/weapons/urls.db

2012-08-11 08:41:22 [1094] squidGuard 1.3 started (1344670882.382)
2012-08-11 08:41:22 [1094] squidGuard ready for requests (1344670882.538)

Re: [squid-users] squidguard not blocking

[J Webster]

#
# CONFIG FILE FOR SQUIDGUARD
#

#dbhome /usr/local/squidGuard/db
#logdir /usr/local/squidGuard/logs
dbhome /var/lib/squidguard
logdir /var/log/squidguard


dest porn {
domainlist porn/domains
urllist porn/urls
}

dest aggressive {
domainlist aggressive/domains
urllist aggressive/urls
}
dest hacking{
domainlist hacking/domains
urllist hacking/urls
}
dest religion   {
domainlist religion/domains
urllist religion/urls
}
dest spyware{
domainlist spyware/domains
urllist spyware/urls
}
dest violence   {
domainlist violence/domains
urllist violence/urls
}
dest weapons{
domainlist weapons/domains
urllist weapons/urls
}

acl {
 default {
pass !porn !aggressive !hacking !religion !spyware 
!violence !weapons !in-addr all

redirect http://www.mysite.co.uk/blockaccess.php
}
 }


[root ~]# service squid restart
Stopping squid:    [  OK  ]
Starting squid: .  [  OK  ]
[root squidguard]# date
Sat Aug 11 08:27:00 BST 2012
[root squidguard]# tail -f squidGuard.log
2012-08-10 17:26:39 [28522] loading dbfile 
/var/lib/squidguard/violence/domains.db

2012-08-10 17:26:39 [28522] init urllist /var/lib/squidguard/violence/urls
2012-08-10 17:26:39 [28522] loading dbfile 
/var/lib/squidguard/violence/urls.db
2012-08-10 17:26:39 [28522] init domainlist 
/var/lib/squidguard/weapons/domains
2012-08-10 17:26:39 [28522] loading dbfile 
/var/lib/squidguard/weapons/domains.db

2012-08-10 17:26:39 [28522] init urllist /var/lib/squidguard/weapons/urls
2012-08-10 17:26:39 [28522] loading dbfile 
/var/lib/squidguard/weapons/urls.db

2012-08-10 17:26:39 [28522] squidGuard 1.3 started (1344615999.035)
2012-08-10 17:26:39 [28522] squidGuard ready for requests (1344615999.039)
2012-08-10 17:26:39 [28522] squidGuard stopped (1344615999.040)







On 10/08/12 23:49, Go Wow wrote:

Is squidguard log config in squidguard.conf file? If not config the log and 
watch the log whether the traffic is hitting SG or not. I feel there some 
config issue in SG.

Let us see your config files and client IP or username.
-Sent via Blackberry

[squid-users] Put all port 80, 443 http https rtmp connections from openvpn through squid?

[J Webster]

Is there a way to push all openvpn connections using http ports through 
a transparent squid and how?
Also, can I log which openvpn certificate/client is accessing which 
pages in this way?
I assume I would have to use an alternative port or use firewall rules 
to only allow squid connections from the network 10.8.x.x

Re: [squid-users] squidguard not blocking

[J Webster]

it is, I get the server IP address when browsing and log is full of HIT MISS 
lines etc

On 10 Aug 2012, at 22:30, "Go Wow"  wrote:

> Check access.log and verify whether the traffic is passing through squid from 
> the client machine.
> 
> -Sent via Blackberry
> 
> -Original Message-
> From: J Webster 
> Date: Fri, 10 Aug 2012 20:34:31 
> To: 
> Subject: [squid-users] squidguard not blocking
> squidguard correctly blocks when I run from the command line:
> [root squidguard]# echo "http://www.porn.com/ - - GET" | squidGuard -c 
> /etc/squid/squidguard.conf -d
> 2012-08-10 17:45:22 [28923] New setting: dbhome: /var/lib/squidguard
> 2012-08-10 17:45:22 [28923] New setting: logdir: /var/log/squidguard
> 2012-08-10 17:45:22 [28923] init domainlist /var/lib/squidguard/porn/domains
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/porn/domains.db
> 2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/porn/urls
> 2012-08-10 17:45:22 [28923] loading dbfile /var/lib/squidguard/porn/urls.db
> 2012-08-10 17:45:22 [28923] init domainlist 
> /var/lib/squidguard/aggressive/domains
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/aggressive/domains.db
> 2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/aggressive/urls
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/aggressive/urls.db
> 2012-08-10 17:45:22 [28923] init domainlist 
> /var/lib/squidguard/hacking/domains
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/hacking/domains.db
> 2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/hacking/urls
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/hacking/urls.db
> 2012-08-10 17:45:22 [28923] init domainlist 
> /var/lib/squidguard/religion/domains
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/religion/domains.db
> 2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/religion/urls
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/religion/urls.db
> 2012-08-10 17:45:22 [28923] init domainlist 
> /var/lib/squidguard/spyware/domains
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/spyware/domains.db
> 2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/spyware/urls
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/spyware/urls.db
> 2012-08-10 17:45:22 [28923] init domainlist 
> /var/lib/squidguard/violence/domains
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/violence/domains.db
> 2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/violence/urls
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/violence/urls.db
> 2012-08-10 17:45:22 [28923] init domainlist 
> /var/lib/squidguard/weapons/domains
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/weapons/domains.db
> 2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/weapons/urls
> 2012-08-10 17:45:22 [28923] loading dbfile 
> /var/lib/squidguard/weapons/urls.db
> 2012-08-10 17:45:22 [28923] squidGuard 1.3 started (1344617122.190)
> 2012-08-10 17:45:22 [28923] squidGuard ready for requests (1344617122.193)
> 2012-08-10 17:45:22 [28923] source not found
> 2012-08-10 17:45:22 [28923] no ACL matching source, using default
> http://localhost/block.html -/- - GET
> 2012-08-10 17:45:22 [28923] squidGuard stopped (1344617122.193)
> 
> Does the url rewriter need to be further up the squid.conf?
> It is right at the end of the conf file at the moment:
> url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf

[squid-users] squidguard not blocking

[J Webster]

squidguard correctly blocks when I run from the command line:
[root squidguard]# echo "http://www.porn.com/ - - GET" | squidGuard -c 
/etc/squid/squidguard.conf -d

2012-08-10 17:45:22 [28923] New setting: dbhome: /var/lib/squidguard
2012-08-10 17:45:22 [28923] New setting: logdir: /var/log/squidguard
2012-08-10 17:45:22 [28923] init domainlist /var/lib/squidguard/porn/domains
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/porn/domains.db

2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/porn/urls
2012-08-10 17:45:22 [28923] loading dbfile /var/lib/squidguard/porn/urls.db
2012-08-10 17:45:22 [28923] init domainlist 
/var/lib/squidguard/aggressive/domains
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/aggressive/domains.db

2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/aggressive/urls
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/aggressive/urls.db
2012-08-10 17:45:22 [28923] init domainlist 
/var/lib/squidguard/hacking/domains
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/hacking/domains.db

2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/hacking/urls
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/hacking/urls.db
2012-08-10 17:45:22 [28923] init domainlist 
/var/lib/squidguard/religion/domains
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/religion/domains.db

2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/religion/urls
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/religion/urls.db
2012-08-10 17:45:22 [28923] init domainlist 
/var/lib/squidguard/spyware/domains
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/spyware/domains.db

2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/spyware/urls
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/spyware/urls.db
2012-08-10 17:45:22 [28923] init domainlist 
/var/lib/squidguard/violence/domains
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/violence/domains.db

2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/violence/urls
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/violence/urls.db
2012-08-10 17:45:22 [28923] init domainlist 
/var/lib/squidguard/weapons/domains
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/weapons/domains.db

2012-08-10 17:45:22 [28923] init urllist /var/lib/squidguard/weapons/urls
2012-08-10 17:45:22 [28923] loading dbfile 
/var/lib/squidguard/weapons/urls.db

2012-08-10 17:45:22 [28923] squidGuard 1.3 started (1344617122.190)
2012-08-10 17:45:22 [28923] squidGuard ready for requests (1344617122.193)
2012-08-10 17:45:22 [28923] source not found
2012-08-10 17:45:22 [28923] no ACL matching source, using default
http://localhost/block.html -/- - GET
2012-08-10 17:45:22 [28923] squidGuard stopped (1344617122.193)

Does the url rewriter need to be further up the squid.conf?
It is right at the end of the conf file at the moment:
url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf

[squid-users] deafult configure options?

[J. Webster]

I just updated to 3.1.18 but am not sure I have all the correct configure 
options.
The version that comes with my distro is 3.1.4 and has:
[root@264219 squid-3.1.18]# squid -v
Squid Cache: Version 3.1.4
configure options:  '--build=i686-pc-linux-gnu' '--host=i686-pc-linux-gnu' 
'--target=i686-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' 
'--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' 
'--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' 
'--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' 
'--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr' 
'--libexecdir=/usr/lib/squid' '--localstatedir=/var' 
'--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' 
'--with-logdir=$(localstatedir)/log/squid' 
'--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' 
'--enable-arp-acl' '--enable-follow-x-forwarded-for' 
'--enable-auth=basic,digest,ntlm,negotiate' 
'--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL,DB,POP3,squid_radius_auth'
 '--enable-ntlm-auth-helpers=smb_lm,no_check,fakeauth' 
'--enable-digest-auth-helpers=password,ldap,eDirectory' 
'--enable-negotiate-auth-helpers=squid_kerb_auth' 
'--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group'
 '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' 
'--enable-delay-pools' '--enable-epoll' '--enable-icap-client' 
'--enable-ident-lookups' '--with-large-files' '--enable-linux-netfilter' 
'--enable-referer-log' '--enable-removal-policies=heap,lru' '--enable-snmp' 
'--enable-ssl' '--enable-storeio=aufs,diskd,ufs' '--enable-useragent-log' 
'--enable-wccpv2' '--enable-esi' '--with-aio' '--with-default-user=squid' 
'--with-filedescriptors=16384' '--with-dl' '--with-openssl' '--with-pthreads' 
'build_alias=i686-pc-linux-gnu' 'host_alias=i686-pc-linux-gnu' 
'target_alias=i686-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall 
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
--param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom 
-fasynchronous-unwind-tables -fpie' 'LDFLAGS=-pie' 'CXXFLAGS=-O2 -g -pipe -Wall 
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
--param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom 
-fasynchronous-unwind-tables -fpie' 'FFLAGS=-O2 -g -pipe -Wall 
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
--param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom 
-fasynchronous-unwind-tables -I/usr/lib/gfortran/modules' 
--with-squid=/builddir/build/BUILD/squid-3.1.4 --enable-ltdl-convenience

However, now that I have upgraded to 3.1.18, I only have:
[root@264219 squid-3.1.18]# squid -v
Squid Cache: Version 3.1.18
configure options:  '--enable-delay-pools' '--enable-useragent-log' 
'--prefix=/usr' '--includedir=/usr/include' '--datadir=/usr/share' 
'--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--localstatedir=/var' 
'--sysconfdir=/etc/squid' --with-squid=/usr/local/sbin/myscripts/squid-3.1.18 
--enable-ltdl-convenience

Is that enough? Is anything defaulted between the versions?
  

RE: [squid-users] limiting connection not working 3.1.4

[J. Webster]

delay_access allow_all still seems to throw an error:
[root@264219 myscripts]# service squid restart
Stopping squid:    [FAILED]
Starting squid:    [FAILED]
2011/12/06 05:48:07| Processing Configuration File: /etc/squid/squid.conf 
(depth 0)
2011/12/06 05:48:07| WARNING: Netmasks are deprecated. Please use CIDR masks 
instead.
2011/12/06 05:48:07| WARNING: IPv4 netmasks are particularly nasty when used to 
compare IPv6 to IPv4 ranges.
2011/12/06 05:48:07| WARNING: For now we will assume you meant to write /32
FATAL: Bungled squid.conf line 65: delay_access allow all
Squid Cache (Version 3.1.4): Terminated abnormally.
CPU Usage: 0.013 seconds = 0.005 user + 0.008 sys
Maximum Resident Size: 19008 KB
Page faults with physical i/o: 0

I have attached the full conf here:

auth_param basic realm MyName proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access deny manager
http_access allow ncsa_users
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xx.xxx.xxx.xxx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
#cache_dir aufs /var/spool/squid 4 16 256
#cache_dir null /null
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none
buffered_logs on
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
half_closed_clients off
visible_hostname http://www.myserver.net
#the DNS is not registered as the server only has an IP address not linked to a 
website
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
client_db off
#coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000
delay_access allow all
forwarded_for off
via off


  

RE: [squid-users] limiting connection not working 3.1.4

[J. Webster]

Also, I now get a:
2011/12/06 05:22:49| squid.conf line 50: refresh_pattern -i (/cgi-bin/|?) 0 0% 0
2011/12/06 05:22:49| refreshAddToList: Invalid regular expression 
'(/cgi-bin/|?)  ': Invalid preceding regular expression
FATAL: Bungled squid.conf line 65: delay_access allow all
Squid Cache (Version 3.1.4): Terminated abnormally.
CPU Usage: 0.013 seconds = 0.007 user + 0.006 sys
Maximum Resident Size: 19008 KB
Page faults with physical i/o: 0
[root@264219 squid-3.1.18]#

  

RE: [squid-users] limiting connection not working 3.1.4

[J. Webster]

> > http_access deny manager
> > http_access allow ncsa_users
>
> So all logged in users have unlimited access?
>
>
> > http_access deny !Safe_ports
> > http_access deny CONNECT !SSL_ports
> > http_access deny to_localhost
> > http_access deny maxuser
>
> These deny rules are placed below the allow rule letting ALL logged in
> users through.
> This means that for all machines on the Internet which can supply one
> of your users insecure plain-text logins:
> * the safe_ports rule preventing viral and P2P abuse relaying through
> Squid has no effect
> * the CONNECT rule preventing blind binary tunneling of data to any
> protocol port through Squid has no effect.
> * you maxuser policy has no effect.

So, I should apply the deny rules above the allow ncsa_users line?
eg
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access deny manager
http_access allow ncsa_users



>
> > http_access allow localhost
> > http_access deny all
> > icp_access allow all
> > http_port 8080
> > http_port xx.xx.xx.xx:80
>
> And what are you expecting to arrive over port 80?
> That port is reserved for reverse-proxy and origin server traffic.
>
I have squid listening on port 80 and 8080 because some clients cannot connect 
on port 8080


> > visible_hostname MyNameProxyServer
>
> Funny domain name. I hope that is obfuscated for the post not in the
> config.
> This is the domain name used in URLs your clients get told to use for
> Squid error and FTP page icons. If it does not resolve back to this or
> another Squid your clients will be facing page load problems on those
> generated responses.

I thought this was just the name presented to the users when they logged on.
If it is meant to be a domain name should it be:
visible_hostname www.mynameproxyserver.com
?

Thanks

  

[squid-users] limiting connection not working 3.1.4

[J. Webster]

I have squid 3.1.4 but using this conf, the rate limiting to 1Mbps does not 
seem to work.
What can I change in the conf / delay parameters?

auth_param basic realm Myname proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xx.xx.xx.xx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
#cache_dir aufs /var/spool/squid 4 16 256
#cache_dir null /null
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern .   0    20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
#acl apache rep_header Server ^Apache
#broken_vary_encoding allow apache
half_closed_clients off
visible_hostname MyNameProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
client_db off
#coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000
forwarded_for off
via off   

[squid-users] limiting connection not working 3.1.4

[J. Webster]

I have squid 3.1.4 but using this conf, the rate limiting to 1Mbps does not 
seem to work.
What can I change in the conf / delay parameters?

auth_param basic realm Myname proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xx.xx.xx.xx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
#cache_dir aufs /var/spool/squid 4 16 256
#cache_dir null /null
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern .   0    20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
#acl apache rep_header Server ^Apache
#broken_vary_encoding allow apache
half_closed_clients off
visible_hostname MyNameProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
client_db off
#coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000
forwarded_for off
via off


  

RE: [squid-users] won't accept port 8080, 80 works

[J. Webster]

any ideas on this?
Thanks


> From: webster_j...@hotmail.com
> To: squid-users@squid-cache.org
> Date: Tue, 27 Sep 2011 08:07:12 +0100
> Subject: [squid-users] won't accept port 8080, 80 works
>
>
> I cannot get squid to connect on port 8080 even though it works on 80.
> Firstly, should this iptables script have a DROP/REJECT command somewhere?
> Port 8080 is open. squid conf is below:
>
> # Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
> *mangle
> :PREROUTING ACCEPT [19588:10233482]
> :INPUT ACCEPT [19588:10233482]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [18858:10334564]
> :POSTROUTING ACCEPT [18858:10334564]
> COMMIT
> # Completed on Fri Sep 16 04:59:49 2011
> # Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
> *filter
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [18851:1052]
> :RH-Firewall-1-INPUT - [0:0]
> -A INPUT -j RH-Firewall-1-INPUT
> -A FORWARD -j RH-Firewall-1-INPUT
> -A RH-Firewall-1-INPUT -i lo -j ACCEPT
> -A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
> -A RH-Firewall-1-INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
> -A RH-Firewall-1-INPUT -p udp -m udp --dport 123 -m state --state NEW -j 
> ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8002 -m state --state NEW -j 
> ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 9001 -m state --state NEW -j 
> ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 1935 -j ACCEPT
> -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 443 -j ACCEPT
> -A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
> COMMIT
> # Completed on Fri Sep 16 04:59:49 2011
> # Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
> *nat
> :PREROUTING ACCEPT [1234:59200]
> :POSTROUTING ACCEPT [338:21268]
> :OUTPUT ACCEPT [338:21268]
> COMMIT
> # Completed on Fri Sep 16 04:59:49 2011
>
>
> http_access deny manager
> http_access allow ncsa_users
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access deny to_localhost
> http_access deny maxuser
> http_access allow localhost
> http_access deny all
> icp_access allow all
> http_port 8080
> http_port xxx.xxx.xx.xx:80
> hierarchy_stoplist cgi-bin ?
> cache_mem 100 MB
> maximum_object_size_in_memory 50 KB
> cache_replacement_policy heap LFUDA
> cache_dir aufs /var/spool/squid 4 16 256
> maximum_object_size 50 MB
>
  

[squid-users] Squidalyser: nothing entered into database

[J. Webster]

Right, I installed everything and ran the mysql script so that it put in x 
thousand rows into the database.
However, when I access: http://mysite.org/cgi-bin/squidalyser.pl I just get a 
blank page. No errors, it doesn't show anything at all.
The webserver is working because if I access 
http://mysite.org/cgi-bin/wordlist.pl then it brings up a webpage.
I just checked the database, there is nothing in it?
Strange, squidparse.pl reported a success:

[root squidparse]# ./squidparse.pl

Running ./squidparse.pl at Wed Sep 28 21:06:42 2011

DB Name: squid
DB Host: localhost
DB User: squidalyser
Squidlog: /var/log/squid/access.log

Expired 1976284 records from the database.
Took 796 seconds to process 1976284 records.


  

[squid-users] lost connection - reconnect automatically

[J. Webster]

I have a squid service with ncsa user auth (login/password).
We have one user who loses their internet connection intermittently and is 
continually being re-presented with the login prompt.
Presumably, the server / browser thinks they have disconnected from the server 
and asks them to re-authenticate.
Is there a way round this?
  

[squid-users] won't accept port 8080, 80 works

[J. Webster]

I cannot get squid to connect on port 8080 even though it works on 80.
Firstly, should this iptables script have a DROP/REJECT command somewhere?
Port 8080 is open. squid conf is below:

# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*mangle
:PREROUTING ACCEPT [19588:10233482]
:INPUT ACCEPT [19588:10233482]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18858:10334564]
:POSTROUTING ACCEPT [18858:10334564]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18851:1052]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8002 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 9001 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 1935 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*nat
:PREROUTING ACCEPT [1234:59200]
:POSTROUTING ACCEPT [338:21268]
:OUTPUT ACCEPT [338:21268]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011


http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xxx.xxx.xx.xx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB
  

[squid-users] squid with Ipad and iphone and ipods

[J. Webster]

Is there anyway to get squid to work with ncsa auth on mobile devices such as 
Ipad and iphone and ipods?
These devices have the ability to enter proxy settings but they do not accept 
username / password prompts and therefore fail squid authentication.
Can I somehow detect a mobile device and perhaps direct access to webpages to a 
bespoke login form?
  

[squid-users] won't accept port 8080, 80 works

[J. Webster]

I cannot get squid to connect on port 8080 even though it works on 80.
Firstly, should this iptables script have a DROP/REJECT command somewhere?
Port 8080 is open. squid conf is below:
[code]
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*mangle
:PREROUTING ACCEPT [19588:10233482]
:INPUT ACCEPT [19588:10233482]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18858:10334564]
:POSTROUTING ACCEPT [18858:10334564]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18851:1052]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8002 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 9001 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 1935 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*nat
:PREROUTING ACCEPT [1234:59200]
:POSTROUTING ACCEPT [338:21268]
:OUTPUT ACCEPT [338:21268]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
[/code]
[code]
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xxx.xxx.xx.xx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB
[/code]
  

[squid-users] deleting headers

[J. Webster]

Is it a bad idea to put this in the conf?

forwarded_for delete
header_access From deny all
header_access Referer deny all
header_access Server deny all
header_access User-Agent deny all
header_access WWW-Authenticate deny all
header_access Link deny all
I accessed a "What's my IP" site and it knew that I was using a proxy, it even 
said squid 2.6.
I believe some sites will block me base don the headers but won;t some sites 
block if headers do not exist?


  

RE: [squid-users] deleting headers

[J. Webster]

I did not unsubscribe, that was someone else.
How can I remove some headers for privacy but have squid work properly on most 
webpages?
confs below


> From: webster_j...@hotmail.com
> To: squid-users@squid-cache.org
> Date: Thu, 22 Sep 2011 16:49:43 +0100
> Subject: [squid-users] deleting headers
>
>
> Is it a bad idea to put this in the conf?
>
> forwarded_for delete
> header_access From deny all
> header_access Referer deny all
> header_access Server deny all
> header_access User-Agent deny all
> header_access WWW-Authenticate deny all
> header_access Link deny all
>
> I accessed a "What's my IP" site and it knew that I was using a proxy, it 
> even said squid 2.6.
> I beleive some sites will block me base don the headers but won;t some sites 
> block if headers do not exist?
>
  

[squid-users] deleting headers

[J. Webster]

Is it a bad idea to put this in the conf?

forwarded_for delete
header_access From deny all
header_access Referer deny all
header_access Server deny all
header_access User-Agent deny all
header_access WWW-Authenticate deny all
header_access Link deny all

I accessed a "What's my IP" site and it knew that I was using a proxy, it even 
said squid 2.6.
I beleive some sites will block me base don the headers but won;t some sites 
block if headers do not exist?
  

[squid-users] won't accept port 8080, 80 works

[J. Webster]

I cannot get squid to connect on port 8080 even though it works on 80.
Firstly, should this iptables script have a DROP/REJECT command somewhere?
Port 8080 is open. squid conf is below:

[code]
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*mangle
:PREROUTING ACCEPT [19588:10233482]
:INPUT ACCEPT [19588:10233482]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18858:10334564]
:POSTROUTING ACCEPT [18858:10334564]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18851:1052]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8002 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 9001 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 1935 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*nat
:PREROUTING ACCEPT [1234:59200]
:POSTROUTING ACCEPT [338:21268]
:OUTPUT ACCEPT [338:21268]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011

[/code]

[code]
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xxx.xxx.xx.xx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB

[/code]   

[squid-users] RE: won't accept port 8080, 80 works

[J. Webster]

From: webster_j...@hotmail.com
To: squid-users@squid-cache.org
Subject: won't accept port 8080, 80 works
Date: Thu, 22 Sep 2011 10:39:53 +0100






.ExternalClass .ecxhmmessage P
{padding:0px;}
.ExternalClass body.ecxhmmessage
{font-size:10pt;font-family:Tahoma;}




I cannot get squid to connect on port 8080 even though it works on 80.
Firstly, should this iptables script have a DROP/REJECT command somewhere?
Port 8080 is open. squid conf is below:

[code]
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*mangle
:PREROUTING ACCEPT [19588:10233482]
:INPUT ACCEPT [19588:10233482]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18858:10334564]
:POSTROUTING ACCEPT [18858:10334564]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [18851:1052]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8002 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 9001 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 1935 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Fri Sep 16 04:59:49 2011
# Generated by iptables-save v1.3.5 on Fri Sep 16 04:59:49 2011
*nat
:PREROUTING ACCEPT [1234:59200]
:POSTROUTING ACCEPT [338:21268]
:OUTPUT ACCEPT [338:21268]
COMMIT
# Completed on Fri Sep 16 04:59:49 2011

[/code]

[code]
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xxx.xxx.xx.xx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB

[/code]

> Date: Thu, 22 Sep 2011 12:55:00 +0530
> From: benjo11...@gmail.com
> To: squid-users@squid-cache.org
> Subject: [squid-users] minus value in output
> 
>   Hi,
> 
> Sometimes i am getting minus value in squidclient mgr:info.
> 
> Cache information for squid:
>  Hits as % of all requests:5min: 0.0%, 60min: 0.0%
>  Hits as % of bytes sent:5min: -0.0%, 60min: -0.0%
>  Memory hits as % of hit requests:5min: 0.0%, 60min: 0.0%
>  Disk hits as % of hit requests:5min: 0.0%, 60min: 0.0%
>  Storage Swap size:1000804 KB
>  Storage Swap capacity: 0.3% used, 99.7% free
>  Storage Mem size:236612 KB
>  Storage Mem capacity: 9.3% used, 90.7% free
>  Mean Object Size:15.83 KB
>  Requests given to unlinkd:0
> 
> 
> Wht does it means?
> 
> 
> Thanks,
> Benjamin

  

RE: [squid-users] 2 squid on the same server

[J. Webster]

>> (logIn only challenges and fetches auth if it is tested, it is only 
>> tested when the IP is in 'other_subnet').

But I need a username/password box to be sent to the user by default unless the 
IP address = 212.234.34.43
The above comment sounds like it will only send a username/password request 
when the IP address = 212.234.34.43
  

RE: [squid-users] 2 squid on the same server

[J. Webster]

> Ah, that tutorial is about writing an authentication helper (ie
> ncsa_auth). Not an ACL helper.
>
> The difference being that external_acl_type ACL helpers auth*orize* the
> request permission to do something in Squid because it matches an IP
> used by some username.
>
> auth_param helpers auth*enticate* some security username:passtoken
> credentials. They do not assign any permissions, just state whether the
> credentials are valid/invalid.
>
>
> The script I was suggesting takes only the IP and produces the username
> for logging. You need some database, or AD login etc mapping which users
> have been assigned which IP. The script uses that source to find the
> username in the background and present it to Squid via "OK
> user=$username" or "ERR" results.
>
>
> The squid.conf looks something like:
>
> external_acl_type IPUser %SRC /path/to/script
>
> auth_param basic program /path/to/ncsa_auth
>
> # VPN subnet intercepted with NAT
> acl ipuser external IPUser
> acl vpn_subnet src 192.168.1.0/24
> http_access allow vpn_subnet ipuser
>
> # regular subnet who can login
> acl logIn proxy_auth REQUIRED
> acl other_subnet src 192.168.2.0/24
> http_access allow other_subnet logIn
>
> # strange machines we don't know.
> http_access deny all
>
Right...sorry, can I leave the VPN out for the moment because I'm confusing 
myself with the setup.
So, the current setup uses ncsa_auth. I need to add a secondary authentication 
mechanism, which checks the external IP address but does not require a username 
or password.
>From what we've said I cannot add 2 mechanisms so I need to pass the auth to a 
>script that can check the IP address. If the IP address does not equal 
>200.212.34.45 then I need to pass the script a username and password pair, 
>which it can check against the existing ncsa_auth squid_passwd file.
Users accesses proxy, if IP=200.212.34.45 OK, else if 
username:password=squid_passwd file OK, else ERR.
Do I even need a script for that or can I simply add acl other_subnet src 
200.212.34.45 to the existing conf?

Current conf:
auth_param basic realm MySquid proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xx.xxx.xxx.198:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
#cache_dir null /null
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
visible_hostname MySquidProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000
  

RE: [squid-users] 2 squid on the same server

[J. Webster]

>
> Something has gone wrong external ACL should not be getting the username
> and password. It should be getting the IP address on STDIN.
>
Of the examples, that I can find they all talk about the username and password 
bewing sent from squid:
http://www.freesoftwaremagazine.com/articles/authentication_with_squid?page=0%2C0
 
How else should the script get the username and password?
I need username, apssword, and IP address and then the script will check if the 
IP matches, if not, it will check the username, and password, otherwise it will 
reject the connection.
  

RE: [squid-users] 2 squid on the same server

[J. Webster]

> Both yes and no and no.
> Yes to "something", any scripting or executable language can be used.
> Via *external_acl_type* (NOT auth_param).
>
STDIN passes the username and password but how does squid pass the IP address 
to the squid helper?
  

RE: [squid-users] 2 squid on the same server

[J. Webster]

> How they interact is entirely up to you and your configuration.
> The http_access lists are a full-blown boolean programming language with
> hundreds of ACL permutations and paths you can configure.
>
> It is perfectly possible to configure in a way where they don't
> interact, BUT you need to configure that to happen.
> Simply listing a check for NCSA auth then an external ACL check for
> IP auth one after the other will case problems. Checking the client
> subnet earlier on the access line can skip one or other auth test and
> avoid a clash.
> This config separation is possible for the external ACL vs auth_param
> checks. Two auth_param types must combine and do the advert thing.
>
Do I need to do something in PHP/perl to make this work?
eg
auth_param basic program /bin/php your_script_location
How can the script check the ncsa password file? I think cheking the IP address 
is probably the easier part


The current conf looks like this, so do I have to replace the ncsa auth with a 
script above that checks the ncsa password file and the IP address?

auth_param basic realm MySquid proxy server
auth_param basic credentialsttl 2 hours
#auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
#replace with
auth_param basic program /bin/php your_script_location

authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
#acl ncsa_users proxy_auth REQUIRED
#replace with
acl AuthenticatedUsers proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
#http_access allow ncsa_users
#replace with
http_access allow AuthenticatedUsers
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port xx.xxx.xxx.198:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
#cache_dir null /null
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
visible_hostname MySquidProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000
cachemgr_passwd mypswd all


if (typeof(lpcurruser) == 'undefined') lpcurruser = ''; if 
(document.getElementById('lpcurruserelt') && 
document.getElementById('lpcurruserelt').value != '') { lpcurruser = 
document.getElementById('lpcurruserelt').value; 
document.getElementById('lpcurruserelt').value = ''; } if (typeof(lpcurrpass) 
== 'undefined') lpcurrpass=''; if (document.getElementById('lpcurrpasselt') && 
document.getElementById('lpcurrpasselt').value != '') { lpcurrpass = 
document.getElementById('lpcurrpasselt').value; 
document.getElementById('lpcurrpasselt').value = ''; } var lploc=1;var 
lponlyfill=null;var link=document.getElementById("i1668"); 
if(link&&typeof(g_lpclicked)=="undefined"){if(document.createEventObject){var 
evt = document.createEventObject();link.fireEvent("onclick",evt);}else{var evt2 
= document.createEvent("MouseEvents");evt2.initMouseEvent("click", true, true, 
document.defaultView, 1, 0, 0, 0, 0, false, false, false, false, 0, 
null);link.dispatchEvent(evt2);}g_lpclicked=1;}lpcurruser = ''; lpcurrpass = 
'';if (typeof(lpcurruser) == 'undefined') lpcurruser = ''; if 
(document.getElementById('lpcurruserelt') && 
document.getElementById('lpcurruserelt').value != '') { lpcurruser = 
document.getElementById('lpcurruserelt').value; 
document.getElementById('lpcurruserelt').value = ''; } if (typeof(lpcurrpass) 
== 'undefined') lpcurrpass=''; if (document.getElementById('lpcurrpasselt') && 
document.getElementById('lpcurrpasselt').value != '') { lpcurrpass = 
document.

RE: [squid-users] 2 squid on the same server

[J. Webster]

 If by "forwarded" you mean NAT. Authentication is not possible. See the
> FAQ about why.
> http://wiki.squid-cache.org/SquidFaq/InterceptionProxy#Why_can.27t_I_use_authentication_together_with_interception_proxying.3F
>
> >
> > Are there any examples for having 2 authentication methods run at
> > once? Does this mean the user would have to pick an option when
>
> The *user* does not know anything or need to. Their browser does it.
>
> > connecting to the server? I don;t think that will work for iPads,
> > xboxes, DVD players, etc accessing a proxy server as they connect
> > automatically without interaction. My current version is 2.6 - will
> > this work with that?

So, 
Connection route A: Direct to proxy listening on port 80 and port 8080 with 
ncsa auth.
Ports 80, 8080, 443 will continue to be accessed with ncsa auth.

Connection route B: VPN with squid logging the websites.
Squid listening on port xxx1.
The logs will only contain an IP address from connections form port xxx1?
I need to make a change in iptables to block outside connections to port xxx1 
and only allow port xxx1 to be accessed form the VPN network.
What do I do with port 443 in this instance? Do I need to make a new https port 
on squid and forward VPN:443 to squid:xxx?

Connection route C: Direct to proxy listening on port xxx2 with IP address auth.
You mentioned in the earlier email chain that if I setup IP auth as well as 
ncsa auth then this will mess up the authentication mechanism.
Is there no other way to have 2 authentication methods running at the same time?
  

RE: [squid-users] 2 squid on the same server

[J. Webster]

> > So, I can do this all with one squid service listening on different ports?
>
> Yes.
>
> > Out of interest, I had a post the other day: "proxy external ip address 
> > acl‏"
> > You mentioned that adding an extra authentication would mess with the 
> > existing ncsa auth.
> > Can I therefore add a 2nd authentication method on a different port and 
> > have 2 authentication methods running at the same time?
>
> Not like that. You can add support for two authentication methods and
> advertise them when challenging for the browser to choose which
> credentials it sends you.
>

If VPN port 80 traffic is forwarded to port 8181 of squid, how can I apply a 
username to the squid logs so I know which user has accessed which oages or 
could I only do it by IP address in  a transparent proxy setup?

Are there any examples for having 2 authentication methods run at once?
Does this mean the user would have to pick an option when connecting to the 
server? I don;t think that will work for iPads, xboxes, DVD players, etc 
accessing a proxy server as they connect automatically without interaction. My 
current version is 2.6 - will this work with that?
  

RE: [squid-users] 2 squid on the same server

[J. Webster]

> yes. yes. wrong. no it does not.
>
> You seem very confused about Squid capabilities...
>
> - *add* a second http_port with "intercept" flag (or "transparent" if
> its an old Squid).
> - configure iptables to pass the VPN port 80 traffic *to* that new
> Squid port.
> - configure iptables to prevent direct client connections to that new
> port.
> - configure squid to not ask for auth from VPN clients.
>
> OR
> - configure the VPN clients to use Squid the same way you configure
> the non-VPN ones.
> - what you do with auth is now optional.

So, I can do this all with one squid service listening on different ports?
Out of interest, I had a post the other day: "proxy external ip address acl‏"
You mentioned that adding an extra authentication would mess with the existing 
ncsa auth.
Can I therefore add a 2nd authentication method on a different port and have 2 
authentication methods running at the same time?


  

[squid-users] 2 squid on the same server

[J. Webster]

Is it possible to run 2 squids on the same server?
I have an existing proxy that has ncsa auth via direct connections to the proxy.
I would also like to route port 80 traffic from a VPN through a transparent 
proxy but without any authentication. Is this possible? I'm guessing I would 
have to block off all connections to this squid through iptables and only allow 
traffic from the VPN. It would also have to be a transparent proxy?
  

RE: [squid-users] proxy external ip address acl

[J. Webster]

> > > Will this mess with the ncsa auth?
> >
> > It will.
> >
> > You have already said they "cannot enter usernames and passwords". So
> > the interference being in the form of not asking for username/password
> > seems to be what you are wanting.
> >
> > I would extend that a bit and maybe check for User-Agent ("browser" ACL)
> > as well as IP. Just in case they change IP.
> >
> > Amos
> > --

>
> Hi
> Sorry, should have been clearer, the current ncsa auth works for the
> majority of clients but we get the occasional client who cannot enter a
> username due to using an iPad or maybe an xbox that cannot enter
> passwords.
> I need to authenticate these by their fixed IP address as well as allow
> other users to authenticate through the normal ncsa auth.
  

[squid-users] proxy external ip address acl

[J Webster]

I currently have a squid proxy using ncsa auth.
I would also like to add an ip address auth for those users that cannot 
enter usernames and passwords (some iphones, DVD players etc.|)

Can I just add an acl like this:

acl external_IP 200.123.45.23
http_access allow external_IP

Will this mess with the ncsa auth?

RE: [squid-users] certificate auth?

[J. Webster]

So, this is not really possible with squid settings but could be 
achieved through use of a webserver accepting certificates and then 
redirecting connections to the proxy?


There are a lot of objects out there that allow proxy settings but 
cannot cope with username and password settings so I am trying to figure 
out a way of authenticating them (eh some iPods with browsers, some 
Xboxes, some TV/satellite boxes with internet enabled connections).


> >
> > On 15/04/2011 11:06, Tom Tux wrote:
> >>
> >> A few weeks ago, there was a post concerning certificate authentication:
> >>
> >>
> >> http://squid-web-proxy-cache.1019090.n4.nabble.com/Client-Certificate-Authentication-td3353759.html
> >>
> >> Regards,
> >> Tom
> >>
> >> 2011/4/15 J. Webster:
> >>>
> >>> Is there a way to generate authentication certificates for access to a
> >>> proxy server instead of or in addition to ncsa auth?
> >>>
> >>> I understand there is an IP address auth check in squid but is this for
> >>> local network (192.168.x.x) only?
> >>>
> >>> My current setup is that users login through ncsa auth and all their IPs
> >>> are external IPs rather than on an internal network, this is for geo IP
> >>> location.
> >>>
> >>
> >
> >
  

[squid-users] certificate auth?

[J. Webster]

Is there a way to generate authentication certificates for access to a proxy 
server instead of or in addition to ncsa auth?

I understand there is an IP address auth check in squid but is this for local 
network (192.168.x.x) only?

My current setup is that users login through ncsa auth and all their IPs are 
external IPs rather than on an internal network, this is for geo IP location.
  

[squid-users] connection not being limited

[J Webster]

I have this at the bottom of my squid conf:
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000

On connection, it seems each individual client is being limited.
However, iftop shows a connection to one site of over 6Mb?!
Any ideas on why this is escaping through the throttle?

server88-xxx-xxx-198.live-servers.net=> iplaydl0.thdo.bbc.co.uk 
169Kb   146Kb   124Kb
<= 
7.11Mb  6.44Mb  5.56Mb

Re: [squid-users] prevent squid being used as spam passthrough

[J Webster]

That's pretty much what I have but is it not possible to use one of these 
ports as a pass through for spam or would the receiving email servers block 
it?


acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports


--
From: "Amos Jeffries" 
Sent: Monday, December 27, 2010 9:36 PM
To: 
Subject: Re: [squid-users] prevent squid being used as spam passthrough


On 27/12/10 09:23, J Webster wrote:

Is it possible for a proxy running on port 80 or 8080 to be used as a
pass through or zone origination for spam email?


Maybe. If it has been configured as an open proxy.
http://wiki.squid-cache.org/SquidFaq/SecurityPitfalls


We have had some users sign up with email addresses such as spambot and
other stuff recently. I suspect these are just bots signing up around
the web but got me thinking whether a proxy could be used in a chain or
tunneled somehow and whether that could be blocked?


The default squid.conf http_access controls are designed to prevent this 
type of thing.


It requires Safe_ports to list only the ports <1024 which are nown to be 
safe for proxy connections-to. As well as SSL_ports for CONNECT tunnels to 
only connect to known HTTPS ports.


You can see the quid default settings at
http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#Squid_configuration

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.10
  Beta testers wanted for 3.2.0.4

Re: [squid-users] refusing connections

[J Webster]

Is it possible that something in this squid.conf might cause a memory block 
or excessive CPU usage that could lead to this:

It seems a coincidence that a server reboot seemed to fix the issue.

auth_param basic realm NameHere proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port XX.XXX.XXX.198:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
#cache_dir null /null
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
visible_hostname NameHereProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000

Re: [squid-users] refusing connections

[J Webster]

Now it is
2799 -rw-r- 1 squid squid 2863056 Dec 27 08:50 
/var/spool/squid/swap.state


but it was root:squid
I changed something in webmin - could that have caused the issue? Why would 
webmin change the file permissions?



--
From: "Travel Factory S.r.l." 
Sent: Sunday, December 26, 2010 11:44 PM
To: "J Webster" ; 
Subject: Re: [squid-users] refusing connections



what are the permissions and who is the owner of
/var/spool/squid/swap.state ?

please do a
ls -lsa /var/spool/squid/swap.state
and report

[squid-users] prevent squid being used as spam passthrough

[J Webster]

Is it possible for a proxy running on port 80 or 8080 to be used as a pass 
through or zone origination for spam email?
We have had some users sign up with email addresses such as spambot and 
other stuff recently. I suspect these are just bots signing up around the 
web but got me thinking whether a proxy could be used in a chain or tunneled 
somehow and whether that could be blocked? 

Re: [squid-users] refusing connections

[J Webster]

Hi
No, that was just me omitting the name but forgot to in the second email.
This morning I allowed, some timeouts to occur in the NCSA auth in case it 
was an IP issue.
I connected, loaded the page and successfully navigated 2 sites, before 
being refused a connection on the 3rd website.
Seems very strange - I turned all local firewalls off and it is the same 
thing.



--
From: "Amos Jeffries" 
Sent: Sunday, December 26, 2010 2:01 AM
To: 
Subject: Re: [squid-users] refusing connections


On 26/12/10 11:46, J Webster wrote:

Hmm. I turned the cache off and restarted and still the same issue so
that means it can;t be the cache?


How did you turn the cache off?  "cache deny all"?

Having a look in to see what version your Squid is I find that the proxy 
whose startup sequence you displayed is not the same one whose 
configuration you displayed.
  The config you showed has a hostname "AAProxyServer". Contacting the 
public IP shown in that startup sequence reports hostname 
"ProxyPlayerProxyServer" and a 2.6 version.


Are you sure you are working from the right instances log?

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.10
  Beta testers wanted for 3.2.0.4

Re: [squid-users] refusing connections

[J Webster]

Hmm. I turned the cache off and restarted and still the same issue so that 
means it can;t be the cache?


--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 11:31 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections


There's nothing strange I can see here.
I guess the next step should really be cachemgr.

Re: [squid-users] refusing connections

[J Webster]

I just rebuilt the cache and no luck - would rebuilding the cache have 
deleted the swap?
Again on restart it works for 10seconds and then starts to refuse 
connections.


Could there be a file permissions error anywhere?

[root squid]# ls -l /var/log/squid
total 1367348
-rw-r- 1 squid squid 314577912 Dec 25 22:20 access.log
-rw-r- 1 squid squid  46312136 Dec 19 00:01 access.log.1.gz
-rw-r- 1 squid squid  51110440 Dec 12 00:01 access.log.2.gz
-rw-r- 1 squid squid  53290163 Dec  5 00:02 access.log.3.gz
-rw-r- 1 squid squid  53867649 Nov 28 00:02 access.log.4.gz
-rw-r- 1 squid squid  49959936 Nov 21 00:02 access.log.5.gz
-rw-r- 1 squid squid 20426 Dec 25 22:20 cache.log
-rw-r- 1 squid squid 17893 Dec 19 00:01 cache.log.1.gz
-rw-r- 1 squid squid 34684 Dec 12 00:01 cache.log.2.gz
-rw-r- 1 squid squid 35869 Dec  5 00:02 cache.log.3.gz
-rw-r- 1 squid squid 39648 Nov 28 00:02 cache.log.4.gz
-rw-r- 1 squid squid 45064 Nov 21 00:02 cache.log.5.gz
-rw-r- 1 root  root 474943 Dec 25 21:45 cache.log.copy
-rw-r--r-- 1 root  root   6023 Dec 25 21:35 squid.out
-rw-r- 1 squid squid 346063721 Dec 25 22:19 store.log
-rw-r- 1 squid squid  82754446 Dec 19 00:02 store.log.1.gz
-rw-r- 1 squid squid  96989188 Dec 12 00:02 store.log.2.gz
-rw-r- 1 squid squid 102610681 Dec  5 00:03 store.log.3.gz
-rw-r- 1 squid squid 104956585 Nov 28 00:03 store.log.4.gz
-rw-r- 1 squid squid  95507257 Nov 21 00:03 store.log.5.gz


--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 11:11 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections


store.log is usually not needed; I'd suggest to just disable it.
swap.state is the cache_dir index; if you delete it it will be rebuilt
at startup.



On Sat, Dec 25, 2010 at 11:09 PM, J Webster  
wrote:

Maybe I should revuild the cache.
Will that also delete swap.state and store.log automatically?

------
From: "J Webster" 
Sent: Saturday, December 25, 2010 10:58 PM
To: "Kinkie" 
Cc: 
Subject: Re: [squid-users] refusing connections


How can I view the cache manager?
I managed to load 2 pages, then tried a third and connection refused.
5mins later I loaded another page and then the next one connection
refused.
These are all different sites and all accessible when accessed directly
without the proxy.

Could be a file permissions somewhere or maybe some sort of blocking
attack on the proxy?

--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 10:55 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections


Anything in the cache manager? Filedescriptor allocation etc.. Can you
access the sites without using the proxy?

On Sat, Dec 25, 2010 at 10:49 PM, J Webster 
wrote:


This is the last bit from the cache log after restarting - seems ok 
but

still refusing connections.
On a restart it seems to connect and load a page but then stops 
halfway

just
like it times out.

2010/12/25 21:47:16| DNS Socket created at 0.0.0.0, port 56340, FD 6
2010/12/25 21:47:16| Adding nameserver 208.67.222.222 from squid.conf
2010/12/25 21:47:16| Adding nameserver 208.67.220.220 from squid.conf
2010/12/25 21:47:16| helperOpenServers: Starting 5 'ncsa_auth' 
processes

2010/12/25 21:47:16| User-Agent logging is disabled.
2010/12/25 21:47:16| Referer logging is disabled.
2010/12/25 21:47:16| Unlinkd pipe opened on FD 16
2010/12/25 21:47:16| Swap maxSize 4096 + 102400 KB, estimated 0
objects
2010/12/25 21:47:16| Target number of buckets: 157932
2010/12/25 21:47:16| Using 262144 Store buckets
2010/12/25 21:47:16| Max Mem  size: 102400 KB
2010/12/25 21:47:16| Max Swap size: 4096 KB
2010/12/25 21:47:16| Local cache digest enabled; rebuild/rewrite every
3600/3600 sec
2010/12/25 21:47:16| Rebuilding storage in /var/spool/squid (CLEAN)
2010/12/25 21:47:16| Using Least Load store dir selection
2010/12/25 21:47:16| Set Current Directory to /var/spool/squid
2010/12/25 21:47:16| Loaded Icons.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 0.0.0.0, port
8080,
FD 20.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 
88.208.237.198,

port 80, FD 21.
2010/12/25 21:47:17| Accepting ICP messages at 0.0.0.0, port 3130, FD
22.
2010/12/25 21:47:17| WCCP Disabled.
2010/12/25 21:47:17| Ready to serve requests.
2010/12/25 21:47:17| Store rebuilding is  0.8% complete
2010/12/25 21:47:19| Done reading /var/spool/squid swaplog (521968
entries)
2010/12/25 21:47:19| Finished rebuilding storage from disk.
2010/12/25 21:47:19|521968 Entries scanned
2010/12/25 21:47:19| 0 Invalid entries.
2010/12/25 21:47:19| 0 With invalid flags.
2010/12/25 21:47:19|521931 Objects loaded.
2010/12/25 21:47:19| 0 Objects expired.
2010/12/25 21:47:19| 0 Objects 

Re: [squid-users] refusing connections

[J Webster]

Maybe I should revuild the cache.
Will that also delete swap.state and store.log automatically?

--
From: "J Webster" 
Sent: Saturday, December 25, 2010 10:58 PM
To: "Kinkie" 
Cc: 
Subject: Re: [squid-users] refusing connections


How can I view the cache manager?
I managed to load 2 pages, then tried a third and connection refused.
5mins later I loaded another page and then the next one connection 
refused.
These are all different sites and all accessible when accessed directly 
without the proxy.


Could be a file permissions somewhere or maybe some sort of blocking 
attack on the proxy?


--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 10:55 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections


Anything in the cache manager? Filedescriptor allocation etc.. Can you
access the sites without using the proxy?

On Sat, Dec 25, 2010 at 10:49 PM, J Webster  
wrote:

This is the last bit from the cache log after restarting - seems ok but
still refusing connections.
On a restart it seems to connect and load a page but then stops halfway 
just

like it times out.

2010/12/25 21:47:16| DNS Socket created at 0.0.0.0, port 56340, FD 6
2010/12/25 21:47:16| Adding nameserver 208.67.222.222 from squid.conf
2010/12/25 21:47:16| Adding nameserver 208.67.220.220 from squid.conf
2010/12/25 21:47:16| helperOpenServers: Starting 5 'ncsa_auth' processes
2010/12/25 21:47:16| User-Agent logging is disabled.
2010/12/25 21:47:16| Referer logging is disabled.
2010/12/25 21:47:16| Unlinkd pipe opened on FD 16
2010/12/25 21:47:16| Swap maxSize 4096 + 102400 KB, estimated 0 
objects

2010/12/25 21:47:16| Target number of buckets: 157932
2010/12/25 21:47:16| Using 262144 Store buckets
2010/12/25 21:47:16| Max Mem  size: 102400 KB
2010/12/25 21:47:16| Max Swap size: 4096 KB
2010/12/25 21:47:16| Local cache digest enabled; rebuild/rewrite every
3600/3600 sec
2010/12/25 21:47:16| Rebuilding storage in /var/spool/squid (CLEAN)
2010/12/25 21:47:16| Using Least Load store dir selection
2010/12/25 21:47:16| Set Current Directory to /var/spool/squid
2010/12/25 21:47:16| Loaded Icons.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 0.0.0.0, port 
8080,

FD 20.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 88.208.237.198,
port 80, FD 21.
2010/12/25 21:47:17| Accepting ICP messages at 0.0.0.0, port 3130, FD 
22.

2010/12/25 21:47:17| WCCP Disabled.
2010/12/25 21:47:17| Ready to serve requests.
2010/12/25 21:47:17| Store rebuilding is  0.8% complete
2010/12/25 21:47:19| Done reading /var/spool/squid swaplog (521968 
entries)

2010/12/25 21:47:19| Finished rebuilding storage from disk.
2010/12/25 21:47:19|521968 Entries scanned
2010/12/25 21:47:19| 0 Invalid entries.
2010/12/25 21:47:19| 0 With invalid flags.
2010/12/25 21:47:19|521931 Objects loaded.
2010/12/25 21:47:19| 0 Objects expired.
2010/12/25 21:47:19| 0 Objects cancelled.
2010/12/25 21:47:19| 6 Duplicate URLs purged.
2010/12/25 21:47:19|31 Swapfile clashes avoided.
2010/12/25 21:47:19|   Took 2.9 seconds (178595.5 objects/sec).
2010/12/25 21:47:19| Beginning Validation Procedure
2010/12/25 21:47:19|262144 Entries Validated so far.
2010/12/25 21:47:19|   Completed Validation Procedure
2010/12/25 21:47:19|   Validated 521931 Entries
2010/12/25 21:47:19|   store_swap_size = 9112264k
2010/12/25 21:47:20| storeLateRelease: released 6 objects


--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 10:41 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections


On Sat, Dec 25, 2010 at 10:38 PM, J Webster 
wrote:


The problem appears to be this:
/var/spool/squid/swap.state: (13) Permission denied

Why would that happen overnight?


Looks like some filesystem corruption happened for some reason. Is
there any other messages like this?


--
   /kinkie







--
/kinkie

Re: [squid-users] refusing connections

[J Webster]

How can I view the cache manager?
I managed to load 2 pages, then tried a third and connection refused.
5mins later I loaded another page and then the next one connection refused.
These are all different sites and all accessible when accessed directly 
without the proxy.


Could be a file permissions somewhere or maybe some sort of blocking attack 
on the proxy?


--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 10:55 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections


Anything in the cache manager? Filedescriptor allocation etc.. Can you
access the sites without using the proxy?

On Sat, Dec 25, 2010 at 10:49 PM, J Webster  
wrote:

This is the last bit from the cache log after restarting - seems ok but
still refusing connections.
On a restart it seems to connect and load a page but then stops halfway 
just

like it times out.

2010/12/25 21:47:16| DNS Socket created at 0.0.0.0, port 56340, FD 6
2010/12/25 21:47:16| Adding nameserver 208.67.222.222 from squid.conf
2010/12/25 21:47:16| Adding nameserver 208.67.220.220 from squid.conf
2010/12/25 21:47:16| helperOpenServers: Starting 5 'ncsa_auth' processes
2010/12/25 21:47:16| User-Agent logging is disabled.
2010/12/25 21:47:16| Referer logging is disabled.
2010/12/25 21:47:16| Unlinkd pipe opened on FD 16
2010/12/25 21:47:16| Swap maxSize 4096 + 102400 KB, estimated 0 
objects

2010/12/25 21:47:16| Target number of buckets: 157932
2010/12/25 21:47:16| Using 262144 Store buckets
2010/12/25 21:47:16| Max Mem  size: 102400 KB
2010/12/25 21:47:16| Max Swap size: 4096 KB
2010/12/25 21:47:16| Local cache digest enabled; rebuild/rewrite every
3600/3600 sec
2010/12/25 21:47:16| Rebuilding storage in /var/spool/squid (CLEAN)
2010/12/25 21:47:16| Using Least Load store dir selection
2010/12/25 21:47:16| Set Current Directory to /var/spool/squid
2010/12/25 21:47:16| Loaded Icons.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 0.0.0.0, port 
8080,

FD 20.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 88.208.237.198,
port 80, FD 21.
2010/12/25 21:47:17| Accepting ICP messages at 0.0.0.0, port 3130, FD 22.
2010/12/25 21:47:17| WCCP Disabled.
2010/12/25 21:47:17| Ready to serve requests.
2010/12/25 21:47:17| Store rebuilding is  0.8% complete
2010/12/25 21:47:19| Done reading /var/spool/squid swaplog (521968 
entries)

2010/12/25 21:47:19| Finished rebuilding storage from disk.
2010/12/25 21:47:19|521968 Entries scanned
2010/12/25 21:47:19| 0 Invalid entries.
2010/12/25 21:47:19| 0 With invalid flags.
2010/12/25 21:47:19|521931 Objects loaded.
2010/12/25 21:47:19| 0 Objects expired.
2010/12/25 21:47:19| 0 Objects cancelled.
2010/12/25 21:47:19| 6 Duplicate URLs purged.
2010/12/25 21:47:19|31 Swapfile clashes avoided.
2010/12/25 21:47:19|   Took 2.9 seconds (178595.5 objects/sec).
2010/12/25 21:47:19| Beginning Validation Procedure
2010/12/25 21:47:19|262144 Entries Validated so far.
2010/12/25 21:47:19|   Completed Validation Procedure
2010/12/25 21:47:19|   Validated 521931 Entries
2010/12/25 21:47:19|   store_swap_size = 9112264k
2010/12/25 21:47:20| storeLateRelease: released 6 objects


--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 10:41 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections


On Sat, Dec 25, 2010 at 10:38 PM, J Webster 
wrote:


The problem appears to be this:
/var/spool/squid/swap.state: (13) Permission denied

Why would that happen overnight?


Looks like some filesystem corruption happened for some reason. Is
there any other messages like this?


--
   /kinkie







--
/kinkie

Re: [squid-users] refusing connections

[J Webster]

This is the last bit from the cache log after restarting - seems ok but 
still refusing connections.
On a restart it seems to connect and load a page but then stops halfway just 
like it times out.


2010/12/25 21:47:16| DNS Socket created at 0.0.0.0, port 56340, FD 6
2010/12/25 21:47:16| Adding nameserver 208.67.222.222 from squid.conf
2010/12/25 21:47:16| Adding nameserver 208.67.220.220 from squid.conf
2010/12/25 21:47:16| helperOpenServers: Starting 5 'ncsa_auth' processes
2010/12/25 21:47:16| User-Agent logging is disabled.
2010/12/25 21:47:16| Referer logging is disabled.
2010/12/25 21:47:16| Unlinkd pipe opened on FD 16
2010/12/25 21:47:16| Swap maxSize 4096 + 102400 KB, estimated 0 objects
2010/12/25 21:47:16| Target number of buckets: 157932
2010/12/25 21:47:16| Using 262144 Store buckets
2010/12/25 21:47:16| Max Mem  size: 102400 KB
2010/12/25 21:47:16| Max Swap size: 4096 KB
2010/12/25 21:47:16| Local cache digest enabled; rebuild/rewrite every 
3600/3600 sec

2010/12/25 21:47:16| Rebuilding storage in /var/spool/squid (CLEAN)
2010/12/25 21:47:16| Using Least Load store dir selection
2010/12/25 21:47:16| Set Current Directory to /var/spool/squid
2010/12/25 21:47:16| Loaded Icons.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 0.0.0.0, port 8080, 
FD 20.
2010/12/25 21:47:17| Accepting proxy HTTP connections at 88.208.237.198, 
port 80, FD 21.

2010/12/25 21:47:17| Accepting ICP messages at 0.0.0.0, port 3130, FD 22.
2010/12/25 21:47:17| WCCP Disabled.
2010/12/25 21:47:17| Ready to serve requests.
2010/12/25 21:47:17| Store rebuilding is  0.8% complete
2010/12/25 21:47:19| Done reading /var/spool/squid swaplog (521968 entries)
2010/12/25 21:47:19| Finished rebuilding storage from disk.
2010/12/25 21:47:19|521968 Entries scanned
2010/12/25 21:47:19| 0 Invalid entries.
2010/12/25 21:47:19| 0 With invalid flags.
2010/12/25 21:47:19|521931 Objects loaded.
2010/12/25 21:47:19| 0 Objects expired.
2010/12/25 21:47:19| 0 Objects cancelled.
2010/12/25 21:47:19| 6 Duplicate URLs purged.
2010/12/25 21:47:19|31 Swapfile clashes avoided.
2010/12/25 21:47:19|   Took 2.9 seconds (178595.5 objects/sec).
2010/12/25 21:47:19| Beginning Validation Procedure
2010/12/25 21:47:19|262144 Entries Validated so far.
2010/12/25 21:47:19|   Completed Validation Procedure
2010/12/25 21:47:19|   Validated 521931 Entries
2010/12/25 21:47:19|   store_swap_size = 9112264k
2010/12/25 21:47:20| storeLateRelease: released 6 objects


--
From: "Kinkie" 
Sent: Saturday, December 25, 2010 10:41 PM
To: "J Webster" 
Cc: 
Subject: Re: [squid-users] refusing connections

On Sat, Dec 25, 2010 at 10:38 PM, J Webster  
wrote:

The problem appears to be this:
/var/spool/squid/swap.state: (13) Permission denied

Why would that happen overnight?


Looks like some filesystem corruption happened for some reason. Is
there any other messages like this?


--
/kinkie

Re: [squid-users] refusing connections

[J Webster]

The problem appears to be this:
/var/spool/squid/swap.state: (13) Permission denied

Why would that happen overnight?
--
From: "J Webster" 
Sent: Saturday, December 25, 2010 10:33 PM
To: 
Subject: [squid-users] refusing connections


For no apparent reason, squid has started refusing connections today.
Any ideas if there is something wrong with the conf below:

auth_param basic realm AA proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port XX.XXX.XXX.XX:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
visible_hostname AAProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000

[squid-users] refusing connections

[J Webster]

For no apparent reason, squid has started refusing connections today.
Any ideas if there is something wrong with the conf below:

auth_param basic realm AA proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port XX.XXX.XXX.XX:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
visible_hostname AAProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000

[squid-users] sluggish squid

[J Webster]

I have had squid installed a while and tonight it is behaving very 
sluggishly.

ANy ideas what I can do to check the problem?
It doesn;t seem it is memory.
Could be bandwidth related but I am getting could not connect errors from 
squid and then occasionally it connects but if it was bandwidth it would 
just be slowloading.

httpd works fine, so does openvpn, and ssh, so must be a squid issue.

There is no syn or ddos and I have about 120 connections.

I have each use limited to 1Mb so really doubt a bandwidth issue...
[root ]# netstat -nat | grep :80 | grep ESTABLISHED| wc -l
84
[root ]# netstat -nat | grep :8080 | grep ESTABLISHED| wc -l
57

CPU load averages 0.05 (1 min) 0.03 (5 mins) 0.00 (15 mins)
Real memory 928.13 MB total, 254.31 MB used

Virtual memory 509.84 MB total, 7.41 MB used

Local disk space 232.06 GB total, 14.23 GB used 

[squid-users] Re: squid cache not updating?

[J Webster]

Any ideas? Do I have to revuild the cache?
Really not sure what to do on this one. Unsure whether the cache is being
updated or it has stopped using the cache, etc.



--
From: "J Webster" 
Sent: Friday, December 03, 2010 8:03 AM
To: 
Subject: squid cache not updating?


I have my cache mounted on a drive at /var/spool/squid.
The other day I tied to mount a new folder also on the same drive, which 
is apparently not the best thing to do.
Since then, I am not sure if my squid cache is updating or not. It seems 
to be stuck at 35Gb use and 16% capacity.

Is there anyway to check if the cache is updating?

[squid-users] Re: squid cache not updating?

[J Webster]

Any ideas?
Really not sure what to do on this one. Unsure whether the cache is being 
updated or it has stopped using the cache, etc.


--
From: "J Webster" 
Sent: Saturday, December 04, 2010 7:14 PM
To: 
Subject: Re: squid cache not updating?


Do I rebuild the cache?

--
From: "J Webster" 
Sent: Friday, December 03, 2010 8:03 AM
To: 
Subject: squid cache not updating?


I have my cache mounted on a drive at /var/spool/squid.
The other day I tied to mount a new folder also on the same drive, which 
is apparently not the best thing to do.
Since then, I am not sure if my squid cache is updating or not. It seems 
to be stuck at 35Gb use and 16% capacity.
Is there anyway to check if the cache is updating? 

[squid-users] Re: squid cache not updating?

[J Webster]

Do I rebuild the cache?

--
From: "J Webster" 
Sent: Friday, December 03, 2010 8:03 AM
To: 
Subject: squid cache not updating?


I have my cache mounted on a drive at /var/spool/squid.
The other day I tied to mount a new folder also on the same drive, which 
is apparently not the best thing to do.
Since then, I am not sure if my squid cache is updating or not. It seems 
to be stuck at 35Gb use and 16% capacity.
Is there anyway to check if the cache is updating? 

[squid-users] squid cache not updating?

[J Webster]

I have my cache mounted on a drive at /var/spool/squid.
The other day I tied to mount a new folder also on the same drive, which 
is apparently not the best thing to do.
Since then, I am not sure if my squid cache is updating or not. It seems 
to be stuck at 35Gb use and 16% capacity.
Is there anyway to check if the cache is updating? 

[squid-users] Re: squid cache not updating?

[J Webster]

I have my cache mounted on a drive at /var/spool/squid.
The other day I tied to mount a new folder also on the same drive, which 
is apparently not the best thing to do.
Since then, I am not sure if my squid cache is updating or not. It seems 
to be stuck at 35Gb use and 16% capacity.
Is there anyway to check if the cache is updating? 

[squid-users] squid cache not updating?

[J Webster]

I have my cache mounted on a drive at /var/spool/squid.
The other day I tied to mount a new folder also on the same drive, which is 
apparently not the best thing to do.
Since then, I am not sure if my squid cache is updating or not. It seems to 
be stuck at 35Gb use and 16% capacity.
Is there anyway to check if the cache is updating? 

Re: [squid-users] 2 NCSA password files

[J Webster]

So, if my users change on a daily basis (sometimes hourly), can I update the 
acl file on the fly.

So, I'd have 1 ncsa file with the username and passwords for all users.
Then 2 acl files with high speed users and low speed users?


--
From: "David Parks" 
Sent: Sunday, November 21, 2010 10:02 AM
To: "'J Webster'" ; 
Subject: RE: [squid-users] 2 NCSA password files

If you write a custom ACL helper you can match users against any criteria 
you define, then implement the delay pools for users that matched your 
custom ACL helper.

[squid-users] 2 NCSA password files

[J Webster]

Is it possible to have 2 NCSA password auth files and then have different 
download speeds per each NCSA file/user group? 

Re: Fwd: Re: [squid-users] Re: Bandwidth split?

[J Webster]

To start off simply and just get the limit working, can I use this:
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 125000/125000
delay_access 1 allow all

That should limit all connections to 1 Mbps.
I have seen varying lines for the last one ranging from allow all, deny all, 
and webmin doesn;t even put in that line at all.
After that, I would like to add in the regexs one by one if it start 
limiting the server.

Will the above just limit by IP connection?
So, I don;t need to bother cross checking the access of the ncsa_users?
Only ncsa_users have access to the server anyway.

Re: Fwd: Re: [squid-users] Re: Bandwidth split?

[J Webster]

256/8 and then * 100 is what I have been doing I think.
Ideally, I need an unlimited server bucket but all the ncsa users to have 
512kbps to 1Mbps individually.
For example, 60 users connected and they all have 1Mbps max. The server is 
100Mbit.


Current conf as per previous emails:
acl magic_words1 url_regex -i 192.168
acl magic_words2 url_regex -i ftp .exe .mp3 .vqf .tar.gz .gz .rpm .zip .rar 
.avi .mpeg .mpe .mpg .qt .ram .rm .iso .raw .wav .mov

# Added nsca_users in a boolean AND fashion
delay_pools 3
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_access 1 allow ncsa_users magic_words1
delay_access 1 deny all
delay_class 2 2
#delay_parameters 2 5000/15 5000/12
delay_parameters 2 32000/15 32000/12
delay_access 2 allow ncsa_users magic_words2
delay_access 2 deny all
delay_class 3 1
# 512Kbit/s fill rate, 1024 Kbit/s reserve
delay_parameters 3 64000/128000
delay_access 3 allow ncsa_users
delay_access 3 deny all

--
From: "Amos Jeffries" 
Sent: Wednesday, November 10, 2010 11:17 AM
To: 
Subject: Re: Fwd: Re: [squid-users] Re: Bandwidth split?


On 10/11/10 05:45, J Webster wrote:

Thanks for the help.
Does anyone know the difference between fill rate and reserve in how
they are applied to restrciting proxy bandwidth?

# 256 Kbit/s fill rate, 1024 Kbit/s reserve




Firstly it is measured in BYTE/sec. So alter you numbers by 8 for 
squid.conf


http://wiki.squid-cache.org/Features/DelayPools

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.9
  Beta testers wanted for 3.2.0.3

Re: Fwd: Re: [squid-users] Re: Bandwidth split?

[J Webster]

I have v2.6
The article seems to show that squid can only ever limit on a single 
aggregate and then per user afterwards: 
http://www.visolve.com/squid/squid27/delaypools.php#delay_pools

Should it be a class 4 bucket instead?


--
From: "Chad Naugle" 
Sent: Tuesday, November 09, 2010 11:05 PM
To: "J Webster" ; 
Subject: Re: Fwd: Re: [squid-users] Re: Bandwidth split?


I could be wrong here, please read up on Visolve, relating to your
version of Squid for more information...

(3.X) http://www.visolve.com/squid/squid30/delaypools.php#delay_pools
(2.7) http://www.visolve.com/squid/squid27/delaypools.php#delay_pools


-
Chad E. Naugle
Tech Support II, x. 7981
Travel Impressions, Ltd.




"J Webster"  11/9/2010 3:32 PM >>>

So, at present, my #3 pool is only allowing 64kbps for the entire
server?
That doesn;t seem right as using iftop on the server reports that the
current speed is 2.22Mb and reached a peak of 5.25Mb total (in/out)
Why use a class 3 at all in most server cases? I suppose you might want
to
restrict proxy server usage to a percentage.
I would like the server to be unlimited but each user using it should
only
get say 1Mb max, maybe less.
So, can I forget the class 3 completely?

--
From: "Chad Naugle" 
Sent: Tuesday, November 09, 2010 9:16 PM
To: "J Webster" ;

Subject: Re: Fwd: Re: [squid-users] Re: Bandwidth split?


Remember, the #3 pool here, should represent the TOTAL allowed rate

for

the Squid Proxy, for all users in 1 bucket.  If you want to limit
individual users' bandwidth, use at least a Class 2, or Class 3

pool.


In my trial & error testing, 64000 does not directly equate to

64k/sec

in actual throughput.  You need to tweak the settings to fit your
situation, until you get desirable results.

Here is my example to get up to 150-300k/sec thruput per a userid,
notice I used a Class 4 pool due to eDirectory IP -> User mappings,

and

deny access to the pool outside of normal business hours, and

"Level1"

employee's.

# Delay Pools
delay_pools 1
delay_class 1 4
delay_access 1 deny allowed_src
delay_access 1 deny edir_root_level1
delay_access 1 deny edir_beth_level1
delay_access 1 deny edir_far_level1
delay_access 1 deny edir_mel_level1
delay_access 1 deny edir_riv_level1
delay_access 1 deny !biz_hours_norm !biz_hours_wknd !biz_hours_ext
delay_access 1 allow all
# Hard 128-256K/sec Limit per/user
delay_parameters 1 -1/-1 -1/-1 128000/256000 128000/256000
delay_initial_bucket_level 25


---------
Chad E. Naugle
Tech Support II, x. 7981
Travel Impressions, Ltd.




"J Webster"  11/9/2010 3:05 PM >>>

This is how it looks at present.
It seems that it might be appliued as aggregate and individual?


Number Class Aggregate limit Individual limit Network limit
 1 Aggregate and individual Unlimited Unlimited=20
2 Aggregate and individual 32000/sec, 15 max 32000/sec, 12
max=20
3 Aggregate 64000/sec, 128000 max=20

Is that 64kbps on #3? Should be enough to view video shouldn;t it?



Travel Impressions made the following annotations
-
"This message and any attachments are solely for the intended

recipient

and may contain confidential or privileged information.  If you are

not

the intended recipient, any disclosure, copying, use, or distribution

of

the information included in this message and any attachments is
prohibited.  If you have received this communication in error,

please

notify us by reply e-mail and immediately and permanently delete

this

message and any attachments.
Thank you."




Travel Impressions made the following annotations
-
"This message and any attachments are solely for the intended recipient
and may contain confidential or privileged information.  If you are not
the intended recipient, any disclosure, copying, use, or distribution of
the information included in this message and any attachments is
prohibited.  If you have received this communication in error, please
notify us by reply e-mail and immediately and permanently delete this
message and any attachments.
Thank you."

Re: Fwd: Re: [squid-users] Re: Bandwidth split?

[J Webster]

So, at present, my #3 pool is only allowing 64kbps for the entire server?
That doesn;t seem right as using iftop on the server reports that the 
current speed is 2.22Mb and reached a peak of 5.25Mb total (in/out)
Why use a class 3 at all in most server cases? I suppose you might want to 
restrict proxy server usage to a percentage.
I would like the server to be unlimited but each user using it should only 
get say 1Mb max, maybe less.

So, can I forget the class 3 completely?

--
From: "Chad Naugle" 
Sent: Tuesday, November 09, 2010 9:16 PM
To: "J Webster" ; 
Subject: Re: Fwd: Re: [squid-users] Re: Bandwidth split?


Remember, the #3 pool here, should represent the TOTAL allowed rate for
the Squid Proxy, for all users in 1 bucket.  If you want to limit
individual users' bandwidth, use at least a Class 2, or Class 3 pool.

In my trial & error testing, 64000 does not directly equate to 64k/sec
in actual throughput.  You need to tweak the settings to fit your
situation, until you get desirable results.

Here is my example to get up to 150-300k/sec thruput per a userid,
notice I used a Class 4 pool due to eDirectory IP -> User mappings, and
deny access to the pool outside of normal business hours, and "Level1"
employee's.

# Delay Pools
delay_pools 1
delay_class 1 4
delay_access 1 deny allowed_src
delay_access 1 deny edir_root_level1
delay_access 1 deny edir_beth_level1
delay_access 1 deny edir_far_level1
delay_access 1 deny edir_mel_level1
delay_access 1 deny edir_riv_level1
delay_access 1 deny !biz_hours_norm !biz_hours_wknd !biz_hours_ext
delay_access 1 allow all
# Hard 128-256K/sec Limit per/user
delay_parameters 1 -1/-1 -1/-1 128000/256000 128000/256000
delay_initial_bucket_level 25


-
Chad E. Naugle
Tech Support II, x. 7981
Travel Impressions, Ltd.




"J Webster"  11/9/2010 3:05 PM >>>

This is how it looks at present.
It seems that it might be appliued as aggregate and individual?


Number Class Aggregate limit Individual limit Network limit
 1 Aggregate and individual Unlimited Unlimited=20
2 Aggregate and individual 32000/sec, 15 max 32000/sec, 12
max=20
3 Aggregate 64000/sec, 128000 max=20

Is that 64kbps on #3? Should be enough to view video shouldn;t it?



Travel Impressions made the following annotations
-
"This message and any attachments are solely for the intended recipient
and may contain confidential or privileged information.  If you are not
the intended recipient, any disclosure, copying, use, or distribution of
the information included in this message and any attachments is
prohibited.  If you have received this communication in error, please
notify us by reply e-mail and immediately and permanently delete this
message and any attachments.
Thank you."

Re: Fwd: Re: [squid-users] Re: Bandwidth split?

[J Webster]

Thanks for the help.
Does anyone know the difference between fill rate and reserve in how they 
are applied to restrciting proxy bandwidth?

# 256 Kbit/s fill rate, 1024 Kbit/s reserve

Re: [squid-users] Re: Bandwidth split?

[J Webster]

It is defines earlier in the conf as:

auth_param basic realm AName proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl cacheadmin src 88.xxx.xxx.xxx
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all


--
From: "Chad Naugle" 
Sent: Tuesday, November 09, 2010 3:10 PM
To: "J Webster" ; 
Subject: Re: [squid-users] Re: Bandwidth split?


So, where is the ncsa_users acl definition?  Is it a external_acl_type,
auth_param, or just a plan ACL ?

Re: [squid-users] Re: Bandwidth split?

[J Webster]

I still have users connecting at around 1.91Mb and faster on the server 
(seen using iftop) so

the delay pools don;t seem to be working for the ncsa_users.
Only thing I can think of is that it's not registering the ncsa users in the 
acl somehow?


acl magic_words1 url_regex -i 192.168
acl magic_words2 url_regex -i ftp .exe .mp3 .vqf .tar.gz .gz .rpm .zip
.rar .avi .mpeg .mpe .mpg .qt .ram .rm .iso .raw .wav .mov
acl restuser proxy_auth ncsa_users

delay_pools 3
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_access 1 allow magic_words1
delay_access 1 deny all
delay_class 2 2
delay_parameters 2 5000/15 5000/12
delay_access 2 allow magic_words2
delay_access 2 deny all
delay_class 3 1
# 256 Kbit/s fill rate, 1024 Kbit/s reserve
delay_parameters 3 32000/128000
delay_access 3 allow restuser
delay_access 3 deny all

Re: [squid-users] Re: Bandwidth split?

[J Webster]

Thanks.
I still have users connecting at around 1.91Mb and faster on the server so 
the delay pools don;t seem to be working.

Only thing I can think of is that it's not registering the ncsa users?

--
From: "Chad Naugle" 
Sent: Monday, November 08, 2010 4:36 PM
To: "J Webster" ; 
Subject: Re: [squid-users] Re: Bandwidth split?


Yes sorry, at work.  See Below.  I am not 100% on fill-rate versus the
other numbers, so I'll leave that up for someone else to reply.  I would
just tinker with the values until you get acceptable results until
then.

acl magic_words1 url_regex -i 192.168
acl magic_words2 url_regex -i ftp .exe .mp3 .vqf .tar.gz .gz .rpm .zip
.rar .avi .mpeg .mpe .mpg .qt .ram .rm .iso .raw .wav .mov
acl restuser proxy_auth ncsa_users

delay_pools 3
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_access 1 allow magic_words1
delay_access 1 deny all
delay_class 2 2
delay_parameters 2 5000/15 5000/12
delay_access 2 allow magic_words2
delay_access 2 deny all
delay_class 3 1
# 256 Kbit/s fill rate, 1024 Kbit/s reserve
delay_parameters 3 32000/128000
delay_access 3 allow restuser
delay_access 3 deny all

Re: [squid-users] Re: Bandwidth split?

[J Webster]

do I need to add this:
delay_access 2 deny all
delay_access 1 deny all
?

Also, what is the difference between fill rate and reserve?
I think I have a fill rate of 256, maybe I should increase this for watching 
video?


I am using iftop on the server, and users still seem to be connecting at 
more than 1Mbps so maybe it isn;t picking up the ncsa users?





From: Chad Naugle
Sent: Monday, November 08, 2010 4:11 PM
To: J Webster ; squid-users@squid-cache.org
Subject: Re: [squid-users] Re: Bandwidth split?


Your problem here is that you are trying to layer delay_pool 1 twice, so I 
corrected the config below adding a third delay_pool for your ncsa_users.

Re: [squid-users] Re: Bandwidth split?

[J Webster]

I have done this but I am not sure if it will pick up the ncsa users.
This should restrict max bandwidth for any 1 user to 1024 (1Mbps)?

acl magic_words1 url_regex -i 192.168
acl magic_words2 url_regex -i ftp .exe .mp3 .vqf .tar.gz .gz .rpm .zip .rar 
.avi .mpeg .mpe .mpg .qt .ram .rm .iso .raw .wav .mov

delay_pools 2
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_access 1 allow magic_words1
delay_class 2 2
delay_parameters 2 5000/15 5000/12
delay_access 2 allow magic_words2

acl restuser proxy_auth ncsa_users
delay_class 1 1
# 256 Kbit/s fill rate, 1024 Kbit/s reserve
delay_parameters 1 32000/128000
delay_access 1 allow restuser
delay_access 1 deny all


--
From: "Chad Naugle" 
Sent: Monday, November 08, 2010 3:57 PM
To: "J Webster" ; ; 
"Chad Naugle" 

Subject: [squid-users] Re: Bandwidth split?


Anyway, I apologize for the short response, I was busy on the phone.  I
would research delay_pools and try to figure out / tweak your config to
meet your needs.  It's not a real straight forward config, but that's
because it is very flexible in how users are limited.  The only thing
that it does not do is control uploading / POST requests.

[squid-users] Re: Bandwidth split?

[J Webster]

I have put in some controls for downloading files like iso, mp3 etc but I 
would like to limit the connection per ip address?


--
From: "J Webster" 
Sent: Sunday, November 07, 2010 9:18 PM
To: 
Subject: Bandwidth split?

It is becoming apparent that some users are hogging the bandwidth on the 
server by downloading videos instead of streaming them.

Any idea on how I can restrict this?
I would like to keep the server as unlimited downloads but split the 
bandwidth at any one time between the users - I figured that this was 
shared automatically but it seems anyone downloading a lot gets more use 
of the bandwidth? 

[squid-users] Bandwidth split?

[J Webster]

It is becoming apparent that some users are hogging the bandwidth on the 
server by downloading videos instead of streaming them.

Any idea on how I can restrict this?
I would like to keep the server as unlimited downloads but split the 
bandwidth at any one time between the users - I figured that this was shared 
automatically but it seems anyone downloading a lot gets more use of the 
bandwidth? 

Re: [squid-users] Limiting user's bandwidth

[J Webster]

Will that share the bandwidth pro rata?
Say the bandwidth is 10Mbps and you have 10 users, they only get 1 each?
Otherwise isn't it shared equally anyway?
There must be a way to apply a kbps limit in case someone is hogging the 
bandwidth?


--
From: "Andrew Beverley" 
Sent: Tuesday, October 26, 2010 11:24 PM
To: "Landy Landy" 
Cc: "Squid-Users" 
Subject: Re: [squid-users] Limiting user's bandwidth


Thanks Andy for your reply and taking your time to help like always.



No problem at all.


> > $tc class add dev eth0 parent 1:0  classid 1:1
> htb rate 900kbit ceil 945kbit

As I understand, correct me if I'm wrong, this rule is telling the
 kernel how much bw we want to use globally or how big is the entire
 bucket.


Yes, but the two are the same, so I would just keep these two parameters
as the same figure. This is the maximum amount of bandwidth that the
whole class can use.


I know the amount of the leafs don't add up to the root's bw but, not
 all clients are connected at the same time.


That's the beauty of HTB. Set your leaf rates to be the maximum amount
you would want them to ever have, if they did happen to all be connected
at the same time. Set the maximum to be the maximum that they should
ever have if it was possible. The prio parameter will then share the
excess bandwidth accordingly, should there be any available.


 I don't know if this is
 the problem or not but, I have similar rules for the LAN interface
 which works pretty well.


I don't know, but in accordance with the above, there is no reason to
not have them all add up.


The weird thing is if I don't use squid caching and just use normal
 FORWARD chain along with these tc script the upload and download
 throtle works fine.


Ah, well the difference is that you are using INPUT/OUTPUT chains with
Squid, not FORWARD, so that will be the difference.

Are you just trying to share bandwidth fairly between users? If so, your
best bet is to change to one leaf for all your clients, but attach a
filter to it that will share bandwidth *by IP address* (see below) - the
default is to share by connection. If you want an overall limit you can
apply that to the one leaf, and then everybody within it will get their
fair share within.

If you want people who are downloading/uploading large amounts to get a
reduced share, then set up an iptables rule to set a MARK based on the
amount of date transferred in that connection. There's a good example at
the following web page, although it's currently a work in progress:

http://www.andybev.com/index.php/Fair_traffic_shaping_an_ADSL_line_for_a_local_network_using_Linux

Andy

Re: [squid-users] upgrade

[J Webster]

So, I could just do yum upgrade squid?

--
From: "Amos Jeffries" 
Sent: Tuesday, August 03, 2010 10:00 AM
To: 
Subject: Re: [squid-users] upgrade


Riaan Nolan wrote:

Centos meh. their repo's are so far behind they think they are in front.

It's better to upgrade. Since I upgraded things started working
properly, like external ACLs with ldap_groups in Active Directory.
No more problems for me.

 > Can I leave the existing cache in place and config files or
I trashed my existing cache, so I would not know if it will work.

Don't compile it from SRC ... get the src RPM e.g
yum install rpm-build openjade linuxdoc-tools openldap-devel pam-devel
openssl-devel httpd rpm-devel
wget
http://www.jur-linux.com/rpms/el-updates/5Client/SRPMS/squid-3.1.0.15-2.el5.src.rpm 
rpm -ivh squid-3.1.0.15-2.el5.src.rpm

rpmbuild -bb squid.spec


Looks like they have 3.1.4 in there too. Either one.



All the best to you :)

ciao/Riaan

On 03/08/2010 14:44, J. Webster wrote:

I currently have squid 2.6 running on centos - they haven't updated =
their
repository yet.
WIll upgrading to 3.1.6 have any performance
enhancements?


Over 2.6 definitely.
A small bit in speed, and a LOT in HTTP/1.1 protocol support which amounts 
to streamlining and bandwidth.



Can I leave the existing cache in place and config files or
will they be =
overwritten during the make commands?


Only existing binaries and documentation gets replaced.

Existing cache is not touched until squid starts. Then some pieces get 
upgraded during normal operation.


Existing config is not touched, new config files should get added as/if 
needed.


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.5

[squid-users] upgrade

[J. Webster]

I currently have squid 2.6 running on centos - they haven't updated =
their 
repository yet.
WIll upgrading to 3.1.6 have any performance 
enhancements?
Can I leave the existing cache in place and config files or 
will they be =
overwritten during the make commands?
  

RE: [squid-users] Limiting to 1 IP address / routers?

[J. Webster]

Is squid somehow checking the local IP address then?
My proxy is a public proxy so any person accessing it does so with a WAN IP.
How does squid know that it is a different computer accessing it to block them 
or is it the ncsa auth that is doing some check?
I only want 1 IP to access it but thought that 1 WAN IP might be ok in some 
cases to have 2 computers behind it.
I don't want someone to give out their username so that someone else can use 
that name from another location (another WAN).
  
_
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] Limiting to 1 IP address / routers?

[J. Webster]

Because in the squid conf there is a setting to limit use to 1 IP address.
So if a user connect to my proxy with username test 1 and then goes to a 2nd 
computer at his home, can he connect using username test1 as well or will it 
block him out?


> Date: Sat, 8 May 2010 13:59:02 -0400
> From: jas...@adventureaquarium.com
> To: webster_j...@hotmail.com; squid-users@squid-cache.org
> Subject: RE: [squid-users] Limiting to 1 IP address / routers?
>
> And???
> I have 120 computers behind 1 WAN address, with NCSA auth. Been that way for 
> years.
>
> Jason
>
>
>
> ..·><((((º>
>
>
>> -Original Message-
>> From: J. Webster [mailto:webster_j...@hotmail.com]
>> Sent: Saturday, May 08, 2010 1:56 PM
>> To: squid-users@squid-cache.org
>> Subject: [squid-users] Limiting to 1 IP address / routers?
>>
>>
>>
>> If there are 3 computers behind a router with 1 WAN IP
>> address, can they all use the proxy server at the same time
>> with the same logon ncsa name?
>> I'm currently running a NCSA user/pass authentication.
>> For example,
>> 192.168.0.1 user test1
>> 192.168.0.2 user test1
>> 192.168.0.3 user test1
>> but all have WAN IP address 88.xxx.xxx.100
>>
>>
>> _
>> http://clk.atdmt.com/UKM/go/195013117/direct/01/
>> We want to hear all your funny, exciting and crazy Hotmail
>> stories. Tell us now
>>
>
>
> This message has been scanned for malware by Websense. www.websense.com
  
_
http://clk.atdmt.com/UKM/go/19780/direct/01/
Do you have a story that started on Hotmail? Tell us now

[squid-users] Limiting to 1 IP address / routers?

[J. Webster]

If there are 3 computers behind a router with 1 WAN IP address, can they all 
use the proxy server at the same time with the same logon ncsa name?
I'm currently running a NCSA user/pass authentication.
For example,
192.168.0.1 user test1
192.168.0.2 user test1
192.168.0.3 user test1
but all have WAN IP address 88.xxx.xxx.100

  
_
http://clk.atdmt.com/UKM/go/195013117/direct/01/
We want to hear all your funny, exciting and crazy Hotmail stories. Tell us now

RE: [squid-users] NCSA upper case sensitive?

[J. Webster]

So, if the squid conf as default has this:
auth_param basic casesensitive off

and someone logs in with MyUserName
then it will be converted to myusername and the authentication will fail?


> Date: Tue, 9 Mar 2010 12:40:41 +1300
> From: squ...@treenet.co.nz
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] NCSA upper case sensitive?
>
> On Mon, 8 Mar 2010 20:40:45 +, "J. Webster" 
> wrote:
>> Is NCSA auth case sensitive for the login name?
>> We have a case recently where it would not take the username as a
> mixture
>> of capitals (eg TestUser) whereas the password works successfully
> whether
>> lower or upper.
>
> NSCA itself is not. Basic authentication is by default.
>
> auth_param basic casesensitive on
>
> Amos
  
_
Do you have a story that started on Hotmail? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] Youtube and BBC iPlayer

[J. Webster]

Many of these video sites are starting to use RTMP, which is causing a problem 
for proxy servers.
Do you know of anyway to reroute port 1935 through the proxy server or to 
somehow catch the RTMP protocol and redirect it?


> Date: Thu, 11 Mar 2010 01:19:09 +1300
> From: squ...@treenet.co.nz
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] Youtube and BBC iPlayer
>
> twintu...@f2s.com wrote:
>> Our bandwidth is been eaten by teachers showing Youtube and BBC iPlayer,
>>
>> Is there a way to get S3.0.19 to effectivly cache this content?
>>
>
> I'm not sure about iPlayer.
>
> YouTube is cachable by removing the default rules blocking dynamic stuff
> being cached. The storeurl feature from 2.7 designed to reduce
> duplicates is not available in 3.0, so the benefit is not great there.
>
> If it is a big problem, I'd suggest going to 2.7 and using the FAQ
> config examples about YouTube until we have that feature ported.
> Or sponsoring someone to do the port for you :) it's not huge.
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE8 or 3.0.STABLE24
> Current Beta Squid 3.1.0.17
  
_
Got a cool Hotmail story? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

[squid-users] NCSA upper case sensitive?

[J. Webster]

Is NCSA auth case sensitive for the login name?
We have a case recently where it would not take the username as a mixture of 
capitals (eg TestUser) whereas the password works successfully whether lower or 
upper.
  
_
We want to hear all your funny, exciting and crazy Hotmail stories. Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] Cache manager analysis

[J. Webster]

gt; authenticate_cache_garbage_interval 1 hour
>> authenticate_ip_ttl 2 hours
>> #acl all src 0.0.0.0/0.0.0.0
>> acl src all
>> acl manager proto cache_object
>> acl localhost src 127.0.0.1
>> acl cacheadmin src 88.xxx.xxx.xxx
>> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
>> acl SSL_ports port 443
>> acl Safe_ports port 80 # http
>> acl Safe_ports port 21 # ftp
>> acl Safe_ports port 443 # https
>> acl Safe_ports port 70 # gopher
>> acl Safe_ports port 210 # wais
>> acl Safe_ports port 1025-65535 # unregistered ports
>> acl Safe_ports port 280 # http-mgmt
>> acl Safe_ports port 488 # gss-http
>> acl Safe_ports port 591 # filemaker
>> acl Safe_ports port 777 # multiling http
>> acl Safe_ports port 1863 # MSN messenger
>> acl ncsa_users proxy_auth REQUIRED
>> acl maxuser max_user_ip -s 2
>> acl CONNECT method CONNECT
>> http_access allow manager localhost
>> http_access allow manager cacheadmin
>> http_access deny !Safe_ports
>> http_access deny CONNECT !SSL_ports
>> http_access deny to_localhost
>> http_access deny manager
>> http_access allow ncsa_users
>> http_access deny maxuser
>> #http_access allow localhost
>> http_access deny all
>> icp_access allow all
>> http_port 8080
>> http_port 88.xxx.xxx.xxx:80
>> hierarchy_stoplist cgi-bin ?
>> cache_mem 100 MB
>> maximum_object_size_in_memory 50 KB
>> cache_replacement_policy heap LFUDA
>> cache_dir aufs /var/spool/squid 4 16 256
>> maximum_object_size 50 MB
>> cache_swap_low 90
>> cache_swap_high 95
>> access_log /var/log/squid/access.log squid
>> cache_log /var/log/squid/cache.log
>> buffered_logs on
>> #acl QUERY urlpath_regex cgi-bin \?
>> #cache deny QUERY
>> refresh_pattern ^ftp: 1440 20% 10080
>> refresh_pattern ^gopher: 1440 0% 1440
>> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>> refresh_pattern . 0 20% 4320
>> quick_abort_min 0 KB
>> quick_abort_max 0 KB
>> acl apache rep_header Server ^Apache
>> broken_vary_encoding allow apache
>> half_closed_clients off
>> cache_mgr a...@aaa.com
>> cachemgr_passwd aaa all
>> visible_hostname ProxyServer
>> log_icp_queries off
>> dns_nameservers 208.67.222.222 208.67.220.220
>> hosts_file /etc/hosts
>> memory_pools off
>> forwarded_for off
>> client_db off
>> coredump_dir /var/spool/squid
>>
>> 
>>> Date: Sat, 13 Feb 2010 18:03:00 +1300
>>> From: squ...@treenet.co.nz
>>> To: squid-users@squid-cache.org
>>> Subject: Re: [squid-users] Cache manager analysis
>>>
>>> J. Webster wrote:
>>>> What is the best place to start with in cache analysis?
>>>> Would it be cache size, memory object size, IO, etc.?
>>>> I'm looking to optimise the settings for my squid server.
>>>
>>> Step 0) migrate to the latest Squid 2.7 or 3.1 or if possible 2.HEAD
>>> (that one is only nominally beta, it's very stable in reality)
>>>
>>> 1) Start by defining 'optimize' ... are you going to prioritize...
>>> Faster service?
>>> More bandwidth saving?
>>> More client connections?
>>>
>>> 2a) For faster service, look at DNS delays, disk IO delays, maximizing
>>> cacheable objects (dynamic objects etc).
>>>
>>> 2b) For pure bandwidth savings start with a look at object cacheablity.
>>> Check dynamics are being cached, ranges are being fetched in full, etc
>>>
>>> 3) Then profile all the objects stored over a reasonably long period,
>>> looking at size. compare with the age of objects being discarded.
>>>
>>> 3a) tune the storage limits to prioritize the storage locations. giving
>>> priority to RAM, then COSS, then AUFS/diskd.
>>>
>>> 3b) set the storage limits as high as possible to maximize amount of
>>> data stored. anywhere.
>>>
>>> 4) take a good long look at your access controls and in particular the
>>> types speedy/fast/slow. You may get some speed benefits from fixing up
>>> the ordering a bit. regex are killers, remote lookups (helpers, or DNS)
>>> are second worst.
>>> (some performance hints below)
>>>
>>> 5) repeat from (2b) as often as possible. concentrate traffic which
>>> seems to logically be storeable but gets a TCP_MISS anyway.
>>>
>>> Objects served from cache lead to faster service ties for those objects,
>>> so the speed vs bandwidth are 

RE: [squid-users] Cache manager analysis

[J. Webster]

SSL_ports port 443
> acl Safe_ports port 80  # http
> acl Safe_ports port 21  # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70  # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl Safe_ports port 1863 # MSN messenger
> acl ncsa_users proxy_auth REQUIRED
> acl maxuser max_user_ip -s 2
> acl CONNECT method CONNECT
> http_access allow manager localhost
> http_access allow manager cacheadmin
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access deny to_localhost
> http_access deny manager
> http_access allow ncsa_users
> http_access deny maxuser
> #http_access allow localhost
> http_access deny all
> icp_access allow all
> http_port 8080
> http_port 88.xxx.xxx.xxx:80
> hierarchy_stoplist cgi-bin ?
> cache_mem 100 MB
> maximum_object_size_in_memory 50 KB
> cache_replacement_policy heap LFUDA
> cache_dir aufs /var/spool/squid 4 16 256
> maximum_object_size 50 MB
> cache_swap_low 90
> cache_swap_high 95
> access_log /var/log/squid/access.log squid
> cache_log /var/log/squid/cache.log
> buffered_logs on
> #acl QUERY urlpath_regex cgi-bin \?
> #cache deny QUERY
> refresh_pattern ^ftp:   144020% 10080
> refresh_pattern ^gopher:14400%  1440
> refresh_pattern -i (/cgi-bin/|\?)  0 0% 0
> refresh_pattern .   0   20% 4320
> quick_abort_min 0 KB
> quick_abort_max 0 KB
> acl apache rep_header Server ^Apache
> broken_vary_encoding allow apache
> half_closed_clients off
> cache_mgr a...@aaa.com
> cachemgr_passwd aaa all
> visible_hostname ProxyServer
> log_icp_queries off
> dns_nameservers 208.67.222.222 208.67.220.220
> hosts_file /etc/hosts
> memory_pools off
> forwarded_for off
> client_db off
> coredump_dir /var/spool/squid
>
> 
>> Date: Sat, 13 Feb 2010 18:03:00 +1300
>> From: squ...@treenet.co.nz
>> To: squid-users@squid-cache.org
>> Subject: Re: [squid-users] Cache manager analysis
>>
>> J. Webster wrote:
>>> What is the best place to start with in cache analysis?
>>> Would it be cache size, memory object size, IO, etc.?
>>> I'm looking to optimise the settings for my squid server.
>>
>> Step 0) migrate to the latest Squid 2.7 or 3.1 or if possible 2.HEAD
>> (that one is only nominally beta, it's very stable in reality)
>>
>> 1) Start by defining 'optimize' ... are you going to prioritize...
>> Faster service?
>> More bandwidth saving?
>> More client connections?
>>
>> 2a) For faster service, look at DNS delays, disk IO delays, maximizing
>> cacheable objects (dynamic objects etc).
>>
>> 2b) For pure bandwidth savings start with a look at object cacheablity.
>> Check dynamics are being cached, ranges are being fetched in full, etc
>>
>> 3) Then profile all the objects stored over a reasonably long period,
>> looking at size. compare with the age of objects being discarded.
>>
>> 3a) tune the storage limits to prioritize the storage locations. giving
>> priority to RAM, then COSS, then AUFS/diskd.
>>
>> 3b) set the storage limits as high as possible to maximize amount of
>> data stored. anywhere.
>>
>> 4) take a good long look at your access controls and in particular the
>> types speedy/fast/slow. You may get some speed benefits from fixing up
>> the ordering a bit. regex are killers, remote lookups (helpers, or DNS)
>> are second worst.
>> (some performance hints below)
>>
>> 5) repeat from (2b) as often as possible. concentrate traffic which
>> seems to logically be storeable but gets a TCP_MISS anyway.
>>
>> Objects served from cache lead to faster service ties for those objects,
>> so the speed vs bandwidth are inter-related somewhat. But there is a
>> tipping point somewhere where tuning one starts to impact the other.
>>
>>
>>>
>>> Server: about 220GB available for the cache, I'm only using 4 MB at 
>>> present as in the config below.
>>> system D2812-A2
>>> /0 bus D2812-A2
>>> /0/0 memory 110KiB BIOS
>>> /0/4 processor Intel(R) Core(TM)2 Duo CPU E7300 @ 2.66GHz
>>> /0/4/5 memory 64KiB L1 cache
>>> /0/4/6 memory 3MiB L2 cache
>>> /0/4/0.1 processor Logical CPU
>>> /0/4/0.2 processor Logical CPU
&

RE: [squid-users] Cache manager analysis

[J. Webster]

Ok - thanks.
2.HEAD - has this been included in the CentOS repository yet? I believe CentOS 
only has 2.6
So, before I even look at the optimising sections, this gives me a squid.conf 
of the following (does this look ok?):

auth_param basic realm Proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
#acl all src 0.0.0.0/0.0.0.0
acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1
acl cacheadmin src 88.xxx.xxx.xxx 127.0.0.1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
#http_access allow manager localhost
#IP 127.0.0.1 added to cacheadmin acl above instead
http_access allow manager cacheadmin
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny manager
http_access allow ncsa_users
http_access deny maxuser
#http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port 88.xxx.xxx.xxx:80
hierarchy_stoplist cgi-bin ?
#cache_mem 100MB
#maybe increase further, check top
cache_mem 256MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
#acl QUERY urlpath_regex cgi-bin \?
#cache deny QUERY
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?)  0 0% 0
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
cache_mgr a...@aaa.com
cachemgr_passwd aaa all
visible_hostname ProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid


> Date: Sat, 13 Feb 2010 18:03:00 +1300
> From: squ...@treenet.co.nz
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] Cache manager analysis
>
> J. Webster wrote:
>> What is the best place to start with in cache analysis?
>> Would it be cache size, memory object size, IO, etc.?
>> I'm looking to optimise the settings for my squid server.
>
> Step 0) migrate to the latest Squid 2.7 or 3.1 or if possible 2.HEAD
> (that one is only nominally beta, it's very stable in reality)
>
> 1) Start by defining 'optimize' ... are you going to prioritize...
> Faster service?
> More bandwidth saving?
> More client connections?
>
> 2a) For faster service, look at DNS delays, disk IO delays, maximizing
> cacheable objects (dynamic objects etc).
>
> 2b) For pure bandwidth savings start with a look at object cacheablity.
> Check dynamics are being cached, ranges are being fetched in full, etc
>
> 3) Then profile all the objects stored over a reasonably long period,
> looking at size. compare with the age of objects being discarded.
>
> 3a) tune the storage limits to prioritize the storage locations. giving
> priority to RAM, then COSS, then AUFS/diskd.
>
> 3b) set the storage limits as high as possible to maximize amount of
> data stored. anywhere.
>
> 4) take a good long look at your access controls and in particular the
> types speedy/fast/slow. You may get some speed benefits from fixing up
> the ordering a bit. regex are killers, remote lookups (helpers, or DNS)
> are second worst.
> (some performance hints below)
>
> 5) repeat from (2b) as often as possible. concentrate traffic which
> seems to logically be storeable but gets a TCP_MISS anyway.
>
> Objects served from cache lead to faster service ties for those objects,
> so the speed vs bandwidth are inter-related somewhat. But there is a
> tipping point somewhere where tuning one starts to impact the other.
>
>
>>
>> Server: about 220GB available for the cache, I'm only using 4 MB at 
>> present as in the config below.
>> system D2812-A2
>> /0 bus D2812-A2
>> /0/0 memory 110KiB BIOS
>> /0/4 p

RE: [squid-users] cache manager access from web

[J. Webster]

Would that work with:
http_access deny manager CONNECT !SSL_ports


> Date: Sat, 13 Feb 2010 20:58:11 +0100
> From: uh...@fantomas.sk
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] cache manager access from web
>
> On 11.02.10 10:46, J. Webster wrote:
>> I have changed the config and can now login to the cache manager.
>> This was in the conf already:
>> http_access deny CONNECT !SSL_ports
>>
>> So, the issue remains whether allowing password access to the cache manager 
>> is enough.
>> How else can this be made more secure? I guess not if the only way for me to 
>> access it is through a public IP address.
>
> I think allowing managr only on https_port should work and help...
>
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> The 3 biggets disasters: Hiroshima 45, Tschernobyl 86, Windows 95
  
_
Tell us your greatest, weirdest and funniest Hotmail stories
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] Cache manager analysis

[J. Webster]

Thanks.
A few questions on this:
(a) when you said this all src all is that meant to be acl src all?
(b) Hint 2: if possible, define an ACL or the network ranges where you accept 
logins. Use it like so
  The logins are accepted form IP addresses that I never know, it is an 
external proxy server for geo location so not sure I can do this? logins will 
only ever by directed to the 88.xxx.xxx.xxx server though?
(c) cache_mem 100 MB
    Bump this up as high as you can go without risking memory swapping.
    Objects served from RAM are 100x faster than objects not.
    Where can I view if memeory swapping is happening?
(D) maximum_object_size 50 MB
    Bump this up too. Holding full ISO CDs and windows service packs can
    boost performance when one is used from the cache. 40GB of disk can
    store a few.
    If I increase this, will the server ever try to store streamed video? I had 
an efficiency problem with the original configuration that came with squid, 
which meant that streamed video was buffering constantly. Not sure what caused 
it but with the current config it does not do that.
If I increase the cache_mem and max object size do I also need to increase this?
maximum_object_size_in_memory 50 KB
(E) 
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY

Drop the QUERY bits above. It's more than halving the things your Squid can 
store.
Remove the acl and the cache deny?
At present, does this stop the cache from storing anything with a ?, ie dynamic 
pages?
What if the same request is made for a dynamic page, will it retrive it from 
the cache (old page) rather then fetch the new dynamic content?

current conf redone below:

auth_param basic realm Proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
#acl all src 0.0.0.0/0.0.0.0
acl src all
acl manager proto cache_object
acl localhost src 127.0.0.1
acl cacheadmin src 88.xxx.xxx.xxx
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access allow manager localhost
http_access allow manager cacheadmin
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny manager
http_access allow ncsa_users
http_access deny maxuser
#http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port 88.xxx.xxx.xxx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
#acl QUERY urlpath_regex cgi-bin \?
#cache deny QUERY
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?)  0 0% 0
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
cache_mgr a...@aaa.com
cachemgr_passwd aaa all
visible_hostname ProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid


> Date: Sat, 13 Feb 2010 18:03:00 +1300
> From: squ...@treenet.co.nz
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] Cache manager analysis
>
> J. Webster wrote:
>> What is the best place to start with in cache analysis?
>> Would it be cache size, memory object size, IO, etc.?
>> I'm looking to optimise the settings for my squid server.
>
> Step 0) migrate to the latest Squid 2.7 or 3.1 or if possible 2.HEAD
> (that one is only nominally beta, it's very stable in reality)
>
> 1) Start by defining 'optimize' ... are you going to prioritize...
> Faster service?
> More bandwidth saving?
> More client connections?
>
> 2a) For faster service, look at DNS delays, disk IO delays, maximizing
> cacheable objects (dy

[squid-users] what happens whens quid cache gets full?

[J. Webster]

I have my squid cache size set to 4 - is this in MB or kb?
What happens when the cache approaches its max size, do I have to manually 
clear it or does squid take care of that?
Thanks
  
_
Got a cool Hotmail story? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

[squid-users] Cache manager analysis

[J. Webster]

What is the best place to start with in cache analysis?
Would it be cache size, memory object size, IO, etc.?
I'm looking to optimise the settings for my squid server.

Server:    about 220GB available for the cache, I'm only using 4 MB at 
present as in the config below.   
  system D2812-A2
/0    bus    D2812-A2
/0/0  memory 110KiB BIOS
/0/4  processor  Intel(R) Core(TM)2 Duo CPU 
E7300  @ 2.66GHz
/0/4/5    memory 64KiB L1 cache
/0/4/6    memory 3MiB L2 cache
/0/4/0.1  processor  Logical CPU
/0/4/0.2  processor  Logical CPU
/0/7  memory 3MiB L3 cache
/0/2a memory 1GiB System Memory
/0/2a/0   memory 1GiB DIMM DDR2 Synchronous 667 
MHz (1.5 ns)
/0/2a/1   memory DIMM DDR2 Synchronous 667 MHz 
(1.5 ns) [empty]
/0/2a/2   memory DIMM DDR2 Synchronous 667 MHz 
(1.5 ns) [empty]
/0/2a/3   memory DIMM DDR2 Synchronous 667 MHz 
(1.5 ns) [empty]
/0/1  processor
/0/1/0.1  processor  Logical CPU
/0/1/0.2  processor  Logical CPU


Current squid.conf:
-
auth_param basic realm Proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl cacheadmin src 88.xxx.xxx.xxx
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access allow manager localhost
http_access allow manager cacheadmin
http_access deny manager
http_access allow ncsa_users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny maxuser
http_access allow localhost
http_access deny all
icp_access allow all
http_port 8080
http_port 88.xxx.xxx.xxx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 4 16 256
maximum_object_size 50 MB
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern .   0   20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
cache_mgr a...@aaa.com
cachemgr_passwd aaa all
visible_hostname ProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
forwarded_for off
client_db off
coredump_dir /var/spool/squid

  
_
Do you have a story that started on Hotmail? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] cache manager access from web

[J. Webster]

I have changed the config and can now login to the cache manager.
This was in the conf already:
http_access deny CONNECT !SSL_ports

So, the issue remains whether allowing password access to the cache manager is 
enough.
How else can this be made more secure? I guess not if the only way for me to 
access it is through a public IP address.




> Date: Wed, 10 Feb 2010 12:49:36 -0900
> From: crobert...@gci.net
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] cache manager access from web
>
> J. Webster wrote:
>> Doesn't the fact that the manager needs a password in previous config lines 
>> mean that they can't access it?
>>
>
> Fair enough, if you are content with that.
>
>> the ncsa_users is only for http access?
>>
>
> The cachemgr interface is accessed via HTTP. It uses a specific request
> method (identified by the ACLs as manager), but it is a subset of HTTP.
>
> Changing the access rules like...
>
> http_access allow manager localhost
> http_access allow manager cacheadmin
> http_access deny manager
> http_access allow ncsa_users
>
> ...prevents those who are allowed to utilize your cache from even
> attempting access to your cachemgr interface (unless they are surfing
> from localhost, or the IP identified by the cacheadmin ACL). The
> default squid.conf has some further denies (such as preventing CONNECT
> requests to non-SSL ports) that are also missing from this configuration
> snippet, so this is not the only avenue for abuse.
>
> Chris
>
  
_
Got a cool Hotmail story? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] cache manager access from web

[J. Webster]

As a side note
 
>> http_access allow ncsa_users
>> http_access allow manager localhost
>> http_access allow manager cacheadmin
>> http_access deny manager
 
cache_manager access (any access, really) is already allowed to 
ncsa_users, no matter if they are accessing from localhost, 
88.xxx.xxx.xx9 or any other IP.  You might want to have a gander at the 
FAQ section on ACLs (http://wiki.squid-cache.org/SquidFaq/SquidAcl).

Doesn't the fact that the manager needs a password in previous config lines 
mean that they can't access it?
the ncsa_users is only for http access?



> Date: Tue, 9 Feb 2010 16:14:31 -0900
> From: crobert...@gci.net
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] cache manager access from web
>
> Amos Jeffries wrote:
>> J. Webster wrote:
>>> I have followed the tutorial here:
>>> http://wiki.squid-cache.org/SquidFaq/CacheManager
>>> and set up acls to access the cache manager cgi on my server. I have
>>> to access this externally for the moment as that is the only access
>>> to the server that I have (SSH or web). The cache manager login
>>> appears when I access: http://myexternalipaddress/cgi-bin/cachemgr.cgi
>>> I have set the cache manager login and password in the squid.conf
>>> # TAG: cache_mgr
>>> # Email-address of local cache manager who will receive
>>> # mail if the cache dies. The default is "root".
>>> #
>>> #Default:
>>> # cache_mgr root
>>> cache_mgr a...@aaa.com
>>> cachemgr_passwd aaa all
>>> #Recommended minimum configuration:
>>> acl all src 0.0.0.0/0.0.0.0
>>> acl manager proto cache_object
>>> acl localhost src 127.0.0.1/255.255.255.255
>>> acl cacheadmin src 88.xxx.xxx.xx9/255.255.255.255 #external IP address?
>>
>> You don't need the /255.255.255.255 bit. Just a single IP address will
>> do.
>>
>>> acl to_localhost dst 127.0.0.0/8
>>> # Only allow cachemgr access from localhost
>
> As a side note
>
>>> http_access allow ncsa_users
>>> http_access allow manager localhost
>>> http_access allow manager cacheadmin
>>> http_access deny manager
>
> cache_manager access (any access, really) is already allowed to
> ncsa_users, no matter if they are accessing from localhost,
> 88.xxx.xxx.xx9 or any other IP. You might want to have a gander at the
> FAQ section on ACLs (http://wiki.squid-cache.org/SquidFaq/SquidAcl).
>
>>>
>>> However, whenever I enter the password and select localhost port 8080
>>> from the cgi script I get:
>>> The following error was encountered:
>>> Cache Access Denied.
>>> Sorry, you are not currently allowed to request:
>>> cache_object://localhost/
>>> from this cache until you have authenticated yourself.
>>
>> Looks like the CGI script does its own internal access to Squid to
>> fetch the page data. But does not have the right login details to pass
>> your "http_access allow ncsa_auth" security config.
>>
>> Amos
>
> Chris
>
  
_
Got a cool Hotmail story? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] DNUMTHREADS

[J. Webster]

Would this dramatically improve performance or it it best left at default?


> Date: Tue, 9 Feb 2010 17:01:46 +1300
> From: squ...@treenet.co.nz
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] DNUMTHREADS
>
> J. Webster wrote:
>> Is it recommended to recompile squid and increase the DNUMTHREADS value?
>> I read that 30 could easily be used on a 500MHz machine and my machine is 
>> more than 2GHz so would it give an improvement to squid performance.
>> I have been reading through this document here, which recommends various 
>> changes including using the reiserfs filesystem.
>> My machine is CentOS.
>>
>> http://blog.last.fm/2007/08/30/squid-optimization-guide
>>
>
> Not sure how he got that info Squid provides the ./configure
> --enable-async-io[=N_THREADS] option as far back as I can see.
>
> It only affects AUFS disk storage.
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE7 or 3.0.STABLE23
> Current Beta Squid 3.1.0.16
  
_
Do you have a story that started on Hotmail? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

[squid-users] cache manager access from web

[J. Webster]

I have followed the tutorial here: 
http://wiki.squid-cache.org/SquidFaq/CacheManager
and set up acls to access the cache manager cgi on my server. I have to access 
this externally for the moment as that is the only access to the server that I 
have (SSH or web). 
The cache manager login appears when I access: 
http://myexternalipaddress/cgi-bin/cachemgr.cgi
I have set the cache manager login and password in the squid.conf
#  TAG: cache_mgr
#   Email-address of local cache manager who will receive
#   mail if the cache dies. The default is "root".
#
#Default:
# cache_mgr root
cache_mgr a...@aaa.com
cachemgr_passwd aaa all
#Recommended minimum configuration:
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl cacheadmin src 88.xxx.xxx.xx9/255.255.255.255 #external IP address?
acl to_localhost dst 127.0.0.0/8
# Only allow cachemgr access from localhost
http_access allow ncsa_users
http_access allow manager localhost
http_access allow manager cacheadmin
http_access deny manager

However, whenever I enter the password and select localhost port 8080 from the 
cgi script I get:
The following error was encountered:
Cache Access Denied.
Sorry, you are not currently allowed to request:
cache_object://localhost/
from this cache until you have authenticated yourself.
  
_
Do you have a story that started on Hotmail? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] Wrong error page showing in browser

[J. Webster]

out of interest, when you issue a
squid -k reconfigure
does this disconnect any connected users?
I assume service squid restart will definitely disconnect users



> From: bodycar...@live.com
> To: contactd...@gmail.com
> CC: squid-users@squid-cache.org
> Date: Mon, 8 Feb 2010 19:25:36 +
> Subject: RE: [squid-users] Wrong error page showing in browser
>
>
> acl academic01 time MTWHF 08:00-18:00
> acl labs src 192.168.3.19-192.168.3.200
> http_access deny academic01 labs
> deny_info ERR_LAB_SCHEDULE labs
>
>
> should work. I believe last match triggers error message. You should not 
> negate academic01.
>
> http_access deny academic01 labs
>
> Means:
>
> If the time is: MTWHF 08:00-18:00
>
> AND
>
> The IP is: 192.168.3.19-192.168.3.200
>
> DENY IT.
>
>
> squid -k reconfigure is sufficient.
>
> J
>
>
>> Date: Mon, 8 Feb 2010 18:46:46 +0100
>> From: contactd...@gmail.com
>> To: squid-users@squid-cache.org
>> Subject: [squid-users] Wrong error page showing in browser
>>
>> Hi all
>>
>> I'm trying to deny proxy access for a block of IPs during a certain time
>> interval.
>> During that time interval when someone from that block tries to access
>> the internet
>> they should get a customized error msg in their browser: ERR_LAB_SCHEDULE.
>>
>> acl academic01 time MTWHF 08:00-18:00
>> acl labs src 192.168.3.19-192.168.3.200
>> http_access deny labs !academic01
>> deny_info ERR_LAB_SCHEDULE labs
>>
>> However, they keep getting the default error msg, and not the customized
>> one.
>> What am I missing here?
>>
>> Thanks
>>
>> Dayo
> _
> Hotmail: Free, trusted and rich email service.
> http://clk.atdmt.com/GBL/go/201469228/direct/01/
  
_
We want to hear all your funny, exciting and crazy Hotmail stories. Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

RE: [squid-users] Wrong error page showing in browser

[J. Webster]

Did you restart the squid server?
I know when I tried this once before, I had to restart for it to pick up the 
custom error pages - maybe they are loaded into a cache at startup?


> Date: Mon, 8 Feb 2010 18:46:46 +0100
> From: contactd...@gmail.com
> To: squid-users@squid-cache.org
> Subject: [squid-users] Wrong error page showing in browser
>
> Hi all
>
> I'm trying to deny proxy access for a block of IPs during a certain time
> interval.
> During that time interval when someone from that block tries to access
> the internet
> they should get a customized error msg in their browser: ERR_LAB_SCHEDULE.
>
> acl academic01 time MTWHF 08:00-18:00
> acl labs src 192.168.3.19-192.168.3.200
> http_access deny labs !academic01
> deny_info ERR_LAB_SCHEDULE labs
>
> However, they keep getting the default error msg, and not the customized
> one.
> What am I missing here?
>
> Thanks
>
> Dayo
  
_
Tell us your greatest, weirdest and funniest Hotmail stories
http://clk.atdmt.com/UKM/go/195013117/direct/01/