date:20081017

*
This message has been scanned by IMSS NIT-Silchar

Dear All Squid users,



The output of the command df in my proxy server is as follows:-

[EMAIL PROTECTED] squid]# df
Filesystem   1K-blocks  Used Available Use% Mounted on
/dev/mapper/PrimaryVol-root
  19838052   6579328  12234724  35% /
/dev/mapper/PrimaryVol-var
  14855176  12993112   1095296  93% /var
/dev/mapper/PrimaryVol-home
  34756272  29035932   3926292  89% /home
/dev/mapper/PrimaryVol-tmp
   4062912139748   3713452   4% /tmp
/dev/sda1   101086 22511 73356  24% /boot
tmpfs   25342412253412   1% /dev/shm





Now the thing is that earlier I used to delete the files generated on
time-t-time basis under /var/spool/squid directory to create more space
for the squid server to run.

But now, there are no core.* files generated in /var/spool/squid
directory, so what should i do to create more space under /var partition,
as of now it shows up as 93%.

At the most the squid server would run for another day and then stop
running!!

Please suggest some pointers to delete some files under /var partition to
create more space !!

Thanks in advance,

jmaan

FW: [squid-users] FW: Load balanced cache-server

2008-10-17 Thread Battsetseg.M

Hi,

Please help me out calculating the hardware requirements for below situation

Best Regards,

Battsetseg. M

Ph: 318115-0554
Mobicom Corporation

-Original Message-
From: Battsetseg.M [mailto:[EMAIL PROTECTED] 
Sent: Friday, October 17, 2008 2:07 PM
To: squid-users@squid-cache.org
Subject: [squid-users] FW: Load balanced cache-server

Hi all,

Were medium sized ISP and want to implement caching. 
After doing some research, we decided to use load balancer with 4-5 servers.
Our exiting gateway bandwidth is totally 300Mbps. After doing some
calculations included in OReilly Web Caching  we got to have a  2800GB of
cache-dir and a huge amount of memory. The question is did I miscalculate?
Or anyone has a better suggestion? 

The calculation specifics are:

Bandwidth 300Mbps
Total 80% of the bandwidth is HTTP
Cache-hit: 40%
Cache-miss: 60%
Non-cacheable objects: 20%


Any help is appreciated.


Best Regards,

Battsetseg. M

Ph: 318115-0554
Mobicom Corporation

Re: [squid-users] Disk Space problem in a squid-proxy server

2008-10-17 Thread Angela Williams

On Friday 17 October 2008, [EMAIL PROTECTED] wrote:
 *

 The output of the command df in my proxy server is as follows:-

 [EMAIL PROTECTED] squid]# df
 Filesystem   1K-blocks  Used Available Use% Mounted on
 /dev/mapper/PrimaryVol-root
   19838052   6579328  12234724  35% /
 /dev/mapper/PrimaryVol-var
   14855176  12993112   1095296  93% /var
 /dev/mapper/PrimaryVol-home
   34756272  29035932   3926292  89% /home
 /dev/mapper/PrimaryVol-tmp
4062912139748   3713452   4% /tmp
 /dev/sda1   101086 22511 73356  24% /boot
 tmpfs   25342412253412   1% /dev/shm





 Now the thing is that earlier I used to delete the files generated on
 time-t-time basis under /var/spool/squid directory to create more space
 for the squid server to run.

 But now, there are no core.* files generated in /var/spool/squid
 directory, so what should i do to create more space under /var partition,
 as of now it shows up as 93%.

 At the most the squid server would run for another day and then stop
 running!!

 Please suggest some pointers to delete some files under /var partition to
 create more space !!

Log files in /var/log? What about the squid store log which you do not need! 
Update your squid.conf to make the store log none

Have a look at your squid.conf file and look for this line
cache_dir

My one I have configured like this.
cache_dir aufs /data/squid 5000 15 256
To give 5G in /data/squid

If you change it you will need to trash the current cache and create new with 
squid -z

Cheers
Ang




-- 
Angela Williams Enterprise Outsourcing
Unix/Linux  Cisco spoken here! Bedfordview
[EMAIL PROTECTED]   Gauteng South Africa

Smile!! Jesus Loves You!!

Re: [squid-users] Header Stripping of Header type other

On fre, 2008-10-17 at 06:09 +0200, WRIGHT Alan [UK] wrote:
 I could use ACL with request_header_access other deny, but this will
 strip some other headers too which is not possible.

You should be able to use any header name in request_header_access. If
not file a bug report.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part

Re: [squid-users] Update Accelerator, Squid and Windows Update Caching

On fre, 2008-10-17 at 06:06 +0100, Richard Wall wrote:

 but I don't see anything evil in the server response headers
 today. I guess the client may be sending no-cache headers...I'll
 double check that later.
 
 Is there some other case that I'm missing?

I think the missing partial object cache is the main culpit for windows
update caching today.

Another minor culpit is that sometimes SSL is used. But I think this is
only some metadata requests.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part

Re: [squid-users] squidnt.com, warning

On fre, 2008-10-17 at 14:40 +1300, Amos Jeffries wrote:

  I have added a warning comment on their download page.

 Which appears to have been moderated out of existence.
 At least the three comments now present are all by 'admin' advertising
 their downloads.

Suspected this would happen. Oh well. Now we at least know for sure they
are hostile. For all we know that Squid download may well be a trojan.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part

Re: [squid-users] FW: Load balanced cache-server


Battsetseg.M wrote:

Hi all,

We’re medium sized ISP and want to implement caching. 
After doing some research, we decided to use load balancer with 4-5 servers.

Our exiting gateway bandwidth is totally 300Mbps. After doing some
calculations included in O’Reilly ‘Web Caching ‘ we got to have a  2800GB of
cache-dir and a huge amount of memory. The question is did I miscalculate?
Or anyone has a better suggestion? 


The calculation specifics are:

Bandwidth 300Mbps
Total 80% of the bandwidth is HTTP
Cache-hit: 40%
Cache-miss: 60%
Non-cacheable objects: 20%


Any help is appreciated.


What calculations?  I don't think it quite works that way.

An estimated conservative HIT rate would be around 30% and it varies.
The non-cacheables are more likely to be the same as MISS rate. Or they 
would become HITs themselves under peak loads.


But, if you have no caching now the HIT savings will be a net gain even 
if its only 1% using a 10MB cache. Which brings the good news that any 
cache size you can afford can be useful.


The approach I'd recommend is to spec out what you can afford and expand 
it over time. That includes starting by only caching a small segment of 
the client base to see how many a single squid (or pair for redundancy) 
can handle it.


Amos
--
Please use Squid 2.7.STABLE4 or 3.0.STABLE10

Re: [squid-users] Update Accelerator, Squid and Windows Update Caching

Richard Wall wrote:

On Fri, Oct 10, 2008 at 12:30 PM, Amos Jeffries [EMAIL PROTECTED] wrote:

Richard Wall wrote:

Hi,

I've been reading through the archive looking for information about
squid 2.6 and windows update caching. The FAQ mentions problems with
range offsets but it's not really clear which versions of Squid this
applies to.

All versions. The FAQ was the result of my experiments mid last year. With
some tweaks made early his year since Vista came out.
We haven't done a intensive experiments with Vista yet.

Hi Amos,

I'm still investigating Windows Update caching (with 2.6.STABLE17/18)

First of all, I have been doing some tests to try and find out the
problem with Squid and Content-Range requests.
* I watch the squid logs as a vista box does its automatic updates
and I can see that *some* of its requests use ranges. (so far I have
only seen these when it requests .psf files...some of which seem to be
very large files...so the range request makes sense) See:
http://groups.google.hr/group/microsoft.public.windowsupdate/browse_thread/thread/af5db07dc2db9713

# zcat squid.log.192.168.1.119.2008-10-16.gz | grep
multipart/byteranges | awk '{print $7}' | uniq | while read URL; do
echo $URL; wget --spider $URL 21 | grep Length; done
http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/10/windows6.0-kb956390-x86_2d03c4b14b5bad88510380c14acd2bffc26436a7.psf
Length: 91,225,471 (87M) [application/octet-stream]
http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/05/windows6.0-kb950762-x86_0cc2989b92bc968e143e1eeae8817f08907fd715.psf
Length: 834,868 (815K) [application/octet-stream]
http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/03/windows6.0-kb948590-x86_ed27763e42ee2e20e676d9f6aa13f18b84d7bc96.psf
Length: 755,232 (738K) [application/octet-stream]
http://www.download.windowsupdate.com/msdownload/update/software/crup/2008/09/windows6.0-kb955302-x86_1e40fd3ae8f95723dbd76f837ba096adb25f3829.psf
Length: 7,003,447 (6.7M) [application/octet-stream]
...

* I have found that curl can make range requests so I've been using
it to test how Squid behavesand it seems to do the right thing. eg
- First ask for a range : The correct range is returned X-Cache: MISS
- Repeat the range request : The correct range is returned X-Cache: MISS
- Request the entire file: The entire file is correctly returned X-Cache: MISS
- Repeat the request: X-Cache: HIT
- Repeat the previous range request: X-Cache: HIT
- Request a different range: X-Cache: HIT

curl --range 1000-1002 --header Pragma: -v -x http://127.0.0.1:3128
http://www.download.windowsupdate.com/msdownload/update/software/secu/2008/05/windows6.0-kb950762-x86_0cc2989b92bc968e143e1eeae8817f08907fd715.psf

/dev/null

Looking back through the archive I find this conversation from 2005:
http://www.squid-cache.org/mail-archive/squid-users/200504/0669.html

...but the behaviour there sounds like a result of setting:
range_offset_limit -1

Seems to me that Squid should do a good job of Windows Update caching.
There is another thread discussing how to override MS update cache
control headers:
http://www.squid-cache.org/mail-archive/squid-users/200508/0596.html

but I don't see anything evil in the server response headers
today. I guess the client may be sending no-cache headers...I'll
double check that later.

Is there some other case that I'm missing?

As I said. I have not seen Vista in detail. I just had to turn off my
old hack to get around the SP1 hanging. (that huge .psf perhapse?)

Never had to do anything with headers.

When I did my testing it was with outdated Win98-WinXP machines (often
needing SP1 in XP's case).

The WU on them made an HTTPS request (seems to be auth-related even
today) requested one or more update indexes fine. Then proceeded to
random-access range requests out of the middle of the update *.cabs
using dynamic urls at various update sites.
This was causing bandwidth blowout with all the MISS'es when I had
several machines a week coming through.

I _think_ but have no confirmation, that the early patch-tuesday
releases were done as large single .CAB files and a particular machine
may only need updating from individual fixes inside them.

As your test showed, fetching the whole file squid can handle the ranges
fine. It's when they are still in MISS state that ranges become trouble.

I'm going to experiment, but if anyone has any positive or
negative experience of Squid and windows update caching, I'd be really
interested to hear from you.

In case Squid cannot do windows update caching by its self, I'm also
looking at integrating Update Accelerator
(http://update-accelerator.advproxy.net/) script with standard squid
2.6 and wondered if anyone else had any experience of this.
The update accelerator script is just a perl wrapper around wget which
is configured as a Squid url_rewrite_program. It's not clear to me
what this script is doing that Squid

[squid-users] Time on squid

2008-10-17 Thread netmail

Hi
When squid generate the message when block an website , the time show is
different respect the linux time ..for example:
The system display : 17.05
Squid : 14.05

How to for sync the time ?
Thanks in advance

Re: [squid-users] Time on squid

2008-10-17 Thread John Doe

GMT time?
Maybe check your logformat (%tg or %tl).

JD




- Original Message 
 From: netmail [EMAIL PROTECTED]
 To: squid-users@squid-cache.org
 Sent: Friday, October 17, 2008 4:37:53 PM
 Subject: [squid-users] Time on squid
 
 Hi
 When squid generate the message when block an website , the time show is
 different respect the linux time ..for example:
 The system display : 17.05
 Squid : 14.05
 
 How to for sync the time ?
 Thanks in advance  


__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

Re: [squid-users] newbie: configuring squid to always check w/origin server

2008-10-17 Thread dukehoops

Henrik,

Thanks for a prompt response. Unfortunatley, seems like we're still missing
something:

The origin server is including

CacheControl: max-age=0
ETag: etag-value

in it's response.

The problems are

1) Squid is not sending

If-None-Match: etag-value

in subsequent requests (unless the browser includes this in it's request in
which case squid just passes it on)

2) When the origin server return 302, squid just passes 302 back to the
browser rather than serving up its cached copy of the image.

Any further tips?
thanks
-nikita

Henrik Nordstrom-5 wrote:

On tor, 2008-10-16 at 16:12 -0700, dukehoops wrote:
1. With what headers should the origin server respond in 3a) and 3b)? In
latter case, it seems like something like Cache-Control:
must-revalidate,
not sure whether to use s-maxage=0 and/or maxage=0

You probably do not need or want must-revalidate, it's a quite hard
directive. max-age is sufficient I think.

You only need must-revalidate (in addition to max-age) if it's
absolutely forbidden to use the last known version when/if revalidation
should fails to contact the web server for some reason.

You only need s-maxage if you want to assign different cache criterias
to shared caches such as Squid and browsers, for example enabling
browsers to cache the image longer than Squid.

2. What params should be used in squid config?

Preferably nothing specific for this since you have control over the web
server..

REgards
Henrik

Nikita Tovstoles
vside.com

--
View this message in context:
http://www.nabble.com/newbie%3A-configuring-squid-to-always-check-w-origin-server-tp20018895p20037231.html
Sent from the Squid - Users mailing list archive at Nabble.com.

Re: [squid-users] Disk Space problem in a squid-proxy server

*
This message has been scanned by IMSS NIT-Silchar



Please see below the output of pwd and df commands:-

[EMAIL PROTECTED] squid]# pwd
/var/log/squid
[EMAIL PROTECTED] squid]# ls -l
total 1005212
-rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log
drwxr-xr-x 2 root  root   4096 2008-10-17 00:49 Backup
-rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log
-rw-r--r-- 1 root  root 135898 2008-10-17 18:27 squid.out
-rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log
[EMAIL PROTECTED] squid]#


In my proxyserver, the access.log is under the path /var/log/squid.


Now please tell me do i need to keep the cache.log file. Can i delete this
file to create more space?

Also what can i do delete/ squeeze the  access.log file ?


I have looked into the squid.conf file, and there is line like this below.

cache_dir ufs /home/squid 30720 16 256

What should I do or can do to reduce the size of the /var partition in my
proxy-server ?

The output if the df command shows up utilization of 94% of the hard disk.

[EMAIL PROTECTED] squid]# df
Filesystem   1K-blocks  Used Available Use% Mounted on
/dev/mapper/PrimaryVol-root
  19838052   6579304  12234748  35% /
/dev/mapper/PrimaryVol-var
  14855176  13145712942696  94% /var
/dev/mapper/PrimaryVol-home
  34756272  29043404   3918820  89% /home
/dev/mapper/PrimaryVol-tmp
   4062912139716   3713484   4% /tmp
/dev/sda1   101086 22511 73356  24% /boot
tmpfs   253424 0253424   0% /dev/shm
[EMAIL PROTECTED] squid]#



Thanks,

jmaan

Re: [squid-users] Disk Space problem in a squid-proxy server

*
This message has been scanned by IMSS NIT-Silchar



Please see below the output of pwd and df commands:-

[EMAIL PROTECTED] squid]# pwd
/var/log/squid
[EMAIL PROTECTED] squid]# ls -l
total 1005212
-rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log
drwxr-xr-x 2 root  root   4096 2008-10-17 00:49 Backup
-rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log
-rw-r--r-- 1 root  root 135898 2008-10-17 18:27 squid.out
-rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log
[EMAIL PROTECTED] squid]#


In my proxyserver, the access.log is under the path /var/log/squid.


Now please tell me do i need to keep the cache.log file. Can i delete this
file to create more space?

Also what can i do delete/ squeeze the  access.log file ?


I have looked into the squid.conf file, and there is line like this below.

cache_dir ufs /home/squid 30720 16 256

What should I do or can do to reduce the size of the /var partition in my
proxy-server ?

The output if the df command shows up utilization of 94% of the hard disk.

[EMAIL PROTECTED] squid]# df
Filesystem   1K-blocks  Used Available Use% Mounted on
/dev/mapper/PrimaryVol-root
  19838052   6579304  12234748  35% /
/dev/mapper/PrimaryVol-var
  14855176  13145712942696  94% /var
/dev/mapper/PrimaryVol-home
  34756272  29043404   3918820  89% /home
/dev/mapper/PrimaryVol-tmp
   4062912139716   3713484   4% /tmp
/dev/sda1   101086 22511 73356  24% /boot
tmpfs   253424 0253424   0% /dev/shm
[EMAIL PROTECTED] squid]#



Thanks,

jmaan

Re: [squid-users] newbie: configuring squid to always check w/origin server

On fre, 2008-10-17 at 10:01 -0700, dukehoops wrote:

 Thanks for a prompt response. Unfortunatley, seems like we're still missing
 something:
  
 The origin server is including
  
 CacheControl: max-age=0
 ETag: etag-value
  
 in it's response.
  
 The problems are 
  
 1) Squid is not sending 
  
 If-None-Match: etag-value

Which Squid version?
 
 2) When the origin server return 302, squid just passes 302 back to the
 browser rather than serving up its cached copy of the image.

302 is redirects.. did you mean 304?

Regards
Henrik


signature.asc
Description: This is a digitally signed message part

Re: [squid-users] newbie: configuring squid to always check w/origin server

2008-10-17 Thread alexgit


Hi Henrik,
We use squid 3.0.STABLE8.
And yes, you are right, the status we send from origin server in 304 not
302.
We tried responses with must-revalidate and without and see no difference.

Thanks a lot,
Alex

P.S. Nikita and I both work on the same project.


Henrik Nordstrom-5 wrote:
 
 On fre, 2008-10-17 at 10:01 -0700, dukehoops wrote:
 
 Thanks for a prompt response. Unfortunatley, seems like we're still
 missing
 something:
  
 The origin server is including
  
 CacheControl: max-age=0
 ETag: etag-value
  
 in it's response.
  
 The problems are 
  
 1) Squid is not sending 
  
 If-None-Match: etag-value
 
 Which Squid version?
  
 2) When the origin server return 302, squid just passes 302 back to the
 browser rather than serving up its cached copy of the image.
 
 302 is redirects.. did you mean 304?
 
 Regards
 Henrik
 
  
 

-- 
View this message in context: 
http://www.nabble.com/newbie%3A-configuring-squid-to-always-check-w-origin-server-tp20018895p20038822.html
Sent from the Squid - Users mailing list archive at Nabble.com.

Re: [squid-users] Disk Space problem in a squid-proxy server

*
This message has been scanned by IMSS NIT-Silchar

Please see below the output of pwd and df commands:-

[EMAIL PROTECTED] squid]# pwd
/var/log/squid
[EMAIL PROTECTED] squid]# ls -l
total 1005212
-rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log
drwxr-xr-x 2 root  root   4096 2008-10-17 00:49 Backup
-rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log
-rw-r--r-- 1 root  root 135898 2008-10-17 18:27 squid.out
-rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log
[EMAIL PROTECTED] squid]#


In my proxyserver, the access.log is under the path /var/log/squid.


Now please tell me do i need to keep the cache.log file. Can i delete this
file to create more space?

Also what can i do delete/ squeeze the  access.log file ?


I have looked into the squid.conf file, and there is line like this below.

cache_dir ufs /home/squid 30720 16 256

What should I do or can do to reduce the size of the /var partition in my
proxy-server ?

The output if the df command shows up utilization of 94% of the hard disk.

[EMAIL PROTECTED] squid]# df
Filesystem   1K-blocks  Used Available Use% Mounted on
/dev/mapper/PrimaryVol-root
  19838052   6579304  12234748  35% /
/dev/mapper/PrimaryVol-var
  14855176  13145712942696  94% /var
/dev/mapper/PrimaryVol-home
  34756272  29043404   3918820  89% /home
/dev/mapper/PrimaryVol-tmp
   4062912139716   3713484   4% /tmp
/dev/sda1   101086 22511 73356  24% /boot
tmpfs   253424 0253424   0% /dev/shm
[EMAIL PROTECTED] squid]#



Thanks,

jmaan

Re: [squid-users] Disk Space problem in a squid-proxy server

2008-10-17 Thread Francois Cami

squid -k rotate
will rotate all .log files for you, you can delete the *.0 files afterwards.

Never delete files without knowing what you are doing ; deleting files
from under squid's nose will lead to unpredictable behaviour :)

You really should read the manuals, and especially the parts related to
logs files and cache_dir entries, to understand what you are doing.

Francois


On Sat, Oct 18, 2008 at 4:52 AM,  [EMAIL PROTECTED] wrote:
 *
 This message has been scanned by IMSS NIT-Silchar



 Please see below the output of pwd and df commands:-

 [EMAIL PROTECTED] squid]# pwd
 /var/log/squid
 [EMAIL PROTECTED] squid]# ls -l
 total 1005212
 -rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log
 drwxr-xr-x 2 root  root   4096 2008-10-17 00:49 Backup
 -rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log
 -rw-r--r-- 1 root  root 135898 2008-10-17 18:27 squid.out
 -rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log
 [EMAIL PROTECTED] squid]#


 In my proxyserver, the access.log is under the path /var/log/squid.


 Now please tell me do i need to keep the cache.log file. Can i delete this
 file to create more space?

 Also what can i do delete/ squeeze the  access.log file ?


 I have looked into the squid.conf file, and there is line like this below.

 cache_dir ufs /home/squid 30720 16 256

 What should I do or can do to reduce the size of the /var partition in my
 proxy-server ?

 The output if the df command shows up utilization of 94% of the hard disk.

 [EMAIL PROTECTED] squid]# df
 Filesystem   1K-blocks  Used Available Use% Mounted on
 /dev/mapper/PrimaryVol-root
  19838052   6579304  12234748  35% /
 /dev/mapper/PrimaryVol-var
  14855176  13145712942696  94% /var
 /dev/mapper/PrimaryVol-home
  34756272  29043404   3918820  89% /home
 /dev/mapper/PrimaryVol-tmp
   4062912139716   3713484   4% /tmp
 /dev/sda1   101086 22511 73356  24% /boot
 tmpfs   253424 0253424   0% /dev/shm
 [EMAIL PROTECTED] squid]#



 Thanks,

 jmaan

[squid-users] Proxyservr Disk Space Problem

*
This message has been scanned by IMSS NIT-Silchar



May I know under which directory should I execute te command as given by you.

Actually, rotation is being done for the log files but still, the
access.log file generated at times are too big on daily basis.





squid -k rotate
will rotate all .log files for you, you can delete the *.0 files afterwards.

Never delete files without knowing what you are doing ; deleting files
from under squid's nose will lead to unpredictable behaviour :)

You really should read the manuals, and especially the parts related to
logs files and cache_dir entries, to understand what you are doing.

Francois


Thanks,

jmaan

[squid-users] Re: Proxyservr Disk Space Problem

2008-10-17 Thread Francois Cami

You can execute that anywhere if squid is in $PATH.

To solve your log file is too big problem, you could rotate
the logs every hour with cron and destroy the *.0 files
belonging to squid. But that's bandaid, not a real solution.


On Sat, Oct 18, 2008 at 7:24 AM,  [EMAIL PROTECTED] wrote:
 *
 This message has been scanned by IMSS NIT-Silchar



 May I know under which directory should I execute te command as given by you.

 Actually, rotation is being done for the log files but still, the
 access.log file generated at times are too big on daily basis.





 squid -k rotate
 will rotate all .log files for you, you can delete the *.0 files afterwards.

 Never delete files without knowing what you are doing ; deleting files
 from under squid's nose will lead to unpredictable behaviour :)

 You really should read the manuals, and especially the parts related to
 logs files and cache_dir entries, to understand what you are doing.

 Francois


 Thanks,

 jmaan

Re: [squid-users] Using Squid as a reverse-proxy to SSL origin?

On tor, 2008-10-16 at 10:56 -0400, Todd Lainhart wrote:

 Could I do the same thing with SSL to the reverse proxy?  That is, the
 reverse proxy is the endpoint for the client, gets the creds, becomes
 the endpoint for the server, decrypts and caches the origin response,
 and then serves cached content encrypted back to the client?

Yes.

 I would
 guess this falls into man-in-the-middle style ugliness, is
 out-of-bounds for SSL and so wouldn't be supported.  But then again I
 was wrong about my original use-case not being supported :-) .

It's supported, and not a man-in-the-middle attack as the reverse proxy
is the administrative endpoint, and as far as the user is concerned is
the authoriative server. The fact that this web server happens to use
HTTP (or HTTP over SSL) to fetch it's content is an implementation
detail.

You'll need a valid certificate on the reverse proxy. The certifiate on
the actual web server may be self-signed or by an internal CA, not
visible to the end-user, only the reverse proxy.


There is one notable limitation however, and that is that the origin
server can not request SSL client certifiacates from the end-user.
because the SSL is terminated at the reverse proxy there is no SSL
between web server and end-user. The proxy can request client
certificates, and may also relay details about the user provided
certificate (not sure such relaying is implemented by Squid yet). The
proxy can also present it's own client certificate to the web server
provint authenticity that it's really a trusted reverse proxy.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part

Re: [squid-users] Complicate ACL affect performance?

On tor, 2008-10-16 at 12:02 +0300, Henrik K wrote:

 Optimizing 1000 x www.foo.bar/randomstuff into a _single_
 www.foobar.com/(r(egex|and(om)?)|fuba[rz]) regex is nowhere near linear.
 Even if it's all random servers, there are only ~30 characters from which
 branches are created from.

Right. 

Would be interesting to see how 50K dstdomain compares to 50k host
patterns merged into a single dstdomain_regex pattern in terms of CPU
usage. Probably a little tweaking of Squid is needed to support such
large patterns, but that's trivial. (squid.conf parser is limited to
4096 characters per line, including folding)

Regards
Henrik


signature.asc
Description: This is a digitally signed message part

Re: [squid-users] Disk Space problem in a squid-proxy server

2008-10-17 Thread Chuck Kollars

  ... At the most the squid server would run for another day and then 
 stop running!! ... Please suggest some pointers to delete some 
 files under /var partition to create more space !! ...

Both your question and all the responses to it I've seen assume Squid is the 
cause of this problem, in other words that Squid is _using_ the missing disk 
space. Although this might be the case, it's not the only possibility. I wonder 
if something else is using the missing disk space, and Squid is just the victim?

(I fear the actions you've taken in the past, deleting various files such as 
core.* and some Squid files, may have been just sticking bandaids on a broken 
leg. They may have bought you a few more months, but without ever addressing 
the real problem.)

When I have a partition that runs short of disk space, the first thing I find 
out is WHO. To do that, I work my way down the directory hierarchy to localize 
the problem to one particular folder, then look closely at all the files in 
that folder to see who created them and when. Even with its obvious holes, this 
simplistic procedure works pretty well in practice almost all the time. Here's 
an example:

cd /var   # assume for this example /var is the problem partition
du -s * | sort -n
# bottommost line is the suspect, let's suppose for example it's 'log'
cd log
du -s * | sort -n   # do it again
# bottommost line is the suspect, let's suppose for example it's 'cups'
cd cups
du -s * | sort -n   # do it again
# stop, as no more sub-folders indicating we're at the bottom
ls -l *
# look at ownership and timestamps on files, imagine how they got this way

The max size of the Squid cache is specified in squid.conf. When it gets full, 
Squid will throw out the oldest files to make room for the new ones, so it will 
keep running all by itself. You should never have to explicitly delete any 
files out of the Squid cache. 

You should make sure all the disk space that squid.conf says might be used is 
actually available (and continues to be available:-). To prevent the problem of 
something else sucking up all the disk space that was supposed to be for Squid, 
I put my Squid cache on a separate partition (not under /var at all) that 
doesn't contain anything else. (As an aside, using a separate partition also 
allows me to improve performance by easily using the 'noatime' mount parameter.)

thanks! -Chuck Kollars

__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

Re: [squid-users] squidnt.com, warning

2008-10-17 Thread Tim Bates


Henrik Nordstrom wrote:

Now we at least know for sure they are hostile.
For all we know that Squid download may well be a trojan.
Did anyone else notice that the site helpfully provides an MD5 hash to 
check, and even provide a tool to check it with? Leaves me wondering if 
both are trojans.


TB

Re: [squid-users] Disk Space problem in a squid-proxy server

2008-10-17 Thread Chris Robertson


Angela Williams wrote:

Have a look at your squid.conf file and look for this line
cache_dir

My one I have configured like this.
cache_dir aufs /data/squid 5000 15 256
To give 5G in /data/squid

If you change it you will need to trash the current cache and create new with 
squid -z
  


This statement is only true if you change the number of directories, 
and/or sub-directories (the 15 and 256 in the example given).  Changing 
the size of the cache (even shrinking it) only requires a squid -k 
reconfigure.  If the used disk space is greater than the allocation, 
objects will be purged until the size on disk is below the high water mark.



Cheers
Ang
  


Chris

Re: [squid-users] Authentication Issue with Squid and mixed BASIC/NTLM auth



Um, something weird is going on. I'm a little scared by the double sets 
of bad news.
Can you confirm that your in-use systems are okay. I haven't led you to 
a point where anything serious is broken? (ie this is all isolated on a 
test machine where its okay to break?)



Chris Natter wrote:

Hmmm, strange. I tested 2.7STABLE4, but it doesn't seem to be stripping
the DOMAIN, it will still accept only DOMAIN\USERNAME. Perhaps I'm
missing something?


I've looked at it closer. And the patches which I saw earlier were for a 
slightly different helper (mapping NTLM front-end auth to LDAP backend)


Henrik informs me that NTLM always needs the domain. Which makes me 
wonder why you didn't in 3.0.




I also tested squid-3.1-20081016, built with a spec file adopted from a
squid3.0STABLE7 Redhat package:

configure \
   --exec_prefix=/usr \
   --bindir=%{_sbindir} \
   --libexecdir=%{_libdir}/squid \
   --localstatedir=/var \
   --datadir=%{_datadir} \
   --sysconfdir=/etc/squid \
   --disable-dependency-tracking \
   --enable-arp-acl \
   --enable-auth=basic,digest,ntlm,negotiate \
 
--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-do

main-NTLM,SASL \
   --enable-cache-digests \
   --enable-cachemgr-hostname=localhost \
   --enable-delay-pools \
   --enable-digest-auth-helpers=password \
   --enable-epoll \
 
--enable-external-acl-helpers=ip_user,ldap_group,unix_group,wbinfo_grou

p \
   --enable-icap-client \
   --enable-ident-lookups \
   --enable-linux-netfilter \
   --enable-ntlm-auth-helpers=SMB,fakeauth \
   --enable-referer-log \
   --enable-removal-policies=heap,lru \
   --enable-snmp \
   --enable-ssl \
   --enable-storeio=aufs,coss,diskd,,ufs \
   --enable-useragent-log \
   --enable-wccpv2 \
   --with-default-user=squid \
   --with-filedescriptors=16384 \
   --with-dl \
   --with-openssl=/usr/kerberos \
   --with-pthreads

And it looks like NTLM could be broken (I don't want to make
assumptions). I was unable to pass credentials in either the
DOMAIN\USERNAME or USERNAME format to OWA through squid. It also forced
an NTLM prompt for Firefox that I had to escape out of before I could
authenticate with BASIC auth.

I wasn't able to test spell-check as I couldn't authenticate to the OWA
server.


That is a worry for us. Thanks for testing and finding the issue.
This is the first bug report on connection pinning.
for our info: did you have the login=PASS on the cache_peer line? and 
woudld you mind sharing the config?


Amos




Thanks!
-Chris
-Original Message-
From: Amos Jeffries [mailto:[EMAIL PROTECTED] 
Sent: Thursday, October 16, 2008 5:37 AM

To: Chris Natter
Cc: squid-users@squid-cache.org
Subject: Re: [squid-users] Authentication Issue with Squid and mixed
BASIC/NTLM auth

Chris Natter wrote:

We were having issues with spell-check in 3.0, I haven't tried any of
the development builds to see if it was resolved though in a later
release. 

 

OWA spell-check just seems to hang when you attempt to spell-check an
email, or gives the try again later prompt. I saw some previous
postings on the archive of the mailing list, but most of them are very
outdated.

I'll have to build an RPM of squid 2.7 and check to see if that solves
both issues.


Ah, now that you mention it I vaguely recall the topic as it flew past a

while back.

Yes, 2.7 is likely the most dependable to have both combos of fixes you 
need.


Without knowing the cause the spellcheck issue _may_ have been resolved 
in 3.1.  Both of the MS workarounds and 'unknown method' support are now


present. If you have a spare moment and are inclined to test it please 
let us know the result. If you still hit bad news for 3.1, its 
definitely a bug that needs looking into at some point.


Amos


Thanks for the help.

-Original Message-
From: Amos Jeffries [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, October 15, 2008 6:46 PM

To: Chris Natter
Cc: squid-users@squid-cache.org
Subject: Re: [squid-users] Authentication Issue with Squid and mixed
BASIC/NTLM auth


Hey all,



I've got a tough situation I'm hoping someone can help me with.



We 'downgraded' from an old 3.0PRE build that a predecessor had setup

on a

reverse proxy, to squid 2.6.STABLE20. The proxy runs your standard

OWA

over Reverse Proxy setup, with login=PASS to an OWA backend running

with

BASIC/NTLM auth. We have to have the NTLM for phones that sync with
ActiveSync.



It seems like something fundamental has changed in the way squid

handles

auth from 3.0 to squid 2.6. Using firefox on 2.6, I can auth with

just

'USERNAME', with IE on 2.6 we have to type DOMAINUSERNAME or
[EMAIL PROTECTED] now. Previously, with squid 3.0, just 'USERNAME' would

work

for auth.



While this seems trivial, anything harder than just 'USERNAME'

boggles

a

lot of users. I'm assuming this has something to do with 'attempting

NTLM'

negotiation? Is there a way around it in squid 2.6?


The cleaner @DOMAIN handling was only added to Squid 2.7+ and 3.0+.

You

Re: [squid-users] Disk Space problem in a squid-proxy server