Re: [squid-users] yahoo mail problem with tproxy (squid 3.1.19, kernel 3.2.21)

2012-07-23 Thread Amos Jeffries 

On 24/07/2012 4:53 p.m., Ming-Ching Tiew wrote:


- Original Message -
From: Amos Jeffries 
To: squid-users@squid-cache.org


One big change in 3.2.0.14 related to TPROXY traffic handling. A bug in 
host_strict_verify was fixed, making the validation > bypass properly when the 
(default) non-strict was configured.

- check that this host_strict_verify directive is ABSENT from your config file, 
or at very least set to OFF.

There is not such directive in my config file.


- check your cache.log for host forgery security alerts, or forwarding loop 
warnings when these requests are being made.

- check your cache.log file for invalid request parsing messages. This may require 
"debug_options ALL,1" to be configured.

The cache.log has these :-

2012/07/24 12:38:34.628| SECURITY ALERT: Host header forgery detected on 
local=219.93.13.235:80 remote=192.168.1.3 FD 13 flags=17 (local IP does not 
match any domain IP)
2012/07/24 12:38:34.628| SECURITY ALERT: By user agent: Mozilla/4.0 
(compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; (R1 1.6); .NET CLR 
3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727)
2012/07/24 12:38:34.628| SECURITY ALERT: on URL: 
http://us.mg6.mail.yahoo.com/neo/launch?.rand=5fsn8p9a1efna

What is the significance ? Is it that my test client machine is infected by 
virus adware or what ?



The HTTP Host: header contains a domain name which does not match the IP 
address the TCP connection is being made to. 
http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery covers the 
problem in some detail. For your case in particular I suspect the DNS 
situations need to be checked.


219.93.13.235 found by the client is not one of the IPs belonging to 
us.mg6.mail.yahoo.com which DNS is supplying to Squid. On the "big name" 
websites this is usually caused by Geo-DNS resolution problems rather 
than client infection. But there is no way for Squid to be sure. The 
only option is for Squid to open a TCP connection directly to that IP 
and hope for the best, or if direct connections are blocked the unable 
to connect comes up.


NOTE: if you are using cache_peer you can currently only send them 
requests where the Host header validates as okay, or were received as 
regular forward-proxy / reverse-proxy traffic. (I'm working on that one 
as I type, but the fix is a few days/weeks away).


If you are *not* using cache_peer then you have TCP connectivity 
problems that need fixing.


You can run 3.1 series for now, or that older beta (ideally not, but if 
you *really* have to its okay for now). There are tweaks and 
improvements around this right up to the squid-3.2.0.18-20120724-r11624 
 
snapshot with more coming. With probably some of the network environment 
situations mentioned in the wiki needing to be fixed as well.


Amos

Re: [squid-users] yahoo mail problem with tproxy (squid 3.1.19, kernel 3.2.21)

2012-07-23 Thread Ming-Ching Tiew 


- Original Message -
From: Amos Jeffries 
To: squid-users@squid-cache.org

> One big change in 3.2.0.14 related to TPROXY traffic handling. A bug in 
> host_strict_verify was fixed, making the validation > bypass properly when 
> the (default) non-strict was configured.
>
> - check that this host_strict_verify directive is ABSENT from your config 
> file, or at very least set to OFF.

There is not such directive in my config file.

> 
> - check your cache.log for host forgery security alerts, or forwarding loop 
> warnings when these requests are being made.
>
> - check your cache.log file for invalid request parsing messages. This may 
> require "debug_options ALL,1" to be configured.

The cache.log has these :-

2012/07/24 12:38:34.628| SECURITY ALERT: Host header forgery detected on 
local=219.93.13.235:80 remote=192.168.1.3 FD 13 flags=17 (local IP does not 
match any domain IP)
2012/07/24 12:38:34.628| SECURITY ALERT: By user agent: Mozilla/4.0 
(compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; (R1 1.6); .NET CLR 
3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727)
2012/07/24 12:38:34.628| SECURITY ALERT: on URL: 
http://us.mg6.mail.yahoo.com/neo/launch?.rand=5fsn8p9a1efna

What is the significance ? Is it that my test client machine is infected by 
virus adware or what ?

Re: [squid-users] Include directive in 2.6?

2012-07-23 Thread Amos Jeffries 

On 24.07.2012 14:19, Baird, Josh wrote:

Hi,

Can someone confirm if the "include" directive is supported in 2.6?
I'm running squid-2.6.STABLE21-6.el5, and have "include
/etc/squid/conf.d/*.conf" in my squid.conf.  No errors are reported,
but the configuration files do not seem to actually be included.

Thanks,

Josh



Not officially. But it was added during the 2.7 series beta cycle, so 
its quite possible RHEL patched their 2.6 package to ignore it or 
possibly support the early version of it.
NP: The initial releases did not support wildcard patterns, just 
absolute file paths, and did not warn very loudly when an include file 
failed to open.



PS, Please upgrade. There are newer RHEL packages available that at 
least get you onto 2.7 series:

  http://people.redhat.com/jskala/squid/
(I wont advise 3.0 or 3.1 at this point as 3.0 is deprecated and the 
3.1 package very outdated - but if you need 3.x features its up to you)


Amos

Re: [squid-users] yahoo mail problem with tproxy (squid 3.1.19, kernel 3.2.21)

2012-07-23 Thread Amos Jeffries 

On 24.07.2012 14:20, Ming-Ching Tiew wrote:

- Original Message -
From: Ming-Ching Tiew

The test is very repeated, ie when I 'make install' from 
squid-3.2.0.12 it works but not

squid-3.2.018.


I meant the tests were very repeatable, squid-3.2.0.12 works,
squid-3.2.0.13 works.
Squid-3.2.0.14 onwards ( tested squid-3.2.0.14, squid-3.2.0.15,
squid-3.2.0.16,
squid-3.2.0.18 ) all start giving problems.

For squid-3.2.0.14, when I try to logon to yahoo mail, I get this
thing below. Other
versions seem to just hang until timeout. I am not trying to finger
point at squid
beta version, but I hope these tests will throw some lights to my
problem with
using squid in tproxy mode :-


One big change in 3.2.0.14 related to TPROXY traffic handling. A bug in 
host_strict_verify was fixed, making the validation bypass properly when 
the (default) non-strict was configured.


 - check that this host_strict_verify directive is ABSENT from your 
config file, or at very least set to OFF.


 - check your cache.log for host forgery security alerts, or forwarding 
loop warnings when these requests are being made.


 - check your cache.log file for invalid request parsing messages. This 
may require "debug_options ALL,1" to be configured.




Amos

Re: [squid-users] yahoo mail problem with tproxy (squid 3.1.19, kernel 3.2.21)

2012-07-23 Thread Ming-Ching Tiew 


- Original Message -
From: Ming-Ching Tiew 
To: "squid-users@squid-cache.org" 

> The test is very repeated, ie when I 'make install' from squid-3.2.0.12 it 
> works but not
> squid-3.2.018.

I meant the tests were very repeatable, squid-3.2.0.12 works, squid-3.2.0.13 
works.
Squid-3.2.0.14 onwards ( tested squid-3.2.0.14, squid-3.2.0.15, squid-3.2.0.16, 
squid-3.2.0.18 ) all start giving problems. 

For squid-3.2.0.14, when I try to logon to yahoo mail, I get this thing below. 
Other
versions seem to just hang until timeout. I am not trying to finger point at 
squid
beta version, but I hope these tests will throw some lights to my problem with
using squid in tproxy mode :-

ERROR
The requested URL could not be retrieved



Invalid Request error was encountered while trying to process the request:

GET /neo/launch?.rand=b1ktfi57od9dm HTTP/1.1
Accept:
 image/gif, image/jpeg, image/pjpeg, image/pjpeg, 
application/x-shockwave-flash, application/vnd.ms-powerpoint, 
application/vnd.ms-excel, application/msword, application/x-ms-xbap, 
application/vnd.ms-xpsdocument, application/xaml+xml, 
application/x-ms-application, */*
Accept-Language: en-us,zh-CN;q=0.7,zh-TW;q=0.3
User-Agent:
 Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; (R1 
1.6); .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727)
Accept-Encoding: gzip, deflate
Host: us.mg6.mail.yahoo.com
Connection: Keep-Alive
Cookie:
 YM.NEO_401430109=; 
B=3c13u8l80rhtv&b=4&d=fhZ_4jJpYFpJGdj7zY3momBX59rysCEBsTfUSw--&s=4n&i=bKhuXma.oOGyPs48n8Zs;
 MSC=t=1343095009X; 
CH=AgBQDcfQACnOUAA0o0AANfIwOTMwUBU3zFAABaHQAAU60AARnaAAKebQAD78kBI1E8AFFzjABS/oMCMzW4AVKto=;
 BA=ba=954&ip=202.46.125.50&t=1343093816; 
F=a=T0kCqogMvSg2966rgvZtTBroHDD9YRIxrhruO5G2Id9YECK2VTCcJQC_osnCHco7IulOaBU-&b=M6y3;
 
Y=v=1&n=b76g5t5k0htpl&l=abvss/o&p=m1tvvmy313220400&iz=52200&r=3l&lg=en-US&intl=us;
 C=mg=1; 
U=mt=qIPNYp2MhYjyHJ68oBizJjAhX4ZPw4zZZ3fShTnx&ux=oufDQB&un=b76g5t5k0htpl;
 YLS=v=1&p=1&n=0; PH=fn=mDjSuQdea6k2x1s-&l=en-US&i=us; 
T=z=pEgDQBpYHIQBR9xcpMzkeu.NjE1NwYzNzYzNDc2N04-&a=QAE&sk=DAALMXwKvPWVxg&ks=EAARH9uRmqjEAc3_qrdxUidnQ--~E&d=c2wBTVRZeU1BRTBNREUwTXpBeE1Eay0BYQFRQUUBZwFLVjc0SFhBNkg2UkFSTFgyS0JUM01CM0cyWQF0aXABSzdTZnlBAXp6AXBFZ0RRQkE3RQ--;
 
RT=s=1343095221696&u=&r=https%3A//login.yahoo.com/config/login_verify2%3F%26.src%3Dym
Some possible problems are:

Missing or unknown request method.

Missing URL.

Missing HTTP Identifier (HTTP/1.0).

•Request is too large.

•Content-Length missing for POST or PUT requests.

•Illegal character in hostname; underscores are not allowed.

•HTTP/1.1 “Expect:” feature is being asked from an HTTP/1.0 software.

Your cache administrator is webmaster.






Generated Tue, 24 Jul 2012 02:00:18 GMT by fedora15 (squid/3.2.0.14)

Re: [squid-users] yahoo mail problem with tproxy (squid 3.1.19, kernel 3.2.21)

2012-07-23 Thread Ming-Ching Tiew 




- Original Message -
From: Ming-Ching Tiew 
To: "squid-users@squid-cache.org" 

> The test is very repeated, ie when I 'make install' from squid-3.2.0.12 it 
> works but not
> squid-3.2.018.

I meant the tests were very repeatable, squid-3.2.0.12 works, squid-3.2.0.13 
works.
Squid-3.2.0.14 onwards ( tested squid-3.2.0.14, squid-3.2.0.15, squid-3.2.0.16, 
squid-3.2.0.18 ) all start giving problems. 

For squid-3.2.0.14, when I try to logon to yahoo mail, I get this thing below. 
Other
versions seem to just hang until timeout. I am not trying to finger point at 
squid
beta version, but I hope these tests will throw some lights to my problem with
using squid in tproxy mode :-

ERROR
The requested URL could not be retrieved



Invalid Request error was encountered while trying to process the request:

GET /neo/launch?.rand=b1ktfi57od9dm HTTP/1.1
Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, 
application/x-shockwave-flash, application/vnd.ms-powerpoint, 
application/vnd.ms-excel, application/msword, application/x-ms-xbap, 
application/vnd.ms-xpsdocument, application/xaml+xml, 
application/x-ms-application, */*
Accept-Language: en-us,zh-CN;q=0.7,zh-TW;q=0.3
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; (R1 
1.6); .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727)
Accept-Encoding: gzip, deflate
Host: us.mg6.mail.yahoo.com
Connection: Keep-Alive
Cookie: YM.NEO_401430109=; 
B=3c13u8l80rhtv&b=4&d=fhZ_4jJpYFpJGdj7zY3momBX59rysCEBsTfUSw--&s=4n&i=bKhuXma.oOGyPs48n8Zs;
 MSC=t=1343095009X; 
CH=AgBQDcfQACnOUAA0o0AANfIwOTMwUBU3zFAABaHQAAU60AARnaAAKebQAD78kBI1E8AFFzjABS/oMCMzW4AVKto=;
 BA=ba=954&ip=202.46.125.50&t=1343093816; 
F=a=T0kCqogMvSg2966rgvZtTBroHDD9YRIxrhruO5G2Id9YECK2VTCcJQC_osnCHco7IulOaBU-&b=M6y3;
 
Y=v=1&n=b76g5t5k0htpl&l=abvss/o&p=m1tvvmy313220400&iz=52200&r=3l&lg=en-US&intl=us;
 C=mg=1; 
U=mt=qIPNYp2MhYjyHJ68oBizJjAhX4ZPw4zZZ3fShTnx&ux=oufDQB&un=b76g5t5k0htpl; 
YLS=v=1&p=1&n=0; PH=fn=mDjSuQdea6k2x1s-&l=en-US&i=us; 
T=z=pEgDQBpYHIQBR9xcpMzkeu.NjE1NwYzNzYzNDc2N04-&a=QAE&sk=DAALMXwKvPWVxg&ks=EAARH9uRmqjEAc3_qrdxUidnQ--~E&d=c2wBTVRZeU1BRTBNREUwTXpBeE1Eay0BYQFRQUUBZwFLVjc0SFhBNkg2UkFSTFgyS0JUM01CM0cyWQF0aXABSzdTZnlBAXp6AXBFZ0RRQkE3RQ--;
 
RT=s=1343095221696&u=&r=https%3A//login.yahoo.com/config/login_verify2%3F%26.src%3Dym
Some possible problems are:

Missing or unknown request method.

Missing URL.

Missing HTTP Identifier (HTTP/1.0).

•Request is too large.

•Content-Length missing for POST or PUT requests.

•Illegal character in hostname; underscores are not allowed.

•HTTP/1.1 “Expect:” feature is being asked from an HTTP/1.0 software.

Your cache administrator is webmaster.






Generated Tue, 24 Jul 2012 02:00:18 GMT by fedora15 (squid/3.2.0.14)

[squid-users] Include directive in 2.6?

2012-07-23 Thread Baird, Josh 
Hi,

Can someone confirm if the "include" directive is supported in 2.6?  I'm 
running squid-2.6.STABLE21-6.el5, and have "include /etc/squid/conf.d/*.conf" 
in my squid.conf.  No errors are reported, but the configuration files do not 
seem to actually be included.

Thanks,

Josh

Re: [squid-users] Query regarding Squid buffer

2012-07-23 Thread Amos Jeffries 

On 24.07.2012 05:11, Panchal Suman wrote:

Hello All,

How to get the number of packets that are in squid buffer when user
is downloading a large file via the squid proxy?


Somewhere between 0 bytes and read_ahead_gap. Normally there are none 
at all. Why do you need to know?



The active_requests cache manager report lists a line
"out.offset 16443, out.size 12705"

out.size have been delivered, out.offset is where the receive side is 
up to presently.


So (out.offset - out.size) is the bytes currently buffered in transit 
through Squid.


Amos

Re: [squid-users] Non-browser applications using NTLM+Squid?

2012-07-23 Thread Amos Jeffries 

On 24.07.2012 05:05, Alex Crow wrote:

Josh,

http_access deny requirentlmhosts

after the allow rule should do it I think.

Alex


If you have an "unprotected" requirentlmhosts ACL the auth challenge 
will be displayed to anyone being tested against it.


What you need is this:

  # require auth from a certain set of hosts
  acl requirentlm proxy_auth REQUIRED
  acl requirentlmhosts src 1.1.1.1/255.255.255.255

  http_acccess deny requirentlmhosts !requirentlm

... followed by any other policies you have. Such as possibly an "allow 
requirentlmhosts" to let these clients through with only authentication 
and then the allow/deny polciy bits for non-requirentlmhosts clients.


Amos

Re: [squid-users] tunnelConnectTimeout(): tunnelState->servers is NULL

2012-07-23 Thread Amos Jeffries 

On 24.07.2012 06:14, Dean Weimer wrote:

I have had a Squid server that has been running for some time, and
all of the sudden started having problems, this server runs as both a
forward and reverse proxy on different ports.  The reverse proxy part
seems to be responding fine, but the forwarding is all of the sudden
logging errors in cache.log file about tunnelSate  The parent server
appears to be running fine, and is serving requests for a few hundred
clients that access it directly without issue.

Of course the full log message is exactly what the subject of this
email message is.
tunnelConnectTimeout(): tunnelState->servers is NULL

The parent server is running 3.1.20 and this server is on 3.1.18,
anyone have any idea what would cause this type of behavior to start
happening?


Something is timing out in CONNECT. Which should not happen normally, 
but Squid closes the connections cleanly so its just a warning.


It's not exactly clear whether this is a timeout event arriving after 
the tunnel has already closed properly (erasing its "servers" list), or 
if it is a real unable-to-connect problem failing after a set of 
retries. Are your client connections through the 3.1.18 proxy having 
trouble with HTTPS?


Amos

RE: [squid-users] How to optimize squid config for live flash video streaming?

2012-07-23 Thread Amos Jeffries 

On 24.07.2012 01:28, Andrew Krupiczka wrote:

Hello Everyone,

Just one comment and question.

1) we're using the squid to optimize HLS (http based) video streaming
exactly via caching, so you can deliver more efficiently a cached
video segments instead of directly communicating with an origin
server(s).
2) does anyone know, if/when collapsed forwarding feature is going to
be incorporated into Squid 3.x?


At this stage there is none working on it AFAIK. The available dev are 
concentrating on making 3.2 ready for stable release.


Amos

[squid-users] tunnelConnectTimeout(): tunnelState->servers is NULL

2012-07-23 Thread Dean Weimer 
I have had a Squid server that has been running for some time, and all of the 
sudden started having problems, this server runs as both a forward and reverse 
proxy on different ports.  The reverse proxy part seems to be responding fine, 
but the forwarding is all of the sudden logging errors in cache.log file about 
tunnelSate  The parent server appears to be running fine, and is serving 
requests for a few hundred clients that access it directly without issue.

Of course the full log message is exactly what the subject of this email 
message is.
tunnelConnectTimeout(): tunnelState->servers is NULL

The parent server is running 3.1.20 and this server is on 3.1.18, anyone have 
any idea what would cause this type of behavior to start happening?

Thanks,
 Dean Weimer
 Network Administrator
 Orscheln Management Co

[squid-users] Query regarding Squid buffer

2012-07-23 Thread Panchal Suman 


Hello All,

How to get the number of packets that are in squid buffer when user is 
downloading a large file via the squid proxy?


Any response would be much appreciated.

Thanks and Regards,
Suman Panchal

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Re: [squid-users] Non-browser applications using NTLM+Squid?

2012-07-23 Thread Alex Crow 

Josh,

http_access deny requirentlmhosts

after the allow rule should do it I think.

Alex

On 23/07/12 15:08, Baird, Josh wrote:

How would I go about only forcing certain hosts to use NTLM auth, but allowing 
everyone else to use the proxy un-authenticated?

I have a ACL that contain's src's of IP's that I need to force to use NTLM:

acl requirentlm proxy_auth REQUIRED
acl requirentlmhosts src 1.1.1.1/255.255.255.255
http_acccess allow requirentlmhosts requirentlm

This takes care of forcing "requirentlmhosts" to auth, but if I have another http_access 
rule that allows everyone else, what keeps "requirentlmhosts" from getting out without 
auth?

Thanks,

Josh

-Original Message-
From: Baird, Josh
Sent: Thursday, July 19, 2012 9:39 PM
To: Eliezer Croitoru; squid-users@squid-cache.org
Subject: RE: [squid-users] Non-browser applications using NTLM+Squid?

Not sure why I didn't think of that.  Thanks!

Josh

From: Eliezer Croitoru [elie...@ngtech.co.il]
Sent: Thursday, July 19, 2012 6:12 PM
To: squid-users@squid-cache.org
Subject: Re: [squid-users] Non-browser applications using NTLM+Squid?

On 7/19/2012 11:29 PM, Baird, Josh wrote:

Hi,

I'm wondering what others are doing about non-browser applications (Anti-virus 
software that fetches updates, instant messengers over HTTP, etc) that sit 
behind a Squid proxy that requires NTLM authentication?  These applications, in 
my experience, use Windows' proxy settings to proxy their outbound traffic, but 
can't speak NTLM, so the application is prevented from proxying any traffic.

Would a Kerberos integrated Squid be a possible solution to this problem?

Thanks,

Josh


very simple.. just allow them all before the authentication acls such as in:

acl updates dstdomain .windowsupdates.microsoft.com .antivirusupdates.org
acl updates1 dst 192.168.0.1/32

http_access allow localnet updates
http_access allow localnet updates1
http_access allow localnet ntlm_auth_helper
http_access deny all


Regards,
Eliezer
--
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer  ngtech.co.il

RE: [squid-users] Non-browser applications using NTLM+Squid?

2012-07-23 Thread Baird, Josh 
How would I go about only forcing certain hosts to use NTLM auth, but allowing 
everyone else to use the proxy un-authenticated?

I have a ACL that contain's src's of IP's that I need to force to use NTLM:

acl requirentlm proxy_auth REQUIRED
acl requirentlmhosts src 1.1.1.1/255.255.255.255
http_acccess allow requirentlmhosts requirentlm

This takes care of forcing "requirentlmhosts" to auth, but if I have another 
http_access rule that allows everyone else, what keeps "requirentlmhosts" from 
getting out without auth?

Thanks,

Josh 

-Original Message-
From: Baird, Josh 
Sent: Thursday, July 19, 2012 9:39 PM
To: Eliezer Croitoru; squid-users@squid-cache.org
Subject: RE: [squid-users] Non-browser applications using NTLM+Squid?

Not sure why I didn't think of that.  Thanks!

Josh

From: Eliezer Croitoru [elie...@ngtech.co.il]
Sent: Thursday, July 19, 2012 6:12 PM
To: squid-users@squid-cache.org
Subject: Re: [squid-users] Non-browser applications using NTLM+Squid?

On 7/19/2012 11:29 PM, Baird, Josh wrote:
> Hi,
>
> I'm wondering what others are doing about non-browser applications 
> (Anti-virus software that fetches updates, instant messengers over HTTP, etc) 
> that sit behind a Squid proxy that requires NTLM authentication?  These 
> applications, in my experience, use Windows' proxy settings to proxy their 
> outbound traffic, but can't speak NTLM, so the application is prevented from 
> proxying any traffic.
>
> Would a Kerberos integrated Squid be a possible solution to this problem?
>
> Thanks,
>
> Josh
>
very simple.. just allow them all before the authentication acls such as in:

acl updates dstdomain .windowsupdates.microsoft.com .antivirusupdates.org
acl updates1 dst 192.168.0.1/32

http_access allow localnet updates
http_access allow localnet updates1
http_access allow localnet ntlm_auth_helper
http_access deny all


Regards,
Eliezer
--
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer  ngtech.co.il

RE: [squid-users] How to optimize squid config for live flash video streaming?

2012-07-23 Thread Andrew Krupiczka 
Hello Everyone,

Just one comment and question.

1) we're using the squid to optimize HLS (http based) video streaming exactly 
via caching, so you can deliver more efficiently a cached video segments 
instead of directly communicating with an origin server(s).
2) does anyone know, if/when collapsed forwarding feature is going to be 
incorporated into Squid 3.x?

Best regards,

Andrew

-Original Message-
From: Amos Jeffries [mailto:squ...@treenet.co.nz] 
Sent: Friday, July 20, 2012 10:54 PM
To: squid-users@squid-cache.org
Subject: Re: [squid-users] How to optimize squid config for live flash video 
streaming?

On 21/07/2012 5:58 a.m., J DJ wrote:
> My squid proxy is not caching (using the "cache deny all" in 
> squid.conf), and I was wondering if anyone had any suggestions or 
> resources on how to optimize it for live flash video streaming?

There is no optimization for live streaming without caching.
The Squid collapsed-forwarding feature for splitting single response to 
multiple clients relies on responses being cacheable (even if not
stored) so that multiple clients are *allowed* to receive them. Live streams 
are usually marked as non-cacheable, and Flash video is even worse since each 
stream has different framing intervals depending on when each client connected 
with adaptive bitrate frames depending on each clients individual end-to-end 
latency.

This assumes the stream is an HTTP data stream. A lot of streams these days are 
ICY protocol, even if displayed by a Flash script interpreter.

Amos

Re: [squid-users] Fw: [ERROR] commonUfsDirOpenSwapLog: Failed to open swap log

2012-07-23 Thread Warren Baker 
On Mon, Jul 23, 2012 at 12:51 PM, Anonymous  wrote:
> Send the contents (sanitized if needed) of your /etc/rc.conf
> hostname="test"
> sshd_enable="YES"
> powerd_enable="YES"
> # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
> dumpdev="NO"
> pdnsd_enable="YES"
> apache22_enable="YES"
> #squid_enable="YES"

Uncomment this

> #/usr/local/sbin/squid
> /usr/local/etc/rc.d/squid.sh

Remove this ^^^

>
> ***
> ls /usr/local/etc/rc.d/
> apache22htcachecleansquid.sh
> bandwidthd.sh.samplepdnsd   webmin


How did you install Squid? Ahh..looking further back in the thread I
see you have a 3.2 version for which there is no port yet.
So in that case you can either amend your squid.sh file to include
rc.subr and add the appropriate checks or use another squid startup
file and modify it to suit your needs.

Squid3.1 port startup file below, but should work fine for your install:

#!/bin/sh
#
# $FreeBSD: ports/www/squid31/files/squid.in,v 1.10 2011/08/31
21:09:18 flo Exp $
#
# PROVIDE: squid
# REQUIRE: LOGIN
# KEYWORD: shutdown
# Note:
# Set "squid_enable=yes" in either /etc/rc.conf, /etc/rc.conf.local or
# /etc/rc.conf.d/squid to activate Squid.
#
# Additional variables you can define in one of these files:
#
# squid_chdir:  the directory into which the rc system moves into before
#   starting Squid. Default: /var/squid
#
# squid_conf:   The configuration file that Squid should use.
#   Default: /usr/local/etc/squid/squid.conf
#
# squid_fib:The alternative routing table id that Squid should use.
#   Default: none
#   See setfib(1) for further details. Note that the setfib(2)
#   system call is not available in FreeBSD versions prior to 7.1.
#
# squid_user:   The user id that should be used to run the Squid master
#   process. Default: squid.
#   Note that you probably need to define "squid_user=root" if
#   you want to run Squid in reverse proxy setups or if you want
#   Squid to listen on a "privileged" port < 1024.
#
# squid_pidfile:
#   The name (including the full path) of the Squid
#   master process' PID file.
#   Default: /var/run/squid/squid.pid.
#   You only need to change this if you changed the
#   corresponding entry in your Squid configuration.
# squid_flags:  Additional commandline arguments for Squid you might want to
#   use. See squid(8) for further details.
#

squid_checkrunning() {
${command} ${command_args} ${squid_flags} -k check 2>/dev/null
}

squid_setfib() {
sysctl net.fibs >/dev/null 2>&1 || return 0
if [ "x${squid_fib}" != "xNONE" ]; then
command="setfib -F ${squid_fib} ${command}"
else
return 0
fi
}

squid_stop() {
echo "Stopping ${name}."
${command} ${command_args} ${squid_flags} -k shutdown
run_rc_command poll
}

. /etc/rc.subr

name=squid
rcvar=${name}_enable
command=/usr/local/sbin/squid
extra_commands=reload
reload_cmd="${command} ${command_args} ${squid_flags} -k reconfigure"
start_precmd="squid_setfib"
stop_precmd="squid_checkrunning"
stop_cmd="squid_stop"

load_rc_config ${name}

squid_chdir=${squid_chdir:-"/var/squid"}
squid_conf=${squid_conf:-"/usr/local/etc/squid/squid.conf"}
squid_enable=${squid_enable:-"NO"}
squid_fib=${squid_fib:-"NONE"}
squid_pidfile=${squid_pidfile:-"/var/run/squid/squid.pid"}
squid_user=${squid_user:-squid}

pidfile=${squid_pidfile}
required_dirs=${squid_chdir}

# squid(8) will not start if ${squid_conf} is not present so try
# to catch that beforehand via ${required_files} rather than make
# squid(8) crash.

required_files=${squid_conf}

# Now make sure that we invoke squid with "-f ${squid_conf}":

command_args="-f ${squid_conf}"

run_rc_command "$1"

# Eof



-- 
.warren

Re: [squid-users] Fw: [ERROR] commonUfsDirOpenSwapLog: Failed to open swap log

2012-07-23 Thread Anonymous 
Send the contents (sanitized if needed) of your /etc/rc.conf 
hostname="test"
sshd_enable="YES"
powerd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
pdnsd_enable="YES"
apache22_enable="YES"
#squid_enable="YES"
#/usr/local/sbin/squid
/usr/local/etc/rc.d/squid.sh

***
ls /usr/local/etc/rc.d/
apache22    htcacheclean    squid.sh
bandwidthd.sh.sample    pdnsd   webmin



- Original Message -
> From: Warren Baker 
> To: Anonymous 
> Cc: Amos Jeffries ; "squid-users@squid-cache.org" 
> 
> Sent: Monday, July 23, 2012 2:11 PM
> Subject: Re: [squid-users] Fw: [ERROR] commonUfsDirOpenSwapLog: Failed to 
> open swap log
> 
> On Mon, Jul 23, 2012 at 11:02 AM, Anonymous  
> wrote:
>>  Well! I have tried every mentioned method and still getting the same old 
> error...!
> 
> Send the contents (sanitized if needed) of your /etc/rc.conf and the
> contents of your /usr/local/etc/rc.d/ directory.
> 
> 
> -- 
> .warren
>

Re: [squid-users] Re: redirection module

2012-07-23 Thread Amos Jeffries 

On 23/07/2012 8:29 p.m., Abhay Singh wrote:

Hi,

  I intend to create some acl-based module (in C) that can analyse the
request url and perform some internal logic (check some conditions,
fetch data from a database) and then either redirect the request to
another URL or send it untouched to the server. Is it possible in
squid or will i have to use some icap reqmod module for this?

Thanks and regards,
Abhay


http://wiki.squid-cache.org/Features/AddonHelpers

Amos

Re: [squid-users] Fw: [ERROR] commonUfsDirOpenSwapLog: Failed to open swap log

2012-07-23 Thread Warren Baker 
On Mon, Jul 23, 2012 at 11:02 AM, Anonymous  wrote:
> Well! I have tried every mentioned method and still getting the same old 
> error...!

Send the contents (sanitized if needed) of your /etc/rc.conf and the
contents of your /usr/local/etc/rc.d/ directory.


-- 
.warren

Re: [squid-users] Fw: [ERROR] commonUfsDirOpenSwapLog: Failed to open swap log

2012-07-23 Thread Anonymous 
Well! I have tried every mentioned method and still getting the same old 
error...!



- Original Message -
> From: Warren Baker 
> To: Anonymous 
> Cc: Amos Jeffries ; "squid-users@squid-cache.org" 
> ; Squid Developers 
> Sent: Monday, July 23, 2012 1:00 PM
> Subject: Re: [squid-users] Fw: [ERROR] commonUfsDirOpenSwapLog: Failed to 
> open swap log
> 
> On Mon, Jul 23, 2012 at 8:02 AM, Anonymous  
> wrote:
>>  squid_enable"YES"
> 
> If you installed Squid from ports then you will have a file called
> /usr/local/etc/rc.d/squid - which is the startup file. You can get the
> startup variables from this file by executing it and passing the
> option rcvar.
> 
> # /usr/local/etc/rc.d/squid rcvar
> # squid
> #
> squid_enable="NO"
> #   (default: "")
> 
> it should return what the startup variable is and what it's value it.
> In your configured setting, above, you are missing the '=' sign. So
> set it to squid_enable="YES" and remove any of your other additional
> references to squid.
> 
> -- 
> .warren
>

[squid-users] Re: redirection module

2012-07-23 Thread Abhay Singh 
Hi,

 I intend to create some acl-based module (in C) that can analyse the
request url and perform some internal logic (check some conditions,
fetch data from a database) and then either redirect the request to
another URL or send it untouched to the server. Is it possible in
squid or will i have to use some icap reqmod module for this?

Thanks and regards,
Abhay

Re: [squid-users] Fw: [ERROR] commonUfsDirOpenSwapLog: Failed to open swap log

2012-07-23 Thread Warren Baker 
On Mon, Jul 23, 2012 at 8:02 AM, Anonymous  wrote:
> squid_enable"YES"

If you installed Squid from ports then you will have a file called
/usr/local/etc/rc.d/squid - which is the startup file. You can get the
startup variables from this file by executing it and passing the
option rcvar.

# /usr/local/etc/rc.d/squid rcvar
# squid
#
squid_enable="NO"
#   (default: "")

it should return what the startup variable is and what it's value it.
In your configured setting, above, you are missing the '=' sign. So
set it to squid_enable="YES" and remove any of your other additional
references to squid.

-- 
.warren