[squid-users] Startup error with client request buffer

2012-03-31 Thread Guillaume Hilt

Hello,

I'm running Squid 3.1.14 (last available version) on Ubuntu 11.10 AMD64.
When i'm trying to run it, it fail with this error :
2012 Mar 31 17:30:44 rendez-vous Client request buffer of 524288 bytes 
cannot hold a request with 1048576 bytes of headers. Change 
client_request_buffer_max or request_header_max_size limits.
FATAL: Client request buffer of 524288 bytes cannot hold a request with 
1048576 bytes of headers. Change client_request_buffer_max or 
request_header_max_size limits.

Squid Cache (Version 3.1.14): Terminated abnormally.
CPU Usage: 0.012 seconds = 0.012 user + 0.000 sys
Maximum Resident Size: 16032 KB
Page faults with physical i/o: 0

client_request_buffer_max_size and request_header_max_size are set to 
2048kB.


Here's my conf :

auth_param basic program /usr/lib/squid3/squid_db_auth --user squid 
--password X --plaintext --persist

auth_param basic children 5
auth_param basic realm Squid
auth_param basic credentialsttl 2 hours
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 10.0.0.0/8# RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16# RFC1918 possible internal network
acl FTP proto FTP
acl SSL_ports port 443 21 20
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443# https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210# wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280# http-mgmt
acl Safe_ports port 488# gss-http
acl Safe_ports port 591# filemaker
acl Safe_ports port 777# multiling http
acl Safe_ports port 901# SWAT
acl purge method PURGE
acl CONNECT method CONNECT
acl My_ports port 80 21 6667
acl db-auth proxy_auth REQUIRED
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow FTP
http_access allow purge localhost
http_access deny purge
http_access deny CONNECT !SSL_ports
http_access allow db-auth
http_access allow localhost
http_access deny all
http_reply_access allow all
icp_access allow localnet
icp_access deny all
htcp_access allow localnet
htcp_access deny all
http_port XX.XX.XX.XX:23
http_port XX.XX.XX.XX:80
hierarchy_stoplist cgi-bin ?
maximum_object_size_in_memory 1 KB
maximum_object_size 1 KB
log_ip_on_direct off
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:  144020% 10080
refresh_pattern ^gopher:   14400%  1440
refresh_pattern -i (/cgi-bin/|\?) 00%  0
refresh_pattern .  0   20% 4320
request_header_max_size 2048 KB
reply_header_max_size 2048 KB
client_request_buffer_max_size 2048 KB
request_header_access Allow allow all
request_header_access Authorization allow all
request_header_access WWW-Authenticate allow all
request_header_access Proxy-Authorization allow all
request_header_access Proxy-Authenticate allow all
request_header_access Cache-Control allow all
request_header_access Content-Encoding allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Expires allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Last-Modified allow all
request_header_access Location allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Content-Language allow all
request_header_access Mime-Version allow all
request_header_access Retry-After allow all
request_header_access Title allow all
request_header_access Connection allow all
request_header_access Proxy-Connection allow all
request_header_access Cookie allow all
request_header_access Set-Cookie allow all
request_header_access User-Agent allow all
request_header_access All deny all
httpd_suppress_version_string on
always_direct allow FTP
forwarded_for delete
client_db off
cache_access_log /dev/null
cache_store_log /dev/null

Any idea ?

Regards,

--
  Guillaume Hilt



Re: [squid-users] Startup error with client request buffer

2012-03-31 Thread Amos Jeffries

On 1/04/2012 5:27 a.m., Guillaume Hilt wrote:

Hello,

I'm running Squid 3.1.14 (last available version) on Ubuntu 11.10 AMD64.
When i'm trying to run it, it fail with this error :
2012 Mar 31 17:30:44 rendez-vous Client request buffer of 524288 bytes 
cannot hold a request with 1048576 bytes of headers. Change 
client_request_buffer_max or request_header_max_size limits.
FATAL: Client request buffer of 524288 bytes cannot hold a request 
with 1048576 bytes of headers. Change client_request_buffer_max or 
request_header_max_size limits.

Squid Cache (Version 3.1.14): Terminated abnormally.
CPU Usage: 0.012 seconds = 0.012 user + 0.000 sys
Maximum Resident Size: 16032 KB
Page faults with physical i/o: 0

client_request_buffer_max_size and request_header_max_size are set to 
2048kB.


You need to be be able to store all the headers, plus the request and 
URL details, plus the HTTP framing bytes in the buffer at once. Having 
the headers size alone only just fit into the buffer is not enough space.


The other main question is why do you expect to have HTTP requests with 
2MB of *headers* arriving?


Unless you have a very good reason I recommend leaving them unset 
(default values). Squid has the highest limits around out of any of the 
HTTP middleware, there is very little chance that requests reaching or 
passing the Squid default limits will be able to pass over the Internet 
reliably even if you raise them for your proxy.




Here's my conf :

auth_param basic program /usr/lib/squid3/squid_db_auth --user squid 
--password X --plaintext --persist

auth_param basic children 5
auth_param basic realm Squid
auth_param basic credentialsttl 2 hours
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 10.0.0.0/8# RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16# RFC1918 possible internal 
network

acl FTP proto FTP
acl SSL_ports port 443 21 20
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443# https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210# wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280# http-mgmt
acl Safe_ports port 488# gss-http
acl Safe_ports port 591# filemaker
acl Safe_ports port 777# multiling http
acl Safe_ports port 901# SWAT
acl purge method PURGE
acl CONNECT method CONNECT
acl My_ports port 80 21 6667
acl db-auth proxy_auth REQUIRED
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow FTP


Um, unlimited FTP access for anyone on the Internet?  see below.


http_access allow purge localhost
http_access deny purge


You have HTCP half-enabled. If you finish that by opening a htcp_port 
and setting htcp_clr_access allow localhost you can probably drop 
PURGE support.



http_access deny CONNECT !SSL_ports


... you already blocked CONNECT !SSL_ports above.


http_access allow db-auth
http_access allow localhost



Allow access to logged in users or any traffic arriving from localhost. 
Notice how there is not mention of a particular protocol. Such as FTP 
versus HTTP versus HTTPS.
That means these rules above are permitting your users access to 
ftp://... without needing that dangrous http_access allow FTP line.



http_access deny all
http_reply_access allow all
icp_access allow localnet
icp_access deny all
htcp_access allow localnet
htcp_access deny all
http_port XX.XX.XX.XX:23
http_port XX.XX.XX.XX:80
hierarchy_stoplist cgi-bin ?
maximum_object_size_in_memory 1 KB
maximum_object_size 1 KB
log_ip_on_direct off
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:  144020% 10080
refresh_pattern ^gopher:   14400%  1440
refresh_pattern -i (/cgi-bin/|\?) 00%  0
refresh_pattern .  0   20% 4320
request_header_max_size 2048 KB
reply_header_max_size 2048 KB
client_request_buffer_max_size 2048 KB
request_header_access Allow allow all
request_header_access Authorization allow all
request_header_access WWW-Authenticate allow all
request_header_access Proxy-Authorization allow all
request_header_access Proxy-Authenticate allow all
request_header_access Cache-Control allow all
request_header_access Content-Encoding allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Expires allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Last-Modified allow all
request_header_access Location allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow