[squid-users] Re: Chunked and Range header support
Thanks for the clarification! Regards, Anita -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Chunked-and-Range-header-support-tp4660869p4660884.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] file count mismatch
this is squid 3.3.6 [root@LARS squid]# tail -n 30 /var/log/squid/cache.log 2013/07/03 06:34:59| Target number of buckets: 40392 2013/07/03 06:34:59| Using 65536 Store buckets 2013/07/03 06:34:59| Max Mem size: 262144 KB 2013/07/03 06:34:59| Max Swap size: 1024 KB 2013/07/03 06:34:59| Rebuilding storage in /home/squid (clean log) 2013/07/03 06:34:59| Using Least Load store dir selection 2013/07/03 06:34:59| Set Current Directory to /home/squid 2013/07/03 06:34:59| Loaded Icons. 2013/07/03 06:34:59| HTCP Disabled. 2013/07/03 06:34:59| Squid plugin modules loaded: 0 2013/07/03 06:34:59| Accepting NAT intercepted HTTP Socket connections at local=0.0.0.0:3128 remote=[::] FD 16 flags=41 2013/07/03 06:34:59| Accepting HTTP Socket connections at local=127.0.0.1:3129 remote=[::] FD 17 flags=9 2013/07/03 06:34:59| Store rebuilding is 1.33% complete 2013/07/03 06:35:00| Done reading /home/squid swaplog (299709 entries) 2013/07/03 06:35:00| Finished rebuilding storage from disk. 2013/07/03 06:35:00|299709 Entries scanned 2013/07/03 06:35:00| 0 Invalid entries. 2013/07/03 06:35:00| 0 With invalid flags. 2013/07/03 06:35:00|299709 Objects loaded. 2013/07/03 06:35:00| 0 Objects expired. 2013/07/03 06:35:00| 0 Objects cancelled. 2013/07/03 06:35:00| 0 Duplicate URLs purged. 2013/07/03 06:35:00| 0 Swapfile clashes avoided. 2013/07/03 06:35:00| Took 1.36 seconds (220842.74 objects/sec). 2013/07/03 06:35:00| Beginning Validation Procedure 2013/07/03 06:35:00| 262144 Entries Validated so far. 2013/07/03 06:35:00| Completed Validation Procedure 2013/07/03 06:35:00| Validated 299707 Entries 2013/07/03 06:35:00| store_swap_size = 8047432.00 KB 2013/07/03 06:35:01| storeLateRelease: released 0 objects [root@LARS squid]# find . -type f | wc -l 299706 this means there are 299706 files in /home/squid including the swap.state file so a total of 299705 objects on disk but swap.state thinks there are 299709 files. another thing I found. cat /var/log/squid/cache.log | grep WARNING | grep swapin 2013/06/08 23:35:45| WARNING: 1 swapin MD5 mismatches 2013/06/09 00:28:59| WARNING: 10 swapin MD5 mismatches 2013/06/09 12:20:56| WARNING: 1 swapin MD5 mismatches 2013/06/09 12:25:46| WARNING: 10 swapin MD5 mismatches 2013/06/09 14:40:18| WARNING: 1 swapin MD5 mismatches 2013/06/10 02:31:02| WARNING: 1 swapin MD5 mismatches 2013/06/10 13:00:37| WARNING: 1 swapin MD5 mismatches 2013/06/10 22:59:53| WARNING: 1 swapin MD5 mismatches 2013/06/12 14:41:23| WARNING: 1 swapin MD5 mismatches 2013/06/15 04:37:03| WARNING: 1 swapin MD5 mismatches 2013/06/28 07:07:33| WARNING: 1 swapin MD5 mismatches 2013/06/28 21:51:19| WARNING: 1 swapin MD5 mismatches 2013/06/29 23:41:49| WARNING: 1 swapin MD5 mismatches 2013/06/30 04:14:24| WARNING: 10 swapin MD5 mismatches 2013/06/30 19:10:49| WARNING: 1 swapin MD5 mismatches 2013/06/30 19:10:49| WARNING: 10 swapin MD5 mismatches 2013/07/02 13:44:11| WARNING: 1 swapin MD5 mismatches 2013/07/02 13:48:35| WARNING: 10 swapin MD5 mismatches 2013/07/02 23:27:32| WARNING: 100 swapin MD5 mismatches 2013/07/03 06:02:18| WARNING: 1 swapin MD5 mismatches 2013/07/03 06:20:43| WARNING: 1 swapin MD5 mismatches 2013/07/03 06:26:59| WARNING: 10 swapin MD5 mismatches 1) Any of the above is something to worry about? 2) Does squid resolve the file mismatch eventually as I reach the max size of cache dir? 3) the swapin MD5 mismatches problem. Is it something I can fix? If so, how? Any other information I can post to help detect where the problem is?
Re: [squid-users] Does squid support X-Forwarded-User
On 3/07/2013 9:39 a.m., Alex Rousskov wrote: On 07/02/2013 03:00 PM, Amos Jeffries wrote: On 3/07/2013 1:26 a.m., Alex Rousskov wrote: On 07/02/2013 02:34 AM, Amos Jeffries wrote: On 2/07/2013 6:33 p.m., Blason wrote: I know squid support X-Forwarded-For but keen to know if I can use X-Forwarded-User so that user authenticated with Proxy can have filtering done on other Application Control box placed in front of Proxy. And Proxy can only be used for caching functionality No. I have not tested it, but is not X-Forwarded-User supported via request_header_add with an appropriate logformat macro in the value? Only for sending. The question was about support for receiving and interpreting. I probably interpreted "in front" the wrong way, sorry. Well, Squid can certainly receive that header and a Squid helper or adapter can interpret it, even providing authentication-like capability if needed. Please note that I am not trying to defend the use of that header for anything, just indicating that some related functionality is available even though Squid itself does not treat X-Forwarded-User specially. Whereas like I initially said a frontend which can add headers is far better off adding Proxy-Authorization with the relevant details. Squid will then support it properly and help detect any dangerous collisions with the two headers. Amos
Re: [squid-users] Does squid support X-Forwarded-User
On 07/02/2013 03:00 PM, Amos Jeffries wrote: > On 3/07/2013 1:26 a.m., Alex Rousskov wrote: >> On 07/02/2013 02:34 AM, Amos Jeffries wrote: >>> On 2/07/2013 6:33 p.m., Blason wrote: I know squid support X-Forwarded-For but keen to know if I can use X-Forwarded-User so that user authenticated with Proxy can have filtering done on other Application Control box placed in front of Proxy. And Proxy can only be used for caching functionality >>> No. >> I have not tested it, but is not X-Forwarded-User supported via >> request_header_add with an appropriate logformat macro in the value? > Only for sending. The question was about support for receiving and > interpreting. I probably interpreted "in front" the wrong way, sorry. Well, Squid can certainly receive that header and a Squid helper or adapter can interpret it, even providing authentication-like capability if needed. Please note that I am not trying to defend the use of that header for anything, just indicating that some related functionality is available even though Squid itself does not treat X-Forwarded-User specially. Cheers, Alex.
Re: [squid-users] Windows RDS Gateway with Squid 3.3.5
On 3/07/2013 2:36 a.m., Stan2k wrote: Hello Everybody Here is the infrastructure I want : Client => Internet => Squid => RDS Gateway => VM Here is my configuration : https_port public_name:443 accel cert=/etc/ssl/private/servercert.pem key=/etc/ssl/private/serverkey.pem cafile=/etc/ssl/private/intermediate.pem capath=/etc/ssl/private/ defaultsite=parentserver.domain.qh version=1 cache_peer parentservername parent 443 0 no-query originserver ssl sslcert=/etc/ssl/private/servercert.crt.pem sslkey=/etc/ssl/private/serverkey.pem sslcapath=/etc/ssl/private/ login=PASSTHRU connection-auth=on ssloptions=ALL name=gateway sslflags=DONT_VERIFY_PEER front-end-https=on no-digest acl RDS dstdomain parentservername cache_peer_access gateway allow all #cache_peer_access gateway deny all http_access allow all Congratulations you have an open proxy. Expect its IP address to be firewalled and blocked by various networks around the world in the next few days if not already. Please follow the guidelines for reverse proxy configuration: Namely that cache_peer_access and http_access restricts allowed requests based on the explicit dstdomain (FQDN) which your peer accepts. If that is not possible at least retain the CONNECT security rules and add these ones which will permit unlimited relay through the peer but nowhere else (still not great, but better than "http_access allow all" as the sole security control): always_direct deny all never_direct allow all miss_access allow all Regarding "miss_access" if you are not going to configure any deny rules for it just remove it from your config file entirely. The default is "allow all". As you can see all is open but i have a problem. My configuration didn't work but yesterday I managed to log me 3 times from the office. Ten minutes after i could no longer log to the machine. I tried to log on at home last night and this morning and it worked. But now nobody can connect to the gateway. You can see the log when i could connect : 1372701961.331 79301 public_ip_client TCP_MISS_ABORTED/000 0 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip - This is followup from a previous connection (which got PINNED). 1372702018.639 8 public_ip_client TCP_MISS/401 695 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain Successful request. The peer responded 401 auth-required. Squid delivered that to the client. 1372702018.735 7 public_ip_client TCP_MISS/401 695 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain Successful request. The peer responded 401 auth-required. Squid delivered that to the client. 1372702025.441 6780 public_ip_client TCP_MISS_ABORTED/000 0 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip - Failed request. Squid relayed it to the peer. The client disconnected after 6.8 seconds and before the peer response could be relayed out to it. 1372702025.441 6686 public_ip_client TCP_MISS_ABORTED/200 7319 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip application/rpc Failed request. Squid relayed it to the peer. The peer processed it and responded 200 OK with some data. The client disconnected after 6.7 seconds and before the peer response could be fully relayed out to it (only 7319 bytes delivered out of an unknown amount greater than 7319). 1372702506.635 8 public_ip_client TCP_MISS/401 695 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain Successful request. The peer responded 401 auth-required. Squid delivered that to the client. 1372702506.728 7 public_ip_client TCP_MISS/401 695 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain Successful request. The peer responded 401 auth-required. Squid delivered that to the client. 1372702514.727 7963 public_ip_client TCP_MISS_ABORTED/200 103543 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip application/rpc Failed request. Squid relayed it to the peer. The peer processed it and responded 200 OK with some data. The client disconnected after 6.7 seconds and before the peer response could be fully relayed out to it (only 103KB delivered). 1372702514.728 8074 public_ip_client TCP_MISS_ABORTED/000 0 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip - 1372703139.182 11 public_ip_client TCP_MISS/401 695 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain 1372703139.295 8 public_ip_client TCP_MISS/401 695 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain 1372703146.054 6851 public_ip_client TCP_MISS_ABORTED/000
Re: [squid-users] Does squid support X-Forwarded-User
On 3/07/2013 1:26 a.m., Alex Rousskov wrote: On 07/02/2013 02:34 AM, Amos Jeffries wrote: On 2/07/2013 6:33 p.m., Blason wrote: I know squid support X-Forwarded-For but keen to know if I can use X-Forwarded-User so that user authenticated with Proxy can have filtering done on other Application Control box placed in front of Proxy. And Proxy can only be used for caching functionality No. I have not tested it, but is not X-Forwarded-User supported via request_header_add with an appropriate logformat macro in the value? Only for sending. The question was about support for receiving and interpreting. Amos
[squid-users] Windows RDS Gateway with Squid 3.3.5
Hello Everybody Here is the infrastructure I want : Client => Internet => Squid => RDS Gateway => VM Here is my configuration : https_port public_name:443 accel cert=/etc/ssl/private/servercert.pem key=/etc/ssl/private/serverkey.pem cafile=/etc/ssl/private/intermediate.pem capath=/etc/ssl/private/ defaultsite=parentserver.domain.qh version=1 cache_peer parentservername parent 443 0 no-query originserver ssl sslcert=/etc/ssl/private/servercert.crt.pem sslkey=/etc/ssl/private/serverkey.pem sslcapath=/etc/ssl/private/ login=PASSTHRU connection-auth=on ssloptions=ALL name=gateway sslflags=DONT_VERIFY_PEER front-end-https=on no-digest acl RDS dstdomain parentservername cache_peer_access gateway allow all #cache_peer_access gateway deny all http_access allow all miss_access allow all #http_access allow RDS #http_access deny all #miss_access allow RDS #miss_access deny all debug_options ALL,2 # # Recommended minimum configuration: # # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # # Recommended minimum Access Permission configuration: # # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager cache_mem 8 MB # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on "localhost" is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow localnet http_access allow localhost # And finally deny all other access to this proxy http_access deny all # Uncomment and adjust the following to add a disk cache directory. cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256 # Leave coredumps in the first cache dir #coredump_dir /usr/local/squid/var/cache/squid # # Add any of your own refresh_pattern entries above these. # refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 As you can see all is open but i have a problem. My configuration didn't work but yesterday I managed to log me 3 times from the office. Ten minutes after i could no longer log to the machine. I tried to log on at home last night and this morning and it worked. But now nobody can connect to the gateway. You can see the log when i could connect : 1372701961.331 79301 public_ip_client TCP_MISS_ABORTED/000 0 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip - 1372702018.639 8 public_ip_client TCP_MISS/401 695 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain 1372702018.735 7 public_ip_client TCP_MISS/401 695 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain 1372702025.441 6780 public_ip_client TCP_MISS_ABORTED/000 0 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip - 1372702025.441 6686 public_ip_client TCP_MISS_ABORTED/200 7319 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - PINNED/private_parentserver_ip application/rpc 1372702506.635 8 public_ip_client TCP_MISS/401 695 RPC_IN_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain 1372702506.728 7 public_ip_client TCP_MISS/401 695 RPC_OUT_DATA https://public_name.com/rpc/rpcproxy.dll? - FIRSTUP_PARENT/private_parentserver_ip text/plain 1372702514.727 79
Re: [squid-users] Does squid support X-Forwarded-User
On 07/02/2013 02:34 AM, Amos Jeffries wrote: > On 2/07/2013 6:33 p.m., Blason wrote: >> I know squid support X-Forwarded-For but keen to know if I can use >> X-Forwarded-User so that user authenticated with Proxy can have filtering >> done on other Application Control box placed in front of Proxy. And Proxy >> can only be used for caching functionality > No. I have not tested it, but is not X-Forwarded-User supported via request_header_add with an appropriate logformat macro in the value? Alex.
Re: [squid-users] CONNECT statistics unaggregated
> That is all discussion about squid-2.7. Did the logging infrastructure change in 3.* so that it would be possible to have separated counters for CONNECT in logs ? Or is it planned ? Thanks, -- Mathieu > > >> >> Cheers, >> >> [1] http://www.squid-cache.org/mail-archive/squid-users/200705/0729.html >> >> -- >> Mathieu > >
Re: [squid-users] CONNECT statistics unaggregated
On 2/07/2013 11:52 p.m., Mathieu GELI wrote: Hi list, I saw an old post about having proper recv_bytes and sent_bytes stats reported in the access.log here [1]. The user is mentioning a patch. I don't see it included in stable or did I miss something ? That is all discussion about squid-2.7. Cheers, [1] http://www.squid-cache.org/mail-archive/squid-users/200705/0729.html -- Mathieu
[squid-users] CONNECT statistics unaggregated
Hi list, I saw an old post about having proper recv_bytes and sent_bytes stats reported in the access.log here [1]. The user is mentioning a patch. I don't see it included in stable or did I miss something ? Cheers, [1] http://www.squid-cache.org/mail-archive/squid-users/200705/0729.html -- Mathieu
Re: [squid-users] Chunked and Range header support
On 2/07/2013 10:52 p.m., anita wrote: Hi All, I am using squid 3.1.16. 1)Can someone tell me how this version of squid handles chunked responses from a server? Does it wait for all the chunks to arrive and send a consolidated response back to the client or it sends as and when it receives it? No version of Squid should be waiting for the entire response. Chunking is normally used for responses with extremely large sizes which are unknown prior to delivery. They do wait for the whole of a single chunk though, or 64KB whichever comes first. The oldest versions with chunking support would not handle >64KB chunks IIRC. 2) Range headers - from my understanding, it looks like they use this for video streaming.. it looks like the client can request a part of the object body to be sent alone to him. Is it correct? In this case, if multiple ranges are requested, is it sent separately or in a consolidated manner? Whatever it is used for Range is a HTTP feature. HTTP protocol is followed. Amos
Re: [squid-users] Connection reset when accessing java servlet report page via squid
On 2/07/2013 10:58 p.m., Visolve Squid Support wrote: Hello, We have a problem with the squid when accessing a servlet page through the squid proxy. It is report page where the inputs are taken from the user and the servlet manipulates the report and present it in the page. Normally it takes around 45-60 seconds to generate the report. So we are getting the Connection reset' message in firefox and 'Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data' in chrome. But works normally without a proxy. Please suggest a solution for this issue if there is any config change need to be done. The default timeouts waiting for responses in Squid are much longer than 60 seconds. Try to locate which ones you have configured shorter which may be closing the connection earlier. Amos
[squid-users] Connection reset when accessing java servlet report page via squid
Hello, We have a problem with the squid when accessing a servlet page through the squid proxy. It is report page where the inputs are taken from the user and the servlet manipulates the report and present it in the page. Normally it takes around 45-60 seconds to generate the report. So we are getting the Connection reset' message in firefox and 'Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data' in chrome. But works normally without a proxy. Please suggest a solution for this issue if there is any config change need to be done. Regards, Manoj
[squid-users] Chunked and Range header support
Hi All, I am using squid 3.1.16. 1)Can someone tell me how this version of squid handles chunked responses from a server? Does it wait for all the chunks to arrive and send a consolidated response back to the client or it sends as and when it receives it? 2) Range headers - from my understanding, it looks like they use this for video streaming.. it looks like the client can request a part of the object body to be sent alone to him. Is it correct? In this case, if multiple ranges are requested, is it sent separately or in a consolidated manner? Thanks in advance. Regards, Anita -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Chunked-and-Range-header-support-tp4660869.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Does squid support X-Forwarded-User
On 2/07/2013 6:33 p.m., Blason wrote: Hi Fellas, I know squid support X-Forwarded-For but keen to know if I can use X-Forwarded-User so that user authenticated with Proxy can have filtering done on other Application Control box placed in front of Proxy. And Proxy can only be used for caching functionality No. There is no useful purpose to X-Forwarded-User been presented, all it does is add security problems. Amos
