[squid-users] HIER_NONE on TCP_MISS?

[bump skier]

Hi,

I'm trying to understand the following behavior I'm seeing with Squid
running in accelerator mode. In short, I'm seeing some TCP_MISS for
requests to a static javascript file which is initially cached and returned
as a cache hit. I suspect the missed cache hits are due to the cache size
being too small and the file eventually getting evicted. However, I'm
confused about what I'm seeing in the Squid access log. For some of the
cache misses I can see in the access log that Squid fetches the file from
the configured origin server but for a vast majority of them I see
HIER_NONE even though Squid is actually returning the file.

Under what situations would Squid fetch content from the origin server
during a cache miss but print HIER_NONE?

Sample Requests from Access Log (actual values have been replaced with
placeholders/fake values)

#What I would expect to see for TCP_MISS - Squid fetches from the origin
server
TIMESTAMP 1.2.3.4 TCP_MISS/200 17310 GET http://www.domain.com/myFile.js
FIRSTUP_PARENT/2.4.6.8 application/javascript

#Squid still appears to be fetching from the origin server however it
prints HIER_NONE in the access log
TIMESTAMP 4.3.2.1 TCP_MISS/200 17310 GET http://www.domain.com/myFile.js
HIER_NONE/- application/javascript

Thanks in advance for any help you can provide.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] NTLM authentication worked in Squid 2.7.STABLE8 Squid Web Proxy, now need it in v3.5 hosted on Windows server 2k12

[Todd Pearson]

I am hosting the squid proxy on Windows 2K12 server.   Squid 2.7.STABLE8 Squid 
Web Proxy version worked well for authentication until recent Windows 10 update 
killed Sha1.  Now I am upgrading to squid proxy version 3.5.x.x to restore 
authentication.  
The below settings are longer available in the 3.5.x.x version since the 
progams do not exist for the new version:
auth_param ntlm program c:/squid/libexec/mswin_ntlm_auth.exe
external_acl_type win_domain_group %LOGIN 
c:/squid/libexec/mswin_check_ad_group.exe -G

What are the equivalent setting for v 3.5.  Once again I am in windows 
environment.  



___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid caching bad objects

[Alex Rousskov]

On 06/26/2017 10:11 AM, Razor Cross wrote:

> We are using squid 3.5. for our server. Recently we have noticed that
> squid is caching incomplete objects in case of chunked response.
> 
> We have gone through the squid code. It looks likes squid is caching
> incomplete response in case of EOF from the server even though it does
> not receive the last empty chunk.
> 
> 
>  if (eof) // already reached EOF
> return COMPLETE_NONPERSISTENT_MSG;

You are looking at the wrong code. HttpStateData::persistentConnStatus()
and related *_MSG codes do not determine whether the entire object was
received. They determine whether

(a) Squid should expect more response bytes and

(b) The connection can be kept open if no more response bytes are expected.

The COMPLETE_NONPERSISTENT_MSG return value is correct here (I am
ignoring the sad fact that we are abusing the word "complete" to cover
both whole and truncated responses).


> Is this expected? Because of this problem, our server ends up serving
> bad objects to the user.

What you describe sounds like a bug, but the exact code you are quoting
is not responsible for that bug. I di not study this in detail, but I
suspect that the COMPLETE_NONPERSISTENT_MSG case in
HttpStateData::processReplyBody() should be changed to call
StoreEntry::lengthWentBad("missing last-chunk") when lastChunk is false
and HttpStateData::flags.chunked is true.


HTH,

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ACLs allow/deny logic

[Vieri]

From: Amos Jeffries 
>> I'd like to allow by default and deny only according to the ACLs I define.
>> 

>> Here's an example with Telegram. I'd like to deny all 
>> application/octet-stream mime types in requests 

>> and replies except for a set of IP addresses or domains.>
> Er, deny is the opposite of allow. So your "example" is to demonstrate 
> the _opposite_ of what you want?
> 

> Not to mention that what you want is the opposite of a well-known 

> Security Best-Practice. Well, your call, but when things go terribly 
> wrong don't say you weren't warned.

My sentence was misleading, I suppose.
My squid.conf has the following structure (which I believe is close to the 
default for a caching http proxy):

ACL definitions

http_access deny ...
http_reply_access deny ...

http_access deny intercepted !localnet

http_access allow localnethttp_access deny all

Is there anything wrong with this?

Vieri
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid 3.5.26 for Microsoft Windows 64-bit is available

[Rafael Akchurin]

Greetings everyone,

The CygWin based build of Squid proxy for Microsoft Windows version 3.5.26 is 
now available (amd64 only!).

* Original release notes are at 
http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.26-RELEASENOTES.html .
* Ready to use MSI package can be downloaded from http://squid.diladele.com .
* List of open issues for the installer - 
https://github.com/diladele/squid-windows/issues

Thanks a lot for Squid developers for making this great software!

Please join our humble efforts to provide ready to run MSI installer for Squid 
on Microsoft Windows with all required dependencies at GitHub -
https://github.com/diladele/squid-windows . Report all issues/bugs/feature 
requests at GitHub project.
Issues about the *MSI installer only* can also be reported to 
supp...@diladele.com .

Best regards,
Rafael Akchurin
Diladele B.V.
https://www.diladele.com

--
Please take a look at Web Safety - our ICAP based web filter server for Squid 
proxy.
The upcoming version 5.1 with "Bypass Blocked Page" functionality is available 
from https://www.diladele.com/download_next_version.html



___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ACLs allow/deny logic

[Amos Jeffries]

On 26/06/17 20:46, Vieri wrote:

Hi,

I'd like to allow by default and deny only according to the ACLs I define.

Here's an example with Telegram. I'd like to deny all application/octet-stream 
mime types in requests and replies except for a set of IP addresses or domains.


Er, deny is the opposite of allow. So your "example" is to demonstrate 
the _opposite_ of what you want?



Not to mention that what you want is the opposite of a well-known 
Security Best-Practice. Well, your call, but when things go terribly 
wrong don't say you weren't warned.



Anyhow ...



acl denied_restricted1_mimetypes_req req_mime_type -i 
"/usr/local/proxy-settings/denied.restricted1.mimetypes"
acl denied_restricted1_mimetypes_rep rep_mime_type -i 
"/usr/local/proxy-settings/denied.restricted1.mimetypes"
acl allowed_restricted1_domains dstdomain -i 
"/usr/local/proxy-settings/allowed.restricted1.domains"
acl allowed_restricted1_ips dst 
"/usr/local/proxy-settings/allowed.restricted1.ips"
http_access deny denied_restricted1_mimetypes_req !allowed_restricted1_domains 
!allowed_restricted1_ips
http_reply_access deny denied_restricted1_mimetypes_rep 
!allowed_restricted1_domains !allowed_restricted1_ips



A line of ACLS is an "AND" condition:

 http_access deny (if) X (and) Y (and) Z

This configuration will deny the mime types except when they come from 
certain IPs *AND* are going to certain domains.



To enact your stated policy you need to do it this way:

 http_access allow allowed_restricted1_domains \
denied_restricted1_mimetypes_req

 http_access allow allowed_restricted1_ips \
denied_restricted1_mimetypes_req

 http_access deny denied_restricted1_mimetypes_req


.. same sort of thing for the reply lines.



# cat /usr/local/proxy-settings/allowed.restricted1.domains
.telegram.org

# cat /usr/local/proxy-settings/allowed.restricted1.ips
149.154.167.91
149.154.165.120

# cat /usr/local/proxy-settings/denied.restricted1.mimetypes
^application/octet-stream$

I see this in access.log:

1498463484.530413 10.215.144.237 TCP_DENIED_REPLY/403 4085 POST 
http://149.154.165.120/api - ORIGINAL_DST/149.154.165.120 text/html

I searched for the relevant parts in cache.log:





I see 2 apparently contradictory log messages (well, for me that is -- I'm 
still learning how to read the log):
The reply for POST http://149.154.165.120/api is DENIED, because it matched 
allowed_restricted1_ips
The reply for POST http://149.154.165.120/api is ALLOWED, because it matched 
allowed_restricted1_ips

Why is this happening?


The servers reply (application/octet-stream) is being denied, and the 
Squid generated error page (text/html) is being allowed.


When a default / implicit action is being done the "matched X" shows the 
name of the last ACL processed - that ACL non-matching was the reason 
for the default/implicit action happening.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] ACLs allow/deny logic

[Vieri]

Hi,

I'd like to allow by default and deny only according to the ACLs I define.

Here's an example with Telegram. I'd like to deny all application/octet-stream 
mime types in requests and replies except for a set of IP addresses or domains.

acl denied_restricted1_mimetypes_req req_mime_type -i 
"/usr/local/proxy-settings/denied.restricted1.mimetypes"
acl denied_restricted1_mimetypes_rep rep_mime_type -i 
"/usr/local/proxy-settings/denied.restricted1.mimetypes"
acl allowed_restricted1_domains dstdomain -i 
"/usr/local/proxy-settings/allowed.restricted1.domains"
acl allowed_restricted1_ips dst 
"/usr/local/proxy-settings/allowed.restricted1.ips"
http_access deny denied_restricted1_mimetypes_req !allowed_restricted1_domains 
!allowed_restricted1_ips
http_reply_access deny denied_restricted1_mimetypes_rep 
!allowed_restricted1_domains !allowed_restricted1_ips

# cat /usr/local/proxy-settings/allowed.restricted1.domains
.telegram.org

# cat /usr/local/proxy-settings/allowed.restricted1.ips
149.154.167.91
149.154.165.120

# cat /usr/local/proxy-settings/denied.restricted1.mimetypes
^application/octet-stream$

I see this in access.log:

1498463484.530413 10.215.144.237 TCP_DENIED_REPLY/403 4085 POST 
http://149.154.165.120/api - ORIGINAL_DST/149.154.165.120 text/html

I searched for the relevant parts in cache.log:

# grep -e "^2017/06/26 09:51:24.48[0-4]" /var/log/squid/cache.test.log_JL
2017/06/26 09:51:24.480 kid1| 28,3| Checklist.cc(70) preCheck: 0x80de0648 
checking slow rules
2017/06/26 09:51:24.480 kid1| 28,5| Acl.cc(138) matches: checking 
http_reply_access
2017/06/26 09:51:24.480 kid1| 28,5| Checklist.cc(400) bannedAction: Action 
'DENIED/0is not banned
2017/06/26 09:51:24.480 kid1| 28,5| Acl.cc(138) matches: checking 
http_reply_access#1
2017/06/26 09:51:24.480 kid1| 28,5| Acl.cc(138) matches: checking 
denied_filetypes
2017/06/26 09:51:24.480 kid1| 28,3| RegexData.cc(51) match: 
aclRegexData::match: checking '/api'
2017/06/26 09:51:24.480 kid1| 28,3| RegexData.cc(62) match: 
aclRegexData::match: looking for 
'(\.ade(\?.*)?$)|(\.adp(\?.*)?$)|(\.app(\?.*)?$)|(\.asd(\?.*)?$)|(\.asf(\?.*)?$)|(\.asx(\?.*)?$)|(\.avi(\?.*)?$)|(\.bas(\?.*)?$)|(\.bat(\?.*)?$)|(\.cab(\?.*)?$)|(\.chm(\?.*)?$)|(\.cmd(\?.*)?$)|(\.cpl(\?.*)?$)|(\.dll$)|(\.exe(\?.*)?$)|(\.fxp(\?.*)?$)|(\.hlp(\?.*)?$)|(\.hta(\?.*)?$)|(\.hto(\?.*)?$)|(\.inf(\?.*)?$)|(\.ini(\?.*)?$)|(\.ins(\?.*)?$)|(\.iso(\?.*)?$)|(\.isp(\?.*)?$)|(\.jse(.?)(\?.*)?$)|(\.jse(\?.*)?$)|(\.lib(\?.*)?$)|(\.lnk(\?.*)?$)|(\.mar(\?.*)?$)|(\.mdb(\?.*)?$)|(\.mde(\?.*)?$)|(\.mp3(\?.*)?$)|(\.mpeg(\?.*)?$)|(\.mpg(\?.*)?$)|(\.msc(\?.*)?$)|(\.msi(\?.*)?$)|(\.msp(\?.*)?$)|(\.mst(\?.*)?$)|(\.ocx(\?.*)?$)|(\.pcd(\?.*)?$)|(\.pif(\?.*)?$)|(\.prg(\?.*)?$)|(\.reg(\?.*)?$)|(\.scr(\?.*)?$)|(\.sct(\?.*)?$)|(\.sh(\?.*)?$)|(\.shb(\?.*)?$)|(\.shs(\?.*)?$)|(\.sys(\?.*)?$)|(\.url(\?.*)?$)|(\.vb(\?.*)?$)|(\.vbe(\?.*)?$)|(\.vbs(\?.*)?$)|(\.vcs(\?.*)?$)|(\.vxd(\?.*)?$)|(\.wmd(\?.*)?$)|(\.wms(\?.*)?$)|(\.wmv(\?.*)?$)|(\.wmz(\?.*)?$)|(\.wsc(\?.*)?$)|(\.wsf(\?.*)?$)|(\.wsh(\?.*)?$)'
2017/06/26 09:51:24.480 kid1| 28,3| Acl.cc(158) matches: checked: 
denied_filetypes = 0
2017/06/26 09:51:24.480 kid1| 28,3| Acl.cc(158) matches: checked: 
http_reply_access#1 = 0
2017/06/26 09:51:24.480 kid1| 28,5| Checklist.cc(400) bannedAction: Action 
'DENIED/0is not banned
2017/06/26 09:51:24.480 kid1| 28,5| Acl.cc(138) matches: checking 
http_reply_access#2
2017/06/26 09:51:24.480 kid1| 28,5| Acl.cc(138) matches: checking 
denied_mimetypes_rep
2017/06/26 09:51:24.480 kid1| 28,3| RegexData.cc(51) match: 
aclRegexData::match: checking 'application/octet-stream'
2017/06/26 09:51:24.480 kid1| 28,3| RegexData.cc(62) match: 
aclRegexData::match: looking for 
'(^application/ecmascript$)|(^application/oebps-package+xml$)|(^application/vnd.amazon.ebook$)|(^application/vnd.android.package-archive$)|(^application/vnd.gmx$)|(^application/vnd.google-earth.kml+xml$)|(^application/vnd.google-earth.kmz$)|(^application/vnd.ms-cab-compressed$)|(^application/vnd.ms-excel.addin.macroenabled.12$)|(^application/vnd.ms-excel.sheet.binary.macroenabled.12$)|(^application/vnd.ms-excel.sheet.macroenabled.12$)|(^application/vnd.ms-excel.template.macroenabled.12$)|(^application/vnd.ms-powerpoint.addin.macroenabled.12$)|(^application/vnd.ms-powerpoint.presentation.macroenabled.12$)|(^application/vnd.ms-powerpoint.slide.macroenabled.12$)|(^application/vnd.ms-powerpoint.slideshow.macroenabled.12$)|(^application/vnd.ms-powerpoint.template.macroenabled.12$)|(^application/vnd.ms-wpl$)|(^application/vnd.ms.wms-hdr.asfv1$)|(^application/vnd.realvnc.bed$)|(^application/vnd.tmobile-livetv$)|(^application/x-authorware-bin$)|(^application/x-cab$)|(^application/x-iso9660-image$)|(^application/x-mms-framed$)|(^application/x-ms-wm$)|(^application/x-msdos-program$)|(^application/x-msdownload$)|(^application/x-shar$)|(^application/x-vbs$)|(^text/vbs$)|(^text/vbscript$)'
2017/06/26 09:51:24.480 kid1| 28,3| Acl.cc(158) matches: checked: 
denied_mimetypes_rep = 0