Re: [squid-users] varyEvaluateMatch

[Alberto Perez]

I have a lot of these too. I rebuild the entire cache dir after I
start seeing this, it was right after I upgrade squid to 3.5.12, and I
remember also I remove ignore-no-cache parameter from refresh_pattern
directives as "squid3 -k parse" instructed to do.

'
2016/02/29 21:50:41| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://cse.google.com/adsense/search/async-ads.js'
'accept-encoding="gzip,deflate"'
2016/02/29 21:50:41| clientProcessHit: Vary object loop!
2016/02/29 21:50:49| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://wd-edge.sharethis.com/button/checkOAuth.esi'
'accept-encoding="gzip,%20deflate"'
2016/02/29 21:50:49| clientProcessHit: Vary object loop!
2016/02/29 21:50:50| varyEvaluateMatch: Oops. Not a Vary match on
second attempt,
'http://w.sharethis.com/button/css/buttons.ab966a004186897711de4a5ed256c924.css'
'accept-encoding="gzip,deflate"'
2016/02/29 21:50:50| clientProcessHit: Vary object loop!
2016/02/29 21:50:50| varyEvaluateMatch: Oops. Not a Vary match on
second attempt,
'http://edge.sharethis.com/share4x/index.a8fc48294ca4cc10eb440189d9f22d7c.html'
'accept-encoding="gzip,deflate"'
2016/02/29 21:50:50| clientProcessHit: Vary object loop!
2016/02/29 21:50:50| varyEvaluateMatch: Oops. Not a Vary match on
second attempt,
'http://w.sharethis.com/share4x/js/st.125da8bfe70ad3e00a35a6e4c65d8bc5.js'
'accept-encoding="gzip,deflate"'
2016/02/29 21:50:50| clientProcessHit: Vary object loop!
2016/02/29 21:50:51| varyEvaluateMatch: Oops. Not a Vary match on
second attempt,
'http://w.sharethis.com/share4x/js/local/es/message.e5cdbb7671ab474b46221772dfb45ce3.js'
'accept-encoding="gzip,deflate"'
2016/02/29 21:50:51| clientProcessHit: Vary object loop!
2016/02/29 21:50:59| varyEvaluateMatch: Oops. Not a Vary match on
second attempt,
'http://active.cache.el-mundo.net/fonts/pt_serif/PTF55F.woff'
'accept-encoding="gzip,%20deflate"'

Perhaps is something in the refresh_patterns config?

This is my snippet related to js files

refresh_pattern -i \.(css|js)$ 14400 99% 518400 override-expire
override-lastmod ignore-reload  ignore-no-store reload-into-ims
ignore-must-revalidate ignore-private

refresh_all_ims

Regards

On 2/29/16, Amos Jeffries  wrote:
> On 1/03/2016 3:23 a.m., joe wrote:
>> Squid Cache: Version 3.5.15-20160224-r13996
>> with 4447  patch
>> 2016/02/29 16:41:51 kid1| varyEvaluateMatch: Oops. Not a Vary match on
>> second attempt, 'http://cstatic.weborama.fr/iframe/external_libs.js'
>> 'accept-encoding="gzip,%20deflate"'
>> 2016/02/29 16:41:51 kid1| clientProcessHit: Vary object loop!
>> 2016/02/29 16:41:52 kid1| varyEvaluateMatch: Oops. Not a Vary match on
>> second attempt, 'http://cstatic.weborama.fr/iframe/external_libs.js'
>> 'accept-encoding="gzip,%20deflate"'
>
> I'm seeing you coming up with quite a mix of cache related issues
> recently. And all this following the bug 4447 issues.
>
> Have you tried erasing your disk cache (or de-configuring it anyway) and
> rebuilding a new one from scratch using the latest fixed Squid build?
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Help with cachemgr and nginx

[Alberto Perez]

Hi everyone,

I am trying to setup cachemgr.cgi in my squid 3.5.2 box with nginx as web
server. I was able to server first page of cachemgr but I can't get further
this page, every credentials I try shows me the same page with no errors.
(attach image)


in my squid.conf file

cache_mgr alberto
cachemgr_passwd managerforsquid40a all

nginx config

location /cgi-bin/ {
alias /usr/local/squid/libexec;
index cachemgr.cgi;
gzip off;
#include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
fastcgi_param  SCRIPT_FILENAME
/usr/local/squid/libexec/cachemgr.cgi;
}

I successfully run other cgi perl scripts with that same configuration
(squish.cgi, but this one is only one page, so i could be missing something
here)

No access denied from nginx response so I asume this is not the problem
here.


Have anyone experience with a similar setup.

Thanks a lot in advance for the time.

Alberto
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] how to realize (external_acl_type) logout without squid -k reconfigure

[Alberto Perez]

Hi, john
To be able to make the logout action instantly effective you should
change your external acl ttl to a lower value, for example 3, in which
case logout will be effective after 3 sec (reasonable value I think).

You should also worry about performance of credentials check because
squid will make this check each 3 sec for each client.

Hope it helps

Regards


On 4/17/15, johnzeng johnzeng2...@yahoo.com wrote:

 Hello Dear Sir :

 We tested external_acl_type for web authentication via writing check.php ,

 and login feature is fine

 squid config is

 external_acl_type session ipv4 concurrency=10 ttl=3600 negative_ttl=0
 cache=1048576 %SRC /opt/check.php

 check.php ( for login) is

 fwrite(STDOUT, $source_ip. OK\n);


 But if we don't squid -k reconfigure , logout feature is bad for me how
 to update Ok or ERR cache entry without squid -k reconfigure

 check.php ( for logout ) is

 fwrite(STDOUT, $source_ip. ERR\n);



 if possible , please give me some advisement





 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] help with tcp_outgoing_address trying to balace traffic based on username

[Alberto Perez]

Hi everyone
I've been trying to make a traffic load balancing between two links
based on username using tcp_outgoing_address

My squid setup only use authorization with an external_acl which
returns the username based on the client ip.

In my first failure trying to setup this, I found (with the help of
Amos) that tcp_outgoing_address only works with fast acls, so ext_user
acl doesn't work there, Amos recommend me to use NOTE acl for matching
annotations in transactions, at first it looks to work fine but now I
am realizing that only a very small part of the traffic is been going
through the correct link.

I can't find any good documentation related to note acl with some
samples or cases of usage, so I hope some one can correct my config
and/or point me the right direction to achieve this.


Inside my external acl I mark some users like this.
// if username is one of fast users
fwrite(STDOUT, OK user=$username clt_conn_tag=55\n);

So theoretically both user and clt_conn_tag values can be matched with
NOTE ACL, as far as i know, I use also clt_conn_tag for testing but it
should be enough with user mark.


#this is fast users declaration tests ive tried with both commented
and uncommented acl.
acl nodo_users ext_user /etc/squid3/users/nodo_users
#acl fast_users note clt_conn_tag 55
acl fast_users note user /etc/squid3/users/nodo_users

nodo_users is a list of fast users,  the tcp_outgoing_address is used
like this

#fast link for fast users
tcp_outgoing_address xx.xx.xx.01 fast_users
# default slow link
tcp_outgoing_address xx.xx.xx.02

Traffic of fast users is intermitent between the two links and much
more using the slow link than the fast link (opposite than expected).
External ACL TTL is 3 sec, so I asume that the correct balancing is
made only when squid checks de acl and mark the request, but this mark
isnt persistent.

Please, any help with this will be appreciated.

Thanks
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] issue with tcp_outgoing_address, trying to balace traffic based on username

[Alberto Perez]

Hi everyone
I've been trying to make a traffic load balancing between two links
based on username using tcp_outgoing_address

My squid setup only use authorization with an external_acl which
returns the username based on the client ip.

In my first failure trying to setup this, I found (with the help of
Amos) that tcp_outgoing_address only works with fast acls, so ext_user
acl doesn't work there, Amos recommend me to use NOTE acl for matching
annotations in transactions, at first it looks to work fine but now I
am realizing that only a very small part of the traffic is been going
through the correct link.

I can't find any good documentation related to note acl with some
samples or cases of usage, so I hope some one can correct my config
and/or point me the right direction to achieve this.


Inside my external acl I mark some users like this.
// if username is one of fast users
fwrite(STDOUT, OK user=$username clt_conn_tag=55\n);

So theoretically both user and clt_conn_tag values can be matched with
NOTE ACL, as far as i know, I use also clt_conn_tag for testing but it
should be enough with user mark.


#this is fast users declaration tests ive tried with both commented
and uncommented acl.
acl nodo_users ext_user /etc/squid3/users/nodo_users
#acl fast_users note clt_conn_tag 55
acl fast_users note user /etc/squid3/users/nodo_users

nodo_users is a list of fast users,  the tcp_outgoing_address is used
like this

#fast link for fast users
tcp_outgoing_address xx.xx.xx.01 fast_users
# default slow link
tcp_outgoing_address xx.xx.xx.02

Traffic of fast users is intermitent between the two links and much
more using the slow link than the fast link (opposite than expected).
External ACL TTL is 3 sec, so I asume that the correct balancing is
made only when squid checks de acl and mark the request, but this mark
isnt persistent.

Please, any help with this will be appreciated.

Thanks
Alberto
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] WARNING: 1 swapin MD5 mismatches and BUG 3279: HTTP reply without Date:

[Alberto Perez]

Another one here not using SMP, and using aufs.

I stopped seen this issue frequently when I reduced my cache size,
from 70 GB to 30 GB now.

Regards

On 3/19/15, Dan Charlesworth d...@getbusi.com wrote:
 Hey Eliezer

 I don't actually use SMP. I could be wrong about the aufs thing; I haven't
 personally tested—and don't currently plan to test—any other cache types. I
 just gleaned that from the comments in the bug reports.

 Kind regards
 Dan


 On 20 March 2015 at 13:45, Eliezer Croitoru elie...@ngtech.co.il wrote:

 Hey Dan and John,

 If indeed this bug is only for UFS\AUFS cache_dir then I would try to
 make
 sure that large-rock will not sustain the same issue.

 I have not seen in any of the bug reports anything that would reproduce
 the issue.
 To make sure the issue is understood and can or cannot be reproduced
 using
 ufs\aufs will give one direction.
 I would try to test large rock in my next testing round with SMP but if
 anyone has some option to test it first I will be glad if it will be done
 to make sure ufs\aufs is the culprit.

 Also if indeed it's with aufs\ufs only with SMP then it means that the
 issue is related to the way SMP can make a ufs\aufs cache_dir dirty and
 there for the answer would be pretty simple to the issue in hands.

 Eliezer

 On 20/03/2015 00:32, Dan Charlesworth wrote:

 Hi John

 This bug has been affecting me on an off for a while as well. I believe
 it
 only affects aufs and, unfortunately, has been around for years.

 See:http://bugs.squid-cache.org/show_bug.cgi?id=3279
 And see:http://bugs.squid-cache.org/show_bug.cgi?id=3483


 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Refresh ACL list only

[Alberto Perez]

I did this once, with external acl, you can specify a ttl of each
external acl and also make your acl reload the list any time you want.

My external acl loaded users (in your case domains) from a list in
hard disk to memory, and then each time squid asked acl verify for a
preset  elapsed time to reuse loaded in memory list or load it again
before answer.

Be aware that dst-domain is a fast acl and you will turn this check
into a slow acl so you should consider if you really need this.

I can provide more details if you decide to use it like this.

Hope it helps



On 3/17/15, Samuel Anderson s...@idsdoc.com wrote:
 Hello all,

 Does anyone know of a way to reload a single ACL list? I have a very
 complicated and large config file that takes around 30 seconds to reload
 when I run the (squid3 -k reconfigure) command. I have several ACL lists
 that need to be updated throughout the day and it would be nice if I could
 only reload those ACL lists and not the entire config. Its problematic
 because while its reloading, the server is effectively down and disrupts
 Internet access for the rest of the users. Below is a small sample of the
 lists that will be updated. If I could add a TTL to the lists so squid
 would reload them periodically without a full reconfigure would be ideal.



 acl GLOBAL-WHITELIST dstdomain /etc/squid3/whitelists/GLOBAL-WHITELIST
 acl UNRESTRICTED-WHITELIST dstdomain
 /etc/squid3/whitelists/UNRESTRICTED-WHITELIST
 acl DEV-WHITELIST dstdomain /etc/squid3/whitelists/DEV-WHITELIST
 acl SALES-WHITELIST dstdomain /etc/squid3/whitelists/SALES-WHITELIST


 Thanks


 --
 Samuel Anderson  |  Information Technology Administrator  |  International
 Document Services

 IDS  |  11629 South 700 East, Suite 200  |  Draper, UT 84020-4607

 --
 CONFIDENTIALITY NOTICE:
 This e-mail and any attachments are confidential. If you are not an
 intended recipient, please contact the sender to report the error and
 delete all copies of this message from your system.  Any unauthorized
 review, use, disclosure or distribution is prohibited.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid intercept config

[Alberto Perez]

Can you share more details about Agressive dynamic content caching
requires some special tweaks I am very interested.

Thanks



On 3/13/15, Yuri Voinov yvoi...@gmail.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1



 13.03.15 23:33, Amos Jeffries пишет:
 On 14/03/2015 5:47 a.m., Monah Baki wrote:

 snip

 half_closed_clients off quick_abort_min 0 KB quick_abort_max 0
 KB vary_ignore_expire on reload_into_ims on memory_pools off
 cache_mem 4096 MB visible_hostname isn-phc-cache
 minimum_object_size 0 bytes

 maximum_object_size 512 MB maximum_object_size 512 KB

 KB value overwriting MB value.


 ipcache_size 1024 ipcache_low 90 ipcache_high 95 cache_swap_low
 98 cache_swap_high 100 fqdncache_size 16384 retry_on_error on
 offline_mode off logfile_rotate 10 dns_nameservers 8.8.8.8
 41.78.211.30




 access.log:

 1426267535.210198 10.0.0.23 TCP_MISS/200 412 GET
 http://jadserve.postrelease.com/trk.gif? -
 ORIGINAL_DST/54.225.133.227 image/gif 1426267535.211198
 10.0.0.23 TCP_MISS/200 412 GET
 http://jadserve.postrelease.com/trk.gif? -
 ORIGINAL_DST/54.225.133.227 image/gif 1426267535.211198
 10.0.0.23 TCP_MISS/200 412 GET
 http://jadserve.postrelease.com/trk.gif? -
 ORIGINAL_DST/54.225.133.227 image/gif 1426267535.223301
 10.0.0.23 TCP_MISS/200 222 GET
 http://rma-api.gravity.com/v1/beacons/log? -
 ORIGINAL_DST/80.239.148.18 text/html 1426267535.244195
 10.0.0.23 TCP_MISS/200 412 GET
 http://jadserve.postrelease.com/trk.gif? -
 ORIGINAL_DST/54.225.133.227 image/gif


 Lots of Akamai hosted requests. Akamai play tricks with DNS
 responses.
 In my installation I've used local Unbound DNS cache and, before it,
 forced DNS interception to him with Cisco. :)

 So, I don't care about any hosts DNS quirks. ;)


 Check your cache.log for security warnings;
 http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery

 Note that objects failing the Host validation are not cacheable.


 1426267535.333423 10.0.0.23 TCP_MISS/200 1420 GET
 http://hpr.outbrain.com/utils/get? - ORIGINAL_DST/50.31.185.42
 text/x-json 1426267535.345412 10.0.0.23 TCP_MISS/200 11179
 GET http://p.visualrevenue.com/? - ORIGINAL_DST/50.31.185.40
 text/javascript 1426267535.346411 10.0.0.23 TCP_MISS/200 423
 GET http://t1.visualrevenue.com/? - ORIGINAL_DST/64.74.232.44
 image/gif

 Not sure about them. Maybe genuine MISS, maybe not.

 Agressive dynamic content caching requires some special tweaks. ;)


 It could also be the issues Antony pointed out, with the objects
 just naturally not being cacheable.


 1426267535.363128 10.0.0.23 TCP_REFRESH_UNMODIFIED/304 327
 GET
 http://z.cdn.turner.com/cnn/.element/widget/video/videoapi/api/js/vendor/jquery.ba-bbq.js


 - - ORIGINAL_DST/80.239.152.153 application/x-javascript

 There is a hit.

 I guess you are new to Squid-3 ? Squid is HTTP/1.1 compliant now
 and the caching rules are slightly different from requirements on
 HTTP/1.0 software. A lot of content that previously could not be
 stored now can (authenticated, private, no-cache, etc.). But being
 sensitive info also requires revalidation in order to be used, so
 they show up like the above.

 Amos

 ___ squid-users mailing
 list squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users

 -BEGIN PGP SIGNATURE-
 Version: GnuPG v2

 iQEcBAEBAgAGBQJVAy/qAAoJENNXIZxhPexGOUEH/2yt1ql+ndo1We1E06LvIZl7
 4PXY1kzuHT6EpOYO9LpLKtE+dPNYJuHKiUEF2hAGz5DP/heKq8PFRBTkMD18sueN
 jm+UfP8BdxgRYuiQWtWNteV0gbH4nOBeJ6QwqlEHMwcsdPtkwWCGA0MS6co+IXKb
 poouP6xQoNddx/UKicu6PQZDj5HRmynTP2c0mJuFEdlQxONgFiP4mqSFBwWhH/B/
 hhdSfxg53xfQ+2B5TsVrKyxmJoIYpHgFZid/pk+Q2bb0WIy8bhHA72EHPjIu5K5Z
 wobLGng+oE0i2erqtZiFR8daGdKcRW7FDYzHi+LJEHJj3i+z0mRIQkGTn3Nxfhg=
 =Cnai
 -END PGP SIGNATURE-
 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid intercept config

[Alberto Perez]

Thanks a lot Yuri,
I made some merge with my config and some of this options, I will see now
how HIT rate it goes, my squid run so limited of bandwidth that I need to
be as much aggressive as I can caching the content.

Thanks again for sharing, very appreciated

Alberto

On Fri, Mar 13, 2015 at 4:01 PM, Yuri Voinov yvoi...@gmail.com wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 This is know-how to himself. ;)

 To be serious,

 you must carefully play with refresh_pattern(s), and some squid.conf
 parameters (and also with store ID feature) to get higher HIT ratio.

 Just for example (this is NOT complete config! No responsibility or
 any guarantees in case of simple copy-n-pasted into your configs! This
 is AS IS example!):

 # Keep swf in cache even if asked not to
 refresh_pattern -i \.(swf)(\?|$)10080   90% 43200
  override-expire
 ignore-reload reload-into-ims ignore-private
 # .NET cache
 refresh_pattern -i \.(as(h|p)x?)(\?|$)  10080   90% 43200
  reload-into-ims
 # Updates: Windows, Adobe, Java
 refresh_pattern -i
 microsoft.com/.*\.(cab|exe|ms[i|u|f|p]|asf|wm[v|a]|dat|zip)
4320
 80% 43200   reload-into-ims
 refresh_pattern -i
 windowsupdate.com/.*\.(cab|exe|ms[i|u|f|p]|asf|wm[v|a]|dat|zip)
 4320 80% 43200  reload-into-ims
 refresh_pattern -i
 my.windowsupdate.website.com/.*\.(cab|exe|ms[i|u|f|p]|asf|wm[v|a]|dat|zip)
 4320 80% 43200  reload-into-ims
 refresh_pattern -i adobe.com/.*\.(zip|exe)  432080% 43200
  reload-into-ims
 refresh_pattern -i java.com/.*\.(zip|exe)   432080% 43200
  reload-into-ims
 refresh_pattern -i sun.com/.*\.(zip|exe)432080% 43200
  reload-into-ims
 refresh_pattern -i google\.com.*\.(zip|exe) 432080% 43200
  reload-into-ims
 refresh_pattern -i macromedia\.com.*\.(zip|exe) 432080% 43200
 reload-into-ims
 # Other long-lived items
 refresh_pattern -i
 \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|webp|flv|mp4)(\?|$)
   14400
 99% 518400  ignore-no-store override-expire ignore-reload
 reload-into-ims ignore-private ignore-must-revalidate
 refresh_pattern -i
 \.((m?|x?|s?)htm(l?)|css|js|xml|php|json)(\?|$) 10080
  90% 86400
 ignore-no-store override-expire override-lastmod reload-into-ims
 ignore-private ignore-must-revalidate
 # Default patterns
 refresh_pattern -i (/cgi-bin/|\?)   0   0%  0
 refresh_pattern .   0   20% 10080   override-lastmod
 reload-into-ims

 The example above also requires some additional cached-related
 parameters to be changed.

 Also, you strictly recommended to research average users activity AND
 play around VARY http headers.

 And others.

 Each squid setup is place-specific. And depending your access/deny
 lists, security policy, users/network activity etc.etc.etc.

 WBR, Yuri

 PS. Your question has NO simple answer. Beware - copy-n-paste any
 foreign config can not guarantee the same results for YOU.

 14.03.15 1:52, Alberto Perez пишет:
  Can you share more details about Agressive dynamic content
  caching requires some special tweaks I am very interested.
 
  Thanks
 
 
 
  On 3/13/15, Yuri Voinov yvoi...@gmail.com wrote:
 
 
  13.03.15 23:33, Amos Jeffries пишет:
  On 14/03/2015 5:47 a.m., Monah Baki wrote:
 
  snip
 
  half_closed_clients off quick_abort_min 0 KB
  quick_abort_max 0 KB vary_ignore_expire on reload_into_ims
  on memory_pools off cache_mem 4096 MB visible_hostname
  isn-phc-cache minimum_object_size 0 bytes
 
  maximum_object_size 512 MB maximum_object_size 512 KB
 
  KB value overwriting MB value.
 
 
  ipcache_size 1024 ipcache_low 90 ipcache_high 95
  cache_swap_low 98 cache_swap_high 100 fqdncache_size 16384
  retry_on_error on offline_mode off logfile_rotate 10
  dns_nameservers 8.8.8.8 41.78.211.30
 
 
 
 
  access.log:
 
  1426267535.210198 10.0.0.23 TCP_MISS/200 412 GET
  http://jadserve.postrelease.com/trk.gif? -
  ORIGINAL_DST/54.225.133.227 image/gif 1426267535.211
  198 10.0.0.23 TCP_MISS/200 412 GET
  http://jadserve.postrelease.com/trk.gif? -
  ORIGINAL_DST/54.225.133.227 image/gif 1426267535.211
  198 10.0.0.23 TCP_MISS/200 412 GET
  http://jadserve.postrelease.com/trk.gif? -
  ORIGINAL_DST/54.225.133.227 image/gif 1426267535.223
  301 10.0.0.23 TCP_MISS/200 222 GET
  http://rma-api.gravity.com/v1/beacons/log? -
  ORIGINAL_DST/80.239.148.18 text/html 1426267535.244195
  10.0.0.23 TCP_MISS/200 412 GET
  http://jadserve.postrelease.com/trk.gif? -
  ORIGINAL_DST/54.225.133.227 image/gif
 
 
  Lots of Akamai hosted requests. Akamai play tricks with DNS
  responses.
  In my installation I've used local Unbound DNS cache and, before
  it, forced DNS interception to him with Cisco. :)
 
  So, I don't care about any hosts DNS quirks. ;)
 
 
  Check your cache.log for security warnings;
  http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery
 
 
 
 Note that objects failing the Host validation are not cacheable.
 
 
  1426267535.333423 10.0.0.23 TCP_MISS/200 1420

Re: [squid-users] issue with tcp_outgoing_address and external acl

[Alberto Perez]

Thank you Amos,

As always you hit it, it worked fine with note acl.

I really appreciate your time to support this community, great help
today. Thanks a lot

God Bless you

Alberto


On 3/5/15, Amos Jeffries squ...@treenet.co.nz wrote:
 On 6/03/2015 10:35 a.m., Alberto Perez wrote:
 Thanks Amos  for the link, I understand tcp_outgoing_address only
 works with fast acl and external acls are slow.

 In order to mitigate this fact and achieve my purpose of share traffic
 among two links depending only of username, who can recommend me a
 workaround?


 In the current (3.4+) Squid you can use the note ACL to check user=
 exists. It is a fast ACL check and does not do anything to trigger auth
 when its absent. It will match usernames added by non-auth helpers as well.

 To specifically limit it to HTTP authenticated users you can also check
 the raw HTTP Proxy-Authentication header contents with req_header ACL
 type.

 Amos


 I was working in mantaining of IPs for those users special, once the
 user login or logout from captive portal I update the list if IPs to
 be used as SRC acl combined with tcp_outgoing_address and worked like
 a charm except for the fact that squid only notice this change if I
 reload configuration, which is a heavy reason to consider another
 solutions.

 It is possible to setup a ttl for this SRC acl, how can I make squid
 note the change in this list without reloading configuration.

 SRC is the client IP the request message was received from. Its part of
 the mesage, there is nothing stored to have a TTL.

 Amos


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] issue with tcp_outgoing_address and external acl

[Alberto Perez]

Hi and thanks to all members in this community for the great support.


Im trying to send traffic for some users through a faster link using
tcp_outgoing_address, I found this directive very usefull and suitable
for this need.

I use a captive portal and my squid only use autorization, so users
are not available with standard ident method.

I wrote an acl to simulate ident process by reading to memory a list
of users from a file and matching external user returned by active
session external acl. For general purposes of autorization this work
fine.

I found that tcp_outgoing_address works good with src ACLs but doesn't
work with this external acl, by the way, has huge ttl so squid doesn't
evaluate it on each request.

Can anyone please tell me if this is possible, or point the right
direction to solve this.

I will be glad to provide more information about my case if is needed.

Thanks a lot in advance

Alberto
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Receiving blank input in External ACL

[Alberto Perez]

Hi to all  and thanks for the time given to read and reply these emails


I am having an strange issue with my external ACL, ocassionally im
getting blank inputs from squid to by proccessed by the external ACL,
for those cases I am returning ERR but squid is complainting showing
in cache.log errors like

2015/02/11 10:01:36| helperHandleRead: unexpected read from
session_active_def #Hlpr5, 4 bytes 'ERR
'

My question is, what should I return in those cases? its normal to
recieve those blanks? how to avoid them if possible?

Im worried about this detail because related to this events (only some
times) squid stop serving to client who fired the event, client has to
change IP to continue surfing.

Here is the definition of the external ACL, its supposed to receive IP

# Set up the normal session helper.
external_acl_type session_active_def concurrency=1 children-max=15
children-startup=12  ipv4 ttl=3 negative_ttl=1 %SRC
/etc/squid3/captive/sessionHelper.php

And here the code

#!/usr/bin/php
?php
error_reporting(0);
$meminstance = new Memcache();
$meminstance-pconnect('127.0.0.1', 11211);
ini_set(memory_limit,($memoryLimit=512).M);
while (!@feof(STDIN)) {
try{

$line = trim(fgets(STDIN));
if (!$line) { shell_exec(echo \SESSION - No client ip error: \
.$line. - . - $(date)  /var/log/squid3/session.log);
fwrite(STDOUT, ERR\n); continue;}

$line = explode( , $line);

$clientip = count($line  1)?$line[1]:false; //1738

if (!$clientip) { shell_exec(echo \SESSION - No client ip error: \
.$line. - . - $(date)  /var/log/squid3/session.log);
fwrite(STDOUT, ERR\n); continue;}

 $username = $meminstance-get($clientip);
$hasInternet = 
$username?$meminstance-get(has_internet_$username):false;
if ($username  $hasInternet){
// extend session
$meminstance-set($clientip, $username, 0,3600); // 
extend 10 min
$meminstance-set(ip_.$username, $clientip, 0, 3600); 
// 10 min
$meminstance-set(has_internet_.$username, true, 0, 
3600); // 10 min
fwrite(STDOUT, OK user=$username\n);
}else{
fwrite(STDOUT, ERR\n);
}
}catch(Exception $e){
fwrite(STDOUT, BH\n);
}
}
exit;
?



Thanks to all
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] External ACL TTL not working as expected

[Alberto Perez]

Thank you Amos for your response,

Sorry to say this solution gives me the same result.

Any other idea?

Thanks a lot

Alberto


On 1/31/15, Amos Jeffries squ...@treenet.co.nz wrote:
 On 31/01/2015 5:14 p.m., Alberto Perez wrote:
 Amos,
 You rock it,
 Thanks a lot, now its working pretty well.

 I was partialy aware of PHP limitation when used to build external
 helpers any way I gave it not too much importance since was working
 fine, I will rewrite code in python ASAP, I appreciate your
 recommendations.

 Please if is not too much to ask, I will like to ask you about another
 issue I have not been able to make it work, im trying to skip access
 logging of a subnet, in old versions of squid (3.1) I was using the
 following configuration


 acl ip_upredes src 10.2.100.0/24

 log_access deny ip_upredes
 log_access allow all


 As of squid 3.4 log_access directive was no longer available I was
 trying to change to

 cache_access_log stdio:/var/log/squid3/access.log
 cache_store_log stdio:/var/log/squid3/store.log
 access_log none ip_upredes


 That should be:

  access_log stdio:/var/log/squid3/access.log squid !ip_upredes

 Amos

 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] External ACL TTL not working as expected

[Alberto Perez]

Hello to everyone,

First of all thanks for the support and for this awesome product.

I'm developing a captive portal with squid 3.4.9 over ubuntu 14.04

Here is my external acl configuration

external_acl_type session_active_def ipv4  %SRC
/etc/squid3/captive/sessionHelper.php  concurrency=100 children=1 -a
-f ttl=1  negative_ttl=0 startup=2 idle=1

acl password external session_active_def

deny_info https://proxy-bump.upr.edu.cu/?url=%u password


And the captive portal autenticate users against Active directory and
save authorization data to memcache server on the same proxy server.

External acl sessionHelper.php  checks for those authorization data in
the memcached server. I have not authentication, only authorization by
ip using that external acl and returning autenticated user so squid
log the username information.

Everything works like a charm except for the frequency squid checks
for those authorization data in the external acl, I has tried every
configuration sample found in the internet but without success, ttl
parameter of external acl doesnt works for me.

Session login and logout doesn't work as expected due to squid doesn't
check this external acl frecuently enough, user can surf after closed
session (removed user authorization data from memcache server) because
squid doesn't authorize anymore same ip requests using the external
acl.

Only way I found to force squid to check for the acl was reloading
configuration like /usr/sbin/squid3 -k reconfigure

But this gives me lot of instability  because squid stop accepting
connections for the time it is reloading configuration, I have lot of
people opening and closing sessions so right now squid is reloading
each minute to cover that demand.

My question: is there a way to force squid to check the external acl
each 1 second or just for every request without reloading
configuration.

I've tried all for over a month and you are my last resort, please
help, and forgive my english also.

Grateful

Alberto
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] External ACL TTL not working as expected

[Alberto Perez]

Amos,
You rock it,
Thanks a lot, now its working pretty well.

I was partialy aware of PHP limitation when used to build external
helpers any way I gave it not too much importance since was working
fine, I will rewrite code in python ASAP, I appreciate your
recommendations.

Please if is not too much to ask, I will like to ask you about another
issue I have not been able to make it work, im trying to skip access
logging of a subnet, in old versions of squid (3.1) I was using the
following configuration


acl ip_upredes src 10.2.100.0/24

log_access deny ip_upredes
log_access allow all


As of squid 3.4 log_access directive was no longer available I was
trying to change to

cache_access_log stdio:/var/log/squid3/access.log
cache_store_log stdio:/var/log/squid3/store.log
access_log none ip_upredes


Starting squid in debug mode shows this

2015/01/30 22:50:15| ACL not found: ip_upredes
FATAL: Bungled /etc/squid3/squid.conf line 98: access_log none ip_upredes

I am suspecting about a missing configuration option when I compiled
squid, so following are my compilation options

configure options:
'--sysconfdir=/etc/squid3'
'--mandir=/usr/share/man'
'--enable-inline'
'--enable-async-io=8'
'--enable-storeio=ufs,aufs,diskd,rock'
'--enable-removal-policies=lru,heap'
'--enable-delay-pools'
'--enable-cache-digests'
'--enable-underscores'
'--enable-icap-client'
'--enable-follow-x-forwarded-for'
'--enable-url-rewrite-helpers=fake'
'--enable-eui'
'--enable-esi'
'--enable-icmp'
'--enable-zph-qos'
'--enable-ecap'
'--enable-ssl'
'--enable-ssl-crtd'
'--disable-translation'
'--with-swapdir=/var/spool/squid3'
'--with-logdir=/var/log/squid3'
'--with-pidfile=/var/run/squid3.pid'
'--with-filedescriptors=65536'
'--with-large-files'
'--with-default-user=proxy'


Again, thanks a lot for this great support, I read a lot of your
answers in this mailing list on the web and I can say you are making a
really GOOD job supporting this community and project.

Thanks, thanks, thanks

Alberto



On 1/30/15, Amos Jeffries squ...@treenet.co.nz wrote:
 On 31/01/2015 9:15 a.m., Alberto Perez wrote:
 Hello to everyone,

 First of all thanks for the support and for this awesome product.


 Welcome. Thank You for getting the use of authorization vs
 authentication correct in your question :-) so nice not to have to start
 with clarifying that.


 I'm developing a captive portal with squid 3.4.9 over ubuntu 14.04

 Here is my external acl configuration

 external_acl_type session_active_def ipv4  %SRC
 /etc/squid3/captive/sessionHelper.php  concurrency=100 children=1 -a
 -f ttl=1  negative_ttl=0 startup=2 idle=1


 http://www.squid-cache.org/Doc/config/external_acl_type/.


 In particular you have the syntax for the helper wrong. The correct
 syntax is:

  external_acl_type name [options] FORMAT /path/to/helper [helper arguments]

 Note the difference between options and helper arguments. Helper
 arguments is part of the command line to be run.


 So this is the correct config line for what you have above is:

  external_acl_type \
   session_active_def \
   concurrency=100 children-max=1 children-startup=1 \
 ipv4 ttl=1 negative_ttl=0 \
   \
   %SRC \
   /etc/squid3/captive/sessionHelper.php -a -f


 I noticed the total number of children was smaller that the number
 indicated to be run at startup. Squid would complain about that as it
 auto-corrects to using startup=1.
 Also, with startup = children the idle parameter is unnecessary. There
 is no amount of child helpers being started later based on load.


 NP: There were a few nasty issues with the Zend engine limiting how long
 a PHP script was allowed to run for. Even the dynamic restart feature
 for Squid helpers was unable to solve that fully. You may want to
 re-write the helper into another language for production use.



 acl password external session_active_def

 deny_info https://proxy-bump.upr.edu.cu/?url=%u password


 Be careful with the use of HTTPS in sessions.

 The initial CONNECT (even the fake-CONNECT Squid uses on intercepted
 traffic) will need accounting for in your session management ACLs. You
 dont want it to be kicking off another session login partway through the
 first.



 HTH
 Amos
 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users