Re: [squid-users] squid with random outgoing ip from pool of 1000 ips

[--Ahmad--]

Ok


but can you confirm me about the list below if its correct ???



should it be like 1/9987 or .088787  formatting ?




acl half10001 random 0.000998001000
acl half10006 random 0.0009930209650350
acl half10011 random 0.0009880657804942
acl half10020 random 0.0009792086759647
acl half10037 random 0.0009626946373158
acl half10043 random 0.0009569328906720
acl half10059 random 0.0009417362622232
acl half10079 random 0.0009230793978373
acl half10082 random 0.0009203129279589
acl half10084 random 0.0009184732224159
acl half10094 random 0.0009093297114627
acl half10098 random 0.0009056978449587
acl half10109 random 0.0008957848329039
acl half10113 random 0.0008922070646991
acl half10114 random 0.0008913148576344
acl half10122 random 0.0008842092457380
acl half10137 random 0.0008710385479118
acl half10154 random 0.0008563487636013
acl half10168 random 0.0008444374977929
acl half10171 random 0.0008419067177676
acl half10173 random 0.0008402237462388
acl half10218 random 0.0008032337005613
acl half10221 random 0.0008008264083574
acl half10222 random 0.0008000255819491
acl half10223 random 0.0007992255563671
acl half10227 random 0.0007960334462989
acl half10247 random 0.0007802631200941
acl half10248 random 0.0007794828569740
.
.
.
.
.

> On Aug 28, 2016, at 12:56 PM, --Ahmad--  wrote:
> 
> just to tell you 
> i updated the acl as below :
> acl half10001 random 0.000998001000
> acl half10006 random 0.0009930209650350
> acl half10011 random 0.0009880657804942
> acl half10020 random 0.0009792086759647
> acl half10037 random 0.0009626946373158
> acl half10043 random 0.0009569328906720
> acl half10059 random 0.0009417362622232
> acl half10079 random 0.0009230793978373
> acl half10082 random 0.0009203129279589
> acl half10084 random 0.0009184732224159
> acl half10094 random 0.0009093297114627
> acl half10098 random 0.0009056978449587
> acl half10109 random 0.0008957848329039
> acl half10113 random 0.0008922070646991
> acl half10114 random 0.0008913148576344
> acl half10122 random 0.0008842092457380
> acl half10137 random 0.0008710385479118
> acl half10154 random 0.0008563487636013
> acl half10168 random 0.0008444374977929
> acl half10171 random 0.0008419067177676
> acl half10173 random 0.0008402237462388
> acl half10218 random 0.0008032337005613
> acl half10221 random 0.0008008264083574
> acl half10222 random 0.0008000255819491
> acl half10223 random 0.0007992255563671
> acl half10227 random 0.0007960334462989
> acl half10247 random 0.0007802631200941
> acl half10248 random 0.0007794828569740
> .
> .
> .
> .
> .
> 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] cache object with vary

[joe]

amos just by switching refresh from chrome to firefox and vice versa
2016/08/29 05:27:46 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://www.annahar.com/'
'accept-encoding="gzip,%20deflate,%20sdch",
user-agent="Mozilla%2F5.0%20(Windows%20NT%205.1)%20AppleWebKit%2F537.36%20(KHTML,%20like%20Gecko)%20Chrome%2F49.0.2623.112%20Safari%2F537.36"'
2016/08/29 05:27:46 kid1| clientProcessHit: Vary object loop!
2016/08/29 05:27:56 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://www.annahar.com/'
'accept-encoding="gzip,%20deflate",
user-agent="Mozilla%2F5.0%20(Windows%20NT%205.1%3B%20rv%3A47.0)%20Gecko%2F20100101%20Firefox%2F47.0"'
2016/08/29 05:27:56 kid1| clientProcessHit: Vary object loop!
2016/08/29 05:28:14 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://www.annahar.com/'
'accept-encoding="gzip,%20deflate,%20sdch",
user-agent="Mozilla%2F5.0%20(Windows%20NT%205.1)%20AppleWebKit%2F537.36%20(KHTML,%20like%20Gecko)%20Chrome%2F49.0.2623.112%20Safari%2F537.36"'
2016/08/29 05:28:14 kid1| clientProcessHit: Vary object loop!
2016/08/29 05:28:26 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://www.annahar.com/'
'accept-encoding="gzip,%20deflate",
user-agent="Mozilla%2F5.0%20(Windows%20NT%205.1%3B%20rv%3A47.0)%20Gecko%2F20100101%20Firefox%2F47.0"'
2016/08/29 05:28:26 kid1| clientProcessHit: Vary object loop!

without store-id 
squid detect the obj and replace by new one if i switch back to another
browser it dose the same
it should keep the obj in cache for chrome and have new obj save for firefox
if you ar saying they ar 2 diff md5 key but it dose not do that



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/cache-object-with-vary-tp4679220p4679235.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Trouble negotiate_kerberos_auth

[Marcio Demetrio Bacci]

Hi Markus, thank you for help me.

When I type the klist command, the result is:

Ticket cache: FILE:/tmp/krb5cc_0
Default principal: rob...@cms.ensino.br
Valid starting   Expires  Service principal
28-08-2016 22:40:53  29-08-2016 08:40:53  krbtgt/cms.ensino...@cms.ensino.br
renew until 29-08-2016 22:40:41

But, I have the following result to command bellow:
/usr/lib64/squid/negotiate_kerberos_auth_test proxy.cms.ensino.br| awk
'{sub(/Token:/,"YR"); print $0}END{print "QQ"}' |
/usr/lib64/squid/negotiate_kerberos_auth -r -s HTTP/proxy.cms.ensino.br

Result:
TT
oYGbMIGYoAMKAQGhCAYGKwYBBQIFooGGBIGDBQEwFKESBBBDTUIuRU5TSU5PLkVCLkJSfmkwZ6ADAgEFoQMCAR6iERgPMjAxNjA4MjkwMTM2MDVaowUCAwK7P6QRGA8yMDE2MDgyOTAxMzYwNVqlBQIDBhpppgMCAQepFRsTPHVuc3BlY2lmaWVkIHJlYWxtPqoLMAmgAwIBAKECMAA=
BH quit command


The HTTP/proxy.cms.ensino.br is in keytab files

I don't have the "test_negotiate_auth.sh" file in
src/auth/negotiate/kerberos, but I have
/usr/lib64/squid/negotiate_kerberos_auth_test, thus I'm using it.

My Linux distribution is CentOS 7

Regards,

Márcio




2016-08-28 15:24 GMT-03:00 Markus Moeller :

>
> HI Marcio,
>
>   The helper need a Kerberos token as input.  Please have a look at
> test_negotiate_auth.sh  which is in src/auth/negotiate/kerberos of the
> trunk version. The squid hostname must match the entry in your keytab and
> you must have done kinit to authenticate against a Kerberos server (e.g.
> AD) as user first.
>
> Regards
> Markus
>
>
> "Marcio Demetrio Bacci"  wrote in message news:CA+
> 0tdyqeat4l5ko4zrjnj1aue64my2re7z95kfdqw7y8sv_...@mail.gmail.com...
> I have trouble to authenticate Squid3 with kerberos in Samba4 domain. I'm
> using CentOS 7 and Squid 3.3.8 (yum install squid)
>
> When I type the bellow command in terminal:
> /usr/lib64/squid/negotiate_kerberos_auth -d -i -s HTTP/
> proxy.cms.ensino...@cms.ensino.br
> john xyz@12345
>
> I have the following error:
> negotiate_kerberos_auth.cc(315): pid=6364 :2016/08/27 10:44:33|
> negotiate_kerberos_auth: DEBUG: Got 'john xyz@12345' from squid (length:
> 14).
> negotiate_kerberos_auth.cc(362): pid=6364 :2016/08/27 10:44:33|
> negotiate_kerberos_auth: ERROR: Invalid request [john xyz@12345]
> BH invalid request
>
>
> Here are my files configuration:
>
> /etc/krb5.conf
> [libdefaults]
> default_realm = CMS.ENSINO.BR
> [realms]
> CMS.ENSINO.BR = {
> kdc = dc1.cms.ensino.br:88
> admin_server = dc1.cms.ensino.br
> default_domain = CMS.ENSINO.BR
> }
> [domain_realm]
> .cms.ensino.br = CMS.ENSINO.BR
> cms.ensino.br = CMS.ENSINO.BR
>
>
>
> Keytab name: FILE:/etc/krb5.keytab
> KVNO Principal
>  
> --
>1 proxy-k$@CMS.ENSINO.BR
>1 proxy-k$@CMS.ENSINO.BR
>1 proxy-k$@CMS.ENSINO.BR
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>1 host/pr...@cms.ensino.br
>1 host/pr...@cms.ensino.br
>1 host/pr...@cms.ensino.br
>1 host/pr...@cms.ensino.br
>1 host/pr...@cms.ensino.br
>1 PROXY$@CMS.ENSINO.BR
>1 PROXY$@CMS.ENSINO.BR
>1 PROXY$@CMS.ENSINO.BR
>1 PROXY$@CMS.ENSINO.BR
>1 PROXY$@CMS.ENSINO.BR
>1 proxy-k$@CMS.ENSINO.BR
>1 proxy-k$@CMS.ENSINO.BR
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 HTTP/pr...@cms.ensino.br
>1 HTTP/pr...@cms.ensino.br
>1 HTTP/pr...@cms.ensino.br
>1 HTTP/pr...@cms.ensino.br
>1 HTTP/pr...@cms.ensino.br
>
>
> Keytab name: FILE:/etc/squid/PROXY.keytab
> KVNO Principal
>  
> --
>1 proxy-k$@CMS.ENSINO.BR
>1 proxy-k$@CMS.ENSINO.BR
>1 proxy-k$@CMS.ENSINO.BR
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 HTTP/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>1 host/proxy.cms.ensino...@cms.ensino.br
>
>
> /etc/sysconfig/squid
> # default squid options
> SQUID_OPTS=""
> # Time to wait for Squid to shut down when asked. Should not be necessary
> # most of the time.
> SQUID_SHUTDOWN_TIMEOUT=100
> # default squid conf file
> SQUID_CONF="/etc/squid/squid.conf"
>
> KRB5_KTNAME=/etc/squid/PROXY.keytab
> export KRB5_KTNAME
>
>
> kinit and klist commands are OK.
>
> Best Regards,
>
> Márcio
>
>
> --
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
>
> ___
> squid-users mailing list
> squid-users@list

Re: [squid-users] TCP_RESET non http requests on port 80

[Alex Rousskov]

On 08/28/2016 03:10 AM, Omid Kosari wrote:
> Alex Rousskov wrote
>> I understand that it works for regular requests. Does it also work (i.e.,
>> does Squid reset the connection) when handling a non-HTTP request on port 80?

> No , when the request is non-HTTP it does not reset the connection .

Great. Now please go back to the simpler configuration I asked you to
test some time ago:

  http_reply_access deny all
  deny_info TCP_RESET all

Does that work for non-HTTP request on port 80?



> config:
> acl test dst 69.58.188.49
> deny_info TCP_RESET test
> http_reply_access deny test 
> 
> 
> =
> test type:
> telnet 123.com 80
> GET / HTTP/1.1
> host: 123.com
> 
> 
> RESULT:
> HTTP/1.1 403 Forbidden

I am confused. Earlier you said "As i mention before the deny_info works
in other configs" and gave a very similar configuration example with
dstdomain ACL. Now you are showing that this example does _not_ work
even with regular requests (you are getting HTTP headers from Squid
instead of a TCP connection reset). Am I missing something?



> config:
> acl test dst 69.58.188.49
> deny_info TCP_RESET test
> adapted_http_access deny test
> 
> 
> =
> test type:
> telnet 123.com 80
> GET / HTTP/1.1
> host: 123.com
> 
> 
> 
> RESULT:
> note:empty, just disconnects the telnet

OK, this works as expected, but we need to get it working for the
http_reply_access IMO.


> =
> test type:
> telnet 123.com 80
> sgsdgsdgsdgsdg
> 
> RESULT:
> HTTP/1.1 400 Bad Request
> Server: squid
> Mime-Version: 1.0
> Date: Sun, 28 Aug 2016 08:56:14 GMT
> Content-Type: text/html;charset=utf-8
> Content-Length: 0
> X-Cache: MISS from cache1
> X-Cache-Lookup: NONE from cache1:3128
> Connection: close

OK, this does not work, as expected (there is no matching request and/or
adapted_http_access is not evaluated at all in this case). You need
http_reply_access.

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid with random outgoing ip from pool of 1000 ips

[Alex Rousskov]

On 08/28/2016 03:56 AM, --Ahmad-- wrote:

> the rotation is very bad and the outgoing ip now is the ip that is
> on my eth0 ipv6 address  only .
> i mean the lists above not working at all and all request go with  1
> ipv6 which is the ip that is on my eth0 address .
> 
> 
> what wrong did i do ?

I do not know. I recommend the following procedure:

1. Start with just one outgoing IP address (that is not the default!)
and make that work well first. Does all from-Squid traffic originate at
that alternative IP address?

2. Move to two outgoing IP addresses (none of which is the default!) and
make that work well. Do you get a nice 50/50 distribution?

3. Use three IP addresses. This step will allow you to test your script.
Again, make sure everything works very well before you proceed any
further. Do measure the actual IP probabilities for a large number of
transactions to verify that your configuration is correct. You should
see ~33% of transactions using each alternative IP.

4. Move on to 1000 IP addresses. Check again.

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid with random outgoing ip from pool of 1000 ips

[Alex Rousskov]

On 08/28/2016 02:04 AM, --Ahmad-- wrote:

> how about the length of the number?
> 
> 1/1000 to which digits should i round ??
> 
> 4 or 5 6 digits ?

I would start with 10 digits or the maximum precision that your
ACL-calculation program allows and decrease the number of digits if
Squid cannot parse the generated probability.

> will squid understand the long digits ?

Yes, but probably up to an [unknown to me] point.

HTH,

Alex.



>> To compensate for the cumulative effect of rules evaluation, you need
>> rule i to have p/(q^i) probability of a match

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Trouble negotiate_kerberos_auth

[Markus Moeller]

HI Marcio,

  The helper need a Kerberos token as input.  Please have a look at 
test_negotiate_auth.sh  which is in src/auth/negotiate/kerberos of the trunk 
version. The squid hostname must match the entry in your keytab and you must 
have done kinit to authenticate against a Kerberos server (e.g. AD) as user 
first.

Regards
Markus 


"Marcio Demetrio Bacci"  wrote in message 
news:ca+0tdyqeat4l5ko4zrjnj1aue64my2re7z95kfdqw7y8sv_...@mail.gmail.com...
I have trouble to authenticate Squid3 with kerberos in Samba4 domain. I'm using 
CentOS 7 and Squid 3.3.8 (yum install squid)


When I type the bellow command in terminal: 
/usr/lib64/squid/negotiate_kerberos_auth -d -i -s 
HTTP/proxy.cms.ensino...@cms.ensino.br
john xyz@12345

I have the following error:
negotiate_kerberos_auth.cc(315): pid=6364 :2016/08/27 10:44:33| 
negotiate_kerberos_auth: DEBUG: Got 'john xyz@12345' from squid (length: 14).
negotiate_kerberos_auth.cc(362): pid=6364 :2016/08/27 10:44:33| 
negotiate_kerberos_auth: ERROR: Invalid request [john xyz@12345]
BH invalid request 


Here are my files configuration:

/etc/krb5.conf
[libdefaults]
default_realm = CMS.ENSINO.BR
[realms]
CMS.ENSINO.BR = {
kdc = dc1.cms.ensino.br:88
admin_server = dc1.cms.ensino.br
default_domain = CMS.ENSINO.BR 
}
[domain_realm]
.cms.ensino.br = CMS.ENSINO.BR
cms.ensino.br = CMS.ENSINO.BR



Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
 --
   1 proxy-k$@CMS.ENSINO.BR
   1 proxy-k$@CMS.ENSINO.BR
   1 proxy-k$@CMS.ENSINO.BR
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br
   1 host/pr...@cms.ensino.br
   1 host/pr...@cms.ensino.br
   1 host/pr...@cms.ensino.br
   1 host/pr...@cms.ensino.br
   1 host/pr...@cms.ensino.br
   1 PROXY$@CMS.ENSINO.BR
   1 PROXY$@CMS.ENSINO.BR
   1 PROXY$@CMS.ENSINO.BR
   1 PROXY$@CMS.ENSINO.BR
   1 PROXY$@CMS.ENSINO.BR
   1 proxy-k$@CMS.ENSINO.BR
   1 proxy-k$@CMS.ENSINO.BR
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 HTTP/pr...@cms.ensino.br
   1 HTTP/pr...@cms.ensino.br
   1 HTTP/pr...@cms.ensino.br
   1 HTTP/pr...@cms.ensino.br
   1 HTTP/pr...@cms.ensino.br


Keytab name: FILE:/etc/squid/PROXY.keytab
KVNO Principal
 --
   1 proxy-k$@CMS.ENSINO.BR
   1 proxy-k$@CMS.ENSINO.BR
   1 proxy-k$@CMS.ENSINO.BR
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 HTTP/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br
   1 host/proxy.cms.ensino...@cms.ensino.br


/etc/sysconfig/squid
# default squid options
SQUID_OPTS=""
# Time to wait for Squid to shut down when asked. Should not be necessary
# most of the time.
SQUID_SHUTDOWN_TIMEOUT=100
# default squid conf file
SQUID_CONF="/etc/squid/squid.conf"

KRB5_KTNAME=/etc/squid/PROXY.keytab
export KRB5_KTNAME



kinit and klist commands are OK.


Best Regards,


Márcio





___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] cache object with vary

[joe]

>If the above wasn't clear enough. This is how squid does it: 

> key: MD5("http://url.com/some.js";) 
>data: vary-marker object ("Vary:Accept-Encoding", ...) 

 >key: MD5("http://url.com/some.js"; + "accept-encoding=") 
 >data: no- Accept-Encoding variant response 

 >key: MD5("http://url.com/some.js"; + "accept-encoding=identity") 
 >data: "identity" variant response 

 >key: MD5("http://url.com/some.js"; + "accept-encoding=gzip") 
 >data: "gzip" variant response 

 >key: MD5("http://url.com/some.js"; + "accept-encoding=deflate") 
 >data: "deflate" variant response 

 >key: MD5("http://url.com/some.js"; + "accept-encoding=deflate,gzip") 
 >data: "deflate,gzip" variant response 

> key: MD5("http://url.com/some.js"; + "accept-encoding=gzip,deflate") 
 >data: "gzip,deflate" variant response 
right so those example they shuld be separet file right depend on vary key
match
 is that right ??
must be as you sayd the store-id then doing some funy things
im going to disable store id and test



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/cache-object-with-vary-tp4679220p4679229.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] cache object with vary

[Amos Jeffries]

On 28/08/2016 12:56 p.m., joe wrote:
> is this bug or its made to work like that
> lets say we have object in cache name 00A5
> url.com/some.js
> vary=accept-encoding="gzip"
> 
> if some browser get the same object
> url.com/some.js
> vary=accept-encoding="deflate"
> 
> the md5 key wont match 

Correct.

> and it delete the old cached object with
> accept-encoding="gzip" and replace with
> new one with vary=accept-encoding="deflate" and prosess as TCP_MISS

Incorrect.

The first thing Squid does is lookup the URL (only). That finda a 'vark
marker' object which tels Squid the Vary header pattern to append to the
hash key and do another lookup for that.

The amended hash key for the second query finds no object ==> a MISS.
Period.

The "gzip" object existence or absence is not related nor touched.

> 
> that will result in "varyEvaluateMatch: Oops. Not a Vary match on second
> attempt
> no match and the code in client_side.cc
> return VARY_CANCEL

IF:
 * the second lookup with the amended hash key *did* find an object, and
 * it was for the same URL, and
 * it has no Vary header;
then a warning message (the above?) is output and the found object will
be replaced with whatever comes back from the MISS resolving actions.


I think you can get yourself into this type of situation when using
Store-ID in ways prohibited by the Store-ID design.

 Requirement #1 for Store-ID is that all objects found by the custom ID
key are identical.

 Variants are non-identical by definition. So at least one variant of
objects that Vary is not going to be identical to objects that lack Vary!


You can also encounter it with SMP workers at times. Since the workers
are processing more traffic than ever before the churn and key hash
collisions rate is potentially greater.


> 
> and in client_side_reply.cc
> 
> case VARY_CANCEL:
> /* varyEvaluateMatch found a object loop. Process as miss */
> debugs(88, DBG_IMPORTANT, "clientProcessHit: Vary object loop!");

NP: the above statements may or may not be true. The code was written a
long time ago and things around it have changed a lot in the meantime.

> http->logType = LOG_TCP_MISS; // we lack a more precise LOG_*_MISS
> code
> processMiss();
> return;
> 
> the way it should be instead of replacing the existing obj  should be
> another object with the 
> new vary
> shuld be 2 file 00A5
> and00A6 example each one has different vary to match the correct
> obj if its gzip or ident or deflate or with useragent
> wen vary not matching shuld be new obj file to be saved as diferent cache
> name 00A7
> so it match the correct object name with its vary
> 

If the above wasn't clear enough. This is how squid does it:

 key: MD5("http://url.com/some.js";)
 data: vary-marker object ("Vary:Accept-Encoding", ...)

 key: MD5("http://url.com/some.js"; + "accept-encoding=")
 data: no- Accept-Encoding variant response

 key: MD5("http://url.com/some.js"; + "accept-encoding=identity")
 data: "identity" variant response

 key: MD5("http://url.com/some.js"; + "accept-encoding=gzip")
 data: "gzip" variant response

 key: MD5("http://url.com/some.js"; + "accept-encoding=deflate")
 data: "deflate" variant response

 key: MD5("http://url.com/some.js"; + "accept-encoding=deflate,gzip")
 data: "deflate,gzip" variant response

 key: MD5("http://url.com/some.js"; + "accept-encoding=gzip,deflate")
 data: "gzip,deflate" variant response

 ... and so on for all possible unique strings that could be sent in
Accept-Encoding.


If one of those 'data' objects contains a 'wrong' response object. The
transaction encountering it MISS'es  / VARY_CANCEL and that store
location gets updated with correct content resulting from the server fetch.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Limit Bandwith for youtube....

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 


28.08.2016 22:39, Yuri Voinov пишет:
>
>
>
> 28.08.2016 21:59, Yuri Voinov пишет:
>
>
>
>
>
>
>   > 28.08.2016 17:34, Matus UHLAR - fantomas пишет:
>
>   > > On 27.08.16 01:10, Yuri Voinov wrote:
>
>   > >>
>
>
>
http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussion?highlight=%28Youtube%29
>
>   > >>
>
>   > >>
>
>   > >> 26.08.2016 23:54, Matus UHLAR - fantomas пишет:
>
>   > >>> On 26.08.16 03:16, Yuri Voinov wrote:
>
>   >  Everything can be much easier. Google
>   Streaming video is not cacheable.
>
>   >  Absolutely.  If users are watching the same
>   video, each time it is
>
>   >  downloaded from the outside. Slowly and
>   sadly.
>
>   > >>>
>
>   > >>> could something like collapsed forwarding solve
>   this problem?
>
>
>
>   > > not videos, streaming. Streamed content should be (at
>   least hypotetically)
>
>   > > possible to receive once, send many times.
>
>   > So?
> Streamed content is always files. Generally with the same chunk size.
With fixed length.
> Only YT encrypt every chunk, so it can't be cached in any way. Another
streamed content can be cached with store-ID.
Sorry - Only YT encrypt every chunk *URL*, so it can't be cached in any way.
>
> Where is collapsed forwarding here?
>
>
>

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXwxPZAAoJENNXIZxhPexGS30H/jKSWUd/X0fdJ7RxYHF1PtA4
D4cW9mqIQJUxKCA8YZkTN0PioWPpwWB6x/ZbSRUnrtcAp/350hG6N8O3hm3k4WlT
XXvPPSfwvAdyuWNA9QHROmfP6z1RMvnnf6avjA3HDdNqmFM2CFYWaZDphWsEEHVa
256etwQAd6Ck7e5jOLqtLcpBZZ4Nol0c9L8x4Ids2xfVMO3FvBP6QAO9uBeEKt/o
cgCgoPCWSFSDlgiQFb0Dk8Js5VfBY4dVau+CsgN+w7L4RJUBT4pT1DTWnxzTOhO0
Z4jvH3MDEtBRm9b3ewWEmN/ZWUVE/fP3BgNevD9fSrDvQyuTLbY7hx2xosHBcGY=
=WNI8
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Limit Bandwith for youtube....

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 


28.08.2016 21:59, Yuri Voinov пишет:
>
>
>
> 28.08.2016 17:34, Matus UHLAR - fantomas пишет:
> > On 27.08.16 01:10, Yuri Voinov wrote:
> >>
>
http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussion?highlight=%28Youtube%29
> >>
> >>
> >> 26.08.2016 23:54, Matus UHLAR - fantomas пишет:
> >>> On 26.08.16 03:16, Yuri Voinov wrote:
>  Everything can be much easier. Google Streaming video is not
cacheable.
>  Absolutely.  If users are watching the same video, each time it is
>  downloaded from the outside. Slowly and sadly.
> >>>
> >>> could something like collapsed forwarding solve this problem?
>
> > not videos, streaming. Streamed content should be (at least
hypotetically)
> > possible to receive once, send many times.
> So?
Streamed content is always files. Generally with the same chunk size.
With fixed length.
Only YT encrypt every chunk, so it can't be cached in any way. Another
streamed content can be cached with store-ID.

Where is collapsed forwarding here?
>

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXwxOnAAoJENNXIZxhPexGbR0H/1lFeUXDAjbY3jeXKDfyeP8o
FBUE5nAhYHCKbdZU+VYLMcm+p0C/t4x59XwjI455gEZZ5vTdNhx/JT+/kQR/AcDc
klwKnkXIvDoxHuxvTrBzoMtlTjnZ4ZLHaEHvv7FoBp4dkY3Sptbzxt6Q2LM9/h9K
ssGSbvPYxuUrNDqc35B9pACxyxhRWWZfl6k/OKPA++Kj8smLiPnd7Hbc84wOm3gA
Y7/3W0N/Qo1p2E/KcxVnkFrd2Nhbe7Jt0yLTRew/5046hSxKEDf37ykP8gFrdUcH
9gbM/gakXRjfrdhLUJnKBDQPnlVqKroAOtN6aKgvhraCbIj9z0CgmkbG9r8X/lE=
=RuBT
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Limit Bandwith for youtube....

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 


28.08.2016 17:34, Matus UHLAR - fantomas пишет:
> On 27.08.16 01:10, Yuri Voinov wrote:
>>
http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussion?highlight=%28Youtube%29
>>
>>
>> 26.08.2016 23:54, Matus UHLAR - fantomas пишет:
>>> On 26.08.16 03:16, Yuri Voinov wrote:
 Everything can be much easier. Google Streaming video is not cacheable.
 Absolutely.  If users are watching the same video, each time it is
 downloaded from the outside. Slowly and sadly.
>>>
>>> could something like collapsed forwarding solve this problem?
>
> not videos, streaming. Streamed content should be (at least hypotetically)
> possible to receive once, send many times.
So?

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXwwp4AAoJENNXIZxhPexGZSUIAITHcsi3CltBKxizehM8EQ5D
RBZ7NnBI88dF1ATfn5x9sW0/4m96G/iXjUOrpa8sNLDbnXaRDb+53RtNR1rs4dSb
pKGa5YvxI218t5LPAGfSMuDpjIkmgnE3/4cDiubasqgIt0yBva40tdWoGjkfxdZJ
ADbmO23EUQ328XNWR+pr+64CegpxeBHtIav1YR1oMJVDRmhVoVZWBc6e+hzEJYfV
jAUbDxZp9Viw/05XInhadcR+l+tCBAjY8uGR1JsVk3FU2EpOxj4RLsnCU27Xng4q
aTwcd3daT+dgEMjYnoOnwFd42oWQT2EA6us7reH8y8f0lMCm8i+Iiwx1J4Rj7Lg=
=9iXx
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Limit Bandwith for youtube....

[Matus UHLAR - fantomas]

On 27.08.16 01:10, Yuri Voinov wrote:

http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussion?highlight=%28Youtube%29


26.08.2016 23:54, Matus UHLAR - fantomas пишет:

On 26.08.16 03:16, Yuri Voinov wrote:

Everything can be much easier. Google Streaming video is not cacheable.
Absolutely.  If users are watching the same video, each time it is
downloaded from the outside. Slowly and sadly.


could something like collapsed forwarding solve this problem?


not videos, streaming. Streamed content should be (at least hypotetically)
possible to receive once, send many times.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid with random outgoing ip from pool of 1000 ips

[--Ahmad--]

just to tell you 
i updated the acl as below :
acl half10001 random 0.000998001000
acl half10006 random 0.0009930209650350
acl half10011 random 0.0009880657804942
acl half10020 random 0.0009792086759647
acl half10037 random 0.0009626946373158
acl half10043 random 0.0009569328906720
acl half10059 random 0.0009417362622232
acl half10079 random 0.0009230793978373
acl half10082 random 0.0009203129279589
acl half10084 random 0.0009184732224159
acl half10094 random 0.0009093297114627
acl half10098 random 0.0009056978449587
acl half10109 random 0.0008957848329039
acl half10113 random 0.0008922070646991
acl half10114 random 0.0008913148576344
acl half10122 random 0.0008842092457380
acl half10137 random 0.0008710385479118
acl half10154 random 0.0008563487636013
acl half10168 random 0.0008444374977929
acl half10171 random 0.0008419067177676
acl half10173 random 0.0008402237462388
acl half10218 random 0.0008032337005613
acl half10221 random 0.0008008264083574
acl half10222 random 0.0008000255819491
acl half10223 random 0.0007992255563671
acl half10227 random 0.0007960334462989
acl half10247 random 0.0007802631200941
acl half10248 random 0.0007794828569740
.
.
.
.
.


but the rotation is very bad and the outgoing ip now is the ip that is on my 
eth0 ipv6 address  only .
i mean the lists above not working at all and all request go with  1 ipv6 which 
is the ip that is on my eth0 address .


what wrong did i do ?



> On Aug 28, 2016, at 11:04 AM, --Ahmad--  wrote:
> 
> thanks for reply 
> 
> how about the length of the number?
> 
> 1/1000 to which digits should i round ??
> 
> 4 or 5 6 digits ?
> 
> 
> 
> AS AN EXAMPLE the 1/1000 probabilities will have the acls as below :
> 
> 
> 0.001 0.000999 0.000998001 0.000997002999 0.000996005996001
> 
> 
> the question is how many digits should i round ?
> 
> will squid understand the long digits ?
> 
> 
> cheers
> 
> 
>> On Aug 28, 2016, at 2:38 AM, Alex Rousskov > > wrote:
>> 
>> On 08/27/2016 04:34 PM, --Ahmad-- wrote:
>> 
>>> i guess i need to create probability 1/1000 for each ip.
>> 
>> Yes, but that is _not_ the same as 1/1000 probability for each
>> tcp_outgoing_address rule, unfortunately. tcp_outgoing_address rules are
>> evaluated top to bottom until the first matches. If you have N rules and
>> each rule has a 1/N probability of a match in isolation, then you will
>> get the following probabilities of a match when the rules are combined:
>> 
>>  rule #0: 1/N   -- good!
>>  rule #1: (1-1/N) * 1/N  -- which is not 1/N
>>  rule #2: (1-1/N) * (1-1/N) * 1/N  -- even less 1/N than rule #2 was
>>  rule #3: (1-1/N) * (1-1/N) * (1-1/N) * 1/N  -- and getting worse!
>>  ...
>> 
>> To simplify equations, let me denote 1/N as p and (1-1/N) as q. With
>> your incorrect 1/N ACLs, you get the following probabilities (I am just
>> rewriting the above using p and q):
>> 
>>  rule #0: p
>>  rule #1: q * p
>>  rule #2: q*q * p
>>  rule #3: q*q*q * p
>>  ...
>> 
>> If you are still unsure, consider the simple case of just 2 rules
>> (instead of 1000). You want the second rule to match 50% of the time. If
>> you give the second rule ACL the same 1/2 probability of a match, then
>> the second rule will only match 1/4 of the time because it will match
>> only when the previous rule did _not_ match (1/2) _and_ when its own ACL
>> matched (1/2): 1/2*1/2 = 1/4.
>> 
>> 
>> To compensate for the cumulative effect of rules evaluation, you need
>> rule i to have p/(q^i) probability of a match (where "q^i" is "q to the
>> power of i"). With that, you will always get the same probability of a
>> match (p) for each rule when that rule is evaluated:
>> 
>>  rule #0: p
>>  rule #1: q * p/q = p
>>  rule #2: q*q * p/(q*q) = p
>>  rule #3: q*q*q * p/(q*q*q) = p
>>  ...
>>  rule #998: q^998 * p/(q^998) = p
>> 
>> To avoid uncertainty, the last rule (rule #999 in the above notation)
>> should use the "all" ACL (i.e., it will always match).
>> 
>> 
>>> how can i create the randomized acls ???
>> 
>> I suggest writing a script that generates 999 ACLs with correct p/(q^i)
>> probability and the corresponding tcp_outgoing_address lines to match them.
>> 
>> Please note that computing ~500 random ACL matches for each outgoing
>> Squid connection (or is it each request?) is not going to be
>> instantaneous! If you are worried about Squid performance, then you may
>> want to add custom Squid code to select a random or round-robin IP
>> address out of a pool of 1000 addresses instead.
>> 
>> [ It is not going to be easy, but if you do it right, the same new
>> configuration interface and underlying code can then be applied to other
>> similar tasks in Squid (e.g., selecting one of several load-balanced
>> ICAP services). In that case, it would be a welcomed feature that may be
>> officially accepted. If you decide to make this generally useful, then I
>> recommend getting your configuration design pre-approved on squid-dev
>> before you implement anything (or before you

Re: [squid-users] TCP_RESET non http requests on port 80

[Omid Kosari]

Alex Rousskov wrote
> I understand that it works for regular requests. Does it also work (i.e.,
> does Squid
> reset the connection) when handling a non-HTTP request on port 80?

No , when the request is non-HTTP it does not reset the connection .



Here is my test results . i would test with 123.com ip address which is
69.58.188.49 .






config:
acl test dst 69.58.188.49
deny_info TCP_RESET test
http_reply_access deny test 


=
test type:
telnet 123.com 80
GET / HTTP/1.1
host: 123.com


RESULT:
HTTP/1.1 403 Forbidden
Server: squid
Mime-Version: 1.0
Date: Sun, 28 Aug 2016 08:45:23 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 5
X-Cache: MISS from cache1
X-Cache-Lookup: MISS from cache1:3128
Connection: keep-alive

reset

note:telnet will not disconnect until i hit few Enter

=
test type:
telnet 123.com 80
sgsdgsdgsdgsdg

RESULT:
HTTP/1.1 400 Bad Request
Server: squid
Mime-Version: 1.0
Date: Sun, 28 Aug 2016 09:00:12 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 0
X-Cache: MISS from cache1
X-Cache-Lookup: NONE from cache1:3128
Connection: close



Connection to host lost.




config:
acl test dst 69.58.188.49
deny_info TCP_RESET test
adapted_http_access deny test


=
test type:
telnet 123.com 80
GET / HTTP/1.1
host: 123.com



RESULT:
note:empty, just disconnects the telnet

=
test type:
telnet 123.com 80
sgsdgsdgsdgsdg

RESULT:
HTTP/1.1 400 Bad Request
Server: squid
Mime-Version: 1.0
Date: Sun, 28 Aug 2016 08:56:14 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 0
X-Cache: MISS from cache1
X-Cache-Lookup: NONE from cache1:3128
Connection: close



Connection to host lost.




--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/TCP-RESET-non-http-requests-on-port-80-tp4679102p4679222.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid with random outgoing ip from pool of 1000 ips

[--Ahmad--]

thanks for reply 

how about the length of the number?

1/1000 to which digits should i round ??

4 or 5 6 digits ?



AS AN EXAMPLE the 1/1000 probabilities will have the acls as below :


0.001 0.000999 0.000998001 0.000997002999 0.000996005996001


the question is how many digits should i round ?

will squid understand the long digits ?


cheers


> On Aug 28, 2016, at 2:38 AM, Alex Rousskov  
> wrote:
> 
> On 08/27/2016 04:34 PM, --Ahmad-- wrote:
> 
>> i guess i need to create probability 1/1000 for each ip.
> 
> Yes, but that is _not_ the same as 1/1000 probability for each
> tcp_outgoing_address rule, unfortunately. tcp_outgoing_address rules are
> evaluated top to bottom until the first matches. If you have N rules and
> each rule has a 1/N probability of a match in isolation, then you will
> get the following probabilities of a match when the rules are combined:
> 
>  rule #0: 1/N   -- good!
>  rule #1: (1-1/N) * 1/N  -- which is not 1/N
>  rule #2: (1-1/N) * (1-1/N) * 1/N  -- even less 1/N than rule #2 was
>  rule #3: (1-1/N) * (1-1/N) * (1-1/N) * 1/N  -- and getting worse!
>  ...
> 
> To simplify equations, let me denote 1/N as p and (1-1/N) as q. With
> your incorrect 1/N ACLs, you get the following probabilities (I am just
> rewriting the above using p and q):
> 
>  rule #0: p
>  rule #1: q * p
>  rule #2: q*q * p
>  rule #3: q*q*q * p
>  ...
> 
> If you are still unsure, consider the simple case of just 2 rules
> (instead of 1000). You want the second rule to match 50% of the time. If
> you give the second rule ACL the same 1/2 probability of a match, then
> the second rule will only match 1/4 of the time because it will match
> only when the previous rule did _not_ match (1/2) _and_ when its own ACL
> matched (1/2): 1/2*1/2 = 1/4.
> 
> 
> To compensate for the cumulative effect of rules evaluation, you need
> rule i to have p/(q^i) probability of a match (where "q^i" is "q to the
> power of i"). With that, you will always get the same probability of a
> match (p) for each rule when that rule is evaluated:
> 
>  rule #0: p
>  rule #1: q * p/q = p
>  rule #2: q*q * p/(q*q) = p
>  rule #3: q*q*q * p/(q*q*q) = p
>  ...
>  rule #998: q^998 * p/(q^998) = p
> 
> To avoid uncertainty, the last rule (rule #999 in the above notation)
> should use the "all" ACL (i.e., it will always match).
> 
> 
>> how can i create the randomized acls ???
> 
> I suggest writing a script that generates 999 ACLs with correct p/(q^i)
> probability and the corresponding tcp_outgoing_address lines to match them.
> 
> Please note that computing ~500 random ACL matches for each outgoing
> Squid connection (or is it each request?) is not going to be
> instantaneous! If you are worried about Squid performance, then you may
> want to add custom Squid code to select a random or round-robin IP
> address out of a pool of 1000 addresses instead.
> 
> [ It is not going to be easy, but if you do it right, the same new
> configuration interface and underlying code can then be applied to other
> similar tasks in Squid (e.g., selecting one of several load-balanced
> ICAP services). In that case, it would be a welcomed feature that may be
> officially accepted. If you decide to make this generally useful, then I
> recommend getting your configuration design pre-approved on squid-dev
> before you implement anything (or before you pay somebody else to
> implement it)! ]
> 
> 
>> is my settings below is correct ??
> 
> No. Your ACL(s) and rule probabilities are wrong. See above.
> 
> 
> HTH,
> 
> Alex.
> 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users