Re: [squid-users] Delay pools not working with squid 4.x , and more Question !!

[Alex Rousskov]

On 9/22/19 10:28 AM, --Ahmad-- wrote:

> Looking forward to hearing that bug fixed .

Me too! However, please do not misinterpret my response as a
confirmation of the bug existence or an implication that somebody is
working on a fix. I do not know whether anybody is working on this. I do
not even recall if somebody has filed a corresponding bug report.

https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F

Alex.


>> On Sep 22, 2019, at 5:07 PM, Alex Rousskov wrote:
>>
>> On 9/22/19 6:25 AM, --Ahmad-- wrote:
>>
>>> i tested squid 4.8 and delay pools not working with it at all .
>>> i reverted back to squid 3.5.x and i had delay pools working .
>>
>>> Q1- do squid 4 support delay pools ?
>>
>> It should. If it does not, there is a bug somewhere.
>>
>>
>>> Q2- with squid 3.5.x we have SMP about 4 childs , and we are running delay 
>>> pools .
>>> does that mean speed ( with all 4 instances ) is 1/1 Mbps
>>> or speed ( with all 4 instances ) is 4/4  Mbps?
>>
>> According to [1], delay pools are not SMP-aware yet so you are
>> essentially configuring individual worker limits: Workers do not share
>> their limits and pools with each other. Hence, the effective Squid
>> instance limit is, very approximately, the aggregate of those configured
>> individual worker limits. For example, if each worker is limited by
>> 1Mbps, then the 4-worker instance may produce up to 4Mbps traffic.
>>
>> In reality, since individual workers usually receive different amounts
>> of traffic (especially until [2] is unblocked), the effective instance
>> limit will be more than 1Mbps and less than 4Mbps.
>>
>> [1] https://wiki.squid-cache.org/Features/SmpScale#What_can_workers_share.3F
>>
>> [2] https://github.com/squid-cache/squid/pull/369
>>
>> Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools not working with squid 4.x , and more Question !!

[--Ahmad--]

Hi Alex thanks for info .
so i can confirm 100 % its a bug 

bec same config exactly work on 3.5

if you recommend me any thing 4.x that work with delay pools or 5.x i would be 
thankful ! 


and thank you very much when you answered me about SMP and delay pools .

all is clear , Looking forward to hearing that bug fixed .

Thanks a lot .

> On Sep 22, 2019, at 5:07 PM, Alex Rousskov  
> wrote:
> 
> On 9/22/19 6:25 AM, --Ahmad-- wrote:
> 
>> i tested squid 4.8 and delay pools not working with it at all .
>> i reverted back to squid 3.5.x and i had delay pools working .
> 
>> Q1- do squid 4 support delay pools ?
> 
> It should. If it does not, there is a bug somewhere.
> 
> 
>> Q2- with squid 3.5.x we have SMP about 4 childs , and we are running delay 
>> pools .
>> does that mean speed ( with all 4 instances ) is 1/1 Mbps
>> or speed ( with all 4 instances ) is 4/4  Mbps?
> 
> According to [1], delay pools are not SMP-aware yet so you are
> essentially configuring individual worker limits: Workers do not share
> their limits and pools with each other. Hence, the effective Squid
> instance limit is, very approximately, the aggregate of those configured
> individual worker limits. For example, if each worker is limited by
> 1Mbps, then the 4-worker instance may produce up to 4Mbps traffic.
> 
> In reality, since individual workers usually receive different amounts
> of traffic (especially until [2] is unblocked), the effective instance
> limit will be more than 1Mbps and less than 4Mbps.
> 
> [1] https://wiki.squid-cache.org/Features/SmpScale#What_can_workers_share.3F
> 
> [2] https://github.com/squid-cache/squid/pull/369
> 
> Alex.
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools not working with squid 4.x , and more Question !!

[Alex Rousskov]

On 9/22/19 6:25 AM, --Ahmad-- wrote:

> i tested squid 4.8 and delay pools not working with it at all .
> i reverted back to squid 3.5.x and i had delay pools working .

> Q1- do squid 4 support delay pools ?

It should. If it does not, there is a bug somewhere.


> Q2- with squid 3.5.x we have SMP about 4 childs , and we are running delay 
> pools .
> does that mean speed ( with all 4 instances ) is 1/1 Mbps
> or speed ( with all 4 instances ) is 4/4  Mbps?

According to [1], delay pools are not SMP-aware yet so you are
essentially configuring individual worker limits: Workers do not share
their limits and pools with each other. Hence, the effective Squid
instance limit is, very approximately, the aggregate of those configured
individual worker limits. For example, if each worker is limited by
1Mbps, then the 4-worker instance may produce up to 4Mbps traffic.

In reality, since individual workers usually receive different amounts
of traffic (especially until [2] is unblocked), the effective instance
limit will be more than 1Mbps and less than 4Mbps.

[1] https://wiki.squid-cache.org/Features/SmpScale#What_can_workers_share.3F

[2] https://github.com/squid-cache/squid/pull/369

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools not working with squid 4.x , and more Question !!

[--Ahmad--]

Hi Amos but squid 4.8  did not get the config below to work :


delay_pools 1
delay_class 1 1
delay_parameters 1 3500/3500
delay_access 1 allow minh


but on squid 3.5 it worked .

Plz For Q2 , what will be speed if we have  4 worker

is it above ? or above * 4 ?

Thanks 



> On Sep 22, 2019, at 1:46 PM, Amos Jeffries  wrote:
> 
> On 22/09/19 10:25 pm, --Ahmad-- wrote:
>> Hello Folks ,
>> 
>> i tested squid 4.8 and delay pools not working with it at all .
>> i reverted back to squid 3.5.x and i had delay pools working .
>> 
>> Q1- do squid 4 support delay pools ?
>> 
> 
> Yes.
> 
>> 
>> Q2- with squid 3.5.x we have SMP about 4 childs , and we are running delay 
>> pools .
>> 
>> say i limited in the main config file 1/1 Mbps 
>> 
> 
> What did you configure *exactly*?
> 
>> does that mean speed ( with all 4 instances ) is 1/1 Mbps
>> or
>> speed ( with all 4 instances ) is 4/4  Mbps
>> 
>> ?
> 
> Neither.
> 
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools not working with squid 4.x , and more Question !!

[Amos Jeffries]

On 22/09/19 10:25 pm, --Ahmad-- wrote:
> Hello Folks ,
> 
> i tested squid 4.8 and delay pools not working with it at all .
> i reverted back to squid 3.5.x and i had delay pools working .
> 
> Q1- do squid 4 support delay pools ?
> 

Yes.

> 
> Q2- with squid 3.5.x we have SMP about 4 childs , and we are running delay 
> pools .
> 
> say i limited in the main config file 1/1 Mbps 
> 

What did you configure *exactly*?

> does that mean speed ( with all 4 instances ) is 1/1 Mbps
> or
>  speed ( with all 4 instances ) is 4/4  Mbps
> 
> ?

Neither.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Delay pools not working with squid 4.x , and more Question !!

[--Ahmad--]

Hello Folks ,

i tested squid 4.8 and delay pools not working with it at all .
i reverted back to squid 3.5.x and i had delay pools working .

Q1- do squid 4 support delay pools ?


Q2- with squid 3.5.x we have SMP about 4 childs , and we are running delay 
pools .

say i limited in the main config file 1/1 Mbps 

does that mean speed ( with all 4 instances ) is 1/1 Mbps
or
 speed ( with all 4 instances ) is 4/4  Mbps

?


Many Thanks 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools and external acl

[Alex Rousskov]

On 10/18/2018 07:23 AM, Danilo V wrote:

> Please check where is my mistake.
> 
> I implemented a custom external ACL

... but you are not _using_ that new "some_group" ACL. An ACL has no
effect unless it is actually used in some ACL-driven directive. You
probably want to add some_group to your http_access rules.

> http_port 3128
> auth_param basic program ...
> acl login proxy_auth REQUIRED
> http_access deny !login
> external_acl_type group ttl=360 ipv4 %LOGIN /ext_danilo_ldap_group.sh
> acl some_group external group Internet_Access
> acl groupInternet note group Internet_Access
> delay_pools 1
> delay_class 1 1
> delay_parameters 1 128000/128000
> delay_access 1 allow groupInternet

Alex.


> The external ACL type which handles such complex non-traffic things is
> clearly listed in the Squid FAQ (and the 'acl' directive documentation)
> as being a "slow" / async ACL type.
> 
> Delay pools is also clearly listed as an access control which only works
> with "fast" category ACL types.
> 
> 


> Your external ACL just needs to supply Squid with a "tag=XX" or
> "group=XX " annotation to label the transaction with whichever group
> matches.
> 
>  # login is required to do group checking...
>  acl login proxy_auth REQUIRED
>  http_access deny !login
> 
> 
>  # the decision to allow the traffic into the proxy does group checks
> and adds annotations...
> 
>  external_acl_type group %LOGIN ...
>  acl some_group external group XX
> 
>  http_access allow some_group_check
> 
> 
>  # the decision of what pool(s) to apply has to work FAST - so uses the
> annotations already present or not present) as its decider:
> 
>  acl groupXX note group XX
> 
>  # or for older Squid
>  acl groupXX note tag XX
> 
>  delay_access N allow groupXX
> 
> 
> Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools and external acl

[Danilo V]

Hi, thanks for your message.

Not working yet. Please check where is my mistake.

I implemented a custom external ACL that checks on active directory via
ldap if a user is member of a particular group. If success returns:
OK group=Internet_Access
Else returns:
ERR

Squid.conf:
http_port 3128
auth_param basic program /usr/lib/squid/basic_ldap_auth -v 3 -b
dc=lab-novo,dc=br,dc=local -R -D
cn=ldap_proxy,ou=gestao_proxy,dc=lab-novo,dc=br,dc=local -w xxx -f
"sAMAccountName=%s" -u uid -P 10.0.0.1:389
acl login proxy_auth REQUIRED
http_access deny !login
external_acl_type group ttl=360 ipv4 %LOGIN /ext_danilo_ldap_group.sh
acl some_group external group Internet_Access
acl groupInternet note group Internet_Access
delay_pools 1
delay_class 1 1
delay_parameters 1 128000/128000
delay_access 1 allow groupInternet

Sqstat confirms that the bandwidth is not being limited.


Date: Wed, 17 Oct 2018 16:38:03 +1300
From: Amos Jeffries 
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Delay pools and external acl
Message-ID: 
Content-Type: text/plain; charset=utf-8

On 16/10/18 11:09 AM, Danilo V wrote:
> Hi all,
>
> Has anyone succeeded applying delay pools on groups from AD?
>
> I'm using squid 3.5.23 with basic_ldap_auth.
> I initially tried to combine mapping groups with external acl type
> (ext_ldap_group_acl) to delay pools. It's a trap :-(
>

A trap?

For starters; "group" is an abstract concept buried in the depths of
authentication which has nothing to do with traffic. It is a purely
human scoping idea. Squid knows nothing of any "group".


The external ACL type which handles such complex non-traffic things is
clearly listed in the Squid FAQ (and the 'acl' directive documentation)
as being a "slow" / async ACL type.

Delay pools is also clearly listed as an access control which only works
with "fast" category ACL types.

<https://wiki.squid-cache.org/SquidFaq/SquidAcl#Fast_and_Slow_ACLs>



> After doing more search I found about class 5 and note acl.
> Has anyone a pratical implementation in this scenario?

Yes several admin have done so. But with custom helpers that integrate
with the new annotation system, or the Kerberos helpers that have been
upgraded to integrate as well. Other helpers have not been updated yet.


Your external ACL just needs to supply Squid with a "tag=XX" or
"group=XX " annotation to label the transaction with whichever group
matches.

 # login is required to do group checking...
 acl login proxy_auth REQUIRED
 http_access deny !login


 # the decision to allow the traffic into the proxy does group checks
and adds annotations...

 external_acl_type group %LOGIN ...
 acl some_group external group XX

 http_access allow some_group_check


 # the decision of what pool(s) to apply has to work FAST - so uses the
annotations already present or not present) as its decider:

 acl groupXX note group XX

 # or for older Squid
 acl groupXX note tag XX

 delay_access N allow groupXX


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools and external acl

[Amos Jeffries]

On 16/10/18 11:09 AM, Danilo V wrote:
> Hi all,
> 
> Has anyone succeeded applying delay pools on groups from AD?
> 
> I'm using squid 3.5.23 with basic_ldap_auth.
> I initially tried to combine mapping groups with external acl type
> (ext_ldap_group_acl) to delay pools. It's a trap :-(
> 

A trap?

For starters; "group" is an abstract concept buried in the depths of
authentication which has nothing to do with traffic. It is a purely
human scoping idea. Squid knows nothing of any "group".


The external ACL type which handles such complex non-traffic things is
clearly listed in the Squid FAQ (and the 'acl' directive documentation)
as being a "slow" / async ACL type.

Delay pools is also clearly listed as an access control which only works
with "fast" category ACL types.





> After doing more search I found about class 5 and note acl.
> Has anyone a pratical implementation in this scenario?

Yes several admin have done so. But with custom helpers that integrate
with the new annotation system, or the Kerberos helpers that have been
upgraded to integrate as well. Other helpers have not been updated yet.


Your external ACL just needs to supply Squid with a "tag=XX" or
"group=XX " annotation to label the transaction with whichever group
matches.

 # login is required to do group checking...
 acl login proxy_auth REQUIRED
 http_access deny !login


 # the decision to allow the traffic into the proxy does group checks
and adds annotations...

 external_acl_type group %LOGIN ...
 acl some_group external group XX

 http_access allow some_group_check


 # the decision of what pool(s) to apply has to work FAST - so uses the
annotations already present or not present) as its decider:

 acl groupXX note group XX

 # or for older Squid
 acl groupXX note tag XX

 delay_access N allow groupXX


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Alex Rousskov]

On 08/07/2018 09:20 AM, Julian Perconti wrote:

> Yesterday i have compiled squid 4.2.
> 
> When site is spliced delay_pools still does not working.
> 
> Any news?

The latest information and suggestions I have is at
http://lists.squid-cache.org/pipermail/squid-users/2018-July/018636.html

Alex.


>> -Mensaje original-
>> De: squid-users  En nombre de
>> Eliezer Croitoru
>> Enviado el: miércoles, 18 de julio de 2018 13:47
>> Para: squid-users@lists.squid-cache.org
>> Asunto: Re: [squid-users] Delay pools in squid4 not working with https
>>
>> Just to mention QUIC related wiki links:
>> - https://wiki.squid-
>> cache.org/KnowledgeBase/Block%20QUIC%20protocol?highlight=%28QUIC%
>> 29
>> - https://wiki.squid-
>> cache.org/ConfigExamples/Intercept/CiscoIOSv15Wccp2?highlight=%28QUIC
>> %29#QUIC.2FSPDY_protocol_blocking
>>
>> Eliezer
>>
>> 
>> Eliezer Croitoru
>> Linux System Administrator
>> Mobile: +972-5-28704261
>> Email: elie...@ngtech.co.il
>>
>>
>>
>> -Original Message-
>> From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On
>> Behalf Of Amos Jeffries
>> Sent: Wednesday, July 11, 2018 12:35 AM
>> To: squid-users@lists.squid-cache.org
>> Subject: Re: [squid-users] Delay pools in squid4 not working with https
>>
>> On 11/07/18 07:50, Paolo Marzari wrote:
>>> My home server just updated from 3.5.27, everything is working fine,
>>> but delay pools seems broken to me.
>>> I capped some devices to 240kb/s and tried to download a debian ISO
>>> with one of them...all good, 240kb/s.
>>> Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed.
>>>
>>> So I tried youtube videos, no cap at all, same problem with facebook.
>>> Revert to 3.5.27 and delays works again with every type of traffic.
>>>
>>> I think there's something wrong with https traffic.
>>>
>>
>> a) is it actually HTTPS traffic?
>>
>> b) are the bytes going through the proxy 2.2Mbps or 240kbps ?
>>
>> I ask because Google/YouTube and Facebook are services using HTTP/2 with
>> high compression features as much as possible. So while the proxy is set to
>> transfer X bytes per second, when hidden inside "HTTPS" those X bytes may
>> show up as 90*X bytes of traffic when decompressed by a Browser.
>>
>> Or the transfer may be QUIC protocol, completely bypassing the HTTP the
>> proxy is counting.
>>
>> Amos
>> ___
>> squid-users mailing list
>> squid-users@lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
>>
>> ___
>> squid-users mailing list
>> squid-users@lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Eliezer Croitoru]

Just to mention QUIC related wiki links:
- 
https://wiki.squid-cache.org/KnowledgeBase/Block%20QUIC%20protocol?highlight=%28QUIC%29
- 
https://wiki.squid-cache.org/ConfigExamples/Intercept/CiscoIOSv15Wccp2?highlight=%28QUIC%29#QUIC.2FSPDY_protocol_blocking

Eliezer


Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: elie...@ngtech.co.il



-Original Message-
From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On Behalf 
Of Amos Jeffries
Sent: Wednesday, July 11, 2018 12:35 AM
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Delay pools in squid4 not working with https

On 11/07/18 07:50, Paolo Marzari wrote:
> My home server just updated from 3.5.27, everything is working fine, 
> but delay pools seems broken to me.
> I capped some devices to 240kb/s and tried to download a debian ISO 
> with one of them...all good, 240kb/s.
> Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed.
> 
> So I tried youtube videos, no cap at all, same problem with facebook.
> Revert to 3.5.27 and delays works again with every type of traffic.
> 
> I think there's something wrong with https traffic.
> 

a) is it actually HTTPS traffic?

b) are the bytes going through the proxy 2.2Mbps or 240kbps ?

I ask because Google/YouTube and Facebook are services using HTTP/2 with high 
compression features as much as possible. So while the proxy is set to transfer 
X bytes per second, when hidden inside "HTTPS" those X bytes may show up as 
90*X bytes of traffic when decompressed by a Browser.

Or the transfer may be QUIC protocol, completely bypassing the HTTP the proxy 
is counting.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Julian Perconti]

> -Mensaje original-
> De: Julian Perconti [mailto:vh1...@yahoo.com.ar]
> Enviado el: jueves, 12 de julio de 2018 21:24
> Para: 'squid-users@lists.squid-cache.org'  cache.org>
> Asunto: RE: [squid-users] Delay pools in squid4 not working with https
> 
> > -Mensaje original-
> > De: Alex Rousskov [mailto:rouss...@measurement-factory.com]
> > Enviado el: jueves, 12 de julio de 2018 21:20
> > Para: Julian Perconti ; squid-users@lists.squid-
> > cache.org
> > Asunto: Re: [squid-users] Delay pools in squid4 not working with https
> >
> > On 07/12/2018 06:16 PM, Julian Perconti wrote:
> > >> De: Alex Rousskov
> > >> If you start splicing/tunneling, it will probably stop working.
> >
> >
> > > Ok, but is not is supposed that this is the normal behaviour?
> >
> >
> > No, Squid should apply delay pools to all traffic.

Ok, I did not know that..

> 
> OK I Will splice https://speed.hetzner.de/ and then tell You what happened
> with delay pool.
> 
> An important thing, the delay_pool cfg that Paolo has is pretty complexthan
> mine.

Confirmed.

Splicing.. speed.hetzner.de

TCP_TUNNEL/200 4452 CONNECT 88.198.248.254:443 - ORIGINAL_DST/88.198.248.254 -

The delay_pool does not work.

Download speed never goes down.

delay_pool class 2 cfg:

delay_pools 1 
delay_class 1 2
delay_access 1 allow all

delay_parameters 1 -1/-1 10/104857600

Version:

Squid Cache: Version 4.1
Service Name: squid

This binary uses OpenSSL 1.1.0f  25 May 2017. For legal restrictions on 
distribution see https://www.openssl.org/source/license.html

> 
> >
> >
> > > I mean, TCP_TUNNEL = squid forward, so squid can not do nothing
> > > about
> > the spliced connection.
> >
> >
> > Squid knows how many bytes it is forwarding, and that is all Squid
> > needs to know to shape traffic.
> >
> > Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Julian Perconti]

> -Mensaje original-
> De: Alex Rousskov [mailto:rouss...@measurement-factory.com]
> Enviado el: jueves, 12 de julio de 2018 21:20
> Para: Julian Perconti ; squid-users@lists.squid-
> cache.org
> Asunto: Re: [squid-users] Delay pools in squid4 not working with https
> 
> On 07/12/2018 06:16 PM, Julian Perconti wrote:
> >> De: Alex Rousskov
> >> If you start splicing/tunneling, it will probably stop working.
> 
> 
> > Ok, but is not is supposed that this is the normal behaviour?
> 
> 
> No, Squid should apply delay pools to all traffic.

OK I Will splice https://speed.hetzner.de/ and then tell You what happened with 
delay pool.

An important thing, the delay_pool cfg that Paolo has is pretty complexthan 
mine.

> 
> 
> > I mean, TCP_TUNNEL = squid forward, so squid can not do nothing about
> the spliced connection.
> 
> 
> Squid knows how many bytes it is forwarding, and that is all Squid needs to
> know to shape traffic.
> 
> Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Alex Rousskov]

On 07/12/2018 06:16 PM, Julian Perconti wrote:
>> De: Alex Rousskov
>> If you start splicing/tunneling, it will probably stop working.


> Ok, but is not is supposed that this is the normal behaviour? 


No, Squid should apply delay pools to all traffic.


> I mean, TCP_TUNNEL = squid forward, so squid can not do nothing about the 
> spliced connection.


Squid knows how many bytes it is forwarding, and that is all Squid needs
to know to shape traffic.

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Julian Perconti]

> -Mensaje original-
> De: Alex Rousskov [mailto:rouss...@measurement-factory.com]
> Enviado el: jueves, 12 de julio de 2018 21:03
> Para: Julian Perconti ; squid-users@lists.squid-
> cache.org
> Asunto: Re: [squid-users] Delay pools in squid4 not working with https
> 
> On 07/12/2018 05:42 PM, Julian Perconti wrote:
> >> De: Alex Rousskov
> >> On 07/12/2018 05:19 PM, Julian Perconti wrote:
> >>
> >>> From my side, the tests were done with full SSL-Bump; downloading a
> >>> file from: https://speed.hetzner.de/
> >>>
> >>> No splice.
> 
> 
> >> My "not working" statement was specific to tunneling code. When Squid
> >> bumps, it does not tunnel, so your tests did not tickle the broken code.
> >> We do not yet know whether prazola is bumping HTTPS traffic.
> >>
> >> Tunneling happens when handling CONNECT requests without SslBump
> and
> >> when splicing TLS traffic with SslBump.
> 
> 
> > My delay_pool cfg is working.
> 
> Yes, I understand. I do not think anybody has claimed that your config should
> not be working. The only claim was that delay pools do not work when Squid
> tunnels traffic. Your Squid does not tunnel traffic.
> 
> 
> > Without splice/tunneling the connection.
> 
> ... and that is why it is working. If you start splicing/tunneling, it will 
> probably
> stop working.

Ok, but is not is supposed that this is the normal behaviour? 

I mean, TCP_TUNNEL = squid forward, so squid can not do nothing about the 
spliced connection.

I don't I am just a squid user... and BTW new in squid SSL intercepts.

> 
> 
> Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Alex Rousskov]

On 07/12/2018 05:42 PM, Julian Perconti wrote:
>> De: Alex Rousskov
>> On 07/12/2018 05:19 PM, Julian Perconti wrote:
>>
>>> From my side, the tests were done with full SSL-Bump; downloading a
>>> file from: https://speed.hetzner.de/
>>>
>>> No splice.


>> My "not working" statement was specific to tunneling code. When Squid
>> bumps, it does not tunnel, so your tests did not tickle the broken code.
>> We do not yet know whether prazola is bumping HTTPS traffic.
>>
>> Tunneling happens when handling CONNECT requests without SslBump and
>> when splicing TLS traffic with SslBump.


> My delay_pool cfg is working.

Yes, I understand. I do not think anybody has claimed that your config
should not be working. The only claim was that delay pools do not work
when Squid tunnels traffic. Your Squid does not tunnel traffic.


> Without splice/tunneling the connection.

... and that is why it is working. If you start splicing/tunneling, it
will probably stop working.


Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Julian Perconti]

> De: Alex Rousskov [mailto:rouss...@measurement-factory.com]
> Enviado el: jueves, 12 de julio de 2018 20:31
> Para: Julian Perconti ; squid-users@lists.squid-
> cache.org
> Asunto: Re: [squid-users] Delay pools in squid4 not working with https
> 
> On 07/12/2018 05:19 PM, Julian Perconti wrote:
> 
> > From my side, the tests were done with full SSL-Bump; downloading a
> > file from: https://speed.hetzner.de/
> >
> > No splice.
> 
> My "not working" statement was specific to tunneling code. When Squid
> bumps, it does not tunnel, so your tests did not tickle the broken code.
> We do not yet know whether prazola is bumping HTTPS traffic.
> 
> Tunneling happens when handling CONNECT requests without SslBump and
> when splicing TLS traffic with SslBump.
> 
> Alex.

My delay_pool cfg is working.

Without splice/tunneling the connection.
When I download a file from https://speed.hetzner.de/ with  the https prefix in 
the URL  downloaded file (without splice anything), the delay slows down the 
download once the limit is reached.

May be I missunderstood something.

Regards


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Alex Rousskov]

On 07/12/2018 05:19 PM, Julian Perconti wrote:

> From my side, the tests were done with full SSL-Bump; downloading a file 
> from: https://speed.hetzner.de/
> 
> No splice.

My "not working" statement was specific to tunneling code. When Squid
bumps, it does not tunnel, so your tests did not tickle the broken code.
We do not yet know whether prazola is bumping HTTPS traffic.

Tunneling happens when handling CONNECT requests without SslBump and
when splicing TLS traffic with SslBump.

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Amos Jeffries]

On 12/07/18 11:39, Julian Perconti wrote:
>>>
>>> El ‎martes‎, ‎10‎ de ‎julio‎ de ‎2018‎ ‎18‎:‎57‎:‎43‎ ‎-03, Alex Rousskov 
>>> escribió: 
>>>
>>>
>>> On 07/10/2018 01:50 PM, Paolo Marzari wrote:
 My home server just updated from 3.5.27, everything is working fine, but
 delay pools seems broken to me.
>>>
 Revert to 3.5.27 and delays works again with every type of traffic.

 I think there's something wrong with https traffic.
>>>
>>> You are probably right. A few days ago, while working on an unrelated
>>> project, we have found a bug in delay pools support for tunneled https
>>> traffic. That support was probably broken by v4 commit 6b2b6cf. We have
>>> not tested v3.5, so I can only confirm that v4 and v5 are broken.
>>>
>>> The bug will be fixed as a side effect of "peering support for SslBump"
>>> changes that should be ready for the official review soon. If you would
>>> like to test our unofficial branch, the code is available at
>>> https://github.com/measurement-factory/squid/tree/SQUID-360-peering-for-SslBump
>>>
>>>
>>> HTH,
>>>
>>> Alex.
>>>
>>> ___
>>> squid-users mailing list
>>> squid-users@lists.squid-cache.org
>>> http://lists.squid-cache.org/listinfo/squid-users
> 
> I can confirm that delay_pools works fine both http and https protocols in 
> squid 4 running debian 9 
> 
> Squid Cache: Version 4.1 

When I looked at the code for Paolos report I found there to be a
difference between SSL-Bumped and non-Bumped traffic.

This hints to me that these opposite reports may due to how the traffic
is being handled.

So Julian, Paolo; if you don't mind can you please say whether you are
using SSL-Bump in your tests and if so whether the test traffic got
splice'd, bump'ed or no SSL-Bump feature use at all ?


There might also still be bugs specific to pool types. We have had a few
in the past that I'm not sure if ever got fixed. Though Paolo's mention
that 3.5 worked okay hints that its probably not those exact issues.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Julian Perconti]

>> 
>> El ‎martes‎, ‎10‎ de ‎julio‎ de ‎2018‎ ‎18‎:‎57‎:‎43‎ ‎-03, Alex Rousskov 
>>  escribió: 
>> 
>> 
>> On 07/10/2018 01:50 PM, Paolo Marzari wrote:
>>> My home server just updated from 3.5.27, everything is working fine, but
>>> delay pools seems broken to me.
>> 
>>> Revert to 3.5.27 and delays works again with every type of traffic.
>>> 
>>> I think there's something wrong with https traffic.
>> 
>> You are probably right. A few days ago, while working on an unrelated
>> project, we have found a bug in delay pools support for tunneled https
>> traffic. That support was probably broken by v4 commit 6b2b6cf. We have
>> not tested v3.5, so I can only confirm that v4 and v5 are broken.
>> 
>> The bug will be fixed as a side effect of "peering support for SslBump"
>> changes that should be ready for the official review soon. If you would
>> like to test our unofficial branch, the code is available at
>> https://github.com/measurement-factory/squid/tree/SQUID-360-peering-for-SslBump
>> 
>> 
>> HTH,
>> 
>> Alex.
>> 
>> ___
>> squid-users mailing list
>> squid-users@lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users

I can confirm that delay_pools works fine both http and https protocols in 
squid 4 running debian 9 

Squid Cache: Version 4.1 
Service Name: squid 
 
Here the cfg: 
 
delay_pools 1 
delay_class 1 2 

delay_access 1 allow all 
 
delay_parameters 1 -1/-1 10/104857600 # ~100KBs/~100MB 
delay_initial_bucket_level 50

Regards
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Alex Rousskov]

On 07/10/2018 01:50 PM, Paolo Marzari wrote:
> My home server just updated from 3.5.27, everything is working fine, but
> delay pools seems broken to me.

> Revert to 3.5.27 and delays works again with every type of traffic.
> 
> I think there's something wrong with https traffic.

You are probably right. A few days ago, while working on an unrelated
project, we have found a bug in delay pools support for tunneled https
traffic. That support was probably broken by v4 commit 6b2b6cf. We have
not tested v3.5, so I can only confirm that v4 and v5 are broken.

The bug will be fixed as a side effect of "peering support for SslBump"
changes that should be ready for the official review soon. If you would
like to test our unofficial branch, the code is available at
https://github.com/measurement-factory/squid/tree/SQUID-360-peering-for-SslBump


HTH,

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[prazola]

A fast check with nbwmon shows 2.2Mbps when using squid 4.1.




--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools in squid4 not working with https

[Amos Jeffries]

On 11/07/18 07:50, Paolo Marzari wrote:
> My home server just updated from 3.5.27, everything is working fine, but
> delay pools seems broken to me.
> I capped some devices to 240kb/s and tried to download a debian ISO with
> one of them...all good, 240kb/s.
> Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed.
> 
> So I tried youtube videos, no cap at all, same problem with facebook.
> Revert to 3.5.27 and delays works again with every type of traffic.
> 
> I think there's something wrong with https traffic.
> 

a) is it actually HTTPS traffic?

b) are the bytes going through the proxy 2.2Mbps or 240kbps ?

I ask because Google/YouTube and Facebook are services using HTTP/2 with
high compression features as much as possible. So while the proxy is set
to transfer X bytes per second, when hidden inside "HTTPS" those X bytes
may show up as 90*X bytes of traffic when decompressed by a Browser.

Or the transfer may be QUIC protocol, completely bypassing the HTTP the
proxy is counting.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Delay pools in squid4 not working with https

[prazola]

My home server just updated from 3.5.27, everything is working fine, but
delay pools seems broken to me.
I capped some devices to 240kb/s and tried to download a debian ISO with one
of them...all good, 240kb/s.
Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed.

So I tried youtube videos, no cap at all, same problem with facebook.
Revert to 3.5.27 and delays works again with every type of traffic.

I think there's something wrong with https traffic.

Here's my delay config section:

acl group288 src 192.168.0.87/32 192.168.0.88/32 192.168.0.84/32
acl groupapo src 192.168.0.56/32 #192.168.0.6/32
acl group656 src 192.168.0.61/32 192.168.0.89/32
acl group656b src 192.168.0.95/32 #192.168.0.112/32 192.168.0.96/32
#192.168.0.6/32
acl group1024 src 192.168.0.92/32
#acl limit5conn maxconn 5
delay_pools 4
delay_class 1 1
delay_class 2 1
delay_class 3 1
delay_class 4 1
delay_parameters 1 288000/308000
delay_parameters 2 595000/64
delay_parameters 3 595200/640400
delay_parameters 4 972000/1024000
delay_access 1 allow group288
delay_access 1 allow groupapo
delay_access 2 allow group656
delay_access 3 allow group656b
delay_access 4 allow group1024
delay_access 1 deny all
delay_access 2 deny all
delay_access 3 deny all
delay_access 4 deny all

Am I missing something in my config?
I need your help squid's gurus...and sorry for bad englando.



--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Delay pools in squid4 not working with https

[Paolo Marzari]

My home server just updated from 3.5.27, everything is working fine, but 
delay pools seems broken to me.
I capped some devices to 240kb/s and tried to download a debian ISO with 
one of them...all good, 240kb/s.

Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed.

So I tried youtube videos, no cap at all, same problem with facebook.
Revert to 3.5.27 and delays works again with every type of traffic.

I think there's something wrong with https traffic.

Here's my delay config section:

   acl group288 src 192.168.0.87/32 192.168.0.88/32 192.168.0.84/32
   acl groupapo src 192.168.0.56/32 #192.168.0.6/32
   acl group656 src 192.168.0.61/32 192.168.0.89/32
   acl group656b src 192.168.0.95/32 #192.168.0.112/32 192.168.0.96/32
   #192.168.0.6/32
   acl group1024 src 192.168.0.92/32
   #acl limit5conn maxconn 5
   delay_pools 4
   delay_class 1 1
   delay_class 2 1
   delay_class 3 1
   delay_class 4 1
   delay_parameters 1 288000/308000
   delay_parameters 2 595000/64
   delay_parameters 3 595200/640400
   delay_parameters 4 972000/1024000
   delay_access 1 allow group288
   delay_access 1 allow groupapo
   delay_access 2 allow group656
   delay_access 3 allow group656b
   delay_access 4 allow group1024
   delay_access 1 deny all
   delay_access 2 deny all
   delay_access 3 deny all
   delay_access 4 deny all

Am I missing something in my config?
I need your help squid's gurus...and sorry for bad englando.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools for authentcated users

[Amos Jeffries]

On 16/09/2016 11:45 p.m., Verónica Ovando wrote:
> Hi!
> 
> I am trying to set up delay pools for AD authenticated users.
> 
> I run Squid 3.4.8 in a Debian 8 server.
> 
> 
> I configured come delay pools, but they really don't have effect. What I want 
> to do is to provide full bandwidth for some pages and create a delay for ALL 
> the others. This is because I can't restrict internet surfing and I need a 
> solution and control bandwidth usage.
> 
> 
> (Squid is working without problems with AD, so I will omit some directives 
> about authentication )
> 

delay_access is a 'fast' category access control. It cannot do auth or
group lookups itself. In order to work with those type of ACL it
requires a previous access control (usually http_access) to have checked
them first and recorded the results as part of the transacion state.


> 
> For example:
> 
> 
> #ACLs**#
> 
> acl AD_Standard external Grupos_AD Standard
> 
> acl redLocal src 90.0.0.0/22
> 
> 
> 
> #Delay Pools**#
> 
> delay_pools 3
> 
> 
> delay_class 1 4
> 
> delay_access 1 allow AD_Standard socialNets
> delay_access 1 deny all
> delay_parameters 1 32000/32000 8000/8000 600/64000 1000/1
> 
> delay_class 2 1
> delay_parameters 2 -1/-1

This is a useless pool. It wastes time calculating bandwidth caps
delays, only to not do any limiting.

Instead of having an "unlimited" pool, simply deny these transactions
from having one of the other pools applied to them. By definition
anything which does not have a pool assigned is unlimited.


Especially since when a transaction meets the criteria for multiple
pools they will *all* have some effect on that transactions bandwidth.
Which can lead to some weird behaviours and (negative!) available
bandwidth values in the byte accounting.


> delay_access 2 allow redLocal redLocal
> delay_access 2 allow redLocal oficiales
> delay_access 2 allow redLocal diarios
> delay_access 2 allow redLocal bancos
> delay_access 2 allow redLocal tarjCred
> delay_access 2 allow redLocal inmueble
> delay_access 2 allow redLocal mails
> delay_access 2 allow redLocal externos
> delay_access 2 allow redLocal varias
> delay_access 2 allow redLocal servicios
> delay_access 2 deny all
> 
> delay_class 3 4
> delay_parameters 3 32000/32000 8000/8000 1/64000 15000/5
> delay_access 3 allow AD_Standard all
> delay_access 3 deny all
> 
> #***#
> 
> 
> So, I am creating three delay_pools, the first one provides 10KB for
> each user (for the AD group Standard), no matters hoy many hosts are
> logged in; the second one provides full usage of the bandwidth for my
> local network to access those pages; the third delay provides up to 50KB
> for the ALL the websites, with exception of those defined in the delay
> 2. Is this correct?

No. The 'restore' is the averaged N/sec byte amount. The smallest of the
parameters for each pool will be the limiting factor.


A transaction that gets assigned to pool #3 will be able to download at
most (ever) 8000 bytes in one second (due to the 8000/8000 bucket). The
other buckets are all larger, so they will refill faster than they are
allowed to drain.
 ** Also the 8000 bucket is the per-network bucket. So you have 8000
bytes/sec being shared by each /24 subnet of clients.


A transaction that gets assigned to pool #1 will be able to download at
most (ever) 8000 bytes in one second (due to the 8000/8000 bucket).
However, the other pools refill at slower rates so it gets more complex...

Assuming that pool #1 is completely full to begin with, and only 1
transaction happens:
 For the 1st second of transfer that maximum 8000 B/sec will happen.
 For the 2nd second of transfer the bandwidth will drop to 3000 B/sec
(there will now only be 3000 bytes in the per-user bucket).
 For the 3rd second of transfer the bandwidth will drop to 1000 B/sec
(the refill rate of the per-user bucket).
 Then 135 seconds later if the transaction is still going the 64000
bucket will be drained and start limiting the transfer to 600 B/sec (the
refill rate of the per-network bucket).

If anything else is going on these buckets may drain faster than
mentioned above as they may (or not) get shared by parallel transaction
and clients.
 If the proxy gets loaded you will increasingly not see the initial
'high' speeds, just the 1000 B/sec or 600 B/sec rates being applied most
of the time.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Delay pools for authentcated users

[Verónica Ovando]

Hi!

I am trying to set up delay pools for AD authenticated users.

I run Squid 3.4.8 in a Debian 8 server.


I configured come delay pools, but they really don't have effect. What I want 
to do is to provide full bandwidth for some pages and create a delay for ALL 
the others. This is because I can't restrict internet surfing and I need a 
solution and control bandwidth usage.


(Squid is working without problems with AD, so I will omit some directives 
about authentication )


For example:


#ACLs**#

acl AD_Standard external Grupos_AD Standard

acl redLocal src 90.0.0.0/22



#Delay Pools**#

delay_pools 3


delay_class 1 4

delay_access 1 allow AD_Standard socialNets
delay_access 1 deny all
delay_parameters 1 32000/32000 8000/8000 600/64000 1000/1

delay_class 2 1
delay_parameters 2 -1/-1
delay_access 2 allow redLocal redLocal
delay_access 2 allow redLocal oficiales
delay_access 2 allow redLocal diarios
delay_access 2 allow redLocal bancos
delay_access 2 allow redLocal tarjCred
delay_access 2 allow redLocal inmueble
delay_access 2 allow redLocal mails
delay_access 2 allow redLocal externos
delay_access 2 allow redLocal varias
delay_access 2 allow redLocal servicios
delay_access 2 deny all

delay_class 3 4
delay_parameters 3 32000/32000 8000/8000 1/64000 15000/5
delay_access 3 allow AD_Standard all
delay_access 3 deny all

#***#

So, I am creating three delay_pools, the first one provides 10KB for each user 
(for the AD group Standard), no matters hoy many hosts are logged in; the 
second one provides full usage of the bandwidth for my local network to access 
those pages; the third delay provides up to 50KB for the ALL the websites, with 
exception of those defined in the delay 2. Is this correct?

Thanks in advance.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools and HTTPS on Squid 3.x

[Eliezer Croitoru]

On 18/02/2016 04:02, Hery Martin wrote:

@Eliezer
I'm using Ubuntu Server 14.04 (not especial decision, because I use to
deploy different distros in a Citrix XenServer test environment)
Have you any guide to implements QOS+Squid? As I said, I saw in many
articles that you have to mark the traffic in Squid to deal with him after
but I'm never tried because didn't had enough information about.


I am in a similar position like you.
I have implemented QOS once or twice but I always need to learn it from 0.
I have seen couple nice scripts in FireHOL and arch linux tutorials.
But I will need to re-read many things to get a hold on how it works and 
should be configured.


If I will have enough time I will try to write about it in the squid 
wiki somewhere in the future.


Eliezer
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools and HTTPS on Squid 3.x

[Hery Martin]

Hey guys!

Finally today I had a chance and tested the version 3.5.12

Just like many of you said, the issue is fixed.

I want to say THANKS to everyone who comments in this thread, all of you
saved my day!

Thanks again!

On Wed, Feb 17, 2016 at 9:02 PM, Hery Martin  wrote:

> Hey guys... First of all say that I'm very thankful about the quick replies
> and advises from everyone.
>
> @Fred and @Alberto:
> Reading across all the thread I think that definitely going to try a 3.5.x
> release tomorrow and I'll report the results here.
>
> @babajaga:
> I use to block this kind of traffic using the domain lists generated by
> Shalla Secure Services (shallalist.de) and also blocks some other
> contents.
>
> @Eliezer
> I'm using Ubuntu Server 14.04 (not especial decision, because I use to
> deploy different distros in a Citrix XenServer test environment)
> Have you any guide to implements QOS+Squid? As I said, I saw in many
> articles that you have to mark the traffic in Squid to deal with him after
> but I'm never tried because didn't had enough information about.
>
> @Alberto:
> Happy to know about your feedback
>
>
>
> On Wed, Feb 17, 2016 at 8:19 PM, Alberto Perez [via Squid Web Proxy Cache]
> <
> ml-node+s1019090n4676071...@n4.nabble.com> wrote:
>
> > I can confirm that this bug is fixed in 3.5.12, I am from Cuba too,
> > used to have two delays, one for http and one for https with the 2x
> > workaround mentioned here, after my last upgrade to 3.5.12 the issue
> > is gone.
> >
> > Also I will highly recommend to use 3.5.x versions, there is a HUGE
> > difference in lot of things including SSL-BUMP, what in your case,
> > with that small amount of workstations, I would suggest to implement
> > as you could easy install the custom certificate in all of them, it
> > may considerably increase your HIT rate and save a lot of bandwidth
> > since we have too much https traffic now days.
> >
> > Kind regards
> >
> > Alberto
> >
> >
> >
> >
> >
> > On 2/17/16, FredB <[hidden email]
> > > wrote:
> >
> > > There was a know bug about delay pool and HTTPS, but as far as I know
> > it's
> > > fixed now
> > > you did a test with 3.5.x ?
> > >
> > > Fred
> > > ___
> > > squid-users mailing list
> > > [hidden email] 
> > > http://lists.squid-cache.org/listinfo/squid-users
> > >
> > ___
> > squid-users mailing list
> > [hidden email] 
> > http://lists.squid-cache.org/listinfo/squid-users
> >
> >
> > --
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://squid-web-proxy-cache.1019090.n4.nabble.com/Delay-Pools-and-HTTPS-on-Squid-3-x-tp4676043p4676071.html
> > To unsubscribe from Delay Pools and HTTPS on Squid 3.x, click here
> > <
> http://squid-web-proxy-cache.1019090.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code=4676043=c2NvcnBpb254aWlAZ21haWwuY29tfDQ2NzYwNDN8MTE2NzYzMzM3NA==
> >
> > .
> > NAML
> > <
> http://squid-web-proxy-cache.1019090.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer=instant_html%21nabble%3Aemail.naml=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
>
>
>
>
> --
> View this message in context:
> http://squid-web-proxy-cache.1019090.n4.nabble.com/Delay-Pools-and-HTTPS-on-Squid-3-x-tp4676043p4676072.html
> Sent from the Squid - Users mailing list archive at Nabble.com.
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools and HTTPS on Squid 3.x

[Hery Martin]

Hey guys... First of all say that I'm very thankful about the quick replies
and advises from everyone.

@Fred and @Alberto:
Reading across all the thread I think that definitely going to try a 3.5.x
release tomorrow and I'll report the results here.

@babajaga:
I use to block this kind of traffic using the domain lists generated by
Shalla Secure Services (shallalist.de) and also blocks some other contents.

@Eliezer
I'm using Ubuntu Server 14.04 (not especial decision, because I use to
deploy different distros in a Citrix XenServer test environment)
Have you any guide to implements QOS+Squid? As I said, I saw in many
articles that you have to mark the traffic in Squid to deal with him after
but I'm never tried because didn't had enough information about.

@Alberto:
Happy to know about your feedback



On Wed, Feb 17, 2016 at 8:19 PM, Alberto Perez [via Squid Web Proxy Cache] <
ml-node+s1019090n4676071...@n4.nabble.com> wrote:

> I can confirm that this bug is fixed in 3.5.12, I am from Cuba too,
> used to have two delays, one for http and one for https with the 2x
> workaround mentioned here, after my last upgrade to 3.5.12 the issue
> is gone.
>
> Also I will highly recommend to use 3.5.x versions, there is a HUGE
> difference in lot of things including SSL-BUMP, what in your case,
> with that small amount of workstations, I would suggest to implement
> as you could easy install the custom certificate in all of them, it
> may considerably increase your HIT rate and save a lot of bandwidth
> since we have too much https traffic now days.
>
> Kind regards
>
> Alberto
>
>
>
>
>
> On 2/17/16, FredB <[hidden email]
> > wrote:
>
> > There was a know bug about delay pool and HTTPS, but as far as I know
> it's
> > fixed now
> > you did a test with 3.5.x ?
> >
> > Fred
> > ___
> > squid-users mailing list
> > [hidden email] 
> > http://lists.squid-cache.org/listinfo/squid-users
> >
> ___
> squid-users mailing list
> [hidden email] 
> http://lists.squid-cache.org/listinfo/squid-users
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://squid-web-proxy-cache.1019090.n4.nabble.com/Delay-Pools-and-HTTPS-on-Squid-3-x-tp4676043p4676071.html
> To unsubscribe from Delay Pools and HTTPS on Squid 3.x, click here
> 
> .
> NAML
> 
>




--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Delay-Pools-and-HTTPS-on-Squid-3-x-tp4676043p4676072.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools and HTTPS on Squid 3.x

[FredB]

There was a know bug about delay pool and HTTPS, but as far as I know it's 
fixed now 
you did a test with 3.5.x ?

Fred
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools and HTTPS on Squid 3.x

[Eliezer Croitoru]

Hey Martin,

I was wondering if you had the chance of trying to enforce some QOS 
policy on the OS level?

Also what OS and distribution are you using?

Eliezer

On 17/02/2016 03:37, Hery Martin wrote:

Hello everybody:

Since a few months ago I'm using squid to provide a solution as small
business proxy in the network of my work place.

I'm from Cuba, in our country the Internet is a very limited resource. I
have only one link of 2Mbps to share with 20 ~ 25 users (even with my
network have more than 60) this is the normal concurrent number.

When I start the squid deployment in my network I started using
2.7stable9 version, I made all arrangements to put it work with my AD to
match ACLs using AD Groups and everything works perfect.

I defined 1 class 2 delay pools to to limits traffic to 12 KBytes/s per
user approx.

delay_pool 1
delay_class 1 2
delay_parameters -1/-1 12228/12228

The delay pool works perfect, I was checking with real-time tool sqstat
and with squidclient mgr:delay

NOW.

I recently upgrade squid to 3.3.8 and I notice that delay pool started
to going wrong when the users surf or download using HTTPS protocol

I checked in real-time and when the users browse HTTPS the pool goes in
negative numbers and start to grow and grow, its very easy to check,
just define a delay pool with 5KB and start a download from an HTTPS
source and you can check it with squidclient mgr:delay, the ip takes
negative pool value and keep growing until the download finish.

Frustrated with this behavior I put different squid versions in a
Virtualization Server and definitely I saw that the problem occurs with
squid 3.x versions, today I made a final test and I think that the
implementation of HTTP v1.1 is maybe related with that problem (I'm not
sure but tomorow I will make a few tests with squid 3.1 where HTTP v1.1
was not yet implemented)

Please, if you have the opportunity, just test this in a Lab
environment, I decided to write to this email list because I asked to
many people that already have implemented squid as proxy in their
networks and they didn't believed to me until I demostrated the issue.

Have anyone information about this bug? There is any hope to fix this
problem at code level?

Anyway, I'm computer systems engineer, I use to write a lot C++ lines
every week... I'm not related with the squid development (never saw the
code in my life) but if somebody have any idea how to fix this and wants
help just count with me.

Greetings from Cuba and sorry about my English :)


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools and HTTPS on Squid 3.x

[Hery Martin]

Hey djch Thanks for your quick reply...

Anyway, I know that delay pools are implemented at software layer, but
maybe the error was just a simple mistake porting the old squid 2 project.
Even when these days we have tools to do this more efficiently like TC-CBQ,
in environments where squid works as no transparent proxy, the delay pools
are very useful, in my country I'm pretty sure that a lot of network
administrator will be thankful about a fix. I was trying to handle this
using TC-CBQ in my proxy server but I read in many articles that you have
to mark the traffic in squid to make it work.

So... Do you think that maybe some developers can take the task if the bug
is reported?

As a secondary thing (Do you know any possible implementation using squid
and TC-CBQ?)

Cheers

On Tue, Feb 16, 2016 at 8:26 PM, djch [via Squid Web Proxy Cache] <
ml-node+s1019090n4676045...@n4.nabble.com> wrote:

> It's been a while since I've looked at this—because the software we use to
> generate our squid.conf just works around now—but we found that Squid 3
> would only enforce exactly half the configured rate on HTTP requests but
> enforce the full rate on HTTPS requests.
>
> So we now make two delay pools for every "restriction": one for HTTP which
> is x2 the byte rate and one for HTTPS which is normal.
>
> I don't we looked much more into it or filed a bug 'cause none of the
> developers seem very keen on pushing delay_pools forward, due their being
> more robust network-level approaches these days.
>
> On Wed, 17 Feb 2016 at 12:37 Hery Martin <[hidden email]
> > wrote:
>
>> Hello everybody:
>>
>> Since a few months ago I'm using squid to provide a solution as small
>> business proxy in the network of my work place.
>>
>> I'm from Cuba, in our country the Internet is a very limited resource. I
>> have only one link of 2Mbps to share with 20 ~ 25 users (even with my
>> network have more than 60) this is the normal concurrent number.
>>
>> When I start the squid deployment in my network I started using
>> 2.7stable9 version, I made all arrangements to put it work with my AD to
>> match ACLs using AD Groups and everything works perfect.
>>
>> I defined 1 class 2 delay pools to to limits traffic to 12 KBytes/s per
>> user approx.
>>
>> delay_pool 1
>> delay_class 1 2
>> delay_parameters -1/-1 12228/12228
>>
>> The delay pool works perfect, I was checking with real-time tool sqstat
>> and with squidclient mgr:delay
>>
>> NOW.
>>
>> I recently upgrade squid to 3.3.8 and I notice that delay pool started to
>> going wrong when the users surf or download using HTTPS protocol
>>
>> I checked in real-time and when the users browse HTTPS the pool goes in
>> negative numbers and start to grow and grow, its very easy to check, just
>> define a delay pool with 5KB and start a download from an HTTPS source and
>> you can check it with squidclient mgr:delay, the ip takes negative pool
>> value and keep growing until the download finish.
>>
>> Frustrated with this behavior I put different squid versions in a
>> Virtualization Server and definitely I saw that the problem occurs with
>> squid 3.x versions, today I made a final test and I think that the
>> implementation of HTTP v1.1 is maybe related with that problem (I'm not
>> sure but tomorow I will make a few tests with squid 3.1 where HTTP v1.1 was
>> not yet implemented)
>>
>> Please, if you have the opportunity, just test this in a Lab environment,
>> I decided to write to this email list because I asked to many people that
>> already have implemented squid as proxy in their networks and they didn't
>> believed to me until I demostrated the issue.
>>
>> Have anyone information about this bug? There is any hope to fix this
>> problem at code level?
>>
>> Anyway, I'm computer systems engineer, I use to write a lot C++ lines
>> every week... I'm not related with the squid development (never saw the
>> code in my life) but if somebody have any idea how to fix this and wants
>> help just count with me.
>>
>> Greetings from Cuba and sorry about my English :)
>> ___
>> squid-users mailing list
>> [hidden email] 
>> http://lists.squid-cache.org/listinfo/squid-users
>>
>
> ___
> squid-users mailing list
> [hidden email] 
> http://lists.squid-cache.org/listinfo/squid-users
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://squid-web-proxy-cache.1019090.n4.nabble.com/Delay-Pools-and-HTTPS-on-Squid-3-x-tp4676043p4676045.html
> To unsubscribe from Delay Pools and HTTPS on Squid 3.x, click here
> 
> .
> NAML
> 

Re: [squid-users] Delay Pools and HTTPS on Squid 3.x

[Dan Charlesworth]

It's been a while since I've looked at this—because the software we use to
generate our squid.conf just works around now—but we found that Squid 3
would only enforce exactly half the configured rate on HTTP requests but
enforce the full rate on HTTPS requests.

So we now make two delay pools for every "restriction": one for HTTP which
is x2 the byte rate and one for HTTPS which is normal.

I don't we looked much more into it or filed a bug 'cause none of the
developers seem very keen on pushing delay_pools forward, due their being
more robust network-level approaches these days.

On Wed, 17 Feb 2016 at 12:37 Hery Martin  wrote:

> Hello everybody:
>
> Since a few months ago I'm using squid to provide a solution as small
> business proxy in the network of my work place.
>
> I'm from Cuba, in our country the Internet is a very limited resource. I
> have only one link of 2Mbps to share with 20 ~ 25 users (even with my
> network have more than 60) this is the normal concurrent number.
>
> When I start the squid deployment in my network I started using 2.7stable9
> version, I made all arrangements to put it work with my AD to match ACLs
> using AD Groups and everything works perfect.
>
> I defined 1 class 2 delay pools to to limits traffic to 12 KBytes/s per
> user approx.
>
> delay_pool 1
> delay_class 1 2
> delay_parameters -1/-1 12228/12228
>
> The delay pool works perfect, I was checking with real-time tool sqstat
> and with squidclient mgr:delay
>
> NOW.
>
> I recently upgrade squid to 3.3.8 and I notice that delay pool started to
> going wrong when the users surf or download using HTTPS protocol
>
> I checked in real-time and when the users browse HTTPS the pool goes in
> negative numbers and start to grow and grow, its very easy to check, just
> define a delay pool with 5KB and start a download from an HTTPS source and
> you can check it with squidclient mgr:delay, the ip takes negative pool
> value and keep growing until the download finish.
>
> Frustrated with this behavior I put different squid versions in a
> Virtualization Server and definitely I saw that the problem occurs with
> squid 3.x versions, today I made a final test and I think that the
> implementation of HTTP v1.1 is maybe related with that problem (I'm not
> sure but tomorow I will make a few tests with squid 3.1 where HTTP v1.1 was
> not yet implemented)
>
> Please, if you have the opportunity, just test this in a Lab environment,
> I decided to write to this email list because I asked to many people that
> already have implemented squid as proxy in their networks and they didn't
> believed to me until I demostrated the issue.
>
> Have anyone information about this bug? There is any hope to fix this
> problem at code level?
>
> Anyway, I'm computer systems engineer, I use to write a lot C++ lines
> every week... I'm not related with the squid development (never saw the
> code in my life) but if somebody have any idea how to fix this and wants
> help just count with me.
>
> Greetings from Cuba and sorry about my English :)
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Delay Pools and HTTPS on Squid 3.x

[Hery Martin]

Hello everybody:

Since a few months ago I'm using squid to provide a solution as small
business proxy in the network of my work place.

I'm from Cuba, in our country the Internet is a very limited resource. I
have only one link of 2Mbps to share with 20 ~ 25 users (even with my
network have more than 60) this is the normal concurrent number.

When I start the squid deployment in my network I started using 2.7stable9
version, I made all arrangements to put it work with my AD to match ACLs
using AD Groups and everything works perfect.

I defined 1 class 2 delay pools to to limits traffic to 12 KBytes/s per
user approx.

delay_pool 1
delay_class 1 2
delay_parameters -1/-1 12228/12228

The delay pool works perfect, I was checking with real-time tool sqstat and
with squidclient mgr:delay

NOW.

I recently upgrade squid to 3.3.8 and I notice that delay pool started to
going wrong when the users surf or download using HTTPS protocol

I checked in real-time and when the users browse HTTPS the pool goes in
negative numbers and start to grow and grow, its very easy to check, just
define a delay pool with 5KB and start a download from an HTTPS source and
you can check it with squidclient mgr:delay, the ip takes negative pool
value and keep growing until the download finish.

Frustrated with this behavior I put different squid versions in a
Virtualization Server and definitely I saw that the problem occurs with
squid 3.x versions, today I made a final test and I think that the
implementation of HTTP v1.1 is maybe related with that problem (I'm not
sure but tomorow I will make a few tests with squid 3.1 where HTTP v1.1 was
not yet implemented)

Please, if you have the opportunity, just test this in a Lab environment, I
decided to write to this email list because I asked to many people that
already have implemented squid as proxy in their networks and they didn't
believed to me until I demostrated the issue.

Have anyone information about this bug? There is any hope to fix this
problem at code level?

Anyway, I'm computer systems engineer, I use to write a lot C++ lines every
week... I'm not related with the squid development (never saw the code in
my life) but if somebody have any idea how to fix this and wants help just
count with me.

Greetings from Cuba and sorry about my English :)
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] delay pools

[Alex Samad]

Hi

Is it possible to implement delay pools such that

if file is less than 10M
then
  allow 60Mb/s
else
  allow 20Mb/s
fi


is that possible the aim is to allow a higher through put for smaller
files, but to limit bigger / longer connections

Alex
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] delay pools

[Amos Jeffries]

On 19/01/2016 6:52 p.m., Alex Samad wrote:
> Hi
> 
> Is it possible to implement delay pools such that
> 
> if file is less than 10M
> then
>   allow 60Mb/s
> else
>   allow 20Mb/s
> fi
> 

There is no "file" in HTTP. Only messages.

Some messages have payloads. Sometimes those payload sizes are known
before they have finished arriving. Usually they are not. Sometimes
those payloads even match the size of a file being transferred with
HTTP. Usually they do not.

You could write a response header ACL to check how many digits there are
in the Content-Length headers. But that will only work sometimes, and
only get you an order-of-magnitue type of check. Which may be good
enough for what I think you are trying to do.

But you will needs Squid-4.0.2 or later for that, where bug 1139 has
been fixed.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Amos Jeffries]

On 4/01/2016 9:42 a.m., Christian Kunkel wrote:
> 
 How many users do you have?
>>>
>>> i wanted to put about 200-500 users on a server. is that possible?
>>
>> Certainly no problem for Squid, and I guess you could assign that number of 
>> separate listening ports for use one per user, but I'll let someone who 
>> knows 
>> more about Squid's internals for such an unusual setup comment on that if 
>> needed.
> 
> ok.

Squid is limited to 64 listening ports. That can be extended a little in
exchange for reducing Squid operating speed, but 200-500 is going very
far. This will cause problems with your stated goal of handling Gbps,
Squid will need some fine tuning to get near that speed as it is.

>>
 - are you trying to limit the *inbound* bandwidth to Squid per user, or
 the *outbound* bandwidth from Squid to each user?
>>>
>>> i want to limit the bandwidth. lets say user has 50mbit but i want him only
>>> to use 10mbit.
>>
>> So, that's the outbound bandwidth from Squid to the user, then?  You don't 
>> mind if Squid fetches the requested content faster than that if it can, and 
>> then feeds it to the user no faster than 10Mbps?
> 
> yep. that can work this way.
>>
>> Is this limit true for all users - ie: is there a single bandwidth limit you 
>> want to apply to all users, or are you trying to set different limits for 
>> different users?
> 
> only one limit for every user.
>>
 - what's the primary reason for wanting to restrict the bandwidth per
 user?
>>>
>>> server has not unlimited speed. better control of the server bandwidth.
>>
>> What total bandwidth are you dealing with?
> 1gbit/s (but i guess its a bit less than that. maybe it will peak at 500mbit)
>> What's the server load when it runs into problems?
> have not tested it so far with so many users.
>> How many concurrent user sessions do you have when the problems occur?
> no problems right. cause not enough load.
>> What are the effects of the problems you're having?
>>
>> Is there any reason you can't use authentication to identify different users?
> it does not work with nated ips.

Authentication does.

> it autheticates with ip adress anyway.

That is *not* Authentication. That is IP based authorization (access
control).

> so it will limit the ip to 10mbit but behind that ip there are maybe 10 or 
> more ppl.

With authentication each of these "ppl" has different credentials and
messages using those credentials are used to count the bandwidth shaping
towards each user.

If your system defines a "user" as being one IP address. Then the IP
address is what the traffic needs to be accounted against.

>>
>> What stops users "investigating" the system, and finding out they can get 
>> extra 
>> bandwidth by using ports which haven't been assigned to them?
> 
> thats the second problem to deal with. there is some kind of a captive portal 
> with login but it opens the port after user autheticates so actually someone 
> else can use that port. so if you have an idea. i would be really thankful :)
> 

FYI: the only thing that Squid can do that OS level QoS controls cannot
easily do is base its shaping on HTTP message header values (ie the
users proxy-auth credentials).

Since you want to do this shaping "per-user" without authentication
credentials to correctly identify what a single "user" actually is and
are instead basing the definition of "user" as stuff coming from an IP
address (that IP based authorization) - then OS level QoS controls
shaping the traffic based on IP address is the best you are going to
get. Despite the NAT related issues.

The captive portal device is the right place for the bandwidth shaping
to be enacted. It has access to both the original client IP and whatever
authentication details is uses.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Amos Jeffries]

On 5/01/2016 1:41 a.m., Christian Kunkel wrote:
> 
> 
>> Am 04.01.2016 um 12:46 schrieb Amos Jeffries:
>>
>> Squid is limited to 64 listening ports. That can be extended a little in
>> exchange for reducing Squid operating speed, but 200-500 is going very
>> far. This will cause problems with your stated goal of handling Gbps,
>> Squid will need some fine tuning to get near that speed as it is.
> 
> ok. so actually i can run 3 or 4 instances of squid to acomplish my goal of 
> 200 users? lets say the server needs to handle that amount of users and not 
> squid. this would work i guess?
> 

Possibly 2 will be needed to hit 1Gbps. And yes, HTTP is stateless
protocol - as load increases you just add more proxies to handle it. But
that is related to the bps speed, not the number of users. Squid is
limited most by the time it takes to parse and process the HTTP messages.

One Squid can handle many thousands of *users* - when the users are
doing the normal low-ish request rates. Or max out your bandwidth by a
single user doing many thousands of requests.


 Is there any reason you can't use authentication to identify different 
 users?
>>> it does not work with nated ips.
>>
>> Authentication does.
> 
> ok. but i can not use authetication. the main os which will be used to 
> connect to squid can not handle http auth headers. no arp and so on. or lets 
> say it this way: no way to get something unique out of the os to autheticate 
> or authorize on. only thing is used are ports. every user gets a unique port 
> to work with. after login through captive this port is redirected to squid. 
> that ports is actually opened for everyone for 48h. after that time user will 
> see captive to login again. lets say: thats not the best way but the best way 
> i could come up with to do something like authetication. maybe there is a 
> better way but i did not find something to make it better.
>>
>>> it autheticates with ip adress anyway.
>>
>> That is *not* Authentication. That is IP based authorization (access
>> control).
> 
> explained above.

What you explained was *not* authentication, and the reasons given are
not relevant to authentication. Which is a good sign that you do not
understand authentication in HTTP.

That is a clear that you are not going to be able to have it any time
soon with your current level of understanding, whether its possible or
not. So I will continue

>>
>>> so it will limit the ip to 10mbit but behind that ip there are maybe 10 or 
>>> more ppl.
>>
>> With authentication each of these "ppl" has different credentials and
>> messages using those credentials are used to count the bandwidth shaping
>> towards each user.
>>
>> If your system defines a "user" as being one IP address. Then the IP
>> address is what the traffic needs to be accounted against.
> 
> main problem of NATed users (in my case): their ip adresses changes from time 
> to time or based on their location. so if ip is used for authorization then a 
> big amount gets ahthorized or they need to relogin constantly. not that nice.

With your current setup you are not going to be able to resolve that
problem, or the one about multiple users behind each IP. It is simply
not possible so long as your tie the IP:port details into the definition
of "user" at the captive portal.

Which is why Anthony and I are making such a fuss about checking whether
you can do proper HTTP authentication. Since that has nothing to do with
NAT, IP, port or any of the problematic TCP layer juggling you are doing
in the portal.

>>

 What stops users "investigating" the system, and finding out they can get 
 extra 
 bandwidth by using ports which haven't been assigned to them?
>>>
>>> thats the second problem to deal with. there is some kind of a captive 
>>> portal with login but it opens the port after user autheticates so actually 
>>> someone else can use that port. so if you have an idea. i would be really 
>>> thankful :)
>>
>> FYI: the only thing that Squid can do that OS level QoS controls cannot
>> easily do is base its shaping on HTTP message header values (ie the
>> users proxy-auth credentials).
> 
> the os does not really save those credentials. every http request then asks 
> for the credentials. thats to messed up this way.

HTTP is a stateless and multiplexed protocol. Each request is designed
to be a standalone description of how to fetch its reply.

>>
>> Since you want to do this shaping "per-user" without authentication
>> credentials to correctly identify what a single "user" actually is and
>> are instead basing the definition of "user" as stuff coming from an IP
>> address (that IP based authorization) - then OS level QoS controls
>> shaping the traffic based on IP address is the best you are going to
>> get. Despite the NAT related issues.
> 
> see above for ports as unique definition of a user.

Which as you repeatedly have said is not providing you with the unique
portion of the requirement - leaving 

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Christian Kunkel]

> Am 04.01.2016 um 12:46 schrieb Amos Jeffries :
> 
> Squid is limited to 64 listening ports. That can be extended a little in
> exchange for reducing Squid operating speed, but 200-500 is going very
> far. This will cause problems with your stated goal of handling Gbps,
> Squid will need some fine tuning to get near that speed as it is.

ok. so actually i can run 3 or 4 instances of squid to acomplish my goal of 200 
users? lets say the server needs to handle that amount of users and not squid. 
this would work i guess?

>>> Is there any reason you can't use authentication to identify different 
>>> users?
>> it does not work with nated ips.
> 
> Authentication does.

ok. but i can not use authetication. the main os which will be used to connect 
to squid can not handle http auth headers. no arp and so on. or lets say it 
this way: no way to get something unique out of the os to autheticate or 
authorize on. only thing is used are ports. every user gets a unique port to 
work with. after login through captive this port is redirected to squid. that 
ports is actually opened for everyone for 48h. after that time user will see 
captive to login again. lets say: thats not the best way but the best way i 
could come up with to do something like authetication. maybe there is a better 
way but i did not find something to make it better.
> 
>> it autheticates with ip adress anyway.
> 
> That is *not* Authentication. That is IP based authorization (access
> control).

explained above.
> 
>> so it will limit the ip to 10mbit but behind that ip there are maybe 10 or 
>> more ppl.
> 
> With authentication each of these "ppl" has different credentials and
> messages using those credentials are used to count the bandwidth shaping
> towards each user.
> 
> If your system defines a "user" as being one IP address. Then the IP
> address is what the traffic needs to be accounted against.

main problem of NATed users (in my case): their ip adresses changes from time 
to time or based on their location. so if ip is used for authorization then a 
big amount gets ahthorized or they need to relogin constantly. not that nice.
> 
>>> 
>>> What stops users "investigating" the system, and finding out they can get 
>>> extra 
>>> bandwidth by using ports which haven't been assigned to them?
>> 
>> thats the second problem to deal with. there is some kind of a captive 
>> portal with login but it opens the port after user autheticates so actually 
>> someone else can use that port. so if you have an idea. i would be really 
>> thankful :)
> 
> FYI: the only thing that Squid can do that OS level QoS controls cannot
> easily do is base its shaping on HTTP message header values (ie the
> users proxy-auth credentials).

the os does not really save those credentials. every http request then asks for 
the credentials. thats to messed up this way.
> 
> Since you want to do this shaping "per-user" without authentication
> credentials to correctly identify what a single "user" actually is and
> are instead basing the definition of "user" as stuff coming from an IP
> address (that IP based authorization) - then OS level QoS controls
> shaping the traffic based on IP address is the best you are going to
> get. Despite the NAT related issues.

see above for ports as unique definition of a user.
> 
> The captive portal device is the right place for the bandwidth shaping
> to be enacted. It has access to both the original client IP and whatever
> authentication details is uses.
> 
> Amos
> 
> __

Kind regards,

Chris
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Antony Stone]

On Sunday 03 January 2016 at 09:42:21, Christian Kunkel wrote:

> Am 03.01.2016 um 01:14 schrieb Antony Stone;
> > >> On Sunday 03 January 2016 at 00:46:39, Christian Kunkel wrote:
> >> 
> >> Hey guys,
> >> 
> >> is there any way i can do some traffic shaping with squid?
> > 
> > Yes, but it's nowhere near as good as doing it with IP tools on the
> > underlying O/S.
> 
> ok. thats what i thought too. any hint there?

http://lartc.org/howto/lartc.qdisc.html
https://wiki.archlinux.org/index.php/Advanced_traffic_control
http://www.squid-cache.org/Doc/config/tcp_outgoing_mark/

> >> I've been thinking to create a multiple ports with squid and limit the
> >> ports. How can i do that?
> > 
> > No idea, without knowing where you're starting from.
> 
> http_port 1337
> http_port 1338
> and so on. every user gets his oen port. by using delay pools or something
> i can limit their speed then?!

How many users do you have?

> >> Or is there a better way?
> > 
> > Almost certainly.
> > 
> > Explain, in as much detail as you can:
> > 
> > - what your networking setup is
> 
> what do you need to know here?

Well, for example:

 - are your clients all in a consistent network range, or are they spread 
across the Internet?

 - are multiple clients NATted behind a single router, or do they have unique 
IP addresses as far as Squid is concerned?

 - if clients are NATted, can you use a VPN so that Squid can see the real IP 
address of each user?

 - are you trying to limit the *inbound* bandwidth to Squid per user, or the 
*outbound* bandwidth from Squid to each user?

 - how many IP addresses and network interfaces does the Squid server have?

 - what's the primary reason for wanting to restrict the bandwidth per user?


Regards,


Antony.

-- 
"In fact I wanted to be John Cleese and it took me some time to realise that 
the job was already taken."

 - Douglas Adams

   Please reply to the list;
 please *don't* CC me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Christian Kunkel]

Am 03.01.2016 um 01:14 schrieb Antony Stone :
> 
>> On Sunday 03 January 2016 at 00:46:39, Christian Kunkel wrote:
>> 
>> Hey guys,
>> 
>> is there any way i can do some traffic shaping with squid?
> 
> Yes, but it's nowhere near as good as doing it with IP tools on the 
> underlying 
> O/S.

ok. thats what i thought too. any hint there?
> 
>> Its a bit complicated in my case. I can not shape through user ip because
>> squid is not running on a local network.
> 
> So, tell us where it is running, then...

its running on a server and can be accessed from the internet.
> 
>> I've been thinking to create a multiple ports with squid and limit the
>> ports. How can i do that?
> 
> No idea, without knowing where you're starting from.

http_port 1337
http_port 1338
and so on. every user gets his oen port. by using delay pools or something i 
can limit their speed then?!
> 
>> Or is there a better way?
> 
> Almost certainly.
> 
> Explain, in as much detail as you can:
> 
> - what your networking setup is
what do you need to know here?
> - which version of Squid you are using
3.5.11
> - which Operating System (and version) you are running it under
debian jessie
> - what you want to achieve by "shaping"
i want to limit the download speed per user.
> 
> The more information you give us, the more we might be able to help you.
> 
> 
> Regards,
> 
> 
> Antony.
> 
> -- 
> There are only 10 types of people in the world:
> those who understand binary notation,
> and those who don't.
> 
>   Please reply to the list;
> please *don't* CC me.
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Christian Kunkel]

> Am 03.01.2016 um 10:13 schrieb Antony Stone 
> :
> 
>> On Sunday 03 January 2016 at 09:42:21, Christian Kunkel wrote:
>> 
>> Am 03.01.2016 um 01:14 schrieb Antony Stone;
> On Sunday 03 January 2016 at 00:46:39, Christian Kunkel wrote:
 
 Hey guys,
 
 is there any way i can do some traffic shaping with squid?
>>> 
>>> Yes, but it's nowhere near as good as doing it with IP tools on the
>>> underlying O/S.
>> 
>> ok. thats what i thought too. any hint there?
> 
> http://lartc.org/howto/lartc.qdisc.html
> https://wiki.archlinux.org/index.php/Advanced_traffic_control
> http://www.squid-cache.org/Doc/config/tcp_outgoing_mark/
> 
ty. i will check that!

 I've been thinking to create a multiple ports with squid and limit the
 ports. How can i do that?
>>> 
>>> No idea, without knowing where you're starting from.
>> 
>> http_port 1337
>> http_port 1338
>> and so on. every user gets his oen port. by using delay pools or something
>> i can limit their speed then?!
> 
> How many users do you have?

i wanted to put about 200-500 users on a server. is that possible?
> 
 Or is there a better way?
>>> 
>>> Almost certainly.
>>> 
>>> Explain, in as much detail as you can:
>>> 
>>> - what your networking setup is
>> 
>> what do you need to know here?
> 
> Well, for example:
> 
> - are your clients all in a consistent network range, or are they spread 
> across the Internet?

across the internet.
> 
> - are multiple clients NATted behind a single router, or do they have unique 
> IP addresses as far as Squid is concerned?

NAT is in place. so no way to use ip as a unique identifier.
> 
> - if clients are NATted, can you use a VPN so that Squid can see the real IP 
> address of each user?

vpn was discussed but we can not use it.
> 
> - are you trying to limit the *inbound* bandwidth to Squid per user, or the 
> *outbound* bandwidth from Squid to each user?
i want to limit the bandwidth. lets say user has 50mbit but i want him only to 
use 10mbit.
> 
> - how many IP addresses and network interfaces does the Squid server have?
> 
right now only one. but it can grow if needed.
> - what's the primary reason for wanting to restrict the bandwidth per user?
server has not unlimited speed. better control of the server bandwidth.
> 
> 
> Regards,
> 
> 
> Antony.
> 
> -- 
> "In fact I wanted to be John Cleese and it took me some time to realise that 
> the job was already taken."
> 
> - Douglas Adams
> 
>   Please reply to the list;
> please *don't* CC me.
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

Kind regards,

Chris
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Antony Stone]

On Sunday 03 January 2016 at 12:35:10, Christian Kunkel wrote:

> > Am 03.01.2016 um 10:13 schrieb Antony Stone:
> > 
> > How many users do you have?
> 
> i wanted to put about 200-500 users on a server. is that possible?

Certainly no problem for Squid, and I guess you could assign that number of 
separate listening ports for use one per user, but I'll let someone who knows 
more about Squid's internals for such an unusual setup comment on that if 
needed.

> > - are you trying to limit the *inbound* bandwidth to Squid per user, or
> > the *outbound* bandwidth from Squid to each user?
> 
> i want to limit the bandwidth. lets say user has 50mbit but i want him only
> to use 10mbit.

So, that's the outbound bandwidth from Squid to the user, then?  You don't 
mind if Squid fetches the requested content faster than that if it can, and 
then feeds it to the user no faster than 10Mbps?

Is this limit true for all users - ie: is there a single bandwidth limit you 
want to apply to all users, or are you trying to set different limits for 
different users?

> > - what's the primary reason for wanting to restrict the bandwidth per
> > user?
> 
> server has not unlimited speed. better control of the server bandwidth.

What total bandwidth are you dealing with?
What's the server load when it runs into problems?
How many concurrent user sessions do you have when the problems occur?
What are the effects of the problems you're having?

Is there any reason you can't use authentication to identify different users?

What stops users "investigating" the system, and finding out they can get extra 
bandwidth by using ports which haven't been assigned to them?



Regards,


Antony.

-- 
If you were ploughing a field, which would you rather use - two strong oxen or 
1024 chickens?

 - Seymour Cray, pioneer of supercomputing

   Please reply to the list;
 please *don't* CC me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Delay Pools or Traffic Shaping per port?!

[Christian Kunkel]

Hey guys,

is there any way i can do some traffic shaping with squid? Its a bit 
complicated in my case. I can not shape through user ip because squid is not 
running on a local network. I've been thinking to create a multiple ports with 
squid and limit the ports. How can i do that? Or is there a better way?

Kind regards,

Chris
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools or Traffic Shaping per port?!

[Antony Stone]

On Sunday 03 January 2016 at 00:46:39, Christian Kunkel wrote:

> Hey guys,
> 
> is there any way i can do some traffic shaping with squid?

Yes, but it's nowhere near as good as doing it with IP tools on the underlying 
O/S.

> Its a bit complicated in my case. I can not shape through user ip because
> squid is not running on a local network.

So, tell us where it is running, then...

> I've been thinking to create a multiple ports with squid and limit the
> ports. How can i do that?

No idea, without knowing where you're starting from.

> Or is there a better way?

Almost certainly.

Explain, in as much detail as you can:

 - what your networking setup is
 - which version of Squid you are using
 - which Operating System (and version) you are running it under
 - what you want to achieve by "shaping"

The more information you give us, the more we might be able to help you.


Regards,


Antony.

-- 
There are only 10 types of people in the world:
those who understand binary notation,
and those who don't.

   Please reply to the list;
 please *don't* CC me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay pools: HTTPS and rep_mime_type support

[Amos Jeffries]

On 18/11/2015 8:53 p.m., Filippo Martinelli wrote:
> I'm struggling with squid delay pools. The documentation I found is very
> poor and on internet there is contradictory and possibly surpassed
> information. I would appreciate if you can point me to exhaustive
> information on delay pools.
> 
> First question:
> 
> acl aclname url_regex -i \.exe
> 
> will it match an HTTPS request ? According to my experience and from some
> posts it will not, so cannot find any easy way to limit the bandwidth used
> to download .exe files from an HTTPS connection. Am I missing something ?
> The only suggestion google gave is to use something like "acl  aclname
> methoid CONNECT" but it is too generic and will not discriminate between
> long .exe download or single page access on HTTPS connections.

"HTTPS" is not a single thing, or message type. It is a term to describe
an entire stack of multiple-layered protocols.

To do anything at all with URL or any other HTTP message details in what
could be termed an "HTTPS request" requires decrypting the TLS layer to
find the HTTP message secured inside it.


ItFrom that description it sounds to me like you are dealing with a
plain-text HTTP message of method CONNECT. There is almost zero
information in those. Apart from the domain name of the server the
client wants to talk to and maybe the client UA device, you are out of
luck using any of the more normal request/reply message details to
decide on the pool.

Squid should still be able to delay pool those CONNECT tunnels though.
But only as a whole thing, and there are still open bugs with unknown
causes. You need a fairly recent version of Squid for it to work even
halfway close to "properly".

For bandwidth control it is often better to use the QoS / TOS
functionality provided by your OS. Squid can output per-request values
for those systems to work with using qos_flows, tcp_outgoing_tos or
tcp_outgoing_mark.


> 
> Second question:
> 
> acl streaming_exe rep_mime_type application/octet-stream
> 
> Can rep_mime_type be used with delay_access poolNumber allow ? Again,
> according to my experience and to some very old posts in internet it will
> not work, but the documentation lacks this important limitation.

Of the current Squid only 4.0.2 or later can do that. (re-)assiging
pools based on HTTP response details was only very recently ported from
Squid-2.6.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools Parameters

[Tecnologia Charne.Net]

El 16/11/15 a las 18:41, Amos Jeffries escribió:
>  
> 8 x 32000 bytes per second = 256000 bits per second = 256kilobits per second
> 8 x 8000 bytes per second = 64000 bits per second = 64 kilobits per second
> 8 x 600 bytes per second = 4800 bits per second
>
> Note the omission of "kilobytes per second" from the first column of numbers.
>
>
> Regards,
>>
>> I arrive to the same conclussion and agree that documentation in
>> http://www.squid-cache.org/Doc/config/delay_parameters/ has some mistakes.
>>
>> Thanks for your time, Antony!
>>
> Mea culpa. Fixing that now.
>
> Amos
>
>


How fast!
Thanks Amos.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Delay pools: HTTPS and rep_mime_type support

[Filippo Martinelli]

I'm struggling with squid delay pools. The documentation I found is very
poor and on internet there is contradictory and possibly surpassed
information. I would appreciate if you can point me to exhaustive
information on delay pools.

First question:

acl aclname url_regex -i \.exe

will it match an HTTPS request ? According to my experience and from some
posts it will not, so cannot find any easy way to limit the bandwidth used
to download .exe files from an HTTPS connection. Am I missing something ?
The only suggestion google gave is to use something like "acl  aclname
methoid CONNECT" but it is too generic and will not discriminate between
long .exe download or single page access on HTTPS connections.

Second question:

acl streaming_exe rep_mime_type application/octet-stream

Can rep_mime_type be used with delay_access poolNumber allow ? Again,
according to my experience and to some very old posts in internet it will
not work, but the documentation lacks this important limitation.

Thanks
Filippo

-- 

For everything there is a season
and a time for every matter under Heaven
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools Parameters

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 


16.11.15 20:49, Tecnología CHARNE.NET пишет:
> Hello!
>
> I'm configuring delay pools on squid 3.5
>
> I don't understand online doc
>
[http://www.squid-cache.org/Versions/v3/3.5/cfgman/delay_parameters.html] about
> delay_parameters
>
> 
> "Note that 8 x 32000 KByte/sec -> 256Kbit/sec.
>   8 x  8000 KByte/sec ->  64Kbit/sec.
>   8 x   600 Byte/sec  -> 4800bit/sec.
> "
> 
>
> It should be
>
> 8 x 32000 KByte/sec -> 256000Kbits/sec
>
> or
>
> 8 x 32KByte/sec -> 256 Kbit/sec
>
>
> What I am missing??
You have forgotten to read fine manuals first.

>
>
> Thanks in advance.
>
>
> Javier.-
>
>
>
>
>
>
>
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJWSe1iAAoJENNXIZxhPexGnt0H/0qUVQKkoQGg/JDdP6JG8548
Y1f2S//dB+19PzKm7RZ2wJMndx5PyMI8Mz/tzzeDjinkaU5lhfDq/pj0COOOGygD
NUpcdrP89le28MVlOnDP1/QudfmBDikWX+L0xZkK5OHZFVj0Kd01jsc151RTm9Vj
gHP2NWKSqW6ApbjDKMLXN6sTinfVYyq35CKG+oN4SjkGSF16eXR/jRAo/02cFQkC
ehXWCyo+7MNFORmadjT8WhSUJdIkgbdq3cks5N6jl6VVUNodqH48fagYy1LBbZjw
ev0CZ4nvGHuSnbdaLpihiDpqTkosjwcunc8hIN/0GjXUvl7PXjf5kPIiv4/9QCY=
=8GJN
-END PGP SIGNATURE-

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools Parameters

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 
Feel free to read Squid Wiki:

http://wiki.squid-cache.org/Features/DelayPools

16.11.15 20:49, Tecnología CHARNE.NET пишет:
> Hello!
>
> I'm configuring delay pools on squid 3.5
>
> I don't understand online doc
>
[http://www.squid-cache.org/Versions/v3/3.5/cfgman/delay_parameters.html] about
> delay_parameters
>
> 
> "Note that 8 x 32000 KByte/sec -> 256Kbit/sec.
>   8 x  8000 KByte/sec ->  64Kbit/sec.
>   8 x   600 Byte/sec  -> 4800bit/sec.
> "
> 
>
> It should be
>
> 8 x 32000 KByte/sec -> 256000Kbits/sec
>
> or
>
> 8 x 32KByte/sec -> 256 Kbit/sec
>
>
> What I am missing??
>
> Thanks in advance.
>
>
> Javier.-
>
>
>
>
>
>
>
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJWSe3MAAoJENNXIZxhPexGqnUH/1DxMlvsy2P5iTCkbKmOcENP
7U3fvchDK6WZ+9vWpQa/YfolCDZhPo3QYaAY+gQTIe6ec6tauVwIzqiyDfP+YiIr
5Wi35rvcSD7/NbzlkKEacCg6TwObyX3aFPkjkq8uOdBal2J1gE3DXU66tSJAmUWl
NZI6FplCLnk5qRgQU9lNF6HDjnTV4tp1E1YokKBGWRkZ51ToX3k5SNQGUNTyx+u4
Rz7W2q9A0e2TqHqExh/HPnWqhIQbO0nagv72MFaSLeHJQ8ZcK25GNxfohJlFMP2d
He9NwKWjd08/rUhDrOd0F/bw4IloI/l2IsUQ6DTuFC6O2z8I/1566OSvKGh23l8=
=6AGf
-END PGP SIGNATURE-

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools Parameters

[Antony Stone]

> 16.11.15 20:49, Tecnología CHARNE.NET пишет:
> > Hello!
> > 
> > I'm configuring delay pools on squid 3.5
> > 
> > I don't understand online doc
> > [http://www.squid-cache.org/Versions/v3/3.5/cfgman/delay_parameters.html]
> > about delay_parameters
> > 
> > 
> > "Note that 8 x 32000 KByte/sec -> 256Kbit/sec.
> >   8 x  8000 KByte/sec ->  64Kbit/sec.
> >   8 x   600 Byte/sec  -> 4800bit/sec.
> > "
> > 
> > 
> > It should be
> > 
> > 8 x 32000 KByte/sec -> 256000Kbits/sec
> > or
> > 8 x 32KByte/sec -> 256 Kbit/sec
> > 
> > What I am missing??

On Monday 16 November 2015 at 15:51:14, Yuri Voinov wrote:

> You have forgotten to read fine manuals first.

On Monday 16 November 2015 at 15:53:00, Yuri Voinov wrote:

> Feel free to read Squid Wiki:
> 
> http://wiki.squid-cache.org/Features/DelayPools



I think this is a little unfair on the original poster.

The arithmetic in the documentation does appear to be incorrect - look at the 
units:

If 8 x 600 bytes per second = 4800 bits per second (which seems reasonable to 
me)

then how can

8 x 8000 kilobytes per second = 64 kilobits per second

and 8 x 32000 kilobytes per second = 256 kilobits per second?

The multiplication by 8 is to convert from bytes to bits.

The units (X per second, or kilo-X per second) should not change.

Therefore I believe the correct calculations should be:

8 x 32000 bytes per second = 256000 bits per second = 256kilobits per second
8 x 8000 bytes per second = 64000 bits per second = 64 kilobits per second
8 x 600 bytes per second = 4800 bits per second

Note the omission of "kilobytes per second" from the first column of numbers.


Regards,


Antony.

-- 
I don't know, maybe if we all waited then cosmic rays would write all our 
software for us. Of course it might take a while.

 - Ron Minnich, Los Alamos National Laboratory

   Please reply to the list;
 please *don't* CC me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools Parameters

[Tecnologia Charne.Net]

>> Feel free to read Squid Wiki:
>> http://wiki.squid-cache.org/Features/DelayPools
> I think this is a little unfair on the original poster.
>
> The arithmetic in the documentation does appear to be incorrect - look at the 
> units:
>
> [...]

> Therefore I believe the correct calculations should be:
>
> 8 x 32000 bytes per second = 256000 bits per second = 256kilobits per second
> 8 x 8000 bytes per second = 64000 bits per second = 64 kilobits per second
> 8 x 600 bytes per second = 4800 bits per second
>
> Note the omission of "kilobytes per second" from the first column of numbers.
>
>
> Regards,


I arrive to the same conclussion and agree that documentation in
http://www.squid-cache.org/Doc/config/delay_parameters/ has some mistakes.

Thanks for your time, Antony!
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools Parameters

[Amos Jeffries]

On 17/11/2015 7:51 a.m., Tecnologia Charne.Net wrote:
>>> Feel free to read Squid Wiki:
>>> http://wiki.squid-cache.org/Features/DelayPools
>> I think this is a little unfair on the original poster.
>>


Yes, the document the original poster was reading *was* the fine manual :-P


>> The arithmetic in the documentation does appear to be incorrect - look at 
>> the 
>> units:
>>
>> [...]
> 
>> Therefore I believe the correct calculations should be:
>>
>> 8 x 32000 bytes per second = 256000 bits per second = 256kilobits per second
>> 8 x 8000 bytes per second = 64000 bits per second = 64 kilobits per second
>> 8 x 600 bytes per second = 4800 bits per second
>>
>> Note the omission of "kilobytes per second" from the first column of numbers.
>>
>>
>> Regards,
> 
> 
> I arrive to the same conclussion and agree that documentation in
> http://www.squid-cache.org/Doc/config/delay_parameters/ has some mistakes.
> 
> Thanks for your time, Antony!
> 

Mea culpa. Fixing that now.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] delay pools

[Amos Jeffries]

On 27/10/2015 7:42 a.m., De Lazzari Matteo wrote:
> 
> Hi, is it possible to use Active directory groups in delay pools
> configuration?

Yes. Although to do it easily will require a Squid-3.4 or later where
transaction annotations are available. Also a helper that sends back the
group=X to Squid about what group(s) the user is in (could be auth
helper or external ACL helper).
 So far only the kerberos auth helper does that and it sends the SSID
value as the group=X value for all the groups listed in the Kerberos token.

With a helper returning the group names to Squid, a "note" type ACL can
be used to check the group=X annotation values in any access control
rules. Including delay_access.


> And someone can tell me an example about how to use
> class 5 delay pool?
> 

That delay pool requires that an external_acl_type helper is being used
and sending some tag=X back to Squid to attach 'tag' each request /
transaction.

That helper has to be tested on one of the *_access rules where async /
slow group lookups will work. The delay_access rules will *not* work
since they are a fast-group check. http_access is the usual place and
the heper decides both whether to allow use of Squid and what to tag the
request with.


You define the pool to be of class 5 with a Bytes/sec rate:
  delay_pools 1 1
  delay_parameters 1 5 20480

You define delay_access to match for the requests that are to have that
pools traffic rate limit applied:
  delay_access 1 allow localnet

Squid will automatically arrange so each unique tag=X value the helper
assigns to those pooled requests will have a pool. All requests to which
the helper replies 'tag=ZZ' will share a one pool, but requests the
helper replies with 'tag=YY' will have a different pool. etc.
 Requests not having a tag at all share one pool (I think, havent
checked that).

That is it.

The difficult bits are that only one tag= value can be assigned to a
transaction, attempts to repeat or alter one assigned wont work, and
that detail about the async/slow access lists being the only ones where
the helper can be checked.


HTH
Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] delay pools

[De Lazzari Matteo]

Hi, is it possible to use Active directory groups in delay pools configuration? 
And someone can tell me an example about how to use class 5 delay pool?

Thanks to all

Classification: Public [ ]  Confidential [X]  Restrict [ ]

Matteo De Lazzari
Information Technology

PREVINET S.p.A.
Via E. Forlanini, 24 - 31022 Preganziol (TV) - ITALY
tel +39 - 0422 1745279
matteo.delazz...@previnet.it
[http://www.previnet.it/images/PrevinetOutlook.jpg]
Ai sensi del D.Lgs. 196/2003 sulla tutela dei dati personali, la presente 
comunicazione e ogni suo allegato e' destinata esclusivamente al soggetto 
indicato quale destinatario o ad eventuali altri soggetti autorizzati a 
riceverla. L'utilizzo non autorizzato e' vietato e potrebbe costituire reato. 
Essa contiene informazioni strettamente confidenziali e riservate, la cui 
comunicazione o diffusione a terzi e' proibita, salvo che non sia stata 
espressamente autorizzata. Se avete ricevuto questa e-mail per errore, Vi 
preghiamo di comunicarlo senza indugio al mittente e di cancellarne ogni 
evidenza dai Vostri supporti.
This message is intended only for the named recipient and may contain 
confidential, proprietary or legally privileged information. Unauthorized 
persons are not permitted access to this information. Any dissemination, 
distribution or copying of this information is strictly prohibited. If you have 
received this message in error, please advise the sender by reply e-mail and 
delete this message and any attachments.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] delay pools question

[Alex Samad]

HI

I have had a look at http://wiki.squid-cache.org/Features/DelayPools

Wondering if somebody can maybe explain how it rate limits downloads.

So I can understand it would be able to limit proxy to client traffic
as squid is the sender and can limit how it sends.

But if I want to limit speed from say microsoft.com to the
organisation how does it organise that.

My limited understanding is you make a request of the ms web servers
and then they send it as fast as they can.

The only way I can think of it happening is slowing the TCP ACK's.  Or
does squid make request for partial ranges of files such as to fit in
the speed requirements.

A
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Delay Pools

[csn233]

On Tue, May 6, 2014 at 3:38 AM, tomaswaldow to...@waldow.ws wrote:
 Hi I have a problem in Squid 3.1.20 with Debian 7.
 The settings of the delay pools are as follows:

 delay_pools 1
 1 2 delay_class
 delay_parameters 1 -1/-1 10/10
 1 delay_access allow localnet! CONNECT

 Should be limited to 100KB but does not work.
 Is ranging between 50 to 55.

Try this:

delay_pools 1
delay_class 1 3
delay_parameters 1 -1/-1 -1/-1 10/10
delay_access 1 allow...

[squid-users] Delay Pools

[tomaswaldow]

Hi I have a problem in Squid 3.1.20 with Debian 7. 
The settings of the delay pools are as follows: 

delay_pools 1 
1 2 delay_class 
delay_parameters 1 -1/-1 10/10 
1 delay_access allow localnet! CONNECT 

Should be limited to 100KB but does not work. 
Is ranging between 50 to 55. 

Can anyone help me? 

PS: sorry for my bad English.



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Delay-Pools-tp4665836.html
Sent from the Squid - Users mailing list archive at Nabble.com.

Re: [squid-users] delay pools status

[Beto Moreno]

  Now that we are here learning, what is the latest bandwidth
management tool squid have for the sysadmins?
  Delays pools like u mention are the old stuff.

  I want to get most from my squid.

Thanks again Amos.

On Fri, Mar 21, 2014 at 7:38 PM, Amos Jeffries squ...@treenet.co.nz wrote:
 On 22/03/2014 4:31 a.m., Beto Moreno wrote:
 Hi, thanks Amos for always share your knowledge.

 No there is 500KB/sec allocated to client response (download) traffic on
 first-come basis.

 This is effectively the same configuration as kernel routing QoS
 controls allowing the Squid process to access 500KB/s of traffic inbound
 for servers and unlimited upload traffic to servers.

 Here squid decide what we can share to each connection, the first
 machine get full bucket, if other machine arrive squid decide what he
 can receive from the bucket and continue the same to others users?
 Until they eat all the bucket and fill again the bucket to share?

 There is two things here:
  1) the delay pool bucket size
  2) the I/O buffer free space

 500KB is larger than the maximum buffer size. The delay pool bucket gets
 more data only once every second.

 So what happens is with only one machine, the client buffer gets to fill
 its buffer several times over several reads until 500KB pool bucket is
 empty.
 The next second there are two machines, and each gets to fill its buffer
 in turn until the delay pool bucket is empty.

 Problem: So long as the delay pool is larger than the client buffer size
 there will be some (unbalanced) sharing of the buffer. If the client
 buffer is smaller than the delay pool you may see unpredictable
 situations where one client gets all the traffic for a few seconds then
 switches to another. With many clients some may get all the bandwidth
 and others none at all.
  This pool type relies heavily on clients having traditional browser
 behaviour. Whee a page was donloaded once then some time later another
 page, etc. With big gaps of no traffic by each client.


 example 2:

  delay_pool_count 1
  delay_class 1 3
  delay_parameters 1 50/562500 -1/-1 4000/4000
  delay_initial_bucket_level 100

 Start will full bucket, here we have one bucket of 500kb/s burst
 562kb/s for all my clients but each on my clients will get =
 32kb/s(max 32kb/s) from their single bucket?

 Remembering the sequence of sharing above. With this class-3 pool type
 each client is only allowed to read up to 4000 bytes from the shared
 pool when it fills its buffer. With a maximum of 4000 bytes total each
 second if it tries to read a few bytes several times.
  The total traffic by all clients gets to 50 Bytes in one second
 then reads are stopped until the global pool bucket refills. ie if 125
 clients all read their 4000 bytes each in one second the 126th client is
 not permitted to read.

 This mostly resolves the balancing problem described above. Relating to
 QoS this pool is effectively the same as QoS controls on the
 client-Squid connections controlling how much traffic got delivered to
 each client by Squid (downloads by client).
  But still pools is slightly worse than proper QoS as it does not cover
 requests/uploads, TCP packet overheads, and large response headers can
 cause buckets to go negative.


 About the QoS, is what I'm trying to manage, but I had been testing
 squid delay-pools and they help in some manner controlling users
 appetite.

 Yes. Any type of control helps in some manner. Squid delay pools are
 just old technology with some strange limits (like not covering uploads
 or TCP overheads). They do not work as well as newer QoS technology
 which had more knowledge and experience behind the design.


 QoS trying to see how to integrate with squid, because once squid
 start controlling inbound is came of difficult to me because 80/443
 are only seen by single client(squid), QoS won't see my lan
 connections to those ports which are who eat my bandwidth, but working
 on.

 qos_flows directive in squid.conf can tag traffic by type so you can
 control HIT/MISS with different rules if you want on the client-squid
 connections.

 tcp_outgoing_tos / tcp_outgoing_mark directives tag traffic on the
 squid-server connections according to ACL matches.

 As I understand it you setup QoS rules based on MARK or TOS to do the
 bandwidth allocation you would be doing with delay pools class and
 parameters directives. Then you setup squid.conf to tag the traffic into
 the QoS pool types the same as you would have done with delay_pool_access.

 Amos

Re: [squid-users] delay pools status

[Beto Moreno]

Hi, thanks Amos for always share your knowledge.

No there is 500KB/sec allocated to client response (download) traffic on
first-come basis.

This is effectively the same configuration as kernel routing QoS
controls allowing the Squid process to access 500KB/s of traffic inbound
for servers and unlimited upload traffic to servers.

Here squid decide what we can share to each connection, the first
machine get full bucket, if other machine arrive squid decide what he
can receive from the bucket and continue the same to others users?
Until they eat all the bucket and fill again the bucket to share?

example 2:

 delay_pool_count 1
 delay_class 1 3
 delay_parameters 1 50/562500 -1/-1 4000/4000
 delay_initial_bucket_level 100

Start will full bucket, here we have one bucket of 500kb/s burst
562kb/s for all my clients but each on my clients will get =
32kb/s(max 32kb/s) from their single bucket?

About the QoS, is what I'm trying to manage, but I had been testing
squid delay-pools and they help in some manner controlling users
appetite.

QoS trying to see how to integrate with squid, because once squid
start controlling inbound is came of difficult to me because 80/443
are only seen by single client(squid), QoS won't see my lan
connections to those ports which are who eat my bandwidth, but working
on.

On Thu, Mar 20, 2014 at 9:48 PM, Amos Jeffries squ...@treenet.co.nz wrote:
 On 21/03/2014 6:04 a.m., Beto Moreno wrote:
 Hi.

 Squid 3.1.22.

 Learning this feature from squid, delay-pools.

 1; aggregate the most simple.
 -If I have a dsl 6Mb/s but I want to share to my users 4MB/s and the
 rest for my other servers.

 delay_pool_count 1
 delay_class 1 1
 delay_parameters 1 50/562500   == 4.5Mb/s Max and hold this to 4Mb/s.
 delay_initial_bucket_level 100  === I want to start with full 4.5MB right?

 Once my clients start surfing, example 10 users, squid will share
 50/10 they will have
 +/- 50Kb/s each one?

 No there is 500KB/sec allocated to client response (download) traffic on
 first-come basis.

 This is effectively the same configuration as kernel routing QoS
 controls allowing the Squid process to access 500KB/s of traffic inbound
 for servers and unlimited upload traffic to servers.

 PS. the delay pools feature in Squid predates QoS and most traffic
 management is now far better done by QoS tools instead of Squid delay pools.



 My network is class C.

 If more clients arrive they will again do some calculations
 rate/numbers of users?

 Exist a way to see the status of the pools connections example with cachemgr?

 Delay pools should be listed in the delay report.

 Amos

Re: [squid-users] delay pools status

[Amos Jeffries]

On 22/03/2014 4:31 a.m., Beto Moreno wrote:
 Hi, thanks Amos for always share your knowledge.
 
 No there is 500KB/sec allocated to client response (download) traffic on
 first-come basis.
 
 This is effectively the same configuration as kernel routing QoS
 controls allowing the Squid process to access 500KB/s of traffic inbound
 for servers and unlimited upload traffic to servers.
 
 Here squid decide what we can share to each connection, the first
 machine get full bucket, if other machine arrive squid decide what he
 can receive from the bucket and continue the same to others users?
 Until they eat all the bucket and fill again the bucket to share?

There is two things here:
 1) the delay pool bucket size
 2) the I/O buffer free space

500KB is larger than the maximum buffer size. The delay pool bucket gets
more data only once every second.

So what happens is with only one machine, the client buffer gets to fill
its buffer several times over several reads until 500KB pool bucket is
empty.
The next second there are two machines, and each gets to fill its buffer
in turn until the delay pool bucket is empty.

Problem: So long as the delay pool is larger than the client buffer size
there will be some (unbalanced) sharing of the buffer. If the client
buffer is smaller than the delay pool you may see unpredictable
situations where one client gets all the traffic for a few seconds then
switches to another. With many clients some may get all the bandwidth
and others none at all.
 This pool type relies heavily on clients having traditional browser
behaviour. Whee a page was donloaded once then some time later another
page, etc. With big gaps of no traffic by each client.

 
 example 2:
 
  delay_pool_count 1
  delay_class 1 3
  delay_parameters 1 50/562500 -1/-1 4000/4000
  delay_initial_bucket_level 100
 
 Start will full bucket, here we have one bucket of 500kb/s burst
 562kb/s for all my clients but each on my clients will get =
 32kb/s(max 32kb/s) from their single bucket?

Remembering the sequence of sharing above. With this class-3 pool type
each client is only allowed to read up to 4000 bytes from the shared
pool when it fills its buffer. With a maximum of 4000 bytes total each
second if it tries to read a few bytes several times.
 The total traffic by all clients gets to 50 Bytes in one second
then reads are stopped until the global pool bucket refills. ie if 125
clients all read their 4000 bytes each in one second the 126th client is
not permitted to read.

This mostly resolves the balancing problem described above. Relating to
QoS this pool is effectively the same as QoS controls on the
client-Squid connections controlling how much traffic got delivered to
each client by Squid (downloads by client).
 But still pools is slightly worse than proper QoS as it does not cover
requests/uploads, TCP packet overheads, and large response headers can
cause buckets to go negative.

 
 About the QoS, is what I'm trying to manage, but I had been testing
 squid delay-pools and they help in some manner controlling users
 appetite.

Yes. Any type of control helps in some manner. Squid delay pools are
just old technology with some strange limits (like not covering uploads
or TCP overheads). They do not work as well as newer QoS technology
which had more knowledge and experience behind the design.

 
 QoS trying to see how to integrate with squid, because once squid
 start controlling inbound is came of difficult to me because 80/443
 are only seen by single client(squid), QoS won't see my lan
 connections to those ports which are who eat my bandwidth, but working
 on.

qos_flows directive in squid.conf can tag traffic by type so you can
control HIT/MISS with different rules if you want on the client-squid
connections.

tcp_outgoing_tos / tcp_outgoing_mark directives tag traffic on the
squid-server connections according to ACL matches.

As I understand it you setup QoS rules based on MARK or TOS to do the
bandwidth allocation you would be doing with delay pools class and
parameters directives. Then you setup squid.conf to tag the traffic into
the QoS pool types the same as you would have done with delay_pool_access.

Amos

[squid-users] delay pools status

[Beto Moreno]

Hi.

Squid 3.1.22.

Learning this feature from squid, delay-pools.

1; aggregate the most simple.
-If I have a dsl 6Mb/s but I want to share to my users 4MB/s and the
rest for my other servers.

delay_pool_count 1
delay_class 1 1
delay_parameters 1 50/562500   == 4.5Mb/s Max and hold this to 4Mb/s.
delay_initial_bucket_level 100  === I want to start with full 4.5MB right?

Once my clients start surfing, example 10 users, squid will share
50/10 they will have
+/- 50Kb/s each one?

My network is class C.

If more clients arrive they will again do some calculations
rate/numbers of users?

Exist a way to see the status of the pools connections example with cachemgr?

Thanks.

Re: [squid-users] delay pools status

[Amos Jeffries]

On 21/03/2014 6:04 a.m., Beto Moreno wrote:
 Hi.
 
 Squid 3.1.22.
 
 Learning this feature from squid, delay-pools.
 
 1; aggregate the most simple.
 -If I have a dsl 6Mb/s but I want to share to my users 4MB/s and the
 rest for my other servers.
 
 delay_pool_count 1
 delay_class 1 1
 delay_parameters 1 50/562500   == 4.5Mb/s Max and hold this to 4Mb/s.
 delay_initial_bucket_level 100  === I want to start with full 4.5MB right?
 
 Once my clients start surfing, example 10 users, squid will share
 50/10 they will have
 +/- 50Kb/s each one?

No there is 500KB/sec allocated to client response (download) traffic on
first-come basis.

This is effectively the same configuration as kernel routing QoS
controls allowing the Squid process to access 500KB/s of traffic inbound
for servers and unlimited upload traffic to servers.

PS. the delay pools feature in Squid predates QoS and most traffic
management is now far better done by QoS tools instead of Squid delay pools.


 
 My network is class C.
 
 If more clients arrive they will again do some calculations
 rate/numbers of users?
 
 Exist a way to see the status of the pools connections example with cachemgr?

Delay pools should be listed in the delay report.

Amos

Re: [squid-users] Delay Pools

[csn233]

 On Sun, Mar 16, 2014 at 9:43 AM, Amos Jeffries squ...@treenet.co.nz wrote:

 Next, I also tried client_delay_pools (3.3.11/3.4.3)

 client_delay_pools 1
 client_delay_access 1 allow all
 client_delay_parameters 1 128000 256000

 This gets connection reset straightaway. What am I missing?


 Information about what the connection reset is coming from?
  Is squid crashing? http://bugs.squid-cache.org/show_bug.cgi?id=3696

 Amos


 Yes, same assertion failed in cache.log, and Squid's crashing and restarting.

Re: [squid-users] Delay Pools

[Amos Jeffries]

On 14/03/2014 4:34 p.m., csn233 wrote:
 Testing delay_pools on 3.3.11 (and other versions too)
 
 delay_pools 1
 delay_class 1 3
 delay_parameters 1 -1/-1 -1/-1 128000/256000
 delay_access 1 allow all
 
 This works as expected. However, when requests goes a bit higher, say
 20 req/s, the CPU shoots up to 100%, and strace shows epoll calls:
 
 epoll_ctl(6, EPOLL_CTL_MOD, 7251, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP,
 {u32=7251, u64=15832364667084217427}}) = 0
 epoll_ctl(6, EPOLL_CTL_MOD, 8068, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP,
 {u32=8068, u64=14612651267063816068}}) = 0
 epoll_wait(6, {{EPOLLOUT, {u32=46, u64=3639753988997382190}},
 {EPOLLOUT, {u32=7111, u64=16029163435841297351}}, {EPOLLOUT,.. [snip]
 
 Is this normal?

Maybe yes, maybe no.

It is normal to see that type of fast looping when there is a lot of
incoming connections or TCP buffers constantly providing new bytes for
handling.

It may be expected when the delay pool slows down reading out of TCP
buffers and causes traffic to become backlogged.

However, I think sockets hitting the delay limits should be omitted from
the polling until they had more bytes allowed. So this may be a bug in
the mechanism doing that.


 
 Next, I also tried client_delay_pools (3.3.11/3.4.3)
 
 client_delay_pools 1
 client_delay_access 1 allow all
 client_delay_parameters 1 128000 256000
 
 This gets connection reset straightaway. What am I missing?
 

Information about what the connection reset is coming from?
 Is squid crashing? http://bugs.squid-cache.org/show_bug.cgi?id=3696

Amos

[squid-users] Delay Pools

[csn233]

Testing delay_pools on 3.3.11 (and other versions too)

delay_pools 1
delay_class 1 3
delay_parameters 1 -1/-1 -1/-1 128000/256000
delay_access 1 allow all

This works as expected. However, when requests goes a bit higher, say
20 req/s, the CPU shoots up to 100%, and strace shows epoll calls:

epoll_ctl(6, EPOLL_CTL_MOD, 7251, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP,
{u32=7251, u64=15832364667084217427}}) = 0
epoll_ctl(6, EPOLL_CTL_MOD, 8068, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP,
{u32=8068, u64=14612651267063816068}}) = 0
epoll_wait(6, {{EPOLLOUT, {u32=46, u64=3639753988997382190}},
{EPOLLOUT, {u32=7111, u64=16029163435841297351}}, {EPOLLOUT,.. [snip]

Is this normal?

Next, I also tried client_delay_pools (3.3.11/3.4.3)

client_delay_pools 1
client_delay_access 1 allow all
client_delay_parameters 1 128000 256000

This gets connection reset straightaway. What am I missing?

[squid-users] Delay Pools bug?

[Dan Charlesworth]

Hi folks

We have two Squid boxes (one with 2.7, one with 3.4.3), which have almost 
identical delay pool configurations, but the 3.4.3 box is limiting the speed to 
around half of what the 2.7 one is; around 32KB/s when it should be 64KB/s

Relevant parts of the configs:

2.7 Config
—
acl 1_cmps src “/path/to/iplists/policy_1

delay_pools 1
delay_class 1 2
http_access allow 1_cmps
delay_access 1 allow  1_cmps
delay_parameters 1 -1/-1 65536/65536
delay_initial_bucket_level 100

3.4.3 Config
—
external_acl_type bandwidth_type ttl=300 children-startup=2 children-idle=1 
children-max=10 %URI %EXT_LOG bandwidth_ext_acl.py

delay_pools 1
delay_class 1 2
acl bandwidth_ext_acl external bandwidth_type 512
http_access allow bandwidth_ext_acl
delay_access 1 allow bandwidth_ext_acl
delay_parameters 1 -1/-1 65536/65536
delay_initial_bucket_level 100

I’m not able to reveal any of the specific external ACL code, but that 
shouldn’t make any difference should it?

Re: [squid-users] Delay Pools bug?

[Amos Jeffries]

On 28/02/2014 2:15 p.m., Dan Charlesworth wrote:
 Hi folks
 
 We have two Squid boxes (one with 2.7, one with 3.4.3), which have almost 
 identical delay pool configurations, but the 3.4.3 box is limiting the speed 
 to around half of what the 2.7 one is; around 32KB/s when it should be 64KB/s
 

We have a strange situation in Squid-3 with delay pools. Both these seem
to be happening:

overcounting
 http://bugs.squid-cache.org/show_bug.cgi?id=3536

undercounting
 http://bugs.squid-cache.org/show_bug.cgi?id=522

Delay pools is a very weak and buggy form of QoS. In general it is
better to use the tcp_outgoing_tos/_mark features and use the far better
system QoS controls.



 Relevant parts of the configs:
 
 2.7 Config
 —
 acl 1_cmps src “/path/to/iplists/policy_1
 
 delay_pools 1
 delay_class 1 2
 http_access allow 1_cmps
 delay_access 1 allow  1_cmps
 delay_parameters 1 -1/-1 65536/65536
 delay_initial_bucket_level 100
 
 3.4.3 Config
 —
 external_acl_type bandwidth_type ttl=300 children-startup=2 children-idle=1 
 children-max=10 %URI %EXT_LOG bandwidth_ext_acl.py
 
 delay_pools 1
 delay_class 1 2
 acl bandwidth_ext_acl external bandwidth_type 512
 http_access allow bandwidth_ext_acl
 delay_access 1 allow bandwidth_ext_acl
 delay_parameters 1 -1/-1 65536/65536
 delay_initial_bucket_level 100
 
 I’m not able to reveal any of the specific external ACL code, but that 
 shouldn’t make any difference should it?
 

No that wont matter for this.

Amos

Re: [squid-users] Delay Pools with Digest and External Auth

[Nils Hügelmann]

Thanks, i've made it working using a modification of your recommendations.

I summarize my solution in case others have a similar problem:

- Class 5 Delay Pools used (limit by Tag)
- External Auth helper program assigns username as EXT_TAG
- When Digest is used, there is a dummy helper that just assigns
username as EXT_TAG
- Dummy helper is activated using http_access allow proxyauth
digest_tagger

- Classification in multiple delay pools is done via other external_auth
ACLs
- These external_auths are activated (to circumvent slow/fast acl
issues) using http_access allow EXTACLNAME !all
- These external_auths need to interpret both the external_auth header
and the digest callback to get the username

Best Regards

Nils
Am 13.05.2013 02:32, schrieb Amos Jeffries:
 On 12/05/2013 8:03 a.m., Nils Hügelmann wrote:
 Hi,

 I want to use both Digest Auth and External Auth (simpleheaderauth)
 for authentification, and need to assign different delay pools to single
 users based on another external_acl (premiumcheck).

 So i have (stripped down for readibility)

 -
 external_acl_type simpleheaderauth %{Proxy-Authorization} simpleauth
 external_acl_type premiumcheck %{Proxy-Authorization} premium
 auth_param digest program digestauth

 acl proxyauth proxy_auth REQUIRED
 acl simpleheaderauth_passed external simpleheaderauth
 acl premiumcheck_passed external premiumcheck

 # activate additional external acls
 http_access allow premiumcheck_passed !all
 http_access allow freethrottled_passed !all

 http_access allow simpleheaderauth_passed
 http_access allow proxyauth
 http_access deny !proxyauth

 http_access deny all
 -

 Which works fine in regards to access control, one can either login via
 simpleheaderauth (external_acl) or via digestauth (auth_param).

 I want to have 2 bandwidth limit levels.

 Situation from here is as follows:

 When using simpleheaderauth:
   - EXT_USER is available (username passed from simpleheaderauth
 external_acl)
   - Tag is available (tag passed from simpleheaderauth external_acl)
   - premiumcheck_passed is properly set

 When using digestauth:
   - LOGIN is available (username passed from auth_param)
   - Tag is not available
   - premiumcheck_passed is not usable

 Delay pools need to work per individual user, so only class 5 pools (
 tagrate ) or class 4 pools ( aggregate, network, individual, user )
 would be possible.

 As simpleheaderauth has no user defined, and digestauth has no tag, my
 first attempt for delay_pools was to create 2 sets of pools with 2
 classes each:

 -
 delay_class 1 5
 delay_class 2 5
 delay_class 3 4
 delay_class 4 4

 # 1st set for simpleheaderauth
 delay_parameters 2 2097152/2097152
 delay_access 2 allow simpleheaderauth_passed premiumcheck_passed

 delay_parameters 1 76800/76800
 delay_access 1 deny premiumcheck_passed
 delay_access 1 allow simpleheaderauth_passed

 # 2nd set for digestauth
 delay_parameters 4 -1/-1 -1/-1 -1/-1 2097152/2097152
 delay_access 4 allow premiumcheck_passed

 delay_parameters 3 -1/-1 -1/-1 -1/-1 76800/76800
 delay_access 3 deny premiumcheck_passed
 delay_access 3 allow all
 -

 1. Can one somehow simplify this by making Tag available for digest, or
 making class 4 username available for external_acl?

 I have work lined up on the TODO list for implementing tag on auth
 interfaces in the next Squid versions.
 If you are able to assist with sponsoring that I can divert some time
 back towards it.

 However, ...

 Alternative #1:
  * make your simple and premium helper lookups produce tags indicating
 those levels.
  * create a dummy external ACL helper lookup test which always
 responds OK tag=digest-auth. Call it only after proxyauth ACL has
 succeeded doing digest.

 eg:
   external_acl_type digestauth %LOGIN basic_fake_auth
   acl digest_tagger external digestauth

   http_access allow proxyauth digest_tagger

 You can then use tag type ACLs for delay_access.


 2. The problem with my attempt is that premiumcheck_passed is not
 evaluated when usind digestauth. Every digestauth user is assigned to
 pool 3, while simpleheaderauth users are properly assigned based on
 premiumcheck_passed. How can i solve this?

 You have isolated the problem pretty accurately. It's root cause is
 the mismatch between delay_access being fast ACL check and the tests
 you are using being slow group ACL.

 Amos

Re: [squid-users] Delay Pools with Digest and External Auth

[Amos Jeffries]

On 12/05/2013 8:03 a.m., Nils Hügelmann wrote:

Hi,

I want to use both Digest Auth and External Auth (simpleheaderauth)
for authentification, and need to assign different delay pools to single
users based on another external_acl (premiumcheck).

So i have (stripped down for readibility)

-
external_acl_type simpleheaderauth %{Proxy-Authorization} simpleauth
external_acl_type premiumcheck %{Proxy-Authorization} premium
auth_param digest program digestauth

acl proxyauth proxy_auth REQUIRED
acl simpleheaderauth_passed external simpleheaderauth
acl premiumcheck_passed external premiumcheck

# activate additional external acls
http_access allow premiumcheck_passed !all
http_access allow freethrottled_passed !all

http_access allow simpleheaderauth_passed
http_access allow proxyauth
http_access deny !proxyauth

http_access deny all
-

Which works fine in regards to access control, one can either login via
simpleheaderauth (external_acl) or via digestauth (auth_param).

I want to have 2 bandwidth limit levels.

Situation from here is as follows:

When using simpleheaderauth:
  - EXT_USER is available (username passed from simpleheaderauth
external_acl)
  - Tag is available (tag passed from simpleheaderauth external_acl)
  - premiumcheck_passed is properly set

When using digestauth:
  - LOGIN is available (username passed from auth_param)
  - Tag is not available
  - premiumcheck_passed is not usable

Delay pools need to work per individual user, so only class 5 pools (
tagrate ) or class 4 pools ( aggregate, network, individual, user )
would be possible.

As simpleheaderauth has no user defined, and digestauth has no tag, my
first attempt for delay_pools was to create 2 sets of pools with 2
classes each:

-
delay_class 1 5
delay_class 2 5
delay_class 3 4
delay_class 4 4

# 1st set for simpleheaderauth
delay_parameters 2 2097152/2097152
delay_access 2 allow simpleheaderauth_passed premiumcheck_passed

delay_parameters 1 76800/76800
delay_access 1 deny premiumcheck_passed
delay_access 1 allow simpleheaderauth_passed

# 2nd set for digestauth
delay_parameters 4 -1/-1 -1/-1 -1/-1 2097152/2097152
delay_access 4 allow premiumcheck_passed

delay_parameters 3 -1/-1 -1/-1 -1/-1 76800/76800
delay_access 3 deny premiumcheck_passed
delay_access 3 allow all
-

1. Can one somehow simplify this by making Tag available for digest, or
making class 4 username available for external_acl?


I have work lined up on the TODO list for implementing tag on auth 
interfaces in the next Squid versions.
If you are able to assist with sponsoring that I can divert some time 
back towards it.


However, ...

Alternative #1:
 * make your simple and premium helper lookups produce tags indicating 
those levels.
 * create a dummy external ACL helper lookup test which always responds 
OK tag=digest-auth. Call it only after proxyauth ACL has succeeded 
doing digest.


eg:
  external_acl_type digestauth %LOGIN basic_fake_auth
  acl digest_tagger external digestauth

  http_access allow proxyauth digest_tagger

You can then use tag type ACLs for delay_access.



2. The problem with my attempt is that premiumcheck_passed is not
evaluated when usind digestauth. Every digestauth user is assigned to
pool 3, while simpleheaderauth users are properly assigned based on
premiumcheck_passed. How can i solve this?


You have isolated the problem pretty accurately. It's root cause is the 
mismatch between delay_access being fast ACL check and the tests you 
are using being slow group ACL.


Amos

[squid-users] Delay Pools with Digest and External Auth

[Nils Hügelmann]

Hi,

I want to use both Digest Auth and External Auth (simpleheaderauth)
for authentification, and need to assign different delay pools to single
users based on another external_acl (premiumcheck).

So i have (stripped down for readibility)

-
external_acl_type simpleheaderauth %{Proxy-Authorization} simpleauth
external_acl_type premiumcheck %{Proxy-Authorization} premium
auth_param digest program digestauth

acl proxyauth proxy_auth REQUIRED
acl simpleheaderauth_passed external simpleheaderauth
acl premiumcheck_passed external premiumcheck

# activate additional external acls
http_access allow premiumcheck_passed !all
http_access allow freethrottled_passed !all

http_access allow simpleheaderauth_passed
http_access allow proxyauth
http_access deny !proxyauth

http_access deny all
-

Which works fine in regards to access control, one can either login via
simpleheaderauth (external_acl) or via digestauth (auth_param).

I want to have 2 bandwidth limit levels.

Situation from here is as follows:

When using simpleheaderauth:
 - EXT_USER is available (username passed from simpleheaderauth
external_acl)
 - Tag is available (tag passed from simpleheaderauth external_acl)
 - premiumcheck_passed is properly set

When using digestauth:
 - LOGIN is available (username passed from auth_param)
 - Tag is not available
 - premiumcheck_passed is not usable

Delay pools need to work per individual user, so only class 5 pools (
tagrate ) or class 4 pools ( aggregate, network, individual, user )
would be possible.

As simpleheaderauth has no user defined, and digestauth has no tag, my
first attempt for delay_pools was to create 2 sets of pools with 2
classes each:

-
delay_class 1 5
delay_class 2 5
delay_class 3 4
delay_class 4 4

# 1st set for simpleheaderauth
delay_parameters 2 2097152/2097152
delay_access 2 allow simpleheaderauth_passed premiumcheck_passed

delay_parameters 1 76800/76800
delay_access 1 deny premiumcheck_passed
delay_access 1 allow simpleheaderauth_passed

# 2nd set for digestauth
delay_parameters 4 -1/-1 -1/-1 -1/-1 2097152/2097152
delay_access 4 allow premiumcheck_passed

delay_parameters 3 -1/-1 -1/-1 -1/-1 76800/76800
delay_access 3 deny premiumcheck_passed
delay_access 3 allow all
-

1. Can one somehow simplify this by making Tag available for digest, or
making class 4 username available for external_acl?

2. The problem with my attempt is that premiumcheck_passed is not
evaluated when usind digestauth. Every digestauth user is assigned to
pool 3, while simpleheaderauth users are properly assigned based on
premiumcheck_passed. How can i solve this?

Thanks

Nils Hügelmann

[squid-users] Delay pools to limit bandwidth to youtube.com

[xbanux baner]

Hello Good People,

My requirement is to limit the overall bandwidth usage of youtube.com
to 128Kbps. The limit should be set for the entire LAN.

Is the below configuration correct?
acl youtube dstdomain youtube.com
delay_pools 1
delay_class 1 2
delay_parameters 1 16000/16000 16000/16000
delay_access 1 allow youtube

Can any please help me out with this?

Thanks in advance.

Re: [squid-users] Delay pools to limit bandwidth to youtube.com

[Amos Jeffries]

On 2013-01-04 01:58, xbanux baner wrote:

Hello Good People,

My requirement is to limit the overall bandwidth usage of youtube.com
to 128Kbps. The limit should be set for the entire LAN.

Is the below configuration correct?
acl youtube dstdomain youtube.com
delay_pools 1
delay_class 1 2
delay_parameters 1 16000/16000 16000/16000
delay_access 1 allow youtube

Can any please help me out with this?

Thanks in advance.



Yes and no.

It will limit youtube.com URLs but not the videos presented by the 
YouTube website, which use many completely different domain names (note 
the 's').


Amos

[squid-users] delay pools and ntlm errors

[Leonardo Bacha Abrantes]

Hi guys,

I'm facing many problems with my squid.

This message appears a lot on the log:

[2012/10/05 15:57:28.523249,  1] libsmb/ntlmssp.c:342(ntlmssp_update)
  got NTLMSSP command 3, expected 1

===

Surf on internet is slow when  delay pools is enabled.


I also had:

FATAL: Too many queued ntlmauthenticator requests

and I increased the value of auth_param ntlm|basic children.

=

my squid.conf:



http_port 3128
append_domain .contoso.local
cache_effective_user squid
cache_mem 2 GB
cache_effective_group squid
forwarded_for off
httpd_suppress_version_string on
visible_hostname proxy.contoso.local
retry_on_error on
pipeline_prefetch on


auth_param ntlm program /usr/bin/ntlm_auth
--helper-protocol=squid-2.5-ntlmssp --domain=contoso
auth_param ntlm children 45
auth_param basic program /usr/bin/ntlm_auth
--helper-protocol=squid-2.5-basic --domain=contoso
auth_param basic children 25
auth_param basic realm Para prosseguir e necessario digitar seu login de rede.
auth_param basic credentialsttl 2 hours


acl localnetwork src xxx.xxx.xxx.xxx/25
acl AuthorizedUsers proxy_auth -i /etc/squid/default_access.acl
#acl unlimitedBandwidth src /etc/squid/unlimited_bandwidth
acl localhost src 127.0.0.1
acl java browser Java/1.4 Java/1.5 Java/1.6

cache_dir ufs /var/spool/squid 6144 16 256
coredump_dir /var/spool/squid
maximum_object_size_in_memory 512 KB
maximum_object_size 64 MB
minimum_object_size 0 KB

acl manager proto cache_object
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 8080# http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1025-65535  # unregistered ports
acl purge method PURGE
acl CONNECT method CONNECT

#delay_pools 1

#delay_class 1 2
#delay_parameters 1 -1/-1 65536/65536
#delay_access 1 deny unlimitedBandwidth localhost
#delay_access 1 allow localnetwork
#delay_access 1 deny all

logformat combined [%tl] %un %a %rm %Ss %Hs %ru
access_log /var/log/squid/access.log squid
access_log /var/log/squid/gerencia.log combined
cache_store_log /var/log/squid/store.log

redirect_program /etc/squidGuard/bin/squidGuard -c
/usr/local/squidGuard/squidGuard.conf
redirect_children 30


http_access deny CONNECT !SSL_ports
http_access allow java
http_access allow  AuthorizedUsers
http_access allow  unlimitedBandwidth
#http_access allow  AuthorizedUsers
http_access deny all

cache_swap_low 90
cache_swap_high 95
dns_nameservers xxx.xxx.xxx.xxx
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern -i (zip|rar|tar\.gz|exe)$  0  50%  259200
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .   0   20% 4320
request_header_access   All allow   all


many thanks!

Re: [squid-users] Delay Pools and limits

[Rafael Gomes]

Amos,

Thanks, with this workaround, I solved my problem. I hope we can fix
this soon.

On Thu, Aug 30, 2012 at 2:03 AM, Amos Jeffries squ...@treenet.co.nz wrote:
 On 30.08.2012 07:47, Rafael Gomes wrote:

 Guys,

 I am using delay pool version 4 to limit bandwidth of user.

 But this is not working well :(

 The bandwidth of this user don't pass of 50KBps, when should keep 100KBps

 Follow my rules:

 acl rafael proxy_auth rafael.gomes

 delay_pools 1
 delay_class 1 4

 delay_parameters 1 -1/-1 -1/-1 -1/-1 10/10
 delay_access 1 allow rafael


 That would be http://bugs.squid-cache.org/show_bug.cgi?id=3536

 Amos



-- 
Rafael Gomes
Consultor em TI
LPIC-1 MCSO
(71) 8318-0284

Atenção: Este e-mail pode conter anexos no formato ODF (Open Document
Format)/ABNT (extensões odt, ods, odp, odb, odg). Antes de pedir os
anexos em outro formato, você pode instalar gratuita e livremente o
BrOffice (http://www.broffice.org).

[squid-users] Delay Pools and limits

[Rafael Gomes]

Guys,

I am using delay pool version 4 to limit bandwidth of user.

But this is not working well :(

The bandwidth of this user don't pass of 50KBps, when should keep 100KBps

Follow my rules:

acl rafael proxy_auth rafael.gomes

delay_pools 1
delay_class 1 4

delay_parameters 1 -1/-1 -1/-1 -1/-1 10/10
delay_access 1 allow rafael

-- 
Rafael Gomes
Consultor em TI
LPIC-1 MCSO
(71) 8318-0284

Atenção: Este e-mail pode conter anexos no formato ODF (Open Document
Format)/ABNT (extensões odt, ods, odp, odb, odg). Antes de pedir os
anexos em outro formato, você pode instalar gratuita e livremente o
BrOffice (http://www.broffice.org).

Re: [squid-users] Delay Pools and limits

[Amos Jeffries]

On 30.08.2012 07:47, Rafael Gomes wrote:

Guys,

I am using delay pool version 4 to limit bandwidth of user.

But this is not working well :(

The bandwidth of this user don't pass of 50KBps, when should keep 
100KBps


Follow my rules:

acl rafael proxy_auth rafael.gomes

delay_pools 1
delay_class 1 4

delay_parameters 1 -1/-1 -1/-1 -1/-1 10/10
delay_access 1 allow rafael


That would be http://bugs.squid-cache.org/show_bug.cgi?id=3536

Amos

[squid-users] delay pools with IP ranges

[Marlon Bastida]

Hi,

I need to create delay pools to do bandwith control.

How can I do to use IP ranges on acl src statements? Not netmasks bits /24.


Tks in advance,
Marlon

Re: [squid-users] delay pools with IP ranges

[Amos Jeffries]

On 16.05.2012 05:38, Marlon Bastida wrote:

Hi,

I need to create delay pools to do bandwith control.


Considered TOS or QoS functionality of Squid? it tends to work better 
than delay pools.




How can I do to use IP ranges on acl src statements? Not netmasks 
bits /24.




Hmm. Question unrelated to delay pools.


The ACL src and dst type syntax is: first-IP [ '-' last-IP] ['/' 
netmask].


  acl blah src 192.168.0.1-192.168.0.5/32
or
  acl blah src 192.168.0.1-192.168.0.5


Amos

Re: [squid-users] Delay pools and ICAP issue in 3.2

[FredB]

- Mail original -
 De: Alex Crow a...@nanogherkin.com
 À: squid-users@squid-cache.org
 Envoyé: Dimanche 8 Janvier 2012 20:04:46
 Objet: [squid-users] Delay pools and ICAP issue in 3.2

 Hi Amos, all,

 I continue testing 3.2 as promised after a brief hiatus (XP clients,
 NTLM auth, external ACLs on NT groups).

 I am pleased to say that in squid-3.2.0.14-20120106-r11479 that
 previous
 issues with external acls deciding users were in a group that they
 weren't (or the opposite) appear to be resolved, at least from
 testing
 on one or two client machines. I will try to extend this to some more
 users ASAP.

 However I have also seen that with both ICAP (to c-icap/clamav) and
 delay pools that browsing stalls on certain sites, especially on a
 class
 3 delay pool with conservative per-client limits (eg 200kB/s
 perclient
 rate, 100kB/s refill). For instance, if I load http://bbc.co.uk/news
 and
 then play a video from that site, then attempt to load a main page
 from
 that site in another tab in Firefox, that tab will just remain a
 blank
 page although the logs do show a few items being processed. If I
 either
 turn off ICAP, or turn of delay pools, all seems well. In fact, if I
 just use a single class 1 pool limiting to 100MB/s it also seems
 fine.

 Delay pools also fail with ICAP unless I exclude streaming media, in
 particular mime type application/x-fcs from ICAP. If I don't exclude
 such things, it is very rare that I can load bbc.co.uk/news at all in
 Firefox. I used some reasonable debug_options settings to try to
 detect
 the problem but I don't see any errors, but the browser just shows
 Waiting for hostname for an hour (as long as I left it) and squid
 cache/access.log show nothing happening.

 I notice a couple of other posts about this. Is this a known problem
 in
 3.2? If not, please provide appropriate debug_options settings and
 I'll
 try to get logs for you in the next 2-4 weeks (I'm afraid I have a
 lot
 on over this time).

 Thanks

 Alex


Maybe there is a link with this http://bugs.squid-cache.org/show_bug.cgi?id=3462

[squid-users] Delay pools and ICAP issue in 3.2

[Alex Crow]

Hi Amos, all,

I continue testing 3.2 as promised after a brief hiatus (XP clients, 
NTLM auth, external ACLs on NT groups).


I am pleased to say that in squid-3.2.0.14-20120106-r11479 that previous 
issues with external acls deciding users were in a group that they 
weren't (or the opposite) appear to be resolved, at least from testing 
on one or two client machines. I will try to extend this to some more 
users ASAP.


However I have also seen that with both ICAP (to c-icap/clamav) and 
delay pools that browsing stalls on certain sites, especially on a class 
3 delay pool with conservative per-client limits (eg 200kB/s perclient 
rate, 100kB/s refill). For instance, if I load http://bbc.co.uk/news and 
then play a video from that site, then attempt to load a main page from 
that site in another tab in Firefox, that tab will just remain a blank 
page although the logs do show a few items being processed. If I either 
turn off ICAP, or turn of delay pools, all seems well. In fact, if I 
just use a single class 1 pool limiting to 100MB/s it also seems fine.


Delay pools also fail with ICAP unless I exclude streaming media, in 
particular mime type application/x-fcs from ICAP. If I don't exclude 
such things, it is very rare that I can load bbc.co.uk/news at all in 
Firefox. I used some reasonable debug_options settings to try to detect 
the problem but I don't see any errors, but the browser just shows 
Waiting for hostname for an hour (as long as I left it) and squid 
cache/access.log show nothing happening.


I notice a couple of other posts about this. Is this a known problem in 
3.2? If not, please provide appropriate debug_options settings and I'll 
try to get logs for you in the next 2-4 weeks (I'm afraid I have a lot 
on over this time).


Thanks

Alex

[squid-users] Delay pools and parenr proxy

[Alex]

part of my squid.conf

acl CACHE peername emts
acl LOCALNET src 192.168.0.0/16
acl ME src 192.168.0.19
...
cache_peer xx.xx.xx.xx parent 3138 0 default proxy-only no-query name=emts
...
delay_pools 1
delay_class 1 2
delay_parameters 1 51000/50 27000/10
delay_access 1 deny CACHE
delay_access 1 allow LOCALNET !ME
delay_access 1 deny all

rule delay_access 1 deny CACHE doesn't work? squid still limit the 
bandwidth

why?

[squid-users] Delay Pools don't work

[Romag]

Hi at all,
i need to limit the bandwith of my subnet (10 subnet) and i try to use
Delay Pool for this..if i use a Delay Pool for all the net it's works
very well, but if i specify a ACL for any subnet the limit don't
work..

My stub of configuration is:

acl SUBNET1 src IP/SUBNET
acl SUBNET2 src IP/SUBNET

delay_pools 1
delay_class 1 3
delay_access 1 allow SUBNET1
delay_access 1 deny all
delay_parameters 1 64000/64000 -1/-1 16000/64000

But the bandwith is not limited.. i need to limit some subnet to 1Mbps
and some to 2Mbps and no limit for localnet.

Thanks!

Re: [squid-users] Delay Pools don't work

[Amos Jeffries]

On 22/06/11 20:36, Romag wrote:

Hi at all,
i need to limit the bandwith of my subnet (10 subnet) and i try to use
Delay Pool for this..if i use a Delay Pool for all the net it's works
very well, but if i specify a ACL for any subnet the limit don't
work..

My stub of configuration is:

acl SUBNET1 src IP/SUBNET
acl SUBNET2 src IP/SUBNET

delay_pools 1
delay_class 1 3
delay_access 1 allow SUBNET1
delay_access 1 deny all
delay_parameters 1 64000/64000 -1/-1 16000/64000


whole network bandwidth is limited to 64,000 bytes/sec (note that is 
bytes) shared between all machines listed in SUBNET1 ACL.


No limitation on /24 subnet.

any one machine (/32) is allowed to consume up to 16000 bytes/sec from 
the Internet.




But the bandwith is not limited.. i need to limit some subnet to 1Mbps
and some to 2Mbps and no limit for localnet.

Thanks!


By dont work you mean what?
 NP: keeping in mind that delay_pool only affects traffic fetched from 
peers or DIRECT from origins. Cache HIT are not delayed.


Which version of Squid are you using ?

Are you sure the machine IP is in SUBNET1 ?
 (sorry if that seems dumb, but it happens sometimes that testers are 
using localhost and trying to match global IPs. Or the reverse)


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.9 and 3.1.12.3

[squid-users] delay pools and reverse proxy mode...

[John Doe]

Hi,

I have a little issue with delay pools and Squid (2.7) in reverse proxy mode...
If the target is not yet in the cache, I get correctly slowed down.
But, once the target is cached, I can dowload at full speed...
If I purge the url from the cache and try again, I get once more slowed down... 
once.
Is this expected behavior in reverse proxy mode?
Do delay pools only apply to squid-backend connections?
Or is there a way to make it apply to client-squid connections?

Thx,
JD

Re: [squid-users] delay pools and reverse proxy mode...

[Amos Jeffries]

On 29/03/11 21:42, John Doe wrote:

Hi,

I have a little issue with delay pools and Squid (2.7) in reverse proxy mode...
If the target is not yet in the cache, I get correctly slowed down.
But, once the target is cached, I can dowload at full speed...
If I purge the url from the cache and try again, I get once more slowed down...
once.
Is this expected behavior in reverse proxy mode?


It is for reverse-proxy with delay pools active unless care is taken to 
omit the site(s) from the pools.



Do delay pools only apply to squid-backend connections?


yes. delay_pools delay pools do.


Or is there a way to make it apply to client-squid connections?


client_delay_pools delay pools. which is available in 3.2 beta.

OR the QoS controls which are available in slightly variant forms with 
all Squid releases since 2.6.


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.11
  Beta testers wanted for 3.2.0.5

Re: [squid-users] delay pools and reverse proxy mode...

[John Doe]

From: Amos Jeffries squ...@treenet.co.nz

 On 29/03/11 21:42, John Doe wrote:
  I have a little  issue with delay pools and Squid (2.7) in reverse proxy 
mode...
  Or is there a  way to make it apply to client-squid connections?
 client_delay_pools  delay pools. which is available in 3.2 beta.

Guess I will have to find another way; or wait for 3.2 stable...

Thank you,
JD

Re: [squid-users] delay pools

[Márcio Luciano Donada]

Em 25/3/2011 21:19, Amos Jeffries escreveu:
 On 26/03/11 10:39, Márcio Luciano Donada wrote:
 I am using version: squid-2.7.9 FreeBSD. I am using Authentication with
 delay pools, as follows:

 auth_param basic program /usr/local/libexec/squid/squid_ldap_auth -R -D
 cn=suporte,dc=xxx,dc=com,dc=br -w f34fadsfsdf -b
 ou=Usuarios,dc=xxx,dc=com,dc=br -f uid=%s -h 192.168.1.1 -d v3
 auth_param basic children 5
 auth_param basic realm Digite sua senha

 acl ldap-auth proxy_auth REQUIRED
 http_access allow ldap-auth
 http_access allow localhost
 http_access deny all
 acl 128kbps proxy_auth /usr/local/etc/squid/user.txt
 acl 256kbps proxy_auth /usr/local/etc/squid/profs.txt
 acl admin proxy_auth /usr/local/etc/squid/admin.txt

 delay_pools 3
 delay_class 1 2
 delay_access 1 allow 128kbps
 delay_access 1 deny all
 delay_class 2 2
 delay_access 2 allow 256kbps
 delay_access 2 deny all
 delay_class 3 2
 delay_access 3 allow admin
 delay_access 3 deny all

 delay_parameters 1 128000/512000 128000/512000
 delay_parameters 2 512000/1024000 512000/1024000
 delay_parameters 3 -1/-1 -1/-1


 But initially it works fine, after 5 min, no longer access anything else
 is simply too slow and nothing works. I wanted to make a control of
 128kbps and 256kbps to stay slow and not having to close the browser and
 restart the operation when

 Couple of problems there:

 Delay pools are measured in *Bytes*. Those numbers are 8x too big for
 Kbps. Did you mean KBps? (upper/lower case matters a *lot* in bps units).

 The first A/B parameter limits the entire network segment bandwidth.
 The second one limits the individual IP.
   Those limits above allow a single user to max out the connection and
 block all other users from getting a single byte through.

 Also, the pool #3 is doing relatively expensive traffic accounting in
 order to do nothing. You can remove it entirely.


 You want something like...

  # pool 1: no network-wide cap, individuals at 128KBps
  delay_parameters 1 -1/-1 131072/131072

  # pool 2: no network-wide cap, individuals at 256KBps
  delay_parameters 2 -1/-1 262144/262144


 or, if you did means Kbps instead of KBps ...

  # pool 1: no network-wide cap, individuals at 128Kbps
  delay_parameters 1 -1/-1 16384/16384

  # pool 2: no network-wide cap, individuals at 256Kbps
  delay_parameters 2 -1/-1 32768/32768


 Amos

Thanks, worked perfectly

-- 
Márcio Luciano Donada
Aurora Alimentos - T.I. Matriz
Coop. Central Oeste Catarinense

[squid-users] Delay pools for VPN users

[Dayo Adewunmi]

Hi

Is there any feasible way of limiting VPN users? They seem to be maxing 
out my link, every now and then. So far, what I've been able to do is 
find out what URL they VPN to and adding that to a delay pool.


Best regards

Dayo

Re: [squid-users] Delay pools for VPN users

[Amos Jeffries]

On 26/03/11 20:00, Dayo Adewunmi wrote:

Hi

Is there any feasible way of limiting VPN users? They seem to be maxing
out my link, every now and then. So far, what I've been able to do is
find out what URL they VPN to and adding that to a delay pool.



VPN has nothing to do with Squid. It operates at the packet level *way* 
down below HTTP.



Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.11
  Beta testers wanted for 3.2.0.5

[squid-users] delay pools

[Márcio Luciano Donada]

I am using version: squid-2.7.9 FreeBSD. I am using Authentication with
delay pools, as follows:

auth_param basic program /usr/local/libexec/squid/squid_ldap_auth -R -D
cn=suporte,dc=xxx,dc=com,dc=br -w f34fadsfsdf -b
ou=Usuarios,dc=xxx,dc=com,dc=br -f uid=%s -h 192.168.1.1 -d v3
auth_param basic children 5
auth_param basic realm Digite sua senha

acl ldap-auth proxy_auth REQUIRED
http_access allow ldap-auth
http_access allow localhost
http_access deny all
acl 128kbps proxy_auth /usr/local/etc/squid/user.txt
acl 256kbps proxy_auth /usr/local/etc/squid/profs.txt
acl admin proxy_auth /usr/local/etc/squid/admin.txt

delay_pools 3
delay_class 1 2
delay_access 1 allow 128kbps
delay_access 1 deny all
delay_class 2 2
delay_access 2 allow 256kbps
delay_access 2 deny all
delay_class 3 2
delay_access 3 allow admin
delay_access 3 deny all

delay_parameters 1 128000/512000 128000/512000
delay_parameters 2 512000/1024000 512000/1024000
delay_parameters 3 -1/-1 -1/-1


But initially it works fine, after 5 min, no longer access anything else
is simply too slow and nothing works. I wanted to make a control of
128kbps and 256kbps to stay slow and not having to close the browser and
restart the operation when

-- 
Márcio Luciano Donada
Aurora Alimentos - T.I. Matriz
Coop. Central Oeste Catarinense

Re: [squid-users] delay pools

[Amos Jeffries]

On 26/03/11 10:39, Márcio Luciano Donada wrote:

I am using version: squid-2.7.9 FreeBSD. I am using Authentication with
delay pools, as follows:

auth_param basic program /usr/local/libexec/squid/squid_ldap_auth -R -D
cn=suporte,dc=xxx,dc=com,dc=br -w f34fadsfsdf -b
ou=Usuarios,dc=xxx,dc=com,dc=br -f uid=%s -h 192.168.1.1 -d v3
auth_param basic children 5
auth_param basic realm Digite sua senha

acl ldap-auth proxy_auth REQUIRED
http_access allow ldap-auth
http_access allow localhost
http_access deny all
acl 128kbps proxy_auth /usr/local/etc/squid/user.txt
acl 256kbps proxy_auth /usr/local/etc/squid/profs.txt
acl admin proxy_auth /usr/local/etc/squid/admin.txt

delay_pools 3
delay_class 1 2
delay_access 1 allow 128kbps
delay_access 1 deny all
delay_class 2 2
delay_access 2 allow 256kbps
delay_access 2 deny all
delay_class 3 2
delay_access 3 allow admin
delay_access 3 deny all

delay_parameters 1 128000/512000 128000/512000
delay_parameters 2 512000/1024000 512000/1024000
delay_parameters 3 -1/-1 -1/-1


But initially it works fine, after 5 min, no longer access anything else
is simply too slow and nothing works. I wanted to make a control of
128kbps and 256kbps to stay slow and not having to close the browser and
restart the operation when


Couple of problems there:

Delay pools are measured in *Bytes*. Those numbers are 8x too big for 
Kbps. Did you mean KBps? (upper/lower case matters a *lot* in bps units).


The first A/B parameter limits the entire network segment bandwidth. The 
second one limits the individual IP.
  Those limits above allow a single user to max out the connection and 
block all other users from getting a single byte through.


Also, the pool #3 is doing relatively expensive traffic accounting in 
order to do nothing. You can remove it entirely.



You want something like...

 # pool 1: no network-wide cap, individuals at 128KBps
 delay_parameters 1 -1/-1 131072/131072

 # pool 2: no network-wide cap, individuals at 256KBps
 delay_parameters 2 -1/-1 262144/262144


or, if you did means Kbps instead of KBps ...

 # pool 1: no network-wide cap, individuals at 128Kbps
 delay_parameters 1 -1/-1 16384/16384

 # pool 2: no network-wide cap, individuals at 256Kbps
 delay_parameters 2 -1/-1 32768/32768


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.11
  Beta testers wanted for 3.2.0.5

[squid-users] Delay-pools (class2) limit more than specified

[Jose Lopes]

Hi,

I'm  using delay pools to limit bandwidth.
My version of squid is Squid Cache: Version 3.1.0.15.

My configs are:

delay_pools 2
delay_access 1 allow client_hosts1
delay_access 1 deny all
delay_class 1 2
delay_parameters 1 131072/131072 65536/65536

delay_access 2 allow client_hosts2
delay_access 2 deny all
delay_class 2 1
delay_parameters 2 131072/131072

Delay_pool 2 works well, one host at client_hosts2 downloads at ~130KB/s .
At delay_pool 1, with one host downloading, it download's at ~33KB/s .
At delay_pool 1, with all hosts (client_hosts1) downloading, the global max 
value of download is ~66KB/s.

Seems like delay_pool of class 2 limit at half of the bandwidth specified.

How do I sort out this problem?

Thanks in advance.
Regards
Jose Lopes

[squid-users] Delay pools

[Sakhi Louw]

Hi,

Does anyone know a good site with detailed information on squid
delay pools.

-- 
Sakhi Louw

[squid-users] Delay pools

[Sakhi Louw]

Hi,

Does anyone know a good site with detailed information on squid
delay pools.

-- 
Sakhi Louw

Re: [squid-users] Delay pools

[Amos Jeffries]

Sakhi Louw wrote:

Hi,

Does anyone know a good site with detailed information on squid
delay pools.



http://wiki.squid-cache.org/Features/DelayPools

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21
  Current Beta Squid 3.1.0.15

[squid-users] Delay Pools advice?

[Roland Roland]

Hello,


i've set up squid about 2 month ago with poorly configured settings just 
to set things going ( i admit it wasnt the right thing to do thoguh time 
were of the essence )
at the moment i decided that it's about time to take a look at each rule 
i've made and fix it to the best i could. so i'm sending this email 
hoping someone would show any mistake i might have done :



I. Info:

1. Our working hours are from 8 am to 6 pm
2. One ISP link that's dedicated for squid/browsing has a 2 mb shared 
bandwidth with 8 GB of quota



II. Required:
1. spread  the bandwidth equally among users at all times (if there;s 2 
ppl online than they should share the 2 mb hence having a faster 
connection than usual (Not Working)
2. Opening websites should have a little burst though downloading files 
should be shaped


III. What i've done:

##working 
Hours

acl work_hours time MTWHF 08:00-20:00
##subnet 
that's shaped

acl limitedto8 src 192.168.75.0/255.255.255.0
##Site's 
that should have a very low bandwidth

acl slowsites url_regex -i /etc/squid/slowsites.txt
##Destination 
that should not be shaped

acl mySubnet dstdom_regex 192.168.75
##Destinations 
that should never be shaped.

acl NoShape url_regex /etc/squid/NoShape.txt
##For 
users would temporary would be allowed to gain higher bandwidth

acl download src 192.168.75.138



delay_pools 2
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_access 1 allow  NoShape download
delay_class 2 3
###limiting 
every user in the subnet 192.168.75.0/24 and allowing NoShape and 
download acl to use the bandwidth freely

delay_access 2 allow limitedto8 !NoShape !download
delay_access 2 deny all
###Bandwidth 
for this delay pool is 512 KB where each user can download at a speed of 
10 KB

delay_parameters 2 512000/512000 -1/-1 1/512000

Re: [squid-users] Delay Pools advice?

[Chris Robertson]

Roland Roland wrote:

Hello,


i've set up squid about 2 month ago with poorly configured settings 
just to set things going ( i admit it wasnt the right thing to do 
thoguh time were of the essence )
at the moment i decided that it's about time to take a look at each 
rule i've made and fix it to the best i could. so i'm sending this 
email hoping someone would show any mistake i might have done :



I. Info:

1. Our working hours are from 8 am to 6 pm
2. One ISP link that's dedicated for squid/browsing has a 2 mb shared 
bandwidth with 8 GB of quota



II. Required:
1. spread  the bandwidth equally among users at all times (if there;s 
2 ppl online than they should share the 2 mb hence having a faster 
connection than usual (Not Working)
2. Opening websites should have a little burst though downloading 
files should be shaped


III. What i've done:

##working 
Hours

acl work_hours time MTWHF 08:00-20:00
##subnet 
that's shaped

acl limitedto8 src 192.168.75.0/255.255.255.0
##Site's 
that should have a very low bandwidth

acl slowsites url_regex -i /etc/squid/slowsites.txt
##Destination 
that should not be shaped

acl mySubnet dstdom_regex 192.168.75
##Destinations 
that should never be shaped.

acl NoShape url_regex /etc/squid/NoShape.txt
##For 
users would temporary would be allowed to gain higher bandwidth

acl download src 192.168.75.138



delay_pools 2
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_access 1 allow  NoShape download
delay_class 2 3
###limiting 
every user in the subnet 192.168.75.0/24 and allowing NoShape and 
download acl to use the bandwidth freely

delay_access 2 allow limitedto8 !NoShape !download
delay_access 2 deny all
###Bandwidth 
for this delay pool is 512 KB where each user can download at a speed 
of 10 KB

delay_parameters 2 512000/512000 -1/-1 1/512000



There's no need for two pools.  Drop the first (non-limiting) pool and 
traffic that isn't shaped by the remaining pool will not be affected.  
As for the second pool, since you only have a /24 network, there is no 
need to use a class 3 pool.  A class 2 will work just fine.


delay_pools 1
delay_class 1 2
#Bandwidth for this delay pool is 512 KB where each user can download at 
a speed of 10 KB

delay_parameters 1 512000/512000 1/512000
#limiting every user in the subnet 192.168.75.0/24 and allowing NoShape 
and download acl to use the bandwidth freely

delay access 1 allow limitedto8 !NoShape !download
delay_access 1 deny all

Chris

Re: [squid-users] Delay Pools advice?

[Amos Jeffries]

On Tue, 08 Dec 2009 17:13:56 +0200, Roland Roland
r_o_l_a_...@hotmail.com
wrote:
 Hello,
 
 
 i've set up squid about 2 month ago with poorly configured settings just

 to set things going ( i admit it wasnt the right thing to do thoguh time

 were of the essence )
 at the moment i decided that it's about time to take a look at each rule

 i've made and fix it to the best i could. so i'm sending this email 
 hoping someone would show any mistake i might have done :
 
 
 I. Info:
 
 1. Our working hours are from 8 am to 6 pm
 2. One ISP link that's dedicated for squid/browsing has a 2 mb shared 
 bandwidth with 8 GB of quota
 

Clarity please: 2 mb ==
 * 2 mb (milli bits?)
 * 2 Mb (mega bits)
 * 2 MB (mega bytes)?

I assume you omitted the 'ps' (per-second) units as well.

being specific of the 2 mb makes the difference between whether we tell
you the config for a 256KiBps or 2MiBps data cap.

 
 II. Required:
 1. spread  the bandwidth equally among users at all times (if there;s 2 
 ppl online than they should share the 2 mb hence having a faster 
 connection than usual (Not Working)
 2. Opening websites should have a little burst though downloading files 
 should be shaped
 
 III. What i've done:
 

##working
 
 Hours
 acl work_hours time MTWHF 08:00-20:00

NP: excludes saturday/sunday. You didn't mention days that in your policy
specs only times.


##subnet
 
 that's shaped
 acl limitedto8 src 192.168.75.0/255.255.255.0

acl limitedto8 src 192.168.75.0/24


##Site's
 
 that should have a very low bandwidth
 acl slowsites url_regex -i /etc/squid/slowsites.txt

##Destination
 
 that should not be shaped
 acl mySubnet dstdom_regex 192.168.75

You want the URLs containing raw-IPs http://192.168.75.*/ and
http://*.192.168.75/ to be available?

Or only the internal destinations?

  acl mySubnet dst 192.168.75.0/24


##Destinations
 
 that should never be shaped.
 acl NoShape url_regex /etc/squid/NoShape.txt

Hmm, surely this is full of domain names and IPs right?

With a little auditing of the file contents you should be able to convert
that to the faster:
  acl noShape dstdomain /etc/squid/NoShape.txt


##For
 
 users would temporary would be allowed to gain higher bandwidth
 acl download src 192.168.75.138
 
 
 
 delay_pools 2
 delay_class 1 2
 delay_parameters 1 -1/-1 -1/-1
 delay_access 1 allow  NoShape download

NP: pool 1 is not useful.
 * it wastes CPU cycles trying to account for all traffic going through
the unlimited users.
 * users which are excluded from the pool #2 below will not be limited
anyway.
 * you can safely drop pool #1


###limiting
 
 every user in the subnet 192.168.75.0/24 and allowing NoShape and 
 download acl to use the bandwidth freely
 delay_access 2 allow limitedto8 !NoShape !download
 delay_access 2 deny all

###Bandwidth
 
 for this delay pool is 512 KB where each user can download at a speed of

 10 KB
 delay_parameters 2 512000/512000 -1/-1 1/512000

NP: the above is a 512 KByte total bandwidth cap (base 1000).

For 512 KiB (base 1024) affecting total and users that should be:
  delay_class 2 2
  delay_parameters 2 524288/524288 10240/10240


To meet your spec of a 2 mb pipe that would be:

 delay_class 2 1

(2 Mbps)  delay_parameters 2 262144/262144
or
(2 MBps)  delay_parameters 2 2097152/2097152

with the same delay_access.

Amos