[squid-users] /var/run/squid gone on every reboot?!

[Heiler Bemerguy]

3.5. Everytime I must recreate this. What I'm missing? 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid caching bad objects

[Alex Rousskov]

On 07/06/2017 10:06 PM, Razor Cross wrote:
> We have disabled eCAP support as part of squid compilation . Is it
> related to this issue ?

No: Assuming my guess about the underlying problem is correct, eCAP is
irrelevant here.

Alex.


> On Thu, Jul 6, 2017 at 10:09 PM, Alex Rousskov wrote:
> 
> On 07/06/2017 07:01 PM, Amos Jeffries wrote:
> 
> > AFAIK Squid currently should not be caching these objects at all. Or
> > when it does use the disk cache as a temporary storage (eg for very
> > large objects) marking them for immediate discard when the abort
> > happens.
> 
> Yes, and the corresponding bug report, with a solution blueprint is at
> http://bugs.squid-cache.org/show_bug.cgi?id=4735
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid caching bad objects

[Razor Cross]

Thanks Alex and Amos for your inputs.

A basic question -
We have disabled eCAP support as part of squid compilation . Is it related
to this issue ?

On Thu, Jul 6, 2017 at 10:09 PM, Alex Rousskov <
rouss...@measurement-factory.com> wrote:

> On 07/06/2017 07:01 PM, Amos Jeffries wrote:
>
> > AFAIK Squid currently should not be caching these objects at all. Or
> > when it does use the disk cache as a temporary storage (eg for very
> > large objects) marking them for immediate discard when the abort
> > happens.
>
> Yes, and the corresponding bug report, with a solution blueprint is at
> http://bugs.squid-cache.org/show_bug.cgi?id=4735
>
>
> > That is being tracked by
> > .
>
> While implementing partial object caching feature would also address
> this alleged bug, that implementation requires a lot more work and,
> depending on the specifics, the resulting feature may be optional so the
> bug fix may still be required anyway.
>
> Alex.
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] youtube videos and squid

[Sergei G]

Hi guys,

I have a very specific scenario in mind to use squid for.

I have 2 kids (2.5 and 4 years old) that are watching iPads and really
using available Comcast bandwidth.  It does not help that they sometimes
just leave those iPads running.

They tend to re-watch youtube videos (click on the same icon that they
liked before).  And that makes me think that squid could help me with
caching off youtube content.  Am I correct?

If not then I have no reason to bother you anymore :)

If squid could help me, then could you point me to a an example
configuration that would work?

As far as hardware I have 2 options:

1. I can install squid on a Raspberry PI 3, if package is readily
available.  that's my preferred solution.
2. I have an old server hardware with more power than RPI 3, but I don't
like to run it, because it is noisy.  It has FreeBSD 10 installed and I can
upgrade it to latest FreeBSD (11?) and isntall squid application that way.


Does squid run on RPI3?  FreeBSD?


Thank you
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid caching bad objects

[Alex Rousskov]

On 07/06/2017 07:01 PM, Amos Jeffries wrote:

> AFAIK Squid currently should not be caching these objects at all. Or
> when it does use the disk cache as a temporary storage (eg for very
> large objects) marking them for immediate discard when the abort
> happens. 

Yes, and the corresponding bug report, with a solution blueprint is at
http://bugs.squid-cache.org/show_bug.cgi?id=4735


> That is being tracked by
> .

While implementing partial object caching feature would also address
this alleged bug, that implementation requires a lot more work and,
depending on the specifics, the resulting feature may be optional so the
bug fix may still be required anyway.

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Huge amount of time_wait connections after upgrade from v2 to v3

[Ivan Larionov]

Hi. Sorry that I'm answering to the old thread. I was on vacation and
didn't have a chance to test the proposed solution.

Dieter, yes, I'm on the old CentOS 6 based OS (Amazon Linux) but with a new
kernel 4.9.27.

Amos, thank you for the suggestions about configure flags and squid config
options, I fixed all issues you pointed to.

Unfortunately following workarounds didn't help:

* client_idle_pconn_timeout 30 seconds
* half_closed_clients on
* client_persistent_connections off
* server_persistent_connections off

However I assumed that this is a bug and that I can find older version
which worked fine. I started testing from 3.1.x all the way to 3.5.26 and
this is what I found:

* All versions until 3.5.21 work fine. There no issues with huge amount of
TIME_WAIT connections under load.
* 3.5.20 is the latest stable version.
* 3.5.21 is the first broken version.
* 3.5.23, 3.5.25, 3.5.26 are broken as well.

This effectively means that bug is somewhere in between 3.5.20 and 3.5.21.

I hope this helps and I hope you'll be able to find an issue. If you can
create a bug report based on this information and post it here it would be
awesome.

Thank you.

On Wed, Jun 7, 2017 at 4:34 AM, Amos Jeffries  wrote:

> On 07/06/17 12:13, Ivan Larionov wrote:
>
>> Hi!
>>
>> We recently updated from squid v2 to v3 and now see huge increase in
>> connections in TIME_WAIT state on our squid servers (verified that this is
>> clients connections).
>>
>
> The biggest change between 2.7 and 3.5 in this area is that 2.7 was
> HTTP/1.0 which closed TCP connections after each request by default, and
> 3.5 is HTTP/1.1 which does not. So connections are more likely to persist
> until they hit some TCP timeout then enter the slow TIME_WAIT process.
>
> There were also some other bugs identified in older 3.5 releases which
> increased the TIME_WAIT specifically. I thought those were almost all fixed
> by now, but YMMV whether you hit the remaining issues.
>  A workaround it to set  c/config/client_idle_pconn_timeout/> to a shorter value than the default
> 2min. eg you might want it to be 30sec or so.
>
>
>
>
>> See versions and amount of such connections under the same load with the
>> same configs (except some incompatible stuff):
>>
>> squid 2.7.STABLE9
>>
>> configure options:  '--program-prefix=' '--prefix=/usr'
>> '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin'
>> '--sysconfdir=/etc' '--includedir=/usr/include' '--libdir=/usr/lib'
>> '--libexecdir=/usr/libexec' '--sharedstatedir=/usr/com'
>> '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr'
>> '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--localstatedir=/var'
>> '--datadir=/usr/share' '--sysconfdir=/etc/squid' '--enable-epoll'
>> '--enable-removal-policies=heap,lru' '--enable-storeio=aufs'
>> '--enable-delay-pools' '--with-pthreads' '--enable-cache-digests'
>> '--enable-useragent-log' '--enable-referer-log' '--with-large-files'
>> '--with-maxfd=16384' '--enable-err-languages=English'
>>
>> # netstat -tn | grep TIME_WAIT | grep 3128 | wc -l
>> 95
>>
>> squid 3.5.25
>>
>> configure options:  '--program-prefix=' '--prefix=/usr'
>> '--exec-prefix=/usr' '--bindir=/usr/sbin' '--sbindir=/usr/sbin'
>> '--sysconfdir=/etc/squid' '--libdir=/usr/lib' '--libexecdir=/usr/lib/squid'
>> '--includedir=/usr/include' '--datadir=/usr/share'
>> '--sharedstatedir=/usr/com' '--localstatedir=/var'
>> '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-epoll'
>> '--enable-removal-policies=heap,lru' '--enable-storeio=aufs'
>> '--enable-delay-pools' '--with-pthreads' '--enable-cache-digests'
>> '--enable-useragent-log' '--enable-referer-log' '--with-large-files'
>> '--with-maxfd=16384' '--enable-err-languages=English' '--enable-htcp'
>>
>
> FYI, these options are not doing anything for Squid-3:
>   '--enable-useragent-log' '--enable-referer-log'
> '--enable-err-languages=English'
>
>
>
>> # netstat -tn | grep TIME_WAIT | grep 3128 | wc -l
>> 11277
>>
>> Config:
>>
>> http_port 0.0.0.0:3128 
>>
>> acl localnet src 10.0.0.0/8  # RFC1918 possible
>> internal network
>> acl localnet src 172.16.0.0/12   # RFC1918
>> possible internal network
>> acl localnet src 192.168.0.0/16  # RFC1918
>> possible internal network
>>
>> acl localnet src fc00::/7   # RFC 4193 local private network range
>> acl localnet src fe80::/10  # RFC 4291 link-local (directly plugged)
>> machines
>>
>> acl SSL_ports port 443
>>
>> acl Safe_ports port 80  # http
>> acl Safe_ports port 21  # ftp
>> acl Safe_ports port 443 # https
>> acl Safe_ports port 70  # gopher
>> acl Safe_ports port 210 # wais
>> acl Safe_ports port 280 # http-mgmt
>> acl Safe_ports port 488 # gss-http
>> acl Safe_ports port 591 # filemaker
>> acl Safe_ports port 777 # multiling http
>> acl Safe_ports port 1025-65

Re: [squid-users] Squid caching bad objects

[Amos Jeffries]

On 07/07/17 03:41, Razor Cross wrote:

Hi Amos,
Do you have any insights on the below issue? Is it fixed on latest squid 
release?  Your inputs would be really helpful




If that means you consider me Squid "owner" or official spokesperson. 
Well, I am one such, but so is Alex. If it helps 
 lists the people most central to 
Squid Project and what their roles are. I defer to Alex most of the time 
when it comes to the internal cache/store operations since his Factory 
team have done the most recent and extensive redesign work for that code.



As to the problem;

AFAIK Squid currently should not be caching these objects at all. Or 
when it does use the disk cache as a temporary storage (eg for very 
large objects) marking them for immediate discard when the abort 
happens. That is being tracked by 
.


If that situation has changed and these objects are now being stored 
incorrectly, that would be a new bug which is both a regression on the 
old safe cache behaviour and blocking the 424 bug. I second Alexs' comments.



Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] The best way to start | stop | reload | status

[erdosain9]

Ok, thank you all!



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/The-best-way-to-start-stop-reload-status-tp4682998p4683005.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid caching bad objects

[Razor Cross]

Hi Amos,
Do you have any insights on the below issue? Is it fixed on latest squid
release?  Your inputs would be really helpful




On Tue, Jun 27, 2017 at 12:04 PM, Razor Cross 
wrote:

>
>
> On Tue, Jun 27, 2017 at 11:34 AM, Alex Rousskov <
> rouss...@measurement-factory.com> wrote:
>
>> On 06/27/2017 10:11 AM, Razor Cross wrote:
>> > On Mon, Jun 26, 2017 at 12:06 PM, Alex Rousskov wrote:
>>
>> > >I suspect that the COMPLETE_NONPERSISTENT_MSG case in
>> > >HttpStateData::processReplyBody() should be changed to call
>> > >StoreEntry::lengthWentBad("missing last-chunk") when lastChunk is
>> false
>> > >and HttpStateData::flags.chunked is true.
>>
>> >   We are able to reproduce the issue . If server socket is closed
>> > after sending first chunk of data, squid is caching the partial object
>> > even though it did not receive the remaining chunks.
>>
>> If you are not going to fix this yourself, please consider filing a bug
>> report, citing this email thread.
>>
>>
>> > I feel it has to
>> > make sure that lastchunk has received before caching the data.
>>
>> That is impossible in general (the response may be too big to buffer)
>> but is also unnecessary in most cases (because Squid can stop caching
>> and delete the being-cached object in-flight). My paragraph quoted above
>> has the blueprint for a possible fix.
>>
>> Thanks for your inputs..
> I just want to hear from squid official forum/owner whether it has fixed
> in any recent squid releases so that we can upgrade/patch the fix.
>
>

> - Cross
>
>>
>>
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] open failed to shm_open(/squid-ssl_session_cache.shm)

[Amos Jeffries]

On 06/07/17 02:30, erdosain9 wrote:

Hi.
Why you say "This is because you use two commands"?
i use systemctl reload squid. or systemctl stop, start



You appear to be posting through Nabble and responding to someone who is 
not a member of the Squid mailing lists. If you do that, please at least 
quote the details you are responding to.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] The best way to start | stop | reload | status

[Amos Jeffries]

On 07/07/17 01:08, erdosain9 wrote:

Hi.
mmm... im having a doubt.
I usually use Systemctl for start, stop, reload, and status; but sometimes i
heard that it was not the best way to do these actions.
Way? I heard something wrong?


Because systemctl is a daemon manager. The "squid" binary is also a 
daemon manager. Having a daemon manager manage another daemon manager 
leads to some very odd behaviours.


People are finding out the hard way what those problems are since 
systemd merged daemon management into the init system on many OS.


NP: In Squid-4 we have completely redesigned the way Squid processes 
integrate so that it can be run under a different daemon manager.




And if not the best way, what would it be?

1) squid -z
2) squid

???



The squid -k options. See 



NP: -z is to format or repair the cache_dir storage area(s).



And from there, how i can stop and reload, status??


"squid -k check" for status, it will exit with an error message if Squid 
is not running, and succeed if one is running.


The others should be obvious from the above documentation manual.



And, if this is the best way, how do I start Squid automatically when the
system boots?


If you are using a packaged binary the vendor should have setup 
appropriate integration to be installed already.


Otherwise, the tools/ directory in Squid source bundles contain whatever 
is needed for integration into the various init systems that version of 
Squid can be used with. For Squid-3 that is any init system that 
supports rc.d scripting.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Dstdomain "there are more than 100 regular expressions"

[Dijxie]

W dniu 06.07.2017 o 15:22, erdosain9 pisze:

Hi. I have this in my cache.log

ad_block, is a list for block publicity.

there is a best way to do that??

2017/07/06 10:35:49| /etc/squid/squid.conf line 55: acl ads dstdom_regex
"/etc/squid/listas/ad_block.lst"
2017/07/06 10:35:49| WARNING: there are more than 100 regular expressions.
Consider using less REs or use rules without expressions like 'dstdomain'.


Thanks to all.



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Dstdomain-there-are-more-than-100-regular-expressions-tp4682999.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


I think best answer will be: ignore that if your squid is running fine, 
or look for external filtering solution like Eliezer's squidblocker or 
squidguard (slow and somehow obsolete, as for me)


Here is good explenation by Amos:
http://squid-web-proxy-cache.1019090.n4.nabble.com/WARNING-there-are-more-than-100-regular-expressions-tp4668529p4668573.html

--
Greets, Dijx.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] The best way to start | stop | reload | status

[Dijxie]

W dniu 06.07.2017 o 15:08, erdosain9 pisze:

Hi.
mmm... im having a doubt.
I usually use Systemctl for start, stop, reload, and status; but sometimes i
heard that it was not the best way to do these actions.
Way? I heard something wrong?
And if not the best way, what would it be?

1) squid -z
2) squid

???

And from there, how i can stop and reload, status??
And, if this is the best way, how do I start Squid automatically when the
system boots?

(Or is it really not relevant?)

Thanks to all!



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/The-best-way-to-start-stop-reload-status-tp4682998.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


squid -k check|reconfigure etc for interaction with squid process. 
Reconfigure is for reload config files, external ACL files, as well as 
for i.e. refreshing cached group members in case if external ACL used.
Usually, the best way to start/stop/restart squid is to use your init, 
i.e. systemctl stop squid, service squid stop, /etc/init.d/squid stop. 
Depending of distro, squid usually has systemd-unit-file or upstart/init 
script. For reload configuration, personally I always use squid -k 
reconfigure.


But if you want to/need to, you do not have to run squid as a 
daemon/service, you can start it via cron or whatever you like. Init is 
usually the best way, since it handles things like order, dependiencies, 
runlevels etc. - that is what is it for.


--
Greets, Dijx.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Dstdomain "there are more than 100 regular expressions"

[erdosain9]

Hi. I have this in my cache.log

ad_block, is a list for block publicity. 

there is a best way to do that??

2017/07/06 10:35:49| /etc/squid/squid.conf line 55: acl ads dstdom_regex
"/etc/squid/listas/ad_block.lst"
2017/07/06 10:35:49| WARNING: there are more than 100 regular expressions.
Consider using less REs or use rules without expressions like 'dstdomain'.


Thanks to all.



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Dstdomain-there-are-more-than-100-regular-expressions-tp4682999.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] The best way to start | stop | reload | status

[erdosain9]

Hi.
mmm... im having a doubt.
I usually use Systemctl for start, stop, reload, and status; but sometimes i
heard that it was not the best way to do these actions.
Way? I heard something wrong?
And if not the best way, what would it be?

1) squid -z
2) squid

???

And from there, how i can stop and reload, status??
And, if this is the best way, how do I start Squid automatically when the
system boots?

(Or is it really not relevant?)

Thanks to all!



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/The-best-way-to-start-stop-reload-status-tp4682998.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users