Re: [squid-users] 3.5.20 run out of my memory.

[Sticher, Jascha]

> Von: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] Im
> Auftrag von Amos Jeffries
> Gesendet: Freitag, 9. Februar 2018 08:37
>
> On 09/02/18 20:30, Sticher, Jascha wrote:
> > Hi,
> >
> >> KiB Mem:   4037016 total,  3729152 used,   307864 free,   120508 buffers
> >> KiB Swap:  8511484 total,0 used,  8511484 free.  2213580 cached Mem
> >
> > this is normal behaviour in Linux - everything that's once read from disk is
> cached in RAM, as long as there is free memory.
> > If the RAM is needed in another way, the cache in memory will be reduced.
> See also: https://www.linuxatemyram.com/
> >
> >
> > Kind regards,
> >
> > Jascha Sticher
> >
> 
> Nice way to say it. Do you mind If I quote you for this in the Squid FAQ
> pages?
> 
> Amos

I don't mind - go ahead.

I'm glad to help!

Kind regards,

Jascha Sticher
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.20 run out of my memory.

[Amos Jeffries]

On 09/02/18 20:30, Sticher, Jascha wrote:
> Hi,
> 
>> KiB Mem:   4037016 total,  3729152 used,   307864 free,   120508 buffers
>> KiB Swap:  8511484 total,0 used,  8511484 free.  2213580 cached Mem
> 
> this is normal behaviour in Linux - everything that's once read from disk is 
> cached in RAM, as long as there is free memory.
> If the RAM is needed in another way, the cache in memory will be reduced. See 
> also: https://www.linuxatemyram.com/
> 
> 
> Kind regards,
> 
> Jascha Sticher
> 

Nice way to say it. Do you mind If I quote you for this in the Squid FAQ
pages?

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.20 run out of my memory.

[Amos Jeffries]

On 09/02/18 14:12, minh hưng đỗ hoàng wrote:
> Thanks alot for your help,
> 
> > https_port 3130 tproxy ssl-bump generate-host-certificates=on
> > dynamic_cert_mem_cache_size=4MB
> > cert=/etc/squid/ssl/e1f19c0494badc8dc14e8c4c56a8b97a.dyn
> 
> Please add sslflags=NO_DEFAULT_CA to the above config line. That should
> reduce the memory usage a lot.
> 
>  
> I have tried this command, but my squid still used alot of my memory for
> cache .
> 
> KiB Mem:   4037016 total,  3729152 used,   307864 free,   120508 buffers
> KiB Swap:  8511484 total,    0 used,  8511484 free.  2213580 cached Mem
> 
> Squid only use about 1.2Gb dram to run, but use alot of memory for
> cached ( 2213580 cached Mem )
> What was cached by my squid with my squid.conf ? Can i reduce or set
> life-time for this cache ?
> 

"cached Mem" is not Squid memory. It is Operating System memory and
nothing to worry about so long as the used and free values are reasonable.

You have "0 used" in Swap, and lots of "free" in main memory. Those are
the most important things.

These pages may be of help understanding what all the numbers mean:





Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.20 run out of my memory.

[Sticher, Jascha]

Hi,

> KiB Mem:   4037016 total,  3729152 used,   307864 free,   120508 buffers
> KiB Swap:  8511484 total,0 used,  8511484 free.  2213580 cached Mem

this is normal behaviour in Linux - everything that's once read from disk is 
cached in RAM, as long as there is free memory.
If the RAM is needed in another way, the cache in memory will be reduced. See 
also: https://www.linuxatemyram.com/


Kind regards,

Jascha Sticher


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.20 run out of my memory.

[minh hưng đỗ hoàng]

Thanks alot for your help,

> https_port 3130 tproxy ssl-bump generate-host-certificates=on
> > dynamic_cert_mem_cache_size=4MB
> > cert=/etc/squid/ssl/e1f19c0494badc8dc14e8c4c56a8b97a.dyn
>
> Please add sslflags=NO_DEFAULT_CA to the above config line. That should
> reduce the memory usage a lot.
>
>
I have tried this command, but my squid still used alot of my memory for
cache .

KiB Mem:   4037016 total,  3729152 used,   307864 free,   120508 buffers
KiB Swap:  8511484 total,0 used,  8511484 free.  2213580 cached Mem

Squid only use about 1.2Gb dram to run, but use alot of memory for cached (
2213580 cached Mem )
What was cached by my squid with my squid.conf ? Can i reduce or set
life-time for this cache ?

-- 
Thanks & Best Regards,
--
Đỗ Hoàng Minh Hưng
Gmail : hoangminh...@gmail.com
SĐT : 01234454115
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Macros

[Alex Rousskov]

On February 8, 2018 13:27:06 Alfredo Daniel Rezinovsky  wrote:


I know there is a macro ${service_name}

I like to know if there are other


See squid.conf.documented. Modern versions have a  section devoted to 
macros, before all the directives are described.




or there's a way to parse environment
variables in squid.conf.


No, IIRC. However, you can preprocess your configuration and/or generate 
include files.



HTH,

Alex.


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Macros

[Yuri]

Indeed :)

You can cover this by write good documentation and share it ;) This is
OpenSource ;) Nothing exists - except you will create by yourself ;)


09.02.2018 01:34, Alfredo Daniel Rezinovsky пишет:
>
> I tried searching in the code and still couldn't find it. But
> Challenge accepted.
>
>
> On 08/02/18 16:28, Yuri wrote:
>> This is OpenSource :) There is no documentation :) (As they say - read
>> the code to get documentation ;))
>>
>>
>> 09.02.2018 01:26, Alfredo Daniel Rezinovsky пишет:
>>> I know there is a macro ${service_name}
>>>
>>> I like to know if there are other or there's a way to parse
>>> environment variables in squid.conf.
>>>
>>> I didn't find this in the on line documentation
>>>
>>> ___
>>> squid-users mailing list
>>> squid-users@lists.squid-cache.org
>>> http://lists.squid-cache.org/listinfo/squid-users
>>
>>
>> ___
>> squid-users mailing list
>> squid-users@lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-- 
*
* C++20 : Bug to the future *
*



signature.asc
Description: OpenPGP digital signature
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Macros

[Alfredo Daniel Rezinovsky]

I tried searching in the code and still couldn't find it. But 
Challenge accepted.



On 08/02/18 16:28, Yuri wrote:

This is OpenSource :) There is no documentation :) (As they say - read
the code to get documentation ;))


09.02.2018 01:26, Alfredo Daniel Rezinovsky пишет:

I know there is a macro ${service_name}

I like to know if there are other or there's a way to parse
environment variables in squid.conf.

I didn't find this in the on line documentation

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Macros

[Yuri]

This is OpenSource :) There is no documentation :) (As they say - read
the code to get documentation ;))


09.02.2018 01:26, Alfredo Daniel Rezinovsky пишет:
> I know there is a macro ${service_name}
>
> I like to know if there are other or there's a way to parse
> environment variables in squid.conf.
>
> I didn't find this in the on line documentation
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-- 
*
* C++20 : Bug to the future *
*




signature.asc
Description: OpenPGP digital signature
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Macros

[Alfredo Daniel Rezinovsky]

I know there is a macro ${service_name}

I like to know if there are other or there's a way to parse environment 
variables in squid.conf.


I didn't find this in the on line documentation

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Certificate Authority with SSLBump

[Yuri]

1. Using mozilla CA bundle instead of system (if exists) for squid.

2. Update mozilla CA bundle by script by cron on regular basis.

3. Have own manually maintained custom add_certs.pem list which combines
with step 2 during updates.

Thats all, folks.


08.02.2018 23:33, FredB пишет:
> Hi All,
>
> In practise how you maintain the CA files? I'm testing SSLBump with Debian 
> Jessie the package ca-certificates provides many certificates but less than 
> the latest Firefox Browser.
> How do you manage to keep all that in check? When a CA is missing you add the 
> pem in you system config or exclude the website from SSLBump?  
>
> EG: From my test https://wiki.squid-cache.org seems unknown (71) Protocol 
> error (TLS code: X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY)
> SSL Certficate error: certificate issuer (CA) not known: /C=US/O=Let's 
> Encrypt/CN=Let's Encrypt Authority X3
>
> Thanks
>
> Regards
> Fred
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-- 
*
* C++20 : Bug to the future *
*




signature.asc
Description: OpenPGP digital signature
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.x or 4.x acting as a transparent http proxy (NOT https)

[Amos Jeffries]

On 08/02/18 10:11, setuid wrote:
> I'll start with the pointedly easy stuff: Squid > 2.6 (tested 3.4, 3.5,
> 4.0 on Ubuntu Xenial, Debian Jessie, FreeSBD 11.1 using iptables, pf,
> ipf, ipfilter) does not work at all, when configured as a transparent
> proxy. Full stop.
> 
> I went through hundreds of posts on dozens of forums, blogs and other
> resources, tried dozens and dozens of configurations suggested by those
> posts, tried all 3 firewall options on BSD, tried two versions of Ubuntu
> and the various versions of Squid from the apt repos, as well as those
> in BSD's ports.
> 
> All of them, 100%, fail in _exactly_ the same way, no matter what my
> configuration was set to. That result, is that _every single http
> request I make_ when Squid is configured as a transparent proxy, results
> in the following response being logged:
> 
> ==
>   07/Feb/2018:15:10:59 -0500.213  0 192.168.1.1 TAG_NONE/400 3583 GET
> / - HIER_NONE/- text/html ("-" "-")
> ==
> 
> When I point a client directly at the proxy, using a browser, curl or
> anything else, I see:
> 
> ==
>   07/Feb/2018:15:12:56 -0500.875 82 192.168.1.1 TCP_MISS/302 333 HEAD
> http://www.java.com/ - HIER_DIRECT/www.java.com - ("-" "curl/7.47.0")
> ==
> 
> These were the same exact request against the same exact Squid instance.

Lets start with the obvious then. HEAD is Not the same as GET. These are
*different* requests. Therefore something major is wrong with your
understanding of the situation.


> If I use Squid 3.5 on Ubuntu or 3.5 and 4.0 on BSD, the logged entry is
> _identical_ for every single http request I make, regardless of origin.
> 
> My Squid configuration is 100% default, identical to the generic config,
> with the exception of the following lines:
> 
> ==
> http_port 3128
> http_port 3129 intercept
> tcp_outgoing_address 192.168.1.25
> debug_options ALL,9
> ==
> 
> I've tried all of the obvious links, blogs and resources I could Google
> up, and 100% of them fail to function as described. Most people I've
> seen on the forums who attempt to get this working, throw their hands up
> in defeat and end up configuring the proxy directly on every client that
> needs it.
> 
> My current environment looks like this:
> 
> [ wireless router: 10.0.1.1 on LAN side, 192.168.1.1 on WAN side ]
> 
> That router has a firewall script on it that says:
> 
> ==
> #!/bin/sh
> PROXY_IP=192.168.2.25
> PROXY_PORT=3128
> LAN_IP=$(nvram get lan_ipaddr)
> LAN_NET=$LAN_IP/$(nvram get lan_netmask)
> 
> iptables -t nat -A PREROUTING -i br0 -s $LAN_NET -d $LAN_NET -p tcp
> --dport 80 -j ACCEPT
> iptables -t nat -A PREROUTING -i br0 -s ! $PROXY_IP -p tcp --dport 80 -j
> DNAT --to $PROXY_IP:$PROXY_PORT
> 
> iptables -t nat -I POSTROUTING -o br0 -s $LAN_NET -d $PROXY_IP -p tcp -j
> SNAT --to $LAN_IP
> iptables -I FORWARD -i br0 -o br0 -s $LAN_NET -d $PROXY_IP -p tcp
> --dport $PROXY_PORT -j ACCEPT
> ==
> 
> This takes every packet that hits the router on :80, and sends it to my
> Squid server on .25, which mangles it and sends it back to 192.168.1.1
> (router), and onward back to client who requested it.
> 

No. This takes HTTP (port 80 syntax) and sends it to a remote proxy
expecting explicit-proxy syntax.


> When I was using 2.6 (without large_file support), I was using this same
> exact configuration, but http_port was set to 'accel', and I didn't need
> _any_ NAT/routing rules on the squid side at all. It all "Just Worked(tm)".


It also "Just Worked" for anyone attacking your network via
CVE-2009-0801 methods. And would provide them with an effective
invisibility cloak while doing so (original IP:ports destroyed by the NAT).
 Not exactly desirable behaviour.



> 
> Now I need to jump through hoops to do pf incantations of rdr/direct-to
> (but direct-to and direct-reply aren't supported on FreeBSD's pf, only
> OpenBSD's pf supports that syntax), and iptables PREROUTING and
> POSTROUTING mojo (also fails).
> 

You need simply *route* traffic properly in the way the Internet was
designed to work. Instead of abusing NAT.

When that is done properly the NAT happens *only* as the final step to
get the traffic into the Squid process by the machine/device/VM Squid is
running on.


> Here's a list of some of the resources I've tried, with 100% failure in
> every case. There are dozens more that I've lost in my browser history now.
> 
> * https://wiki.squid-cache.org/ConfigExamples/Intercept/Ipfw
> *
> https://wiki.squid-cache.org/SquidFaq/InterceptionProxy#Interception_Caching_packet_redirection_for_OpenBSD_PF

Hmm. I see that page is updated and missing quote a few things. Thanks
for bringing this to attention.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users