Re: [squid-users] transparent proxy https and self signed certificate error
On 05/10/14 18:44, Amos Jeffries wrote: > PS. Google with Chrome appear these days to be the champions of > unbreakable TLS, their software is continually being updated to > use/invent new TLS features that close loopholes in TLS design which > allow ssl-bump to take place. What worked last month has no guarantee > of working today, same again next month. That can't be right? I mean, sslbump doesn't rely on any "bugs" - it is simply a CA and so any browser that thinks it's a CA should be happy going to any https website using appropriate certs signed by that CA? I know Chrome has *cert pinning* (ie they hardwired the CAs that Google knows *.google.com uses into Chrome), but that isn't a "loophole". sslbump seems to work as well as can be expected. But pinning also appears to be growing in stature (Firefox now does it too), so there are less and less sites that sslbump can work on. I wanted to use sslbump so that we could run AV and filtering on https links, but pinning means our "exclude list" of https sites is getting larger and larger - and includes Cloud providers the badguys are housing their malware on - which means our AV still can't catch it :-( -- Cheers Jason Haar Corporate Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] transparent proxy https and self signed certificate error
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 5/10/2014 1:29 p.m., Robert Watson wrote: > using squid 3.4.8, compiled from source with ./configure flags > --enable-icap-client --enable-ssl --enable-ssl-crtd configured > iptables for transparent proxy (redirect 80 to 3128) and everything > works fine > > configured iptables for transparent proxy (redirect 443 to 3127) > but can't get transparent proxy for https to work my squid.conf > ... # Squid https port https_port 3127 intercept ssl-bump > generate-host-certificates=on dynamic_cert_mem_cache_size=4MB > cert=/etc/squid/ssl_cert/XXX.pem acl broken_sites dstdomain > .example.com ssl_bump none localhost ssl_bump none broken_sites > ssl_bump server-first all sslproxy_cert_error allow all > sslproxy_flags DONT_VERIFY_PEER sslcrtd_program > /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB > sslcrtd_children 32 startup=5 idle=1 > > when visiting google (or any other https site) chrome complains > NET::ERR_CERT_AUTHORITY_INVALID I tried using internet explorer as > admin and imported the self signed certificate but that hasn't > helped > > can anyone please with how to debug this thanks, Robert To debug you will need a packet capture with full packet bodies (tcpdump -s 0) of the TCP connection between browser and Squid, and the connection between Squid and server. Wireshark should be able to decrypt the TLS/SSL handshakes to see what differences or corruption is happening. FYI: When testing be sure to clear/empty the ssl_crtd database if any changes are made to CA keys. PS. Google with Chrome appear these days to be the champions of unbreakable TLS, their software is continually being updated to use/invent new TLS features that close loopholes in TLS design which allow ssl-bump to take place. What worked last month has no guarantee of working today, same again next month. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUMNrEAAoJELJo5wb/XPRj7QAIAMVZ5SOc+X8vWlMdbgyNhNJR k//TmLRMdwZ1qxFBHTF3t+I7JVua2b+DDp0fU6Ubq6WvoARNBQGPQdI0XfOtrnLQ 3lsBCkU8NZuXt2LeoKG6eNPaNyuhom7HeFzmwELgM4SuASxbO4mpBxET8Tg1XYwQ VdSruqwx0hwhb5g4yeXWEIflkILc1A5cTAAbNGXIHpWbqMmwvnav5KWCfDhesHEU CdxuyZJnUZwv/uRYSaiiYebUECTS/Zl8JkGvCXe5zheLwT2Wcor3urUXIK3gPToz dy8FJ7lRGSSIJNkiQO4iNwI28vYkJHP2u3yFMFOdu4r/jN7WRgaY2LSpaQF+pqc= =teuE -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] RPM Packages
This question is probably specifically for Eliezer. My question is this, On the RPM repository at http://www1.ngtech.co.il/rpm/ There is an RPM package for version 3.4.5 for Oracle Linux 6. I installed this a few months ago when I was preparing to go live with a new Squid instance and now after all of the testing is complete, I'd like to update to the latest RPM which is only for CentOS 6. Eliezer, can you tell me how much different is the CentOS 6 build from the OEL 6 please? Thanks and Regards John ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Best OS for latest squid
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 5/10/2014 4:49 p.m., Douglas Davenport wrote: > I'm starting from scratch with an AWS based squid setup, I would > like to be able stay up to date with the latest squid releases to > have all the sslbump fixes. Can someone suggest what is best to > use, Centos 6, Ubuntu 14 or another distro? I see a lot of the > binary releases lag behind, does squid build easily on a particular > platform? Sorry if this question has been covered, I searched but > only found discussion about hardware specs. Thanks! > We currently do regular integration testing with successful results on Debian Sid, Ubuntu Precise & Saucy & Trusty, CentOS 6 & 7, FreeBSD 9.1 & 10, OpenBSD 5.4, Fedora 19. Using GCC, clang, and ICC compilers where available. Other OS usually have good results as well with the exception of Windows and MacOS where SMP functionality used by Squid is missing or broken. FWIW: Old OS releases with older compilers generally work best with old Squid releases with matching level of compiler support. Not that such a situation is desirabe for use in todays Internet. NOTE: For tracking latest Squid in future you will want GCC 4.9+ or clang 3.4+ compilers. The Squid-3.4+ series build best performance optimizations with them and 3.6 series about to begin development will probably require C++11 at some point soon. Amos PS. personally I am a Debian "fanboi", with Ubuntu a close second. That comes down to package management tools and their multi-arch support though. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUMMmyAAoJELJo5wb/XPRjl6YIAKj+8puStnRkYgi9CkIxV8Kg xJbnVTuD2yKKbNr090KcDJ6IbJ1dxtZlc7PIZVu3l/pkQ01lACSo75Q6wdgo2+FS mydorgDOB9aDY9CZj24xvBlyZCUoLq/3mGf4ttMKp+QJ6lU5pKwrElw7dkKdmNs5 djjF8/w8/tUEdWI+FBMJBsT9m+Wl1eLPZyFULU0hTRhAmMPV1Ms6sYXT8myjYvUR Ha20lSeQWNshSjHfKlkgX1BycpgSTOYvQ9BUK9U5beMHBWVvQUxI+A1WNcWINqXN f84ryIi+tyWHexweg3eKRWzh0+O/NVBtqIbXOJjGqVZxqzLuYjUoI4hNM23bAEc= =lbBe -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] Best OS for latest squid
I'm starting from scratch with an AWS based squid setup, I would like to be able stay up to date with the latest squid releases to have all the sslbump fixes. Can someone suggest what is best to use, Centos 6, Ubuntu 14 or another distro? I see a lot of the binary releases lag behind, does squid build easily on a particular platform? Sorry if this question has been covered, I searched but only found discussion about hardware specs. Thanks! ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] transparent proxy https and self signed certificate error
using squid 3.4.8, compiled from source with ./configure flags --enable-icap-client --enable-ssl --enable-ssl-crtd configured iptables for transparent proxy (redirect 80 to 3128) and everything works fine configured iptables for transparent proxy (redirect 443 to 3127) but can't get transparent proxy for https to work my squid.conf ... # Squid https port https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/XXX.pem acl broken_sites dstdomain .example.com ssl_bump none localhost ssl_bump none broken_sites ssl_bump server-first all sslproxy_cert_error allow all sslproxy_flags DONT_VERIFY_PEER sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB sslcrtd_children 32 startup=5 idle=1 when visiting google (or any other https site) chrome complains NET::ERR_CERT_AUTHORITY_INVALID I tried using internet explorer as admin and imported the self signed certificate but that hasn't helped can anyone please with how to debug this thanks, Robert ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
I suspect that the language setting is causing it.
If $LANG is different from "C" it may have a huge impact on the
performance of regular expression evaluation (not only in Squid but also
awk, sed etc.)
Try this:
LANG=C /etc/init.d/squid start
and see if Squid improves.
Marcus
> Hi,
>
> I have 2 squid boxes . Same version,OS and almost same config,hardware .
> Both have same problem also . normally cpu usage by squid is very high . I
> have tried this guide http://wiki.squid-cache.org/SquidFaq/SquidProfiling
> and found more than 85% of cpu usage is by re_search_internal symbol name
> .
> Please help me to solve this problem
>
> samples %image name app name symbol
> name
> 3297480 89.3408 libc-2.15.so squid3
> re_search_internal
>
>
> squid3 -v
> Squid Cache: Version 3.1.20
> configure options: '--build=x86_64-linux-gnu' '--prefix=/usr'
> '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
> '--infodir=${prefix}/share/info' '--sysconfdir=/etc'
> '--localstatedir=/var'
> '--libexecdir=${prefix}/lib/squid3' '--srcdir=.'
> '--disable-maintainer-mode'
> '--disable-dependency-tracking' '--disable-silent-rules'
> '--datadir=/usr/share/squid3' '--sysconfdir=/etc/squid3'
> '--mandir=/usr/share/man' '--with-cppunit-basedir=/usr' '--enable-inline'
> '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd'
> '--enable-removal-policies=lru,heap' '--enable-delay-pools'
> '--enable-cache-digests' '--enable-underscores' '--enable-icap-client'
> '--enable-follow-x-forwarded-for'
> '--enable-auth=basic,digest,ntlm,negotiate'
> '--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SASL,SMB,YP,DB,POP3,getpwnam,squid_radius_auth,multi-domain-NTLM'
> '--enable-ntlm-auth-helpers=smb_lm,'
> '--enable-digest-auth-helpers=ldap,password'
> '--enable-negotiate-auth-helpers=squid_kerb_auth'
> '--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group'
> '--enable-arp-acl' '--enable-esi' '--enable-zph-qos' '--enable-wccpv2'
> '--disable-translation' '--with-logdir=/var/log/squid3'
> '--with-pidfile=/var/run/squid3.pid' '--with-filedescriptors=65536'
> '--with-large-files' '--with-default-user=proxy'
> '--enable-linux-netfilter'
> 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector
> --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall'
> 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now'
> 'CPPFLAGS=-D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector
> --param=ssp-buffer-size=4 -Wformat -Werror=format-security'
> --with-squid=/build/buildd/squid3-3.1.20
>
>
>
> Linux 3.5.0-51-generic #76-Ubuntu SMP Thu May 15 21:19:10 UTC 2014 x86_64
> x86_64 x86_64 GNU/Linux
>
>
>
>
>
>
> --
> View this message in context:
> http://squid-web-proxy-cache.1019090.n4.nabble.com/High-cpu-usage-by-re-search-internal-tp4667550p4667655.html
> Sent from the Squid - Users mailing list archive at Nabble.com.
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] redirect all ports to squid
Hello, AFAIK it is possible to use redocks software ( http://darkk.net.ru/redsocks/ ) with squid. On Wed, Oct 1, 2014 at 1:49 AM, James Harper wrote: >> >> It's possible to redirect all ports to squid ? thru iptables ? >> For example port 25 smtp,143 imap, etc... >> Can squid handle that. In transparent mode. > > Yes. Kind of. You need: > . An appropriate rule in iptables nat table that ends with -j REDIRECT > --to-ports 3129 (or whatever port you are listening on for this traffic) > . A https_port definition in squid.conf on that port with ssl-bump and a > certificate (certificate doesn't get used unless you are doing actual https > but the syntax requires it) and a port name > . an acl attached to the name of the listeners myportname > . an ssl_bump none that matches the traffic you are interested in (all if you > aren't doing https interception) > > Now that you know you can do it, consider: > . I've asked this question on the list and the response from people who > really do know what they are talking about is that squid is not designed as a > general tcp proxy and there are probably other solutions that work better > . squid currently doesn't allow a sensible termination of the connection if > it isn't allowed, or if there is nothing listening at the other end. Your > smtp/pop3/imap/etc application won't like that. > . you have to do authentication out-of-band (eg ident), but that's the same > with transparent http anyway > > To do this really nicely, squid would need: > . a "tcp_port" instead of "http_port" designed for exactly this sort of thing > . a way to call out to the destination before accepting the connection so > that a 'connection refused' could be given if there is nothing listening > . a way to simply drop the connection if it doesn't succeed rather than the > default response squid gives > . a way to redirect traffic to a helper (eg SMTP/IMAP/POP3 filter to scan for > viruses, etc) (maybe this already exists already via other means?) > > So in short it works, but not as well as it could, and you might be better of > finding another solution. The main reason I was interested is that Squid > already has a very nice acl implementation, and there are already a number of > good log analysis tools for it. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
Thanks a lot . The latest file with your helps is here http://pastebin.com/8yytTWqA Any other tricks appreciated . -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/High-cpu-usage-by-re-search-internal-tp4667550p4667661.html Sent from the Squid - Users mailing list archive at Nabble.com. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 5/10/2014 4:12 a.m., Amos Jeffries wrote: > On 5/10/2014 3:34 a.m., Omid Kosari wrote: >> Mehdi Sarmadi wrote >>> Hey >>> >>> Alright. About refresh pattern you have a very excessive list >>> IMHO. I don't know about your hardware but generally for a >>> typical general purpose SMB server hardware, that's too much. >>> If you want to stick with it and can't reduce the list. Check, >>> how many core's you machine have. You should know squid >>> naturally sticks . A solution is to start multiple squid >>> instances, that way you can have squid(refresh_pattern) load >>> distributed on more than one CPU core, thus you'll get better >>> performance. >>> >>> Hope it helps Cheers > >> Thanks for the tip . It has a core i3 cpu so it has 4 cores . >> Unfortunately squid does not load fine across all cores >> specially in older versions like mine v 3.1 . Multi instance has >> its own complexity and headaches . i am trying to have clean >> design to be away from those problems . It was very useful if >> squid could do the refresh_pattern jobs by other cores . or some >> trick like that . > > > Here are some tips for your patterns: > > * all the (.+\.)? at the begining are useless complication. > Remove. > > * so are the .*?$ at the end of some patterns. This bit is also > probably doing more harm than good. Because the $ hints to regex > that it should scan right-to-left and the path and query portions > of the URL is the largest pieces to scan over. Remove. > > * the following two lines are redundant. The first will match > everything the second would have caught. Drop the second one. > > refresh_pattern -i \.htm 120 50% 10080 reload-into-ims > refresh_pattern -i \.html 120 50% 10080 reload-into-ims > > * the pattern below the comment "#Very aggressive 120 Days" > contains duplicates. > > > there are probably some smaller fixes, but those are the biggest I > can see without suggesting you drop those patterns entirely. > > You would do well from an upgrade of Squid. The later versions > have eliminated the need for ignore-no-cache, ignore-private, > ignore-auth (the latter two there do really, really bad things). > > I am also curios why you are ignoring must-revalidate? it is a > bandwidth reduction mechanism. > I'm finding some more the more I look. Things like (10\.10\.34\.34|peyvandha\.ir) being in the pattern set at the top means that the section of pattern later (10\.10\.34\.34|peyvandha\.ir| in the 120 day set is useless. Similar things in the "All files" set. * rar for example is listed twice, * (jp(e?g|e|2)|jpg matches jpg ... or jpg. Also when matching one character from multiple use square bracket syntax. Instead of things like ms(i|u|p) ... make it: ms[iup] Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUMBEEAAoJELJo5wb/XPRjjxoH/RcYxK+rRSwdUqbetPJOdtMf e0ApMl2N6pc+dD8HpY5ZcAB5rSOSEepinETVgpeOI33mDT7H4m7NlUvLStHBTzan SFWpoGWICUDiDwh0I/nMJTBByz8la073Rw7uKwl0AL2j3P9WtJoPyd4J8pSbs+BD 5YcQyJU0nWxlx05qbayl7Fe1R3wCWwA3xWtLTXnxEqnfQ+u69g4o+0XWSh7A0tC9 WVcl1BE4GaBXjSFKdkx1waR6ZIXeaEuI0GRSe57MCVk2e5P/nkLGwBvepw/dr7YN nHbeC4LI/vOKhS0yZVsa5z6y10Ov2vLTnAf2kXgpA2Ud7lZY72MAGYbhJNnWNiU= =y94q -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 5/10/2014 3:34 a.m., Omid Kosari wrote: > Mehdi Sarmadi wrote >> Hey >> >> Alright. About refresh pattern you have a very excessive list >> IMHO. I don't know about your hardware but generally for a >> typical general purpose SMB server hardware, that's too much. If >> you want to stick with it and can't reduce the list. Check, how >> many core's you machine have. You should know squid naturally >> sticks . A solution is to start multiple squid instances, that >> way you can have squid(refresh_pattern) load distributed on more >> than one CPU core, thus you'll get better performance. >> >> Hope it helps Cheers > > Thanks for the tip . It has a core i3 cpu so it has 4 cores . > Unfortunately squid does not load fine across all cores specially > in older versions like mine v 3.1 . Multi instance has its own > complexity and headaches . i am trying to have clean design to be > away from those problems . It was very useful if squid could do the > refresh_pattern jobs by other cores . or some trick like that . > Here are some tips for your patterns: * all the (.+\.)? at the begining are useless complication. Remove. * so are the .*?$ at the end of some patterns. This bit is also probably doing more harm than good. Because the $ hints to regex that it should scan right-to-left and the path and query portions of the URL is the largest pieces to scan over. Remove. * the following two lines are redundant. The first will match everything the second would have caught. Drop the second one. refresh_pattern -i \.htm 120 50% 10080 reload-into-ims refresh_pattern -i \.html 120 50% 10080 reload-into-ims * the pattern below the comment "#Very aggressive 120 Days" contains duplicates. there are probably some smaller fixes, but those are the biggest I can see without suggesting you drop those patterns entirely. You would do well from an upgrade of Squid. The later versions have eliminated the need for ignore-no-cache, ignore-private, ignore-auth (the latter two there do really, really bad things). I am also curios why you are ignoring must-revalidate? it is a bandwidth reduction mechanism. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUMA5PAAoJELJo5wb/XPRjkbwIAJtwCUvi1NYfg6dkggRth902 MG/x9mzhEVGE7Js7EyctD01Rgte/DUO3WIkbEBAkVK1t3Qlb1D8r60axtKBZzbd1 TsJ+kiJP4MB6C48yPwZlTe3/XwacgrF62UtocGNfJEHjm+9ZnyQL6EhAMJFJAHdM 6kJg35/AuThbEKrfTOG0u4BuMd0bnvNb1VHAOiRoaBAWDcW0N3uK3vrMM0XT5teA J5ht0GaLQoum/YesgwEf84emqD0Ukv4SZSj73w1fO4B4lEEC6aQ2IkOHycYp/fCh tLPeMekwZa4aQxLjgVJNEZHhcen91ttZPyLQUByZFhMShAJCJsRpQwpVpiHt/ug= =SEQ1 -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
Mehdi Sarmadi wrote > Hey > > Alright. About refresh pattern you have a very excessive list IMHO. I > don't know about your hardware but generally for a typical general purpose > SMB server hardware, that's too much. If you want to stick with it and > can't reduce the list. > Check, how many core's you machine have. You should know squid naturally > sticks . A solution is to start multiple squid instances, that way you can > have squid(refresh_pattern) load distributed on more than one CPU core, > thus you'll get better performance. > > Hope it helps > Cheers Thanks for the tip . It has a core i3 cpu so it has 4 cores . Unfortunately squid does not load fine across all cores specially in older versions like mine v 3.1 . Multi instance has its own complexity and headaches . i am trying to have clean design to be away from those problems . It was very useful if squid could do the refresh_pattern jobs by other cores . or some trick like that . -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/High-cpu-usage-by-re-search-internal-tp4667550p4667658.html Sent from the Squid - Users mailing list archive at Nabble.com. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
Thanks . I did it . When all refresh_pattern lines commented except following default ones refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 squid uses 40~60% cpu usage . Adding just following 2 lines increases more than 20% to squid cpu usage ! refresh_pattern -i (.+\.)?(microsoft|windowsupdate)\.com/.*?\.(cab|exe|dll|ms[i|u|f]|asf|wm[v|a]|dat|zip|iso|psf)$ 10080 100% 172800 ignore-no-store ignore-reload ignore-private ignore-must-revalidate override-expire override-lastmod refresh_pattern -i (.+\.)?apple\.com/.*?\.(pkg|dmg|tar|pkm|ipsw|exe)$ 10080 100% 172800 ignore-no-store ignore-reload ignore-private And here is my complete refresh_pattern http://pastebin.com/UtG7Gdv1 . I hope you provide some tips to solve it . Here is my cachemgr info http://pastebin.com/Ewcmwyvw Thanks in advance -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/High-cpu-usage-by-re-search-internal-tp4667550p4667657.html Sent from the Squid - Users mailing list archive at Nabble.com. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 5/10/2014 1:04 a.m., Omid Kosari wrote: > Hi, > > I have 2 squid boxes . Same version,OS and almost same > config,hardware . Both have same problem also . normally cpu usage > by squid is very high . I have tried this guide > http://wiki.squid-cache.org/SquidFaq/SquidProfiling and found more > than 85% of cpu usage is by re_search_internal symbol name . Please > help me to solve this problem Ah. regex bites again. re_search_internal is a regex pattern match being scanned on some data. Go through your squid.conf and minimize the amount of regex ACLs and other directives you are using. If you want some assistance with that post it here and us chickens can peck at it. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUL/WwAAoJELJo5wb/XPRjaPQIAM/nO1afdt8nUr8jXY+zlBuf qO0t8ODr+Bg58kbiV1s3J2yntFpQncEcttVPsN/h6tqVkdEfhjruU+IdnEU7caRi bQM0EPmAXs1FAfFWy+KfeIVFqSPKp0Bh8cV0wxgM39084002PQopQoZ/iHzEyuBg S1H8jF2/2iqL2MKDEN4e6HwfUuBy9unuJLikD5h1lNwriW31avEtoicnI/w/r8qd be0TmrOWeKkRe5gcIl5D/lFUghKWcgyDGePEwnM2LUrm/oOufEPD59THP/j8GNur 5/BvxmYzo/ojOl843HMbCtcaNzyoUKXO/1M1GfDclCN/7pIJ4t86mbVmk5d5R7M= =pIdC -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] High cpu usage by re_search_internal
Hi,
I have 2 squid boxes . Same version,OS and almost same config,hardware .
Both have same problem also . normally cpu usage by squid is very high . I
have tried this guide http://wiki.squid-cache.org/SquidFaq/SquidProfiling
and found more than 85% of cpu usage is by re_search_internal symbol name .
Please help me to solve this problem
samples %image name app name symbol
name
3297480 89.3408 libc-2.15.so squid3
re_search_internal
squid3 -v
Squid Cache: Version 3.1.20
configure options: '--build=x86_64-linux-gnu' '--prefix=/usr'
'--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
'--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var'
'--libexecdir=${prefix}/lib/squid3' '--srcdir=.' '--disable-maintainer-mode'
'--disable-dependency-tracking' '--disable-silent-rules'
'--datadir=/usr/share/squid3' '--sysconfdir=/etc/squid3'
'--mandir=/usr/share/man' '--with-cppunit-basedir=/usr' '--enable-inline'
'--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd'
'--enable-removal-policies=lru,heap' '--enable-delay-pools'
'--enable-cache-digests' '--enable-underscores' '--enable-icap-client'
'--enable-follow-x-forwarded-for'
'--enable-auth=basic,digest,ntlm,negotiate'
'--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SASL,SMB,YP,DB,POP3,getpwnam,squid_radius_auth,multi-domain-NTLM'
'--enable-ntlm-auth-helpers=smb_lm,'
'--enable-digest-auth-helpers=ldap,password'
'--enable-negotiate-auth-helpers=squid_kerb_auth'
'--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group'
'--enable-arp-acl' '--enable-esi' '--enable-zph-qos' '--enable-wccpv2'
'--disable-translation' '--with-logdir=/var/log/squid3'
'--with-pidfile=/var/run/squid3.pid' '--with-filedescriptors=65536'
'--with-large-files' '--with-default-user=proxy' '--enable-linux-netfilter'
'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector
--param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall'
'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now'
'CPPFLAGS=-D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector
--param=ssp-buffer-size=4 -Wformat -Werror=format-security'
--with-squid=/build/buildd/squid3-3.1.20
Linux 3.5.0-51-generic #76-Ubuntu SMP Thu May 15 21:19:10 UTC 2014 x86_64
x86_64 x86_64 GNU/Linux
--
View this message in context:
http://squid-web-proxy-cache.1019090.n4.nabble.com/High-cpu-usage-by-re-search-internal-tp4667550p4667655.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] redirect all ports to squid
Spam detection software, running on the system "master.squid-cache.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Hi, Yes, we can redirect the ports to squid through our firewall rules. Check below lines to redirect the ports. We have some different methods to do. 1. In first Method: First, we need to machine that squid will be running on, You do not need iptables or any special kernel options on this machine, just squid. You will, however, need the 'http_accel' options as described above. [...] Content analysis details: (5.9 points, 5.0 required) pts rule name description -- -- 0.0 HTML_MESSAGE BODY: HTML included in message 1.6 RCVD_IN_BRBL_LASTEXT RBL: No description available. [182.73.50.82 listed in bb.barracudacentral.org] 3.6 RCVD_IN_PBLRBL: Received via a relay in Spamhaus PBL [182.73.50.82 listed in zen.spamhaus.org] 0.7 RCVD_IN_XBLRBL: Received via a relay in Spamhaus XBL 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines The original message was not completely plain text, and may be unsafe to open with some email clients; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor. --- Begin Message --- Hi, Yes, we can redirect the ports to squid through our firewall rules. Check below lines to redirect the ports. We have some different methods to do. 1. In first Method: First, we need to machine that squid will be running on, You do not need iptables or any special kernel options on this machine, just squid. You will, however, need the 'http_accel' options as described above. You'll want to use the following set of commands on iptables-box: * iptables -t nat -A PREROUTING -i eth0 -s ! *squid-box* -p tcp --dport 80 -j DNAT --to *squid-box*:3128 * iptables -t nat -A POSTROUTING -o eth0 -s *local-network* -d *squid-box* -j SNAT --to *iptables-box* * iptables -A FORWARD -s *local-network* -d *squid-box* -i eth0 -o eth0 -p tcp --dport 3128 -j ACCEPT 2. And have another method: * iptables -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s *squid-box* * iptables -t mangle -A PREROUTING -j MARK --set-mark 3 -p tcp --dport 80 * ip rule add fwmark 3 table 2 * ip route add default via *squid-box* dev eth1 table 2 (OR) iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128 Regards, Visolve Squid On 9/30/2014 10:11 PM, hadi wrote: It's possible to redirect all ports to squid ? thru iptables ? For example port 25 smtp,143 imap, etc... Can squid handle that. In transparent mode. --- End Message --- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] redirect all ports to squid
Spam detection software, running on the system "master.squid-cache.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Hi, Yes, we can redirect the ports to squid through our firewall rules. Check below lines to redirect the ports. We have some different methods to do. 1. In first Method: First, we need to machine that squid will be running on, You do not need iptables or any special kernel options on this machine, just squid. You will, however, need the 'http_accel' options as described above. [...] Content analysis details: (5.9 points, 5.0 required) pts rule name description -- -- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: squid-cache.org] 3.6 RCVD_IN_PBLRBL: Received via a relay in Spamhaus PBL [182.73.50.82 listed in zen.spamhaus.org] 0.7 RCVD_IN_XBLRBL: Received via a relay in Spamhaus XBL 1.6 RCVD_IN_BRBL_LASTEXT RBL: No description available. [182.73.50.82 listed in bb.barracudacentral.org] 0.0 HTML_MESSAGE BODY: HTML included in message 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines The original message was not completely plain text, and may be unsafe to open with some email clients; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor. --- Begin Message --- Hi, Yes, we can redirect the ports to squid through our firewall rules. Check below lines to redirect the ports. We have some different methods to do. 1. In first Method: First, we need to machine that squid will be running on, You do not need iptables or any special kernel options on this machine, just squid. You will, however, need the 'http_accel' options as described above. You'll want to use the following set of commands on iptables-box: * iptables -t nat -A PREROUTING -i eth0 -s ! *squid-box* -p tcp --dport 80 -j DNAT --to *squid-box*:3128 * iptables -t nat -A POSTROUTING -o eth0 -s *local-network* -d *squid-box* -j SNAT --to *iptables-box* * iptables -A FORWARD -s *local-network* -d *squid-box* -i eth0 -o eth0 -p tcp --dport 3128 -j ACCEPT 2. And have another method: * iptables -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s *squid-box* * iptables -t mangle -A PREROUTING -j MARK --set-mark 3 -p tcp --dport 80 * ip rule add fwmark 3 table 2 * ip route add default via *squid-box* dev eth1 table 2 (OR) iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128 Regards, Visolve Squid On 9/30/2014 10:11 PM, hadi wrote: It's possible to redirect all ports to squid ? thru iptables ? For example port 25 smtp,143 imap, etc... Can squid handle that. In transparent mode. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users --- End Message --- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] redirect all ports to squid
Spam detection software, running on the system "master.squid-cache.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Hi, Yes, we can redirect the ports to squid through our firewall rules. Check below lines to redirect the ports. We have some different methods to do. 1. In first Method: First, we need to machine that squid will be running on, You do not need iptables or any special kernel options on this machine, just squid. You *will*, however, need the 'http_accel' options as described above. [...] Content analysis details: (5.9 points, 5.0 required) pts rule name description -- -- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: squid-cache.org] 0.0 HTML_MESSAGE BODY: HTML included in message 1.6 RCVD_IN_BRBL_LASTEXT RBL: No description available. [182.73.50.82 listed in bb.barracudacentral.org] 0.7 RCVD_IN_XBLRBL: Received via a relay in Spamhaus XBL [182.73.50.82 listed in zen.spamhaus.org] 3.6 RCVD_IN_PBLRBL: Received via a relay in Spamhaus PBL 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines The original message was not completely plain text, and may be unsafe to open with some email clients; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor. --- Begin Message --- Hi, Yes, we can redirect the ports to squid through our firewall rules. Check below lines to redirect the ports. We have some different methods to do. 1. In first Method: First, we need to machine that squid will be running on, You do not need iptables or any special kernel options on this machine, just squid. You *will*, however, need the 'http_accel' options as described above. You'll want to use the following set of commands on iptables-box: * iptables -t nat -A PREROUTING -i eth0 -s ! *squid-box* -p tcp --dport 80 -j DNAT --to *squid-box*:3128 * iptables -t nat -A POSTROUTING -o eth0 -s *local-network* -d *squid-box* -j SNAT --to *iptables-box* * iptables -A FORWARD -s *local-network* -d *squid-box* -i eth0 -o eth0 -p tcp --dport 3128 -j ACCEPT 2. And have another method: * iptables -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s *squid-box* * iptables -t mangle -A PREROUTING -j MARK --set-mark 3 -p tcp --dport 80 * ip rule add fwmark 3 table 2 * ip route add default via *squid-box* dev eth1 table 2 (OR) iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128 Regards, Visolve Squid On 9/30/2014 10:11 PM, hadi wrote: It's possible to redirect all ports to squid ? thru iptables ? For example port 25 smtp,143 imap, etc... Can squid handle that. In transparent mode. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users --- End Message --- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
