Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
Hi. On 12.01.2015 19:06, Amos Jeffries wrote: > > I am confident that those types of leaks do not exist at al in Squid 3.4. > > These rounds of mmory exhaustion problems are caused by pseudo-leaks, > where Squid incorrectly holds onto memory (has not forgotten it > though) far longer than it should be. > Could you please clarify for me what is the "Long Strings" pool and how can I manage it's size ? After start the largest consuming pool is the mem_node one, but it usually stops increasing after a few days (somewhere around the cache_memory border - don't know if it's it, or just a coincedence). "Long Strings", however, keep raising and raising, and after some days it becomes the largest one. I'm using the following settings: cache_mem 512 MB cache_dir diskd /var/squid/cache 1100 16 256 after few days SNMP reports that the clients amount is around 1700. Thanks. Eugene. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/2015 1:15 a.m., Yuri Voinov wrote: > > Yep. > > Memory leaking - if it really it - will be occurs on all > platforms. > > If not - this is OS-specific issue. libc, malloc library problem. > But not squid itself. > By definition a memory leak is memory which is "forgotten" / dereference by the application (squid) without correctly calling free/delete to release the memory back to the OS. I am confident that those types of leaks do not exist at al in Squid 3.4. These rounds of mmory exhaustion problems are caused by pseudo-leaks, where Squid incorrectly holds onto memory (has not forgotten it though) far longer than it should be. > > 12.01.2015 18:06, Eugene M. Zheganin пишет: >> Hi. > >> On 12.01.2015 16:41, Eugene M. Zheganin wrote: >>> I'm now also having a strong impression that squid is leaking >>> memory. Now, when 3.4.x is able to handle hundreds of users >>> during several hours I notice that it's memory usage is >>> constantly increasing. My patience always ends at the point of >>> 1.5 Gigs memory usage, where server memory starts to be >>> exhausted (squid is running with lots of other stuff) and I >>> restart it. This is happening on exactly the same config the >>> 3.3.13 was running, so ... I have cache_mem set to 512 megs, >>> diskd, medium sized cache_dir and lots of users. Is something >>> changed drastically in 3.4.x comparing to the 3.3.13, or is it, >>> as it seems, a memory leak ? >> Squid 3.4 on FreeBSD is by default compiling with the >> --enable-debug-cbdata option and when 45th log selector is at >> it's default 1, cache.log is filling with CBData memory leaking >> alarms. Here is the list for the last 40 minutes, sorted with the >> occurrence count: > >> 104136 Checklist.cc:160 81438 Checklist.cc:187 177226 >> Checklist.cc:320 84861 Checklist.cc:45 89151 CommCalls.cc:21 >> 22069 DiskIO/DiskDaemon/DiskdIOStrategy.cc:353 120 >> UserRequest.cc:166 29 UserRequest.cc:172 55814 >> clientStream.cc:235 5966 client_side_reply.cc:93 4516 >> client_side_request.cc:134 5568 dns_internal.cc:1131 4859 >> dns_internal.cc:1140 86 event.cc:90 7770 external_acl.cc:1426 >> 1548 fqdncache.cc:340 7467 helper.cc:856 39905 ipcache.cc:353 >> 11880 store.cc:1611 181959 store_client.cc:154 256951 >> store_client.cc:337 6835 ufs/UFSStoreState.cc:333 > >> are those all false alarms ? Those have been confirmed as false alerts. The C-style code releases locks first while still holding a pointer (and logs the warning about leak) then frees that pointer (outside the cbdata code) ... the C++ code does it the other way around with cbdata code doing free before releasing the final lock. I am back at the head-scratching stage about where those cbdata issues are coming from. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUs9TrAAoJELJo5wb/XPRjHMsIAL48yDQP91+DaX3F1ctY4uYw hFMMiSL48rd/4KYvwXs+7Cy2HEzcF7mMQpkcY92MdX0PYtsTAIDnP5Cggm56knqB aV4UtwgethI5qvcJlGRXrCpsmqEYjvC4w7PEdvEBRyzAZTv9g36hWMMc9IZtMX6R Yqga9pnUPvIl1Cm1e/5eD/kPfxJm4KMdBrW1jRyhD9z515F+1rko5rD0wIGZBomn KFi/ieb52JPa5+nZLKgn3q/8EQyvsVC56J0UeuZwPj1qrZE13fLAEawydxyv9/PJ RbTaoYAOzcXTTwylOrTRntnwZr+qN5pSr7eUVYv+XdEoVHrII49RlwtB6qJrUfM= =ychL -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yep. Memory leaking - if it really it - will be occurs on all platforms. If not - this is OS-specific issue. libc, malloc library problem. But not squid itself. 12.01.2015 18:06, Eugene M. Zheganin пишет: > Hi. > > On 12.01.2015 16:41, Eugene M. Zheganin wrote: >> I'm now also having a strong impression that squid is leaking memory. >> Now, when 3.4.x is able to handle hundreds of users during several >> hours I notice that it's memory usage is constantly increasing. My >> patience always ends at the point of 1.5 Gigs memory usage, where >> server memory starts to be exhausted (squid is running with lots of >> other stuff) and I restart it. This is happening on exactly the same >> config the 3.3.13 was running, so ... I have cache_mem set to 512 >> megs, diskd, medium sized cache_dir and lots of users. Is something >> changed drastically in 3.4.x comparing to the 3.3.13, or is it, as it >> seems, a memory leak ? > Squid 3.4 on FreeBSD is by default compiling with the > --enable-debug-cbdata option and when 45th log selector is at it's > default 1, cache.log is filling with CBData memory leaking alarms. Here > is the list for the last 40 minutes, sorted with the occurrence count: > > 104136 Checklist.cc:160 > 81438 Checklist.cc:187 > 177226 Checklist.cc:320 > 84861 Checklist.cc:45 > 89151 CommCalls.cc:21 > 22069 DiskIO/DiskDaemon/DiskdIOStrategy.cc:353 > 120 UserRequest.cc:166 > 29 UserRequest.cc:172 > 55814 clientStream.cc:235 > 5966 client_side_reply.cc:93 > 4516 client_side_request.cc:134 > 5568 dns_internal.cc:1131 > 4859 dns_internal.cc:1140 > 86 event.cc:90 > 7770 external_acl.cc:1426 > 1548 fqdncache.cc:340 > 7467 helper.cc:856 > 39905 ipcache.cc:353 > 11880 store.cc:1611 > 181959 store_client.cc:154 > 256951 store_client.cc:337 > 6835 ufs/UFSStoreState.cc:333 > > are those all false alarms ? > > Thanks. > Eugene. > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUs7rJAAoJENNXIZxhPexGmk0IAJmEmfJ1aBLL9DlrrnHM95JU 8VeGgsQi/wVpAjS2z1JA5eDJZ6WY5tBycVkJsKK0SBaHXrFjTHQkEUuy4iFQLpkV q8xZ4Y/X0rKJ6ayy6XSHaEd4wznlthteCTI5ImTW9fiL7PXfW7mci+o2g6lUPNa7 edtep7gp04ICmkLdq1F6P5InxksoLpc1iavV281SRowPDv151TFlZ5cn0A3fmqIv J/Pi19ss3vabiU3VXEvhiA5duxtx2lIs+BMZpU2Q3L9nQhvUf2pa8xMRBRF95RCd 8Pagb7Exzc/0/2JIjt8oCeV60Rr7xOUcwoxOXRC/4EBxzTWBH7FOkrnmBpVGNnM= =NJuF -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yep. Memory leaking - if it really it - will be occurs on all platforms. If not - this is OS-specific issue. libc, malloc library problem. But not squid itself. 12.01.2015 18:06, Eugene M. Zheganin пишет: -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUs7rFAAoJENNXIZxhPexG7EYIAKfrJU2JU6gulc11PMnuNrq1 P0lMm5WYTh2joRj6j3mCsiMR8FyolOQCxT298wY3/zXlY4bvluuwwqJ9hW4JiuMn RGXK5iJvGg8zr0yABiDoNXFLgUMVdW8NqibSfecRds7ZZkjEhnn8tUO+2jU03ZBy dZzg7TavNOeRextBJCaknr4IKwvoQWQsiparTF91wJSg9YfQ7oHsWellTlbI7uPC r/2opE2nOtKF+PEbspmzXgzt76RBe1xNM4dWikbeZOPzXvg0n7Iwbhd6TSTfWLS0 Wb4HAPB7qVJ52Nx3lYjFYHrIMqKClrj+ETQVJp7CKOZCxP25jMyF+F1Oa9d9ZxE= =mVwt -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
Hi. On 12.01.2015 16:41, Eugene M. Zheganin wrote: > I'm now also having a strong impression that squid is leaking memory. > Now, when 3.4.x is able to handle hundreds of users during several > hours I notice that it's memory usage is constantly increasing. My > patience always ends at the point of 1.5 Gigs memory usage, where > server memory starts to be exhausted (squid is running with lots of > other stuff) and I restart it. This is happening on exactly the same > config the 3.3.13 was running, so ... I have cache_mem set to 512 > megs, diskd, medium sized cache_dir and lots of users. Is something > changed drastically in 3.4.x comparing to the 3.3.13, or is it, as it > seems, a memory leak ? Squid 3.4 on FreeBSD is by default compiling with the --enable-debug-cbdata option and when 45th log selector is at it's default 1, cache.log is filling with CBData memory leaking alarms. Here is the list for the last 40 minutes, sorted with the occurrence count: 104136 Checklist.cc:160 81438 Checklist.cc:187 177226 Checklist.cc:320 84861 Checklist.cc:45 89151 CommCalls.cc:21 22069 DiskIO/DiskDaemon/DiskdIOStrategy.cc:353 120 UserRequest.cc:166 29 UserRequest.cc:172 55814 clientStream.cc:235 5966 client_side_reply.cc:93 4516 client_side_request.cc:134 5568 dns_internal.cc:1131 4859 dns_internal.cc:1140 86 event.cc:90 7770 external_acl.cc:1426 1548 fqdncache.cc:340 7467 helper.cc:856 39905 ipcache.cc:353 11880 store.cc:1611 181959 store_client.cc:154 256951 store_client.cc:337 6835 ufs/UFSStoreState.cc:333 are those all false alarms ? Thanks. Eugene. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Looks like an OS-specific issue. I don't see any memory leaking on my boxes (running Solaris 10, yes ;)). Moreover, helpers is corrrectly got an release memory. 12.01.2015 17:41, Eugene M. Zheganin пишет: > Hi. > > On 09.01.2015 06:12, Amos Jeffries wrote: >> Grand total: >> => 9.5 GB of RAM just for Squid. >> >> .. then there is whatever memory the helper programs, other software >> on the server and operating system all need. >> > I'm now also having a strong impression that squid is leaking memory. > Now, when 3.4.x is able to handle hundreds of users during several hours > I notice that it's memory usage is constantly increasing. My patience > always ends at the point of 1.5 Gigs memory usage, where server memory > starts to be exhausted (squid is running with lots of other stuff) and I > restart it. This is happening on exactly the same config the 3.3.13 was > running, so ... I have cache_mem set to 512 megs, diskd, medium sized > cache_dir and lots of users. Is something changed drastically in 3.4.x > comparing to the 3.3.13, or is it, as it seems, a memory leak ? > > Thanks. > Eugene. > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUs7R6AAoJENNXIZxhPexGrKwH/1T9k9zGwgEqQeg/6+u1z1bV kShvT7TOVNGXHMXWEka2NWjn/o973nHRAUbwBd6MUMsRSd0o3hOBYnVByYAI/6UM X/CmZpADcTMS/WSAFIrSlqj/Ml1HOafOQcmMrxw6h5jJ9qoO/O8oPHGiBpiAIjGh eMtwX0qiyfx+Xy8ncYUial/JtQPm3jsxBuCofBHatqeAA9vPyng+a+e/C4MKILX/ D4EbzCGd8CBzH8vKGkPIwUKbXJ3j79yf7ve+u+YREX/DuJ68uroHJEOo8lNnHr+/ cVoJ70C3Nju0ZYE2Dme8kYJ7764k+K4sdlD10mBXroMxqgBoUZTqIl+Tx/G8o5c= =E/KG -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
Hi. On 09.01.2015 00:10, Doug Sampson wrote: > Man, I empathize with you. Have you tried running Squid 3.4.x on > FreeBSD 9.3? Sometimes I wonder if it's FreeBSD 10.x that's causing > the issue... It's not. FreeBSD 9.x branch was a crappy release from it's start. Eugene. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
Hi. On 09.01.2015 06:12, Amos Jeffries wrote: > Grand total: > => 9.5 GB of RAM just for Squid. > > .. then there is whatever memory the helper programs, other software > on the server and operating system all need. > I'm now also having a strong impression that squid is leaking memory. Now, when 3.4.x is able to handle hundreds of users during several hours I notice that it's memory usage is constantly increasing. My patience always ends at the point of 1.5 Gigs memory usage, where server memory starts to be exhausted (squid is running with lots of other stuff) and I restart it. This is happening on exactly the same config the 3.3.13 was running, so ... I have cache_mem set to 512 megs, diskd, medium sized cache_dir and lots of users. Is something changed drastically in 3.4.x comparing to the 3.3.13, or is it, as it seems, a memory leak ? Thanks. Eugene. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 9/01/2015 8:10 a.m., Doug Sampson wrote:
>> Hi,
>>
>> I have the similar problem on FreeBSD 10.1-STABLE #1 r275861
>> with squid-3.4.10. I also applied MEMPOOLS=1 when starting squid.
>> I experience the process slowing down and unacceptable
>> performance.
>>
>> Squid is configured to use kerberos and ntlm authentication and
>> lap group authentication. other settings:
>>
~14 MB for Squid process memory.
>> cache_replacement_policy heap LFUDA cache_mem 4096 MB
+ 4096 MB for RAM cache
+ 60 MB for RAM cache index
>> maximum_object_size 32 MB cache_dir diskd /usr/local/squid/cache
>> 32768 32 256
+ 540 MB for disk cache index
+ (600 * 16 * 0.5) MB for each active client connection state.
==> 4800 MB
NP: modern web browsers open up to 8 parallel connections to load a
page ("happy eyeballs" makes that 16 TCP sockets) per client ==> ~8 MB
per active client.
Grand total:
=> 9.5 GB of RAM just for Squid.
.. then there is whatever memory the helper programs, other software
on the server and operating system all need.
>>
>> I have seen the following errors in cache.log:
>>
>> FATAL: Received Segment Violation...dying. FATAL: Received Bus
>> Error...dying.
need a backtrace to tell what the "Segment Violation" is about.
"Bus Error" is your CPU or RAM futzing up. The OS or hardware is
broken. Quite possibly this is the result of runnign out of RAM and
swap space.
>>
>> after this the squid restarts. The system has 10GB of memory and
>> is working as internal cache for ~600 users.
>>
>> Please point me in the right direction. I have no problem
>> running squid33-3.3.13 on FreeBSD 9.3-STABLE #0 r270210.
>>
>> Thank you very much.
>>
>> Regards,
>>
>> lk
>
> Man, I empathize with you. Have you tried running Squid 3.4.x on
> FreeBSD 9.3? Sometimes I wonder if it's FreeBSD 10.x that's causing
> the issue...
>
> I tried the shell variable MEMPOOLS=1 and that quickly made the
> situation a lot worse. Swap space would get filled up very quickly
> and the system would slow down quickly before crashing.
>
All MEMPOOLS=1 does is make Squid internally recycle the memory that
gets free'd. Re-using it for other objects of the same type that are
allocated right after its free'd. So new memory only gets allocated
once the recycling "pool" for an object type is empty, everything
allocated so far actively in use.
The main point of using MEMPOOLS=1 is a debugging aid to track down
leaks, by a) reduced OS allocator calls making frequent ones obvious,
and b) the mgr:mem report more clearly listing what objects are being
free'd (chunked pool contains recycled objects) and which are not
(pool always empty).
Steve Hill has come up with some patches that resolve memory issues
with the new 3.4 helper annotations feature when using Negotiate or
NTMLM auth helpers. For authenticated clients with long connection
times or high traffic volumes the state can accumulate up to quite
large amounts of memory. see the "Debugging slow proxy" thread of
earlier this week.
HTH
Amos
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBAgAGBQJUrysEAAoJELJo5wb/XPRjFFMH/RNhVzPWfFixDDarpHUFQMoE
6QLkaD5wwQfRp3xfBW/QfZ9vvu04zptFOvNr4766jGsh9RSGZdNZwgPc9pCADDrv
Xs0HCO1VDMxGoBjFfaS2XJ5DLX2zQdbYlZKb0yghxXSYzg0ZZEhmsKO8r1Exp8j7
KO95yP3z/8agmwXbU2sJ1esqRC7IfW2sF/DtU8cPTzUf0cKEyjGoCbVrNAN1qKUH
jLf1iw8sNr3xGwFG/WmQmKpYTsXInSp4GmwXgCSQ0T5DCLVDtWuvnXpobxR8iuSW
zIX/CDKTe58lfVP4EXdzJNBgiuEiVZaRccynbk2L8HRvJ8kt9Pk50P5tWXM5FMw=
=lv6G
-END PGP SIGNATURE-
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
> Doug Sampson writes: >> Hi, >> >> I have the similar problem on FreeBSD 10.1-STABLE #1 r275861 with >> squid-3.4.10. I also applied MEMPOOLS=1 when starting squid. I >> experience the process slowing down and unacceptable performance. >> >> Squid is configured to use kerberos and ntlm authentication and lap >> group authentication. other settings: >> >> cache_replacement_policy heap LFUDA >> cache_mem 4096 MB >> maximum_object_size 32 MB >> cache_dir diskd /usr/local/squid/cache 32768 32 256 >> >> I have seen the following errors in cache.log: >> >> FATAL: Received Segment Violation...dying. >> FATAL: Received Bus Error...dying. >> >> after this the squid restarts. The system has 10GB of memory and is >> working as internal cache for ~600 users. >> >> Please point me in the right direction. I have no problem running >> squid33-3.3.13 on FreeBSD 9.3-STABLE #0 r270210. >> >> Thank you very much. >> >> Regards, >> >> lk > Man, I empathize with you. Have you tried running Squid 3.4.x on > FreeBSD 9.3? Sometimes I wonder if it's FreeBSD 10.x that's causing > the issue... I am running FreeBSD 9.3-STABLE #1 r274084 with Squid 3.4.9 as a cache in DMZ without authentication. There are no problems. lk > I tried the shell variable MEMPOOLS=1 and that quickly made the > situation a lot worse. Swap space would get filled up very quickly and > the system would slow down quickly before crashing. > Any other ideas? > ~Doug ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I mean, cache_mem is too big. First, 600 users is not very big installation to do huge memory cache. 1-2 Gb will be ehough. With fast disk cache. Second - 4 Gb ram is near 32-bit address limit. Possible 32-bit library or somewhat meets this limit. Is OS and Squid completely 64-bit? 09.01.2015 1:10, Doug Sampson пишет: >> Hi, >> >> I have the similar problem on FreeBSD 10.1-STABLE #1 r275861 with >> squid-3.4.10. I also applied MEMPOOLS=1 when starting squid. I >> experience the process slowing down and unacceptable performance. >> >> Squid is configured to use kerberos and ntlm authentication and lap >> group authentication. other settings: >> >> cache_replacement_policy heap LFUDA >> cache_mem 4096 MB >> maximum_object_size 32 MB >> cache_dir diskd /usr/local/squid/cache 32768 32 256 >> >> I have seen the following errors in cache.log: >> >> FATAL: Received Segment Violation...dying. >> FATAL: Received Bus Error...dying. >> >> after this the squid restarts. The system has 10GB of memory and is >> working as internal cache for ~600 users. >> >> Please point me in the right direction. I have no problem running >> squid33-3.3.13 on FreeBSD 9.3-STABLE #0 r270210. >> >> Thank you very much. >> >> Regards, >> >> lk > > Man, I empathize with you. Have you tried running Squid 3.4.x on FreeBSD 9.3? Sometimes I wonder if it's FreeBSD 10.x that's causing the issue... > > I tried the shell variable MEMPOOLS=1 and that quickly made the situation a lot worse. Swap space would get filled up very quickly and the system would slow down quickly before crashing. > > Any other ideas? > > ~Doug > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUrvaqAAoJENNXIZxhPexGTigIAKCNyOeAD03un/vxFINUN2Yb 0cZLNzSjIDkzhepXkfQo2yazcTSoGoJ5QFSwMj9yljFCve2JjTj1O6KT4JxbOzvG vMpEqY6KjuCsENU8vm6lBq+D7PmtZw20V+nofA1V3y1L9CGCuygVTe33FuWNIVwA VQKmhV+0IaG8vIdHgKtYKUK9n4zV48DAPurpjTf4UotzD0+BYef/BOatrQ1HGp9/ z6cmqB9FGIxOk5MZIgeer/3bjkYng7Q60zm7pHnMFhb265+0bsMzkvGl/PMvzKib ASRdTqy+rZiE/Y48KfcjTjY4Eyvk69U3I5iuu7xU+CvawMsBhifRn79A7e0eHTQ= =gLo3 -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
> Hi, > > I have the similar problem on FreeBSD 10.1-STABLE #1 r275861 with > squid-3.4.10. I also applied MEMPOOLS=1 when starting squid. I > experience the process slowing down and unacceptable performance. > > Squid is configured to use kerberos and ntlm authentication and lap > group authentication. other settings: > > cache_replacement_policy heap LFUDA > cache_mem 4096 MB > maximum_object_size 32 MB > cache_dir diskd /usr/local/squid/cache 32768 32 256 > > I have seen the following errors in cache.log: > > FATAL: Received Segment Violation...dying. > FATAL: Received Bus Error...dying. > > after this the squid restarts. The system has 10GB of memory and is > working as internal cache for ~600 users. > > Please point me in the right direction. I have no problem running > squid33-3.3.13 on FreeBSD 9.3-STABLE #0 r270210. > > Thank you very much. > > Regards, > > lk Man, I empathize with you. Have you tried running Squid 3.4.x on FreeBSD 9.3? Sometimes I wonder if it's FreeBSD 10.x that's causing the issue... I tried the shell variable MEMPOOLS=1 and that quickly made the situation a lot worse. Swap space would get filled up very quickly and the system would slow down quickly before crashing. Any other ideas? ~Doug ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
Hi, I have the similar problem on FreeBSD 10.1-STABLE #1 r275861 with squid-3.4.10. I also applied MEMPOOLS=1 when starting squid. I experience the process slowing down and unacceptable performance. Squid is configured to use kerberos and ntlm authentication and lap group authentication. other settings: cache_replacement_policy heap LFUDA cache_mem 4096 MB maximum_object_size 32 MB cache_dir diskd /usr/local/squid/cache 32768 32 256 I have seen the following errors in cache.log: FATAL: Received Segment Violation...dying. FATAL: Received Bus Error...dying. after this the squid restarts. The system has 10GB of memory and is working as internal cache for ~600 users. Please point me in the right direction. I have no problem running squid33-3.3.13 on FreeBSD 9.3-STABLE #0 r270210. Thank you very much. Regards, lk ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/12/2014 4:22 a.m., Doug Sampson wrote: >> Nothing particularly stands out as leaking. Although the cache >> memory pages (mem_node) in-use size is suspiciously close to half >> what you say the OS is reporting. >> >> That makes me suspect that your OS is rounding up its allocations >> to 8KB of memory for each node. If that is the case the simplest >> workaround is to reduce cache_mem size down to below the point >> where the box will swap. > > Okay, I'll keep dropping the cache_mem down until the system stops > swapping. Hopefully not too much. > > Would it make sense to add additional memory in order to reduce the > possibility of swapping? Say, 1GB more? If you can afford that option, then yes. > >> If you are game for it I have been wondering if we need to >> enable chunking for 64-bit systems. To test that run squid with >> environment variable MEMPOOLS=1. Memory should then be allocated >> in larger blocks, but utilized much more compactly within those >> blocks for an overall saving on objects like mem_node. It is >> currently a rarely used feature though, so I'm not sure if there >> are any issues hiding. >> > > I'm reluctant to do this on a production server. Interfacing with > our customers via social media et al is an important part of our > business. > Sure. No problem. > Nevertheless, I looked into how the MEMPOOLS parameter could be > implemented in a FreeBSD machine. I couldn't find a post that > clearly indicates how to implement it. The nearest reference I > could find was that FreeBSD implements UMA which apparently with my > limited knowledge is a parallel memory implementation of MEMPOOLS. > Correct me if I'm terribly mistaken. > It is a shell environment variable Squid checks for when starting up. Add the line: export MEMPOOLS=1 to the Squid init script, then run the script to restart the proxy. likewise, remove the line and restart to stop using it. NP: you may or may not need the "export" keyword depending on your shell. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUiHMhAAoJELJo5wb/XPRjx7wIAIkcEUEvdQ+vN5QbtEhBxERK 7iofy58y5BgPkoargu2BkLJXmLd+mAuanJOslL45RAyp1BlBJuDVi+rGX/rGvIBj OOFbnIx99qbyysG4gkZcqu8S3TESzfJyQ8QTFGB39GK69fNXTymcK2+S+3GVLW/P 9u95j9EwMCYw5BZQcd5QJhaoLQqrkQeJ7/7jNXcB/7AEs5Ho4xT07tV66RS9c0BK SUQrYEbYHOx+Xm4dOf2e1oF8JD8ilZ+Hyn+/7+pRMVkbvMN3fq/nG6Ibrhoo2CU+ JcwfK0a7/pgbEiJPb7TDCiaPjIHnkXnyI4d8eBRvEDbLytww3gXuETuqD/1OBFg= =v4R3 -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
> Nothing particularly stands out as leaking. Although the cache memory > pages (mem_node) in-use size is suspiciously close to half what you > say the OS is reporting. > > That makes me suspect that your OS is rounding up its allocations to > 8KB of memory for each node. If that is the case the simplest > workaround is to reduce cache_mem size down to below the point where > the box will swap. Okay, I'll keep dropping the cache_mem down until the system stops swapping. Hopefully not too much. Would it make sense to add additional memory in order to reduce the possibility of swapping? Say, 1GB more? > If you are game for it I have been wondering if we need to enable > chunking for 64-bit systems. To test that run squid with environment > variable MEMPOOLS=1. > Memory should then be allocated in larger blocks, but utilized much > more compactly within those blocks for an overall saving on objects > like mem_node. It is currently a rarely used feature though, so I'm > not sure if there are any issues hiding. > I'm reluctant to do this on a production server. Interfacing with our customers via social media et al is an important part of our business. Nevertheless, I looked into how the MEMPOOLS parameter could be implemented in a FreeBSD machine. I couldn't find a post that clearly indicates how to implement it. The nearest reference I could find was that FreeBSD implements UMA which apparently with my limited knowledge is a parallel memory implementation of MEMPOOLS. Correct me if I'm terribly mistaken. ~Doug ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/12/2014 11:17 a.m., Doug Sampson wrote: >> On 26/11/2014 8:59 a.m., Doug Sampson wrote: >>> >>> Thanks, Amos, for your pointers. >>> >>> I've commented out all the fresh_patterns lines appearing >>> above the last two lines. >>> >>> I also have dropped diskd in favor of using aufs exclusively, >>> taking out the min-size parameter. I've commented out the >>> diskd_program support option. In the previous version of squid >>> (2.7) I had split the cache_dir into two types with great >>> success using coss and aufs. Previously I had only aufs and >>> performance wasn't where I wanted it. Apparently coss is no >>> longer supported in the 3.x version of squid atop FreeBSD. >> >> COSS has been replaced with Rock storage type in Squid-3. They >> should be used in roughly similar ways in terms of traffic >> optimization. >> >>> >>> The pathname for the cache swap logs have been fixed. >>> Apparently this came from a squid.conf example that I copied in >>> parts. Would this be the reason why we are seeing the error >>> messages in /var/log/messages regarding swapping mentioned in >>> my original post? >> >> No. I think that is coming out of the OS kernel memory >> management. It uses the term "swap" as well in regards to disk >> backed virtual memory. >> >> If your system is "swapping" (using that disk backed "swap >> memory") while running Squid then you will get terrible >> performance as a matter of course since the Squid cache index and >> cache_meme is often very large in RAM and accessed often. >> >>> >>> The hierarchy_stoplist line has been stripped out as you say it >>> is deprecated. >>> >>> The mem .TSV file is attached herewith. >>> >>> Currently I have the cache_dir located on the OS disk and all >>> of the cache logging files on a second drive. Is this the >>> optimal setup of cache-dir and logs? >> >> I would do it the other way around. Logs are appended with a >> small amount of data each transaction, whereas the main cache_dir >> has a relativey large % of the bandwidth throughput being written >> out to it constantly (less % in recent Squid, but still a lot). >> The dik most likely to die early is the one holding cache_dir. >> > I'm still running into the issue of being out of available space in > the swapfile on my system. I've attached another TSV file > indicating the various type of memory being in use and whatnot. Is > there anything in there that screams out? > Nothing particularly stands out as leaking. Although the cache memory pages (mem_node) in-use size is suspiciously close to half what you say the OS is reporting. That makes me suspect that your OS is rounding up its allocations to 8KB of memory for each node. If that is the case the simplest workaround is to reduce cache_mem size down to below the point where the box will swap. If you are game for it I have been wondering if we need to enable chunking for 64-bit systems. To test that run squid with environment variable MEMPOOLS=1. Memory should then be allocated in larger blocks, but utilized much more compactly within those blocks for an overall saving on objects like mem_node. It is currently a rarely used feature though, so I'm not sure if there are any issues hiding. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUgptwAAoJELJo5wb/XPRjLbUIAJjORv/denfKNTA53cxrN7CU Cbyc3m6yqiM4Mi0J0VNS2qPUJYd69sSGkz4Q4XtpoKLPjO8i8glcVDg8ncoKMokj E3rJbpj4tcdukPmvJ8XlQRqlAfYANTQK6wGvfhmQOlNqPjtKi6eB7HZR75LWNFlM aj/6Ij15cfM0OI9KQW5PyKwxLwTmZhDn3g0RTHdbzho9GZOjfZEYA0Wn2eMDNqOM JIwsamBp6KFCN7w/6Mw2I0+guVOpFALDd35M39HdapEuzwauuNrt5F1DvD+rSIa+ 8iBKRcDtB3tsaFWyIilFoxCbTTdtdggDhiN0FfT/KEebtZb+A5PI+IQIFWszNzc= =5tf8 -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
> On 26/11/2014 8:59 a.m., Doug Sampson wrote: > > > > Thanks, Amos, for your pointers. > > > > I've commented out all the fresh_patterns lines appearing above > > the last two lines. > > > > I also have dropped diskd in favor of using aufs exclusively, > > taking out the min-size parameter. I've commented out the > > diskd_program support option. In the previous version of squid > > (2.7) I had split the cache_dir into two types with great success > > using coss and aufs. Previously I had only aufs and performance > > wasn't where I wanted it. Apparently coss is no longer supported in > > the 3.x version of squid atop FreeBSD. > > COSS has been replaced with Rock storage type in Squid-3. They should > be used in roughly similar ways in terms of traffic optimization. > > > > > The pathname for the cache swap logs have been fixed. Apparently > > this came from a squid.conf example that I copied in parts. Would > > this be the reason why we are seeing the error messages in > > /var/log/messages regarding swapping mentioned in my original > > post? > > No. I think that is coming out of the OS kernel memory management. It > uses the term "swap" as well in regards to disk backed virtual memory. > > If your system is "swapping" (using that disk backed "swap memory") > while running Squid then you will get terrible performance as a matter > of course since the Squid cache index and cache_meme is often very > large in RAM and accessed often. > > > > > The hierarchy_stoplist line has been stripped out as you say it is > > deprecated. > > > > The mem .TSV file is attached herewith. > > > > Currently I have the cache_dir located on the OS disk and all of > > the cache logging files on a second drive. Is this the optimal > > setup of cache-dir and logs? > > I would do it the other way around. Logs are appended with a small > amount of data each transaction, whereas the main cache_dir has a > relativey large % of the bandwidth throughput being written out to it > constantly (less % in recent Squid, but still a lot). The dik most > likely to die early is the one holding cache_dir. > I'm still running into the issue of being out of available space in the swapfile on my system. I've attached another TSV file indicating the various type of memory being in use and whatnot. Is there anything in there that screams out? Amos, you said earlier that it was the OS system that needed to be tuned. Are there any references to where I can fine-tune it for Squid usage? I looked here http://oss.org.cn/man/newsoft/squid/Squid_FAQ/FAQ-8.html#how-much-ram and I'm unable to figure out a way to decrease the amount of memory that I could use. I tried limiting cache_mem to 1344 MB from some higher value but that didn't work. What are some of the methods that FreeBSD 10.0 users using to limit the use of memory that Squid uses? Sort of fixing the memory leaks, it looks like I need to consider the possibility of restarting the Squid service on a regular basis (i.e. at least once a week) in order to enable Squid to perform at an acceptable level and to avoid clogging /var/log/messages with such messages as follows: <...> +swap_pager_getswapspace(15): failed +swap_pager_getswapspace(2): failed +swap_pager_getswapspace(16): failed +swap_pager_getswapspace(16): failed <...> Is this a common practice among squid admins to restart squid periodically? Thank you! ~Doug Current memory usage: Pool Obj SizeChunksAllocatedIn UseIdleAllocations SavedRate (bytes)KB/ch obj/ch(#) used free part %Frag (#) (KB) high (KB) high (hrs) %Tot(#) (KB) high (KB) high (hrs) %alloc(#) (KB) high (KB)(#) %cnt %vol(#)/sec mem_node 4136292390 1180982 1336651 4.76 86.464 292191 1180178 1336651 4.76 99.932 199 804 16173 1641718 1.322 21.088 0.001 Short Strings 401701206 66454 73289 4.76 4.865 1699725 66396 73289 4.76 99.913 1481 58 250 37674459 30.347 4.680 0.029 HttpHeaderEntry56796862 43579 47925 4.76 3.191 796197 43543 47925 4.76 99.917 665 37 163 10727306 8.641 1.866 0.008 HttpReply 28074302 20317 22720 4.76 1.487 74275 20310 22720 4.76 99.964 27 8 135 754702 0.608 0.656 0.001 MemObject 24074278 17409 19473 4.76 1.275 74273 17408 19473 4.76 99.993 5 2 121 187591 0.151 0.140 0.000 StoreEntry10479949 8120 9020 4.76 0.594 79944 8120 9020 4.76 99.994 5 1 53 198382 0.160 0.064 0.000 HttpHdrCc 9650739 4757 5279 4.76 0.348 50725 4756 5279 4.76 99.972 14 2 41 602296 0.485 0.180 0.000 Medium Strings12837543 4693 4889 3.29 0.344 37426 4679 4889 3.29 99.688 117 15 48 2579153 2.078 1.025 0.002 cbdata MemBuf (9) 6474339 4647 5197 4.76 0.340 74310 4645 5197 4.76 99.961 29 2 31 1332078 1.073 0.265 0.001 Long Strings 5124170 2085 2085 0.04 0.153 4105 2053 2085 0.04 98.441 65 33 130 389656 0.314 0.620 0.000 LRU policy node2474220 1740 1946 4.76 0.127 74217 1740 1946 4.7
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
I used the ugly tuning: set vm.defer_swapspace_pageouts to 1. But it is may caused some issue when the physic memory is really exhausted. I have no much time to investigate the right way, but I think maybe vm.swap_idle_threshold1/vm.swap_idle_threshold2 or vm.overcommit etc. maybe harmful. Simon 在 14/12/1 23:16, Doug Sampson 写道: Maybe your problem is related to sysctl mib tuning about swap/overcommit etc. I did not observed memory leak with squid 3.4.4, but FB 10 do swap frequently than old version. Could you elaborate a bit more? That went over my head. What could I do in terms of tuning the system? ~Doug ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
>Maybe your problem is related to sysctl mib tuning about > swap/overcommit etc. I did not observed memory leak with squid 3.4.4, > but FB 10 do swap frequently than old version. Could you elaborate a bit more? That went over my head. What could I do in terms of tuning the system? ~Doug ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
Maybe your problem is related to sysctl mib tuning about swap/overcommit etc. I did not observed memory leak with squid 3.4.4, but FB 10 do swap frequently than old version. Simon ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/11/2014 8:59 a.m., Doug Sampson wrote: > > Thanks, Amos, for your pointers. > > I've commented out all the fresh_patterns lines appearing above > the last two lines. > > I also have dropped diskd in favor of using aufs exclusively, > taking out the min-size parameter. I've commented out the > diskd_program support option. In the previous version of squid > (2.7) I had split the cache_dir into two types with great success > using coss and aufs. Previously I had only aufs and performance > wasn't where I wanted it. Apparently coss is no longer supported in > the 3.x version of squid atop FreeBSD. COSS has been replaced with Rock storage type in Squid-3. They should be used in roughly similar ways in terms of traffic optimization. > > The pathname for the cache swap logs have been fixed. Apparently > this came from a squid.conf example that I copied in parts. Would > this be the reason why we are seeing the error messages in > /var/log/messages regarding swapping mentioned in my original > post? No. I think that is coming out of the OS kernel memory management. It uses the term "swap" as well in regards to disk backed virtual memory. If your system is "swapping" (using that disk backed "swap memory") while running Squid then you will get terrible performance as a matter of course since the Squid cache index and cache_meme is often very large in RAM and accessed often. > > The hierarchy_stoplist line has been stripped out as you say it is > deprecated. > > The mem .TSV file is attached herewith. > > Currently I have the cache_dir located on the OS disk and all of > the cache logging files on a second drive. Is this the optimal > setup of cache-dir and logs? I would do it the other way around. Logs are appended with a small amount of data each transaction, whereas the main cache_dir has a relativey large % of the bandwidth throughput being written out to it constantly (less % in recent Squid, but still a lot). The dik most likely to die early is the one holding cache_dir. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUfCoRAAoJELJo5wb/XPRjvtsIAJskATDCJDf4IZdapvHXeJAn Ug8KkCrV4/HU5ajeG/Z0G7czvzU34LNXCDtblLWp2JD9LDBK9NRTlOO6q5IoTtAY qqw5YXFrKbDt16A6bNHuVDn1Mxh+vHG64KG6cPabcG4lR3EnAlS8/WoBebAZDHCq 1Ds0LfPJJKNg6CEKiaXavwv+zv5rEsZBCZtQmq8+8hymn7ytCBOo/IN5+UKUkhZg cEo3RgZYhVjo5msATRTjR83Ow+4MCEKaejFuRsFVlI6tapJlZ6u2M/0XgxGRbQyv NqXJoxLunXlOMEftpyPWY52EOhH7XkQzTLLky+mzHQK4P9/jPMv8NuwUxZhS6Bk= =Etqf -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
> On 25/11/2014 9:06 a.m., Doug Sampson wrote: > > Recently due to squid 2.7 being EOL'ed, we migrated our squid > > server to version 3.4.9 on a FreeBSD 10.0-RELEASE running on 64-bit > > hardware. We started seeing paging file being swapped out > > eventually running out of available memory. From the time squid > > gets started it usually takes about two days before we see these > > entries in /var/log/messages as follows: > > > > +swap_pager_getswapspace(16): failed +swap_pager_getswapspace(16): > > failed +swap_pager_getswapspace(16): failed > > +swap_pager_getswapspace(12): failed +swap_pager_getswapspace(16): > > failed +swap_pager_getswapspace(12): failed > > +swap_pager_getswapspace(6): failed +swap_pager_getswapspace(16): > > failed > > > > Looking at the 'top' results, I see that the swap file has been > > totally exhausted. Memory used by squid hovers around 2.3GB out of > > the total 3GB of system memory. > > > > I am not sure what is causing these memory leaks. After rebooting, > > squid-internal-mgr/info shows the following statistics: > > > > Squid Object Cache: Version 3.4.9 Build Info: Start Time: Mon, 24 > > Nov 2014 18:39:08 GMT Current Time: Mon, 24 Nov 2014 19:39:13 GMT > > Connection information for squid: Number of clients accessing > > cache: 18 Number of HTTP requests received:10589 Number of ICP > > messages received: 0 Number of ICP messages sent: 0 Number of > > queued ICP replies: 0 Number of HTCP messages received: 0 Number of > > HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP > > requests per minute since start:176.2 Average ICP messages per > > minute since start: 0.0 Select loop called: 763993 times, 4.719 ms > > avg Cache information for squid: Hits as % of all requests: 5min: > > 3.2%, 60min: 17.0% Hits as % of bytes sent: 5min: 2.0%, 60min: > > 6.7% Memory hits as % of hit requests: 5min: 0.0%, 60min: 37.2% > > Disk hits as % of hit requests: 5min: 22.2%, 60min: 33.2% Storage > > Swap size: 7361088 KB Storage Swap capacity: 58.5% used, 41.5% > > free Storage Mem size: 54348 KB Storage Mem capacity: 3.9% > used, > > 96.1% free Mean Object Size:23.63 KB Requests given to unlinkd: > > 1 > > Median Service Times (seconds) 5 min60 min: HTTP Requests > > (All): 0.10857 0.19742 Cache Misses: 0.10857 0.32154 > > Cache Hits:0.08265 0.01387 Near Hits: > > 0.15048 0.12106 Not-Modified Replies: 0.00091 0.00091 DNS > > Lookups: 0.05078 0.05078 ICP Queries: 0.0 > > 0.0 Resource usage for squid: UP Time: 3605.384 seconds CPU > > Time: 42.671 seconds CPU Usage: 1.18% CPU Usage, 5 minute avg: > > 0.72% CPU Usage, 60 minute avg: 1.17% Maximum Resident Size: 845040 > > KB Page faults with physical i/o: 20 Memory accounted for: Total > > accounted: 105900 KB memPoolAlloc calls: 2673353 > > memPoolFree calls:2676487 File descriptor usage for squid: > > Maximum number of file descriptors: 87516 Largest file desc > > currently in use:310 Number of file desc currently in use: > > 198 Files queued for open: 0 Available number of > > file descriptors: 87318 Reserved number of file descriptors: 100 > > Store Disk files open: 0 Internal Data > > Structures: 311543 StoreEntries 4421 StoreEntries with MemObjects > > 4416 Hot Object Cache Items 311453 on-disk objects > > > > I will post another one tomorrow that will indicate growing > > memory/swapfile consumption. > > > > Here is my squid.conf: > > > > # OPTIONS FOR AUTHENTICATION # > > > - > > > > > # 1st four lines for > > auth_param basic children 5 auth_param basic realm Squid > > proxy-caching web server auth_param basic credentialsttl 2 hours > > auth_param basic casesensitive off # next three lines for kerberos > > authentication (needed to use usernames) # used in conjunction > > with "acl auth proxy_auth" line below #auth_param negotiate program > > /usr/local/libexec/squid/negotiate_kerberos_auth -i #auth_param > > negotiate children 50 startup=10 idle=5 #auth_param negotiate > > keep_alive on > > > > > > # ACCESS CONTROLS # > > > - > > > > > # Example rule allowing access from your local networks. > > # Adapt to list your (internal) IP networks from where browsing # > > should be allowed #acl manager proto cache_object acl manager > > url_regex -i ^cache_object:// /squid-internal-mgr/ acl adminhost > > src 192.168.1.149 acl localnet src 192.168.1.0/24 # RFC1918 > > possible internal network acl localnet src fc00::/7 # RFC > > 4193 local private network range acl localnet src fe80::/10 > > # RFC 4291 link-local (directly plugged) machines acl webserver src > > 198.168.1.35 acl some_big_clients src 192.168.1.149/32 #CI53 > > > > # We want to limit downloads of th
Re: [squid-users] Memory Leak Squid 3.4.9 on FreeBSD 10.0 x64
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25/11/2014 9:06 a.m., Doug Sampson wrote: > Recently due to squid 2.7 being EOL'ed, we migrated our squid > server to version 3.4.9 on a FreeBSD 10.0-RELEASE running on 64-bit > hardware. We started seeing paging file being swapped out > eventually running out of available memory. From the time squid > gets started it usually takes about two days before we see these > entries in /var/log/messages as follows: > > +swap_pager_getswapspace(16): failed +swap_pager_getswapspace(16): > failed +swap_pager_getswapspace(16): failed > +swap_pager_getswapspace(12): failed +swap_pager_getswapspace(16): > failed +swap_pager_getswapspace(12): failed > +swap_pager_getswapspace(6): failed +swap_pager_getswapspace(16): > failed > > Looking at the 'top' results, I see that the swap file has been > totally exhausted. Memory used by squid hovers around 2.3GB out of > the total 3GB of system memory. > > I am not sure what is causing these memory leaks. After rebooting, > squid-internal-mgr/info shows the following statistics: > > Squid Object Cache: Version 3.4.9 Build Info: Start Time: Mon, 24 > Nov 2014 18:39:08 GMT Current Time: Mon, 24 Nov 2014 19:39:13 GMT > Connection information for squid: Number of clients accessing > cache:18 Number of HTTP requests received:10589 Number of ICP > messages received:0 Number of ICP messages sent: 0 Number of > queued ICP replies: 0 Number of HTCP messages received: 0 Number of > HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP > requests per minute since start: 176.2 Average ICP messages per > minute since start: 0.0 Select loop called: 763993 times, 4.719 ms > avg Cache information for squid: Hits as % of all requests: 5min: > 3.2%, 60min: 17.0% Hits as % of bytes sent: 5min: 2.0%, 60min: > 6.7% Memory hits as % of hit requests:5min: 0.0%, 60min: 37.2% > Disk hits as % of hit requests: 5min: 22.2%, 60min: 33.2% Storage > Swap size:7361088 KB Storage Swap capacity: 58.5% used, 41.5% > free Storage Mem size:54348 KB Storage Mem capacity: 3.9% used, > 96.1% free Mean Object Size: 23.63 KB Requests given to unlinkd: 1 > Median Service Times (seconds) 5 min60 min: HTTP Requests > (All): 0.10857 0.19742 Cache Misses: 0.10857 0.32154 > Cache Hits:0.08265 0.01387 Near Hits: > 0.15048 0.12106 Not-Modified Replies: 0.00091 0.00091 DNS > Lookups: 0.05078 0.05078 ICP Queries: 0.0 > 0.0 Resource usage for squid: UP Time:3605.384 seconds CPU > Time: 42.671 seconds CPU Usage: 1.18% CPU Usage, 5 minute avg: > 0.72% CPU Usage, 60 minute avg: 1.17% Maximum Resident Size: 845040 > KB Page faults with physical i/o: 20 Memory accounted for: Total > accounted: 105900 KB memPoolAlloc calls: 2673353 > memPoolFree calls:2676487 File descriptor usage for squid: > Maximum number of file descriptors: 87516 Largest file desc > currently in use:310 Number of file desc currently in use: > 198 Files queued for open: 0 Available number of > file descriptors: 87318 Reserved number of file descriptors: 100 > Store Disk files open: 0 Internal Data > Structures: 311543 StoreEntries 4421 StoreEntries with MemObjects > 4416 Hot Object Cache Items 311453 on-disk objects > > I will post another one tomorrow that will indicate growing > memory/swapfile consumption. > > Here is my squid.conf: > > # OPTIONS FOR AUTHENTICATION # > - > > # 1st four lines for > auth_param basic children 5 auth_param basic realm Squid > proxy-caching web server auth_param basic credentialsttl 2 hours > auth_param basic casesensitive off # next three lines for kerberos > authentication (needed to use usernames) # used in conjunction > with "acl auth proxy_auth" line below #auth_param negotiate program > /usr/local/libexec/squid/negotiate_kerberos_auth -i #auth_param > negotiate children 50 startup=10 idle=5 #auth_param negotiate > keep_alive on > > > # ACCESS CONTROLS # > - > > # Example rule allowing access from your local networks. > # Adapt to list your (internal) IP networks from where browsing # > should be allowed #acl manager proto cache_object acl manager > url_regex -i ^cache_object:// /squid-internal-mgr/ acl adminhost > src 192.168.1.149 acl localnet src 192.168.1.0/24 # RFC1918 > possible internal network acl localnet src fc00::/7 # RFC > 4193 local private network range acl localnet src fe80::/10 > # RFC 4291 link-local (directly plugged) machines acl webserver src > 198.168.1.35 acl some_big_clients src 192.168.1.149/32 #CI53 > > # We want to limit downloads of these type of files # Put this all > in one line acl magic_words url_regex -i ftp .exe .mp3 .vqf .tar.gz >
