Re: [squid-users] FileDescriptor Issues
Hi All I have recompiled squid with 6400 FDS I tried with 65535 and I got a warning that 65535 is not a multiple of 64 and it may cause some problems on some systems. so I changed it to 6400 I completed the installation started Squid now it's showing 6400 although the system is set to 65535 I have one question, from your experiences with squid, would 6400 FDS be enough? Thank you all for your help Regards Adam - Original Message - From: Bradley, Stephen W. Mr. bradl...@muohio.edu To: a...@gmail adbas...@googlemail.com Sent: Tuesday, March 23, 2010 2:02 PM Subject: RE: [squid-users] FileDescriptor Issues A problem I found is that you have to set ulimit BEFORE you compile it as well. I built everything from scratch and everytime I rebuild it I have to : ulimit -HSn XX ( being whatever you want it to be) In /etc/init.d/squid (the script I use) [snip] PATH=/usr/bin:/sbin:/bin:/usr/sbin export PATH ulimit -HSn 32768 [snip] That way every time I run the script it makes sure that it sets the FDs up to where they need to be. I'm guessing that if you have a busy server it is crashing after a little while of running... ;-) steve -Original Message- From: a...@gmail [mailto:adbas...@googlemail.com] Sent: Monday, March 22, 2010 11:10 PM To: Amos Jeffries; squid-users@squid-cache.org Subject: Re: [squid-users] FileDescriptor Issues Thanks Amos for this tip I will try that and keep you posted Regards Adam - Original Message - From: Amos Jeffries squ...@treenet.co.nz To: squid-users@squid-cache.org Sent: Tuesday, March 23, 2010 2:54 AM Subject: Re: [squid-users] FileDescriptor Issues On Tue, 23 Mar 2010 02:19:40 -, a...@gmail adbas...@googlemail.com wrote: Thanks Ivan for your suggestion But in my case it's slightly different I have no squid in /etc/default/squid /etc/init.d/mine is located in /usr/local/squid/sbin/squidunless I try this/usr/local/squid/sbin/squid SQUID_MAXFD=4096 /etc/default/squid is a configuration file for configuring the system init.d/squid script. It does not exist normally, you create it only when overrides are needed. .../sbin/squid is supposed to be the binary application which gets run. And then restart it, but I am not sure I am using Ubuntu HardyI think this tip is for the Squid that is packaged with Ubuntu and not the compiledSquid Bash environment shells resets the descriptors down again towards 1024 each time a new one is generated. It _always_ must be increased to the wanted limit before running Squid. Whether you do it manually on the command line each time, or in the init.d script, or in some other custom starter script. My Ubuntu systems show default OS limits of just over 24K FD available. Building Squid with: ulimit -HSn 65535 ./configure --with-filedescriptors=65535 ... make install starting: squid -f /etc/squid.conf squid shows 1024 starting: ulimit -Hsn 64000 squid -f /etc/squid.conf squid shows 64000 Amos
Re: [squid-users] FileDescriptor Issues
Hi, Which OS are you using? my start up script is located here /usr/local/squid/sbin/squid The packaged one I had before, the startup script was located in /etc/init.d/squid3 But not the compiled version Thank you Regards Adam - Original Message - From: Bradley, Stephen W. Mr. bradl...@muohio.edu To: a...@gmail adbas...@googlemail.com Sent: Tuesday, March 23, 2010 2:02 PM Subject: RE: [squid-users] FileDescriptor Issues A problem I found is that you have to set ulimit BEFORE you compile it as well. I built everything from scratch and everytime I rebuild it I have to : ulimit -HSn XX ( being whatever you want it to be) In /etc/init.d/squid (the script I use) [snip] PATH=/usr/bin:/sbin:/bin:/usr/sbin export PATH ulimit -HSn 32768 [snip] That way every time I run the script it makes sure that it sets the FDs up to where they need to be. I'm guessing that if you have a busy server it is crashing after a little while of running... ;-) steve -Original Message- From: a...@gmail [mailto:adbas...@googlemail.com] Sent: Monday, March 22, 2010 11:10 PM To: Amos Jeffries; squid-users@squid-cache.org Subject: Re: [squid-users] FileDescriptor Issues Thanks Amos for this tip I will try that and keep you posted Regards Adam - Original Message - From: Amos Jeffries squ...@treenet.co.nz To: squid-users@squid-cache.org Sent: Tuesday, March 23, 2010 2:54 AM Subject: Re: [squid-users] FileDescriptor Issues On Tue, 23 Mar 2010 02:19:40 -, a...@gmail adbas...@googlemail.com wrote: Thanks Ivan for your suggestion But in my case it's slightly different I have no squid in /etc/default/squid /etc/init.d/mine is located in /usr/local/squid/sbin/squidunless I try this/usr/local/squid/sbin/squid SQUID_MAXFD=4096 /etc/default/squid is a configuration file for configuring the system init.d/squid script. It does not exist normally, you create it only when overrides are needed. .../sbin/squid is supposed to be the binary application which gets run. And then restart it, but I am not sure I am using Ubuntu HardyI think this tip is for the Squid that is packaged with Ubuntu and not the compiledSquid Bash environment shells resets the descriptors down again towards 1024 each time a new one is generated. It _always_ must be increased to the wanted limit before running Squid. Whether you do it manually on the command line each time, or in the init.d script, or in some other custom starter script. My Ubuntu systems show default OS limits of just over 24K FD available. Building Squid with: ulimit -HSn 65535 ./configure --with-filedescriptors=65535 ... make install starting: squid -f /etc/squid.conf squid shows 1024 starting: ulimit -Hsn 64000 squid -f /etc/squid.conf squid shows 64000 Amos
[squid-users] FileDescriptor Issues
Hi All, I have tried everything so far I definitely have increased my file descriptors on my Ubuntu OS from 1024 to 46622 But when I start Squid 3.0 STABLE25 I doesn't seem to detect the real descriptor's size I have checked the sysctl.conf, and I have checked the system to make sure that the correct size /etc/sysctl.confWhen I run this I more /proc/sys/fs/file-maxI get 46622But Squid3.0 seem to only detect 1024Is there anything that I am not doing please? I don't know what else to do Thank you Regards Adam
Re: [squid-users] FileDescriptor Issues
Hi, Did you try using ulimit? Best Regards, Al On Mon, 22 Mar 2010, a...@gmail wrote: Date: Mon, 22 Mar 2010 17:42:47 - From: a...@gmail adbas...@googlemail.com To: squid-users@squid-cache.org Subject: [squid-users] FileDescriptor Issues Hi All, I have tried everything so far I definitely have increased my file descriptors on my Ubuntu OS from 1024 to 46622 But when I start Squid 3.0 STABLE25 I doesn't seem to detect the real descriptor's size I have checked the sysctl.conf, and I have checked the system to make sure that the correct size /etc/sysctl.confWhen I run this I more /proc/sys/fs/file-maxI get 46622But Squid3.0 seem to only detect 1024Is there anything that I am not doing please? I don't know what else to do Thank you Regards Adam
Re: [squid-users] FileDescriptor Issues
Hi, Al Yes I did thanks for the suggestion I am trying to figure out why is Squid refusing to aknowledge the available size on the system Unless of course it's a bug on either sides, I mean on Squid's side and Ubuntu side, But I have checked some Ubuntu forums and people used the same methods I used and it seems very strange that when I start Squid I get 1024 instead of 46622 or whatever the number I put Regards Adam - Original Message - From: Al - Image Hosting Services az...@zickswebventures.com To: a...@gmail adbas...@googlemail.com Cc: squid-users@squid-cache.org Sent: Monday, March 22, 2010 6:13 PM Subject: Re: [squid-users] FileDescriptor Issues Hi, Did you try using ulimit? Best Regards, Al On Mon, 22 Mar 2010, a...@gmail wrote: Date: Mon, 22 Mar 2010 17:42:47 - From: a...@gmail adbas...@googlemail.com To: squid-users@squid-cache.org Subject: [squid-users] FileDescriptor Issues Hi All, I have tried everything so far I definitely have increased my file descriptors on my Ubuntu OS from 1024 to 46622 But when I start Squid 3.0 STABLE25 I doesn't seem to detect the real descriptor's size I have checked the sysctl.conf, and I have checked the system to make sure that the correct size /etc/sysctl.confWhen I run this I more /proc/sys/fs/file-maxI get 46622But Squid3.0 seem to only detect 1024Is there anything that I am not doing please? I don't know what else to do Thank you Regards Adam
Re: [squid-users] FileDescriptor Issues
Hello All, I have solved the problem, I managed to increase the filedescriptor from 1024 This what I have done on (Ubuntu hardy) it should work on most Ubuntu OS and Debians I first needed to see the max that my System can support run this command first: cat /proc/sys/fs/file-max it will display the maximum that you're system can currently handle to increase that number you need first to run this command let's assume X is a number 46900 echo proc/sys/fs/file-max (where the xx is the number you want to add) you then need to add this into the file /etc/sysctl.conf file fs.file-max = X (that same number again) After you've done this. check again with this command systcl -p It's all stored in /proc/sys/fs/file-nr (just run this command to get the output) To modify the limit descriptors per session We need to add this to our limits.conf emacs or vi /etc/security/limits.conf and add * soft nofile X * hard nofileX Note you can use either or both of the above two lines And you can use a specific user instead of the wildcard * which is at the beginning of each line, it means to all users on your system save it and then you can check with ulimit -n if you still get 1024 you probably need to reboot your system altogether, on mine it didn't show until I rebooted anyway. I hope this will help someone somewhere at some point Regards Adam - Original Message - From: Al - Image Hosting Services az...@zickswebventures.com To: a...@gmail adbas...@googlemail.com Cc: squid-users@squid-cache.org Sent: Monday, March 22, 2010 6:13 PM Subject: Re: [squid-users] FileDescriptor Issues Hi, Did you try using ulimit? Best Regards, Al On Mon, 22 Mar 2010, a...@gmail wrote: Date: Mon, 22 Mar 2010 17:42:47 - From: a...@gmail adbas...@googlemail.com To: squid-users@squid-cache.org Subject: [squid-users] FileDescriptor Issues Hi All, I have tried everything so far I definitely have increased my file descriptors on my Ubuntu OS from 1024 to 46622 But when I start Squid 3.0 STABLE25 I doesn't seem to detect the real descriptor's size I have checked the sysctl.conf, and I have checked the system to make sure that the correct size /etc/sysctl.confWhen I run this I more /proc/sys/fs/file-maxI get 46622But Squid3.0 seem to only detect 1024Is there anything that I am not doing please? I don't know what else to do Thank you Regards Adam
[squid-users] FileDescriptor Issues
I have solved the problem, I managed to increase the filedescriptor My system now reads 65535 But Squid still says only 1024 fileDescriptors available What can I do to fix this please, I have rebooted the system and Squid several times I am running out of ideas Any help would be appreciated Regards Adam
Re: [squid-users] FileDescriptor Issues
Have you set the descriptor size in the squid start up script? see here http://paulgoscicki.com/archives/2007/01/squid-warning-your-cache-is-running-out-of-filedescriptors/ cheers Ivan On Tue, Mar 23, 2010 at 12:45 PM, a...@gmail adbas...@googlemail.com wrote: I have solved the problem, I managed to increase the filedescriptor My system now reads 65535 But Squid still says only 1024 fileDescriptors available What can I do to fix this please, I have rebooted the system and Squid several times I am running out of ideas Any help would be appreciated Regards Adam
Re: [squid-users] FileDescriptor Issues
Thanks Ivan for your suggestion But in my case it's slightly different I have no squid in /etc/default/squid /etc/init.d/mine is located in /usr/local/squid/sbin/squidunless I try this/usr/local/squid/sbin/squid SQUID_MAXFD=4096 And then restart it, but I am not sure I am using Ubuntu HardyI think this tip is for the Squid that is packaged with Ubuntu and not the compiledSquidThanks for your suggestion I appreciate itRegardsAdamFrom: Ivan . ivan...@gmail.comTo: a...@gmail adbas...@googlemail.com Cc: squid-users@squid-cache.org Sent: Tuesday, March 23, 2010 1:50 AM Subject: Re: [squid-users] FileDescriptor Issues Have you set the descriptor size in the squid start up script? see here http://paulgoscicki.com/archives/2007/01/squid-warning-your-cache-is-running-out-of-filedescriptors/ cheers Ivan On Tue, Mar 23, 2010 at 12:45 PM, a...@gmail adbas...@googlemail.com wrote: I have solved the problem, I managed to increase the filedescriptor My system now reads 65535 But Squid still says only 1024 fileDescriptors available What can I do to fix this please, I have rebooted the system and Squid several times I am running out of ideas Any help would be appreciated Regards Adam
Re: [squid-users] FileDescriptor Issues
Sorry I haven't set it in the Start up script But I will try it right away Regards Adam - Original Message - From: Ivan . ivan...@gmail.com To: a...@gmail adbas...@googlemail.com Cc: squid-users@squid-cache.org Sent: Tuesday, March 23, 2010 1:50 AM Subject: Re: [squid-users] FileDescriptor Issues Have you set the descriptor size in the squid start up script? see here http://paulgoscicki.com/archives/2007/01/squid-warning-your-cache-is-running-out-of-filedescriptors/ cheers Ivan On Tue, Mar 23, 2010 at 12:45 PM, a...@gmail adbas...@googlemail.com wrote: I have solved the problem, I managed to increase the filedescriptor My system now reads 65535 But Squid still says only 1024 fileDescriptors available What can I do to fix this please, I have rebooted the system and Squid several times I am running out of ideas Any help would be appreciated Regards Adam
Re: [squid-users] FileDescriptor Issues
On Tue, 23 Mar 2010 02:19:40 -, a...@gmail adbas...@googlemail.com wrote: Thanks Ivan for your suggestion But in my case it's slightly different I have no squid in /etc/default/squid /etc/init.d/mine is located in /usr/local/squid/sbin/squidunless I try this/usr/local/squid/sbin/squid SQUID_MAXFD=4096 /etc/default/squid is a configuration file for configuring the system init.d/squid script. It does not exist normally, you create it only when overrides are needed. .../sbin/squid is supposed to be the binary application which gets run. And then restart it, but I am not sure I am using Ubuntu HardyI think this tip is for the Squid that is packaged with Ubuntu and not the compiledSquid Bash environment shells resets the descriptors down again towards 1024 each time a new one is generated. It _always_ must be increased to the wanted limit before running Squid. Whether you do it manually on the command line each time, or in the init.d script, or in some other custom starter script. My Ubuntu systems show default OS limits of just over 24K FD available. Building Squid with: ulimit -HSn 65535 ./configure --with-filedescriptors=65535 ... make install starting: squid -f /etc/squid.conf squid shows 1024 starting: ulimit -Hsn 64000 squid -f /etc/squid.conf squid shows 64000 Amos
Re: [squid-users] FileDescriptor Issues
Thanks Amos for this tip I will try that and keep you posted Regards Adam - Original Message - From: Amos Jeffries squ...@treenet.co.nz To: squid-users@squid-cache.org Sent: Tuesday, March 23, 2010 2:54 AM Subject: Re: [squid-users] FileDescriptor Issues On Tue, 23 Mar 2010 02:19:40 -, a...@gmail adbas...@googlemail.com wrote: Thanks Ivan for your suggestion But in my case it's slightly different I have no squid in /etc/default/squid /etc/init.d/mine is located in /usr/local/squid/sbin/squidunless I try this/usr/local/squid/sbin/squid SQUID_MAXFD=4096 /etc/default/squid is a configuration file for configuring the system init.d/squid script. It does not exist normally, you create it only when overrides are needed. .../sbin/squid is supposed to be the binary application which gets run. And then restart it, but I am not sure I am using Ubuntu HardyI think this tip is for the Squid that is packaged with Ubuntu and not the compiledSquid Bash environment shells resets the descriptors down again towards 1024 each time a new one is generated. It _always_ must be increased to the wanted limit before running Squid. Whether you do it manually on the command line each time, or in the init.d script, or in some other custom starter script. My Ubuntu systems show default OS limits of just over 24K FD available. Building Squid with: ulimit -HSn 65535 ./configure --with-filedescriptors=65535 ... make install starting: squid -f /etc/squid.conf squid shows 1024 starting: ulimit -Hsn 64000 squid -f /etc/squid.conf squid shows 64000 Amos
