Re: [squid-users] Disk Space problem in a squid-proxy server
Chuck Kollars wrote: ... At the most the squid server would run for another day and then stop running!! ... Please suggest some pointers to delete some files under /var partition to create more space !! ... Both your question and all the responses to it I've seen assume Squid is the cause of this problem, in other words that Squid is _using_ the missing disk space. Although this might be the case, it's not the only possibility. I wonder if something else is using the missing disk space, and Squid is just the victim? Hes made a reasonable assumption on previous experience. A few weeks ago when he came in it was Squid with an overly large cache configured and dumping core files when the cache dir overloaded. This time squids logs are getting extra large, so he needs to rotate them as advised last time also. But something else is also at fault this time around by the evidence given. (I fear the actions you've taken in the past, deleting various files such as core.* and some Squid files, may have been just sticking bandaids on a broken leg. They may have bought you a few more months, but without ever addressing the real problem.) When I have a partition that runs short of disk space, the first thing I find out is WHO. To do that, I work my way down the directory hierarchy to localize the problem to one particular folder, then look closely at all the files in that folder to see who created them and when. Even with its obvious holes, this simplistic procedure works pretty well in practice almost all the time. Here's an example: cd /var # assume for this example /var is the problem partition du -s * | sort -n # bottommost line is the suspect, let's suppose for example it's 'log' cd log du -s * | sort -n # do it again # bottommost line is the suspect, let's suppose for example it's 'cups' cd cups du -s * | sort -n # do it again # stop, as no more sub-folders indicating we're at the bottom ls -l * # look at ownership and timestamps on files, imagine how they got this way The max size of the Squid cache is specified in squid.conf. When it gets full, Squid will throw out the oldest files to make room for the new ones, so it will keep running all by itself. You should never have to explicitly delete any files out of the Squid cache. You should make sure all the disk space that squid.conf says might be used is actually available (and continues to be available:-). To prevent the problem of something else sucking up all the disk space that was supposed to be for Squid, I put my Squid cache on a separate partition (not under /var at all) that doesn't contain anything else. (As an aside, using a separate partition also allows me to improve performance by easily using the 'noatime' mount parameter.) I gave him all this same advice last time. Ho took it. the cache is in /home and being happy. It's that other thing now cluttering up /var still causing issues when it blocks squid log files. So bakck to the du procedure again is best to find the second culprit. Amos -- Please use Squid 2.7.STABLE4 or 3.0.STABLE9
Re: [squid-users] Disk Space problem in a squid-proxy server
* This message has been scanned by IMSS NIT-Silchar Can I reconfigure the current squid cache ? If so please tell which all of the lines needs to changed so that the prxy server can made runnable without much hindrance to the users of the proxy server. Asli if the contens would get trasshed if the number of directories and its sub-directories have been changed ? If at all I change according to my requirement what should i set ? As of noew the /var partition shows up the disk utilization as 96%. At the most the squid server may run for another few hours and I am in a dilemma as to what should I do make cache squeeze and make the squid run normally. Kindly help!!! Angela Williams wrote: Have a look at your squid.conf file and look for this line cache_dir My one I have configured like this. cache_dir aufs /data/squid 5000 15 256 To give 5G in /data/squid If you change it you will need to trash the current cache and create new with squid -z This statement is only true if you change the number of directories, and/or sub-directories (the 15 and 256 in the example given). Changing the size of the cache (even shrinking it) only requires a squid -k reconfigure. If the used disk space is greater than the allocation, objects will be purged until the size on disk is below the high water mark. Cheers Ang Chris * This message has been scanned by IMSS-NIT Silchar Thanks Regards Jyotishmaan Ray Computer Center,NITS National Institute of Technology Silchar Assam, INDIA Pin - 788 010 Tele: 0091-3842-233179 Fax: 0091-3842-233797 Mobile: 09435554598 URL: http://www.nits.ac.in
Re: [squid-users] Disk Space problem in a squid-proxy server
* This message has been scanned by IMSS NIT-Silchar Can I reconfigure the current squid cache ? If so please tell which all of the lines needs to changed so that the prxy server can made runnable without much hindrance to the users of the proxy server. Asli if the contens would get trasshed if the number of directories and its sub-directories have been changed ? If at all I change according to my requirement what should i set ? As of noew the /var partition shows up the disk utilization as 96%. At the most the squid server may run for another few hours and I am in a dilemma as to what should I do make cache squeeze and make the squid run normally. Angela Williams wrote: Have a look at your squid.conf file and look for this line cache_dir My one I have configured like this. cache_dir aufs /data/squid 5000 15 256 To give 5G in /data/squid If you change it you will need to trash the current cache and create new with squid -z This statement is only true if you change the number of directories, and/or sub-directories (the 15 and 256 in the example given). Changing the size of the cache (even shrinking it) only requires a squid -k reconfigure. If the used disk space is greater than the allocation, objects will be purged until the size on disk is below the high water mark. Cheers Ang Chris
Re: [squid-users] Disk Space problem in a squid-proxy server
On 17.10.08 21:17, Francois Cami wrote: squid -k rotate will rotate all .log files for you, you can delete the *.0 files afterwards. it's log files, not .log files, they don't need to be named *.log and if logfile_rotate is 0, they won't get renamed, only reopened, so it's up to other process to rename them first ... -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Chernobyl was an Windows 95 beta test site.
Re: [squid-users] Disk Space problem in a squid-proxy server
[EMAIL PROTECTED] wrote: * This message has been scanned by IMSS NIT-Silchar Can I reconfigure the current squid cache ? No need. The cache IS NOT IN /var and cannot now be the cause of the problem. Amos If so please tell which all of the lines needs to changed so that the prxy server can made runnable without much hindrance to the users of the proxy server. Asli if the contens would get trasshed if the number of directories and its sub-directories have been changed ? If at all I change according to my requirement what should i set ? As of noew the /var partition shows up the disk utilization as 96%. At the most the squid server may run for another few hours and I am in a dilemma as to what should I do make cache squeeze and make the squid run normally. Kindly help!!! Angela Williams wrote: Have a look at your squid.conf file and look for this line cache_dir My one I have configured like this. cache_dir aufs /data/squid 5000 15 256 To give 5G in /data/squid If you change it you will need to trash the current cache and create new with squid -z This statement is only true if you change the number of directories, and/or sub-directories (the 15 and 256 in the example given). Changing the size of the cache (even shrinking it) only requires a squid -k reconfigure. If the used disk space is greater than the allocation, objects will be purged until the size on disk is below the high water mark. Cheers Ang Chris * This message has been scanned by IMSS-NIT Silchar Thanks Regards Jyotishmaan Ray Computer Center,NITS National Institute of Technology Silchar Assam, INDIA Pin - 788 010 Tele: 0091-3842-233179 Fax: 0091-3842-233797 Mobile: 09435554598 URL: http://www.nits.ac.in -- Please use Squid 2.7.STABLE4 or 3.0.STABLE9
Re: [squid-users] Disk Space problem in a squid-proxy server
* This message has been scanned by IMSS NIT-Silchar Dear All Squid users, The output of the command df in my proxy server is as follows:- [EMAIL PROTECTED] squid]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/PrimaryVol-root 19838052 6579328 12234724 35% / /dev/mapper/PrimaryVol-var 14855176 12993112 1095296 93% /var /dev/mapper/PrimaryVol-home 34756272 29035932 3926292 89% /home /dev/mapper/PrimaryVol-tmp 4062912139748 3713452 4% /tmp /dev/sda1 101086 22511 73356 24% /boot tmpfs 25342412253412 1% /dev/shm Now the thing is that earlier I used to delete the files generated on time-t-time basis under /var/spool/squid directory to create more space for the squid server to run. But now, there are no core.* files generated in /var/spool/squid directory, so what should i do to create more space under /var partition, as of now it shows up as 93%. At the most the squid server would run for another day and then stop running!! Please suggest some pointers to delete some files under /var partition to create more space !! Thanks in advance, jmaan
Re: [squid-users] Disk Space problem in a squid-proxy server
On Friday 17 October 2008, [EMAIL PROTECTED] wrote: * The output of the command df in my proxy server is as follows:- [EMAIL PROTECTED] squid]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/PrimaryVol-root 19838052 6579328 12234724 35% / /dev/mapper/PrimaryVol-var 14855176 12993112 1095296 93% /var /dev/mapper/PrimaryVol-home 34756272 29035932 3926292 89% /home /dev/mapper/PrimaryVol-tmp 4062912139748 3713452 4% /tmp /dev/sda1 101086 22511 73356 24% /boot tmpfs 25342412253412 1% /dev/shm Now the thing is that earlier I used to delete the files generated on time-t-time basis under /var/spool/squid directory to create more space for the squid server to run. But now, there are no core.* files generated in /var/spool/squid directory, so what should i do to create more space under /var partition, as of now it shows up as 93%. At the most the squid server would run for another day and then stop running!! Please suggest some pointers to delete some files under /var partition to create more space !! Log files in /var/log? What about the squid store log which you do not need! Update your squid.conf to make the store log none Have a look at your squid.conf file and look for this line cache_dir My one I have configured like this. cache_dir aufs /data/squid 5000 15 256 To give 5G in /data/squid If you change it you will need to trash the current cache and create new with squid -z Cheers Ang -- Angela Williams Enterprise Outsourcing Unix/Linux Cisco spoken here! Bedfordview [EMAIL PROTECTED] Gauteng South Africa Smile!! Jesus Loves You!!
Re: [squid-users] Disk Space problem in a squid-proxy server
* This message has been scanned by IMSS NIT-Silchar Please see below the output of pwd and df commands:- [EMAIL PROTECTED] squid]# pwd /var/log/squid [EMAIL PROTECTED] squid]# ls -l total 1005212 -rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log drwxr-xr-x 2 root root 4096 2008-10-17 00:49 Backup -rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log -rw-r--r-- 1 root root 135898 2008-10-17 18:27 squid.out -rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log [EMAIL PROTECTED] squid]# In my proxyserver, the access.log is under the path /var/log/squid. Now please tell me do i need to keep the cache.log file. Can i delete this file to create more space? Also what can i do delete/ squeeze the access.log file ? I have looked into the squid.conf file, and there is line like this below. cache_dir ufs /home/squid 30720 16 256 What should I do or can do to reduce the size of the /var partition in my proxy-server ? The output if the df command shows up utilization of 94% of the hard disk. [EMAIL PROTECTED] squid]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/PrimaryVol-root 19838052 6579304 12234748 35% / /dev/mapper/PrimaryVol-var 14855176 13145712942696 94% /var /dev/mapper/PrimaryVol-home 34756272 29043404 3918820 89% /home /dev/mapper/PrimaryVol-tmp 4062912139716 3713484 4% /tmp /dev/sda1 101086 22511 73356 24% /boot tmpfs 253424 0253424 0% /dev/shm [EMAIL PROTECTED] squid]# Thanks, jmaan
Re: [squid-users] Disk Space problem in a squid-proxy server
* This message has been scanned by IMSS NIT-Silchar Please see below the output of pwd and df commands:- [EMAIL PROTECTED] squid]# pwd /var/log/squid [EMAIL PROTECTED] squid]# ls -l total 1005212 -rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log drwxr-xr-x 2 root root 4096 2008-10-17 00:49 Backup -rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log -rw-r--r-- 1 root root 135898 2008-10-17 18:27 squid.out -rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log [EMAIL PROTECTED] squid]# In my proxyserver, the access.log is under the path /var/log/squid. Now please tell me do i need to keep the cache.log file. Can i delete this file to create more space? Also what can i do delete/ squeeze the access.log file ? I have looked into the squid.conf file, and there is line like this below. cache_dir ufs /home/squid 30720 16 256 What should I do or can do to reduce the size of the /var partition in my proxy-server ? The output if the df command shows up utilization of 94% of the hard disk. [EMAIL PROTECTED] squid]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/PrimaryVol-root 19838052 6579304 12234748 35% / /dev/mapper/PrimaryVol-var 14855176 13145712942696 94% /var /dev/mapper/PrimaryVol-home 34756272 29043404 3918820 89% /home /dev/mapper/PrimaryVol-tmp 4062912139716 3713484 4% /tmp /dev/sda1 101086 22511 73356 24% /boot tmpfs 253424 0253424 0% /dev/shm [EMAIL PROTECTED] squid]# Thanks, jmaan
Re: [squid-users] Disk Space problem in a squid-proxy server
* This message has been scanned by IMSS NIT-Silchar Please see below the output of pwd and df commands:- [EMAIL PROTECTED] squid]# pwd /var/log/squid [EMAIL PROTECTED] squid]# ls -l total 1005212 -rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log drwxr-xr-x 2 root root 4096 2008-10-17 00:49 Backup -rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log -rw-r--r-- 1 root root 135898 2008-10-17 18:27 squid.out -rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log [EMAIL PROTECTED] squid]# In my proxyserver, the access.log is under the path /var/log/squid. Now please tell me do i need to keep the cache.log file. Can i delete this file to create more space? Also what can i do delete/ squeeze the access.log file ? I have looked into the squid.conf file, and there is line like this below. cache_dir ufs /home/squid 30720 16 256 What should I do or can do to reduce the size of the /var partition in my proxy-server ? The output if the df command shows up utilization of 94% of the hard disk. [EMAIL PROTECTED] squid]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/PrimaryVol-root 19838052 6579304 12234748 35% / /dev/mapper/PrimaryVol-var 14855176 13145712942696 94% /var /dev/mapper/PrimaryVol-home 34756272 29043404 3918820 89% /home /dev/mapper/PrimaryVol-tmp 4062912139716 3713484 4% /tmp /dev/sda1 101086 22511 73356 24% /boot tmpfs 253424 0253424 0% /dev/shm [EMAIL PROTECTED] squid]# Thanks, jmaan
Re: [squid-users] Disk Space problem in a squid-proxy server
squid -k rotate will rotate all .log files for you, you can delete the *.0 files afterwards. Never delete files without knowing what you are doing ; deleting files from under squid's nose will lead to unpredictable behaviour :) You really should read the manuals, and especially the parts related to logs files and cache_dir entries, to understand what you are doing. Francois On Sat, Oct 18, 2008 at 4:52 AM, [EMAIL PROTECTED] wrote: * This message has been scanned by IMSS NIT-Silchar Please see below the output of pwd and df commands:- [EMAIL PROTECTED] squid]# pwd /var/log/squid [EMAIL PROTECTED] squid]# ls -l total 1005212 -rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log drwxr-xr-x 2 root root 4096 2008-10-17 00:49 Backup -rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log -rw-r--r-- 1 root root 135898 2008-10-17 18:27 squid.out -rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log [EMAIL PROTECTED] squid]# In my proxyserver, the access.log is under the path /var/log/squid. Now please tell me do i need to keep the cache.log file. Can i delete this file to create more space? Also what can i do delete/ squeeze the access.log file ? I have looked into the squid.conf file, and there is line like this below. cache_dir ufs /home/squid 30720 16 256 What should I do or can do to reduce the size of the /var partition in my proxy-server ? The output if the df command shows up utilization of 94% of the hard disk. [EMAIL PROTECTED] squid]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/PrimaryVol-root 19838052 6579304 12234748 35% / /dev/mapper/PrimaryVol-var 14855176 13145712942696 94% /var /dev/mapper/PrimaryVol-home 34756272 29043404 3918820 89% /home /dev/mapper/PrimaryVol-tmp 4062912139716 3713484 4% /tmp /dev/sda1 101086 22511 73356 24% /boot tmpfs 253424 0253424 0% /dev/shm [EMAIL PROTECTED] squid]# Thanks, jmaan
Re: [squid-users] Disk Space problem in a squid-proxy server
... At the most the squid server would run for another day and then stop running!! ... Please suggest some pointers to delete some files under /var partition to create more space !! ... Both your question and all the responses to it I've seen assume Squid is the cause of this problem, in other words that Squid is _using_ the missing disk space. Although this might be the case, it's not the only possibility. I wonder if something else is using the missing disk space, and Squid is just the victim? (I fear the actions you've taken in the past, deleting various files such as core.* and some Squid files, may have been just sticking bandaids on a broken leg. They may have bought you a few more months, but without ever addressing the real problem.) When I have a partition that runs short of disk space, the first thing I find out is WHO. To do that, I work my way down the directory hierarchy to localize the problem to one particular folder, then look closely at all the files in that folder to see who created them and when. Even with its obvious holes, this simplistic procedure works pretty well in practice almost all the time. Here's an example: cd /var # assume for this example /var is the problem partition du -s * | sort -n # bottommost line is the suspect, let's suppose for example it's 'log' cd log du -s * | sort -n # do it again # bottommost line is the suspect, let's suppose for example it's 'cups' cd cups du -s * | sort -n # do it again # stop, as no more sub-folders indicating we're at the bottom ls -l * # look at ownership and timestamps on files, imagine how they got this way The max size of the Squid cache is specified in squid.conf. When it gets full, Squid will throw out the oldest files to make room for the new ones, so it will keep running all by itself. You should never have to explicitly delete any files out of the Squid cache. You should make sure all the disk space that squid.conf says might be used is actually available (and continues to be available:-). To prevent the problem of something else sucking up all the disk space that was supposed to be for Squid, I put my Squid cache on a separate partition (not under /var at all) that doesn't contain anything else. (As an aside, using a separate partition also allows me to improve performance by easily using the 'noatime' mount parameter.) thanks! -Chuck Kollars __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: [squid-users] Disk Space problem in a squid-proxy server
Angela Williams wrote: Have a look at your squid.conf file and look for this line cache_dir My one I have configured like this. cache_dir aufs /data/squid 5000 15 256 To give 5G in /data/squid If you change it you will need to trash the current cache and create new with squid -z This statement is only true if you change the number of directories, and/or sub-directories (the 15 and 256 in the example given). Changing the size of the cache (even shrinking it) only requires a squid -k reconfigure. If the used disk space is greater than the allocation, objects will be purged until the size on disk is below the high water mark. Cheers Ang Chris
Re: [squid-users] Disk Space problem in a squid-proxy server
[EMAIL PROTECTED] wrote: * This message has been scanned by IMSS NIT-Silchar Please see below the output of pwd and df commands:- [EMAIL PROTECTED] squid]# pwd /var/log/squid [EMAIL PROTECTED] squid]# ls -l total 1005212 -rw-r--r-- 1 squid squid 502314096 2008-10-17 22:40 access.log drwxr-xr-x 2 root root 4096 2008-10-17 00:49 Backup -rw-r--r-- 1 squid squid920314 2008-10-17 22:40 cache.log -rw-r--r-- 1 root root 135898 2008-10-17 18:27 squid.out -rw-r--r-- 1 squid squid 524928293 2008-10-17 22:40 store.log [EMAIL PROTECTED] squid]# In my proxyserver, the access.log is under the path /var/log/squid. Now please tell me do i need to keep the cache.log file. Can i delete this file to create more space? Also what can i do delete/ squeeze the access.log file ? squid -k rotate I have looked into the squid.conf file, and there is line like this below. cache_dir ufs /home/squid 30720 16 256 Thats 2/3 the drive space of /home. seems reasonable for a dedicated proxy box. What should I do or can do to reduce the size of the /var partition in my proxy-server ? The output if the df command shows up utilization of 94% of the hard disk. [EMAIL PROTECTED] squid]# df Filesystem 1K-blocks Used Available Use% Mounted on ... 14855176 13145712942696 94% /var 34756272 29043404 3918820 89% /home ... store_log none will kill the store log entirely. Its not really needed. Regular rotate's as mentioned earlier and by others will keep the access.log from growing so huge. BUT, the squid files total to only 1GB. Whatever is causing the rest of the 15GB to fill up does not appear to be Squid from those traces. Are you sure the cores from earlier stopped rather than changing location? Amos -- Please use Squid 2.7.STABLE4 or 3.0.STABLE9
Re: [squid-users] Disk Space problem in a squid-proxy server
Hi! On Thursday 16 October 2008, [EMAIL PROTECTED] wrote: * This message has been scanned by IMSS NIT-Silchar Dear All Squid Users, I have a proxy server, where the df shows the following : [EMAIL PROTECTED] /]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/PrimaryVol-root 19838052 6484280 12329772 35% / /dev/mapper/PrimaryVol-var 14855176 12631912 1456496 90% /var /dev/mapper/PrimaryVol-home 34756272 29035612 3926612 89% /home /dev/mapper/PrimaryVol-tmp 4062912205720 3647480 6% /tmp /dev/sda1 101086 22511 73356 24% /boot tmpfs 253424 0253424 0% /dev/shm On executing the du command under /home it shows the following ( copying here a certain portion of the output of the command du): 9304./squid/00/DA 8748./squid/00/55 3364./squid/00/C6 10204 ./squid/00/E9 6336./squid/00/62 4996./squid/00/57 8264./squid/00/16 5428./squid/00/0A 8972./squid/00/31 6344./squid/00/DD 6740./squid/00/68 5168./squid/00/9D 5004./squid/00/DC 15868 ./squid/00/E4 6464./squid/00/07 8708./squid/00/A1 4728./squid/00/51 3588./squid/00/3B 4468./squid/00/91 4508./squid/00/D1 5064./squid/00/D3 7652./squid/00/BA 5828./squid/00/B1 3484./squid/00/88 . My question, is can i delete the above folders to make more free space available under /home directory ? Simple answer? No! You might want to have a look at your squid.conf file and see how much diskspace is allocated for its cache. Look for a line like this default. cache_dir ufs /var/cache/squid 100 16 256 The 100 is the size in Meg in this default. Remember that if you change the size it really means trashing the old cache and have squid recreate and nice new empty one with squid -z Please do not hijack an existing thread as many users with threaded news readers will not see it! Cheers Ang Cheers Ang -- Angela Williams Enterprise Outsourcing Unix/Linux Cisco spoken here! Bedfordview [EMAIL PROTECTED] Gauteng South Africa Smile!! Jesus Loves You!!
