Re: /etc/daily /tmp purge mods; skip open files with fstat test
On 2015-05-14 Thu 11:24 AM |, Todd C. Miller wrote: On Thu, 14 May 2015 17:48:49 +0100, Stuart Henderson wrote: Even not allowing for TOCTOU problems I'd be wary of running fstat automatically. If it is only used to detect when a file is in use that we would otherwise delete it seems reasonable. It is always annoying when daily removes /tmp files that are actually in use just because of their date. In the diff, there's no /dev/null redirects. Any errors could be obvious. Maybe some adventurous men could experiment with it for a fortnight see how it goes. Cool, -- http://www.stuff.co.nz/travel/themes/adventure/68507731/swiss-daredevil-yves-jetman-rossy-soars-over-dubai
Re: /etc/daily /tmp purge mods; skip open files with fstat test
On Thu, 14 May 2015 17:48:49 +0100, Stuart Henderson wrote: On 2015/05/14 17:45, Craig Skinner wrote: $ fstat -u _squid -f /tmp Even not allowing for TOCTOU problems I'd be wary of running fstat automatically. If it is only used to detect when a file is in use that we would otherwise delete it seems reasonable. It is always annoying when daily removes /tmp files that are actually in use just because of their date. - todd
/etc/daily /tmp purge mods; skip open files with fstat test
Hi tech,
Here's a daily(8) email, showing purged unreferenced open files in /tmp:
- Forwarded message from Charlie Root r...@britvault.co.uk -
...
..
Checking filesystems:
** /dev/rwd0a (NO WRITE)
** Last Mounted on /
** Root file system
2474 files, 24716 used, 14539 free (275 frags, 1783 blocks, 0.7% fragmentation)
...
..
.
** /dev/rwd0j (NO WRITE)
** Last Mounted on /tmp
UNREF FILE I=3 OWNER=_squid MODE=100600
SIZE=8 MTIME=May 3 01:30 2015
CLEAR? no
UNREF FILE I=4 OWNER=_squid MODE=100600
SIZE=8216 MTIME=May 3 01:30 2015
CLEAR? no
UNREF FILE I=5 OWNER=_squid MODE=100600
SIZE=44 MTIME=May 3 01:30 2015
CLEAR? no
55 files, 157 used, 248826 free (42 frags, 31098 blocks, 0.0% fragmentation)
...
..
- End forwarded message -
Here are Squid's /tmp files following a daemon restart:
$ fstat -u _squid -f /tmp
USER CMD PID FD MOUNTINUM MODE R/WSZ|DV
_squid squid 66734 /tmp3 -rw--- rw8
_squid squid 66735 /tmp4 -rw--- rw 8216
_squid squid 66736 /tmp5 -rw--- rw 44
$ find /tmp -maxdepth 1 -type f -user _squid -ls
34 -rw---1 _squid wheel 8 May 10 01:30
/tmp/e378568b39344b2594a38dc74f20c87d5ded81fbd3e2f3e210844537fad87caf.shm
4 20 -rw---1 _squid wheel8216 May 10 01:30
/tmp/9c0c04602d25164c1e3019612d4c1914336991e7917bb5b92efe3eb6731eefd7.shm
54 -rw---1 _squid wheel 44 May 10 01:30
/tmp/d51f7f4517395797c558a832fcca245513620ade8a553bf4259712be6ac4663c.shm
Here's a diff of a modified /etc/daily /tmp purge portion:
o replace test(1) '-L' with '-h' due to:
-L ... Do not rely on its existence; use -h instead
o don't cd nor find(1) execdir, rather full path find.
o file find stage;-
o read found skip directories for rm(1),
check found item isn't open with fstat.
o securely random pattern overwrite stale files.
o directory find stage;-
o find only empty directories for rmdir(1).
o 5 day stale directories.
o similarily order ignores of .X11-unix, .ICE-unix portslocks.
o also purge stale;-
o pipes.
o sockets.
o dangling symlinks.
Index: daily
===
RCS file: /cvs/src/etc/daily,v
retrieving revision 1.83
diff -u -p -r1.83 daily
--- daily 29 Apr 2015 00:10:44 - 1.83
+++ daily 14 May 2015 15:53:00 -
@@ -45,16 +45,32 @@ start_part Running daily.local:
run_script daily.local
next_part Removing scratch and junk files:
-if [ -d /tmp -a ! -L /tmp ]; then
- cd /tmp {
- find -x . \
- \( -path './ssh-*' -o -path ./.X11-unix -o -path ./.ICE-unix \
- -o -path ./portslocks -o -path './tmux-*' \) \
- -prune -o -type f -atime +7 -execdir rm -f -- {} \; 2/dev/null
- find -x . -type d -mtime +1 ! -path ./vi.recover ! -path ./.X11-unix \
- ! -path ./.ICE-unix ! -path ./portslocks ! -name . \
- -execdir rmdir -- {} \; /dev/null 21; }
-fi
+[[ -d /tmp ! -h /tmp ]]
+{
+ # stale files, not still held open:
+ find -x /tmp \( -path '/tmp/ssh-*' -o -path '/tmp/tmux-*' \
+ -o -path /tmp/.X11-unix -o -path /tmp/.ICE-unix \
+ -o -path /tmp/portslocks \) -prune \
+ -o -type f -atime +7 | while read found
+ do
+ [[ -d ${found} ]] continue
+ fstat ${found} | grep -q ${found}$ || rm -P -- ${found}
+ done
+
+ # stale dangling symlinks:
+ find -Lx /tmp -type l -ctime +14 \
+ -exec rm -- {} \;
+
+ # stale pipes sockets:
+ find -x /tmp \( -type p -o -type s \) -ctime +40 \
+ -exec rm -- {} \;
+
+ # stale directories:
+ find -x /tmp -type d -mtime +5 -empty \
+ ! -name /tmp ! -path /tmp/vi.recover \
+ ! -path /tmp/.X11-unix ! -path /tmp/.ICE-unix \
+ ! -path /tmp/portslocks -exec rmdir -- {} \;
+}
# Additional junk directory cleanup would go like this:
#if [ -d /scratch -a ! -L /scratch ]; then
Would it be a good idea to move the /scratch example out of the
/etc/daily script, and into daily(8), as an example for daily.local?
Re: /etc/daily /tmp purge mods; skip open files with fstat test
On 2015/05/14 17:45, Craig Skinner wrote: $ fstat -u _squid -f /tmp Even not allowing for TOCTOU problems I'd be wary of running fstat automatically.
Re: /etc/daily /tmp purge mods; skip open files with fstat test
On 5/14/15, Craig Skinner skin...@britvault.co.uk wrote:
[...]
Here's a diff of a modified /etc/daily /tmp purge portion:
o replace test(1) '-L' with '-h' due to:
-L ... Do not rely on its existence; use -h instead
Interesting that FreeBSD[1] and MacOS X say the opposite.
SUSv4 (one I have handy) has the exact same text for both
options:
True if pathname resolves to an existing
directory entry for a symbolic link. False if
pathname cannot be resolved, or if pathname
resolves to an existing directory entry for a
file that is not a symbolic link. If the final
component of pathname is a symbolic link, that
symbolic link is not followed
--patrick
[1]
https://www.freebsd.org/cgi/man.cgi?query=testapropos=0sektion=0manpath=FreeBSD+10.1-RELEASEarch=defaultformat=html
o don't cd nor find(1) execdir, rather full path find.
o file find stage;-
o read found skip directories for rm(1),
check found item isn't open with fstat.
o securely random pattern overwrite stale files.
o directory find stage;-
o find only empty directories for rmdir(1).
o 5 day stale directories.
o similarily order ignores of .X11-unix, .ICE-unix portslocks.
o also purge stale;-
o pipes.
o sockets.
o dangling symlinks.
Index: daily
===
RCS file: /cvs/src/etc/daily,v
retrieving revision 1.83
diff -u -p -r1.83 daily
--- daily 29 Apr 2015 00:10:44 - 1.83
+++ daily 14 May 2015 15:53:00 -
@@ -45,16 +45,32 @@ start_part Running daily.local:
run_script daily.local
next_part Removing scratch and junk files:
-if [ -d /tmp -a ! -L /tmp ]; then
- cd /tmp {
- find -x . \
- \( -path './ssh-*' -o -path ./.X11-unix -o -path ./.ICE-unix \
- -o -path ./portslocks -o -path './tmux-*' \) \
- -prune -o -type f -atime +7 -execdir rm -f -- {} \; 2/dev/null
- find -x . -type d -mtime +1 ! -path ./vi.recover ! -path ./.X11-unix \
- ! -path ./.ICE-unix ! -path ./portslocks ! -name . \
- -execdir rmdir -- {} \; /dev/null 21; }
-fi
+[[ -d /tmp ! -h /tmp ]]
+{
+ # stale files, not still held open:
+ find -x /tmp \( -path '/tmp/ssh-*' -o -path '/tmp/tmux-*' \
+ -o -path /tmp/.X11-unix -o -path /tmp/.ICE-unix \
+ -o -path /tmp/portslocks \) -prune \
+ -o -type f -atime +7 | while read found
+ do
+ [[ -d ${found} ]] continue
+ fstat ${found} | grep -q ${found}$ || rm -P -- ${found}
+ done
+
+ # stale dangling symlinks:
+ find -Lx /tmp -type l -ctime +14 \
+ -exec rm -- {} \;
+
+ # stale pipes sockets:
+ find -x /tmp \( -type p -o -type s \) -ctime +40 \
+ -exec rm -- {} \;
+
+ # stale directories:
+ find -x /tmp -type d -mtime +5 -empty \
+ ! -name /tmp ! -path /tmp/vi.recover \
+ ! -path /tmp/.X11-unix ! -path /tmp/.ICE-unix \
+ ! -path /tmp/portslocks -exec rmdir -- {} \;
+}
# Additional junk directory cleanup would go like this:
#if [ -d /scratch -a ! -L /scratch ]; then
Would it be a good idea to move the /scratch example out of the
/etc/daily script, and into daily(8), as an example for daily.local?
