Re: Fun Scripting Problem
On 2013-02-13 12:27, Tim Daneliuk tun...@tundraware.com wrote:
I know how to do this in Python, but I really want to do it in
straight Bourne shell. I have some ideas, but I thought I'd
give you folks a crack at this Big Fun:
a) You have a directory of files - say they're logs - generated
at nondeterministic intervals. You may get more than one a day,
more than one a month, none, or hundreds.
b) To conserve space, you want to keep the last file generated
in any given month (the archive goes back for an unspecified
number of years), and delete all the files generated prior to
that last file in that same month.
c) Bonus points if the problem is solved generally for either files
or directories generated as described above.
These are not actually logs, and no, I don't think logrotate can
do this ... or can it?
You can try using stat/date to print the month in which each file was
modified, e.g:
freefall:/home/keramida$ stat -f '%m %N' prs.tar.bz2
1359910210 prs.tar.bz2
freefall:/home/keramida$ date -f '%s' -j '1359910210' '+%Y-%m'
2013-02
Having the mtime of the file in seconds since the epoch (1359910210)
should be easy to sort through, e.g. you can find the last modification
time with a 'sort -n | tail -1' pipe:
freefall:/home/keramida/w/doc-head/el_GR.ISO8859-7/share$ touch
xml/glossary.ent
freefall:/home/keramida/w/doc-head/el_GR.ISO8859-7/share$ find xml -exec
stat -f '%m %N' {} +
1360060289 xml
1360060288 xml/navibar.l10n.ent
1360060288 xml/trademarks.ent
1360060288 xml/libcommon.xsl
1360060288 xml/header.l10n.ent
= 1360942284 xml/glossary.ent
1360060289 xml/freebsd.dsl
1360060289 xml/teams.ent
1360060289 xml/freebsd.ent
1360060289 xml/l10n.ent
1360060289 xml/mailing-lists.ent
1360060289 xml/newsgroups.ent
1360060289 xml/translators.ent
1360060289 xml/catalog.xml
1360060289 xml/entities.ent
1360060289 xml/catalog
1360060289 xml/urls.ent
freefall:/home/keramida/w/doc-head/el_GR.ISO8859-7/share$ find xml -exec
stat -f '%m %N' {} + | sort -n | tail -1
1360942284 xml/glossary.ent
Then you can convert the epoch-based times to '%Y-%m' timestamps in a
loop, e.g.:
freefall:/home/keramida/w/doc-head/el_GR.ISO8859-7/share$ find xml \
-exec stat -f '%m %N' {} + | while read mtime fname ; do
echo ${mtime} $( date -f '%s' -j ${mtime} '+%Y-%m' ) ${fname}
done
1360060289 2013-02 xml
1360060288 2013-02 xml/navibar.l10n.ent
1360060288 2013-02 xml/trademarks.ent
1360060288 2013-02 xml/libcommon.xsl
1360060288 2013-02 xml/header.l10n.ent
1360942284 2013-02 xml/glossary.ent
1360060289 2013-02 xml/freebsd.dsl
1360060289 2013-02 xml/teams.ent
1360060289 2013-02 xml/freebsd.ent
1360060289 2013-02 xml/l10n.ent
1360060289 2013-02 xml/mailing-lists.ent
1360060289 2013-02 xml/newsgroups.ent
1360060289 2013-02 xml/translators.ent
1360060289 2013-02 xml/catalog.xml
1360060289 2013-02 xml/entities.ent
1360060289 2013-02 xml/catalog
1360060289 2013-02 xml/urls.ent
Having the mtime in seconds as the first column is still conducive to
sorting / tail:
freefall:/home/keramida/w/doc-head/el_GR.ISO8859-7/share$ find xml \
-exec stat -f '%m %N' {} + | while read mtime fname ; do
echo ${mtime} $( date -f '%s' -j ${mtime} '+%Y-%m' ) ${fname};
done | sort -n | tail -1
1360942284 2013-02 xml/glossary.ent
From that point it should be trivial to select all files whose timestamp
is smaller than or equal to 1360942284 - one month of seconds.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fun Scripting Problem
On Wed, Feb 13, 2013 at 03:13:06PM -0600, Robert Bonomi wrote:
here's a one-liner:
rm ` \
stat -f %SB %B %N * \
| sort -k5nr \
| cut -c1-7,17-20,32- \
| awk 'BEGIN {a=;b=0;c=0} $1==a $2==b $3=c {print
$4;}{a=$1;b=$2;c=$3}' \
I'm never comfortable calling something like that a one-liner. If it
runs over 80 columns of width, that (to me) doesn't really qualify as a
one-liner.
--
Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ]
signature.asc
Description: Digital signature
Fun Scripting Problem
I know how to do this in Python, but I really want to do it in straight Bourne shell. I have some ideas, but I thought I'd give you folks a crack at this Big Fun: a) You have a directory of files - say they're logs - generated at nondeterministic intervals. You may get more than one a day, more than one a month, none, or hundreds. b) To conserve space, you want to keep the last file generated in any given month (the archive goes back for an unspecified number of years), and delete all the files generated prior to that last file in that same month. c) Bonus points if the problem is solved generally for either files or directories generated as described above. These are not actually logs, and no, I don't think logrotate can do this ... or can it? -- --- Tim Daneliuk ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: Fun Scripting Problem
(apologies for top-post) As tempted as I am, I think newsyslog(8) may be what you want. Missing information in your post is how you intend to timestamp the files -- by filename? by content? If by-content, then is it a good assumption that the data is one entry per-line? ... and if-so, is the timestamp in that line? These are all questions that would be needed to script what you're asking for (not that I'm volunteering or anything like that). -- Devin From: owner-freebsd-questi...@freebsd.org [owner-freebsd-questi...@freebsd.org] on behalf of Tim Daneliuk [tun...@tundraware.com] Sent: Wednesday, February 13, 2013 10:27 AM To: FreeBSD Mailing List Subject: Fun Scripting Problem I know how to do this in Python, but I really want to do it in straight Bourne shell. I have some ideas, but I thought I'd give you folks a crack at this Big Fun: a) You have a directory of files - say they're logs - generated at nondeterministic intervals. You may get more than one a day, more than one a month, none, or hundreds. b) To conserve space, you want to keep the last file generated in any given month (the archive goes back for an unspecified number of years), and delete all the files generated prior to that last file in that same month. c) Bonus points if the problem is solved generally for either files or directories generated as described above. These are not actually logs, and no, I don't think logrotate can do this ... or can it? -- --- Tim Daneliuk ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fun Scripting Problem
On 02/13/2013 12:38 PM, Teske, Devin wrote: (apologies for top-post) As tempted as I am, I think newsyslog(8) may be what you want. Missing information in your post is how you intend to timestamp the files -- by filename? by content? If by-content, then is it a good assumption that the data is one entry per-line? ... and if-so, is the timestamp in that line? These are all questions that would be needed to script what you're asking for (not that I'm volunteering or anything like that). The only way to determine the date of the file is by looking at its stat info. There is nothing the file name or content that could be used to infer this. -- --- Tim Daneliuk ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fun Scripting Problem
Tim Daneliuk tun...@tundraware.com writes: On 02/13/2013 12:38 PM, Teske, Devin wrote: (apologies for top-post) As tempted as I am, I think newsyslog(8) may be what you want. Missing information in your post is how you intend to timestamp the files -- by filename? by content? If by-content, then is it a good assumption that the data is one entry per-line? ... and if-so, is the timestamp in that line? These are all questions that would be needed to script what you're asking for (not that I'm volunteering or anything like that). The only way to determine the date of the file is by looking at its stat info. There is nothing the file name or content that could be used to infer this. Well, you can use stat to output the year, month, timestamp, and file name in a fixed format for all of the files, sort them, and then cycle through the list, deleting every file that has a year and month that are the same as the following one in the list. The looping can be done with sh read or with sed. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fun Scripting Problem
Date: Wed, 13 Feb 2013 12:27:31 -0600
From: Tim Daneliuk tun...@tundraware.com
Subject: Fun Scripting Problem
I know how to do this in Python, but I really want to do it in
straight Bourne shell. I have some ideas, but I thought I'd
give you folks a crack at this Big Fun:
a) You have a directory of files - say they're logs - generated
at nondeterministic intervals. You may get more than one a day,
more than one a month, none, or hundreds.
b) To conserve space, you want to keep the last file generated
in any given month (the archive goes back for an unspecified
number of years), and delete all the files generated prior to
that last file in that same month.
c) Bonus points if the problem is solved generally for either files
or directories generated as described above.
These are not actually logs, and no, I don't think logrotate can
do this ... or can it?
here's a one-liner:
rm ` \
stat -f %SB %B %N * \
| sort -k5nr \
| cut -c1-7,17-20,32- \
| awk 'BEGIN {a=;b=0;c=0} $1==a $2==b $3=c {print
$4;}{a=$1;b=$2;c=$3}' \
`
This selects on creation date. change the B (both of them) in the stat
call to use a different timestamp
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fun Scripting Problem
On Wed, 13 Feb 2013 12:53:32 -0600,
Tim Daneliuk tun...@tundraware.com said:
T The only way to determine the date of the file is by looking at its stat
T info. There is nothing the file name or content that could be used to
T infer this.
Being a pedantic twit, I interpreted stat info to mean info you can
get from something that calls stat(). The script below runs on BSD,
Linux, or Solaris if you have GNU find installed. Season to taste.
--
Karl Vogel I don't speak for the USAF or my company
Mom was so overprotective, she only let us play Rock, Paper.
---
#!/bin/ksh
# usage: keepfiles DIR
export PATH=/usr/local/bin:/bin:/usr/bin
tag=${0##*/}
here=$(pwd)
tmp=$here/keep$$
# Sanity check.
die () { echo $tag: FATAL: $@ 2; exit 1; }
# Don't repeat yourself. Use undocumented option %Ts to
# avoid dealing with fractional seconds.
listdir () { find . -type f -printf %TY%Tm%Td %Ts|%p\n; }
case $# in
0) die need a directory ;;
*) work=$1 ;;
esac
# List files by date generated.
mkdir $tmp|| die $tmp: mkdir failed
test -d $work || die $work: not a directory
cd $work || die cd $work failed
months=$(listdir | cut -c1-6 | sort -u)
for mon in $months
do
listdir | # Get the files to check,
grep ^$mon | # ... look for each month,
sort -n | # ... sort by modtime,
cut -d'|' -f2 | # ... get the path,
tail -1 | # ... last one's oldest,
sed -e 's!^\(.*\)!mv \1 '$tmp'!' | # ... write a mv command,
sh # ... and run it.
done
# Clean up.
cd $here
echo Keep the stuff in $tmp
exit 0
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fun Scripting Problem
On 02/13/2013 03:13 PM, Robert Bonomi wrote:
Date: Wed, 13 Feb 2013 12:27:31 -0600
From: Tim Daneliuk tun...@tundraware.com
Subject: Fun Scripting Problem
I know how to do this in Python, but I really want to do it in
straight Bourne shell. I have some ideas, but I thought I'd
give you folks a crack at this Big Fun:
a) You have a directory of files - say they're logs - generated
at nondeterministic intervals. You may get more than one a day,
more than one a month, none, or hundreds.
b) To conserve space, you want to keep the last file generated
in any given month (the archive goes back for an unspecified
number of years), and delete all the files generated prior to
that last file in that same month.
c) Bonus points if the problem is solved generally for either files
or directories generated as described above.
These are not actually logs, and no, I don't think logrotate can
do this ... or can it?
here's a one-liner:
rm ` \
stat -f %SB %B %N * \
| sort -k5nr \
| cut -c1-7,17-20,32- \
| awk 'BEGIN {a=;b=0;c=0} $1==a $2==b $3=c {print
$4;}{a=$1;b=$2;c=$3}' \
`
This selects on creation date. change the B (both of them) in the stat
call to use a different timestamp
Thanks to all that took the time. Interesting responses. It will
be fun to cook up my own version.
--
Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
