Re: grep, maybe
After a night's sleep, I realized I might even be able to make Ben happy:
#!/bin/sh
cd /path/to/toplevel/dir
find -type d | while read i
do
grep moe $i/* /dev/null echo $i
done | while read d
do
mv $d /path/to/destination || echo mv for $d didn't work: $?
done
-Ken
P.S. It deals gracefully with spaces (I checked -- hadn't been sure about
how the second read would work).
P.P.S. It does appear that find is terminated for the second loop. (Or
so said the ps I threw into it to check.)
P.^3S. For millions of hits, your memory might start to bog. In which
case, if you expect that, it's definitely time to go with the temporary
file thing.
On Thu, October 29, 2009 1:31 pm, mark wrote:
On Thu, Oct 29, 2009 at 11:38 AM, Maurice
mauri...@cds-cumberland.orgwrote:
Looking for some guidance;
I have several files within several folders (5 files per folder, and
thousands of folders) that I need to search a text file within each
folder for a word match (like three_little_pigs.txt, and I need to find
moe, if he's listed) and then when a match is found I need to move
(not copy) that entire folder (and it's 3~5 files contained within) to
another location...
#!/bin/sh
cd [top level directory] grep -l -r [search string in double quotes]
/tmp/file_names_found
2/tmp/grep.errs
cat /tmp/file_names_found|xargs -i basename {} /tmp/dir_names_to_move
dir_list=`sort /tmp/dir_names_to_move|unique` for DIRECTORY in
$(dir_list); do
mvdir $DIRECTORY [new location here] STATUS=$?
if [ $STATUS -ne 0 ]; then echo mvdir returned status $STATUS end done
exit
You'll have to come up with a way to create the value for new location
here that is unique so you don't overlay all the directories into the
same path and lose everything, so maybe test it with a copy first. You'll
also want to check /tmp/grep.errs for any error messages. I hope this
helps.
mark
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is believed to be clean.
___
gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
On Fri, Oct 30, 2009 at 8:15 AM, Ken D'Ambrosio k...@jots.org wrote: ... might even be able to make Ben happy ... Oh, a challenge, eh? ;-) cd /path/to/toplevel/dir find -type d | while read i Could be just: find /path/to/toplevel/dir -type d | while read i grep moe $i/* /dev/null echo $i Definitely should be at least: grep moe $i/* /dev/null echo $i (Putting the star inside the quotes prevents the shell from globing it, thus leading to grep trying to open a file named * in the directory.) Could be: grep -q moe $i/* echo $i (The -q switch to grep causes it to just exit with true on the first match, making things faster if the match is earlier in the file. Also avoids the need to throw away the output.) P.S. It deals gracefully with spaces (I checked -- hadn't been sure about how the second read would work). Yah, read reads lines. Lines are split into tokens on spaces and put into parameters you give, but the last parameter gets the rest of the line. So with one parameter, you get the whole line. P.P.S. It does appear that find is terminated for the second loop. (Or so said the ps I threw into it to check.) Cool. Thanks for checking. And for letting us know. :) P.^3S. For millions of hits, your memory might start to bog. In which case, if you expect that, it's definitely time to go with the temporary file thing. Or increase your swap space. ;-) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
grep -r greps all files recursively. grep -l outputs only the names of files which contain matching text. To move the folders, you would have to process that output to select the directory, then move the directory. Probably a perl or shell scripting task. AFAIK grep has nothing so specific as moving the folders the files are in, but it gets you 80% of the way there. --DTVZ On Thu, Oct 29, 2009 at 11:38 AM, Maurice mauri...@cds-cumberland.orgwrote: Looking for some guidance; I have several files within several folders (5 files per folder, and thousands of folders) that I need to search a text file within each folder for a word match (like three_little_pigs.txt, and I need to find moe, if he's listed) and then when a match is found I need to move (not copy) that entire folder (and it's 3~5 files contained within) to another location... I'm thinking grep, but don't know the correct syntax to make all this happen. I can easily find all the folders (1949 of them) and the word match 3923 times within the text file(s)... Any ideas??? -- -Maurice Pelletier Child Development Services - Cumberland County 50 Depot Road Falmouth, ME 04105 207-781-8881 (voice) 207-781-8855 (fax) www.cds-cumberland.org Linux -- it's not just for breakfast anymore... -Moe ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
On Thu, Oct 29, 2009 at 11:38 AM, Maurice mauri...@cds-cumberland.org wrote: I need to search a text file within each folder for a word match (like three_little_pigs.txt, and I need to find moe, if he's listed) and then when a match is found I need to move (not copy) that entire folder (and it's 3~5 files contained within) to another location... Well, the following will search for any file containing moe (exactly), and list matching files: grep -l -r moe /path/to/top/level/directory If you want a case-insensitive search (match moe, Moe, MoE, etc.): grep -i -l -r moe /path/to/top/level/directory If you want to search for multiple names, separate them by vertical bars (|). You will have to quote the search string to keep the bars from being interpreted by the shell: grep -i -l -r 'moe|larry|curly' /path/to/top/level/directory Moving just the files would be pretty easy. Moving the *directories* would be trickier. We can't move it when grep is still in the directory, so we'll have to save a list somewhere, then parse the list to find the directory names, then eliminate the duplicates, then move the directories. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
On Thu, Oct 29, 2009 at 11:38 AM, Maurice mauri...@cds-cumberland.orgwrote:
Looking for some guidance;
I have several files within several folders (5 files per folder, and
thousands of folders) that I need to search a text file within each
folder for a word match (like three_little_pigs.txt, and I need to find
moe, if he's listed) and then when a match is found I need to move
(not copy) that entire folder (and it's 3~5 files contained within) to
another location...
#!/bin/sh
cd [top level directory]
grep -l -r [search string in double quotes] /tmp/file_names_found
2/tmp/grep.errs
cat /tmp/file_names_found|xargs -i basename {} /tmp/dir_names_to_move
dir_list=`sort /tmp/dir_names_to_move|unique`
for DIRECTORY in $(dir_list); do
mvdir $DIRECTORY [new location here]
STATUS=$?
if [ $STATUS -ne 0 ]; then
echo mvdir returned status $STATUS
end
done
exit
You'll have to come up with a way to create the value for new location
here that is unique so you don't overlay all the directories into the same
path and lose everything, so maybe test it with a copy first. You'll also
want to check /tmp/grep.errs for any error messages. I hope this helps.
mark
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
mark prg...@gmail.com writes:
On Thu, Oct 29, 2009 at 11:38 AM, Maurice mauri...@cds-cumberland.org wrote:
Looking for some guidance;
I have several files within several folders (5 files per folder, and
thousands of folders) that I need to search a text file within each
folder for a word match (like three_little_pigs.txt, and I need to find
moe, if he's listed) and then when a match is found I need to move
(not copy) that entire folder (and it's 3~5 files contained within) to
another location...
#!/bin/sh
cd [top level directory]
grep -l -r [search string in double quotes] /tmp/file_names_found 2/tmp/
grep.errs
cat /tmp/file_names_found|xargs -i basename {} /tmp/dir_names_to_move
dir_list=`sort /tmp/dir_names_to_move|unique`
for DIRECTORY in $(dir_list); do
mvdir $DIRECTORY [new location here]
STATUS=$?
if [ $STATUS -ne 0 ]; then
echo mvdir returned status $STATUS
end
done
exit
I'd probably go with something like:
grep --recursive --files-with-matches $searchstring $topdir \
| xargs --max-args=1 dirname \
| sort --unique \
| xargs mv --target-directory=$newloc
The only thing that really sucks about that is that the involvement of
`sort' means that the mv needs to wait until *all* of the files have
been grep'd, instead of just moving the directory as soon as *any*
file in it is found to contain the search-term.
And, depending on whether your directories vary in the depth of their
nesting, and whether you prefer to move the more- or less-deep
directories first (or at all), you may need to add a --reverse to
the `sort' line or do more complicated filtering with an additional
grep (on the *paths*) or sed.
--
Don't be afraid to ask (Lf.((Lx.xx) (Lr.f(rr.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
Oooh! A challenge! Here's my solution:
#!/bin/sh
cd /path/to/toplevel/dir
find -type d | while read i
do
grep moe $i/* mv $i /path/to/destination || echo mv didn't work: $?
done
-Ken
On Thu, October 29, 2009 1:31 pm, mark wrote:
On Thu, Oct 29, 2009 at 11:38 AM, Maurice
mauri...@cds-cumberland.orgwrote:
Looking for some guidance;
I have several files within several folders (5 files per folder, and
thousands of folders) that I need to search a text file within each
folder for a word match (like three_little_pigs.txt, and I need to find
moe, if he's listed) and then when a match is found I need to move
(not copy) that entire folder (and it's 3~5 files contained within) to
another location...
#!/bin/sh
cd [top level directory] grep -l -r [search string in double quotes]
/tmp/file_names_found
2/tmp/grep.errs
cat /tmp/file_names_found|xargs -i basename {} /tmp/dir_names_to_move
dir_list=`sort /tmp/dir_names_to_move|unique` for DIRECTORY in
$(dir_list); do
mvdir $DIRECTORY [new location here] STATUS=$?
if [ $STATUS -ne 0 ]; then echo mvdir returned status $STATUS end done
exit
You'll have to come up with a way to create the value for new location
here that is unique so you don't overlay all the directories into the
same path and lose everything, so maybe test it with a copy first. You'll
also want to check /tmp/grep.errs for any error messages. I hope this
helps.
mark
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is believed to be clean.
___
gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
On Thu, Oct 29, 2009 at 2:31 PM, Ken D'Ambrosio k...@jots.org wrote: find -type d | while read i do grep moe $i/* mv $i /path/to/destination || echo mv didn't work: $? done Hmmm, does the find execute concurrently with the grep? If it does, then you're liable to confuse the hell out of find if you manage to move a directory which is a component of the path it's currently trasversing. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
On Thu, Oct 29, 2009 at 2:20 PM, Joshua Judson Rosen roz...@geekspace.com wrote: grep --recursive --files-with-matches $searchstring $topdir \ | xargs --max-args=1 dirname \ | sort --unique \ | xargs mv --target-directory=$newloc I like it. I didn't know about the --target-directory option to mv(1). That'll come in handy in the future. Thanks... Hmmm, file names with spaces are likely to foul things; xargs splits on any whitespace by default. Maybe: grep --recursive --files-with-matches $searchstring $topdir \ | xargs --delimiter=\\n --max-args=1 dirname \ | sort --unique \ | xargs --delimiter=\\n mv --target-directory=$newloc Explicitly specifying the delimiter as newline means newline only. Whitespace within a line is ignored. (It will still fail if a file name contains a *newline*, but that's pathological, while file names with spaces are quite common.) Don't be afraid to ask (Lf.((Lx.xx) (Lr.f(rr. Okay, I'll ask: What does that stuff to the right mean? Some kind of LISP? -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
Interesting point. Three replies: 1) If you're mv'ing onto the same filesystem, the inodes will never change, and it won't matter. 2) I imagine grep will only spit out a status (which is what the parses) after it's finished running, though I'd have to verify that empirically. 3) You can cheat, and do something like this: cat $i/* | grep -l moe mv $i /destpath || echo mv didn't work: $? -Ken On Thu, October 29, 2009 4:41 pm, Ben Scott wrote: On Thu, Oct 29, 2009 at 2:31 PM, Ken D'Ambrosio k...@jots.org wrote: find -type d | while read i do grep moe $i/* mv $i /path/to/destination || echo mv didn't work: $? done Hmmm, does the find execute concurrently with the grep? If it does, then you're liable to confuse the hell out of find if you manage to move a directory which is a component of the path it's currently trasversing. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
Wups! Sorry -- I read your question wrong: Does *find* run concurrently*. I'd had other concerns, and mis-read your question to fit my thinking. D'oh! It's been my experience that find doesn't get confused, it just gets miffed, and moves on. -Ken On Thu, October 29, 2009 5:03 pm, Ken D'Ambrosio wrote: Interesting point. Three replies: 1) If you're mv'ing onto the same filesystem, the inodes will never change, and it won't matter. 2) I imagine grep will only spit out a status (which is what the parses) after it's finished running, though I'd have to verify that empirically. 3) You can cheat, and do something like this: cat $i/* | grep -l moe mv $i /destpath || echo mv didn't work: $? -Ken On Thu, October 29, 2009 4:41 pm, Ben Scott wrote: On Thu, Oct 29, 2009 at 2:31 PM, Ken D'Ambrosio k...@jots.org wrote: find -type d | while read i do grep moe $i/* mv $i /path/to/destination || echo mv didn't work: $? done Hmmm, does the find execute concurrently with the grep? If it does, then you're liable to confuse the hell out of find if you manage to move a directory which is a component of the path it's currently trasversing. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: grep, maybe
Ben Scott dragonh...@gmail.com writes: On Thu, Oct 29, 2009 at 2:20 PM, Joshua Judson Rosen roz...@geekspace.com wrote: Don't be afraid to ask (Lf.((Lx.xx) (Lr.f(rr. Okay, I'll ask: What does that stuff to the right mean? The other half of the whole habanero pepper. :) More lucidly: a combinator. ;) It's a pun. Actually, somehow this never occurred to me before, but since the sentence as a whole is `an imperative', and the `question' to be asked is functional, I guess it's... a functional imperative? So maybe it's a deeper (or worse) joke than I originally intended Some kind of LISP? Almost. Did you have any luck googling for it? :) -- Don't be afraid to ask (Lf.((Lx.xx) (Lr.f(rr. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
