Re: [PATCH] Add new program: magic
Bo Borgerson wrote: > As I thought more about this functionality I realized that it may be > more broadly useful. Any utility that can operate on multiple input > files could benefit. I wondered if it would be possible in a > non-invasive way to provide this service to other tools. This is what > I came up with. > > Instead of: > $ sort --magic-open a.gz b.bz2 c.txt > > I run: > $ magic sort a.gz b.bz2 c.txt I like this sort of general purpose utility that can work with a broad set of things much better than hacks to every utility. It is definitely a better direction. But I don't like the name. The name is too generic and doesn't give a clue as to what it actually does. This is probably better to name something like daylight-commander or some such (with apologies to Nortan and midnight-commander). Additionally I have to note that bash (and probably other shells) already supply this capability in a generic way. sort <(zcat a) <(zcat b) c > Then I realized that this automatic fifo management might be more > useful still. In addition to checking the `magic' bytes at the > beginning of regular files for known decompression programs, I thought > it might be useful to allow an arbitrary sub-command to be used as an > input. This is getting to be too heuristic driven (too error prone) for my tastes. > For example, to compare the output of two versions of a program: > $ magic diff "ls -l" "src/ls -l" diff <(ls -l) <(./src/ls -l) > Or to compare files on two remote machines: > $ magic diff "ssh host1 cat /etc/passwd" "ssh host2 cat /etc/passwd" diff <(ssh -n host1 cat /etc/passwd) <(ssh -n host2 cat /etc/passwd) Bob ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH] Add new program: magic
Hi Bo, > As I mentioned last week, I've patched my local `sort' to allow > automatic decompression of input files if an option, --magic-open, is > passed on the command line. > > As I thought more about this functionality I realized that it may be > more broadly useful. Any utility that can operate on multiple input > files could benefit. I wondered if it would be possible in a > non-invasive way to provide this service to other tools. This is what > I came up with. > > Instead of: > $ sort --magic-open a.gz b.bz2 c.txt > > I run: > $ magic sort a.gz b.bz2 c.txt This sounds very similar to the zrun command from the moreutils package: | This is a growing collection of the unix tools that nobody thought | to write thirty years ago. | | So far, it includes the following utilities: | - isutf8: check if a file or standard input is utf-8 | - sponge: soak up standard input and write to a file | - ts: timestamp standard input | - vidir: edit a directory in your text editor | - vipe: insert a text editor into a pipe | - combine: combine the lines in two files using boolean operations | - ifdata: get network interface info without parsing ifconfig output | - pee: tee standard input to pipes | - zrun: automatically uncompress arguments to command | - mispipe: pipe two commands, returning the exit status of the first | - lckdo: execute a program with a lock held Regards - Peter Edwards ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH] Add new program: magic
[EMAIL PROTECTED] wrote: | So far, it includes the following utilities: | - sponge: soak up standard input and write to a file Eh? That sounds like 'cat > file'... -- Matthew Look! A randomly selected yet highly appropriate signature! ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH] Add new program: magic
On Wed, Apr 9, 2008 at 8:52 PM, Matthew Woehlke <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: > > > > > | So far, it includes the following utilities: > > > > | - sponge: soak up standard input and write to a file > > > > Eh? That sounds like 'cat > file'... So you can read from and write to an existing file. Much better solution that the prepend solution I had suggested. cat a b | sponge b Brock ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH] Add new program: magic
Matthew Woehlke wrote: > [EMAIL PROTECTED] wrote: >> | So far, it includes the following utilities: >> | - sponge: soak up standard input and write to a file > > Eh? That sounds like 'cat > file'... I was wondering that too. It basically buffers the file in mem so that one can modify a file through a filter without the need for explicit temporary files. Pádraig. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH] Add new program: magic
On Wed, Apr 9, 2008 at 9:48 PM, Bob Proulx <[EMAIL PROTECTED]> wrote: > I like this sort of general purpose utility that can work with a broad > set of things much better than hacks to every utility. It is > definitely a better direction. > > But I don't like the name. The name is too generic and doesn't give a > clue as to what it actually does. This is probably better to name > something like daylight-commander or some such (with apologies to > Nortan and midnight-commander). Point taken :) I knew I would need to change the name. I should have done so before submitting the draft. Do you have any suggestions for a more descriptive name? How about 'fargs', as a contraction of fifo-args? I'll use that at least for the duration of this message. > Additionally I have to note that bash (and probably other shells) > already supply this capability in a generic way. > > sort <(zcat a) <(zcat b) c Yeah, bash is great! There are some differences, though: $ fargs wc input/* 3141895 12183 /tmp/fargsBsoaWi/HACKING.gz 5601796 31786 /tmp/fargsBsoaWi/THANKS.bz2 177 9766908 input/TODO 10514667 50877 total $ wc <(zcat input/HACKING.gz) <(bzcat input/THANKS.bz2) input/TODO 3141895 12183 /dev/fd/63 5601796 31786 /dev/fd/62 177 9766908 input/TODO 10514667 50877 total First, with the bash syntax you need to enumerate the set of commands you want to run. Second, for programs that use filenames in output or diagnostic messages, the fifos produced by fargs may be somewhat more legible. Also, so far as I know there isn't any way currently to use --files0-from=F style argument passing, as with `wc' (and pending in `sort'). There may be a trick here that I just haven't learned yet, but I'd like to be able to do the following (more so with `sort', but I'll use `wc' for illustration). $ find input/ -type f --files0 | fargs wc --files0-from=- 177 9766908 input/TODO 3141895 12183 /tmp/fargsLywSy2/HACKING.gz 5601796 31786 /tmp/fargsLywSy2/THANKS.bz2 10514667 50877 total > This is getting to be too heuristic driven (too error prone) for my > tastes. The precedence is `directive>option>file>command'. There are some cases where the interpretation may be surprising. If you have a file in your current working directory named 'ls' and you intend to use 'ls' as a sub-command, then: $ fargs wc ls Won't give you what you want. For this reason I included the 'exec:' directive: $ fargs wc exec:ls If you manage to produce a compressed file named '-l' and you run: $ fargs wc -l You could be waiting for a long time. For this reason I included the 'file:' directive: $ fargs wc file:-l Finally, because `fargs' doesn't know which options of the invoked command take arguments, if you run: $ fargs sort -o output input/* and `output' already exists, you may be in for a surprise. For this reason I included the 'skip:' directive: $ fargs sort -o skip:output input/* Of course this last case could be avoided by using the long-option form: $ fargs sort --output=output input/* Note that regardless of whether arbitrary sub-commands are allowed, this last case is an issue. > diff <(ssh -n host1 cat /etc/passwd) <(ssh -n host2 cat /etc/passwd) That one really looked like a nail when I'd just finished building my shiny new hammer. ;) Thanks, Bo ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
