Hi,
As I mentioned last week, I've patched my local `sort' to allow
automatic decompression of input files if an option, --magic-open, is
passed on the command line.
As I thought more about this functionality I realized that it may be
more broadly useful. Any utility that can operate on multiple input
files could benefit. I wondered if it would be possible in a
non-invasive way to provide this service to other tools. This is what
I came up with.
Instead of:
$ sort --magic-open a.gz b.bz2 c.txt
I run:
$ magic sort a.gz b.bz2 c.txt
This creates a temporary fifo and opens a decompression program for
each compressed input. The corresponding files in the argument list
are replaced by these temporary fifos. When the command completes,
the fifos are removed. If a signal is received and execution is
terminated prematurely, the fifos are removed.
Then I realized that this automatic fifo management might be more
useful still. In addition to checking the `magic' bytes at the
beginning of regular files for known decompression programs, I thought
it might be useful to allow an arbitrary sub-command to be used as an
input.
For example, to compare the output of two versions of a program:
$ magic diff "ls -l" "src/ls -l"
Or to compare files on two remote machines:
$ magic diff "ssh host1 cat /etc/passwd" "ssh host2 cat /etc/passwd"
I've attached a basic implementation of this tool. It supports both
magic-number-based auto-decompression of known formats and sub-command
inputs through auto-maintained fifos. It supports both command-line
argument interpretation and --files0-from=F interpretation. It
supports explicit directives to override interpretation precedence.
Is this something that might be worth including in coreutils?
Thanks,
Bo
From 0a16d0698590f137e04c8351f8f14383147e827f Mon Sep 17 00:00:00 2001
From: Bo Borgerson <[EMAIL PROTECTED]>
Date: Sun, 6 Apr 2008 17:54:08 -0400
Subject: [PATCH] Add new program: magic
* AUTHORS: Register as the author.
* NEWS: Advertise new program.
* README: List new program.
* doc/coreutils.texi: Describe new program.
* man/Makefile.am: Add new program.
* man/magic.x: Add new man page template.
* po/POTFILES.in: Add new program.
* src/Makefile.am: Add new program.
* src/magic.c: Add new program.
* tests/misc/Makefile.am: Add new test.
* tests/misc/help-version: Accomodate new program.
* tests/misc/magic: Test new program.
Signed-off-by: Bo Borgerson <[EMAIL PROTECTED]>
---
AUTHORS | 1 +
NEWS | 3 +
README | 2 +-
doc/coreutils.texi | 73 ++++-
man/Makefile.am | 1 +
man/magic.x | 4 +
po/POTFILES.in | 1 +
src/Makefile.am | 2 +-
src/magic.c | 978 +++++++++++++++++++++++++++++++++++++++++++++++
tests/misc/Makefile.am | 1 +
tests/misc/help-version | 1 +
tests/misc/magic | 89 +++++
12 files changed, 1152 insertions(+), 4 deletions(-)
create mode 100644 man/magic.x
create mode 100644 src/magic.c
create mode 100755 tests/misc/magic
diff --git a/AUTHORS b/AUTHORS
index 807857f..a79bec3 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -42,6 +42,7 @@ link: Michael Stone
ln: Mike Parker, David MacKenzie
logname: FIXME: unknown
ls: Richard Stallman, David MacKenzie
+magic: Bo Borgerson
md5sum: Ulrich Drepper, Scott Miller, David Madore
mkdir: David MacKenzie
mkfifo: David MacKenzie
diff --git a/NEWS b/NEWS
index e208b30..48affd5 100644
--- a/NEWS
+++ b/NEWS
@@ -82,6 +82,9 @@ GNU coreutils NEWS -*- outline -*-
Fix a non-portable use of sed in configure.ac.
[bug introduced in coreutils-6.9.92]
+** New programs
+
+magic: run a program with multiple piped inputs
* Noteworthy changes in release 6.9.92 (2008-01-12) [beta]
diff --git a/README b/README
index 7a608f4..548b832 100644
--- a/README
+++ b/README
@@ -10,7 +10,7 @@ The programs that can be built with this package are:
[ arch base64 basename cat chcon chgrp chmod chown chroot cksum comm cp
csplit cut date dd df dir dircolors dirname du echo env expand expr
factor false fmt fold groups head hostid hostname id install join kill
- link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl nohup
+ link ln logname ls magic md5sum mkdir mkfifo mknod mktemp mv nice nl nohup
od paste pathchk pinky pr printenv printf ptx pwd readlink rm rmdir
runcon seq sha1sum sha224sum sha256sum sha384sum sha512sum shred shuf
sleep sort split stat stty su sum sync tac tail tee test touch tr true
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 5a6f2c3..274606b 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -75,6 +75,7 @@
* ln: (coreutils)ln invocation. Make links between files.
* logname: (coreutils)logname invocation. Print current login name.
* ls: (coreutils)ls invocation. List directory contents.
+* magic: (coreutils)magic invocation. Run with piped inputs.
* md5sum: (coreutils)md5sum invocation. Print or check MD5 digests.
* mkdir: (coreutils)mkdir invocation. Create directories.
* mkfifo: (coreutils)mkfifo invocation. Create FIFOs (named pipes).
@@ -190,7 +191,7 @@ Free Documentation License''.
* Working context:: pwd stty printenv tty
* User information:: id logname whoami groups users who
* System context:: date uname hostname hostid
-* Modified command invocation:: chroot env nice nohup su
+* Modified command invocation:: chroot env magic nice nohup su
* Process control:: kill
* Delaying:: sleep
* Numeric operations:: factor seq
@@ -418,6 +419,7 @@ Modified command invocation
* chroot invocation:: Run a command with a different root directory
* env invocation:: Run a command in a modified environment
+* magic invocation:: Run a command with piped inputs
* nice invocation:: Run a command with modified niceness
* nohup invocation:: Run a command immune to hangups
* su invocation:: Run a command with substitute user and group ID
@@ -678,7 +680,7 @@ However, some of the programs documented here do produce
other exit status values and a few associate different
meanings with the values @samp{0} and @samp{1}.
Here are some of the exceptions:
[EMAIL PROTECTED], @command{env}, @command{expr},
[EMAIL PROTECTED], @command{env}, @command{expr}, @command{magic},
@command{nice}, @command{nohup}, @command{printenv},
@command{sort}, @command{su}, @command{test}, @command{tty}.
@@ -13355,6 +13357,7 @@ user, etc.
@menu
* chroot invocation:: Modify the root directory.
* env invocation:: Modify environment variables.
+* magic invocation:: Pipe inputs as if they were files.
* nice invocation:: Modify niceness.
* nohup invocation:: Immunize to hangups.
* su invocation:: Modify user and group ID.
@@ -13509,6 +13512,72 @@ Exit status:
the exit status of @var{command} otherwise
@end display
[EMAIL PROTECTED] magic invocation
[EMAIL PROTECTED] @command{magic}: Run a command with piped inputs
+
[EMAIL PROTECTED] magic
[EMAIL PROTECTED] fifos, piped inputs, run a command with piped
[EMAIL PROTECTED] decompression, automatic fifo
+
[EMAIL PROTECTED] runs a command with a modified environment.
+Synopsis:
+
[EMAIL PROTECTED]
+magic [EMAIL PROTECTED]@dots{} @var{command} @var{arg} [EMAIL PROTECTED]@dots{}
[EMAIL PROTECTED] example
+
[EMAIL PROTECTED] to @var{command} are examined and modified as follows.
+For any @var{arg} that is a regular file that appears to be compressed, a
+fifo is created and attached to an appropriate decompression program.
+That @var{arg} is replaced with the name of the fifo.
+For any @var{arg} that appears to be a sub-command, a fifo is created and
+attached to the sub-command. That @var{arg} is replaced with the name of
+the fifo.
+All temporary fifos are cleaned up after @var{command} finishes.
+
+If @var{command} is invoked with the special option --files0-from=F, then
+additional @var{arg}s will interpreted from @var{f}, and piped through a
+fifo to @var{command}.
+Any other @var{arg} that appears to be an option is left unmodified.
+
+Interpretation of @var{arg}s can be affected by prefixing with one of three
+explicit directives:
+
[EMAIL PROTECTED]
+ `skip:' Pass this @var{arg} through unaffected
+ `file:' Treat this @var{arg} as a file (even if it looks like an option)
+ `exec:' Treat this @var{arg} as a sub-command
+ (even if there's a file with the same name)
[EMAIL PROTECTED] example
+
+The program accepts the following options. Also see @ref{Common options}.
+Options must precede operands.
+
[EMAIL PROTECTED] @samp
+
[EMAIL PROTECTED] -T @var{dir}
[EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] -T
[EMAIL PROTECTED] --temporary-directory
+put temporary directories in @var{dir}, instead of @var{$TMPDIR} or /tmp.
+
[EMAIL PROTECTED] -M @var{MAX}
[EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] -M
[EMAIL PROTECTED] --max-directory-entries
+Put at most @var{max} fifos in any directory. A new temporary directory
+will be created for each @var{max} fifos.
+
[EMAIL PROTECTED] table
+
[EMAIL PROTECTED] exit status of @command{magic}
+Exit status:
+
[EMAIL PROTECTED]
+1 if @command{magic} initialization fails
+the exit status of @var{command} otherwise
[EMAIL PROTECTED] display
+
@node nice invocation
@section @command{nice}: Run a command with modified niceness
diff --git a/man/Makefile.am b/man/Makefile.am
index 7164b3b..56c7664 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -71,6 +71,7 @@ link.1: $(common_dep) $(srcdir)/link.x ../src/link.c
ln.1: $(common_dep) $(srcdir)/ln.x ../src/ln.c
logname.1: $(common_dep) $(srcdir)/logname.x ../src/logname.c
ls.1: $(common_dep) $(srcdir)/ls.x ../src/ls.c
+magic.1: $(common_dep) $(srcdir)/magic.x ../src/magic.c
md5sum.1: $(common_dep) $(srcdir)/md5sum.x ../src/md5sum.c
mkdir.1: $(common_dep) $(srcdir)/mkdir.x ../src/mkdir.c
mkfifo.1: $(common_dep) $(srcdir)/mkfifo.x ../src/mkfifo.c
diff --git a/man/magic.x b/man/magic.x
new file mode 100644
index 0000000..3427df3
--- /dev/null
+++ b/man/magic.x
@@ -0,0 +1,4 @@
+[NAME]
+magic \- automatically decompress input files
+[DESCRIPTION]
+.\" Add any additional description here
diff --git a/po/POTFILES.in b/po/POTFILES.in
index e975109..019747d 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -73,6 +73,7 @@ src/link.c
src/ln.c
src/logname.c
src/ls.c
+src/magic.c
src/md5sum.c
src/mkdir.c
src/mkfifo.c
diff --git a/src/Makefile.am b/src/Makefile.am
index 44d802e..db584b7 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -30,7 +30,7 @@ EXTRA_PROGRAMS = \
$(no_install__progs) \
$(build_if_possible__progs) \
[ chcon chgrp chown chmod cp dd dircolors du \
- ginstall link ln dir vdir ls mkdir \
+ ginstall link ln dir vdir ls magic mkdir \
mkfifo mknod mktemp \
mv nohup readlink rm rmdir shred stat sync touch unlink \
cat cksum comm csplit cut expand fmt fold head join groups md5sum \
diff --git a/src/magic.c b/src/magic.c
new file mode 100644
index 0000000..7813473
--- /dev/null
+++ b/src/magic.c
@@ -0,0 +1,978 @@
+/* magic -- read from compressed files and pipelines
+ Copyright (C) 2008 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+
+/* magic - run a program with multiple piped inputs
+
+ Written by Bo Borgerson. */
+
+#include <config.h>
+#include <getopt.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <signal.h>
+#include "system.h"
+#include "error.h"
+#include "long-options.h"
+#include "readtokens0.h"
+#include "quote.h"
+#include "xstrtol.h"
+
+#define PROGRAM_NAME "magic"
+
+#define AUTHORS "Bo Borgerson"
+
+#define GZIP_MAGIC "\037\213" /* Magic for gzip files. */
+#define GZIP_MAGIC_OLD "\037\236" /* Magic for old gzip files. */
+#define BZIP_MAGIC "\102\132" /* Magic for bzip files. */
+
+#ifndef DEFAULT_TMPDIR
+# define DEFAULT_TMPDIR "/tmp"
+#endif
+
+/* Use SA_NOCLDSTOP as a proxy for whether the sigaction machinery is
+ present. */
+#ifndef SA_NOCLDSTOP
+# define SA_NOCLDSTOP 0
+/* No sigprocmask. Always 'return' zero. */
+# define sigprocmask(How, Set, Oset) (0)
+# define sigset_t int
+# if ! HAVE_SIGINTERRUPT
+# define siginterrupt(sig, flag) /* empty */
+# endif
+#endif
+
+#define FILES_FROM_OPT "--files0-from="
+#define FILES_FROM_OPT_LEN strlen (FILES_FROM_OPT);
+
+/* By default we won't put more than this many fifos in a single directory.
+ Can be overridden on the command-line with the --max-directory-entries
+ (-M) option. */
+#define DEFAULT_MAX_DIRENTS 128
+
+#define SECONDS_BETWEEN_REAPS 4
+
+static char *program_name;
+
+static struct option const long_options[] = {
+ {"temporary-directory", required_argument, NULL, 'T'},
+ {"max-directory-entries", required_argument, NULL, 'M'},
+ {GETOPT_HELP_OPTION_DECL},
+ {GETOPT_VERSION_OPTION_DECL},
+ {NULL, 0, NULL, 0}
+};
+
+static void
+usage (int status)
+{
+ if (status != EXIT_SUCCESS)
+ fprintf (stderr, _("Try `%s --help' for more information.\n"),
+ program_name);
+ else
+ {
+ printf (_("\
+Usage: %s [OPTION]... COMMAND ARG [ARG]...\n\
+ Or: %s [OPTION]... COMMAND [OPTION]... --fils0-from=F\n\
+"), program_name, program_name);
+
+ fputs (_("\
+Examine the ARGs to COMMAND. For any ARG that is a regular file that\n\
+appears to be compressed, create a fifo, attach an appropriate\n\
+decompression program and replace the argument with the name of the fifo.\n\
+"), stdout);
+ fputs (_("\
+For any ARG that appears to be a sub-command, create a fifo, attach\n\
+the sub-command and replace the argument with the name of the fifo.\n\
+Leave any ARG that appears to be an option as-is.\n\
+Clean up all the temporary fifos after COMMAND finishes.\n\
+\n\
+"), stdout);
+ fputs (_("\
+Interpretation of ARGs can be affected by prefixing with one of three\n\
+explicit directives:\n\
+\n\
+"), stdout);
+ printf (_("\
+ %s Pass this ARG through unaffected\n\
+"), quote ("skip:"));
+ printf (_("\
+ %s Treat this ARG as a file (even if it looks like an option)\n\
+"), quote ("file:"));
+ printf (_("\
+ %s Treat this ARG as a sub-command\n\
+ (even if there's a file with the same name)\n\
+"), quote ("exec:"));
+ fputs (_("\
+\n\
+If COMMAND is invoked with the special option --files0-from=F, then\n\
+additional ARGS will interpreted from F, and piped through a fifo to\n\
+COMMAND.\n\
+"), stdout);
+ printf (_("\
+\n\
+Options (must come before COMMAND):\n\
+\n\
+ -T, --temporary-directory=DIR put temporary directories in DIR,\n\
+ not $TMPDIR or %s;\n\
+ -M, --max-directory-entries=MAX put at most MAX fifos in any directory;\n\
+ create a new directory after MAX\n\
+"), DEFAULT_TMPDIR);
+ fputs (HELP_OPTION_DESCRIPTION, stdout);
+ fputs (VERSION_OPTION_DESCRIPTION, stdout);
+ emit_bug_reporting_address ();
+ }
+ exit (status);
+}
+
+/* The set of signals we need to look out for. */
+static int const all_sig[] =
+{
+ /* The usual suspects, minus ALRM, which is used for reaping. */
+ SIGHUP, SIGINT, SIGPIPE, SIGQUIT, SIGTERM,
+#ifdef SIGPOLL
+ SIGPOLL,
+#endif
+#ifdef SIGPROF
+ SIGPROF,
+#endif
+#ifdef SIGVTALRM
+ SIGVTALRM,
+#endif
+#ifdef SIGXCPU
+ SIGXCPU,
+#endif
+#ifdef SIGXFSZ
+ SIGXFSZ,
+#endif
+};
+
+enum { nsigs = sizeof all_sig / sizeof all_sig[0] };
+
+/* The total number of forked processes that have not been reaped yet. */
+static size_t nprocs;
+
+/* The PID of our main child (the one who will be reading from our fifos). */
+static int command_pid;
+
+/* The exit status of our main child. */
+static int command_status;
+
+/* A list of the names of our fifos (so we can unlink them later). */
+static char **fifos;
+
+/* How many fifos are in our list. */
+static size_t fifo_count;
+
+/* How many fifos we've got space allocated for currently. */
+static size_t fifo_alloc;
+
+/* A list of temporary directories containing our fifos. */
+static char **fifo_dirs;
+
+/* How many temporary directories we have in our list. */
+static size_t fifo_dir_count = 0;
+
+/* How many directories we have space allocated for currently. */
+static size_t fifo_dir_alloc = 0;
+
+/* Directory where we create our temporary directories. */
+static char *fifo_dir_base;
+
+/* How many fifos we've put into the last temporary directory we created. */
+static size_t cur_fifo_dir_fifos = 0;
+
+/* How many fifos we'll put into a directory before creating a new one.
+ Can be set on the command-line with --max-directory-entries (-M). */
+static size_t max_fifo_dir_fifos = DEFAULT_MAX_DIRENTS;
+
+/* If our COMMAND uses --files0-from=F, then we need to take ARGs from that
+ file, and transform the --files0-from=F option to point to a fifo we'll
+ write to. */
+static char *files_from = NULL;
+
+/* The length first of the prefix "--files0-from=", then of our full
+ replacement */
+static size_t files_from_opt_len;
+
+/* The string containing first the prefix "--files0-from", then our
+ full replacement. */
+static char *files_from_opt;
+
+/* The index in the ARG list of the --files0-from option. */
+static int files_from_opt_i = 0;
+
+/* The set of signals that are caught. */
+static sigset_t caught_signals;
+
+/* Critical section status. */
+struct cs_status
+{
+ bool valid;
+ sigset_t sigs;
+};
+
+/* Enter a critical section. */
+static struct cs_status
+cs_enter (void)
+{
+ struct cs_status status;
+ status.valid = (sigprocmask (SIG_BLOCK,
+ &caught_signals, &status.sigs) == 0);
+ return status;
+}
+
+/* Leave a critical section. */
+static void
+cs_leave (struct cs_status status)
+{
+ if (status.valid)
+ {
+ /* Ignore failure when restoring the signal mask. */
+ sigprocmask (SIG_SETMASK, &status.sigs, NULL);
+ }
+}
+
+/* Clean up after ourselves. First unlink fifos, then rmdir directories.
+ Free memory, even though we currently only clean up at exit. */
+
+static void
+cleanup (void)
+{
+ int i;
+
+ for (i = 0; i < fifo_count; i++)
+ {
+ unlink (fifos[i]);
+ free (fifos[i]);
+ }
+ if (fifos)
+ {
+ free (fifos);
+ fifos = NULL;
+ }
+ for (i = 0; i < fifo_dir_count; i++)
+ {
+ if(rmdir (fifo_dirs[i]) == -1)
+ error (0, errno, _("rmdir failed: %s"), fifo_dirs[i]);
+ free (fifo_dirs[i]);
+ fifo_dirs[i] = NULL;
+ }
+ if (fifo_dirs)
+ {
+ free (fifo_dirs);
+ fifo_dirs = NULL;
+ }
+ if (fifo_dir_base)
+ {
+ free (fifo_dir_base);
+ fifo_dir_base = NULL;
+ }
+}
+
+/* Cleanup actions to take when exiting. */
+
+static void
+exit_cleanup (void)
+{
+ /* Clean up in a critical section so that a signal handler does not
+ try to clean up too. */
+ struct cs_status cs = cs_enter ();
+ cleanup ();
+ cs_leave (cs);
+
+ close_stdout ();
+}
+
+/* Handle interrupts and hangups. */
+
+
+static void
+sighandler (int sig)
+{
+ size_t i;
+ static int n_sig = 0;
+ if (! SA_NOCLDSTOP)
+ signal (sig, SIG_IGN);
+
+ cleanup ();
+
+ for (i = 0; i < (1<<30); i++);
+
+ signal (sig, SIG_DFL);
+ raise (sig);
+}
+
+/* Wrapper around dup2 for convenience. */
+
+static void
+dup2_or_die (int oldfd, int newfd)
+{
+ if (dup2 (oldfd, newfd) < 0)
+ error (EXIT_FAILURE, errno, _("dup2 failed"));
+}
+
+/* If 0 < PID, wait for the child process with that PID to exit.
+ If PID is -1, clean up a random child process which has finished.
+ If PID is -1 and no processes have quit yet, return 0 without waiting. */
+
+static pid_t
+reap (pid_t pid)
+{
+ int status;
+ pid_t cpid = waitpid (pid, &status, pid < 0 ? WNOHANG : 0);
+
+ if (cpid < 0)
+ error (EXIT_FAILURE, errno, _("waitpid"));
+
+ else if (0 < cpid)
+ {
+ if (! WIFEXITED (status) || WEXITSTATUS (status))
+ error (0, 0, _("%s terminated abnormally"),
+ (cpid == command_pid)?"command":"child");
+ if (cpid == command_pid)
+ command_status = status;
+ else
+ --nprocs;
+ }
+
+ return cpid;
+}
+
+/* Keep reaping finished children as long as there are more to reap.
+ This doesn't block waiting for any of them, it only reaps those
+ that are already dead. This is the sigaction handler for ALRM.
+ It reschedules itself for a second later after it's done. */
+
+static void
+reap_some (int sig)
+{
+ pid_t pid;
+
+ while (0 < nprocs && (pid = reap (-1)));
+
+ if (nprocs)
+ alarm (SECONDS_BETWEEN_REAPS);
+}
+
+/* Fork a child process. Don't let the child know where the fifos
+ and temporary directories are. We want to clean them up ourselves.
+ Return the PID of the child or -1 if fork failed. */
+
+static pid_t
+safe_fork (void)
+{
+#if HAVE_WORKING_FORK
+ char **saved_fifos;
+ int saved_errno;
+ int i;
+ unsigned int wait_retry = 1;
+ pid_t pid IF_LINT (= -1);
+ struct cs_status cs;
+
+
+ /* This is so the child process won't delete our temp files
+ if it receives a signal before exec-ing. */
+ cs = cs_enter ();
+ saved_fifos = fifos;
+ fifos = NULL;
+
+ pid = fork ();
+ saved_errno = errno;
+ if (pid)
+ fifos = saved_fifos;
+
+ if (0 == pid)
+ for (i = 0; i < nsigs; i++)
+ signal (all_sig[i], SIG_DFL);
+
+ cs_leave (cs);
+ errno = saved_errno;
+
+ if (0 < pid)
+ ++nprocs;
+
+ return pid;
+
+#else /* ! HAVE_WORKING_FORK */
+ return -1;
+#endif
+}
+
+/* Create a temporary directory into which we'll put fifos. */
+
+static void
+create_fifo_dir (void)
+{
+ static char const slashbase[] = "/magicXXXXXX";
+ struct cs_status cs;
+ int saved_errno;
+ size_t len = strlen (fifo_dir_base);
+ char *success;
+ char *fifo_dir;
+
+ if (fifo_dir_count == fifo_dir_alloc)
+ fifo_dirs = X2NREALLOC (fifo_dirs, &fifo_dir_alloc);
+
+ /* One extra for the trailing slash */
+ fifo_dirs[fifo_dir_count] = xmalloc (len + sizeof slashbase + 1);
+
+ fifo_dir = fifo_dirs[fifo_dir_count++];
+
+ memcpy (fifo_dir, fifo_dir_base, len);
+ memcpy (fifo_dir + len, slashbase, sizeof slashbase);
+
+
+ /* Create the directory in a critical section, to avoid races. */
+ cs = cs_enter ();
+ success = mkdtemp (fifo_dir);
+ saved_errno = errno;
+ cs_leave (cs);
+ errno = saved_errno;
+
+ if (!success)
+ {
+ /* Note that the last FIFO_DIR doesn't get freed, but we're exiting
+ anyway. */
+ fifo_dir_count--;
+ error (EXIT_FAILURE, errno,
+ _("cannot create temporary directory: %s"), fifo_dir);
+ }
+
+ /* Append a slash so we don't have to worry about it later. */
+ fifo_dir[len + sizeof slashbase - 1] = '/';
+ fifo_dir[len + sizeof slashbase] = '\0';
+
+ cur_fifo_dir_fifos = 0;
+}
+
+/* Use DIR as our base temp dir. */
+static void
+set_temp_dir (char const *dir)
+{
+ size_t len;
+ if (fifo_dir_base)
+ {
+ if (!STREQ (fifo_dir_base, dir))
+ error (EXIT_FAILURE, 0, _("multiple temp dirs specified"));
+ }
+ else
+ {
+ len = strlen (dir);
+ fifo_dir_base = xmalloc (len + 1);
+ memcpy (fifo_dir_base, dir, len);
+ }
+}
+
+/* Create a new fifo with a name resembling the file or command it's
+ associated with. */
+static char *
+new_fifo (char const *file_name)
+{
+ char *fifo_tag = strrchr (file_name, '/');
+ size_t len1, len2, len3 = 0;
+ size_t len3_alloc = INT_BUFSIZE_BOUND (unsigned int) + 1;
+ char i_str [INT_BUFSIZE_BOUND (unsigned int) + 1];
+ unsigned int i = 0;
+ int still_trying = 1;
+ char *fifo;
+ char *fifo_dir;
+
+ if (fifo_tag)
+ fifo_tag++;
+ else
+ fifo_tag = (char *)file_name;
+
+ if (fifo_count == fifo_alloc)
+ fifos = X2NREALLOC (fifos, &fifo_alloc);
+
+ if (!fifo_dir_count || cur_fifo_dir_fifos == max_fifo_dir_fifos)
+ create_fifo_dir ();
+
+ cur_fifo_dir_fifos++;
+
+ fifo_dir = fifo_dirs[fifo_dir_count - 1];
+
+ len1 = strlen (fifo_dir);
+ len2 = strlen (fifo_tag);
+
+ fifos[fifo_count] = xmalloc (len1 + len2 + len3_alloc);
+
+ fifo = fifos[fifo_count++];
+
+ memcpy (fifo, fifo_dir, len1);
+ memcpy (fifo + len1, fifo_tag, len2 + 1);
+
+ /* There may be files with the same name in different directories, so
+ we need to handle collisions. This gets inefficient as we get more
+ identical filenames. */
+ while (still_trying && (i < UINT_MAX))
+ {
+
+ if (i)
+ {
+ sprintf (i_str, "-%d", i);
+ len3 = strlen (i_str);
+ memcpy (fifo + len1 + len2, i_str, len3 + 1);
+ }
+
+ if ((still_trying = mkfifo (fifo, 0600)) != 0)
+ {
+ if (errno != EEXIST)
+ error (EXIT_FAILURE, errno, _("cannot create fifo %s"),
+ quote (fifo));
+ }
+
+ i++;
+ }
+
+ return fifo;
+}
+
+/* See if this ARG matches a filename. If it does, and it's a regular file,
+ see if its magic corresponds to a compression format and set up for
+ auto-decompression. Returns the filename if it's a file, but non-regular
+ or not compressed, the fifo name if compressed or NULL if not a file. */
+static char *
+handle_file (int i, char *p)
+{
+ int filefd, pipefd;
+
+ struct stat st;
+
+ char magic[2] = {0,0};
+ char *open_prog = NULL;
+
+ pid_t helper_pid;
+
+
+ if (stat (p, &st) == 0)
+ {
+ if (S_ISREG (st.st_mode))
+ {
+
+ if ((filefd = open (p, O_RDONLY)) == -1)
+ error (EXIT_FAILURE, errno, _("open failed: %s"), p);
+
+ if ((read (filefd, &magic, 2)) == -1)
+ error (EXIT_FAILURE, errno, _("read failed: %s"), p);
+
+
+ if (!memcmp (magic, GZIP_MAGIC, 2)
+ || !memcmp (magic, GZIP_MAGIC_OLD, 2))
+ open_prog = "gzip";
+
+ else if (!memcmp (magic, BZIP_MAGIC, 2))
+ open_prog = "bzip2";
+
+ if (open_prog)
+ {
+ char *fifo;
+
+ lseek (filefd, 0, SEEK_SET);
+
+ fifo = new_fifo (p);
+
+ helper_pid = safe_fork ();
+
+ if (helper_pid > 0)
+ {
+ close (filefd);
+
+ return fifo;
+
+ }
+ else if (helper_pid == 0)
+ {
+
+ close (STDIN_FILENO);
+ close (STDOUT_FILENO);
+
+ /* This will block. */
+ if ((pipefd = open (fifo, O_WRONLY)) == -1)
+ error (EXIT_FAILURE, errno, _("open failed: %s"),
+ fifo);
+
+ dup2_or_die (pipefd, STDOUT_FILENO);
+ dup2_or_die (filefd, STDIN_FILENO);
+
+ execlp (open_prog, open_prog, "-c", "-f", "-d", NULL);
+
+ error (EXIT_FAILURE, errno, _("execlp failed: %s"),
+ open_prog);
+
+ }
+ else
+ error (EXIT_FAILURE, errno, _("fork failed: %s"),
+ open_prog);
+
+ }
+
+ }
+ return p; /* It's a file, at least */
+ }
+
+ return NULL;
+}
+
+/* Treat this ARG as a sub-command to be executed. Hook its output up to
+ a fifo, and return the name of the fifo. */
+static char *
+handle_exec (int i, char *p)
+{
+ int pipefd;
+ char *fifo = new_fifo (p);
+
+ pid_t helper_pid = safe_fork ();
+
+ if (helper_pid > 0)
+ {
+
+ return fifo;
+
+ }
+ else if (helper_pid == 0)
+ {
+
+ close (STDIN_FILENO);
+ close (STDOUT_FILENO);
+
+ /* This will block. */
+ if ((pipefd = open (fifo, O_WRONLY)) == -1)
+ error (EXIT_FAILURE, errno, _("open failed: %s"),
+ fifo);
+
+ dup2_or_die (pipefd, STDOUT_FILENO);
+
+ execlp ("sh", "sh", "-c", p, NULL);
+
+ error (EXIT_FAILURE, errno, _("execlp failed: sh -c %s"), quote (p));
+
+ }
+
+ error (EXIT_FAILURE, errno, _("fork failed"));
+
+}
+
+
+/* If this ARG looks like an option, return it. Otherwise, return NULL.
+ Also, if it looks like a --files0-from option, set up for files_from
+ processing. */
+static char *
+handle_option (int i, char *p)
+{
+ if (!strncmp (p, files_from_opt, files_from_opt_len))
+ {
+ files_from = p + files_from_opt_len;
+ files_from_opt_i = i;
+ return p;
+ }
+ else if (*p == '-')
+ return p;
+
+ return NULL;
+}
+
+/* If this ARG starts with a recognized directive prefix then dispatch
+ to the appropriate handler. */
+static char *
+handle_directive (int i, char *p)
+{
+ char *d_file = "file:";
+ char *d_skip = "skip:";
+ char *d_exec = "exec:";
+
+ size_t len = strlen (d_file);
+
+ if (!strncmp (p, d_file, len))
+ return handle_file (i, p + len);
+
+ if (!strncmp (p, d_skip, len))
+ return p + len;
+
+ if (!strncmp (p, d_exec, len))
+ return handle_exec (i, p + len);
+
+ return NULL;
+}
+
+/* Cascade through ARG handlers in order of precedence. Return the new
+ value to go in ARG's place. */
+static char *
+handle_arg (int i, char *p)
+{
+ char *retval;
+
+ if ((retval = handle_directive (i, p)) != NULL)
+ return retval;
+
+ if ((retval = handle_option (i, p)) != NULL)
+ return retval;
+
+ if ((retval = handle_file (i, p)) != NULL)
+ return retval;
+
+ if ((retval = handle_exec (i, p)) != NULL)
+ return retval;
+
+ return p;
+}
+
+/* Specify the maximum number of fifos we'll create in a single directory
+ before creating a new temporary directory. */
+static void
+specify_max_dirent (int oi, char c, char const *s)
+{
+ uintmax_t n;
+ enum strtol_error e = xstrtoumax (s, NULL, 10, &n, "");
+
+ if (e == LONGINT_OK)
+ {
+ max_fifo_dir_fifos = n;
+ if (n == max_fifo_dir_fifos)
+ {
+ if (0 < max_fifo_dir_fifos)
+ return;
+ e = LONGINT_INVALID;
+ }
+ else
+ e = LONGINT_OVERFLOW;
+ }
+
+ xstrtol_fatal (e, oi, c, long_options, s);
+}
+
+
+int
+main (int argc, char **argv)
+{
+ int c = 0, i;
+ int oi = -1;
+ char **new_argv;
+ struct fstatus *fstatus;
+ struct Tokens tok;
+ char **files;
+ char *fifo;
+ int nfiles = 0;
+
+ files_from_opt_len = FILES_FROM_OPT_LEN;
+ files_from_opt = xmalloc (files_from_opt_len + 1);
+
+ memcpy (files_from_opt, FILES_FROM_OPT, files_from_opt_len);
+
+ initialize_main (&argc, &argv);
+ program_name = argv[0];
+ setlocale (LC_ALL, "");
+ bindtextdomain (PACKAGE, LOCALEDIR);
+ textdomain (PACKAGE);
+
+ initialize_exit_failure (EXIT_FAILURE);
+ atexit (exit_cleanup);
+
+ /* This signal handler setup block is basically lifted straight out of
+ sort.c, which also manages temporary files that need to be cleaned up
+ before exiting, if possible. */
+ {
+ size_t i;
+
+#if SA_NOCLDSTOP
+ struct sigaction act;
+
+ sigemptyset (&caught_signals);
+ for (i = 0; i < nsigs; i++)
+ {
+ sigaction (all_sig[i], NULL, &act);
+ if (act.sa_handler != SIG_IGN)
+ sigaddset (&caught_signals, all_sig[i]);
+ }
+
+ act.sa_handler = sighandler;
+ act.sa_mask = caught_signals;
+ act.sa_flags = 0;
+
+ for (i = 0; i < nsigs; i++)
+ if (sigismember (&caught_signals, all_sig[i]))
+ sigaction (all_sig[i], &act, NULL);
+#else
+ for (i = 0; i < nsigs; i++)
+ if (signal (all_sig[i], SIG_IGN) != SIG_IGN)
+ {
+ signal (all_sig[i], sighandler);
+ siginterrupt (all_sig[i], 1);
+ }
+#endif
+ }
+
+
+ parse_long_options (argc, argv, PROGRAM_NAME, PACKAGE_NAME, VERSION,
+ usage, AUTHORS, (char const *) NULL);
+
+ while ((c = getopt_long (argc, argv, "+T:M:", long_options, &oi)) != -1)
+ switch (c)
+ {
+ case 'T':
+ set_temp_dir (optarg);
+ break;
+
+ case 'M':
+ specify_max_dirent (oi, c, optarg);
+ break;
+
+ default:
+ usage (EXIT_FAILURE);
+ break;
+ }
+
+ if (argc - optind < 2)
+ usage (EXIT_FAILURE);
+
+ if (!fifo_dir_base)
+ {
+ char const *tmp_dir = getenv ("TMPDIR");
+ set_temp_dir (tmp_dir ? tmp_dir : DEFAULT_TMPDIR);
+ }
+
+ argc -= optind;
+ argv += optind;
+
+ new_argv = xnmalloc (argc + 1, sizeof *argv);
+ memcpy (new_argv, argv, (argc + 1) * sizeof *argv);
+
+ for (i = 1; i < argc; i++)
+ {
+ char *p = new_argv[i];
+
+ new_argv[i] = handle_arg (i, new_argv[i]);
+ }
+
+ if (files_from)
+ {
+ FILE *stream;
+
+ if (STREQ (files_from, "-"))
+ stream = stdin;
+ else
+ {
+ stream = fopen (files_from, "r");
+ if (stream == NULL)
+ error (EXIT_FAILURE, errno, _("cannot open %s for reading"),
+ quote (files_from));
+ }
+
+ readtokens0_init (&tok);
+
+ if (! readtokens0 (stream, &tok))
+ error (EXIT_FAILURE, 0, _("cannot read file names from %s"),
+ quote (files_from));
+
+ files = tok.tok;
+ nfiles = tok.n_tok;
+ }
+
+ if (files_from)
+ {
+ pid_t writer_pid;
+
+ for (i = 0; i < nfiles; i++)
+ if ((fifo = handle_arg (i, files[i])) != NULL)
+ files[i] = fifo;
+
+ fifo = new_fifo (files_from);
+
+ writer_pid = safe_fork ();
+
+ if (0 < writer_pid)
+ {
+ size_t fifo_len = strlen (fifo) + 1;
+ files_from_opt = xrealloc (files_from_opt,
+ files_from_opt_len + fifo_len);
+
+ /* Put the new --files0-from=F option in place of the old one. */
+ memcpy (files_from_opt + files_from_opt_len, fifo, fifo_len);
+ new_argv[files_from_opt_i] = files_from_opt;
+ }
+ else if (writer_pid == 0)
+ {
+ int writefd;
+ size_t written;
+ size_t arglen;
+ /* This will block. */
+ if ((writefd = open (fifo, O_WRONLY)) == -1)
+ error (EXIT_FAILURE, errno, _("open failed: %s"), fifo);
+ for (i = 0; i < nfiles; i++)
+ {
+ arglen = strlen (files[i]) + 1;
+ if ((written = write (writefd, files[i], arglen)) != arglen)
+ error (EXIT_FAILURE, errno, _("write failed: %s"), fifo);
+ }
+ return 0;
+ }
+ else
+ error (EXIT_FAILURE, errno, _("fork failed"));
+ }
+
+
+ command_pid = safe_fork ();
+
+ if (command_pid > 0)
+ {
+ struct sigaction alarm_handler;
+
+ if (--nprocs)
+ {
+ sigemptyset (&alarm_handler.sa_mask);
+
+ alarm_handler.sa_handler = reap_some;
+
+ /* Restart the blocking wait() after we clean up zombie helpers. */
+ alarm_handler.sa_flags = SA_RESTART;
+
+ sigaction (SIGALRM, &alarm_handler, NULL);
+
+ /* This sets an alarm for itself. It will run every
+ SECONDS_BETWEEN_REAPS seconds while we're waiting for COMMAND
+ to finish. */
+ reap_some (0);
+ }
+
+ /* This blocks. We're stuck here (except for occasional non-blocking
+ reap sessions for helper children) until COMMAND is done. */
+ reap (command_pid);
+
+ if (WIFEXITED (command_status))
+ command_status = WEXITSTATUS (command_status);
+ else if (WIFSIGNALED (command_status))
+ command_status = WTERMSIG (command_status) + 128;
+
+ return command_status;
+ }
+ else if (command_pid == 0)
+ {
+
+ execvp (new_argv[0], new_argv);
+
+ error (EXIT_FAILURE, errno, _("exec of %s failed"),
+ quote (new_argv[0]));
+ }
+ else
+ error (EXIT_FAILURE, 0, _("fork failed"));
+}
+
+/*
+ * Local variables:
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tests/misc/Makefile.am b/tests/misc/Makefile.am
index 17a0ec0..7b52b91 100644
--- a/tests/misc/Makefile.am
+++ b/tests/misc/Makefile.am
@@ -72,6 +72,7 @@ TESTS = \
groups-version \
head-c \
head-pos \
+ magic \
md5sum \
md5sum-newline \
mknod \
diff --git a/tests/misc/help-version b/tests/misc/help-version
index 3696736..3870fd5 100755
--- a/tests/misc/help-version
+++ b/tests/misc/help-version
@@ -143,6 +143,7 @@ groups_args=--version
pathchk_args=$tmp_in
yes_args=--version
logname_args=--version
+magic_args=--version
nohup_args=--version
printf_args=foo
seq_args=10
diff --git a/tests/misc/magic b/tests/misc/magic
new file mode 100755
index 0000000..ddedf9b
--- /dev/null
+++ b/tests/misc/magic
@@ -0,0 +1,89 @@
+#!/bin/sh
+# Test magic
+
+# Copyright (C) 2008 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+: ${srcdir=.}
+. $srcdir/../require-perl
+
+me=`echo $0|sed 's,.*/,,'`
+exec $PERL -w -I$srcdir/.. -MCoreutils -M"CuTmpdir qw($me)" -- - <<\EOF
+#/
+require 5.003;
+use strict;
+
+(my $program_name = $0) =~ s|.*/||;
+
+my $prog = 'magic';
+
+# get rid of potential differences between invocation and environments
+my $subst = {OUT_SUBST => 's/(?:\S+magic.{6}|\s+)//g'};
+
+# Turn off localization of executable's ouput.
[EMAIL PROTECTED](LANGUAGE LANG LC_ALL)} = ('C') x 3;
+
+open my $ofh, ">", "bzip2" or die "Couldn't open bzip2 for write: $!";
+print $ofh "#!/bin/sh\ntail -n +2\n";
+close $ofh;
+system ("chmod +x bzip2");
+
+$ENV{PATH} = ".:$ENV{PATH}";
+
+my @Tests =
+ (
+ # invalid naked invocation
+ ['usage', {EXIT=>1},
+ {ERR => "Try `$prog --help' for more information.\n"}],
+
+ # invalid invocation with no command operands
+ ['usage-2', 'wc', {EXIT=>1},
+ {ERR => "Try `$prog --help' for more information.\n"}],
+
+ # successful invocation with unmodified input
+ ['ok', 'wc', {IN=>{in=>'a'}}, {OUT => "0 1 1 in\n"}],
+
+ # successful invocation with option
+ ['ok-1', 'wc', '-l', {IN=>{in=>'a'}}, {OUT => "0 in\n"}],
+
+ # successful invocation with `compressed' input
+ ['ok-2', 'wc', {IN=>{in=>"BZ\na"}}, $subst, {OUT => "011/in"}],
+
+ # successful invocation with sub-command
+ ['ok-3', 'wc', '"ls bzip2"', $subst, {OUT => "116/lsbzip2"}],
+
+ # successful invocation with both
+ ['ok-4', 'wc', {IN=>{in=>"BZ\na"}}, '"ls bzip2"', $subst,
+ {OUT => "011/in116/lsbzip2127total"}],
+
+ # specification of an invalid temporary directory
+ ['bad', '-T', 'does/not/exist', 'wc', {IN=>{in=>"BZ\na"}}, '"ls bzip2"',
+ {EXIT=>1},
+ {ERR_SUBST => 's/\/magic.+/\/magic/'},
+ {ERR => "$prog: cannot create temporary directory: ".
+ "does/not/exist/magic\n"}],
+
+ # need at least -M of 1
+ ['bad-2', '-M', '0', 'wc', {IN=>{in=>"BZ\na"}}, '"ls bzip2"',
+ {EXIT=>1}, {ERR => "$prog: invalid -M argument `0'\n"}],
+
+ );
+
+my $save_temps = $ENV{DEBUG};
+my $verbose = $ENV{VERBOSE};
+
+my $fail = run_tests ($program_name, $prog, [EMAIL PROTECTED], $save_temps, $verbose);
+exit $fail;
+EOF
--
1.5.2.5
_______________________________________________
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
