On Sun, Apr 6, 2008 at 4:30 PM, Jim Meyering <[EMAIL PROTECTED]> wrote:
> s/texti/texi/
> Please use capitals and periods in ChangeLogs. ;-)
> s/null/NUL/
> Split the string. Otherwise, your addition pushes its length beyond
> a portability limit whose exact number I forget but it's around 500.
> No big deal, but it's good practice to alphabetize.
> This should have only 1 year number: 2008.
Thanks. I'll try to catch this sort of thing myself in the future.
> Now, your doc change will be to add this line:
>
> @files0fromOption{sort,}
I added an argument to the macro that specifies output for sub-lists,
since it's 'a total' for wc and du, but 'sorted output' for sort.
> If the file is based on some other, please indicate that.
> That will help me as reviewer, and future maintainers.
>
> E.g., I put this comment in the wc test of --files0-from:
>
> # This file bears a striking resemblance to tests/du/files0-from.
Unfortunately I didn't just copy one of the relevant test files. If
it would be easier for maintenance to have a more direct copy I can
redo it. I added a line at the top indicating that this test script
covers a lot of the same ground as the wc-files0-from tests.
Thanks,
Bo
From 404e23daf6874e4d36e2048de569bcac057b7400 Mon Sep 17 00:00:00 2001
From: Bo Borgerson <[EMAIL PROTECTED]>
Date: Thu, 3 Apr 2008 18:42:57 -0400
Subject: [PATCH] Add new sort option --files0-from=F
* src/sort.c: Support new option.
* tests/misc/sort-files0-from: Test new option.
* tests/misc/Makefile.am: Indicate new test.
* docs/coreutils.texi: Explain new option.
* NEWS: Advertise new option.
Signed-off-by: Bo Borgerson <[EMAIL PROTECTED]>
---
NEWS | 5 ++
doc/coreutils.texi | 12 +++--
src/sort.c | 65 ++++++++++++++++++++++++---
tests/misc/Makefile.am | 1 +
tests/misc/sort-files0-from | 106 +++++++++++++++++++++++++++++++++++++++++++
5 files changed, 178 insertions(+), 11 deletions(-)
create mode 100755 tests/misc/sort-files0-from
diff --git a/NEWS b/NEWS
index e208b30..492c4e9 100644
--- a/NEWS
+++ b/NEWS
@@ -55,6 +55,11 @@ GNU coreutils NEWS -*- outline -*-
options --general-numeric-sort/-g, --month-sort/-M, --numeric-sort/-n
and --random-sort/-R, resp.
+ sort accepts a new option, --files0-from=F, that specifies a file
+ containing a null-separated list of files to sort. This list is used
+ instead of filenames passed on the command-line to avoid problems with
+ maximum command-line (argv) length.
+
** Improvements
id and groups work around an AFS-related bug whereby those programs
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 5a6f2c3..9ac7bbf 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -3074,7 +3074,7 @@ Print only the newline counts.
@opindex --max-line-length
Print only the maximum line lengths.
[EMAIL PROTECTED] files0fromOption{cmd,withTotalOption}
[EMAIL PROTECTED] files0fromOption{cmd,withTotalOption,subListOutput}
@itemx [EMAIL PROTECTED]
@opindex [EMAIL PROTECTED]
@cindex including files from @command{\cmd\}
@@ -3084,13 +3084,13 @@ This is useful \withTotalOption\
when the list of file names is so long that it may exceed a command line
length limitation.
In such cases, running @command{\cmd\} via @command{xargs} is undesirable
-because it splits the list into pieces and makes @command{\cmd\} print a
-total for each sublist rather than for the entire list.
+because it splits the list into pieces and makes @command{\cmd\} print
+\subListOutput\ for each sublist rather than for the entire list.
One way to produce a list of null-byte-terminated file names is with @sc{gnu}
@command{find}, using its @option{-print0} predicate.
Do not specify any @var{FILE} on the command line when using this option.
@end macro
[EMAIL PROTECTED],}
[EMAIL PROTECTED],,a total}
For example, to find the length of the longest line in any @file{.c} or
@file{.h} file in the current hierarchy, do this:
@@ -3670,6 +3670,8 @@ Terminate with an error if @var{prog} exits with nonzero status.
Whitespace and the backslash character should not appear in
@var{prog}; they are reserved for future use.
[EMAIL PROTECTED],,sorted output}
+
@item -k @var{pos1}[,@var{pos2}]
@itemx [EMAIL PROTECTED],@var{pos2}]
@opindex -k
@@ -9757,7 +9759,7 @@ Does not affect other symbolic links. This is helpful for finding
out the disk usage of directories, such as @file{/usr/tmp}, which
are often symbolic links.
[EMAIL PROTECTED], with the @option{--total} (@option{-c}) option}
[EMAIL PROTECTED], with the @option{--total} (@option{-c}) option,a total}
@optHumanReadable
diff --git a/src/sort.c b/src/sort.c
index 8b2eec5..e67ce80 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -37,6 +37,7 @@
#include "posixver.h"
#include "quote.h"
#include "randread.h"
+#include "readtokens0.h"
#include "stdio--.h"
#include "stdlib--.h"
#include "strnumcmp.h"
@@ -304,8 +305,9 @@ usage (int status)
{
printf (_("\
Usage: %s [OPTION]... [FILE]...\n\
+ or: %s [OPTION]... --files0-from=F\n\
"),
- program_name);
+ program_name, program_name);
fputs (_("\
Write sorted concatenation of all FILE(s) to standard output.\n\
\n\
@@ -342,22 +344,24 @@ Other options:\n\
-C, --check=quiet, --check=silent like -c, but do not report first bad line\n\
--compress-program=PROG compress temporaries with PROG;\n\
decompress them with PROG -d\n\
- -k, --key=POS1[,POS2] start a key at POS1, end it at POS2 (origin 1)\n\
- -m, --merge merge already sorted files; do not sort\n\
+ --files0-from=F read input from the files specified by\n\
+ NUL-terminated names in file F\n\
"), stdout);
fputs (_("\
+ -k, --key=POS1[,POS2] start a key at POS1, end it at POS2 (origin 1)\n\
+ -m, --merge merge already sorted files; do not sort\n\
-o, --output=FILE write result to FILE instead of standard output\n\
-s, --stable stabilize sort by disabling last-resort comparison\n\
- -S, --buffer-size=SIZE use SIZE for main memory buffer\n\
"), stdout);
printf (_("\
+ -S, --buffer-size=SIZE use SIZE for main memory buffer\n\
-t, --field-separator=SEP use SEP instead of non-blank to blank transition\n\
-T, --temporary-directory=DIR use DIR for temporaries, not $TMPDIR or %s;\n\
multiple options specify multiple directories\n\
- -u, --unique with -c, check for strict ordering;\n\
- without -c, output only the first of an equal run\n\
"), DEFAULT_TMPDIR);
fputs (_("\
+ -u, --unique with -c, check for strict ordering;\n\
+ without -c, output only the first of an equal run\n\
-z, --zero-terminated end lines with 0 byte, not newline\n\
"), stdout);
fputs (HELP_OPTION_DESCRIPTION, stdout);
@@ -394,6 +398,7 @@ enum
{
CHECK_OPTION = CHAR_MAX + 1,
COMPRESS_PROGRAM_OPTION,
+ FILES0_FROM_OPTION,
RANDOM_SOURCE_OPTION,
SORT_OPTION
};
@@ -407,6 +412,7 @@ static struct option const long_options[] =
{"compress-program", required_argument, NULL, COMPRESS_PROGRAM_OPTION},
{"dictionary-order", no_argument, NULL, 'd'},
{"ignore-case", no_argument, NULL, 'f'},
+ {"files0-from", required_argument, NULL, FILES0_FROM_OPTION},
{"general-numeric-sort", no_argument, NULL, 'g'},
{"ignore-nonprinting", no_argument, NULL, 'i'},
{"key", required_argument, NULL, 'k'},
@@ -2752,6 +2758,8 @@ main (int argc, char **argv)
bool posixly_correct = (getenv ("POSIXLY_CORRECT") != NULL);
bool obsolete_usage = (posix2_version () < 200112);
char **files;
+ char *files_from = NULL;
+ struct Tokens tok;
char const *outfile = NULL;
initialize_main (&argc, &argv);
@@ -2955,6 +2963,10 @@ main (int argc, char **argv)
compress_program = optarg;
break;
+ case FILES0_FROM_OPTION:
+ files_from = optarg;
+ break;
+
case 'k':
key = key_init (&key_buf);
@@ -3099,6 +3111,47 @@ main (int argc, char **argv)
}
}
+ if (files_from)
+ {
+ FILE *stream;
+
+ /* When using --files0-from=F, you may not specify any files
+ on the command-line. */
+ if (nfiles)
+ {
+ error (0, 0, _("extra operand %s"), quote (files[0]));
+ fprintf (stderr, "%s\n",
+ _("file operands cannot be combined with --files0-from"));
+ usage (SORT_FAILURE);
+ }
+
+ if (STREQ (files_from, "-"))
+ stream = stdin;
+ else
+ {
+ stream = fopen (files_from, "r");
+ if (stream == NULL)
+ error (SORT_FAILURE, errno, _("cannot open %s for reading"),
+ quote (files_from));
+ }
+
+ readtokens0_init (&tok);
+
+ if (! readtokens0 (stream, &tok) || fclose (stream) != 0)
+ error (SORT_FAILURE, 0, _("cannot read file names from %s"),
+ quote (files_from));
+
+ if (tok.n_tok)
+ {
+ free (files);
+ files = tok.tok;
+ nfiles = tok.n_tok;
+ }
+ else
+ error (SORT_FAILURE, 0, _("no input from %s"),
+ quote (files_from));
+ }
+
/* Inheritance of global options to individual keys. */
for (key = keylist; key; key = key->next)
{
diff --git a/tests/misc/Makefile.am b/tests/misc/Makefile.am
index 17a0ec0..7fcd0b8 100644
--- a/tests/misc/Makefile.am
+++ b/tests/misc/Makefile.am
@@ -98,6 +98,7 @@ TESTS = \
shred-remove \
shuf \
sort-compress \
+ sort-files0-from \
sort-merge \
sort-rand \
split-a \
diff --git a/tests/misc/sort-files0-from b/tests/misc/sort-files0-from
new file mode 100755
index 0000000..d5403a5
--- /dev/null
+++ b/tests/misc/sort-files0-from
@@ -0,0 +1,106 @@
+#!/bin/sh
+# Test "sort --files0-from=F".
+# Many of these tests are based on tests/misc/wc-files0-from.
+
+# Copyright (C) 2008 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+: ${srcdir=.}
+. $srcdir/../require-perl
+
+me=`echo $0|sed 's,.*/,,'`
+exec $PERL -w -I$srcdir/.. -MCoreutils -M"CuTmpdir qw($me)" -- - <<\EOF
+require 5.003;
+use strict;
+
+(my $program_name = $0) =~ s|.*/||;
+my $prog = 'sort';
+
+# Turn off localisation of executable's ouput.
[EMAIL PROTECTED](LANGUAGE LANG LC_ALL)} = ('C') x 3;
+
+`echo a > in1`;
+`echo b > in2`;
+`echo c > in3`;
+
+my $fn = "arglist1";
+my $list0 = "in1\0in2\0in3\0";
+open my $fh, ">", $fn or warn "Couldn't open '$fn' for write: $!";
+print $fh $list0;
+close $fh;
+
+`touch empty`;
+
+my $usage_err = <<EO_USAGE;
+$prog: extra operand `extra'
+file operands cannot be combined with --files0-from
+Try `sort --help' for more information.
+EO_USAGE
+
+my @Tests =
+ (
+
+ # nonexistent files0-from input file
+ ['f1', "--files0-from=nonexistent", {EXIT => 2},
+ {ERR_SUBST => 's/reading:.+/reading/'},
+ {ERR => "$prog: cannot open `nonexistent' for reading\n"}],
+
+ # empty files0-from input file
+ ['f2', "--files0-from=empty", {EXIT => 2},
+ {ERR => "$prog: no input from `empty'\n"}],
+
+ # command-line argument wth files0-from input
+ ['f3', "--files0-from=$fn", {IN => {extra => 'operand'}}, {EXIT => 2},
+ {ERR => $usage_err}],
+
+ # successful files0-from file
+ ['f4', "--files0-from=$fn", {OUT => "a\nb\nc\n"}],
+
+ # successful files0-from stdin
+ ['f5', '--files0-from=-', '<', {IN => $list0 }, {OUT => "a\nb\nc\n"}],
+
+ # one NUL
+ ['nul-1', '--files0-from=-', '<', {IN=>"\0"}, {EXIT=>2},
+ {ERR => "$prog: open failed: : No such file or directory\n"}],
+
+ # two NULs, one fatal error
+ ['nul-2', '--files0-from=-', '<', {IN=>"\0\0"}, {EXIT=>2},
+ {ERR => "$prog: open failed: : No such file or directory\n"}],
+
+ # one file name, no NUL
+ ['1', '--files0-from=-', '<', {IN=>"in1"}, {OUT=>"a\n"}],
+
+ # one file name, with NUL
+ ['1a', '--files0-from=-', '<', {IN=>"in1\0"}, {OUT=>"a\n"}],
+
+ # two file names, no final NUL
+ ['2', '--files0-from=-', '<', {IN=>"in1\0in2"}, {OUT=>"a\nb\n"}],
+
+ # two file names, with final NUL
+ ['2a', '--files0-from=-', '<', {IN=>"in1\0in2\0"}, {OUT=>"a\nb\n"}],
+
+ # Ensure that sort dies even with a good file after a null filename
+ ['zero-len', '--files0-from=-', '<', {IN=>{f=>"\0in1\0"}},
+ {ERR => "$prog: open failed: : No such file or directory\n"},
+ {EXIT=>2} ],
+
+ );
+
+my $save_temps = $ENV{DEBUG};
+my $verbose = $ENV{VERBOSE};
+
+my $fail = run_tests ($program_name, $prog, [EMAIL PROTECTED], $save_temps, $verbose);
+exit $fail;
+EOF
--
1.5.2.5
_______________________________________________
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
