Re: head(1): fully support the legacy -count syntax
On Mon, Oct 11, 2021 at 09:13:16AM -0500, Scott Cheloha wrote:
> On Sun, Oct 10, 2021 at 08:26:04PM -0400, gwes wrote:
> > On 10/10/21 5:03 PM, Scott Cheloha wrote:
> > > [...]
> > >
> > > If we want to have the unportable legacy syntax then it should work
> > > like other option arguments. Option arguments can be respecified
> > > multiple times in most other utilities. The last such appearance of
> > > an option argument is the one the utility uses. That's how option
> > > arguments work, for the most part.
> > >
> > > We could remove the legacy syntax and shave a couple lines of code.
> > >
> > > OTOH, supporting it fully is super easy. I've provided a patch.
> > >
> > The man page says head [-count | -n count] [file ...]
> > There are only two valid switches.
> > One, the other, or none.
> > Using more than one is -undefined-.
> > I doubt it ever has been defined.
> > It could be asserted that any more than 1 switch is an error.
>
> No.
>
> It wouldn't be an error, it would be undefined. Just like you said.
If we have survived with a crippled backwards compat since '95, it's
pretty pointless to re-add it now.
I also started thinking about e.g. kill(1). We don't support multiple
signals there.
Thinking a bit more, and testing some, I'm pretty sure the following
change to usage() and the man page would suffice.
- usage: head [-count | -n count] [file ...]
+ usage: head [-count] [-n count] [file ...]
/Alexander
Index: head.1
===
RCS file: /cvs/src/usr.bin/head/head.1,v
retrieving revision 1.23
diff -u -p -r1.23 head.1
--- head.1 25 Oct 2015 21:50:32 - 1.23
+++ head.1 14 Oct 2021 21:28:31 -
@@ -37,7 +37,8 @@
.Nd display first few lines of files
.Sh SYNOPSIS
.Nm head
-.Op Fl Ar count | Fl n Ar count
+.Op Fl Ar count
+.Op Fl n Ar count
.Op Ar
.Sh DESCRIPTION
The
Index: head.c
===
RCS file: /cvs/src/usr.bin/head/head.c,v
retrieving revision 1.22
diff -u -p -r1.22 head.c
--- head.c 10 Oct 2021 15:57:25 - 1.22
+++ head.c 14 Oct 2021 21:28:31 -
@@ -114,6 +114,6 @@ main(int argc, char *argv[])
static void
usage(void)
{
- fputs("usage: head [-count | -n count] [file ...]\n", stderr);
+ fputs("usage: head [-count] [-n count] [file ...]\n", stderr);
exit(1);
}
Re: head(1): fully support the legacy -count syntax
On Sun, Oct 10, 2021 at 08:26:04PM -0400, gwes wrote:
> On 10/10/21 5:03 PM, Scott Cheloha wrote:
> > [...]
> >
> > If we want to have the unportable legacy syntax then it should work
> > like other option arguments. Option arguments can be respecified
> > multiple times in most other utilities. The last such appearance of
> > an option argument is the one the utility uses. That's how option
> > arguments work, for the most part.
> >
> > We could remove the legacy syntax and shave a couple lines of code.
> >
> > OTOH, supporting it fully is super easy. I've provided a patch.
> >
> The man page says head [-count | -n count] [file ...]
> There are only two valid switches.
> One, the other, or none.
> Using more than one is -undefined-.
> I doubt it ever has been defined.
> It could be asserted that any more than 1 switch is an error.
No.
It wouldn't be an error, it would be undefined. Just like you said.
UB is not an argument for preserving the current behavior. If
anything, UB suggests we should do the least surprising thing. Lucky
for us, the least surprising thing is easy to pinpoint because we
have decades of prior art to use as a guide.
Let's look at the timeline:
- head(1) first appears in 1BSD. It accepted multiple -count
arguments and only used the last one found, if any:
https://github.com/dspinellis/unix-history-repo/blob/BSD-1/s6/head.c#L26
- BSD head(1) then kept this behavior in all releases up through 4.3BSD:
https://github.com/dspinellis/unix-history-repo/blob/BSD-4_3/usr/src/ucb/head.c#L39
- 4.3-Tahoe and 4.3-Reno then lost the behavior:
https://github.com/dspinellis/unix-history-repo/blob/BSD-4_3_Tahoe/usr/src/ucb/head.c#L43
https://github.com/dspinellis/unix-history-repo/blob/BSD-4_3_Reno/usr/src/usr.bin/head/head.c#L45
- Acceptance of multiple -count arguments then reappears in 1992
during the development of 4.4BSD:
https://svnweb.freebsd.org/csrg/usr.bin/head/head.c?revision=52824=markup#l97
- The behavior is then present in all 4.4 releases (Lite1, Lite2, and 4.4):
https://github.com/dspinellis/unix-history-repo/blob/BSD-4_4_Lite1/usr/src/usr.bin/head/head.c#L126
https://github.com/dspinellis/unix-history-repo/blob/BSD-4_4_Lite2/usr/src/usr.bin/head/head.c#L128
https://github.com/dspinellis/unix-history-repo/blob/BSD-4_4/usr/src/usr.bin/head/head.c#L126
- FreeBSD picked up the behavior from one of the 4.4-Lite trees in 1994:
https://cgit.freebsd.org/src/commit/usr.bin/head/head.c?id=9b50d9027575220cb6dd09b3e62f03f511e908b8
- It is present in FreeBSD's head(1) to this day:
https://cgit.freebsd.org/src/tree/usr.bin/head/head.c#n192
- NetBSD picked up the behavior from 4.4-Lite2 in 1997:
http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/head/head.c?rev=1.7=text/x-cvsweb-markup_with_tag=MAIN
- It is present in NetBSD's head(1) to this day:
http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/head/head.c?rev=1.24=text/x-cvsweb-markup_with_tag=MAIN
- DragonflyBSD got their head(1) implementation from FreeBSD in 2003.
They have always had the behavior, and have it to this day:
https://github.com/DragonFlyBSD/DragonFlyBSD/blob/master/usr.bin/head/head.c#L160
- OpenSolaris head(1) supported multiple -count arguments at release
in 2005:
https://github.com/illumos/illumos-gate/blob/7c478bd95313f5f23a4c958a745db2134aa03244/usr/src/cmd/head/head.c#L76
- I don't have the source code but I'd imagine the support for multiple
-count arguments was present in many older Solaris releases before
OpenSolaris appeared.
- illumos (and presumably Solaris) head(1) retains the behavior to
this day:
https://github.com/illumos/illumos-gate/blob/a9e414682948591ec63d5ab2cd11ba55603b59fa/usr/src/cmd/head/head.c#L83
> If you have an old man page that says {[ -count | -n count ] ...}
> then your argument has some merit.
The documentation is not an ironclad guarantee of behavior. Only the
source code can have behavior. I have demonstrated a particular
behavior in the BSD head(1) source code across decades of releases.
--
In summary:
Theo forked NetBSD in 1995, before they imported the -Lite2 changes
for head(1). The fact that OpenBSD's head(1) does not support
multiple -count arguments is an accident of timing, not conscious
choice.
Re: head(1): fully support the legacy -count syntax
On 10/10/21 5:03 PM, Scott Cheloha wrote:
On Sun, Oct 10, 2021 at 02:36:32PM -0600, Theo de Raadt wrote:
Stuart Henderson wrote:
x1> On 2021/10/10 14:26, Scott Cheloha wrote:
On Sun, Oct 10, 2021 at 12:31:22PM -0600, Theo de Raadt wrote:
Bryan Steele wrote:
On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote:
On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote:
did anyone ever use it this way, or are you getting ahead of yourself.
I don't understand the question.
I've only ever seen it used with -count as the first argument, can't
say it's every occoured to me to type "head file -10".
That is not what I proposed. Reread my first message:
https://marc.info/?l=openbsd-tech=163388435528203=2
i.e. "head -2 -3 somefile" is taken as -3.
This is unportable syntax, GNU head doesn't support it, current OpenBSD head
doesn't support it,
... obviously. That's why I posted the patch. :)
and it doesn't seem to be really meaningful.
Additionally I don't think we've ever had a problem with this in ports.
I think we would be better served to keep things as-is and not support it.
Seems that FreeBSD is the odd one out here?
If we're only going to support it as the first argument then we've
created a "gotcha", a special exception to the expected behavior. We
claim to support the legacy syntax but we don't actually fully support
it. The requirement that the legacy syntax be the first option
argument to head(1) in order to work as expected is undocumented.
So we could document this peculiarity...
... or we could just use my tiny patch and fully support it and
everything will work as documented today.
Indeed, the problem is our code supports this
but noone else supports it
NetBSD and FreeBSD support it. It was a standard part of the syntax
in SUSv2. I showed you in a private mail that it was supported in
1BSD through 4.3BSD, and then again in 4.4BSD some time in 1992.
Somewhere along the way between CSRG and NetBSD and our repository
the support was removed.
well, someone might accidentally use it in a script they write on OpenBSD
and... it is unportable, the behaviour is either different, or an error
condition
So who benefits? Noone, the way I see it.
If we want to have the unportable legacy syntax then it should work
like other option arguments. Option arguments can be respecified
multiple times in most other utilities. The last such appearance of
an option argument is the one the utility uses. That's how option
arguments work, for the most part.
We could remove the legacy syntax and shave a couple lines of code.
OTOH, supporting it fully is super easy. I've provided a patch.
The man page says head [-count | -n count] [file ...]
There are only two valid switches.
One, the other, or none.
Using more than one is -undefined-.
I doubt it ever has been defined.
It could be asserted that any more than 1 switch is an error.
If you have an old man page that says {[ -count | -n count ] ...}
then your argument has some merit.
geoff steckel
Re: head(1): fully support the legacy -count syntax
On Sun, Oct 10, 2021 at 02:36:32PM -0600, Theo de Raadt wrote: > Stuart Henderson wrote: > > x1> On 2021/10/10 14:26, Scott Cheloha wrote: > > > On Sun, Oct 10, 2021 at 12:31:22PM -0600, Theo de Raadt wrote: > > > > Bryan Steele wrote: > > > > > > > > > On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > > > > > > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > > > > > > did anyone ever use it this way, or are you getting ahead of > > > > > > > yourself. > > > > > > > > > > > > I don't understand the question. > > > > > > > > > > I've only ever seen it used with -count as the first argument, can't > > > > > say it's every occoured to me to type "head file -10". > > > > > > That is not what I proposed. Reread my first message: > > > > > > https://marc.info/?l=openbsd-tech=163388435528203=2 > > > > i.e. "head -2 -3 somefile" is taken as -3. > > > > This is unportable syntax, GNU head doesn't support it, current OpenBSD head > > doesn't support it, ... obviously. That's why I posted the patch. :) > > and it doesn't seem to be really meaningful. > > Additionally I don't think we've ever had a problem with this in ports. > > I think we would be better served to keep things as-is and not support it. > > Seems that FreeBSD is the odd one out here? If we're only going to support it as the first argument then we've created a "gotcha", a special exception to the expected behavior. We claim to support the legacy syntax but we don't actually fully support it. The requirement that the legacy syntax be the first option argument to head(1) in order to work as expected is undocumented. So we could document this peculiarity... ... or we could just use my tiny patch and fully support it and everything will work as documented today. > Indeed, the problem is our code supports this > > but noone else supports it NetBSD and FreeBSD support it. It was a standard part of the syntax in SUSv2. I showed you in a private mail that it was supported in 1BSD through 4.3BSD, and then again in 4.4BSD some time in 1992. Somewhere along the way between CSRG and NetBSD and our repository the support was removed. > well, someone might accidentally use it in a script they write on OpenBSD > > and... it is unportable, the behaviour is either different, or an error > condition > > So who benefits? Noone, the way I see it. If we want to have the unportable legacy syntax then it should work like other option arguments. Option arguments can be respecified multiple times in most other utilities. The last such appearance of an option argument is the one the utility uses. That's how option arguments work, for the most part. We could remove the legacy syntax and shave a couple lines of code. OTOH, supporting it fully is super easy. I've provided a patch.
Re: head(1): fully support the legacy -count syntax
Stuart Henderson wrote: x1> On 2021/10/10 14:26, Scott Cheloha wrote: > > On Sun, Oct 10, 2021 at 12:31:22PM -0600, Theo de Raadt wrote: > > > Bryan Steele wrote: > > > > > > > On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > > > > > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > > > > > did anyone ever use it this way, or are you getting ahead of > > > > > > yourself. > > > > > > > > > > I don't understand the question. > > > > > > > > I've only ever seen it used with -count as the first argument, can't > > > > say it's every occoured to me to type "head file -10". > > > > That is not what I proposed. Reread my first message: > > > > https://marc.info/?l=openbsd-tech=163388435528203=2 > > i.e. "head -2 -3 somefile" is taken as -3. > > This is unportable syntax, GNU head doesn't support it, current OpenBSD head > doesn't support it, and it doesn't seem to be really meaningful. > Additionally I don't think we've ever had a problem with this in ports. > I think we would be better served to keep things as-is and not support it. > Seems that FreeBSD is the odd one out here? Indeed, the problem is our code supports this but noone else supports it well, someone might accidentally use it in a script they write on OpenBSD and... it is unportable, the behaviour is either different, or an error condition So who benefits? Noone, the way I see it.
Re: head(1): fully support the legacy -count syntax
On Sun, Oct 10, 2021 at 09:11:50PM +0100, Stuart Henderson wrote: > On 2021/10/10 14:26, Scott Cheloha wrote: > > On Sun, Oct 10, 2021 at 12:31:22PM -0600, Theo de Raadt wrote: > > > Bryan Steele wrote: > > > > > > > On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > > > > > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > > > > > did anyone ever use it this way, or are you getting ahead of > > > > > > yourself. > > > > > > > > > > I don't understand the question. > > > > > > > > I've only ever seen it used with -count as the first argument, can't > > > > say it's every occoured to me to type "head file -10". > > > > That is not what I proposed. Reread my first message: > > > > https://marc.info/?l=openbsd-tech=163388435528203=2 > > i.e. "head -2 -3 somefile" is taken as -3. > > This is unportable syntax, GNU head doesn't support it, current OpenBSD head > doesn't support it, and it doesn't seem to be really meaningful. > Additionally I don't think we've ever had a problem with this in ports. > I think we would be better served to keep things as-is and not support it. > Seems that FreeBSD is the odd one out here? > FreeBSD and NetBSD merged an obsolete() function from Lite-2 which converts all -NUMBER arguments into -nNUMBER and then lets getopt take care of picking the last -n argument. If we want to do this, we should probably follow their pattern.
Re: head(1): fully support the legacy -count syntax
On 2021/10/10 14:26, Scott Cheloha wrote: > On Sun, Oct 10, 2021 at 12:31:22PM -0600, Theo de Raadt wrote: > > Bryan Steele wrote: > > > > > On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > > > > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > > > > did anyone ever use it this way, or are you getting ahead of yourself. > > > > > > > > I don't understand the question. > > > > > > I've only ever seen it used with -count as the first argument, can't > > > say it's every occoured to me to type "head file -10". > > That is not what I proposed. Reread my first message: > > https://marc.info/?l=openbsd-tech=163388435528203=2 i.e. "head -2 -3 somefile" is taken as -3. This is unportable syntax, GNU head doesn't support it, current OpenBSD head doesn't support it, and it doesn't seem to be really meaningful. Additionally I don't think we've ever had a problem with this in ports. I think we would be better served to keep things as-is and not support it. Seems that FreeBSD is the odd one out here?
Re: head(1): fully support the legacy -count syntax
On Sun, Oct 10, 2021 at 02:26:32PM -0500, Scott Cheloha wrote: > On Sun, Oct 10, 2021 at 12:31:22PM -0600, Theo de Raadt wrote: > > Bryan Steele wrote: > > > > > On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > > > > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > > > > did anyone ever use it this way, or are you getting ahead of yourself. > > > > > > > > I don't understand the question. > > > > > > I've only ever seen it used with -count as the first argument, can't > > > say it's every occoured to me to type "head file -10". > > That is not what I proposed. Reread my first message: > > https://marc.info/?l=openbsd-tech=163388435528203=2 Yes, sorry. It's been a day.. > > POSIX says options before arguments. That is what we support. We don't > > support options after arguments. > > Yep. >
Re: head(1): fully support the legacy -count syntax
On Sun, Oct 10, 2021 at 12:31:22PM -0600, Theo de Raadt wrote: > Bryan Steele wrote: > > > On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > > > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > > > did anyone ever use it this way, or are you getting ahead of yourself. > > > > > > I don't understand the question. > > > > I've only ever seen it used with -count as the first argument, can't > > say it's every occoured to me to type "head file -10". That is not what I proposed. Reread my first message: https://marc.info/?l=openbsd-tech=163388435528203=2 > POSIX says options before arguments. That is what we support. We don't > support options after arguments. Yep.
Re: head(1): fully support the legacy -count syntax
Bryan Steele wrote: > On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > > did anyone ever use it this way, or are you getting ahead of yourself. > > > > I don't understand the question. > > I've only ever seen it used with -count as the first argument, can't > say it's every occoured to me to type "head file -10". POSIX says options before arguments. That is what we support. We don't support options after arguments.
Re: head(1): fully support the legacy -count syntax
On Sun, Oct 10, 2021 at 12:18:55PM -0500, Scott Cheloha wrote: > On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > > did anyone ever use it this way, or are you getting ahead of yourself. > > I don't understand the question. I've only ever seen it used with -count as the first argument, can't say it's every occoured to me to type "head file -10". > The -count syntax was fully supported in the first revision of head(1): > > https://svnweb.freebsd.org/csrg/usr.bin/head/head.c?view=markup=1026 > > The -count syntax was fully supported through 4.4BSD: > > https://svnweb.freebsd.org/csrg/usr.bin/head/head.c?revision=69237=markup > > The -count syntax was also standard in SUSv2: > > https://pubs.opengroup.org/onlinepubs/7908799/xcu/head.html > > ... and then dropped in SUSv3 (POSIX-2001): > > https://pubs.opengroup.org/onlinepubs/009695399/utilities/head.html > > FreeBSD maintains full support for the -count syntax: > > https://cgit.freebsd.org/src/tree/usr.bin/head/head.c#n191 > > ... so clearly people have used it this way. If we're going to > support the -count syntax at all, why not fully support it? We can do > so with very little code. > >
Re: head(1): fully support the legacy -count syntax
On Sun, Oct 10, 2021 at 10:51:29AM -0600, Theo de Raadt wrote: > did anyone ever use it this way, or are you getting ahead of yourself. I don't understand the question. The -count syntax was fully supported in the first revision of head(1): https://svnweb.freebsd.org/csrg/usr.bin/head/head.c?view=markup=1026 The -count syntax was fully supported through 4.4BSD: https://svnweb.freebsd.org/csrg/usr.bin/head/head.c?revision=69237=markup The -count syntax was also standard in SUSv2: https://pubs.opengroup.org/onlinepubs/7908799/xcu/head.html ... and then dropped in SUSv3 (POSIX-2001): https://pubs.opengroup.org/onlinepubs/009695399/utilities/head.html FreeBSD maintains full support for the -count syntax: https://cgit.freebsd.org/src/tree/usr.bin/head/head.c#n191 ... so clearly people have used it this way. If we're going to support the -count syntax at all, why not fully support it? We can do so with very little code.
Re: head(1): fully support the legacy -count syntax
did anyone ever use it this way, or are you getting ahead of yourself.
Scott Cheloha wrote:
> Hi,
>
> head(1) takes line count arguments in two ways. The legacy (1977)
> syntax is "-count" [1]. The "new" (1992) syntax is "-n count" [2].
> In either case, "count" must be a positive decimal value.
>
> Somewhere along the way, support for the legacy syntax was neutered.
> At present it only works as expected if -count is the first option
> argument to head(1), i.e. this works:
>
> $ head -20 file
>
> but this is an error:
>
> $ head -20 -25 file
>
> I'm not a fan of this. If we're going to support the legacy syntax I
> think it should behave like option arguments for other utilities. You
> should be able to specify -count multiple times. We do this for
> tail(1), which has a similar legacy syntax problem.
>
> The easiest way to transparently support such invocations is to use
> the GNU double-colon extension for getopt(3). We then rebuild the
> number string with asprintf(3), parse it with strtonum(3), and free
> the string.
>
> Before:
>
> $ jot 10 | head -5 -6
> head: unknown option -- 6
> usage: head [-count | -n count] [file ...]
>
> After:
>
> $ jot 10 | head -5 -6
> 1
> 2
> 3
> 4
> 5
> 6
>
> --
>
> ok?
>
> [1]
> https://svnweb.freebsd.org/csrg/usr.bin/head/head.c?revision=1026=markup
>
> [2]
> https://svnweb.freebsd.org/csrg/usr.bin/head/head.c?revision=52824=markup
>
> Index: head.c
> ===
> RCS file: /cvs/src/usr.bin/head/head.c,v
> retrieving revision 1.22
> diff -u -p -r1.22 head.c
> --- head.c10 Oct 2021 15:57:25 - 1.22
> +++ head.c10 Oct 2021 16:44:01 -
> @@ -49,27 +49,30 @@ int
> main(int argc, char *argv[])
> {
> const char *errstr;
> + char *str;
> FILE*fp;
> longcnt;
> int ch, firsttime;
> longlinecnt = 10;
> - int status = 0;
> + int error, status = 0;
>
> if (pledge("stdio rpath", NULL) == -1)
> err(1, "pledge");
>
> - /* handle obsolete -number syntax */
> - if (argc > 1 && argv[1][0] == '-' &&
> - isdigit((unsigned char)argv[1][1])) {
> - linecnt = strtonum(argv[1] + 1, 1, LONG_MAX, );
> - if (errstr != NULL)
> - errx(1, "count is %s: %s", errstr, argv[1] + 1);
> - argc--;
> - argv++;
> - }
> -
> - while ((ch = getopt(argc, argv, "n:")) != -1) {
> +#define OPTSTR "0::1::2::3::4::5::6::7::8::9::n:"
> + while ((ch = getopt(argc, argv, OPTSTR)) != -1) {
> switch (ch) {
> + case '0': case '1': case '2': case '3': case '4':
> + case '5': case '6': case '7': case '8': case '9':
> + error = asprintf(, "%c%s",
> + ch, (optarg == NULL) ? "" : optarg);
> + if (error == -1)
> + err(1, "asprintf");
> + linecnt = strtonum(str, 1, LONG_MAX, );
> + if (errstr != NULL)
> + errx(1, "count is %s: %s", errstr, str);
> + free(str);
> + break;
> case 'n':
> linecnt = strtonum(optarg, 1, LONG_MAX, );
> if (errstr != NULL)
>
