Re: bug#8969: improve synchronization between examples in the manual and test cases

[Stefano Lattarini]

On Friday 01 July 2011, Stefano Lattarini wrote:
> Severity: wishlist
> thanks
> 
> Reference:
>  
> 
> On Tuesday 28 June 2011, Ralf Wildenhues wrote:
> > * Stefano Lattarini wrote on Tue, Jun 28, 2011 at 08:24:13AM CEST:
> > > On Monday 27 June 2011, Ralf Wildenhues wrote:
> > > > Sure.  Well, do they use some format already?
> > > >
> > > Well, there are only two of them, and both follows this format:
> > > 
> > > @c The test case for the setup described here is
> > > @c test/subdircond2.test
> > > @c Try to keep it in sync.
> > > 
> > > (which is wrongish BTW, as the testsuite subdirectory is `tests/',
> > > not `test/')
> > > 
> > > I'd go with one of these simple formats:
> > >   @c Keep in sync with subdircond2.test
> > 
> > I like this one (with a trailing dot ;-)
> > 
> > ;-)
> > 
> I'll soon post a patch that improves the synchronization between
> some examples from the manual and some test cases.  In the process
> of writing it, I've noticed that various non-trivial examples from
> the manual are not tested; and while I've remedied to part of this
> problem with my patch, a lot remains to be done; so I'm opening
> this bug to avoid forgetting about the issue.
> 
> Regards,
>   Stefano
>
And here it is the promised patch.  I will push in 72 hours if there
is no objection by then.

Regards,
  Stefano
From 2dd820b423bea12eac7b94fd8ed2563ee21cf9e1 Mon Sep 17 00:00:00 2001
Message-Id: <2dd820b423bea12eac7b94fd8ed2563ee21cf9e1.1309473910.git.stefano.lattar...@gmail.com>
From: Stefano Lattarini 
Date: Thu, 30 Jun 2011 20:05:25 +0200
Subject: [PATCH] docs, tests: synchronize examples from docs to tests

* tests/README (Writing test cases): Give suggestions on how to
keep test cases and examples in the documentation synchronized.
* doc/automake.texi: Improve or fix existing testcase-referencing
comments, and add many new ones.
* HACKING (Administrivia): Suggest to test complex examples and
idioms from the manual.
* tests/specflg8.test: Improve synchronization with the example
in the manual.
* tests/output11.test:Likewise.
* tests/txinfo21.test:Likewise.
* tests/interp.test: Likewise.  Since we are at it, and enable
the `errexit' shell flag, do related changes, and add trailing
`:'command.
* tests/amhello-cflags.test: New test.
* tests/amhello-cross-compile.test: Likewise.
* tests/amhello-binpkg.test: Likewise.
* tests/tests-environment-backcompat: Likewise.
* tests/parallel-tests-log-compiler-example.test: Likewise.
* tests/Makefile.am (TESTS): Update.
---
 ChangeLog  |   23 
 HACKING|7 ++-
 doc/automake.texi  |   54 ++
 tests/Makefile.am  |5 ++
 tests/Makefile.in  |5 ++
 tests/README   |7 ++
 tests/amhello-binpkg.test  |   44 +++
 tests/amhello-cflags.test  |   49 
 tests/amhello-cross-compile.test   |   54 ++
 tests/interp.test  |   18 --
 tests/parallel-tests-log-compiler-example.test |   71 
 tests/specflg8.test|   35 ++--
 tests/tests-environment-backcompat.test|   65 ++
 tests/txinfo21.test|4 +-
 14 files changed, 402 insertions(+), 39 deletions(-)
 create mode 100755 tests/amhello-binpkg.test
 create mode 100755 tests/amhello-cflags.test
 create mode 100755 tests/amhello-cross-compile.test
 create mode 100755 tests/parallel-tests-log-compiler-example.test
 create mode 100755 tests/tests-environment-backcompat.test

diff --git a/ChangeLog b/ChangeLog
index f1e10c2..c77326d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,26 @@
+2011-07-01  Stefano Lattarini  
+
+	docs, tests: synchronize examples from docs to tests
+	* tests/README (Writing test cases): Give suggestions on how to
+	keep test cases and examples in the documentation synchronized.
+	* doc/automake.texi: Improve or fix existing testcase-referencing
+	comments, and add many new ones.
+	* HACKING (Administrivia): Suggest to test complex examples and
+	idioms from the manual.
+	* tests/specflg8.test: Improve synchronization with the example
+	in the manual.
+	* tests/output11.test:Likewise.
+	* tests/txinfo21.test:Likewise.
+	* tests/interp.test: Likewise.  Since we are at it, and enable
+	the `errexit' shell flag, do related changes, and add trailing
+	`:'command.
+	* tests/amhello-cflags.test: New test.
+	* tests/amhello-cross-compile.test: Likewise.
+	* tests/amhello-binpkg.test: Likewise.
+	* tests/tests-environment-backcompat: Likewise.
+	* tests/parallel-tests-log-compiler-example.test: Likewise.
+	* tests/Makefile.am (TESTS): Update.
+
 2011-06-23  Stefano Lattarini  
 
 	docs: avoid

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Stefano Lattarini]

On Friday 01 July 2011, Stefano Lattarini wrote:
> On Friday 01 July 2011, Stefano Lattarini wrote:
> > I'll soon post a patch that improves the synchronization between
> > some examples from the manual and some test cases.  In the process
> > of writing it, I've noticed that various non-trivial examples from
> > the manual are not tested; and while I've remedied to part of this
> > problem with my patch, a lot remains to be done; so I'm opening
> > this bug to avoid forgetting about the issue.
> > 
> And here it is the promised patch.  I will push in 72 hours if there
> is no objection by then.
> 
Oops, one of the "amhello" tests has an unecessary requirement (copy &
paste blunder).  Consider the following squashed in:

  diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
  index 8da3495..47add73 100755
  --- a/tests/amhello-binpkg.test
  +++ b/tests/amhello-binpkg.test
  @@ -17,7 +17,6 @@
   # Document an example from the manual about the `amhello' package:
   # using DESDIR to build simple, no-frills binary packages.
   
  -required=i586-mingw32msvc-gcc
   . ./defs || Exit 1
   
   set -e

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Stefano Lattarini]

Hello Peter.

On Friday 01 July 2011, Peter Breitenlohner wrote:
> On Fri, 1 Jul 2011, Stefano Lattarini wrote:
> 
> > Oops, one of the "amhello" tests has an unecessary requirement (copy &
> > paste blunder).  Consider the following squashed in:
> >
> >  diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
> >  index 8da3495..47add73 100755
> >  --- a/tests/amhello-binpkg.test
> >  +++ b/tests/amhello-binpkg.test
> >  @@ -17,7 +17,6 @@
> >   # Document an example from the manual about the `amhello' package:
> >   # using DESDIR to build simple, no-frills binary packages.
> 
> Hi Stefano,
> 
> could this be a typo (DESDIR vs DESTDIR)?
>
Yes it is; fixed now.  Thanks for spotting it!

And since we are out to hunt typos, all the new amhello tests have the
same error in the heading descriptions, using: 
  "Document an example from the manual about the `amhello' package:"
where this should be used instead:
  "Test an example from the manual about the `amhello' package:"
Consider these fixed too.

Thanks,
  Stefano

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Peter Breitenlohner]

On Fri, 1 Jul 2011, Stefano Lattarini wrote:


Oops, one of the "amhello" tests has an unecessary requirement (copy &
paste blunder).  Consider the following squashed in:

 diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
 index 8da3495..47add73 100755
 --- a/tests/amhello-binpkg.test
 +++ b/tests/amhello-binpkg.test
 @@ -17,7 +17,6 @@
  # Document an example from the manual about the `amhello' package:
  # using DESDIR to build simple, no-frills binary packages.


Hi Stefano,

could this be a typo (DESDIR vs DESTDIR)?

Regards
Peter Breitenlohner 

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Stefano Lattarini]

On Friday 01 July 2011, Stefano Lattarini wrote:
> Hello Peter.
> 
> On Friday 01 July 2011, Peter Breitenlohner wrote:
> > On Fri, 1 Jul 2011, Stefano Lattarini wrote:
> > 
> > > Oops, one of the "amhello" tests has an unecessary requirement (copy &
> > > paste blunder).  Consider the following squashed in:
> > >
> > >  diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
> > >  index 8da3495..47add73 100755
> > >  --- a/tests/amhello-binpkg.test
> > >  +++ b/tests/amhello-binpkg.test
> > >  @@ -17,7 +17,6 @@
> > >   # Document an example from the manual about the `amhello' package:
> > >   # using DESDIR to build simple, no-frills binary packages.
> > 
> > Hi Stefano,
> > 
> > could this be a typo (DESDIR vs DESTDIR)?
> >
> Yes it is; fixed now.  Thanks for spotting it!
> 
> And since we are out to hunt typos, all the new amhello tests have the
> same error in the heading descriptions, using: 
>   "Document an example from the manual about the `amhello' package:"
> where this should be used instead:
>   "Test an example from the manual about the `amhello' package:"
> Consider these fixed too.
> 
> Thanks,
>   Stefano
>
I've applied this patch to maint, merged maint into master, merged master
into testsuite-work, and pushed.

Regards,
  Stefano

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Ralf Wildenhues]

* Stefano Lattarini wrote on Fri, Jul 01, 2011 at 12:59:53AM CEST:
> Subject: [PATCH] docs, tests: synchronize examples from docs to tests
> 
> * tests/README (Writing test cases): Give suggestions on how to
> keep test cases and examples in the documentation synchronized.
> * doc/automake.texi: Improve or fix existing testcase-referencing
> comments, and add many new ones.
> * HACKING (Administrivia): Suggest to test complex examples and
> idioms from the manual.
> * tests/specflg8.test: Improve synchronization with the example
> in the manual.
> * tests/output11.test:Likewise.
> * tests/txinfo21.test:Likewise.
> * tests/interp.test: Likewise.  Since we are at it, and enable
> the `errexit' shell flag, do related changes, and add trailing
> `:'command.
> * tests/amhello-cflags.test: New test.
> * tests/amhello-cross-compile.test: Likewise.
> * tests/amhello-binpkg.test: Likewise.
> * tests/tests-environment-backcompat: Likewise.
> * tests/parallel-tests-log-compiler-example.test: Likewise.
> * tests/Makefile.am (TESTS): Update.


> --- /dev/null
> +++ b/tests/amhello-binpkg.test
> @@ -0,0 +1,44 @@

> +# Document an example from the manual about the `amhello' package:
> +# using DESDIR to build simple, no-frills binary packages.

(DESTDIR was already fixed IIUC)

> +required=i586-mingw32msvc-gcc
> +. ./defs || Exit 1
> +
> +set -e
> +
> +cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
> +  || fatal_ "cannot get amhello tarball"
> +
> +tar zxf amhello-1.0.tar.gz

The z flag is not portable to all tars, the portable spelling is
  gzip -dc amhello-1.0.tar.gz | tar xf -

(several instances).  I'm ok with fixing the manual also, although it's
usually clear for people still having to use those vendor tars (and
inconvenient for the rest).

> +cd amhello-1.0
> +
> +./configure --prefix /usr
> +make
> +make DESTDIR="`pwd`/inst" install
> +cd inst
> +find . -type f -print > ../files.lst
> +tar cvf amhello-1.0-i686.tar.gz `cat ../files.lst` > t
> +LC_ALL=C sort t > tar.got
> +
> +diff - tar.got <<'END'
> +./usr/bin/hello
> +./usr/share/doc/amhello/README
> +END
[...]

Nice patch btw!

Thanks,
Ralf

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Stefano Lattarini]

On Wednesday 06 July 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Fri, Jul 01, 2011 at 12:59:53AM CEST:
> > Subject: [PATCH] docs, tests: synchronize examples from docs to tests
> > 
> > * tests/README (Writing test cases): Give suggestions on how to
> > keep test cases and examples in the documentation synchronized.
> > * doc/automake.texi: Improve or fix existing testcase-referencing
> > comments, and add many new ones.
> > * HACKING (Administrivia): Suggest to test complex examples and
> > idioms from the manual.
> > * tests/specflg8.test: Improve synchronization with the example
> > in the manual.
> > * tests/output11.test:Likewise.
> > * tests/txinfo21.test:Likewise.
> > * tests/interp.test: Likewise.  Since we are at it, and enable
> > the `errexit' shell flag, do related changes, and add trailing
> > `:'command.
> > * tests/amhello-cflags.test: New test.
> > * tests/amhello-cross-compile.test: Likewise.
> > * tests/amhello-binpkg.test: Likewise.
> > * tests/tests-environment-backcompat: Likewise.
> > * tests/parallel-tests-log-compiler-example.test: Likewise.
> > * tests/Makefile.am (TESTS): Update.
> 
> 
> > --- /dev/null
> > +++ b/tests/amhello-binpkg.test
> > @@ -0,0 +1,44 @@
> 
> > +# Document an example from the manual about the `amhello' package:
> > +# using DESDIR to build simple, no-frills binary packages.
> 
> (DESTDIR was already fixed IIUC)
> 
> > +required=i586-mingw32msvc-gcc
> > +. ./defs || Exit 1
> > +
> > +set -e
> > +
> > +cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
> > +  || fatal_ "cannot get amhello tarball"
> > +
> > +tar zxf amhello-1.0.tar.gz
> 
> The z flag is not portable to all tars, the portable spelling is
>   gzip -dc amhello-1.0.tar.gz | tar xf -
> 
> (several instances).
>
Oops, sorry.  Fixed by the attached patch.  OK for maint?  I'll wait
the customary 72 hours before pushing.

> I'm ok with fixing the manual also, although it's
> usually clear for people still having to use those vendor tars (and
> inconvenient for the rest).
>
Yes, I'd say we leave the examples in manual untouched.  Agreed?

> > +cd amhello-1.0
> > +
> > +./configure --prefix /usr
> > +make
> > +make DESTDIR="`pwd`/inst" install
> > +cd inst
> > +find . -type f -print > ../files.lst
> > +tar cvf amhello-1.0-i686.tar.gz `cat ../files.lst` > t
> > +LC_ALL=C sort t > tar.got
> > +
> > +diff - tar.got <<'END'
> > +./usr/bin/hello
> > +./usr/share/doc/amhello/README
> > +END
> [...]
> 
> Nice patch btw!
>
Thanks :-)

Regards,
  Stefano
From d7e03b029b3ae322ed719ef501810d7334eaf288 Mon Sep 17 00:00:00 2001
Message-Id: 
From: Stefano Lattarini 
Date: Wed, 6 Jul 2011 10:31:03 +0200
Subject: [PATCH] tests: portability fixes in tests on amhello examples

* tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
extract a gzip-compressed tarball, that's unportable to some
tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
idiom instead.
* tests/amhello-cflags.test: Likewise.
* tests/amhello-cross-compile.test: Likewise.

Suggestion from Ralf Wildenhues.
---
 ChangeLog|   11 +++
 tests/amhello-binpkg.test|3 ++-
 tests/amhello-cflags.test|4 +++-
 tests/amhello-cross-compile.test |3 ++-
 4 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index c77326d..c6f8d05 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2011-07-06  Stefano Lattarini  
+
+	tests: portability fixes in tests on amhello examples
+	* tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
+	extract a gzip-compressed tarball, that's unportable to some
+	tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
+	idiom instead.
+	* tests/amhello-cflags.test: Likewise.
+	* tests/amhello-cross-compile.test: Likewise.
+	Suggestion from Ralf Wildenhues.
+
 2011-07-01  Stefano Lattarini  
 
 	docs, tests: synchronize examples from docs to tests
diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
index 8085a23..951f713 100755
--- a/tests/amhello-binpkg.test
+++ b/tests/amhello-binpkg.test
@@ -24,7 +24,8 @@ set -e
 cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
   || fatal_ "cannot get amhello tarball"
 
-tar zxf amhello-1.0.tar.gz
+# "tar zxf" in unportable to some tar implementations.
+gzip -dc amhello-1.0.tar.gz | tar xf -
 cd amhello-1.0
 
 ./configure --prefix /usr
diff --git a/tests/amhello-cflags.test b/tests/amhello-cflags.test
index 1c3e516..25b0a52 100755
--- a/tests/amhello-cflags.test
+++ b/tests/amhello-cflags.test
@@ -26,8 +26,10 @@ set -e
 cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
   || fatal_ "cannot get amhello tarball"
 
-tar zxf amhello-1.0.tar.gz
+# "tar zxf" in unportable to some tar implementations.
+gzip -dc amhello-1.0.tar.gz | tar xf -
 cd amhello-1.0
+
 mkdir debug optim
 cd debug
 ../configure CFLAGS='-g -O0'
diff --git a/tests/amhello-cross-compile.test b/tests/amhello-cross-compile.test
index 862a079..7c05fc0 100755
--- a/tests/amhello-cross-compile.test
+++ b/te

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Ralf Wildenhues]

* Stefano Lattarini wrote on Wed, Jul 06, 2011 at 10:32:56AM CEST:
> Oops, sorry.  Fixed by the attached patch.  OK for maint?  I'll wait
> the customary 72 hours before pushing.

Thanks.  OK, but please remove all the comments about tar unportability
in the tests.  It's so obvious.  ;-)
(and there are probably a dozen other places in the Automake source tree
that you'd have to put the comment at as well, for any amount of
consistency.  I think having one comment, the one that already exists in
lib/am/distdir.am, is fully sufficient.)

> > I'm ok with fixing the manual also, although it's
> > usually clear for people still having to use those vendor tars (and
> > inconvenient for the rest).
> >
> Yes, I'd say we leave the examples in manual untouched.  Agreed?

OK.

> Subject: [PATCH] tests: portability fixes in tests on amhello examples
> 
> * tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
> extract a gzip-compressed tarball, that's unportable to some
> tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
> idiom instead.
> * tests/amhello-cflags.test: Likewise.
> * tests/amhello-cross-compile.test: Likewise.

Thanks,
Ralf

Re: bug#8969: improve synchronization between examples in the manual and test cases

[Stefano Lattarini]

On Wednesday 06 July 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Wed, Jul 06, 2011 at 10:32:56AM CEST:
> > Oops, sorry.  Fixed by the attached patch.  OK for maint?  I'll wait
> > the customary 72 hours before pushing.
> 
> Thanks.  OK, but please remove all the comments about tar unportability
> in the tests.  It's so obvious.  ;-)
>
OK done.

> (and there are probably a dozen other places in the Automake source tree
> that you'd have to put the comment at as well, for any amount of
> consistency.  I think having one comment, the one that already exists in
> lib/am/distdir.am, is fully sufficient.)
> 
> > > I'm ok with fixing the manual also, although it's
> > > usually clear for people still having to use those vendor tars (and
> > > inconvenient for the rest).
> > >
> > Yes, I'd say we leave the examples in manual untouched.  Agreed?
> 
> OK.
> 
> > Subject: [PATCH] tests: portability fixes in tests on amhello examples
> > 
> > * tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
> > extract a gzip-compressed tarball, that's unportable to some
> > tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
> > idiom instead.
> > * tests/amhello-cflags.test: Likewise.
> > * tests/amhello-cross-compile.test: Likewise.
> 
> Thanks,
> Ralf
> 
Pushed now.

Thanks,
  Stefano