Re: AMTAR brokenness

2004-04-23 Thread Alexandre Duret-Lutz
 Paul == Paul Eggert [EMAIL PROTECTED] writes:

 Paul Alexandre Duret-Lutz [EMAIL PROTECTED] writes:
  I've prepared this.  However I'm not really proud of the name
  `filenames-99'.  Maybe someone can suggest some nicer name?

 Paul I don't know what the rules are for Automake option spelling, but how
 Paul about filename-length-max=99?  This is because the obvious
 Paul generalization is to be able to specify whatever length you like.

Thanks for the suggestion.  I'm checking this in and will close the PR.

2004-04-23  Alexandre Duret-Lutz  [EMAIL PROTECTED]

For PR automake/414:
Introduce option filename-length-max=99.
* doc/automake.texi (Options): Document it.
* configure.ac (AM_INIT_AUTOMAKE): Use it.
* lib/Automake/Options.pm (_process_option_list): Recognize it.
* automake.in (handle_dist): Set FILENAME_FILTER.
* lib/am/distdir.am (distdir) [FILENAME_FILTER]: Diagnose long
filenames.
* tests/fn99.test: New file.
* tests/Makefile.am (TESTS): Add fn99.test.

Index: NEWS
===
RCS file: /cvs/automake/automake/NEWS,v
retrieving revision 1.268
diff -u -r1.268 NEWS
--- NEWS19 Apr 2004 23:20:07 -  1.268
+++ NEWS23 Apr 2004 21:55:04 -
@@ -45,8 +45,8 @@
 * Diagnose AC_CONFIG_AUX_DIR calls following AM_INIT_AUTOMAKE. (PR/49)
 
 * Tar format can be chosen with the new options tar-v7, tar-ustar, and
-  tar-pax.
-
+  tar-pax.  The new option filename-length-max=99 helps diagnosing
+  filenames that are too long for tar-v7.
 
 New in 1.8:
 
Index: automake.in
===
RCS file: /cvs/automake/automake/automake.in,v
retrieving revision 1.1553
diff -u -r1.1553 automake.in
--- automake.in 17 Apr 2004 18:46:05 -  1.1553
+++ automake.in 23 Apr 2004 21:55:08 -
@@ -3498,9 +3498,13 @@
 if rule 'dist-hook';
   $transform{'DIST-TARGETS'} = join(' ', @dist_targets);
 
+  my $flm = option ('filename-length-max');
+  my $filename_filter = $flm ? '.' x $flm-[1] : '';
+
   $output_rules .= file_contents ('distdir',
   new Automake::Location,
-  %transform);
+  %transform,
+  FILENAME_FILTER = $filename_filter);
 }
 
 
Index: configure.ac
===
RCS file: /cvs/automake/automake/configure.ac,v
retrieving revision 1.14
diff -u -r1.14 configure.ac
--- configure.ac10 Apr 2004 17:18:01 -  1.14
+++ configure.ac23 Apr 2004 21:55:10 -
@@ -1,6 +1,6 @@
 # Process this file with autoconf to produce a configure script.
 
-# Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003
+# Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004
 # Free Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
@@ -36,7 +36,7 @@
 # bothered by `missing'.
 am_AUTOCONF=${AUTOCONF-autoconf}
 
-AM_INIT_AUTOMAKE([1.6 dist-bzip2])
+AM_INIT_AUTOMAKE([1.6 dist-bzip2 filename-length-max=99])
 
 # The API version is the base version.  We must guarantee
 # compatibility for all releases with the same API version.
Index: doc/automake.texi
===
RCS file: /cvs/automake/automake/doc/automake.texi,v
retrieving revision 1.32
diff -u -r1.32 automake.texi
--- doc/automake.texi   19 Apr 2004 23:20:07 -  1.32
+++ doc/automake.texi   23 Apr 2004 21:55:16 -
@@ -5812,6 +5812,16 @@
 Hook @code{dist-tarZ} to @code{dist}.
 @trindex dist-tarZ
 
[EMAIL PROTECTED] @code{filename-length-max=99}
[EMAIL PROTECTED] Option, filename-length-max=99
[EMAIL PROTECTED] filename-length-max=99
+Abort if filenames longer than 99 characters are found during
[EMAIL PROTECTED] dist}.  Such long filenames are generally considered not to
+be portable in tarballs.  See the @code{tar-v7} and @code{tar-ustar}
+options below.  This option should be used in the top-level
[EMAIL PROTECTED] or as an argument of @code{AM_INIT_AUTOMAKE} in
[EMAIL PROTECTED], it will be ignored otherwise.
+
 @item @code{no-define}
 @cindex Option, no-define
 This options is meaningful only when passed as an argument to
@@ -5919,6 +5929,17 @@
 @item @code{tar-v7}
 @itemx @code{tar-ustar}
 @itemx @code{tar-pax}
[EMAIL PROTECTED] Option, tar-v7
[EMAIL PROTECTED] Option, tar-ustar
[EMAIL PROTECTED] Option, tar-pax
[EMAIL PROTECTED] tar formats
[EMAIL PROTECTED] v7 tar format
[EMAIL PROTECTED] ustar format
[EMAIL PROTECTED] pax format
[EMAIL PROTECTED] tar-v7
[EMAIL PROTECTED] tar-ustar
[EMAIL PROTECTED] tar-pax
+
 These three mutually exclusive options select the tar format to use
 when generating tarballs with @code{make dist}.  (The tar file created
 is then compressed according to the set of @code{no-dist-gzip},
@@ -5935,7 

Re: AMTAR brokenness

2004-04-20 Thread Alexandre Duret-Lutz
 Paul == Paul Eggert [EMAIL PROTECTED] writes:

[...]

 Paul Many tar implementations have trouble with path names longer than 99
 Paul bytes.  This includes the current GNU tar official latest non-alpha
 Paul release (which is buggy in this area).  It would be reasonable to add
 Paul an automake option that checks for longer-than-99-byte file names, for
 Paul people who are worried about such things.  But I don't think it needs
 Paul to be high priority.

I've prepared this.  However I'm not really proud of the name
`filenames-99'.  Maybe someone can suggest some nicer name?

2004-04-20  Alexandre Duret-Lutz  [EMAIL PROTECTED]

For PR automake/414:
Introduce option filenames-99.
* doc/automake.texi (Options): Document it.
* configure.ac (AM_INIT_AUTOMAKE): Use it.
* lib/Automake/Options.pm (_process_option_list): Recognize it.
* automake.in (handle_dist): Set FILENAMES_99.
* lib/am/distdir.am (distdir) [FILENAMES_99]: Diagnose long
filenames.
* tests/fn99.test: New file.
* tests/Makefile.am (TESTS): Add fn99.test.

Index: NEWS
===
RCS file: /cvs/automake/automake/NEWS,v
retrieving revision 1.268
diff -u -r1.268 NEWS
--- NEWS19 Apr 2004 23:20:07 -  1.268
+++ NEWS20 Apr 2004 20:30:56 -
@@ -45,8 +45,8 @@
 * Diagnose AC_CONFIG_AUX_DIR calls following AM_INIT_AUTOMAKE. (PR/49)
 
 * Tar format can be chosen with the new options tar-v7, tar-ustar, and
-  tar-pax.
-
+  tar-pax.  The new option filenames-99 helps diagnosing filenames
+  that are too long for tar-v7.
 
 New in 1.8:
 
Index: automake.in
===
RCS file: /cvs/automake/automake/automake.in,v
retrieving revision 1.1553
diff -u -r1.1553 automake.in
--- automake.in 17 Apr 2004 18:46:05 -  1.1553
+++ automake.in 20 Apr 2004 20:31:00 -
@@ -3500,7 +3500,8 @@
 
   $output_rules .= file_contents ('distdir',
   new Automake::Location,
-  %transform);
+  %transform,
+  FILENAMES_99 = !! option 'filenames-99');
 }
 
 
Index: configure.ac
===
RCS file: /cvs/automake/automake/configure.ac,v
retrieving revision 1.14
diff -u -r1.14 configure.ac
--- configure.ac10 Apr 2004 17:18:01 -  1.14
+++ configure.ac20 Apr 2004 20:31:02 -
@@ -1,6 +1,6 @@
 # Process this file with autoconf to produce a configure script.
 
-# Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003
+# Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004
 # Free Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
@@ -36,7 +36,7 @@
 # bothered by `missing'.
 am_AUTOCONF=${AUTOCONF-autoconf}
 
-AM_INIT_AUTOMAKE([1.6 dist-bzip2])
+AM_INIT_AUTOMAKE([1.6 dist-bzip2 filenames-99])
 
 # The API version is the base version.  We must guarantee
 # compatibility for all releases with the same API version.
Index: doc/automake.texi
===
RCS file: /cvs/automake/automake/doc/automake.texi,v
retrieving revision 1.32
diff -u -r1.32 automake.texi
--- doc/automake.texi   19 Apr 2004 23:20:07 -  1.32
+++ doc/automake.texi   20 Apr 2004 20:31:07 -
@@ -5812,6 +5812,16 @@
 Hook @code{dist-tarZ} to @code{dist}.
 @trindex dist-tarZ
 
[EMAIL PROTECTED] @code{filenames-99}
[EMAIL PROTECTED] Option, filenames-99
[EMAIL PROTECTED] filenames-99
+Abort if filenames longer than 99 characters are found during
[EMAIL PROTECTED] dist}.  Such long filenames are generally considered not to
+be portable in tarballs.  See the @code{tar-v7} and @code{tar-ustar}
+options below.  This option should be used in the top-level
[EMAIL PROTECTED] or as an argument of @code{AM_INIT_AUTOMAKE} in
[EMAIL PROTECTED], it will be ignored otherwise.
+
 @item @code{no-define}
 @cindex Option, no-define
 This options is meaningful only when passed as an argument to
@@ -5919,6 +5929,17 @@
 @item @code{tar-v7}
 @itemx @code{tar-ustar}
 @itemx @code{tar-pax}
[EMAIL PROTECTED] Option, tar-v7
[EMAIL PROTECTED] Option, tar-ustar
[EMAIL PROTECTED] Option, tar-pax
[EMAIL PROTECTED] tar formats
[EMAIL PROTECTED] v7 tar format
[EMAIL PROTECTED] ustar format
[EMAIL PROTECTED] pax format
[EMAIL PROTECTED] tar-v7
[EMAIL PROTECTED] tar-ustar
[EMAIL PROTECTED] tar-pax
+
 These three mutually exclusive options select the tar format to use
 when generating tarballs with @code{make dist}.  (The tar file created
 is then compressed according to the set of @code{no-dist-gzip},
@@ -5935,7 +5956,8 @@
 given longer filenames some tar implementations will diagnose the
 problem while other will generate broken tarballs or use non-portable
 extensions.  Furthermore, the 

Re: AMTAR brokenness

2004-04-20 Thread Paul Eggert
Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

 I've prepared this.  However I'm not really proud of the name
 `filenames-99'.  Maybe someone can suggest some nicer name?

I don't know what the rules are for Automake option spelling, but how
about filename-length-max=99?  This is because the obvious
generalization is to be able to specify whatever length you like.




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-19 Thread Paul Eggert
Thanks for writing that.  Some minor points:

Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

 [EMAIL PROTECTED] selects the new pax format defined by POSIX
 +1003.1-2001.  It supports filenames with up to 65535 characters.

Hmm, where did that 65535 come from?  I don't know of any limit of
65535 bytes (presumably you meant bytes, not characters) that is
required by POSIX.

Also, there are moves to change the pax format (so far in an
upward-compatible way, but you never know).  Perhaps you should
mention that tar-pax is intended to be the most recent version of
the pax interchange format, not necessarily the 2001 version.  (This
problem probably afflicts the GNU tar documentation as well)

 +# Substitute a variable $(am__untar) that exctract such

exctract - extract

 +  cpio)
 +am__tar='find $$tardir -print | cpio -H $1 -L -o'
 +am__tar_='find $tardir -print | cpio -H $1 -L -o'
 +am__untar='cpio -i'

Wouldn't it be safer to use 'cpio -H tar -i' rather than 'cpio -i'?
The latter autodetects, but I worry that autodetection isn't infallible.
(I don't know about cpio to be sure here.)




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-19 Thread Alexandre Duret-Lutz
 Paul == Paul Eggert [EMAIL PROTECTED] writes:

 Paul Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

  [EMAIL PROTECTED] selects the new pax format defined by POSIX
  +1003.1-2001.  It supports filenames with up to 65535 characters.

 Paul Hmm, where did that 65535 come from?  I don't know of any limit of
 Paul 65535 bytes (presumably you meant bytes, not characters) that is
 Paul required by POSIX.

I read that in the Heirloom pax manpage
  http://heirloom.sourceforge.net/man/pax.1.html

Maybe its maintainer can enlighten us.

 Paul Also, there are moves to change the pax format (so far in an
 Paul upward-compatible way, but you never know).  Perhaps you should
 Paul mention that tar-pax is intended to be the most recent version of
 Paul the pax interchange format, not necessarily the 2001 version.  (This
 Paul problem probably afflicts the GNU tar documentation as well)

Will do.

  +  cpio)
  +am__tar='find $$tardir -print | cpio -H $1 -L -o'
  +am__tar_='find $tardir -print | cpio -H $1 -L -o'
  +am__untar='cpio -i'

 Paul Wouldn't it be safer to use 'cpio -H tar -i' rather than 'cpio -i'?
 Paul The latter autodetects, but I worry that autodetection isn't infallible.
 Paul (I don't know about cpio to be sure here.)

It shouldn't hurt.
-- 
Alexandre Duret-Lutz





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-19 Thread Sergey Poznyakoff
Alexandre Duret-Lutz [EMAIL PROTECTED] wrote:

 [EMAIL PROTECTED] selects the new pax format defined by POSIX
 +1003.1-2001.  It supports filenames with up to 65535 characters.

POSIX 1003.1-2001 specification does not impose any limit on
lengths of files stored in PAX interchange format.

Regards,
Sergey




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-19 Thread Gunnar Ritter
Alexandre Duret-Lutz [EMAIL PROTECTED] wrote:

  Paul == Paul Eggert [EMAIL PROTECTED] writes:

  Paul Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

   [EMAIL PROTECTED] selects the new pax format defined by POSIX
   +1003.1-2001.  It supports filenames with up to 65535 characters.

  Paul Hmm, where did that 65535 come from?  I don't know of any limit of
  Paul 65535 bytes (presumably you meant bytes, not characters) that is
  Paul required by POSIX.

 I read that in the Heirloom pax manpage
   http://heirloom.sourceforge.net/man/pax.1.html

 Maybe its maintainer can enlighten us.

The reason why I introduced this was the following: When cpio/pax
encounters a broken archive with a very large number in the file
name length field, it tries to allocate a lot of memory to store
the file name; it will eventually fail to allocate that memory
and terminate operation immediately. That's bad because if the
header was just skipped, another valid header could follow (with
-k/-K specified).

Thus I introduced some sane limit here. Since cpio does not
currently change its working directory, it is restriced to the
open() limit of PATH_MAX anyway and will get ENAMETOOLONG, which
is at 4096 on most systems. I chose 65535 because I've never seen
a Unix system which even remotely supports names of such length
for open(). But the value is arbitrary and could be raised to any
value as long as enough memory is available to store a file name.

Gunnar

-- 
http://omnibus.ruf.uni-freiburg.de/~gritter




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-19 Thread Alexandre Duret-Lutz
 adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

[...]
 Paul the existing code uses the -o option, which is [not portable]
[...]

Speaking about `o', I've just discovered that `missing' contains
some magic to strip that flag when tar fails, and retry.

It means that Roger's troubles should vanish once tar is changed
to report filenames too long for `o' with $? != 0.
-- 
Alexandre Duret-Lutz





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-19 Thread Alexandre Duret-Lutz
Here is the patch I'm installing.  Besides the doc update, and
the cpio -H $1 -i thing, I also changed the name of the cache
variable to include the _AM_PROG_TAR argument (so that
subpackages with different tar-xxx option do not share the same
cache variable).

As I've said, I'm also deliberately telling the tar2.test
failing on system that cannot create pax archive to get some
feedback from people using CVS Automake.  (I tried Darwin and
Solaris8, where this test fails.)  I'll correct this before the
release.

2004-04-20  Alexandre Duret-Lutz  [EMAIL PROTECTED]

For PR automake/414:
Introduce options tar-v7, tar-ustar, and tar-pax to select
tar format.
* doc/automake.texi (Options): Document them.
* lib/Automake/Options.pm (_process_option_list): Process
these new options.
* lib/am/distdir.am (dist-gzip, dist-bzip2, dist-tarZ, dist,
distcheck): Adjust to use am__tar and am__untar.
* m4/tar.m4: New file.
* m4/Makefile.am (dist_m4data_DATA): Add tar.m4.
* m4/init.m4 (AM_INIT_AUTOMAKE): Support the new options
and call _AM_PROG_TAR.
* tests/tar.test, tests/tar2.test, tests/tar3.test: New files.
* tests/Makefile.am (TESTS): Add them.

Index: NEWS
===
RCS file: /cvs/automake/automake/NEWS,v
retrieving revision 1.267
diff -u -r1.267 NEWS
--- NEWS17 Apr 2004 18:46:05 -  1.267
+++ NEWS19 Apr 2004 23:04:37 -
@@ -44,6 +44,9 @@
 
 * Diagnose AC_CONFIG_AUX_DIR calls following AM_INIT_AUTOMAKE. (PR/49)
 
+* Tar format can be chosen with the new options tar-v7, tar-ustar, and
+  tar-pax.
+
 
 New in 1.8:
 
Index: doc/automake.texi
===
RCS file: /cvs/automake/automake/doc/automake.texi,v
retrieving revision 1.31
diff -u -r1.31 automake.texi
--- doc/automake.texi   2 Apr 2004 07:14:26 -   1.31
+++ doc/automake.texi   19 Apr 2004 23:04:44 -
@@ -5916,6 +5916,46 @@
 @file{subdir/file.cxx}, then the output file would be
 @file{subdir/file.o}.
 
[EMAIL PROTECTED] @code{tar-v7}
[EMAIL PROTECTED] @code{tar-ustar}
[EMAIL PROTECTED] @code{tar-pax}
+These three mutually exclusive options select the tar format to use
+when generating tarballs with @code{make dist}.  (The tar file created
+is then compressed according to the set of @code{no-dist-gzip},
[EMAIL PROTECTED] and @code{dist-tarZ} options in use.)
+
+These options must be passed as argument to @code{AM_INIT_AUTOMAKE}
+(@xref{Macros}) because they can causes new configure check to be
+performed.  Automake will complain if it sees such option in a
[EMAIL PROTECTED] variable.
+
[EMAIL PROTECTED] selects the old V7 tar format.  This is the historical
+default.  This antiquated format is understood by all tar
+implementations and supports filenames with up to 99 characters. When
+given longer filenames some tar implementations will diagnose the
+problem while other will generate broken tarballs or use non-portable
+extensions.  Furthermore, the V7 format cannot store empty
+directories.
+
[EMAIL PROTECTED] selects the ustar format defined by POSIX
+1003.1-1988.  This format is believed to be old enough to be portable.
+It fully supports directories, and stores filenames with up to 255
+characters.  However you may run against broken tar implementations
+that incorrectly handle filenames longer than 99 characters (please
+report them to @email{bug-automake@@gnu.org} so we can document this
+accurately).
+
[EMAIL PROTECTED] selects the new pax interchange format defined by POSIX
+1003.1-2001.  It does not limit the length of filenames.  However,
+this format is very young and should probably be restricted to
+packages which target only very modern platforms.  There are moves to
+change the pax format in an upward-compatible way, so this option may
+refer to a more recent version in the future.
+
[EMAIL PROTECTED] knows several ways to construct these formats.  It
+will not abort if it cannot find a tool up to the task (so that the
+package can still be built), but @code{make dist} will fail.
+
 @item @var{version}
 @cindex Option, version
 A version number (e.g. @samp{0.30}) can be specified.  If Automake is not
Index: lib/Automake/Options.pm
===
RCS file: /cvs/automake/automake/lib/Automake/Options.pm,v
retrieving revision 1.2
diff -u -r1.2 Options.pm
--- lib/Automake/Options.pm 30 Sep 2003 19:05:54 -  1.2
+++ lib/Automake/Options.pm 19 Apr 2004 23:04:44 -
@@ -1,4 +1,4 @@
-# Copyright (C) 2003  Free Software Foundation, Inc.
+# Copyright (C) 2003, 2004  Free Software Foundation, Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -270,6 +270,22 @@
 || $_ eq 'cygnus' || $_ eq 'no-dependencies')
{
 

Re: [Bug-tar] Re: AMTAR brokenness

2004-04-18 Thread Sergey Poznyakoff
Alexandre Duret-Lutz [EMAIL PROTECTED] wrote:

  Sergey able to produce correct 'ustar' archives. Whatever
  Sergey options you give to previous versions of GNU tar, the
  Sergey produced archives will deviate from the standard.
 
 Ouch!  Is that a `will always deviate' or a `can deviate in some
 conditions (like -o does)'?

It is will always deviate.

 So as I sees it the question is: in this conditions, will
 old-GNU-tar-without-o deviate more from ustar than
 old-GNU-tar-with-o deviates from V7?

The distance between the produced archive and the required
format will be almost the same in both cases. The main difference
when using -o is that archives produced with this option do not
contain symbolic ownership information (usernames/group names).

Regards,
Sergey 




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-18 Thread Sergey Poznyakoff
Roger Leigh [EMAIL PROTECTED] wrote:

 I didn't realise it supported generating tar files.  For the cpio I
 have (GNU cpio version 2.5):
 
ustar  The POSIX.1 tar format.  Also recognizes GNU tar archives,
   which are similar but not identical.
 
 How does this differ from tar --format=posix.  Isn't the POSIX tar
 format for the current GNU tar now the same thing?

No, it isn't. The option --format=posix means POSIX.1-2001 format
(a.k.a. pax format), whereas ustar is POSIX.1-1988. The former
is atually a superset of the latter.

Regards,
Sergey




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-18 Thread Sergey Poznyakoff
Paul Eggert [EMAIL PROTECTED] wrote:

 Given all the hassles that will accompany any change, perhaps we
 should give the maintainer a more gradual upgrade path.
 
 For example, we could add an automake macro AM_TAR_FORMAT.
[...]

That's a nice idea. I'd vote for it.

 I'd put cpio last on the list, with the lowest priority.

I agree.

 Traditional
 UNIX cpio had serious bugs whenever two files had inode numbers that
 were the same modulo 2**16.  This problem persisted for many years (up
 until I stopped using cpio -- perhaps it still has the bug for all I
 know).  GNU cpio cannot yet handle large files or archives (i.e.,
 larger than 2 GiB on typical x86 hosts).

The CVS version is able to handle large files, however most
distributions still ship cpio binaries compiled without large
file support.

Regards,
Sergey




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-18 Thread Alexandre Duret-Lutz
 Paul == Paul Eggert [EMAIL PROTECTED] writes:

[...]
 Paul Given all the hassles that will accompany any change, perhaps we
 Paul should give the maintainer a more gradual upgrade path.

 Paul For example, we could add an automake macro AM_TAR_FORMAT.
 Paul AM_TAR_FORMAT([v7]) would use the current behavior (flawed as it is --
 Paul at least it's something people are familiar with).
 Paul AM_TAR_FORMAT([ustar]) would attempt to generate ustar format, using
 Paul the commands tar --format=ustar, tar (if not GNU tar), pax -x
 Paul ustar, and cpio -H ustar, in that order.  Likewise for other
 Paul formats as we find the need to support them.  
[...]

Sounds great.  Here is a preliminary patch for this.  I have
still to test it more extensively, but that should be enough for
comments.

(BTW the real purpose of tar2.test is to get report from
people on platforms where the pax format is not supported.)

2004-04-18  Alexandre Duret-Lutz  [EMAIL PROTECTED]

For PR automake/414:
Introduce options tar-v7, tar-ustar, and tar-pax to select
tar format.
* doc/automake.texi (Options): Document them.
* lib/Automake/Options.pm (_process_option_list): Process
these new options.
* lib/am/distdir.am (dist-gzip, dist-bzip2, dist-tarZ, dist,
distcheck): Adjust to use am__tar and am__untar.
* m4/tar.m4: New file.
* m4/Makefile.am (dist_m4data_DATA): Add tar.m4.
* m4/init.m4 (AM_INIT_AUTOMAKE): Support the new options
and call _AM_PROG_TAR.
* tests/tar.test, tests/tar2.test, tests/tar3.test: New files.
* tests/Makefile.am (TESTS): Add them.

Index: NEWS
===
RCS file: /cvs/automake/automake/NEWS,v
retrieving revision 1.267
diff -u -r1.267 NEWS
--- NEWS17 Apr 2004 18:46:05 -  1.267
+++ NEWS18 Apr 2004 21:12:28 -
@@ -44,6 +44,9 @@
 
 * Diagnose AC_CONFIG_AUX_DIR calls following AM_INIT_AUTOMAKE. (PR/49)
 
+* Tar format can be chosen with the new options tar-v7, tar-ustar, and
+  tar-pax.
+
 
 New in 1.8:
 
Index: doc/automake.texi
===
RCS file: /cvs/automake/automake/doc/automake.texi,v
retrieving revision 1.31
diff -u -r1.31 automake.texi
--- doc/automake.texi   2 Apr 2004 07:14:26 -   1.31
+++ doc/automake.texi   18 Apr 2004 21:12:34 -
@@ -5916,6 +5916,40 @@
 @file{subdir/file.cxx}, then the output file would be
 @file{subdir/file.o}.
 
[EMAIL PROTECTED] @code{tar-v7}
[EMAIL PROTECTED] @code{tar-ustar}
[EMAIL PROTECTED] @code{tar-pax}
+These three mutually exclusive options select the tar format to use
+when generating tarballs with @code{make dist}.  (The tar file created
+is then compressed according to the set of @code{no-dist-gzip},
[EMAIL PROTECTED] and @code{dist-tarZ} options in use.)
+
+These options must be passed as argument to @code{AM_INIT_AUTOMAKE}
+(@xref{Macros}) because they can causes new configure check to be
+performed.  Automake will complain if it sees such option in a
[EMAIL PROTECTED] variable.
+
[EMAIL PROTECTED] selects the old V7 tar format.  This is the historical
+default.  This antiquated format is understood by all tar
+implementations and supports filenames with up to 99 characters. When
+given longer filenames some tar implementations will diagnose the
+problem while other will generate broken tarballs or use non-portable
+extensions.  Furthermore, the V7 format cannot store empty
+directories.
+
[EMAIL PROTECTED] selects the ustar format defined by POSIX
+1003.1-1988.  This format is believed to be old enough to be portable.
+It fully supports directories, and stores filenames with up to 255
+characters.  However you may run against broken tar implementations
+that incorrectly handle filenames longer than 99 characters (please
+report them to @email{bug-automake@@gnu.org} so we can document this
+accurately).
+
[EMAIL PROTECTED] selects the new pax format defined by POSIX
+1003.1-2001.  It supports filenames with up to 65535 characters.
+However this format is very young and should probably be avoided in
+all packages that do not target only modern systems.
+
 @item @var{version}
 @cindex Option, version
 A version number (e.g. @samp{0.30}) can be specified.  If Automake is not
Index: lib/Automake/Options.pm
===
RCS file: /cvs/automake/automake/lib/Automake/Options.pm,v
retrieving revision 1.2
diff -u -r1.2 Options.pm
--- lib/Automake/Options.pm 30 Sep 2003 19:05:54 -  1.2
+++ lib/Automake/Options.pm 18 Apr 2004 21:12:35 -
@@ -1,4 +1,4 @@
-# Copyright (C) 2003  Free Software Foundation, Inc.
+# Copyright (C) 2003, 2004  Free Software Foundation, Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -270,6 +270,22 @@
 || $_ eq 

Re: [Bug-tar] Re: AMTAR brokenness

2004-04-18 Thread Eric Sunshine
On Sun, 18 Apr 2004 23:18:05 +0200, Alexandre Duret-Lutz wrote:
 [EMAIL PROTECTED] selects the new pax format defined by POSIX
 +1003.1-2001.  It supports filenames with up to 65535 characters.
 +However this format is very young and should probably be avoided in
 +all packages that do not target only modern systems.

Perhaps, a bit more idiomatic:

However, this format is very young and should probably be restricted to  
packages which target only very modern platforms.

-- ES




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-17 Thread Paul Eggert
Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

 How about this scheme:

In the light of later discussion, how about this scheme instead?

  Use the first of the following commands that works:

tar --format=ustar
tar
pax -x ustar

I wouldn't bother with POSIXLY_CORRECT=1 or tar -o, as they don't
really help much these days.

 On second though, shouldn't we try to use pax first?  tar is no
 longer a POSIX requirement, right?

The tar command has never been a POSIX requirement.  However, pax
has never caught on, for various reasons, and it hasn't been
road-tested as much as tar has.  It makes sense to use pax if
tar is not available, but I wouldn't make pax my first choice.

 Also do we really need to try `-o'?

I vaguely recall that RMS used to suggest -o in the GNU coding
standards, for compatibility with v7 'tar', but that requirement is no
longer necessary (it's not in the standards now, anyway).

 Besides GNU tar, is there some tar implementation that does not
 default to the ustar format?

Many tar implementations have trouble with path names longer than 99
bytes.  This includes the current GNU tar official latest non-alpha
release (which is buggy in this area).  It would be reasonable to add
an automake option that checks for longer-than-99-byte file names, for
people who are worried about such things.  But I don't think it needs
to be high priority.




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-17 Thread Ralph Schleicher
Roger Leigh [EMAIL PROTECTED] writes:

 I'm using the patch I posted.  If and when anyone complains, I'll
 instruct them to build and install the latest GNU tar (or build it for
 them).  I require the functionality for long pathnames, and I don't
 want to have to support broken tools.  For my needs, 99 chars is not
 at all sufficient.

Have you ever considered using 'cpio -H ustar' instead of tar?

-- 
Ralph




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-17 Thread Alexandre Duret-Lutz
 Paul == Paul Eggert [EMAIL PROTECTED] writes:

 Paul Alexandre Duret-Lutz [EMAIL PROTECTED] writes:
  How about this scheme:

 Paul In the light of later discussion, how about this scheme instead?

 Paul Use the first of the following commands that works:

 Paul tar --format=ustar
 Paul tar
 Paul pax -x ustar

This looks satisfactory when GNU tar 1.13.93 is installed, or
when GNU tar is not installed.  However, on most GNU/Linux
setups, which have GNU tar 1.13.25 installed, my understanding
is that will cause GNU-tarballs to be produced.

What if we try hard not to use such old GNU tar versions?

   Use `tar --format=ustar' if possible.
   If `tar' is GNU's:
  Use `pax -x ustar' if that works,
  or fall back to plain `tar'.
   Otherwise
  Use plain `tar',
  or fall back to `pax -x ustar'.

[...]

 Paul The tar command has never been a POSIX requirement.  However, pax
 Paul has never caught on, for various reasons, and it hasn't been
 Paul road-tested as much as tar has.  It makes sense to use pax if
 Paul tar is not available, but I wouldn't make pax my first choice.

I trust you.  However my impression from Sergey comments is that
older GNU tar versions should not be a choice at all when it
comes to creating ustar archives :(

What about Ralph's `cpio -H ustar' suggestion?  I've never used
cpio.  It looks like the -H option is non-standard, but is
supported by many implementations.  Probably this is even less
exercised than `pax -x ustar'.

 Paul Many tar implementations have trouble with path names
 Paul longer than 99 bytes.  This includes the current GNU tar
 Paul official latest non-alpha release (which is buggy in this
 Paul area).  It would be reasonable to add an automake option
 Paul that checks for longer-than-99-byte file names, for
 Paul people who are worried about such things.  

I agree.
-- 
Alexandre Duret-Lutz





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-17 Thread Roger Leigh
Ralph Schleicher [EMAIL PROTECTED] writes:

 Roger Leigh [EMAIL PROTECTED] writes:

 I'm using the patch I posted.  If and when anyone complains, I'll
 instruct them to build and install the latest GNU tar (or build it for
 them).  I require the functionality for long pathnames, and I don't
 want to have to support broken tools.  For my needs, 99 chars is not
 at all sufficient.

 Have you ever considered using 'cpio -H ustar' instead of tar?

I didn't realise it supported generating tar files.  For the cpio I
have (GNU cpio version 2.5):

   ustar  The POSIX.1 tar format.  Also recognizes GNU tar archives,
  which are similar but not identical.

How does this differ from tar --format=posix.  Isn't the POSIX tar
format for the current GNU tar now the same thing?


-- 
Roger Leigh

Printing on GNU/Linux?  http://gimp-print.sourceforge.net/
GPG Public Key: 0x25BFB848.  Please sign and encrypt your mail.




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-17 Thread Paul Eggert
Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

  Paul tar --format=ustar
  Paul tar
  Paul pax -x ustar

 This looks satisfactory when GNU tar 1.13.93 is installed, or
 when GNU tar is not installed.  However, on most GNU/Linux
 setups, which have GNU tar 1.13.25 installed, my understanding
 is that will cause GNU-tarballs to be produced.

OK, but hold on a second: what's the goal here?  If the goal is to
produce ustar format, then you're correct: we shouldn't try plain
tar (without checking it, anyway).  But if the goal is to produce
tarballs that work, then the most likely consumer of the tarball right
now is GNU tar 1.13.25 or earlier -- which chokes on some ustar
tarballs.

Given all the hassles that will accompany any change, perhaps we
should give the maintainer a more gradual upgrade path.

For example, we could add an automake macro AM_TAR_FORMAT.
AM_TAR_FORMAT([v7]) would use the current behavior (flawed as it is --
at least it's something people are familiar with).
AM_TAR_FORMAT([ustar]) would attempt to generate ustar format, using
the commands tar --format=ustar, tar (if not GNU tar), pax -x
ustar, and cpio -H ustar, in that order.  Likewise for other
formats as we find the need to support them.  The default could be v7
for now, but we can warn people that it is subject to change: it could
change to ustar, then posix, for example.

That way, adventurous maintainers could use AM_TAR_FORMAT([ustar])
now, whereas more conservative types would avoid any change in
behavior.

 What about Ralph's `cpio -H ustar' suggestion?

I'd put cpio last on the list, with the lowest priority.  Traditional
UNIX cpio had serious bugs whenever two files had inode numbers that
were the same modulo 2**16.  This problem persisted for many years (up
until I stopped using cpio -- perhaps it still has the bug for all I
know).  GNU cpio cannot yet handle large files or archives (i.e.,
larger than 2 GiB on typical x86 hosts).




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Paul Eggert
Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

 1. The --format=posix options is a GNU Tar option, which means `make dist'
can no longer be run with other make implementations.

Presumably you meant tar implementations and not make
implementations?  But this is already the case, as the existing code
uses the -o option, which is specific to GNU tar (the -o option has a
different meaning with traditional tar, and traditional tar rejects -o
when creating archives).

 2. The --format option is a new option of tar 1.13.93, so the
produced Makefile will not even run with widespread tar versions.

True: any such patch along these lines would have to configure this,
depending on whether tar supported the option in question.  Come to
think of it, while you're at it you could also see whether -o is
supported, so that the makefile ports to non-GNU tar implementations.

 3. The POSIX format is young

Quite true.  Few implementations support the POSIX 1003.1-2001 format.

 Forcing the ustar format might be a possibility (is it?), if
 that can be done portably (that probably involves a configure
 check).

Yes, I think this is the best approach: that is, use --format=ustar if
available, then -o if that works, otherwise don't use anything.

 Another idea is to arrange the dist rules so users have
 a way to override the tar options.

That also might be helpful, though I think not as useful; most users
won't want to futz with this.




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Sergey Poznyakoff
Hello,

Actually I was planning to address this issue a bit later, since I
do not have a solution for automake yet. Anyway: 

Alexandre Duret-Lutz [EMAIL PROTECTED] wrote:

  Roger + $(AMTAR) chf - $(distdir) --format=posix | GZIP=$(GZIP_ENV) gzip -c 
 $(distdir).tar.gz
 [...]
 
 Sorry for the late comment.  I can see at least three reasons
 against this patch:

I agree with all three. The format used for distribution tarballs should
be a reasonably old one, so that any existing version of tar be able to
cope with it. 

On the other hand, let me notice that the current practice of using
(so called) V7 format for make dist falls within the first reason you
mentioned:

 The --format=posix options is a GNU Tar option, which means `make dist'
 can no longer be run with other make implementations.  Automake makefiles 
 should be portable and this breaks it.

The -o option used by automake is as well a GNU Tar option, and it
has another meaning for other tar implementations. According to
POSIX specs, -o stands for 'Assign to  extracted  files  the  user
and group  identifiers  of  the  user running the program', i.e.
it corresponds to GNU tar --no-same-owner option. Let's take as
examples two other tar implementations:

1) Sun tar. Its manpage says:

 o Ownership. Assign to  extracted  files  the  user  and
   group  identifiers  of  the  user running the program,
   rather than those on  tarfile.
   [...]

2) Star:

 -nochown, -o
   Do  not restore owner and group of files.  This may
   be used if super  user  privileges are  needed  to
   overwrite existing files but the
   local ownership of the existing files should not change.

Thus, using -o in Makefiles is at best irrelevant to creation of
archives.

Returning to the question which format should be chosen for
distribution tarballs: I would recommend the 'ustar' format.
It also has a limitation on the maximum length of stored pathnames,
but the limitation is 256 characters, which is reasonably large.
Besides, ustar archives can be read by any existing implementation
of tar.

Regards,
Sergey






Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Alexandre Duret-Lutz
 Paul == Paul Eggert [EMAIL PROTECTED] writes:

 Paul Alexandre Duret-Lutz [EMAIL PROTECTED] writes:
  1. The --format=posix options is a GNU Tar option, which means `make dist'
  can no longer be run with other make implementations.

 Paul Presumably you meant tar implementations and not make
 Paul implementations?  

Yep.

 Paul But this is already the case, as the existing code uses
 Paul the -o option, which is specific to GNU tar (the -o
 Paul option has a different meaning with traditional tar, and
 Paul traditional tar rejects -o when creating archives).

Thanks, I didn't know that.  My comment was only based on the
fact that the automake test suite uses it and nobody ever
complained about it.  But of course the test suite doesn't
verify the output format.

Looking into why I never noticed myself, I can see that

  Solaris 8's tar ignores `o' when creating archives (I guess it
  create a ustar file, though I do not know how to verify this).

  FreeBSD/NetBSD/OpenBSD/Darwin's tar understand `o' as a V7 request 
  like GNU Tar.

  OSF1's tar understand's `o' as GNU tar does when creating archives,
  and as Solaris does when extracting archives.

I cannot check more platform.  But at least it's interesting
that it works on more that just GNU tar.

  Forcing the ustar format might be a possibility (is it?), if
  that can be done portably (that probably involves a configure
  check).

 Paul Yes, I think this is the best approach: that is, use --format=ustar if
 Paul available, then -o if that works, otherwise don't use anything.

  Another idea is to arrange the dist rules so users have
  a way to override the tar options.

 Paul That also might be helpful, though I think not as useful; most users
 Paul won't want to futz with this.

Ok.  I'll try to do both of these this week-end and Cc the patch
to bug-tar for comments.
-- 
Alexandre Duret-Lutz





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Alexandre Duret-Lutz
 Sergey == Sergey Poznyakoff [EMAIL PROTECTED] writes:

[...]

 Sergey Returning to the question which format should be chosen for
 Sergey distribution tarballs: I would recommend the 'ustar' format.
 Sergey It also has a limitation on the maximum length of stored pathnames,
 Sergey but the limitation is 256 characters, which is reasonably large.
 Sergey Besides, ustar archives can be read by any existing implementation
 Sergey of tar.

By the way, the pax manpage from the Heirloom Toolchest
(http://heirloom.berlios.de/man/pax.1.html) mentions that due
to implementation errors, file names longer than 99 characters
can not considered to be generally portable when taking about
the ustar format.

I think it's the only place where I've read this.  Has anybody
seen such implementation?  I'm tempted to think we shouldn't
care.
-- 
Alexandre Duret-Lutz





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Alexandre Duret-Lutz
 adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:
 Paul == Paul Eggert [EMAIL PROTECTED] writes:

[...]
  Forcing the ustar format might be a possibility (is it?), if
  that can be done portably (that probably involves a configure
  check).

 Paul Yes, I think this is the best approach: that is, use --format=ustar if
 Paul available, then -o if that works, otherwise don't use anything.
[...]
 adl Ok.  I'll try to do both of these this week-end and Cc the patch
 adl to bug-tar for comments.

On second though, shouldn't we try to use pax first?  tar is no
longer a POSIX requirement, right?

Also do we really need to try `-o'?  This sounds odd because we
are trying to get rid of the 100-char limit, and using `-o'
would means `make dist' will work for deep trees on some
platform and not on some others.

Besides GNU tar, is there some tar implementation that does not
default to the ustar format?

How about this scheme:

  Use `pax -x ustar' if it works.

  Otherwise try `tar --format=ustar'.

  Otherwise use `POSIXLY_CORRECT=1 tar'.  (I'm assuming that
  POSIXLY_CORRECT=1 will coerce older tar versions into 
  producing ustar format.  Am I wrong, or is there another 
  way?)

-- 
Alexandre Duret-Lutz





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Sergey Poznyakoff
Alexandre Duret-Lutz [EMAIL PROTECTED] wrote:

 On second though, shouldn't we try to use pax first?  tar is no
 longer a POSIX requirement, right?

Right, but I'd say it's too early to relay on pax.
 
 Also do we really need to try `-o'?

No, we do not.

 Besides GNU tar, is there some tar implementation that does not
 default to the ustar format?

GNU tar 1.14 will be the last release that uses non-portable GNU format
by default. Subsequent versions will default to 'ustar' format.
 
 How about this scheme:
 
   Use `pax -x ustar' if it works.
 
   Otherwise try `tar --format=ustar'.

This looks OK (though see my comment above).

   Otherwise use `POSIXLY_CORRECT=1 tar'.  (I'm assuming that
   POSIXLY_CORRECT=1 will coerce older tar versions into 
   producing ustar format.  Am I wrong, or is there another 
   way?)

No, it will not. Previous versions of GNU tar were not able to produce
correct 'ustar' archives. Whatever options you give to previous versions
of GNU tar, the produced archives will deviate from the standard.
Using -o does not help much, either: if the file list to be archived
contains long filenames or symbolic links, tar will create GNU extension
headers for them, thus hurting the portability of the resulting archive.

Regards,
Sergey




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Sergey Poznyakoff
Alexandre Duret-Lutz [EMAIL PROTECTED] wrote:

 I think it's the only place where I've read this.  Has anybody
 seen such implementation?  I'm tempted to think we shouldn't
 care.

Agreed.

Regards,
Sergey




Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Bob Friesenhahn
On Fri, 16 Apr 2004, Alexandre Duret-Lutz wrote:

 By the way, the pax manpage from the Heirloom Toolchest
 (http://heirloom.berlios.de/man/pax.1.html) mentions that due
 to implementation errors, file names longer than 99 characters
 can not considered to be generally portable when taking about
 the ustar format.

 I think it's the only place where I've read this.  Has anybody
 seen such implementation?  I'm tempted to think we shouldn't
 care.

More than likely such implementations exist.  Usually when tar
programs break while extracting long file names, they continue on,
leaving bad file names in their wake.  Probably the user won't even
notice until something doesn't work.

Is there a reason to allow file names longer than 99 characters in a
package?  Clearly this is non-portable.  Why not enforce a maximum
file name length of 99 characters in Automake?  One way to enforce
this is to use sed to truncate file names longer than 99 characters
before passing them to tar so that tar complains/fails during 'make
dist'.

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Alexandre Duret-Lutz
 Sergey == Sergey Poznyakoff [EMAIL PROTECTED] writes:

[...]

  Otherwise use `POSIXLY_CORRECT=1 tar'.  (I'm assuming that
  POSIXLY_CORRECT=1 will coerce older tar versions into 
  producing ustar format.  Am I wrong, or is there another 
  way?)

 Sergey No, it will not. Previous versions of GNU tar were not
 Sergey able to produce correct 'ustar' archives. Whatever
 Sergey options you give to previous versions of GNU tar, the
 Sergey produced archives will deviate from the standard.

Ouch!  Is that a `will always deviate' or a `can deviate in some
conditions (like -o does)'?

 Sergey Using -o does not help much, either: if the file list
 Sergey to be archived contains long filenames or symbolic
 Sergey links, tar will create GNU extension headers for them,
 Sergey thus hurting the portability of the resulting archive.

We don't have symbolic links.  Only regular files and non-empty
directories; furthermore ownership is not important.

So as I sees it the question is: in this conditions, will
old-GNU-tar-without-o deviate more from ustar than
old-GNU-tar-with-o deviates from V7?

If the answer is yes, then I guess we must keep using `o' with
older GNU tar versions, and people with long filenames will have
to get a recent tar version if they want portable tarballs.
Otherwise it seems we simply do not need `o'.
-- 
Alexandre Duret-Lutz





Re: [Bug-tar] Re: AMTAR brokenness

2004-04-16 Thread Roger Leigh
Bob Friesenhahn [EMAIL PROTECTED] writes:

 Is there a reason to allow file names longer than 99 characters in a
 package?  Clearly this is non-portable.  Why not enforce a maximum
 file name length of 99 characters in Automake?

I experience the breakage when using Doxygen to generate a reference
manual from C++ code.  Some of the filenames it generates are over 80
chars, due to using namespaces, and long classnames etc.  If the path
is called libfoobar-baz-12.32.44/doc/libfoobar-baz/html/., that's an
extra 46 chars in the path.  The 99 char limit has been comfortably
exceeded, and I had no control over this!

[There are options to produce mangled 8.3 names, but I want them to be
human readable.]

 One way to enforce this is to use sed to truncate file names longer
 than 99 characters before passing them to tar so that tar
 complains/fails during 'make dist'.

What if the truncated names already exist?  You will then silently
produce a broken archive.  Better to do

  find . | wc -L

to get an accurate figure, and then bail out.

I'm using the patch I posted.  If and when anyone complains, I'll
instruct them to build and install the latest GNU tar (or build it for
them).  I require the functionality for long pathnames, and I don't
want to have to support broken tools.  For my needs, 99 chars is not
at all sufficient.


Regards,
Roger

-- 
Roger Leigh

Printing on GNU/Linux?  http://gimp-print.sourceforge.net/
GPG Public Key: 0x25BFB848.  Please sign and encrypt your mail.




Re: AMTAR brokenness

2004-04-15 Thread Alexandre Duret-Lutz

 Roger == Roger Leigh [EMAIL PROTECTED] writes:
[...]
 Roger -   $(AMTAR) chof - $(distdir) | GZIP=$(GZIP_ENV) gzip -c 
$(distdir).tar.gz
 Roger +   $(AMTAR) chf - $(distdir) --format=posix | GZIP=$(GZIP_ENV) gzip -c 
$(distdir).tar.gz
[...]

Sorry for the late comment.  I can see at least three reasons
against this patch:

1. The --format=posix options is a GNU Tar option, which means `make dist'
   can no longer be run with other make implementations.  Automake makefiles 
   should be portable and this breaks it.

2. The --format option is a new option of tar 1.13.93, so the
   produced Makefile will not even run with widespread tar versions.

3. The POSIX format is young and AFAICT many platforms out there
   cannot read it.  (Sometimes tar won't read it but you can
   read it with the pax utility, sometimes neither work.
   E.g. such archives cause pax and tar to segfault on the OSF1
   V4.0 box I have access to.)

I'm not denying there is a problem to fix, but the above change
really breaks more things than it cures.  Without it, the only
people annoyed are those with 100+-character-long filenames.
With it _all_ packages produce non-portable tarballs using
non-portable syntax.

Forcing the ustar format might be a possibility (is it?), if
that can be done portably (that probably involves a configure
check).  Another idea is to arrange the dist rules so users have
a way to override the tar options.
-- 
Alexandre Duret-Lutz





AMTAR brokenness

2004-04-01 Thread Roger Leigh
Recent versions of GNU tar correctly use the -o option.  This breaks
any project with path lengths 100.  Previously GNU tar incorrectly
produced a POSIX tar file with -o, but now it produces a crippled
UNIX v7 tarfile.

PR/414 has been filed against automake.  A patch against automake
1.8.3 follows, which uses the correct --format=posix syntax.  Both the
current and patched forms do rely on GNU tar.  A configure check to
check for GNY tar might be better.  Either way, GNU tar should *not*
be given the -o option, since it causes horrible breakage.

Regards,
Roger


--- /tmp/distdir.am 2004-04-01 12:32:29.0 +0100
+++ distdir.am  2004-04-01 12:30:26.0 +0100
@@ -238,19 +238,19 @@
 GZIP_ENV = --best
 .PHONY: dist-gzip
 dist-gzip: distdir
-   $(AMTAR) chof - $(distdir) | GZIP=$(GZIP_ENV) gzip -c $(distdir).tar.gz
+   $(AMTAR) chf - $(distdir) --format=posix | GZIP=$(GZIP_ENV) gzip -c 
$(distdir).tar.gz
$(am__remove_distdir)
 
 ?BZIP2?DIST_ARCHIVES += $(distdir).tar.bz2
 .PHONY: dist-bzip2
 dist-bzip2: distdir
-   $(AMTAR) chof - $(distdir) | bzip2 -9 -c $(distdir).tar.bz2
+   $(AMTAR) chf - $(distdir) --format=posix | bzip2 -9 -c $(distdir).tar.bz2
$(am__remove_distdir)
 
 ?COMPRESS?DIST_ARCHIVES += $(distdir).tar.Z
 .PHONY: dist-tarZ
 dist-tarZ: distdir
-   $(AMTAR) chof - $(distdir) | compress -c $(distdir).tar.Z
+   $(AMTAR) chf - $(distdir) --format=posix | compress -c $(distdir).tar.Z
$(am__remove_distdir)
 
 ?SHAR?DIST_ARCHIVES += $(distdir).shar.gz
@@ -281,9 +281,9 @@
 
 .PHONY: dist dist-all
 dist dist-all: distdir
-?GZIP? $(AMTAR) chof - $(distdir) | GZIP=$(GZIP_ENV) gzip -c $(distdir).tar.gz
-?BZIP2?$(AMTAR) chof - $(distdir) | bzip2 -9 -c $(distdir).tar.bz2
-?COMPRESS? $(AMTAR) chof - $(distdir) | compress -c $(distdir).tar.Z
+?GZIP? $(AMTAR) chf - $(distdir) --format=posix | GZIP=$(GZIP_ENV) gzip -c 
$(distdir).tar.gz
+?BZIP2?$(AMTAR) chf - $(distdir) --format=posix | bzip2 -9 -c 
$(distdir).tar.bz2
+?COMPRESS? $(AMTAR) chf - $(distdir) --format=posix | compress -c 
$(distdir).tar.Z
 ?SHAR? shar $(distdir) | GZIP=$(GZIP_ENV) gzip -c $(distdir).shar.gz
 ?ZIP?  -rm -f $(distdir).zip
 ?ZIP?  zip -rq $(distdir).zip $(distdir)

-- 
Roger Leigh

Printing on GNU/Linux?  http://gimp-print.sourceforge.net/
GPG Public Key: 0x25BFB848.  Please sign and encrypt your mail.