Re: setup.exe: Invalid or unsupported tar format
--- Brian Dessent ha scritto: The setup.exe source (install.cc, Installer::installOne) looks like it expects an error peeking from try_decompress in this situation, but running in the debugger shows try_decompress-peek() successfully reading '0'. Note that if you bunzip2 the empty package, you get a 10K tarball which is all zeroes, so success isn't completely unexpected... $ tar --show-defaults --format=gnu -f- -b20 --quoting-style=escape --rmt-command=/usr/sbin/rmt.exe --rsh-command=/usr/bin/rsh $ ls -lrt 10240 May 6 11:30 prova1.tar my guess 20*512= 10240 bytes Regards Marco Tante idee per la salvaguardia del nostro Pianeta su Yahoo! For Good http://it.promotions.yahoo.com/forgood/environment.html
Re: setup.exe: Invalid or unsupported tar format
On Mon, May 05, 2008 at 08:13:58PM -0600, Eric Blake wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Brian Dessent on 5/5/2008 3:11 PM: | I have an empty meta-package to pull in other packages copied from one | of the _obsolete packages. I understand these are created via 'tar -T | /dev/null -cjf foo.tar.bz2' (see | http://cygwin.com/ml/cygwin-apps/2008-04/msg00105.html), and diff | reports the results identical; the file size is the expected 46 bytes. It sounds like we need to replace all the 46-byte files with 14 byte ones? ~ But the empty file is technically not a valid tar file, since it lacks the required trailing blocks. Yes, I'm confused by this. I did replace all of the zero length tar files with the 46 byte versions. That's what I've always done when creating an empty package. cgf
Re: setup.exe: Invalid or unsupported tar format
Christopher Faylor wrote: Yes, I'm confused by this. I did replace all of the zero length tar files with the 46 byte versions. That's what I've always done when creating an empty package. Here is a summary of the situation: The current stable/release version of setup will never warn if it can't understand a package, it will just silently skip it. Mostly, this is what you want, as it means it will cope just fine with both the 14b and 46b packages. However a while ago someone reported that a package wasn't installing and after investigation it was because the maintainer had used something like cmake with its own libtar implementation that used an incompatible magic. Setup was silently refusing to install the package even though everything indicated that it was to be installed -- very confusing. I added support for this new magic value and rewrote the tar checking code to explicitly warn if it cannot grok a package. This change existed only in testing/snapshot versions so few people would have experienced it. Unbeknownst to me at the time, this meant it would report on both the 14b packages (since they contain nothing after decompression) and the 46b packages (since they don't actually contain a tar header but 10k of \0 which is unrecognisable to setup as a valid tar header.) I admit that I don't regularly install packages from _obsolete so I never saw it in any of my testing. Some time later someone reported using a setup snapshot and seeing the warning for an empty 14b package. At the time I attributed this to the 14b package not being a 46b package, and added code to setup to skip the 14b package without a warning, assuming incorrectly that the 46b case also worked as no one had reported that -- until now. What I intend to do is add another special-case for checking that if the first 512 bytes are all zeros to assume it's an empty tar file and suppress the warning. This will mean that both 46b and 14b packages are correctly handled, as well as still preserving the warning if a real package is unrecognisable. But until I can add that, if you are using a snapshot version only the 14b packages will be correctly handled; the 46b ones will warn. This is mostly just a nuisance as installation continues after the warning is dismissed and it's not like you're missing anything from the empty package, but it's a nuisance nonetheless and setup ought to be able to handle both types of empty package. I don't think there's any need to convert anything on the mirrors again because the next actual release version of setup should cope with both, and people using snapshots should be able to deal. Brian
setup.exe: Invalid or unsupported tar format
I've built setup.exe from source (updated today), and the ChangeLog says Brian Dessent fixed the Invalid or unsupported tar format problem with empty tar.bz2 files, but I am still seeing the error popup. I have an empty meta-package to pull in other packages copied from one of the _obsolete packages. I understand these are created via 'tar -T /dev/null -cjf foo.tar.bz2' (see http://cygwin.com/ml/cygwin-apps/2008-04/msg00105.html), and diff reports the results identical; the file size is the expected 46 bytes. The setup.exe source (install.cc, Installer::installOne) looks like it expects an error peeking from try_decompress in this situation, but running in the debugger shows try_decompress-peek() successfully reading '0'. Note that if you bunzip2 the empty package, you get a 10K tarball which is all zeroes, so success isn't completely unexpected... $ cygcheck -cd | grep 'bzip\|tar\|cygwin' bzip2 1.0.3-2 cygwin 1.5.25-7 mingw-bzip2 1.0.3-2 tar 1.19-1 Am I missing anything? I'm not sure the best way to fix this; the following patch worked, but seems unlikely to actually be correct in all cases. Is an empty tarball the only case where the peek value will be zero? Index: install.cc === RCS file: /cvs/cygwin-apps/setup/install.cc,v retrieving revision 2.83 diff -u -p -w -r2.83 install.cc --- install.cc 9 Apr 2008 02:25:27 - 2.83 +++ install.cc 5 May 2008 20:15:13 - @@ -259,7 +259,7 @@ Installer::installOne (packagemeta pkgm /* Decompression succeeded but we couldn't grok it as a valid tar archive. */ char c; - if (try_decompress-peek (c, 1) 0) + if (try_decompress-peek (c, 1) 0 || !c) // assume 0 indicates empty tarball /* In some cases, maintainers have uploaded bzipped 0-byte files as dummy packages instead of empty tar files. This is common enough that we should not treat this as an -- Bryan Thrall FlightSafety International [EMAIL PROTECTED]
Re: setup.exe: Invalid or unsupported tar format
Thrall, Bryan wrote: I've built setup.exe from source (updated today), and the ChangeLog says Brian Dessent fixed the Invalid or unsupported tar format problem with empty tar.bz2 files, but I am still seeing the error popup. I have an empty meta-package to pull in other packages copied from one of the _obsolete packages. I understand these are created via 'tar -T /dev/null -cjf foo.tar.bz2' (see http://cygwin.com/ml/cygwin-apps/2008-04/msg00105.html), and diff reports the results identical; the file size is the expected 46 bytes. That fix was for empty bzipped files, i.e. the result after decompression is a 0 byte file, so it's not exactly relevant to this situation. The setup.exe source (install.cc, Installer::installOne) looks like it expects an error peeking from try_decompress in this situation, but running in the debugger shows try_decompress-peek() successfully reading '0'. Note that if you bunzip2 the empty package, you get a 10K tarball which is all zeroes, so success isn't completely unexpected... This code should not have ever been reached if the file contained a valid tar header, since archive::extract should not have returned NULL in that case. For a valid-yet-empty tar file (i.e. what the 46 byte thing is supposed to be) archive::extract should succeed but the first call to next_file_name will return an empty string. But it seems that tar -T /dev/null doesn't in fact produce anything resembling a valid tar file, it simply spits out 10k of zeros. And I checked all the old existing 46 byte .tar.bz2 files, and they are in fact just 10k of bzipped zeros. Sigh. Why does tar do this? This causes ::extract to return NULL (since it looks nothing like a tar file) but it's also not an empty file, so peek doesn't fail either. And again, to recap: the reason for generating an error instead of silently moving on in this situation was to diagnose the case where the maintainer used a strange tar implementation to create a package -- without the error the package will be silently skipped which is very confusing. But it seems like this nonsense 10k of zeros will have to be special cased in order to be able to detect the desired error case from the bogus error case. In the mean time, I suggest you use a real empty file as workaround (i.e. bzip2 /dev/null file.tar.bz2) as that should correctly be detected as an empty package if you're using the current setup. Brian
RE: setup.exe: Invalid or unsupported tar format
Brian Dessent wrote on Monday, May 05, 2008 4:12 PM: And again, to recap: the reason for generating an error instead of silently moving on in this situation was to diagnose the case where the maintainer used a strange tar implementation to create a package -- without the error the package will be silently skipped which is very confusing. But it seems like this nonsense 10k of zeros will have to be special cased in order to be able to detect the desired error case from the bogus error case. In the mean time, I suggest you use a real empty file as workaround (i.e. bzip2 /dev/null file.tar.bz2) as that should correctly be detected as an empty package if you're using the current setup. Thanks for the explanation. I used 'bzip2 /dev/null file.tar.bz2', generating a 14 byte tar.bz2 file, and setup.exe handles it just fine. -- Bryan Thrall FlightSafety International [EMAIL PROTECTED]
Re: setup.exe: Invalid or unsupported tar format
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Brian Dessent on 5/5/2008 3:11 PM: | I have an empty meta-package to pull in other packages copied from one | of the _obsolete packages. I understand these are created via 'tar -T | /dev/null -cjf foo.tar.bz2' (see | http://cygwin.com/ml/cygwin-apps/2008-04/msg00105.html), and diff | reports the results identical; the file size is the expected 46 bytes. It sounds like we need to replace all the 46-byte files with 14 byte ones? ~ But the empty file is technically not a valid tar file, since it lacks the required trailing blocks. | | But it seems that tar -T /dev/null doesn't in fact produce anything | resembling a valid tar file, it simply spits out 10k of zeros. Perhaps this is worth a question on the upstream tar list? According to http://www.opengroup.org/onlinepubs/009695399/utilities/pax.html, it looks like a valid tar file can exist without files, but needs 2 blocks of all 0 bytes to identify the end of the tar file. And since the default tar configures with tar options of -b20, I'm actually surprised it's not 20k of NUL bytes (2 blocks * 20 units of 512 per block). You can reduce the size of a valid tarball with 'tar -b1 -c -T /dev/null', but that's still a 1k file of all NULs. | | In the mean time, I suggest you use a real empty file as workaround | (i.e. bzip2 /dev/null file.tar.bz2) as that should correctly be | detected as an empty package if you're using the current setup. By the way, the special 1.7 setup has this error enabled by default, making it report failure on any attempt to install packages like 'gcc' which are empty tarball placeholders with dependencies elsewhere when trying out the 1.7 tree. - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgfvuYACgkQ84KuGfSFAYBH+QCfZomHQERLb7wBUUAP7+S26lUL fIEAnR6LS68JtzQOr8hKiDZPgMgUj2c1 =sLs1 -END PGP SIGNATURE-
