Re: [tahoe-dev] 1.9.0 tarball broken
On Wed, Nov 9, 2011 at 11:43 AM, David-Sarah Hopwood david-sa...@jacaranda.org wrote: Why doesn't the setup code force the umask? It seems like this should not depend on the environment. Yes, this is a bug in setuptools. No, if it is a bug, it is a bug in distutils. We need to stop using setuptools. Switching to another tool would be trading out these bugs for different bugs. In any case, casual assertions in mailing list threads probably isn't a very productive way forward on that issue. Regards, Zooko ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
On Sun, Nov 13, 2011 at 5:16 PM, David-Sarah Hopwood david-sa...@jacaranda.org wrote: No, if it is a bug, it is a bug in distutils. I don't make a strong distinction between setuptools and distutils, because setuptools heavily patches distutils without attempting to maintain any modular separation between them. It's just easier to say setuptools than setuptools plus distutils as patched by setuptools. Well, the reason to make the distinction is that if we stopped using setuptools, as per Brian's unsuck branch, or if we switched to a different tool, such as distribute, pip, distutils2, paver, or whatever (note: some of those tools may be different types of thing), then that'd probably not change this particular behavior. Regards, Zooko ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
On 13/11/11 17:01, Zooko O'Whielacronx wrote: On Sun, Nov 13, 2011 at 5:16 PM, David-Sarah Hopwood david-sa...@jacaranda.org wrote: No, if it is a bug, it is a bug in distutils. I don't make a strong distinction between setuptools and distutils, because setuptools heavily patches distutils without attempting to maintain any modular separation between them. It's just easier to say setuptools than setuptools plus distutils as patched by setuptools. Well, the reason to make the distinction is that if we stopped using setuptools, as per Brian's unsuck branch, or if we switched to a different tool, such as distribute, pip, distutils2, paver, or whatever (note: some of those tools may be different types of thing), then that'd probably not change this particular behavior. It probably would if the replacement did not use the implementation of 'setup.py sdist' that is in distutils. I would be in favour of using a replacement that is not based on distutils at all. (Note that this doesn't preclude providing reasonably close compatibility with the 'setup.py' command interface.) -- David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com signature.asc Description: OpenPGP digital signature ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
On 11/8/11 7:53 PM, Brian Warner wrote: Oops. I think I used our recipe's python setup.py sdist step to create the tarballs, but if that does the wrong thing with file nodes, then maybe we need to fix the recipe. I think I found the problem: the automation that builds the tarballs recently moved to a new buildslave, and I forgot to set the new slave up with the --umask=022 flag (so it defaulted to 002). I just fixed that, so future tarballs should be better. I'll repack the existing 1.9.0 tarballs tomorrow and put them back at the same URLs, and republish the hashes. thanks, -Brian ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
I'll repack the existing 1.9.0 tarballs tomorrow and put them back at the same URLs, and republish the hashes. A new version number please - packaging systems expect tarballs once published to be invariant (since they store their own hashes), and the same file name with different contents looks like an attack. From my viewpoint, it's better to just let this be than to change the contents of an already published file. Why doesn't the setup code force the umask? It seems like this should not depend on the environment. pgpaM25ynDbgG.pgp Description: PGP signature ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
On 11/9/11 5:02 AM, Greg Troxel wrote: I'll repack the existing 1.9.0 tarballs tomorrow and put them back at the same URLs, and republish the hashes. A new version number please - packaging systems expect tarballs once published to be invariant (since they store their own hashes), and the same file name with different contents looks like an attack. Sure thing. If you're ok with your workaround, I'll probably wait until the weekend (and after the Summit) to spin 1.9.1. It won't be as trivial as I'd like, since it'll involve figuring out some branching issues (the changes that landed on trunk since 1.9.0 was tagged are too new for release, so we'll need to make a 1.9.1 that's only a few (documentation) lines different than 1.9.0, and that means a branch, and darcs does not make branching easy. Why doesn't the setup code force the umask? It seems like this should not depend on the environment. It'd be great if it behaved that way. Tarballs are generated by a script, in the make tarballs target of the source tree's top-level Makefile. The execution that created the 1.9.0 tarballs was logged here: https://tahoe-lafs.org/buildbot-tahoe-lafs/builders/tarballs/builds/871/steps/tarballs/logs/stdio It uses the standard Distutils setup.py sdist command, which apparently doesn't set the umask and just inherits it from the environment. What I think I'll do is add a umask 002 setting in the Makefile target. (it's non-ideal, because changing the umask is harder to unwind than, say, setting an environment variable, so running 'make tarballs' will have a subtle lingering side-effect on the shell in which it is run). cheers, -Brian ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
Brian Warner war...@lothar.com writes: On 11/9/11 5:02 AM, Greg Troxel wrote: A new version number please - packaging systems expect tarballs once published to be invariant (since they store their own hashes), and the same file name with different contents looks like an attack. Sure thing. If you're ok with your workaround, I'll probably wait until the weekend (and after the Summit) to spin 1.9.1. It won't be as trivial My workaround is just 3 lines in a makefile to chmod it, and the package builds fine, so I'm actually 100% fine with it indefinitely. I don't know if it's bothering other packagers, but random fixups are the norm at some rate. as I'd like, since it'll involve figuring out some branching issues (the changes that landed on trunk since 1.9.0 was tagged are too new for release, so we'll need to make a 1.9.1 that's only a few (documentation) lines different than 1.9.0, and that means a branch, and darcs does not make branching easy. I guess that's another reason to move to git, besides #1226 :-) Why doesn't the setup code force the umask? It seems like this should not depend on the environment. It'd be great if it behaved that way. Tarballs are generated by a script, in the make tarballs target of the source tree's top-level Makefile. The execution that created the 1.9.0 tarballs was logged here: https://tahoe-lafs.org/buildbot-tahoe-lafs/builders/tarballs/builds/871/steps/tarballs/logs/stdio It uses the standard Distutils setup.py sdist command, which apparently doesn't set the umask and just inherits it from the environment. What I think I'll do is add a umask 002 setting in the Makefile target. (it's non-ideal, because changing the umask is harder to unwind That makes sense to me, but I don't really understand the details. than, say, setting an environment variable, so running 'make tarballs' will have a subtle lingering side-effect on the shell in which it is run). But isn't that shell only for that make target? And it seems there is no need to unwind within the tarball creation process. pgpxPpg9u5b3H.pgp Description: PGP signature ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
On 09/11/11 09:49, Brian Warner wrote: On 11/9/11 5:02 AM, Greg Troxel wrote: I'll repack the existing 1.9.0 tarballs tomorrow and put them back at the same URLs, and republish the hashes. A new version number please - packaging systems expect tarballs once published to be invariant (since they store their own hashes), and the same file name with different contents looks like an attack. Sure thing. If you're ok with your workaround, I'll probably wait until the weekend (and after the Summit) to spin 1.9.1. -1 on releasing a 1.9.1 for this issue. A mistake in generating the tarball doesn't justify a new release. We've had such mistakes before and have not made a new release. If the code had already been packaged in OS distributions, that would be a different case, but it hasn't (and such distributions have a convention for changing the package version without requiring an upstream release). Why doesn't the setup code force the umask? It seems like this should not depend on the environment. Yes, this is a bug in setuptools. We need to stop using setuptools. What I think I'll do is add a umask 002 setting in the Makefile target. (it's non-ideal, because changing the umask is harder to unwind than, say, setting an environment variable, so running 'make tarballs' will have a subtle lingering side-effect on the shell in which it is run). Can you use a subshell to avoid that (something like /bin/sh -c 'umask 022 ...')? -- David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com signature.asc Description: OpenPGP digital signature ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
* Greg Troxel g...@ir.bbn.com wrote: My workaround is just 3 lines in a makefile to chmod it, and the package builds fine, so I'm actually 100% fine with it indefinitely. I don't know if it's bothering other packagers, but random fixups are the norm at some rate. FWIW, Botan tarballs have the same issue; it's dealt with in the buildscript and that's it. -- left blank, right bald pgpCgKR9PUVtm.pgp Description: PGP signature ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] 1.9.0 tarball broken
On 11/8/11 4:22 PM, Greg Troxel wrote: The 1.9.0 tarball is different from the 1.9.0b1 tarball; files/directories are 0600/0700, while in 1.9.0b1 they are the more sane 0644/0755. The standard install seems to copy them as is, resulting in root-owned files in the package that are unreadable by mere mortals. We've had this issue before, and I'm not sure if there is a bug open about the tarball creation process - certainly things like permissions in the tarball should be deterministic. Oops. I think I used our recipe's python setup.py sdist step to create the tarballs, but if that does the wrong thing with file nodes, then maybe we need to fix the recipe. I'll make a note to write a make_release_tarballs.py script, put it in the tree, and include a self-test that opens up the newly-generated tarball and checks the permissions. For 1.9, I'll make a new (fixed) tarball and put it up in the same place as the old one. I'll send out mail with the new hashes when I do that (maybe tomorrow). cheers, -Brian ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
