Re: Generated changes and patch systems
On Tue, May 27, 2008 at 06:06:28PM -0700, Russ Allbery wrote: Neil Williams [EMAIL PROTECTED] writes: With these gtk-doc files, it's not so much that the tmpl/*.sgml files are generated but that a tool essential to the build modifies them in a way that cannot be patched because the results of the patches are variable according to that third party tool. If the tool behaves and only behaves in that way (I haven't checked), that tool is broken and we should fix that tool. I've run into a similar situation before with a vaugely borked upstream build system - it was straightforward enough to work around the problem by moving the files out of the way and then replacing them if there was a backup present when clean was run. -- You grabbed my hand and we fell into it, like a daydream - or a fever. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Generated changes and patch systems
On Sun, 2008-05-25 at 08:40 -0700, Russ Allbery wrote: Neil Williams [EMAIL PROTECTED] writes: On Sun, 2008-05-25 at 13:19 +0100, Mark Brown wrote: If you run autotools at build time you should also ensure that the changes which autotools makes are reverted in the clean target. This means that your diff doesn't get cluttered with automatically generated things and ensures that repeated builds of the package produce the same diff.gz. I haven't seen any other packages doing that - is there an example involving aclocal.m4 somewhere? Lots of other packages do this -- one of mine off the top of my head is xml-security-c. Nope. No mention of aclocal.m4 in debian/rules for that package, just /usr/share/misc/config.guess and config.sub. I need to patch configure and configure.ac in this package, that means that aclocal.m4 is constantly being changed and reverted. It shouldn't matter - it really should not. I see no purpose in lintian (or dpkg come to that) testing for changes in aclocal.m4 - IMHO it should simply be exempt from this check. No distro can risk patching aclocal.m4 - by it's nature it is transient and volatile and subject to large changes entirely at the mercy of external factors. Any build system that tries is simply going to break constantly, AFAICT. No matter how I wrap the cp foo foo.safe mv foo.safe foo in debian/rules, the first build can be OK but all subsequent builds end up with aclocal.m4 in the .diff.gz. Besides, replacing aclocal.m4 after it has just been updated by ./configure is not a good idea to me. I really don't want to do all that for the tmpl/* files as well - I don't see the need, copying dozens of files into foo.safe or foo.upstream and then moving them back? Just delete them then. ??? That simply does not work. The problem is that running gtk-doc not only requires tmpl/*.sgml files to exist but it *then modifies them*! Even though I don't install these files into the package, I cannot delete them and I cannot move them out of the way or the documentation is not updated. Again, first build can be OK, second build generates a dozen spurious warnings (because the files are modified after the .diff.gz is created but cannot be reliably reverted before the next build). I don't see that I should provide out-dated documentation (by dropping --enable-gtk-doc) just because it suits lintian - neither can I patch the updates because the updates themselves are generated by a third party tool so I can neither control the changes made nor maintain clean patches. I suspect the issue arises because upstream happen to prepare the release tarballs on Ubuntu or Fedora where the tool version differs. The precise reason is inconsequential - the problem is that Debian should not need to care about these modifications. It's taking 'divergence' one stage too far. With these gtk-doc files, it's not so much that the tmpl/*.sgml files are generated but that a tool essential to the build modifies them in a way that cannot be patched because the results of the patches are variable according to that third party tool. The changes then affect the files that are packaged. Some of these are formatting changes - whitespace changes, extra lines, comment changes. Other changes cause sections or entire pages to migrate within the final files. The sense of the docs doesn't appear to change, just the internal structure - as determined by the differing versions of the tools used. The CDBS build doesn't do anything different - it's just that lintian doesn't produces any warnings for this check in CDBS packages, ever. The point is to not put mechanically generated changes in the diff, since it makes it much harder to review what the maintainer has actually done. I don't see how it can be prevented. Take a look at the current .diff.gz for libgpewidget 0.115-5 in the archive. I think most people take the approach of just deleting such files in the clean target, which will exclude their changes from the diff. Sorry, it is not as simple as that - the package simply won't build if tmpl/*.sgml are removed, no rule to make tmpl/*.sgml. The tmpl/*.sgml files cannot be cleaned and removing aclocal.m4 is simply pointless (and creates an even larger .diff.gz). I still don't see what problem this test is trying to solve - AFAICT the problem didn't exist in the first place and all this extra work appears pointless. I still think patch-system-but-direct-changes-in-diff should be downgraded to info only, if kept at all. I fail to see what I'm doing wrong - or even why it matters to lintian. We risk swapping a minor problem that only occurs when the maintainer upgrades from one version to another (and which could be fixed by dpkg-source ignoring changes to generated files in the .diff.gz or not applying generated changes that do exist in the .diff.gz) for a major FTBFS error every time some random third party package is updated by erroneously patching files that contain generated
Re: Generated changes and patch systems
Neil Williams [EMAIL PROTECTED] writes: On Sun, 2008-05-25 at 08:40 -0700, Russ Allbery wrote: Lots of other packages do this -- one of mine off the top of my head is xml-security-c. Nope. No mention of aclocal.m4 in debian/rules for that package, just /usr/share/misc/config.guess and config.sub. Uh, it's the same problem. Surely the generalization is obvious? I guess look at shibboleth-sp if it's not I need to patch configure and configure.ac in this package, that means that aclocal.m4 is constantly being changed and reverted. It shouldn't matter - it really should not. Right, so delete it in the clean rule. I see no purpose in lintian (or dpkg come to that) testing for changes in aclocal.m4 - IMHO it should simply be exempt from this check. Absolutely not -- you're introducing unexpected changes to the packaging diff file, and that's exactly the point of this check. Delete the modified and regenerated files in the clean rule and then you won't have this problem. No distro can risk patching aclocal.m4 - by it's nature it is transient and volatile and subject to large changes entirely at the mercy of external factors. Any build system that tries is simply going to break constantly, AFAICT. Exactly, which is why lintian is making sure that you don't introduce patches to it in your *.diff.gz inadvertantly, since it's not a sane thing to do when you're using a patch system. The preferred method to handle it as far as I'm concerned is doing what you're doing -- running autotools at build time. In that case, you need to remove the regenerated files in your clean rule. The other way to do it, particularly if you need a very specific version of autotools, is to run them yourself and save them as a patch, but I think that's a bad idea except in very specific situations. I really don't want to do all that for the tmpl/* files as well - I don't see the need, copying dozens of files into foo.safe or foo.upstream and then moving them back? Just delete them then. ??? That simply does not work. The problem is that running gtk-doc not only requires tmpl/*.sgml files to exist but it *then modifies them*! This is extremely unusual. Are you *sure* that it does an inplace edit of the files during the build process? This is almost never what build systems do; instead, they generate files from other files using templating systems. They may ship the results of that operation and then redo it during the build, in which case you need to delete the regenerated files in your clean rule. If they really expect the files to exist and to edit them in-place during the build, upstream is doing something insane, and it's really something that should be fixed upstream; a lintian warning is drawing your attention to a very broken behavior. Even though I don't install these files into the package, I cannot delete them and I cannot move them out of the way or the documentation is not updated. Again, first build can be OK, second build generates a dozen spurious warnings (because the files are modified after the .diff.gz is created but cannot be reliably reverted before the next build). Right, lintian is diagnosing build system brokenness. I'm fairly happy with lintian's behavior here; what it's drawing attention to is exactly the kind of thing that breaks repeated package builds or causes NMUs to introduce spurious package diffs, and is something that should really ideally be fixed. With these gtk-doc files, it's not so much that the tmpl/*.sgml files are generated but that a tool essential to the build modifies them in a way that cannot be patched because the results of the patches are variable according to that third party tool. If the tool behaves and only behaves in that way (I haven't checked), that tool is broken and we should fix that tool. The CDBS build doesn't do anything different - it's just that lintian doesn't produces any warnings for this check in CDBS packages, ever. Well, yes, it does, if CDBS uses a recognized patch system, but the list of patch systems that lintian recognizes is fairly limited at this point. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
