nodist_noinst_SCRIPTS and `make distcheck`
Hello, I'm using automake 1.11.6 (from debian 7 stable). I have a script in my project, which creates the ChangeLog from the git log (it's the script from gnulib). Since the script is meant only for `make dist`, it's neither distributed nor installed. I didn't put it in any variable, not even nodist_noinst_SCRIPTS. Everything seemed to work, including `make dist`. Now I tried `make distcheck` for the first time. It fails because it cannot find the script. When it copies the srcdir content into a new temporary place, it simply forgets to take the script, maybe because there are no targets for it at all. So the result is an error, file not found. I tried making the ChangeLog target depend on that script and/or define nodist_noinst_SCRIPTS, but then the result is another error: all-am requires util/git2cl.pl, but there are no rules for building it. Of course I can add a fake rule for the script, which does nothing, but is that a good solution or a dirty workaround? Q: Why don't I just distribute the script? A: It's meant only for maintainers and requires the full git history, so it's useless in a tarball. What should I do? I do want my `make distcheck` to succeed, and this issue seems to be the only thing preventing that. Thanks! fr33domlover PGP key ID: 937A67EF signature.asc Description: PGP signature
Re: nodist_noinst_SCRIPTS and `make distcheck`
On 2014-10-20 17:51 +0300, fr33domlover wrote: I have a script in my project, which creates the ChangeLog from the git log (it's the script from gnulib). Since the script is meant only for `make dist`, it's neither distributed nor installed. I didn't put it in any variable, not even nodist_noinst_SCRIPTS. Everything seemed to work, including `make dist`. Now I tried `make distcheck` for the first time. It fails because it cannot find the script. When it copies the srcdir content into a new temporary place, it simply forgets to take the script, maybe because there are no targets for it at all. [snip details]. There are two related things that distcheck is testing here, and either one of them may be tripping you up. First, distcheck is checking that users can run make dist from your tarball. Second, distcheck is checking that all this works properly in VPATH builds (i.e., with srcdir != builddir). For the first point, in principle it is OK to have ChangeLog generated automatically from your VCS. But you need to be careful that the distributed ChangeLog's prerequisites are present and that it is up-to-date, so that make dist from the tarball does not attempt to re-generate it (since obviously this process will not function). One way to solve this is to sidestep it entirely with a dist-hook, which can test if you are building from VCS, then generate the ChangeLog as appropriate. Something like this (untested): dist-hook: dist-create-changelog dist-create-changelog: if test -d $(srcdir)/.git; then \ : generate $(distdir)/ChangeLog here; \ fi .PHONY: dist-create-changelog Here we rely on the fact that Automake will automagically include ChangeLog in the tarball if it is present in srcdir. You may want to also add a distcheck-hook to ensure that this actually happens. For the second point, if you are not routinely testing VPATH builds from your git tree at all, just be aware that distcheck may uncover bugs related to this feature. Hope that helps, -- Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)
Re: nodist_noinst_SCRIPTS and `make distcheck`
Hello Nick, On 2014-10-20 Nick Bowler nbow...@elliptictech.com wrote: There are two related things that distcheck is testing here, and either one of them may be tripping you up. First, distcheck is checking that users can run make dist from your tarball. Indeed `make dist` succeeds. Second, distcheck is checking that all this works properly in VPATH builds (i.e., with srcdir != builddir). I didn't try, but I assume it will work because the only problem is that script, which IS present in the right place. The Makefile.am makes sure it will be taken from $srcdir. The problem happens because `make distcheck` copies files from the source repo into a new temporary srcdir. For the first point, in principle it is OK to have ChangeLog generated automatically from your VCS. But you need to be careful that the distributed ChangeLog's prerequisites are present and that it is up-to-date, so that make dist from the tarball does not attempt to re-generate it (since obviously this process will not function). One way to solve this is to sidestep it entirely with a dist-hook, which can test if you are building from VCS, then generate the ChangeLog as appropriate. Something like this (untested): dist-hook: dist-create-changelog dist-create-changelog: if test -d $(srcdir)/.git; then \ : generate $(distdir)/ChangeLog here; \ fi .PHONY: dist-create-changelog Here we rely on the fact that Automake will automagically include ChangeLog in the tarball if it is present in srcdir. You may want to also add a distcheck-hook to ensure that this actually happens. This is exactly what I do :-) And like I said, it does work. For the second point, if you are not routinely testing VPATH builds from your git tree at all, just be aware that distcheck may uncover bugs related to this feature. It's going to be the first release, so I didn't try distcheck until now. The line which generates the ChangeLog in the snippet above requires that script to be present in $srcdir - but distcheck doesn't copy it to its temporary srcdir, so it's not present. The solution to the problem of .git not being present during distcheck may be: In distcheck-hook, take it from the right place (i.e. $(srcdir)/../.git) and then try to make the ChangeLog again. The thing about the script is, that while I can do the same (pick the script from the original $srcdir), it would be somewhat wrong design-wise - if a script is used for `make dist`, then `make distcheck` should copy it into the temporary srcdir just like the source code. While not installed nor distributed, the script is *used* during the process and is therefore required. How do I tell `make distcheck` to do that? I did try to have the ChangeLog target depend on the script file, but then instead of being satisfied by it (since the script file exists), it complains there's no target for it (but none needed since it's a file and it already exists). Is there a solution for this in automake? If not, I'll take a look at the implementation of distcheck and see if anything can be done about it (maybe some kind of variable(s) which means files used for inst/dist/both but are themselves nodist_noinst). fr33domlover PGP key ID: 937A67EF signature.asc Description: PGP signature
Re: nodist_noinst_SCRIPTS and `make distcheck`
On 2014-10-20 20:25 +0300, fr33domlover wrote: There are two related things that distcheck is testing here, and either one of them may be tripping you up. First, distcheck is checking that users can run make dist from your tarball. Indeed `make dist` succeeds. Second, distcheck is checking that all this works properly in VPATH builds (i.e., with srcdir != builddir). I didn't try, but I assume it will work because the only problem is that script, which IS present in the right place. The Makefile.am makes sure it will be taken from $srcdir. The problem happens because `make distcheck` copies files from the source repo into a new temporary srcdir. Not exactly. Distcheck first creates a distribution tarball (i.e., make dist), THEN it unpacks the tarball into a temporary srcdir and tests that. In other words, distcheck is directly testing the 'user experience' when they unpack a tarball you publish. Part of that user experience is that the following sequence should work: - download your package tarball from a website. - unpack it - ./configure make dist Since your script is not distributed, that sequence must not require the script to work. [...] It's going to be the first release, so I didn't try distcheck until now. The line which generates the ChangeLog in the snippet above requires that script to be present in $srcdir - but distcheck doesn't copy it to its temporary srcdir, so it's not present. You should not need to copy the script at all, as it should be run at 'make dist' time or earlier. This will happen directly in your VCS- controlled srcdir, before distcheck unpacks the tarball to test. The solution to the problem of .git not being present during distcheck may be: In distcheck-hook, take it from the right place (i.e. $(srcdir)/../.git) and then try to make the ChangeLog again. The thing about the script is, that while I can do the same (pick the script from the original $srcdir), it would be somewhat wrong design-wise - if a script is used for `make dist`, then `make distcheck` should copy it into the temporary srcdir just like the source code. While not installed nor distributed, the script is *used* during the process and is therefore required. I'm not sure I understand. When make distcheck tests the distribution, it tries to run make dist from the tarball. This must succeed *without* running your script at all, as the git history will not be available in this scenario. So when building from the tarball, your distribution should copy the already-included ChangeLog instead of generating a new one. As I mentioned, one solution is to use a dist-hook to generate the ChangeLog only when certain conditions (i.e., building from source control) are met. There are other possibile solutions: for example, Automake generates ChangeLog in builddir using a phony target. Personally, I think a dist-hook is simpler. How do I tell `make distcheck` to do that? I did try to have the ChangeLog target depend on the script file, but then instead of being satisfied by it (since the script file exists), it complains there's no target for it (but none needed since it's a file and it already exists). You can't have ChangeLog depend on your script because the script is not distributed, so this dependency can never be satisfied in the tarball. Is there a solution for this in automake? Yes, use a dist-hook. Cheers, -- Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)
Re: nodist_noinst_SCRIPTS and `make distcheck`
On 2014-10-20 Nick Bowler nbow...@elliptictech.com wrote: Not exactly. Distcheck first creates a distribution tarball (i.e., make dist), THEN it unpacks the tarball into a temporary srcdir and tests that. In other words, distcheck is directly testing the 'user experience' when they unpack a tarball you publish. Part of that user experience is that the following sequence should work: - download your package tarball from a website. - unpack it - ./configure make dist Since your script is not distributed, that sequence must not require the script to work. Thanks! Now I understand how distcheck really works. I'll just make the ChangeLog target check if the script and .git are available, as you suggest. Problem solved :-) signature.asc Description: PGP signature
