2015-12-17 7:38 GMT+01:00 Rainer Gerhards <[email protected]>: > 2015-12-17 2:04 GMT+01:00 Thomas D. <[email protected]>: >> Hi, >> >> Michael Biebl wrote: >>> The test-suite fails here: >>> >>> make[5]: *** No rule to make target 'json_null_array.sh', needed by >>> 'json_null_array.sh.log'. Schluss. >>> >>> Looks like a file is missing (once again) from the dist tarball, like in >>> https://github.com/rsyslog/rsyslog/issues/484 >> >> Yup, I already created a PR: https://github.com/rsyslog/rsyslog/pull/707 >> >> Rainer / Florian: May I ask how your release workflow looks like? I am >> wondering how this can be avoided in future. Don't you run final tests on >> the release tarball you are considering to upload? Maybe we can change the >> build bot to always use the tarball `make dist` creates instead of git >> checkouts or add an additional job which produces daily snapshots (identical >> to release tarballs) which will be tested (unpack + configure + make + make >> check...). > > Well, the core problem is that I cannot run test testbench under "make > distcheck" due to environmental differences in that mode. We already > had a couple of discussion threads on this and nobody was yet able to > provide a solution. Getting one would be great. > > I think we should do an as complete check as possible within travis. > If we had full make distcheck capability, that would be esay. As we do > not have, I currently run a make check first, then disable the > testbench, then run a make distcheck. In essence, this means we can > automatically check completeness for all but the testbench. Running a > test on the final release tarball *again* would bring up the same > issue or none. So I think there is no added value in it. > > The idea you mention with creating the tarball during test run sounds > very promising. I will give it a try. However, I will try to integrate > it into travis, as really every commit should pass this cleanly. I try > to put such general things into travis, because that is tightly > integrated with github. For 8.15, I have greatly expanded this, for > example I now run the clang static analyzer by default to detect > potential issues early (there are still three files that need some > more work to be included in that test). Builbot is my last line of > defense, it just tries different environments. As it is not that > tightly integrated into github, that final step requires more manual > labor and so I would rather like travis to find as much as possible. > > Regarding daily tarballs, we had them, but with the change of the new > build system we found nobody who wants to take care of the web > infrastructure on that system (discussed here at length) and so we do > not have them any longer. > > So it's not just that we are all being too lazy to do proper > verification, it's a pretty complex task to do. Suggestions like the > one with building out of the tarball are very useful in trying to get > us closer (I still would love to be able to use make distcheck, and I > also would love to be able to do unit tests, which is another issue we > have disucssed at length but have no solution for).
I implemented Thomas suggestion with doing the checks out of the dist tarball: https://github.com/rgerhards/rsyslog/commit/82c7eccd5a3aae2b349b79121dcff65c626b49c8 This happens under travis and so is run for each commit. This should avoid any such problems in the future. In fact, it worked so well that it detected one more missing file ;) Thanks for the suggestion and hope everybody will find this useful. Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
