Re: Fwd: New bali-phy version
Hi Benjamin, On Sun, May 06, 2018 at 06:36:51PM +0300, Benjamin Redelings wrote: > OK, I figured this out, now that I can debug the pbuilder fails. Thanks! You are welcome - package uploaded. > Yeah, that was a misleading error message from meson. It was actually > failing to find python to run the script. I removed the Build-Depends on > python 2, and added a quilt patch to change the interpreter to > /usr/bin/python3. I might change this upstream, if this also work on > homebrew, although on homebrew it is not necessary since /usr/bin/python > recently changed to point to python3 H, as far as I understand simply making /usr/bin/python to python3 is not the recommended approach since python3 is considered a "different programming language". But well, it forces people to do the right move away from Python 2. :-) > The extra error indirectly resulted from the fact that pbuilder runs the > tests with HOME=/nonexistant. This caused an extra error message. I > changed the test script to stop requiring that the error messages are > EXACTLY x, but that instead the error messages contain all lines in x. Yes, different HOME is another source of unexpected issues when using pbuilder. :-) > As a result, I can now build the package with gbp buildpackage @ pbuilder. > The repo on salsa should be up-to-date now. > > Moreover I'd recommend to switch to Python3. > The test script works in both python2 and python3. So the question is just > whether /usr/bin/python3 is a usable name on other OSs where /usr/bin/python > just works. I suspect it is. If you change tp python3 in upstream for the next release its fine. If not please add a short DEP3 header to the patch (feel free to ask if you have no idea what DEP3 might be!) Kind regards Andreas. -- http://fam-tille.de
Re: Fwd: New bali-phy version
OK, I figured this out, now that I can debug the pbuilder fails. Thanks! On 05/05/2018 08:13 PM, Andreas Tille wrote: On Sat, May 05, 2018 at 04:41:59PM +0200, Andreas Tille wrote: FileNotFoundError: [Errno 2] No such file or directory: '/build/bali-phy-3.1+dfsg/tests/run-tests.py': '/build/bali-phy-3.1+dfsg/tests/run-tests.py' No idea why this error message was printed. What was really missing was /usr/bin/python. I've added this in my latest commit to the Build-Depends. Missing Build-Depends are usually causing issued between a build on the local machine compared to pbuilder. Yeah, that was a misleading error message from meson. It was actually failing to find python to run the script. I removed the Build-Depends on python 2, and added a quilt patch to change the interpreter to /usr/bin/python3. I might change this upstream, if this also work on homebrew, although on homebrew it is not necessary since /usr/bin/python recently changed to point to python3 So while this is solved now I get a set of other errors: ... 28/28 bali-phy testsuite FAIL63.29 s OK:27 FAIL: 1 SKIP: 0 TIMEOUT:0 The output from the failed tests: 28/28 bali-phy testsuite FAIL63.29 s --- command --- /build/bali-phy-3.1.1+dfsg/tests/run-tests.py /build/bali-phy-3.1.1+dfsg/obj-x86_64-linux-gnu/src/bali-phy --package-path=/build/bali-phy-3.1.1+dfsg/obj-x86_64-linux-gnu/src/builtins:/build/bali-phy-3.1.1+dfsg --- Listing only the last 100 lines from a long log. --- Running test: parse/partitions/link/5 ... FAIL! ['error'] Running test: parse/partitions/link/7 ... FAIL! ['error'] Running test: parse/partitions/link/1 ... ok Running test: parse/partitions/link/4 ... FAIL! ['error'] ... Running test: prob_prog/if-then-else/1 ... ok Running test: IO/errors/Codons/2 ... FAIL! ['error'] Running test: IO/errors/Codons/stop/2 ... FAIL! ['error'] Running test: IO/errors/Codons/stop/1 ... FAIL! ['error'] Running test: IO/errors/Codons/AUTGC ... FAIL! ['error'] Running test: IO/errors/Codons/3 ... FAIL! ['error'] Running test: IO/errors/Codons/1 ... FAIL! ['error'] Running test: IO/errors/Triplets/2 ... FAIL! ['error'] Running test: IO/errors/Triplets/AUTGC ... FAIL! ['error'] Running test: IO/errors/Triplets/1 ... FAIL! ['error'] Running test: IO/errors/DNA-RNA/2 ... FAIL! ['error'] Running test: IO/errors/DNA-RNA/3 ... FAIL! ['error'] Running test: IO/errors/DNA-RNA/1 ... FAIL! ['error'] FAIL! (41 unexpected failures, 2 expected failures, 91 tests total) --- Full log written to /build/bali-phy-3.1.1+dfsg/obj-x86_64-linux-gnu/meson-logs/testlog.txt FAILED: meson-test /usr/bin/python3 -u /usr/bin/meson test --no-rebuild --print-errorlogs ninja: build stopped: subcommand failed. If the build works on your local machine I'd recommend to seek for 1. Further missing Build-Depends 2. Attempts to access remote resources (which is not possible in pbuilder) The extra error indirectly resulted from the fact that pbuilder runs the tests with HOME=/nonexistant. This caused an extra error message. I changed the test script to stop requiring that the error messages are EXACTLY x, but that instead the error messages contain all lines in x. As a result, I can now build the package with gbp buildpackage @ pbuilder. The repo on salsa should be up-to-date now. Moreover I'd recommend to switch to Python3. The test script works in both python2 and python3. So the question is just whether /usr/bin/python3 is a usable name on other OSs where /usr/bin/python just works. I suspect it is. macosports wants /usr/bin/env python, since they don't want to run the python from /usr/bin, but that sounds like a security risk to me... Hope this helps Very much! thanks, -BenRI
Re: Fwd: New bali-phy version
Hi Andreas, On 05/05/2018 05:41 PM, Andreas Tille wrote: Hi Benjamin, On Sat, May 05, 2018 at 02:56:56PM +0200, Benjamin Redelings wrote: uscan --verbose --force-download gbp import-orig --pristine-tar bali-phy_3.1+dfsg.orig.tar.xz since the pristine-tar branch was not populated. Either you forgot the import-orig step or you did not pushed. 1. Hmm... yes I think I only pushed master. My workflow notes now look like this: % uscan --verbose --force-download % gbp import-orig --pristine-tar ../bali-phy_+dfsg.orig.tar.xz % gbp dch % git push origin --all % debuild -S % cd .. % sudo pdebuild --update % sudo pdebuild --build bali-phy_version.dsc Does this look right? This does not look wrong but it is not needed to force `git push --all`. I only use gbp clone ... git push This might depend on git config push.default . I think if the value is "simple", then you need --all to push the pristine-tar and upstream branches. which pushes branch master, upstream and pristine-tar. Moreover, I'd recommend to use gbp-buildpackage This is helpful. Currently, this is not calling the hook scripts, so I used cowbuilder to run the hook scripts, and then gbp-buildpackage to build a full package. I have modified /etc/pbuilder/buildd-config.sh to set HOOKDIR=/var/cache/pbuilder/hooks, and added a hookfile C10_something to /root/.pbuilder/hooks and /home/bredelings/.pbuilder/hooks and /var/cache/pbuilder/hooks but gbp-buildpackage is still not calling the hooks on failure. instead of sudo pdebuild --build in any case the sudo should not be needed here. May be you re-read Debian Med policy about how to use gbp. Please also let us know if this documentation is not really helpful - than we need to enhance it to make it better. Hmm... I somehow have a hard time using the debian-med policy as a guide, whereas other documentation like the new maintainer's guide is easier to follow. It feels like the debian-med policy is aimed at people who already understand debian packaging. For example, it says "set the devscripts variables DEBEMAIL and DEBFULLNAME", but does not say that this means editting ~/.devscripts to set environment variables. The "git tips" section also feels like it is full of lots of pieces of information that I have a hard time connecting. My apologies for being slow at this! In general, I think learning how to package for Debian is a confusing process, because there are so many different documents one needs to read, and they all have a different style for managing packages. As a result, I think being able to ask questions in person has been very helpful. 1. For the debian-med policy, would it be possible to generate the HTML version so that the section headings are numbered? I think this would make the structure more clear. 2. The git tips section has many little tips, but perhaps it could be organized into (a) an overview of setting up and updating package repos and (b) an FAQ-type section. I mostly need part (a). This could maybe be chronologically organized into (a.1) setting up config files (a.2) creating a new local git repo, and (a.3) updating the repo for a new version. 3. There is a nice sequence of commands for "To create a new local git repository" (a.2). Can we add a recommended sequence of commands for "Adding a new upstream release to the repo"? (a.3) Maybe it would look a little bit like this? % uscan --verbose --force-download % gbp import-orig --pristine-tar ../PKG_VERSION.orig.tar.xz % gbp dch % gbp-buildpackage // run pbuilder to build in a chroot % git push --all % git push --tags I think it is very helpful to have a basic working set of commands like this. People can then play around with these command, read the man pages, etc. Right now the policy doesn't say anything about uscan, and that is a big time-saver. However, it does seem like you have to set up a number of config files for gbp-buildpackage to actually work, and also probably run 'pbuilder --create --distribution sid' first, or something like that. 4. I suspect that some of this chould go upstream to the new maintainers guide. The workflow there probably works fine. But this stuff about using uscan and gbp is entirely missing from section 8.3 "New upstream release", which could more accurately be called "New upstream release (without git)". 5. It might be worth mentioning the pbuilder hook for debugging build problems. pbuilder has a large learning curve, and it wasn't immediately clear what the hooks were for. So, it could have taken me a very long time to figure this out. Anyway, I hope I am not causing trouble, and I hope these comments from a beginner are helpful. FileNotFoundError: [Errno 2] No such file or directory: '/build/bali-phy-3.1+dfsg/tests/run-tests.py': '/build/bali-phy-3.1+dfsg/tests/run-tests.py' FAILED: meson-test /usr/bin/python3 -u /usr/bin/meson test --no-rebuild --print-error
Re: Fwd: New bali-phy version
Hi again, On Sat, May 05, 2018 at 04:41:59PM +0200, Andreas Tille wrote: > > > > FileNotFoundError: [Errno 2] No such file or directory: > > > '/build/bali-phy-3.1+dfsg/tests/run-tests.py': > > > '/build/bali-phy-3.1+dfsg/tests/run-tests.py' No idea why this error message was printed. What was really missing was /usr/bin/python. I've added this in my latest commit to the Build-Depends. Missing Build-Depends are usually causing issued between a build on the local machine compared to pbuilder. So while this is solved now I get a set of other errors: ... 28/28 bali-phy testsuite FAIL63.29 s OK:27 FAIL: 1 SKIP: 0 TIMEOUT:0 The output from the failed tests: 28/28 bali-phy testsuite FAIL63.29 s --- command --- /build/bali-phy-3.1.1+dfsg/tests/run-tests.py /build/bali-phy-3.1.1+dfsg/obj-x86_64-linux-gnu/src/bali-phy --package-path=/build/bali-phy-3.1.1+dfsg/obj-x86_64-linux-gnu/src/builtins:/build/bali-phy-3.1.1+dfsg --- Listing only the last 100 lines from a long log. --- Running test: parse/partitions/link/5 ... FAIL! ['error'] Running test: parse/partitions/link/7 ... FAIL! ['error'] Running test: parse/partitions/link/1 ... ok Running test: parse/partitions/link/4 ... FAIL! ['error'] ... Running test: prob_prog/if-then-else/1 ... ok Running test: IO/errors/Codons/2 ... FAIL! ['error'] Running test: IO/errors/Codons/stop/2 ... FAIL! ['error'] Running test: IO/errors/Codons/stop/1 ... FAIL! ['error'] Running test: IO/errors/Codons/AUTGC ... FAIL! ['error'] Running test: IO/errors/Codons/3 ... FAIL! ['error'] Running test: IO/errors/Codons/1 ... FAIL! ['error'] Running test: IO/errors/Triplets/2 ... FAIL! ['error'] Running test: IO/errors/Triplets/AUTGC ... FAIL! ['error'] Running test: IO/errors/Triplets/1 ... FAIL! ['error'] Running test: IO/errors/DNA-RNA/2 ... FAIL! ['error'] Running test: IO/errors/DNA-RNA/3 ... FAIL! ['error'] Running test: IO/errors/DNA-RNA/1 ... FAIL! ['error'] FAIL! (41 unexpected failures, 2 expected failures, 91 tests total) --- Full log written to /build/bali-phy-3.1.1+dfsg/obj-x86_64-linux-gnu/meson-logs/testlog.txt FAILED: meson-test /usr/bin/python3 -u /usr/bin/meson test --no-rebuild --print-errorlogs ninja: build stopped: subcommand failed. If the build works on your local machine I'd recommend to seek for 1. Further missing Build-Depends 2. Attempts to access remote resources (which is not possible in pbuilder) Moreover I'd recommend to switch to Python3. Hope this helps Andreas. -- http://fam-tille.de
Re: Fwd: New bali-phy version
Hi Benjamin, On Sat, May 05, 2018 at 02:56:56PM +0200, Benjamin Redelings wrote: > > > > uscan --verbose --force-download > > gbp import-orig --pristine-tar bali-phy_3.1+dfsg.orig.tar.xz > > > > since the pristine-tar branch was not populated. Either you forgot the > > import-orig step or you did not pushed. > 1. Hmm... yes I think I only pushed master. My workflow notes now look like > this: > > % uscan --verbose --force-download > % gbp import-orig --pristine-tar ../bali-phy_+dfsg.orig.tar.xz > % gbp dch > > % git push origin --all > % debuild -S > % cd .. > % sudo pdebuild --update > % sudo pdebuild --build bali-phy_version.dsc > > Does this look right? This does not look wrong but it is not needed to force `git push --all`. I only use gbp clone ... git push which pushes branch master, upstream and pristine-tar. Moreover, I'd recommend to use gbp-buildpackage instead of sudo pdebuild --build in any case the sudo should not be needed here. May be you re-read Debian Med policy about how to use gbp. Please also let us know if this documentation is not really helpful - than we need to enhance it to make it better. > > FileNotFoundError: [Errno 2] No such file or directory: > > '/build/bali-phy-3.1+dfsg/tests/run-tests.py': > > '/build/bali-phy-3.1+dfsg/tests/run-tests.py' > > FAILED: meson-test > > /usr/bin/python3 -u /usr/bin/meson test --no-rebuild --print-errorlogs > > ninja: build stopped: subcommand failed. > > dh_auto_test: cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 MESON_TESTTHREADS=4 > > ninja test returned exit code 1 > > make: *** [debian/rules:12: build] Error 1 > > dpkg-buildpackage: error: debian/rules build gave error exit status 2 > > ... > > > > Can you please get a fresh clone and check the potential diff? > 2. This is strange. I can reproduce the error, but I don't understand it. > The file really should be there. Also, this does not happen if I just use > debuild instead of using pbuilder. > > Um, pbuilder seems quite good at cleaning up after itself. Is it possible > to log in with the build directory untouch and figure out where the errors > are coming from? I haven't managed to figure this out yet... You might like to check out pbuilder hooks[1]. I'm using the following: $ cat /var/cache/pbuilder/hooks/C99_failed_build #!/bin/bash # https://lists.debian.org/debian-mentors/2015/02/msg00066.html echo -n 'Installing convenience apps: ' for p in mc less bash-completion ; do apt-get install -y --force-yes --no-install-recommends $p &> /dev/null echo -n "$p " done echo /bin/bash -i /dev/tty 2> /dev/tty I think you can find more examples in /usr/share/doc/pbuilder/examples. Kind regards Andreas. [1] https://wiki.ubuntu.com/PbuilderHowto#Running_a_Shell_When_Build_Fails_.28Intro_to_Hook_Scripts.29 https://wiki.debian.org/PbuilderTricks -- http://fam-tille.de
Re: Fwd: New bali-phy version
Hi Andreas, On 05/04/2018 09:14 AM, Andreas Tille wrote: Hi Benjamin, I've done uscan --verbose --force-download gbp import-orig --pristine-tar bali-phy_3.1+dfsg.orig.tar.xz since the pristine-tar branch was not populated. Either you forgot the import-orig step or you did not pushed. 1. Hmm... yes I think I only pushed master. My workflow notes now look like this: % uscan --verbose --force-download % gbp import-orig --pristine-tar ../bali-phy_+dfsg.orig.tar.xz % gbp dch % git push origin --all % debuild -S % cd .. % sudo pdebuild --update % sudo pdebuild --build bali-phy_version.dsc Does this look right? So there might be some difference to your local Git repository and there is a slight chance that this is causing the following build error (even if I doubt this reason for the issue): ... 25/28 trees-distances --help OK 0.01 s 26/28 alignment-thin --help OK 0.01 s 27/28 alignments-diff --help OK 0.01 s Traceback (most recent call last): File "/usr/bin/meson", line 26, in sys.exit(main()) File "/usr/bin/meson", line 23, in main return mesonmain.run(sys.argv[1:], launcher) File "/usr/share/meson/mesonbuild/mesonmain.py", line 278, in run return mtest.run(remaining_args) File "/usr/share/meson/mesonbuild/mtest.py", line 712, in run return th.doit() File "/usr/share/meson/mesonbuild/mtest.py", line 463, in doit self.run_tests(tests) File "/usr/share/meson/mesonbuild/mtest.py", line 621, in run_tests self.drain_futures(futures) File "/usr/share/meson/mesonbuild/mtest.py", line 637, in drain_futures self.process_test_result(result.result()) File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result return self.__get_result() File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result raise self._exception File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run result = self.fn(*self.args, **self.kwargs) File "/usr/share/meson/mesonbuild/mtest.py", line 232, in run return self._run_cmd(wrap + cmd + self.test.cmd_args + self.options.test_args) File "/usr/share/meson/mesonbuild/mtest.py", line 276, in _run_cmd preexec_fn=preexec_fn if not is_windows() else None) File "/usr/lib/python3.6/subprocess.py", line 709, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/build/bali-phy-3.1+dfsg/tests/run-tests.py': '/build/bali-phy-3.1+dfsg/tests/run-tests.py' FAILED: meson-test /usr/bin/python3 -u /usr/bin/meson test --no-rebuild --print-errorlogs ninja: build stopped: subcommand failed. dh_auto_test: cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 MESON_TESTTHREADS=4 ninja test returned exit code 1 make: *** [debian/rules:12: build] Error 1 dpkg-buildpackage: error: debian/rules build gave error exit status 2 ... Can you please get a fresh clone and check the potential diff? 2. This is strange. I can reproduce the error, but I don't understand it. The file really should be there. Also, this does not happen if I just use debuild instead of using pbuilder. Um, pbuilder seems quite good at cleaning up after itself. Is it possible to log in with the build directory untouch and figure out where the errors are coming from? I haven't managed to figure this out yet... PS: The next days I'm not as active as usual - please excuse any delays from my side. No worries, thanks for any help! It still seems pretty quick. -BenRI
Re: Fwd: New bali-phy version
Hi Benjamin, On Thu, May 03, 2018 at 05:24:11PM -0400, Benjamin Redelings wrote: > I've updated the bali-phy repo on salsa for a new version (3.1). I > added a single patch; I'll fix this upstream before the next version. > > I don't think I can upload, so if that's right, can you check the > new version and upload for me? I've done uscan --verbose --force-download gbp import-orig --pristine-tar bali-phy_3.1+dfsg.orig.tar.xz since the pristine-tar branch was not populated. Either you forgot the import-orig step or you did not pushed. So there might be some difference to your local Git repository and there is a slight chance that this is causing the following build error (even if I doubt this reason for the issue): ... 25/28 trees-distances --help OK 0.01 s 26/28 alignment-thin --help OK 0.01 s 27/28 alignments-diff --help OK 0.01 s Traceback (most recent call last): File "/usr/bin/meson", line 26, in sys.exit(main()) File "/usr/bin/meson", line 23, in main return mesonmain.run(sys.argv[1:], launcher) File "/usr/share/meson/mesonbuild/mesonmain.py", line 278, in run return mtest.run(remaining_args) File "/usr/share/meson/mesonbuild/mtest.py", line 712, in run return th.doit() File "/usr/share/meson/mesonbuild/mtest.py", line 463, in doit self.run_tests(tests) File "/usr/share/meson/mesonbuild/mtest.py", line 621, in run_tests self.drain_futures(futures) File "/usr/share/meson/mesonbuild/mtest.py", line 637, in drain_futures self.process_test_result(result.result()) File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result return self.__get_result() File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result raise self._exception File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run result = self.fn(*self.args, **self.kwargs) File "/usr/share/meson/mesonbuild/mtest.py", line 232, in run return self._run_cmd(wrap + cmd + self.test.cmd_args + self.options.test_args) File "/usr/share/meson/mesonbuild/mtest.py", line 276, in _run_cmd preexec_fn=preexec_fn if not is_windows() else None) File "/usr/lib/python3.6/subprocess.py", line 709, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/build/bali-phy-3.1+dfsg/tests/run-tests.py': '/build/bali-phy-3.1+dfsg/tests/run-tests.py' FAILED: meson-test /usr/bin/python3 -u /usr/bin/meson test --no-rebuild --print-errorlogs ninja: build stopped: subcommand failed. dh_auto_test: cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 MESON_TESTTHREADS=4 ninja test returned exit code 1 make: *** [debian/rules:12: build] Error 1 dpkg-buildpackage: error: debian/rules build gave error exit status 2 ... Can you please get a fresh clone and check the potential diff? Kind regards Andreas. PS: The next days I'm not as active as usual - please excuse any delays from my side. -- http://fam-tille.de
Fwd: New bali-phy version
Forgot to Cc list. Forwarded Message Subject:New bali-phy version Date: Thu, 3 May 2018 17:23:38 -0400 From: Benjamin Redelings To: Andreas Tille Hi Andreas, I've updated the bali-phy repo on salsa for a new version (3.1). I added a single patch; I'll fix this upstream before the next version. I don't think I can upload, so if that's right, can you check the new version and upload for me? -BenRI
