Re: CI are forcing tests for ports where tests are disabled
On 18/3/2024 09:35, Daniel J. Luke wrote: On Mar 12, 2024, at 12:20 PM, grey wrote: But, I am guessing, it's either not that simple, or at a minimum, I might need some additional insights into how to placate it. The test phase (like most other phases) doesn't run as root by default. When you run `port test` it's failing because it can't read ssh-keysign which has "-rws--x--x 1 root admin" permissions (so it /can't/ work). You can - stop setting 'tests.run yes' since that's the port advertising that 'port test' will work (and it doesn't), tell macports to run the test phase as root (test.asroot yes should work here, but when I did a quick test it didn't and I didn't have a chance to see why), or alter the openssh tests/build to not need to be run as root. There was also a ticket about this, and some fixes for base have gone into git. Until they are released, 'test.ignore_archs yes' can be used to disable this particular test for ports that install unreadable files. - Josh
Re: CI are forcing tests for ports where tests are disabled
On Mar 12, 2024, at 12:20 PM, grey wrote: > But, I am guessing, it's either not that simple, or at a minimum, I > might need some additional insights into how to placate it. The test phase (like most other phases) doesn't run as root by default. When you run `port test` it's failing because it can't read ssh-keysign which has "-rws--x--x 1 root admin" permissions (so it /can't/ work). You can - stop setting 'tests.run yes' since that's the port advertising that 'port test' will work (and it doesn't), tell macports to run the test phase as root (test.asroot yes should work here, but when I did a quick test it didn't and I didn't have a chance to see why), or alter the openssh tests/build to not need to be run as root. -- Daniel J. Luke
Re: CI are forcing tests for ports where tests are disabled
I think I may have run into a similar issue with how tests have changed in CI with 2.9.x MacPorts and the OpenSSH 9.7p1 PR I submitted yesterday: https://github.com/macports/macports-ports/pull/22981 Going back to reverting to 9.6p1 locally and running port -d -v test I see that it fails with the same error; whereas when the CI checks for that ran with my PR in December of 2023, the CI checks passed. The man page for port test is pretty slim: test Tests portname. So I am guessing this is the spot to reference for more descriptive explanations of test.cmd and test.args https://guide.macports.org/#reference.phases.test Whereas the net/openssh Portfile just has these defined at the moment: test.runyes test.target tests I've include the output of manually invoking "make test" in the following Trac issue I opened regarding this CI buildbot failure: https://trac.macports.org/ticket/69478#comment:2 Which, as you can see, ends with "all tests passed"; so I don't think this is an issue with upstream/OpenSSH's tests. Not to mention, running make tests from the OpenSSH tarball is really quite time consuming and probably not the best use of CI buildbot resources? I can't make heads or tails of this commit from 2.9.0 which introduced the "simple default tests for all ports" https://github.com/macports/macports-base/commit/ddc26554e6ed49ce8501069b967284739fa6d34e But, I am guessing, it's either not that simple, or at a minimum, I might need some additional insights into how to placate it. Thanks!
Re: CI are forcing tests for ports where tests are disabled
Thank you, Josh, I will look into changing that then. > On Feb 29, 2024, at 2:46 PM, Joshua Root wrote: > > Oh I see, the R portgroup is overriding the test phase entirely, which makes > test.run useless. Don't do that; set test.cmd, test.args and so on. > > - Josh > > On 29/2/2024 18:34, Sergio Had wrote: >> I believe, they were not forced for R stuff until /very/ recently. >> I found a solution which should work to fix running tests for R packages >> even when tests are unsupported and forced to run, but this situation is >> quite fragile. >> For the context: there are some R packages which are in themselves trivial, >> but required as dependencies for some important ones. By default R checks >> require all /optional/ dependencies to be installed, which sometimes means >> /a lot/ of stuff to build. Adding every such optional dependency to MacPorts >> just to support testing is a) practically unfeasible and b) hardly needed. >> So while I tend to add support for testing wherever find it important or >> wherever it does not take too much of effort, there are a number of packages >> which have /test.run no – /and that is unlikely to change. >> While default behavior of R can be changed via passing a variable in >> environment, this a) does not guarantee that some tests won’t fail due to >> missing optional dependency nevertheless and b) is not clearly a superior >> choice, since it becomes less clear if some optional dependencies are >> missing (which we may care about in specific cases). >>> On Feb 29, 2024, at 2:17 PM, Joshua Root wrote: >>> >>> On 29/2/2024 17:01, Sergey Fedorov wrote: There is something broken with CI now. Tests phase must not be run when it is disabled (test.run is set to no), but it is. >>> >>> Built-in tests can always be run. See 2.9 release notes. >>> >>> - Josh >
Re: CI are forcing tests for ports where tests are disabled
Oh I see, the R portgroup is overriding the test phase entirely, which makes test.run useless. Don't do that; set test.cmd, test.args and so on. - Josh On 29/2/2024 18:34, Sergio Had wrote: I believe, they were not forced for R stuff until /very/ recently. I found a solution which should work to fix running tests for R packages even when tests are unsupported and forced to run, but this situation is quite fragile. For the context: there are some R packages which are in themselves trivial, but required as dependencies for some important ones. By default R checks require all /optional/ dependencies to be installed, which sometimes means /a lot/ of stuff to build. Adding every such optional dependency to MacPorts just to support testing is a) practically unfeasible and b) hardly needed. So while I tend to add support for testing wherever find it important or wherever it does not take too much of effort, there are a number of packages which have /test.run no – /and that is unlikely to change. While default behavior of R can be changed via passing a variable in environment, this a) does not guarantee that some tests won’t fail due to missing optional dependency nevertheless and b) is not clearly a superior choice, since it becomes less clear if some optional dependencies are missing (which we may care about in specific cases). On Feb 29, 2024, at 2:17 PM, Joshua Root wrote: On 29/2/2024 17:01, Sergey Fedorov wrote: There is something broken with CI now. Tests phase must not be run when it is disabled (test.run is set to no), but it is. Built-in tests can always be run. See 2.9 release notes. - Josh
Re: CI are forcing tests for ports where tests are disabled
I believe, they were not forced for R stuff until very recently. I found a solution which should work to fix running tests for R packages even when tests are unsupported and forced to run, but this situation is quite fragile. For the context: there are some R packages which are in themselves trivial, but required as dependencies for some important ones. By default R checks require all optional dependencies to be installed, which sometimes means a lot of stuff to build. Adding every such optional dependency to MacPorts just to support testing is a) practically unfeasible and b) hardly needed. So while I tend to add support for testing wherever find it important or wherever it does not take too much of effort, there are a number of packages which have test.run no – and that is unlikely to change. While default behavior of R can be changed via passing a variable in environment, this a) does not guarantee that some tests won’t fail due to missing optional dependency nevertheless and b) is not clearly a superior choice, since it becomes less clear if some optional dependencies are missing (which we may care about in specific cases). > On Feb 29, 2024, at 2:17 PM, Joshua Root wrote: > > On 29/2/2024 17:01, Sergey Fedorov wrote: >> There is something broken with CI now. >> Tests phase must not be run when it is disabled (test.run is set to no), but >> it is. > > Built-in tests can always be run. See 2.9 release notes. > > - Josh
Re: CI are forcing tests for ports where tests are disabled
On 29/2/2024 17:01, Sergey Fedorov wrote: There is something broken with CI now. Tests phase must not be run when it is disabled (test.run is set to no), but it is. Built-in tests can always be run. See 2.9 release notes. - Josh
CI are forcing tests for ports where tests are disabled
There is something broken with CI now. Tests phase must not be run when it is disabled (test.run is set to no), but it is.
