Dear Duncan and Simon,
thank you both very much for you help.
I can make the test more informative and also break it down into substeps. But
I am unsure whether CRAN policies allow to use their system for such testing
steps. I rather think not. Though I must say that I still do not know how to
otherwise fix the error. Can anyone, ideally a CRAN maintainer, confirm that
this is okay?
Regarding the tolerance, the test compares to small integers. In this specific
situation both sides are 5L on my local system. The tolerance is there to
ensure that
ncol(compareStat) * testalpha is not something like 4.9 due to floating
point approximations and we end up with 4L when as.integer() is applied. This
is not happening in the concrete example, since ncol(compareStat) * testalpha =
5.67. I am very sure that the error is on the left hand side and compare is
larger than 5.
In fact, tol in
rejected[, i] <- teststat[i, ] > ret[i] + tol
may need to be larger, since teststat contains quite large values. I think
there is a realistic chance that a floating point error occurs at this point.
But once again, I do not want to send a random guess to CRAN when I cannot test
whether this has fixed the problem or not. I have tested the old code with
--disable-long-double and compiler flags such as -ffloat-store and
-fexcess-precision=standard and it works. I do not know to what degree this
ensures that it works on all systems.
Many thanks and best wishes,
Florian
From: Simon Urbanek
Sent: 22 May 2023 00:25
To: Pein, Florian
Cc: r-package-devel@r-project.org
Subject: [External] Re: [R-pkg-devel] Test fails on M1 Mac on CRAN, but not on
macOS builder
This email originated outside the University. Check before clicking links or
attachments.
Florian,
looking at the notes for 2.1-4 it says the tolerance has the wrong sign, i.e.
you're adding it to the value on both sides of the interval (instead of
subtracting for the lower bound). In your latest version the tolerances get
added everywhere so that makes even less sense to me, but then I don't know
what you actually intended to be completely honest. All I say, simply make sure
you get the logic for the tolerance intervals right.
Cheers,
Simon
> On 19/05/2023, at 9:49 PM, Pein, Florian wrote:
>
> Dear everyone,
> my R package stepR
> (https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fweb%2Fpackages%2FstepR%2F&data=05%7C01%7Cpein%40live.lancs.ac.uk%7C6cc9896064a34616d3c108db5a52aa15%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C638203083314419558%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J1MeMbuYtFfl6Qc6N142pgiychUS1I5Tdc%2Bz005lsLc%3D&reserved=0<https://cran.r-project.org/web/packages/stepR/>)
> fails the CRAN package checks on M1 Mac, but the error does not occur on the
> macOS builder
> (https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2Fmacbuilder%2Fsubmit.html&data=05%7C01%7Cpein%40live.lancs.ac.uk%7C6cc9896064a34616d3c108db5a52aa15%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C638203083314419558%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=BRu4Nser4a1YggooWKHc%2BFewcIRbL3d08%2BeO8Ovv7N0%3D&reserved=0<https://mac.r-project.org/macbuilder/submit.html>).
> So, I am unable to reproduce the error and hence unable to fix it (starring
> at the code did not help either).
>
> The relevant part is
>
> * checking tests ...
> Running ��testthat.R�� [35s/35s]
> [36s/36s] ERROR
> Running the tests in ��tests/testthat.R�� failed.
>
>> test_check("stepR")
> [ FAIL 1 | WARN 0 | SKIP 23 | PASS 22741 ]
>
> Failed tests
>
> �w�w Failure ('test-critVal.R:2463:3'): family 'hsmuce' works
> �w�w�w�w�w�w�w�w�w�w�w�w�w�w�w�w�w�w�w�w
> compare <= as.integer(ncol(compareStat) * testalpha + tolerance) is not TRUE
>
> `actual`: FALSE
> `expected`: TRUE
> sqrt
> Backtrace:
> �g
> 1. �|�wstepR (local) testVector(...) at test-critVal.R:2463:2
> 2. �|�wtestthat::expect_true(...) at test-critVal.R:50:2
>
> [ FAIL 1 | WARN 0 | SKIP 23 | PASS 22741 ]
> Error: Test failures
> Execution halted
>
>
> Has anyone an idea how to tackle this problem?
>
> The test code is long (a full version is available on CRAN). The following is
> the code part that I think is relevant (once again I cannot reproduce the
> error, so I am also unable to give a minimal reproducible example, I can only
> guess one):
>
> library(stepR)
> library(testthat)
>
&
