Re: [R-pkg-devel] using portable simd instructions
Hi Jesse, What I've done is to use a mix of compile-time detection of compiler SIMD support and run-time detection of SIMD hardware support. At package load, SIMD-specific versions of functions are installed in a symbol table. It's not perfect and it can be hard to support evolving platforms, especially now that ARM is more prevalent. However, it does allow for distribution on CRAN as it uses only autoconf, POSIX make, and no specific compiler. At compile time: 1. Use a configure script to detect the platform and any SIMD instructions supported by the compiler. This is also the time to identify the compiler flags necessary to enable instruction sets. Unlike what the existing autoconf macros do, you can ignore whether or not the host system supports the instruction sets (with the exception when compiling with Solaris Studio - it won't let you load a binary with instructions not supported by the host, even if they cannot be executed). 2. Use makefiles to conditionally compile different versions of the functions you want, one for each level of instruction set supported by the compiler, using the flags detected above. They all should be in different files with different symbols. For example: partition_sse2.c defines partition_sse2(), partition_avx.c defines partition_avx(), etc., while partition.c defines partition_c() - a fall-back compiled without any SIMD instructions. Note that echoing compilations with SIMD flags will trigger a check warning, as those units are not inherently portable. That is addressed below. At run time: 1. On package load, detect what instruction sets are supported by the host. On x86 machines, this usually involves a call to cpuid. 2. For the maximum level of instruction set supported by the host, install the relevant symbol for each function into a symbol table. Using the example above, a header defines an external function pointer partition(), which gets set to one of the SIMD-specific implementations. In setting that up, I found Agner Fog's notes on CPU dispatching to be extremely helpful. They can be found here: https://www.agner.org/optimize. I use this strategy in the dbarts package, the code for which is here: https://github.com/vdorie/dbarts. Best, Vince On Tue, Mar 26, 2024 at 10:45 AM Dirk Eddelbuettel wrote: > > On 26 March 2024 at 10:53, jesse koops wrote: > | How can I make this portable and CRAN-acceptable? > > But writing (or borrowing ?) some hardware detection via either configure / > autoconf or cmake. This is no different than other tasks decided at > install-time. > > Start with 'Writing R Extensions', as always, and work your way up from > there. And if memory serves there are already a few other packages with > SIMD > at CRAN so you can also try to take advantage of the search for a 'token' > (here: 'SIMD') at the (unofficial) CRAN mirror at GitHub: > >https://github.com/search?q=org%3Acran%20SIMD&type=code > > Hth, Dirk > > -- > dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel > [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] help fixing CRAN package sos-now fails on GitHub
A simple fix might be to redefine findFn in the global environment of the
vignette, have it explicitly call sos::findFn, and return a result
sanitized for LaTeX output. According to this:
https://stackoverflow.com/questions/32865384/function-to-sanitize-strings-for-latex-compilation
it looks like there are a couple of options for that.
On Mon, May 22, 2023 at 1:39 PM Spencer Graves <
spencer.gra...@effectivedefense.org> wrote:
> Hi, Uwe et al.:
>
>
> The sos vignette sos.Rnw now fails on GitHub, complaining of a
> LaTeX
> error in "{\tt pspline_ checker}". I don't find it in the code for
> sos.Rnw. It may be triggered by a response to an internet search that's
> different from before.
>
>
> https://github.com/sbgraves237/sos
>
>
> Uwe suggests I postprocess output with gsub("_", "_",
> output).
> It's not clear to me where I should apply this. I am not eager to
> change the code in the package, because it could break something else
> that's currently working.
>
>
> Suggestions?
> Thanks
> Spencer Graves
>
>
> On 5/16/23 7:41 AM, Uwe Ligges wrote:
> >
> >
> > On 16.05.2023 14:02, Spencer Graves wrote:
> >>
> >>
> >> On 5/16/23 6:06 AM, Uwe Ligges wrote:
> >>>
> >>>
> >>> On 16.05.2023 01:46, Spencer Graves wrote:
> Hello, All:
>
>
> The sos package is failing some CRAN checks, complaining:[1]
>
>
> LaTeX errors:
> ! Missing $ inserted.
>
> $
> l.303 {\tt pspline_
> checker} in the
> >>>
> >>>
> >>> I can only guess this is part of the response you got from some sos
> >>> request? I cannot reproduce it currently.
> >>>
> >>> So check:
> >>> Does your package pass check if some function names including an
> >>> underscore in the name is returned from an sos request?
> >>
> >>
> >> Hi, Uwe et al.:
> >>
> >>
> >>Thanks, Uwe, for your reply.
> >>
> >>
> >>It's complaining about something in a vignette that has been
> >> part of the package since it appeared in The R Journal in Volume 1/2
> >> in 2009. I received an email from Prof. Ripley complaining that it
> >> reported problems ("WARN") on some of the CRAN checks. When I asked,
> >> Prof. Ripley reply's reply included:
> >>
> >>
> >> >> l.303 {\tt pspline_
> >> >> checker} in the
> >> >> ! ==> Fatal error occurred, no output PDF file produced!
> >> >>
> >> >> Underlines need to be escaped in LaTeX. And as your results
> >> depend on
> >> >> Internet downloads,
> >> >>
> >> >> "Packages which use Internet resources should fail gracefully with
> an
> >> >> informative message if the resource is not available or has changed
> >> >> (and not give a check warning nor error)."
> >> >>
> >> >> applies: you need to anticipate that the results might include
> >> >> underlines.
> >>
> >>
> >>I don't know how to detect, let alone fix the "Underlines" that
> >> "need to be escaped in LaTeX."
> >
> > If you receive an output, postprocess it via
> >
> > gsub("_", "_", output)
> >
> >
> >
> >>
> >>
> >>Regarding the other issue that "Packages which use Internet
> >> resources should fail gracefully with an informative message if the
> >> resource is not available or has changed (and not give a check warning
> >> nor error)", I assume I should wrap in "try" all tests in *.Rd files
> >> that access the Internet and make sure that they don't fail "R CMD
> >> check" if the Internet is not available.
> >
> > Yes.
> >
> > Best,
> > Uwe Ligges
> >
> >
> >>
> >>Comments?
> >>Thanks again,
> >>Spencer Graves
> >>
> >>
> >> p.s. Yesterday I remember I got "WARN" on three of six CRAN checks
> >> against r-devel on different platforms and NOTE on four of the seven
> >> other CRAN checks. Today I see "WARN" on only two. If I just wait,
> >> these "WARN" problems may go away by themselves. However, Prof.
> >> Ripley gave me other problems to fix, and I want to support our kind,
> >> smart and generous English professor.
> >>
> >>>
> >>> Best,
> >>> Uwe Ligges
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> ! Emergency stop.
>
> $
> l.303 {\tt pspline_
> checker} in the
> ! ==> Fatal error occurred, no output PDF file produced!
> --- failed re-building 'sos.Rnw'
>
>
> I can NOT replicate these locally nor with GitHub action, and
> I failed to find 'psp' in 'sos.Rnw'.[2] This raises two issue:
>
>
> OBVIOUS: What can I do to fix this error, or at least to understand
> it better?
>
>
> SUBTLE: How can I configure "github action", so it can replicate
> the errors reported on CRAN?
>
>
> Thanks,
> Spencer
>
>
> [1]
>
>
> https://cran.r-project.org
>
>
> [2]
>
>
> https://github.com/sbgraves237/sos
>
>
> >
Re: [R-pkg-devel] How to update "SystemRequirements: C++11"?
This might be helpful. Using autoconf and ax_cxx_compile_stdcxx ( https://www.gnu.org/software/autoconf-archive/ax_cxx_compile_stdcxx.html) in package_root/tools/m4: configure.ac AC_PREREQ(2.64) AC_INIT(yourpackagename, 1.0, y...@email.com) AC_CONFIG_SRCDIR([src/main.cpp]) AC_CONFIG_AUX_DIR([tools/build-aux]) AC_CONFIG_FILES([src/Makevars]) m4_include([tools/m4/ax_cxx_compile_stdcxx.m4]) AX_CXX_COMPILE_STDCXX([11], [], [mandatory]) AX_CXX_COMPILE_STDCXX([14], [], [optional]) AX_CXX_COMPILE_STDCXX([17], [], [optional]) MAX_SUPPORTED_CXX_VERSION=CXX11 if test -z "$HAVECXX14"; then MAX_SUPPORTED_CXX_VERSION=CXX14 if test -z "$HAVECXX17"; then MAX_SUPPORTED_CXX_VERSION=CXX17 fi fi AC_SUBST(MAX_SUPPORTED_CXX_VERSION) AC_OUTPUT Makevars.in --- CXX_STD = @MAX_SUPPORTED_CXX_VERSION@ # usual make stuff DESCRIPTION --- # usual description stuff SystemRequirements: C++11 # add authors of compiler macro to authors field On Mon, Feb 6, 2023 at 6:10 PM Avraham Adler wrote: > "If a package does have a src/Makevars[.win] file then also setting > the make variable ‘CXX_STD’ there is recommended," > > Avi > > On Mon, Feb 6, 2023 at 10:58 PM Iñaki Ucar > wrote: > > > > On Mon, 6 Feb 2023 at 23:27, Avraham Adler > wrote: > > > > > > On Mon, Feb 6, 2023 at 9:46 PM Duncan Murdoch < > murdoch.dun...@gmail.com> wrote: > > > > > > > > On 06/02/2023 4:01 p.m., Duncan Murdoch wrote: > > > > > On 06/02/2023 3:46 p.m., Winston Chang wrote: > > > > >> I recently submitted a package to CRAN with "SystemRequirements: > C++11". > > > > >> This raises the following NOTE on R-devel, and I was asked to fix > and > > > > >> resubmit: > > > > >> > > > > >> * checking C++ specification ... NOTE > > > > >> Specified C++11: please update to current default of C++17 > > > > >> > > > > >> If I understand correctly, I have two options, neither of which > will work: > > > > >> 1. Remove "SystemRequirements: C++11" entirely. The problem with > this is > > > > >> that on systems with older versions of R (3.6 and below, I > believe), it may > > > > >> try to compile the package with an older C++ standard, which will > fail. > > > > >> 2. Update it to "SystemRequirements: C++17". The problem here is > that on > > > > >> systems that don't have a C++17 compiler, the package won't build > -- even > > > > >> though the package only actually requires a C++11 compiler. > > > > >> > > > > >> How should I deal with this? > > > > > > > > > > Are you allowed to say "C++11 or C++17"? > > > > > > > > Actually I can partially answer this: the test is not based on > > > > SystemRequirements, but on the log of the build, which now contains > > > > lines like this: > > > > > > > >using C compiler: ‘Apple clang version 12.0.0 > (clang-1200.0.32.28)’ > > > >using C++ compiler: ‘Apple clang version 12.0.0 > (clang-1200.0.32.28)’ > > > >using C++11 > > > >using SDK: ‘MacOSX11.1.sdk’ > > > > > > > > I get "using C++11" because of this line in Makevars: > > > > > > > >CXX_STD = CXX11 > > > > > > > > So I guess the answer is to change your Makevars to make that line > > > > conditional on running under an old version of R. If you figure out > how > > > > to do that, could you post the recipe here? > > > > > > I believe this is closely related to this email [1]. Dirk and I and > > > Aymeric have been going through the same issue with nloptr as well. > > > Our current opinion is to leave it out, as it would restrict more > > > users to demand C++17, which is unneeded, than to have users with over > > > a decade old compiler which does not have C++11. Dirk and Aymeric, > > > please chime in if I have misstated. Regardless, according to > > > Professor Ripley, if you require specific features, you should mark > > > them in _both_ Makevars and SystemRequirements, which is implied in > > > the "r-devel" version of WRE which adds the words "also" in section > > > 1.2.4. > > > > Could you please point to the specific passage? According to [1], > > section 1.2.4, declaring the C++ standard in SystemRequirements is > > only mandatory for C++17 or later. > > > > [1] > https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Using-C_002b_002b-code > > > > -- > > Iñaki Úcar > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel > [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
