Re: [R-pkg-devel] RFC: an interface to manage use of parallelism in packages

[Vladimir Dergachev]

On Wed, 25 Oct 2023, Ivan Krylov wrote:


Summary: at the end of this message is a link to an R package
implementing an interface for managing the use of execution units in R
packages. As a package maintainer, would you agree to use something
like this? Does it look sufficiently reasonable to become a part of R?
Read on for why I made these particular interface choices.

My understanding of the problem stated by Simon Urbanek and Uwe Ligges
[1,2] is that we need a way to set and distribute the CPU core
allowance between multiple packages that could be using very different
methods to achieve parallel execution on the local machine, including
threads and child processes. We could have multiple well-meaning
packages, each of them calling each other using a different parallelism
technology: imagine parallel::makeCluster(getOption('mc.cores'))
combined with parallel::mclapply(mc.cores = getOption('mc.cores')) and
with an OpenMP program that also spawns getOption('mc.cores') threads.
A parallel BLAS or custom multi-threading using std::thread could add
more fuel to the fire.



Hi Ivan,

  Generally, I like the idea. A few comments:

  * from a package developer point of view, I would prefer to have a clear 
idea of how many threads I could use. So having a core R function like 
"getMaxThreads()" or similar would be useful. What that function returns 
could be governed by a package.


  In fact, it might be a good idea to allow to have several packages 
implementing "thread governors" for different situations.


  * it would make sense to think through whether we want (or not) to allow 
package developers to call omp_set_num_threads() or whether this is done 
by R.


  This is hairier than you might think. Allowing it forces every package 
to call omp_set_num_threads() before OMP block, because there is no way to 
know which packaged was called before.


  Not allowing to call omp_set_num_threads() might make it difficult to 
use all the threads, and force R to initialize OpenMP on startup.


 * Speaking of initialization of OpenMP, I have seen situations where 
spawning some regular pthread threads and then initializing OpenMP forces 
all pthread threads to a single CPU.


  I think this is because OpenMP sets thread affinity for all the process 
threads, but only distributes its own.


 * This also raises the question of how affinity is managed. If you have 
called makeForkCluster() to create 10 R instances and then each uses 2 
OpenMP threads, you do not want those occupying only 2 cpu execution 
threads instead of 20.


 * From the user perspective, it might be useful to be able to limit 
number of threads per package by using patterns or regular expressions.
Often, the reason for limiting number of threads is to reduce memory 
usage.


 * Speaking of memory usage, glibc has parameters like MALLOC_ARENA_MAX 
that have great impact on memory usage of multithreaded programs. I 
usually set it to 1, but then I take extra care to make as few memory 
allocation calls as possible within individual threads.


best

Vladimir Dergachev

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] PkgA imports PkgB, and PkgB suggests PkgA?

[Shu Fai Cheung]

Hi All,

I vaguely recall that, on CRAN, if PkgA imports PkgB, then PkgB cannot
import PkgA. (Please correct me if I am wrong.)

How about this?

PkgA imports PkgB (because PkgA has some helper functions for using PkgB)
PkgB suggests PkgA (because some vignettes or examples in PkgB use
those helpers from PkgA)

Is this allowed on CRAN?

Regards,
Shu Fai

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] [r-package-devel] Win.Metafile and package check - "Found the platform-specific device:"

[Vladimir Dergachev]

On Fri, 3 Nov 2023, wayne.w.jo...@shell.com wrote:


Dear R-Package-Devel,

As part of GWSDAT package (https://github.com/waynegitshell/GWSDAT) we support 
the option to output plots to a WMF 
(https://r-graphics.org/recipe-output-vector-wmf) format if, and only if,  the 
user is on Windows. However, when I run the package checks on here it complains 
about using a platform specific function with the following message:

 Found the platform-specific device:
   'win.metafile'
 dev.new() is the preferred way to open a new device, in the unlikely
 event one is needed.

In my opinion this is a false positive - and a similar issue has previously 
been reported here: 
https://stackoverflow.com/questions/70585796/unable-to-understand-1-note-in-devtoolscheck-caused-by-a-platform-specific-d

Any ideas on how I modify the code and package submission to automatically pass 
the checks?


Two suggestions:

  * let users specify the graphics device they want

  * reading manpage for dev.new() it accepts a bunch of options - there is 
probably a way to request the metafile device you want. But I could not 
find that in documentation.


best

Vladimir Dergachev



Thanks,

Wayne




Wayne Jones
Principal Data Scientist
Decarbonisation Data Science

Tel: +44 (0) 207 934 4330
Projects and Technology, Shell Research Limited, Shell Centre, York Road, 
London, SE1 7NA
Email: wayne.w.jo...@shell.com
Intranet: 
Shell.ai
Internet: www.shell.ai


[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel



__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Rmarkdown fails if (quote) r (space) is used

[J C Nash]

Yes. An initial space does the trick. Thanks. J

On 2023-11-03 11:48, Serguei Sokol wrote:

Le 03/11/2023 à 15:54, J C Nash a écrit :

I've spent a couple of hours with an Rmarkdown document where I
was describing some spherical coordinates made up of a radius r and
some angles. I wanted to fix the radius at 1.

In my Rmarkdown text I wrote

    Thus we have `r = 1` ...

To avoid a confusion between inline code and fixed font typesetting, could it be

    Thus we have ` r = 1` ...

(with a space after an opening quote)?

Best,
Serguei.



This caused failure to render with "unexpected =". I was using Rstudio
at first and didn't see the error msg.

If I use "radius R" and `R = 1`, things are fine, or `r=1` with no space,
but the particular "(quote) r (space)" seems to trigger code block processing.

Perhaps this note can save others some wasted time.

I had thought (obviously incorrectly) that one needed ```{r something}
to start the code chunk.

JN

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel




__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Rmarkdown fails if (quote) r (space) is used

[Serguei Sokol]

Le 03/11/2023 à 15:54, J C Nash a écrit :

I've spent a couple of hours with an Rmarkdown document where I
was describing some spherical coordinates made up of a radius r and
some angles. I wanted to fix the radius at 1.

In my Rmarkdown text I wrote

    Thus we have `r = 1` ...
To avoid a confusion between inline code and fixed font typesetting, 
could it be


   Thus we have ` r = 1` ...

(with a space after an opening quote)?

Best,
Serguei.



This caused failure to render with "unexpected =". I was using Rstudio
at first and didn't see the error msg.

If I use "radius R" and `R = 1`, things are fine, or `r=1` with no space,
but the particular "(quote) r (space)" seems to trigger code block 
processing.


Perhaps this note can save others some wasted time.

I had thought (obviously incorrectly) that one needed ```{r something}
to start the code chunk.

JN

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Rmarkdown fails if (quote) r (space) is used

[Jeff Newmiller via R-package-devel]

A code chunk does always begin with a triple backtick at the beginning of a 
line. The term for what you encountered is "inline code" used to embed computed 
results into the markdown text as though you had typed them directly.

Check out 
https://www.r-bloggers.com/2017/12/how-to-show-r-inline-code-blocks-in-r-markdown
 for relevant discussion.

On November 3, 2023 7:54:22 AM PDT, J C Nash  wrote:
>I've spent a couple of hours with an Rmarkdown document where I
>was describing some spherical coordinates made up of a radius r and
>some angles. I wanted to fix the radius at 1.
>
>In my Rmarkdown text I wrote
>
>   Thus we have `r = 1` ...
>
>This caused failure to render with "unexpected =". I was using Rstudio
>at first and didn't see the error msg.
>
>If I use "radius R" and `R = 1`, things are fine, or `r=1` with no space,
>but the particular "(quote) r (space)" seems to trigger code block processing.
>
>Perhaps this note can save others some wasted time.
>
>I had thought (obviously incorrectly) that one needed ```{r something}
>to start the code chunk.
>
>JN
>
>__
>R-package-devel@r-project.org mailing list
>https://stat.ethz.ch/mailman/listinfo/r-package-devel

-- 
Sent from my phone. Please excuse my brevity.

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] Rmarkdown fails if (quote) r (space) is used

[J C Nash]

I've spent a couple of hours with an Rmarkdown document where I
was describing some spherical coordinates made up of a radius r and
some angles. I wanted to fix the radius at 1.

In my Rmarkdown text I wrote

   Thus we have `r = 1` ...

This caused failure to render with "unexpected =". I was using Rstudio
at first and didn't see the error msg.

If I use "radius R" and `R = 1`, things are fine, or `r=1` with no space,
but the particular "(quote) r (space)" seems to trigger code block processing.

Perhaps this note can save others some wasted time.

I had thought (obviously incorrectly) that one needed ```{r something}
to start the code chunk.

JN

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] [r-package-devel] Win.Metafile and package check - "Found the platform-specific device:"

[Wayne.W.Jones]

Dear R-Package-Devel,

As part of GWSDAT package (https://github.com/waynegitshell/GWSDAT) we support 
the option to output plots to a WMF 
(https://r-graphics.org/recipe-output-vector-wmf) format if, and only if,  the 
user is on Windows. However, when I run the package checks on here it complains 
about using a platform specific function with the following message:

  Found the platform-specific device:
'win.metafile'
  dev.new() is the preferred way to open a new device, in the unlikely
  event one is needed.

In my opinion this is a false positive - and a similar issue has previously 
been reported here: 
https://stackoverflow.com/questions/70585796/unable-to-understand-1-note-in-devtoolscheck-caused-by-a-platform-specific-d

Any ideas on how I modify the code and package submission to automatically pass 
the checks?

Thanks,

Wayne




Wayne Jones
Principal Data Scientist
Decarbonisation Data Science

Tel: +44 (0) 207 934 4330
Projects and Technology, Shell Research Limited, Shell Centre, York Road, 
London, SE1 7NA
Email: wayne.w.jo...@shell.com
Intranet: 
Shell.ai
Internet: www.shell.ai


[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package bioOED has been removed from CRAN just for personal reasons

[Vladimir Dergachev]

On Wed, 1 Nov 2023, David Hugh-Jones wrote:


Aside from the package question, surely the other issue here is that Prof
Ripley’s email is extraordinarily rude. Any paid employee would be sacked
for that. I appreciate R and CRAN are volunteer-run organisations, but I
don’t think that should be an excuse for this level of, frankly, toxicity.
Why is he allowed to get away with it?


So one thing to keep in mind that doing volunteer public facing work tends 
to expose people to all kinds of unreasonable requests.


Those who endure often become more direct, and that's fine. Or, and 
people in commercial companies can be very direct too.


One thing that helps is to be extra-polite to a person who is doing a lot 
of volunteer work, and who is likely way oversubscribed.


Focusing on practical matters, if you take a step back things look 
pretty good:


Your package has dependency on a package that you have not written and
that is maintained outside CRAN. It was bound to break sooner or later.

However, the last time you updated bioOED was in 2019 and there was no 
need to do anything for more than 3 years. That's amazing ! And probably 
made possible by being a little bit too direct on occasion.


best

Vladimir Dergachev



David


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package bioOED has been removed from CRAN just for personal reasons

[David Hugh-Jones]

Guys,

Martin, I am very sorry you spent all that time. I am sceptical that this
is a correct interpretation of the law, given what I see on Twitter daily,
and I would not take a university as an authority on this. But that is
neither here nor there – if I made you waste your time, that's my bad and
I'm sorry.

More generally, I have absolutely no wish to spend time beating up on any
individual – and people have kindly sent comments off-list that gave me
context about the specific case. But persistent rudeness is harmful, and
there ought to be a way to address it. Many projects use codes of conduct
for this purpose.

David


On Fri, 3 Nov 2023 at 11:15, Martin Maechler 
wrote:

> Dear R-package-devel readers *and* notably writers,
>
> In Europe there are (diverse) laws about privacy etc, and those,
> (and/or some politeness) do not allow
> free citing of private e-mail communications in public
> nor ad hominem remarks in such public communication.
>
> For this reason, I (as mailing list co-maintainer, and notably
> responsible for the lawfulness of the public mailing list archives on
> our web server) now spent about 2 hours  to carefully obey such
> privacy requirements (carefully editing + recreating all the html
> archives).
> {In another similar case, my employer, ETH Zurich, did urge me
> "from above" to spend my time with such a tedious and ungratifying job ...}
> This is *NOT* something I'd be happy to redo.
>
> So *please* do not misuse such a public mailing list and do keep
> your private opinions private in such a case in the future!
>
> > Spencer Graves  on Thu, 2 Nov 2023 15:29:29 -0500 writes:
>
> > On 11/2/23 2:52 PM, Rolf Turner wrote:
> >>
> >> On Wed, 1 Nov 2023 16:10:34 + David Hugh-Jones wrote:
> >>
> >>> Aside from the package question, surely the other issue
> >>> here is that ’s email is extraordinarily
> >>> rude. Any paid employee would be sacked for that. I
> >>> appreciate R and CRAN are volunteer-run organisations,
> >>> but I don’t think that should be an excuse for this
> >>> level of, frankly, toxicity. Why is he allowed to get
> >>> away with it?
> >>>
> >>> David
>
> >>
> >> I've just had a look at the initial posting in this
> >> thread
> >>
> >> and can see nothing rude or offensive in the email
> >> that was copied and pasted into that  posting.
> >>
> >> I find *your* email far more offensive than anything that
> >>  has ever written.  Get a life.
> >>
> >> cheers,
> >>
> >> Rolf Turner
> >>
> >> P.S.  See fortunes::fortune(88).
> >>
> >> R. T.
>
>
> > Hi, David:
>
>
> []
>
>
> > I've been in the military, and I've learned to ignore
> > the tone and look for the value in comments I
> > receive. I've learned a lot from , and
> > others. When the tone seemed less supportive or even
> > insulting, I'm very glad the person took the time to
> > comment and didn't decide not to reply for fear of
> > offending me. I'm more productive and a better human for
> > all the help I've gotten from this and other R-related
> > lists.
>
>
> > fortunes::fortune('Spencer Graves')
>
> which can you get 4 different answers; using `showMatches` argument
> (which I think I had added), e.g. now gives
>
> > fortune("Spencer Graves", showMatches = TRUE)
> Matching row numbers: 90, 124, 177, 271
>
> Rolf Turner: In the middle of a Saturday morning (in my Time Zone!) I send
> out a plea for help, and in
> just over 20 minutes my problem is solved!
> I don't think you get service like that anywhere else. This R-help list is
> BLOODY AMAZING!
> Spencer Graves: 'The sun never sets on the (former) British Empire.'
> Today, it never sets on R-Help.
>-- Rolf Turner and Spencer Graves
>   R-help (May 2005)
>
> and then
>
> > fortune(90)
>
> Our great-great grandchildren as yet unborn may read some of the stupid
> questions and/or answers that I
> and perhaps others give from time to time. I'd rather get flamed for
> saying something stupid in public on
> this list than to continue to provide substandard service to the people
> with whom I work because I
> perpetrated the same mistake in an environment in which no one questioned
> so effectively my errors.
>-- Spencer Graves (in a discussion on whether answers on R-help should
> be more polite)
>   R-help (December 2004)
>
> > sg
>
> Martin
>
>
> --
> Martin Maechler
> ETH Zurich  and  R Core team
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package bioOED has been removed from CRAN just for personal reasons

[Martin Maechler]

Dear R-package-devel readers *and* notably writers,

In Europe there are (diverse) laws about privacy etc, and those,
(and/or some politeness) do not allow
free citing of private e-mail communications in public
nor ad hominem remarks in such public communication.

For this reason, I (as mailing list co-maintainer, and notably
responsible for the lawfulness of the public mailing list archives on
our web server) now spent about 2 hours  to carefully obey such
privacy requirements (carefully editing + recreating all the html archives).
{In another similar case, my employer, ETH Zurich, did urge me
"from above" to spend my time with such a tedious and ungratifying job ...}
This is *NOT* something I'd be happy to redo.

So *please* do not misuse such a public mailing list and do keep
your private opinions private in such a case in the future!

> Spencer Graves  on Thu, 2 Nov 2023 15:29:29 -0500 writes:

> On 11/2/23 2:52 PM, Rolf Turner wrote:
>> 
>> On Wed, 1 Nov 2023 16:10:34 + David Hugh-Jones wrote:
>> 
>>> Aside from the package question, surely the other issue
>>> here is that ’s email is extraordinarily
>>> rude. Any paid employee would be sacked for that. I
>>> appreciate R and CRAN are volunteer-run organisations,
>>> but I don’t think that should be an excuse for this
>>> level of, frankly, toxicity. Why is he allowed to get
>>> away with it?
>>> 
>>> David

>> 
>> I've just had a look at the initial posting in this
>> thread
>>
>> and can see nothing rude or offensive in the email
>> that was copied and pasted into that  posting.
>> 
>> I find *your* email far more offensive than anything that
>>  has ever written.  Get a life.
>> 
>> cheers,
>> 
>> Rolf Turner
>> 
>> P.S.  See fortunes::fortune(88).
>> 
>> R. T.


> Hi, David:


[]


> I've been in the military, and I've learned to ignore
> the tone and look for the value in comments I
> receive. I've learned a lot from , and
> others. When the tone seemed less supportive or even
> insulting, I'm very glad the person took the time to
> comment and didn't decide not to reply for fear of
> offending me. I'm more productive and a better human for
> all the help I've gotten from this and other R-related
> lists.


> fortunes::fortune('Spencer Graves')

which can you get 4 different answers; using `showMatches` argument
(which I think I had added), e.g. now gives

> fortune("Spencer Graves", showMatches = TRUE)
Matching row numbers: 90, 124, 177, 271 

Rolf Turner: In the middle of a Saturday morning (in my Time Zone!) I send out 
a plea for help, and in
just over 20 minutes my problem is solved!
I don't think you get service like that anywhere else. This R-help list is 
BLOODY AMAZING!
Spencer Graves: 'The sun never sets on the (former) British Empire.' Today, it 
never sets on R-Help.
   -- Rolf Turner and Spencer Graves
  R-help (May 2005)

and then

> fortune(90)

Our great-great grandchildren as yet unborn may read some of the stupid 
questions and/or answers that I
and perhaps others give from time to time. I'd rather get flamed for saying 
something stupid in public on
this list than to continue to provide substandard service to the people with 
whom I work because I
perpetrated the same mistake in an environment in which no one questioned so 
effectively my errors.
   -- Spencer Graves (in a discussion on whether answers on R-help should be 
more polite)
  R-help (December 2004)

> sg

Martin


--
Martin Maechler
ETH Zurich  and  R Core team

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel