Re: [Rd] bug report for make check

[Tomas Kalibera]

We've fixed the test to allow the current behavior of all.equal.POSIXt 
for now. It became clear that figuring out how to best improve 
all.equal.POSIXt would take much longer.

Best
Tomas

On 10/30/20 12:00 PM, Kasper Daniel Hansen wrote:
> Thanks, I'm going to shut up then.
>
> Despite having been reported multiple times, it was not at all clear 
> to me that this was being worked on with the intention of addressing 
> make check - that is one of the limitations of the communication 
> system we use.
>
> Best,
> Kasper
>
> On Fri, Oct 30, 2020 at 11:38 AM Tomas Kalibera 
> mailto:tomas.kalib...@gmail.com>> wrote:
>
> Dear Kasper,
>
> if you want to submit a bug via bugzilla, please first read
>
> https://www.r-project.org/bugs.html
>
> and you will learn there that there is an email address to use when
> asking for an account, not all R-devel mailing list readers need
> to read
> that.
>
> Moreover, I can see you already have an account in R bugzilla.
>
> Moreover, bugs can be reported also on R-devel mailing list and
> this has
> been reported already several times, we are working on it, it is not
> clear what is the right way to fix this.
>
> Thanks for your patience,
> Tomas
>
> On 10/30/20 11:24 AM, Kasper Daniel Hansen wrote:
> > I would like to request access to bugzilla to file a bug report
> on make
> > check for R-devel.
> >
> > Following changes to all.equal.POSIXt,
> >    make check
> > now reports an error if the environment variable TZ is set to
> >    TZ="US/Eastern"
> > (and likely other values). This can be addressed by using the
> argument
> > check.tzone=FALSE in the call to all.equal.POSIXt in
> reg-tests-2.R. A patch
> > has been posted by Sebastian Meyer in the R-devel thread
> "timezone tests
> > and R-devel".
> >
>
>
>
> -- 
> Best,
> Kasper



[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] parallel PSOCK connection latency is greater on Linux?

[Jeff]

Could TCP_NODELAY and TCP_QUICKACK be exposed to the R user so that 
they might determine what is best for their potentially latency- or 
throughput-sensitive application?


Best,
Jeff

On Mon, Nov 2, 2020 at 14:05, Iñaki Ucar  
wrote:
On Mon, 2 Nov 2020 at 02:22, Simon Urbanek 
 wrote:


 It looks like R sockets on Linux could do with TCP_NODELAY -- 
without (status quo):


How many network packets are generated with and without it? If there
are many small writes and thus setting TCP_NODELAY causes many small
packets to be sent, it might make more sense to set TCP_QUICKACK
instead.

Iñaki


 Unit: microseconds
expr  min   lq mean  median   uq 
 max
  clusterEvalQ(cl, iris) 1449.997 43991.99 43975.21 43997.1 44001.91 
48027.83

  neval
   1000

 exactly the same machine + R but with TCP_NODELAY enabled in 
R_SockConnect():


 Unit: microseconds
expr min lq mean  median  uq 
 max neval
  clusterEvalQ(cl, iris) 156.125 166.41 180.8806 170.247 174.298 
5322.234  1000


 Cheers,
 Simon


 > On 2/11/2020, at 3:39 AM, Jeff  wrote:
 >
 > I'm exploring latency overhead of parallel PSOCK workers and 
noticed that serializing/unserializing data back to the main R 
session is significantly slower on Linux than it is on Windows/MacOS 
with similar hardware. Is there a reason for this difference and is 
there a way to avoid the apparent additional Linux overhead?

 >
 > I attempted to isolate the behavior with a test that simply 
returns an existing object from the worker back to the main R 
session.

 >
 > library(parallel)
 > library(microbenchmark)
 > gcinfo(TRUE)
 > cl <- makeCluster(1)
 > (x <- microbenchmark(clusterEvalQ(cl, iris), times = 1000, unit = 
"us"))

 > plot(x$time, ylab = "microseconds")
 > head(x$time, n = 10)
 >
 > On Windows/MacOS, the test runs in 300-500 microseconds depending 
on hardware. A few of the 1000 runs are an order of magnitude slower 
but this can probably be attributed to garbage collection on the 
worker.

 >
 > On Linux, the first 5 or so executions run at comparable speeds 
but all subsequent executions are two orders of magnitude slower 
(~40 milliseconds).

 >
 > I see this behavior across various platforms and hardware 
combinations:

 >
 > Ubuntu 18.04 (Intel Xeon Platinum 8259CL)
 > Linux Mint 19.3 (AMD Ryzen 7 1800X)
 > Linux Mint 20 (AMD Ryzen 7 3700X)
 > Windows 10 (AMD Ryzen 7 4800H)
 > MacOS 10.15.7 (Intel Core i7-8850H)
 >
 > __
 > R-devel@r-project.org mailing list
 > https://stat.ethz.ch/mailman/listinfo/r-devel
 >

 __
 R-devel@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-devel




--
Iñaki Úcar


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] parallel PSOCK connection latency is greater on Linux?

[Iñaki Ucar]

On Mon, 2 Nov 2020 at 14:29, Jeff  wrote:
>
> Could TCP_NODELAY and TCP_QUICKACK be exposed to the R user so that
> they might determine what is best for their potentially latency- or
> throughput-sensitive application?

I think it makes sense (with a sensible default). E.g., Julia does this [1-2].

[1] https://docs.julialang.org/en/v1/stdlib/Sockets/#Sockets.nagle
[2] https://docs.julialang.org/en/v1/stdlib/Sockets/#Sockets.quickack

-- 
Iñaki Úcar

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] vignettes present in 2 folders or won't work

[Georgi Boshnakov]

From: Duncan Murdoch 
To: Mark van der Loo 
Cc: Dirk Eddelbuettel , r-devel



Further to Duncan's comments:

> It would be nice if the documents in inst/doc were linked to on the CRAN 
> landing page of a package. I think that documents under inst/doc are a 
> bit hard to find if package authors do not create (possibly many) links 
> to them in Rd files or vignettes.

There is the seemingly underused option "package" of help():

help(package = "pkgname", help_type = "html")

The vignettes and other documents (including sources of vignettes, etc) are at 
the top of the html page (help_type is used in case the default for help is 
text format, when the output is less convenient in this case). 

What is shown can be customised by a custom index.tml under inst/doc (described 
in WRE).  An inconvenience for users of devtools::check()  is that it wipes out 
inst/doc (but it does ask for confirmation).


Georgi Boshnakov



--

Message: 12
Date: Mon, 2 Nov 2020 05:22:02 -0500
From: Duncan Murdoch 
To: Mark van der Loo 
Cc: Dirk Eddelbuettel , r-devel

Subject: Re: [Rd] vignettes present in 2 folders or won't work
Message-ID: <92e51613-b647-5d72-793a-dd64e0eda...@gmail.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

On 02/11/2020 4:07 a.m., Mark van der Loo wrote:
> 
> 
> On Sun, Nov 1, 2020 at 10:39 PM Duncan Murdoch  > wrote:
> 
> On 01/11/2020 2:57 p.m., Dirk Eddelbuettel wrote:
>  >
>  > The closest to a canonical reference for a static vignette is the
> basic blog
>  > post by Mark at
>  >
>  >
> 
> https://www.markvanderloo.eu/yaRb/2019/01/11/add-a-static-pdf-vignette-to-an-r-package/
>  >
>  > which I follow in a number of packages.
>  >
>  > Back to the original point by Alexandre: No, I do _not_ think we
> can do
>  > without a double copy of the _pre-made_ pdf ("input") and the
> _resulting_ pdf
>  > ("output").
>  >
>  > That bugs me a little too but I take it as a given as static /
> pre-made
>  > vignettes are non-standard (given lack of any mention in WRE, and
> the pretty
>  > obvious violation of the "spirit of the law" of vignette which is
> after all
>  > made to run code, not to avoid it). Yet uses for static vignettes
> are pretty
>  > valid and here we are with another clear as mud situation.
>  >
> 
> In many cases such files aren't vignettes.
> 
> By definition, packages should contain plain text source code for
> vignettes.  They can contain other PDF files in inst/doc, but if you
> don't include the plain text source, those aren't vignettes.
> 
> An exception would be a package that contains the source code but
> doesn't want to require CRAN or other users to run it, because it's too
> time-consuming, or needs obscure resources.  The CRAN policy
> discusses this.
> 
> Duncan Murdoch
> 
> 
> It would be nice if the documents in inst/doc were linked to on the CRAN 
> landing page of a package. I think that documents under inst/doc are a 
> bit hard to find if package authors do not create (possibly many) links 
> to them in Rd files or vignettes.

What I'd suggest is that you write a "browseDocs" function that displays 
them in some nice format (similar to "browseVignettes").  Maybe CRAN 
would choose to add a new category listing its results, but at a 
minimum, you could very easily add a vignette called "Other documents" 
that contains a list of links.   It wouldn't be as prominent as 
"Vignettes" on CRAN, but you could make the display as prominent as you 
want on your own web page.

Duncan Murdoch




--

Subject: Digest Footer

___
R-devel@r-project.org mailing list  DIGESTED
https://stat.ethz.ch/mailman/listinfo/r-devel


--

End of R-devel Digest, Vol 213, Issue 1
***
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] parallel PSOCK connection latency is greater on Linux?

[Iñaki Ucar]

On Mon, 2 Nov 2020 at 02:22, Simon Urbanek  wrote:
>
> It looks like R sockets on Linux could do with TCP_NODELAY -- without (status 
> quo):

How many network packets are generated with and without it? If there
are many small writes and thus setting TCP_NODELAY causes many small
packets to be sent, it might make more sense to set TCP_QUICKACK
instead.

Iñaki

> Unit: microseconds
>expr  min   lq mean  median   uq  max
>  clusterEvalQ(cl, iris) 1449.997 43991.99 43975.21 43997.1 44001.91 48027.83
>  neval
>   1000
>
> exactly the same machine + R but with TCP_NODELAY enabled in R_SockConnect():
>
> Unit: microseconds
>expr min lq mean  median  uq  max neval
>  clusterEvalQ(cl, iris) 156.125 166.41 180.8806 170.247 174.298 5322.234  1000
>
> Cheers,
> Simon
>
>
> > On 2/11/2020, at 3:39 AM, Jeff  wrote:
> >
> > I'm exploring latency overhead of parallel PSOCK workers and noticed that 
> > serializing/unserializing data back to the main R session is significantly 
> > slower on Linux than it is on Windows/MacOS with similar hardware. Is there 
> > a reason for this difference and is there a way to avoid the apparent 
> > additional Linux overhead?
> >
> > I attempted to isolate the behavior with a test that simply returns an 
> > existing object from the worker back to the main R session.
> >
> > library(parallel)
> > library(microbenchmark)
> > gcinfo(TRUE)
> > cl <- makeCluster(1)
> > (x <- microbenchmark(clusterEvalQ(cl, iris), times = 1000, unit = "us"))
> > plot(x$time, ylab = "microseconds")
> > head(x$time, n = 10)
> >
> > On Windows/MacOS, the test runs in 300-500 microseconds depending on 
> > hardware. A few of the 1000 runs are an order of magnitude slower but this 
> > can probably be attributed to garbage collection on the worker.
> >
> > On Linux, the first 5 or so executions run at comparable speeds but all 
> > subsequent executions are two orders of magnitude slower (~40 milliseconds).
> >
> > I see this behavior across various platforms and hardware combinations:
> >
> > Ubuntu 18.04 (Intel Xeon Platinum 8259CL)
> > Linux Mint 19.3 (AMD Ryzen 7 1800X)
> > Linux Mint 20 (AMD Ryzen 7 3700X)
> > Windows 10 (AMD Ryzen 7 4800H)
> > MacOS 10.15.7 (Intel Core i7-8850H)
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



-- 
Iñaki Úcar

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] vignettes present in 2 folders or won't work

[Duncan Murdoch]

On 02/11/2020 4:07 a.m., Mark van der Loo wrote:



On Sun, Nov 1, 2020 at 10:39 PM Duncan Murdoch > wrote:


On 01/11/2020 2:57 p.m., Dirk Eddelbuettel wrote:
 >
 > The closest to a canonical reference for a static vignette is the
basic blog
 > post by Mark at
 >
 >

https://www.markvanderloo.eu/yaRb/2019/01/11/add-a-static-pdf-vignette-to-an-r-package/
 >
 > which I follow in a number of packages.
 >
 > Back to the original point by Alexandre: No, I do _not_ think we
can do
 > without a double copy of the _pre-made_ pdf ("input") and the
_resulting_ pdf
 > ("output").
 >
 > That bugs me a little too but I take it as a given as static /
pre-made
 > vignettes are non-standard (given lack of any mention in WRE, and
the pretty
 > obvious violation of the "spirit of the law" of vignette which is
after all
 > made to run code, not to avoid it). Yet uses for static vignettes
are pretty
 > valid and here we are with another clear as mud situation.
 >

In many cases such files aren't vignettes.

By definition, packages should contain plain text source code for
vignettes.  They can contain other PDF files in inst/doc, but if you
don't include the plain text source, those aren't vignettes.

An exception would be a package that contains the source code but
doesn't want to require CRAN or other users to run it, because it's too
time-consuming, or needs obscure resources.  The CRAN policy
discusses this.

Duncan Murdoch


It would be nice if the documents in inst/doc were linked to on the CRAN 
landing page of a package. I think that documents under inst/doc are a 
bit hard to find if package authors do not create (possibly many) links 
to them in Rd files or vignettes.


What I'd suggest is that you write a "browseDocs" function that displays 
them in some nice format (similar to "browseVignettes").  Maybe CRAN 
would choose to add a new category listing its results, but at a 
minimum, you could very easily add a vignette called "Other documents" 
that contains a list of links.   It wouldn't be as prominent as 
"Vignettes" on CRAN, but you could make the display as prominent as you 
want on your own web page.


Duncan Murdoch

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] vignettes present in 2 folders or won't work

[Mark van der Loo]

On Sun, Nov 1, 2020 at 10:39 PM Duncan Murdoch 
wrote:

> On 01/11/2020 2:57 p.m., Dirk Eddelbuettel wrote:
> >
> > The closest to a canonical reference for a static vignette is the basic
> blog
> > post by Mark at
> >
> >
> https://www.markvanderloo.eu/yaRb/2019/01/11/add-a-static-pdf-vignette-to-an-r-package/
> >
> > which I follow in a number of packages.
> >
> > Back to the original point by Alexandre: No, I do _not_ think we can do
> > without a double copy of the _pre-made_ pdf ("input") and the
> _resulting_ pdf
> > ("output").
> >
> > That bugs me a little too but I take it as a given as static / pre-made
> > vignettes are non-standard (given lack of any mention in WRE, and the
> pretty
> > obvious violation of the "spirit of the law" of vignette which is after
> all
> > made to run code, not to avoid it). Yet uses for static vignettes are
> pretty
> > valid and here we are with another clear as mud situation.
> >
>
> In many cases such files aren't vignettes.
>
> By definition, packages should contain plain text source code for
> vignettes.  They can contain other PDF files in inst/doc, but if you
> don't include the plain text source, those aren't vignettes.
>
> An exception would be a package that contains the source code but
> doesn't want to require CRAN or other users to run it, because it's too
> time-consuming, or needs obscure resources.  The CRAN policy discusses
> this.
>
> Duncan Murdoch
>
>
It would be nice if the documents in inst/doc were linked to on the CRAN
landing page of a package. I think that documents under inst/doc are a bit
hard to find if package authors do not create (possibly many) links to them
in Rd files or vignettes.

Cheers,
Mark

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel