Re: [Rd] Bug with `[<-.POSIXlt` on specific OSes
> Davis Vaughan writes:
> I've got a bit more information about this one. It seems like it
> (only? not sure) appears when `TZ = "UTC"`, which is why I didn't see
> it before on my Mac, which defaults to `TZ = ""`. I think this is at
> least explainable by the fact that those "optional" fields aren't
> technically needed when the time zone is UTC.
Exactly. Debugging `[<-.POSIlt` with
x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
Sys.setenv(TZ = "UTC")
x[1] <- NA
shows we get into
value <- unclass(as.POSIXlt(value))
if (ici) {
for (n in names(x)) names(x[[n]]) <- nms
}
for (n in names(x)) x[[n]][i] <- value[[n]]
where
Browse[2]> names(value)
[1] "sec" "min" "hour" "mday" "mon" "year" "wday" "yday" "isdst"
Browse[2]> names(x)
[1] "sec""min""hour" "mday" "mon""year" "wday" "yday"
[9] "isdst" "zone" "gmtoff"
Without having looked at the code, the docs say
‘zone’ (Optional.) The abbreviation for the time zone in force at
that time: ‘""’ if unknown (but ‘""’ might also be used for
UTC).
‘gmtoff’ (Optional.) The offset in seconds from GMT: positive
values are East of the meridian. Usually ‘NA’ if unknown,
but ‘0’ could mean unknown.
so perhaps we should fill with the values for the unknown case?
-k
> I can reproduce this now on my personal Mac:
> ```
> x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
> Sys.setenv(TZ = "")
> x[1] <- NA
> x
> #> [1] NA
> x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
> Sys.setenv(TZ = "America/New_York")
> x[1] <- NA
> x
> #> [1] NA
> x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
> Sys.setenv(TZ = "UTC")
> x[1] <- NA
> #> Error in x[[n]][i] <- value[[n]] : replacement has length zero
> x
> #> [1] "2013-01-31 CST"
> ```
> Here are `sessionInfo()` and `Sys.getenv("TZ")` outputs for 3 GitHub
> Actions platforms where the bug exists (note they all set `TZ = "UTC"`!):
> Linux:
> ```
>> sessionInfo()
> R version 4.2.1 (2022-06-23)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 18.04.6 LTS
> Matrix products: default
> BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
> LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
> locale:
> [1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
> [4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8LC_MESSAGES=C.UTF-8
> [7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C
> [10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
> loaded via a namespace (and not attached):
> [1] compiler_4.2.1
>> Sys.getenv("TZ")
> [1] "UTC"
> ```
> Mac:
> ```
>> sessionInfo()
> R version 4.2.1 (2022-06-23)
> Platform: x86_64-apple-darwin17.0 (64-bit)
> Running under: macOS Big Sur ... 10.16
> Matrix products: default
> BLAS:
> /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
> LAPACK:
> /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
> locale:
> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
> loaded via a namespace (and not attached):
> [1] compiler_4.2.1
>> Sys.getenv("TZ")
> [1] "UTC"
> ```
> Windows:
> This is the best I can get you, sorry (remote worker issues), but note that
> it does also say `tz UTC` like the others.
> ```
> version R version 4.2.1 (2022-06-23 ucrt)
> os Windows Server x64 (build 20348)
> system x86_64, mingw32
> ui RTerm
> language (EN)
> collate English_United States.utf8
> ctype English_United States.utf8
> tz UTC
> date 2022-10-11
> ```
> And here is my Mac where the bug doesn't show up by default because `TZ =
> ""`:
> ```
>> sessionInfo()
> R version 4.2.1 (2022-06-23)
> Platform: x86_64-apple-darwin17.0 (64-bit)
> Running under: macOS Big Sur ... 10.16
> Matrix products: default
> BLAS:
> /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
> LAPACK:
> /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
> locale:
> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
> loaded via a namespace (and not attached):
> [1] compiler_4.2.1
>> Sys.getenv("TZ")
> [1] ""
>> Sys.timezone()
> [1] "America/New_York"
> ```
> -Davis
> On Thu, Oct 6, 2022 at 9:33 AM Davis Vaughan wrote:
>> Hi all,
>>
>> I have found another POSIXlt bug while I've been fiddling around with it.
>> This one only appears on specific OSes, because it has to do with the fact
>> that the `gmtoff` field is optional, and isn't always used on all OSes. It
>> also doesn't seem to be specific to
Re: [Rd] Bug with `[<-.POSIXlt` on specific OSes
I've got a bit more information about this one. It seems like it (only? not
sure) appears when `TZ = "UTC"`, which is why I didn't see it before on my
Mac, which defaults to `TZ = ""`. I think this is at least explainable by
the fact that those "optional" fields aren't technically needed when the
time zone is UTC.
I can reproduce this now on my personal Mac:
```
x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
Sys.setenv(TZ = "")
x[1] <- NA
x
#> [1] NA
x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
Sys.setenv(TZ = "America/New_York")
x[1] <- NA
x
#> [1] NA
x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
Sys.setenv(TZ = "UTC")
x[1] <- NA
#> Error in x[[n]][i] <- value[[n]] : replacement has length zero
x
#> [1] "2013-01-31 CST"
```
Here are `sessionInfo()` and `Sys.getenv("TZ")` outputs for 3 GitHub
Actions platforms where the bug exists (note they all set `TZ = "UTC"`!):
Linux:
```
> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
[4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8LC_MESSAGES=C.UTF-8
[7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.2.1
> Sys.getenv("TZ")
[1] "UTC"
```
Mac:
```
> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur ... 10.16
Matrix products: default
BLAS:
/Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
LAPACK:
/Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.2.1
> Sys.getenv("TZ")
[1] "UTC"
```
Windows:
This is the best I can get you, sorry (remote worker issues), but note that
it does also say `tz UTC` like the others.
```
version R version 4.2.1 (2022-06-23 ucrt)
os Windows Server x64 (build 20348)
system x86_64, mingw32
ui RTerm
language (EN)
collate English_United States.utf8
ctype English_United States.utf8
tz UTC
date 2022-10-11
```
And here is my Mac where the bug doesn't show up by default because `TZ =
""`:
```
> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur ... 10.16
Matrix products: default
BLAS:
/Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
LAPACK:
/Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.2.1
> Sys.getenv("TZ")
[1] ""
> Sys.timezone()
[1] "America/New_York"
```
-Davis
On Thu, Oct 6, 2022 at 9:33 AM Davis Vaughan wrote:
> Hi all,
>
> I have found another POSIXlt bug while I've been fiddling around with it.
> This one only appears on specific OSes, because it has to do with the fact
> that the `gmtoff` field is optional, and isn't always used on all OSes. It
> also doesn't seem to be specific to r-devel, I think it has been there
> awhile.
>
> Here is the bug:
>
> ```
> x <- as.POSIXlt(as.POSIXct("2013-01-31", tz = "America/Chicago"))
>
> # Oh no!
> x[1] <- NA
> #> Error in x[[n]][i] <- value[[n]] : replacement has length zero
> ```
>
> If you look at the objects, you can see that `x` has a `gmtoff` field, but
> `NA` (when converted to POSIXlt, which is what `[<-.POSIXlt` does) does not:
>
> ```
> unclass(x)
> #> $sec
> #> [1] 0
> #>
> #> $min
> #> [1] 0
> #>
> #> $hour
> #> [1] 0
> #>
> #> $mday
> #> [1] 31
> #>
> #> $mon
> #> [1] 0
> #>
> #> $year
> #> [1] 113
> #>
> #> $wday
> #> [1] 4
> #>
> #> $yday
> #> [1] 30
> #>
> #> $isdst
> #> [1] 0
> #>
> #> $zone
> #> [1] "CST"
> #>
> #> $gmtoff
> #> [1] -21600
> #>
> #> attr(,"tzone")
> #> [1] "America/Chicago" "CST" "CDT"
>
> unclass(as.POSIXlt(NA))
> #> $sec
> #> [1] NA
> #>
> #> $min
> #> [1] NA
> #>
> #> $hour
> #> [1] NA
> #>
> #> $mday
> #> [1] NA
> #>
> #> $mon
> #> [1] NA
> #>
> #> $year
> #> [1] NA
> #>
> #> $wday
> #> [1] NA
> #>
> #> $yday
> #> [1] NA
> #>
> #> $isdst
> #> [1] -1
> #>
> #> attr(,"tzone")
> #> [1] "UTC"
> ```
>
> The problem seems to be that `[<-.POSIXlt` assumes that if the field was
> there in `x` then it must also be there in `va
Re: [Rd] Rscript -e EXPR fails to launch if stdin is closed
There's still 2 weeks till code freeze for 4.2.2, and porting the fix would be
trivial. As long as there is no risk that someone will get the bright idea of
changing a critical package to depend on R >= 4.2.2...
- pd
> On 10 Oct 2022, at 18:34 , Henrik Bengtsson
> wrote:
>
> Thank you Peter for the quick fix. Will this make it into R-patched
> to become R 4.2.2 soon?
>
> I can confirm that the fix resolved also the original problem report
> involving launching a parallel PSOCK cluster from within a 'processx'
> background process
> (https://stackoverflow.com/questions/73962109/why-are-the-workers-failing-to-connect-when-calling-makepsockcluster-from-an-e/73991833#73991833
> and https://github.com/r-lib/callr/issues/236)
>
>
> /Henrik
>
> On Mon, Oct 10, 2022 at 5:54 AM peter dalgaard wrote:
>>
>> It seems to work simply to do "if (ifd >= 0)..." (the ifp test is fine
>> since ifp is FILE* and initialized to NULL). Will commit (to r-devel for
>> now).
>>
>> -pd
>>
>>> On 10 Oct 2022, at 11:07 , peter dalgaard wrote:
>>>
>>> He!
>>>
>>> Yes, that looks like a blunder.
>>>
>>> mkstemp() returns -1 on failure, not 0, so the test on ifd (and I suppose
>>> also the one on ifp) is wrong. And of course, once you close file
>>> descriptor 0, mkstemp() chooses the 1st available fd, i.e. 0, for its
>>> return value.
>>>
>>> -pd
>>>
On 9 Oct 2022, at 20:25 , Henrik Bengtsson
wrote:
Rscript fails to launch if the standard input (stdin) is closed, e.g.
$ Rscript --vanilla -e 42 0<&-
Fatal error: creating temporary file for '-e' failed
This appear to only happen with `-e EXPR`, e.g. it works when doing:
$ echo "42" > script.R
$ Rscript --vanilla script.R 0<&-
[1] 42
and:
$ R --vanilla 0<&-
R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
...
>
TROUBLESHOOTING:
$ strace Rscript --vanilla -e 42 0<&-
execve("/home/hb/shared/software/CBI/R-4.2.1-gcc9/bin/Rscript",
["Rscript", "--vanilla", "-e", "42"], 0x7fff9f476418 /* 147 vars */) =
0
brk(NULL) = 0x5625ca9e6000
arch_prctl(0x3001 /* ARCH_??? */, 0x7fff23b4d260) = -1 EINVAL (Invalid
argument)
...
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}) = 0
write(1, "Fatal error: creating temporary "..., 53Fatal error:
creating temporary file for '-e' failed
) = 53
exit_group(2) = ?
+++ exited with 2 +++
which points to src/unix/system.c:
ifd = mkstemp(ifile);
if (ifd > 0)
ifp = fdopen(ifd, "w+");
if(!ifp) R_Suicide(_("creating temporary file for '-e' failed"));
One rationale for having closed standard files (including stdin) is to
avoid leaking file descriptors, cf.
https://wiki.sei.cmu.edu/confluence/display/c/FIO22-C.+Close+files+before+spawning+processes
and https://danwalsh.livejournal.com/53603.html. The background for
reporting on this was that `system()` fails to work in processx
spawned processes, which closes the standard files by default in
processx (<= 3.7.0).
Best,
Henrik
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>> --
>>> Peter Dalgaard, Professor,
>>> Center for Statistics, Copenhagen Business School
>>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>>> Phone: (+45)38153501
>>> Office: A 4.23
>>> Email: pd@cbs.dk Priv: pda...@gmail.com
>>>
>>
>> --
>> Peter Dalgaard, Professor,
>> Center for Statistics, Copenhagen Business School
>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>> Phone: (+45)38153501
>> Office: A 4.23
>> Email: pd@cbs.dk Priv: pda...@gmail.com
>>
--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd@cbs.dk Priv: pda...@gmail.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] A potential POSIXlt->Date bug introduced in r-devel
> Davis Vaughan > on Fri, 7 Oct 2022 11:00:23 -0400 writes: > Martin, > FWIW, I scoured the docs using GitHub's new code search preview but can't > seem to find any reference to the fact that POSIXlt fields are internally > recycled (even though lubridate seems to have been relying on this for > quite some time). Thank you, Davis, for searching and confirming that this has never been documented. And so, in that sense there's no bug, because only non-documented behaviour has changed. Still, with svn rev 83062 I have finally committed a version of the C code underlying as.Date.POSIXlt() which (I think/hope) does deal correctly with such partially filled POSIXlt objects (it does with your relatively benign example and with the more nasty one I had mentioned). BTW, at least length() has been working correctly for these, for a bit less than a day now, too, since svn rev 83056 . These are still only some of several steps to correctly support such objects. Notably the `[` and `[<-` methods will need a substantial update, too (as you have noticed for the latter in your other R-devel post on Oct 6, Subject "Bug with `[<-.POSIXlt` on specific OSes"). Note that I'd be quite grateful for well documented platform dependent behavior. In addition to a typical sessionInfo() this would also need to report the current setting of a "TZ" environment variable, where I have noticed [on Linux, Fedora 36] that there may be different behavior between TZ entirely undefined and a TZ="" setting (which I intuitively assumed would work equivalently). Best, Martin > -Davis > On Fri, Oct 7, 2022 at 8:52 AM Martin Maechler > wrote: >> > Martin Maechler >> > on Thu, 6 Oct 2022 10:15:29 +0200 writes: >> >> > Davis Vaughan >> > on Wed, 5 Oct 2022 17:04:11 -0400 writes: >> >> >> Hi all, >> >> >> I think I have discovered a bug in the conversion from POSIXlt to >> Date that has been introduced in r-devel. >> >> >> It affects lubridate, but surprisingly didn't cause test failures >> there. [..] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
