Re: [R] Is simplify2array working for dimension > 2?
Re your mention in your other mail (which didn't go to r-help) of this part of the doc: "The default value, TRUE, returns a vector or matrix if appropriate, whereas if simplify = "array" the result may be an array of “rank” (==length(dim(.))) **one higher than the result of FUN(X[[i]]).**" It's not I understood this, but it makes some sense: sapply(1:2, function(i) diag(3)) is a matrix, but sapply(1:2, function(i) diag(3), simplify = "array") is a rank-3 array. So this is the "appropriate case", and recursively simplifying nested list to high rank arrays is not something that's supposed to be done. And indeed, simplify2array(list(diag(2), diag(2))) does yield a rank-3 array as well. And it works for a list of rank-3 arrays, converted to rank-4, etc. So list of array is ok, list of list is not, except for rank 2. The behavior for rank 2 led me to think it applied as well for higher rank, and the doc for the 'higher' argument seemed to confirm this, but I was a bit optimistic. Thanks for your answer. I believe the doc may be improved a little bit, but the intent looks clearer now. Le jeu. 8 févr. 2024 à 10:32, Bert Gunter a écrit : > Jean-Claude: > > Well, here's my "explanation". Caveat emptor! > > Note that: > "simplify2array() is the utility called from sapply() when simplify is > not false" > > and > > > sapply(a, I, simplify = "array") > [,1] [,2] > [1,] list,2 list,2 > [2,] list,2 list,2 > > So it seems that simplify2array() is not intended to operate in the > way that you expected, i.e. that recursive simplification is done. > And, indeed, if you check the code for the function, you will see that > that is the case. Perhaps the key phrase in the docs is in the > sapply() part that says: > > "sapply is a user-friendly version and wrapper of lapply by default > returning a vector, matrix or, if simplify = "array", an array ***if > appropriate***, by applying simplify2array(). " In other words, > recursive simplification is considered not "appropriate". > > FWIW I also find this somewhat confusing and think that explicitly > saying that recursive simplification is not done might make it less > so. But writing docs that address all our possible misconceptions is > pretty difficult (or impossible!), and maybe adding that explicit > caveat would confuse others even more... :-( > > Cheers, > Bert > > > > > > > > > > On Thu, Feb 8, 2024 at 12:12 AM Jean-Claude Arbaut > wrote: > > > > Reading the doc for ?simplify2array, I got the impression that with the > > 'higher = T' argument the function returns an array of dimension greater > > than 2 when it makes sense (the doc says "when appropriate", which is > > rather vague). I would expect > > > > a <- list( > > list(list(1, 2), list(3, 4)), > > list(list(5, 6), list(7, 8)) > > ) > > simplify2array(a, higher = T) > > > > to return the same (possibly up to a dimension permutation) as > > array(1:8, dim = c(2, 2, 2)) > > > > However, in this case simplify2array returns a matrix (i.e. 2 dimensional > > array), whose elements are lists. > > It's the same as > > structure(list(list(1, 2), list(3, 4), list(5, 6), list(7, 8)), dim = > c(2, > > 2)) > > > > I get the same behavior with > > a <- list( > > list(c(1, 2), c(3, 4)), > > list(c(5, 6), c(7, 8)) > > ) > > but then the matrix elements are numeric vectors instead of lists. > > > > Did I miss something to get the result I expected with this function? Or > is > > it a bug? Or maybe the function is not supposed to return a higher > > dimensional array, and I didn't understand the documentation correctly? > > > > There is a workaround, one can do for instance > > array(unlist(a), dim = c(2, 2, 2)) > > and there may be better options (checking dimensions?). > > > > In case it's important: running R 4.3.2 on Debian 12.4. > > > > Best regards, > > > > Jean-Claude Arbaut > > > > [[alternative HTML version deleted]] > > > > __ > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Is simplify2array working for dimension > 2?
Reading the doc for ?simplify2array, I got the impression that with the 'higher = T' argument the function returns an array of dimension greater than 2 when it makes sense (the doc says "when appropriate", which is rather vague). I would expect a <- list( list(list(1, 2), list(3, 4)), list(list(5, 6), list(7, 8)) ) simplify2array(a, higher = T) to return the same (possibly up to a dimension permutation) as array(1:8, dim = c(2, 2, 2)) However, in this case simplify2array returns a matrix (i.e. 2 dimensional array), whose elements are lists. It's the same as structure(list(list(1, 2), list(3, 4), list(5, 6), list(7, 8)), dim = c(2, 2)) I get the same behavior with a <- list( list(c(1, 2), c(3, 4)), list(c(5, 6), c(7, 8)) ) but then the matrix elements are numeric vectors instead of lists. Did I miss something to get the result I expected with this function? Or is it a bug? Or maybe the function is not supposed to return a higher dimensional array, and I didn't understand the documentation correctly? There is a workaround, one can do for instance array(unlist(a), dim = c(2, 2, 2)) and there may be better options (checking dimensions?). In case it's important: running R 4.3.2 on Debian 12.4. Best regards, Jean-Claude Arbaut [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error with installed.packages with R 3.4.0 on Windows
Thank you all for the help. I will try the patch. Best regards, Jean-Claude Arbaut 2017-04-29 0:33 GMT+02:00 Duncan Murdoch : > On 28/04/2017 5:58 PM, Uwe Ligges wrote: > >> >> >> On 28.04.2017 19:10, Dimitri Liakhovitski wrote: >> >>> When I click on "r patched snapshot build" here >>> <https://cran.r-project.org/bin/windows/base/>, it take me here >>> <https://cran.r-project.org/bin/windows/base/rpatched.html> , it >>> says: Download R-3.3.3 Patched build for Windows >>> <https://cran.r-project.org/bin/windows/base/R-3.3.3patched-win.exe> >>> However, I am unclear how can one get to the patched 3.4.0 version? >>> >> >> If you are on Windows, you did the roight things, but the page has to be >> updated.CCing Duncan who maintains these pages. >> > > Thanks, I missed that update. It is now building 3.4.0-patched, so that > version should be available on the mirrors in a few hours. > > Duncan Murdoch > > >> Best, >> Uwe >> >> >> >> Thank you! >>> >>> On Fri, Apr 28, 2017 at 7:58 AM, Uwe Ligges >>> >> <mailto:lig...@statistik.tu-dortmund.de>> wrote: >>> >>> >>> >>> On 28.04.2017 10:45, Thierry Onkelinx wrote: >>> >>> Dear Peter, >>> >>> It actually breaks install.packages(). So it is not that >>> innocent. >>> >>> >>> And hence, as Peter exoplained, it is already fixed inn R-patched, >>> thanks to Tomas Kalibera. >>> >>> Best, >>> Uwe Ligges >>> >>> >>> >>> >>> >>> >>> Best regards, >>> >>> Thierry >>> >>> >>> Op 28 apr. 2017 10:36 a.m. schreef "peter dalgaard" >>> mailto:pda...@gmail.com>>: >>> >>> Yes, we noticed this in the last days of the code freeze before >>> release and >>> shied away from inserting a workaround, partly because we >>> couldn't see what >>> the root of the problem might be. >>> >>> For the purposes of installed.packages it is relatively harmless >>> to treat >>> the NA condition as FALSE, since it is just a matter of whether >>> a cache is >>> valid. I.e., it might cause an unnecessary cache rebuild. For >>> other >>> situations it might be more of an issue. >>> >>> The workaround (NA -> FALSE, basically) is in place in R-patched >>> and >>> R-devel. >>> >>> -pd >>> >>> On 28 Apr 2017, at 07:47 , Thierry Onkelinx >>> mailto:thierry.onkel...@inbo.be>> >>> >>> wrote: >>> >>> >>> We have several computers with the same problem. >>> >>> Op 28 apr. 2017 7:25 a.m. schreef "Jean-Claude Arbaut" >>> mailto:arbau...@gmail.com> >>> : >>> >>> Hello, >>> >>> I am currently getting a strange error when I call >>> installed.packages(): >>> >>> Error in if (file.exists(dest) && file.mtime(dest) > >>> file.mtime(lib) && : >>> missing value where TRUE/FALSE needed >>> Calls: installed.packages >>> >>> >>> I am working with R 3.4.0 on Windows. I didn't get this >>> error with R >>> >>> 3.3.3. >>> >>> Apparently, file.mtime() is returning NA well applied to a >>> directory, and >>> this causes the entire && expression to be NA, then the "if" >>> fails because >>> it needs either T or F. >>> The source of "installed.packages" seems to be roughly the >>> same as in R >>> 3.3.3, so I wonder if there have been other changes in R, >>> maybe the >>> >>> logical >>> >>> operators, that would make this function fail. >>> >>> Any idea? >>> >>> Best regards, >>> >>>
[R] Error with installed.packages with R 3.4.0 on Windows
Hello, I am currently getting a strange error when I call installed.packages(): Error in if (file.exists(dest) && file.mtime(dest) > file.mtime(lib) && : missing value where TRUE/FALSE needed Calls: installed.packages I am working with R 3.4.0 on Windows. I didn't get this error with R 3.3.3. Apparently, file.mtime() is returning NA well applied to a directory, and this causes the entire && expression to be NA, then the "if" fails because it needs either T or F. The source of "installed.packages" seems to be roughly the same as in R 3.3.3, so I wonder if there have been other changes in R, maybe the logical operators, that would make this function fail. Any idea? Best regards, Jean-Claude Arbaut [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] C/C++/Fortran Rolling Window Regressions
This may be useful:
Sven Hammarling and Craig Lucas
"Updating the QR factorization and the least squares problem"
http://eprints.ma.man.ac.uk/1192/01/covered/MIMS_ep2008_111.pdf
http://www.maths.manchester.ac.uk/~clucas/updating/
2016-07-21 20:02 GMT+02:00 jeremiah rounds :
> Hi,
>
> A not unusual task is performing a multiple regression in a rolling window
> on a time-series.A standard piece of advice for doing in R is something
> like the code that follows at the end of the email. I am currently using
> an "embed" variant of that code and that piece of advice is out there too.
>
> But, it occurs to me that for such an easily specified matrix operation
> standard R code is really slow. rollapply constantly returns to R
> interpreter at each window step for a new lm. All lm is at its heart is
> (X^t X)^(-1) * Xy, and if you think about doing that with Rcpp in rolling
> window you are just incrementing a counter and peeling off rows (or columns
> of X and y) of a particular window size, and following that up with some
> matrix multiplication in a loop. The psuedo-code for that Rcpp
> practically writes itself and you might want a wrapper of something like:
> rolling_lm (y=y, x=x, width=4).
>
> My question is this: has any of the thousands of R packages out there
> published anything like that. Rolling window multiple regressions that
> stay in C/C++ until the rolling window completes? No sense and writing it
> if it exist.
>
>
> Thanks,
> Jeremiah
>
> Standard (slow) advice for "rolling window regression" follows:
>
>
> set.seed(1)
> z <- zoo(matrix(rnorm(10), ncol = 2))
> colnames(z) <- c("y", "x")
>
> ## rolling regression of width 4
> rollapply(z, width = 4,
>function(x) coef(lm(y ~ x, data = as.data.frame(x))),
>by.column = FALSE, align = "right")
>
> ## result is identical to
> coef(lm(y ~ x, data = z[1:4,]))
> coef(lm(y ~ x, data = z[2:5,]))
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Truncated file upon reading a text file with 0xff characters
Thank you for the answer. I was about to ask why I should avoid text
connections, but actually I just noticed that with a binary connection
for the read, the problem disappears (I mean, I replace "rt" with "rb"
in the file open).
R is even clever enough that, when feeded the latin1 file after an
options(encoding="UTF-8") and no encoding in the readLines, it returns
correctly a string with encoding "unknown" and byte 0xff in the raw
representation (I would have expected at least a warning, but it
silently reads bad UTF-8 bytes as simply raw bytes, it seems)
Thus the text connection does something more that causes a problem.
Maybe it tries to translate characters twice?
And this problem remains with read.table. Not surprising: by
inspecting the source, I see it uses open(file "rt").
Jean-Claude Arbaut
2016-03-15 21:24 GMT+01:00 Duncan Murdoch :
> I think you've identified a bug (or more than one) here, but your message is
> so long, I haven't had time to go through it all. I'd suggest that you
> write up a shorter version for the bug list. The shorter version would
>
> 1. Write the latin1 file using writeBin.
> 2. Set options(encoding = "") and read it without error.
> 3. Set options(encoding = "UTF-8") and get an error even if you explicitly
> set encoding when reading.
> 4. Set options(encoding = "latin1") and also get an error with or without
> explicitly setting the encoding.
>
> I would limit the tests to readLines; read.table is much more complicated,
> and isn't necessary to illustrate the problem. It just confuses things by
> bringing it into the discussion.
>
> You should also avoid bringing text mode connections into the discussion
> unless they are necessary.
>
> Duncan Murdoch
>
>
> On 15/03/2016 3:05 PM, Jean-Claude Arbaut wrote:
>>
>> Hello R users,
>>
>> I am having problems to read a CSV file that contains names with character
>> ÿ.
>> In case it doesn't print correctly, it's Unicode character 00FF or LATIN
>> SMALL
>> LETTER Y WITH DIAERESIS.
>> My computer has Windows 7 and R 3.2.4.
>>
>> Initially, I configured my computer to run options(encoding="UTF-8")
>> in my .Rprofile,
>> since I prefer this encoding, for portability. Good and modern
>> standard, I thought.
>> Rather than sending a large file, here is how to reproduce my problem:
>>
>>options(encoding="UTF-8")
>>
>>f <- file("test.txt", "wb")
>>writeBin(as.integer(c(65, 13, 10, 66, 255, 67, 13, 10, 68, 13, 10)),
>> f, size=1)
>>close(f)
>>read.table("test.txt", encoding="latin1")
>>f <- file("test.txt", "rt")
>>readLines(f, encoding="latin1")
>>close(f)
>>
>> I write a file with three lines, in binary to avoid any translation:
>> A
>> B\xffC
>> D
>>
>> Upon reading I get only:
>>
>>> read.table("test.txt", encoding="latin1")
>> V1
>>1 A
>>2 B
>>Warning messages:
>>1: In read.table("test.txt", encoding = "latin1") :
>> invalid input found on input connection 'test.txt'
>>2: In read.table("test.txt", encoding = "latin1") :
>> incomplete final line found by readTableHeader on 'test.txt'
>>> readLines(f, encoding="latin1")
>>[1] "A" "B"
>>Warning messages:
>>1: In readLines(f, encoding = "latin1") :
>> invalid input found on input connection 'test.txt'
>>2: In readLines(f, encoding = "latin1") :
>> incomplete final line found on 'test.txt'
>>
>> Hence the file is truncated. However, character \xff is a valid latin1
>> character,
>> as one can check for instance at
>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1
>> I tried with an UTF-8 version of this file:
>>
>>f <- file("test.txt", "wb")
>>writeBin(as.integer(c(65, 13, 10, 66, 195, 191, 67, 13, 10, 68, 13,
>> 10)), f, size=1)
>>close(f)
>>read.table("test.txt", encoding="UTF-8")
>>f <- file("test.txt", "rt")
>>readLines(f, encoding="UTF-8")
>>close(f)
>>
>> Since this character ÿ is encoded as two bytes 195, 191 in UTF-8, I would
>> expect
>> that I get my complete file. But I don't. Instead, I get:
>>
>>>
[R] Truncated file upon reading a text file with 0xff characters
uot;BÿC" "D"
The second line is correctly encoded, but the encoding is just not
"marked" in one case.
With the UTF-8 bytes:
f <- file("test.txt", "wb")
writeBin(as.integer(c(65, 13, 10, 66, 195, 191, 67, 13, 10, 68, 13,
10)), f, size=1)
close(f)
a <- read.table("test.txt", encoding="UTF-8")$V1
Encoding(a)
iconv(a[2], toRaw=T)
a
a <- read.table("test.txt")$V1
Encoding(a)
iconv(a[2], toRaw=T)
a
This will yield:
> a <- read.table("test.txt", encoding="UTF-8")$V1
> Encoding(a)
[1] "unknown" "UTF-8" "unknown"
> iconv(a[2], toRaw=T)
[[1]]
[1] 42 c3 bf 43
> a
[1] "A" "BÿC" "D"
> a <- read.table("test.txt")$V1
> Encoding(a)
[1] "unknown" "unknown" "unknown"
> iconv(a[2], toRaw=T)
[[1]]
[1] 42 c3 bf 43
> a
[1] "A""BÿC" "D"
Both are correctly read (the raw bytes are ok), but the second one doesn't print
correctly because the encoding is not "marked".
My thoughts:
With options(encoding="native.enc"), the characters read are not
translated, and are read
as raw bytes, which can get an encoding mark to print correctly (otherwise it
prints as native, that is mostly latin1).
With options(encoding="latin1"), and reading the UTF-8 file, I guess it's mostly
like the preceding: the characters are read as raw, and marked as
UTF-8, which works.
With options(encoding="latin1"), and reading the latin1 file (with the
0xFF byte),
I don't understand what happens. The file gets truncated almost as if 0xFF were
an EOF character - which is perplexing, since I think that in C, 0xFF
is sometimes
(wrongly) confused with EOF.
And with options(encoding="UTF-8"), I am not sure what happens.
Questions:
* What's wrong with options(encoding="latin1")?
* Is it unsafe to use another option(encoding) than the default
native.enc, on Windows?
* Is it safe to assume that with native.enc R reads raw characters
and, only when requested,
marks an encoding afterwards? (that is, I get "unknown" by default
which is printed
as latin1 on Windows, and if I enforce another encoding, it will be
used whatever
the bytes really are)
* What does really happen with another option(encoding), especially UTF-8?
* If I save a character variable to an Rdata file, is the file usable
on another OS,
or on the same with another default encoding (by changing
options())? Does it depend
whether the character string has un "unknown" encoding or an explicit one?
* Is there a way (preferably an options()) to tell R to read text
files as UTF-8 by default?
Would it work with any one of read.table(), readLines(), or even source()?
I thought options(encoding="UTF-8") would do, but it fails on the
examples above.
Best regards,
Jean-Claude Arbaut
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] ACF values with confidence limits + Plot Extaction
a <- as.ts(rnorm(20))
png("acf.png")
a.acf <- acf(a)
dev.off()
# to see what is available
names(a.acf)
unclass(a.acf)
2016-03-06 20:06 GMT+01:00 Preetam Pal :
> Thanks, Michael. Appreciate it.
> But suppose I go for the plot, how to extract it from R ... say, I want to
> save it as a .png file.
> Regards,
> Preetam
>
> On Sun, Mar 6, 2016 at 3:39 PM, Michael Dewey
> wrote:
>
>> According to the documentation acf returns what you want. It also says
>> that it returns it invisibly if plot = TRUE which I imagine is what you are
>> doing.
>>
>> So try
>>
>> res <- acf(insert_parameters_here, plot = FALSE)
>>
>> and then look at res
>>
>>
>>
>> On 06/03/2016 07:40, Preetam Pal wrote:
>>
>>> Hi R-users,
>>>
>>> I have a time series of residuals and I want to get the ACF
>>> (autocorrelation) values till lag = 12, along with the 12 upper/lower
>>> confidence limits. I understand that acf(residual) would give me the plot,
>>> but I will also need the actual values as an array etc. Plus, I'll have to
>>> extract the plot from R as well. Is there a way to achieve these two?
>>>
>>> Appreciate your help.
>>>
>>> Regards,
>>> Preetam
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> __
>>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>> --
>> Michael
>> http://www.dewey.myzen.co.uk/home.html
>>
>
>
>
> --
> Preetam Pal
> (+91)-9432212774
> M-Stat 2nd Year, Room No. N-114
> Statistics Division, C.V.Raman
> Hall
> Indian Statistical Institute, B.H.O.S.
> Kolkata.
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
