[R] Tidyverse/dplyr solution for filling values of a tibble/dataframe from a column with a nested list.
Hi experts.
I have a tibble with a column containing a nested list (>>
data type to be specific).
Looks something like the following (but in R/Arrow format):
ID
Nestedvals
001
[[1]](1,0.1)[[2]](2,0.2)[[3]](3,0.3)[[4]](4,0.4)[[5]](5,0.5)
002
[[1]](1,0.1)[[2]](2,0.2)[[3]](3,0.3)[[4]](4,0.4)
003
[[1]](1,0.1)[[2]](2,0.2)[[3]](3,0.3)
004
[[1]](1,0.1)[[2]](2,0.2)
005
[[1]](1,0.1)
Basically, each list contains a set of doubles, with the first indicating a
specific index (based on the 0 beginning python index), and a certain value
(e.g. 0.5).
What I would like to do is generate set of columns based on the rang of unique
indexes of each nested list. e.g.:
col_1, col_2, col_3, col_4, col_5
Which I have done with the following:
tibble[paste0("col_", 1:5)] <- 0
And then replace each 0 with the value (second number in the nested list),
based on the index (first number in each nested list), for each row of the
tibble.
I wrote a function to split each nested list:
nestsplit <- function(x, y) {
`unlist(lapply(x, [[`, y))
}
And then generate unique columns with the column names (by index) and values of
interest to append to the tibble:
tibble <-
tibble |> rowwise() |> mutate(index_names = list(paste0(
"col_", as.character(nestsplit(nestedvals, 1))
)),
index_values = list(nestsplit(nestedvals, 2)))
But I would like to see if there is an efficient, tidyverse/dplyr-based
solution to individually assign these values rather than writing a loop to
assign each of them by row.
So that an output like this:
ID
Nestedvals
col_1
col_2
col_3
col_4
col_5
001
0
0
0
0
0
002
0
0
0
0
0
003
0
0
0
0
0
004
0
0
0
0
0
005
0
0
0
0
0
Looks instead like the following:
ID
Nestedvals
col_1
col_2
col_3
col_4
col_5
001
0.1
0.2
0.3
0.4
0.5
002
0.1
0.2
0.3
0.4
0
003
0.1
0.2
0.3
0
0
004
0.1
0.2
0
0
0
005
0.1
0
0
0
0
-
I would love to give an example to simulate the exact nature of the data, but
I'm unfortunately not sure how to recreate this class for an example:
> typeof(tibble$var)
[1] "list"
> class(tibble$var)
[1] "arrow_list""vctrs_list_of" "vctrs_vctr""list"
The closest I have ever been able to get is with:
tibble(ID = c("001", "002", "003", "004", "005"), nestedvals =
list(list(c(1,0.1),c(2,0.2),c(3,0.3),c(4,0.4),c(5,0.5)),list(c(1,0.1),c(2,0.2),c(3,0.3),c(4,0.4)),list(c(1,0.1),c(2,0.2),c(3,0.3)),list(c(1,0.1),c(2,0.2)),list(c(1,0.1
Which gives a list datatype instead of >>
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Mass General Brigham
Compliance HelpLine at https://www.massgeneralbrigham.org/complianceline
.
Please note that this e-mail is not secure (encrypted). If you do not wish to
continue communication over unencrypted e-mail, please notify the sender of
this message immediately. Continuing to send or respond to e-mail after
receiving this message means you understand and accept this risk and wish to
continue to communicate over unencrypted e-mail.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Import multiple tif raster
В Thu, 18 Apr 2024 19:04:04 +0200
пишет:
> Solution: direct and full path to the .tif files.
> I confused back and forward slash
Congratulations on solving the problem yourself!
Here's a few more options that could be useful in the future:
1. Double the backslashes. Between single quotes ('...') and double
quotes ("..."), backslashes have a special meaning. This makes it
possible to enter characters without typing them directly (e.g. '\uf6'
results in "ö") or put quotes inside quoted strings (e.g. message('\'')
prints a single quote), but in order to mean \, you have to type \\:
path = "C:\\Users\\Sibylle
Stöckli\\Desktop\\NCCS_Impacts_Lot2_2022\\InVEST\\Species_Input\\valpar_bee_presence"
2. Use the special "raw character constant" syntax where backslashes
don't have a special meaning:
path = r"{C:\Users\Sibylle
Stöckli\Desktop\NCCS_Impacts_Lot2_2022\InVEST\Species_Input\valpar_bee_presence}"
These two options are explained in help(Quotes).
3. Instead of typing the path, use choose.dir() (Windows only, I think)
and select it interactively. You can print() the return value and copy
and paste it into the script later.
--
Best regards,
Ivan
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] [Tagged] Re: Import multiple tif raster
Your original code with relative path would also work if you did not put the leading slash. On April 18, 2024 10:04:04 AM PDT, "SIBYLLE STÖCKLI via R-help" wrote: >Dear community >Dear Ivan > >Thanks a lot. The code works now. Solution: direct and full path to the .tif >files. >I confused back and forward slash > > >#first import all files in a single folder as a list >rastlist <- list.files(path = "C:/Users/Sibylle >Stöckli/Desktop/NCCS_Impacts_Lot2_2022/InVEST/Species_Input/valpar_bee_presence", > pattern='.tif$', all.files= T, full.names= T) > >At the end I got a warning >#to run a function on an individual raster e.g., plot >> boxplot(allrasters[[1]]) >> >> p<-boxplot(allrasters[[1]]) >Warning message: >In .local(x, ...) : taking a sample of 1e+05 cells >> >Not all of the characters in C:/Users/Sibylle >Stöckli/Desktop/Test_Rasterfile.R could be encoded using ASCII. To save using >a different encoding, choose "File | Save with Encoding..." from the main menu. >> > >Kind regards >Sibylle > >-Original Message- >From: R-help On Behalf Of SIBYLLE STÖCKLI via >R-help >Sent: Thursday, April 18, 2024 6:48 PM >To: 'Ivan Krylov' ; 'SIBYLLE STÖCKLI via R-help' > >Subject: Re: [R] Import multiple tif raster > >Dear Ivan > >Thanks a lot. > >I tried now to provide the full path. However probably the "ö" in the path >produces the error, would that be possible? > >> #first import all files in a single folder as a list rastlist <- >> list.files(path = "C:\Users\Sibylle >> Stöckli\Desktop\NCCS_Impacts_Lot2_2022\InVEST\Species_Input\valpar_bee >> _presence", pattern='.tif$', all.files= T, full.names= T) >Error: '\U' used without hex digits in character string (:1:35) > >Kind regards >Sibylle > >-Original Message- >From: Ivan Krylov >Sent: Thursday, April 18, 2024 2:39 PM >To: SIBYLLE STÖCKLI via R-help >Cc: sibylle.stoec...@gmx.ch >Subject: Re: [R] Import multiple tif raster > >В Thu, 18 Apr 2024 11:08:33 +0200 >SIBYLLE STÖCKLI via R-help пишет: > >> > #to check the index numbers of all imported raster list elements >> > allrasters >> list() >> > >> > #call single raster element >> > allrasters[[1]] >> Error in allrasters[[1]] : subscript out of bounds > >`allrasters` is an empty list, so it doesn't have a first item. >Therefore, allrasters[[1]] is an error. Why is this so? Probably because >list.files(...) above returned an empty vector. > >Check rastlist and/or length(rastlist). Make sure that the path you're giving >to list.files (the one starting with >/NCCS_Impacts_Lot2_2022/InVEST/...) exists on your computer. Did you mean to >start it with a slash, making it an absolute path starting from the root of >the filesystem? > >-- >Best regards, >Ivan > >__ >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. > >__ >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. -- Sent from my phone. Please excuse my brevity. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Import multiple tif raster
Dear community Dear Ivan Thanks a lot. The code works now. Solution: direct and full path to the .tif files. I confused back and forward slash #first import all files in a single folder as a list rastlist <- list.files(path = "C:/Users/Sibylle Stöckli/Desktop/NCCS_Impacts_Lot2_2022/InVEST/Species_Input/valpar_bee_presence", pattern='.tif$', all.files= T, full.names= T) At the end I got a warning #to run a function on an individual raster e.g., plot > boxplot(allrasters[[1]]) > > p<-boxplot(allrasters[[1]]) Warning message: In .local(x, ...) : taking a sample of 1e+05 cells > Not all of the characters in C:/Users/Sibylle Stöckli/Desktop/Test_Rasterfile.R could be encoded using ASCII. To save using a different encoding, choose "File | Save with Encoding..." from the main menu. > Kind regards Sibylle -Original Message- From: R-help On Behalf Of SIBYLLE STÖCKLI via R-help Sent: Thursday, April 18, 2024 6:48 PM To: 'Ivan Krylov' ; 'SIBYLLE STÖCKLI via R-help' Subject: Re: [R] Import multiple tif raster Dear Ivan Thanks a lot. I tried now to provide the full path. However probably the "ö" in the path produces the error, would that be possible? > #first import all files in a single folder as a list rastlist <- > list.files(path = "C:\Users\Sibylle > Stöckli\Desktop\NCCS_Impacts_Lot2_2022\InVEST\Species_Input\valpar_bee > _presence", pattern='.tif$', all.files= T, full.names= T) Error: '\U' used without hex digits in character string (:1:35) Kind regards Sibylle -Original Message- From: Ivan Krylov Sent: Thursday, April 18, 2024 2:39 PM To: SIBYLLE STÖCKLI via R-help Cc: sibylle.stoec...@gmx.ch Subject: Re: [R] Import multiple tif raster В Thu, 18 Apr 2024 11:08:33 +0200 SIBYLLE STÖCKLI via R-help пишет: > > #to check the index numbers of all imported raster list elements > > allrasters > list() > > > > #call single raster element > > allrasters[[1]] > Error in allrasters[[1]] : subscript out of bounds `allrasters` is an empty list, so it doesn't have a first item. Therefore, allrasters[[1]] is an error. Why is this so? Probably because list.files(...) above returned an empty vector. Check rastlist and/or length(rastlist). Make sure that the path you're giving to list.files (the one starting with /NCCS_Impacts_Lot2_2022/InVEST/...) exists on your computer. Did you mean to start it with a slash, making it an absolute path starting from the root of the filesystem? -- Best regards, Ivan __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Import multiple tif raster
Dear Ivan Thanks a lot. I tried now to provide the full path. However probably the "ö" in the path produces the error, would that be possible? > #first import all files in a single folder as a list > rastlist <- list.files(path = "C:\Users\Sibylle > Stöckli\Desktop\NCCS_Impacts_Lot2_2022\InVEST\Species_Input\valpar_bee_presence", > pattern='.tif$', all.files= T, full.names= T) Error: '\U' used without hex digits in character string (:1:35) Kind regards Sibylle -Original Message- From: Ivan Krylov Sent: Thursday, April 18, 2024 2:39 PM To: SIBYLLE STÖCKLI via R-help Cc: sibylle.stoec...@gmx.ch Subject: Re: [R] Import multiple tif raster В Thu, 18 Apr 2024 11:08:33 +0200 SIBYLLE STÖCKLI via R-help пишет: > > #to check the index numbers of all imported raster list elements > > allrasters > list() > > > > #call single raster element > > allrasters[[1]] > Error in allrasters[[1]] : subscript out of bounds `allrasters` is an empty list, so it doesn't have a first item. Therefore, allrasters[[1]] is an error. Why is this so? Probably because list.files(...) above returned an empty vector. Check rastlist and/or length(rastlist). Make sure that the path you're giving to list.files (the one starting with /NCCS_Impacts_Lot2_2022/InVEST/...) exists on your computer. Did you mean to start it with a slash, making it an absolute path starting from the root of the filesystem? -- Best regards, Ivan __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Import multiple tif raster
В Thu, 18 Apr 2024 11:08:33 +0200 SIBYLLE STÖCKLI via R-help пишет: > > #to check the index numbers of all imported raster list elements > > allrasters > list() > > > > #call single raster element > > allrasters[[1]] > Error in allrasters[[1]] : subscript out of bounds `allrasters` is an empty list, so it doesn't have a first item. Therefore, allrasters[[1]] is an error. Why is this so? Probably because list.files(...) above returned an empty vector. Check rastlist and/or length(rastlist). Make sure that the path you're giving to list.files (the one starting with /NCCS_Impacts_Lot2_2022/InVEST/...) exists on your computer. Did you mean to start it with a slash, making it an absolute path starting from the root of the filesystem? -- Best regards, Ivan __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Import multiple tif raster
Dear community My aim is to import multiple .tif raster files using the help here: https://stackoverflow.com/questions/52746936/how-to-efficiently-import-multi ple-raster-tif-files-into-r Does anyone now about the error "subscripts out of bounds"? I am not sure about the check with alllrasters: is list() ok or should there be a number (the number of imported .tif files? Kind regards Sibylle > library(terra) > library(raster) > getwd() [1] "C:/Users/Sibylle Stöckli/Desktop" > > #first import all files in a single folder as a list > rastlist <- list.files(path = "/NCCS_Impacts_Lot2_2022/InVEST/Species_Input/bee_presence", pattern='.tif$', all.files= T, full.names= T) > > #import all raster files in folder using lapply > allrasters <- lapply(rastlist, raster) > > #to check the index numbers of all imported raster list elements > allrasters list() > > #call single raster element > allrasters[[1]] Error in allrasters[[1]] : subscript out of bounds __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] missing() fails to detect missing
I'm not really sure how verboten it is to have methods that omit "...". The
code seems to be instrumented to handle that case.
The catch is that it does so by inserting a call to .local where .local is the
method function, e.g. if you insert a browser() call at the start of the
method, you'll see
> foo("a")
Called from: .local(x, y, ...)
Browse[1]> sys.status()
$sys.calls
$sys.calls[[1]]
foo("a")
$sys.calls[[2]]
foo("a")
$sys.calls[[3]]
.local(x, y, ...)
$sys.calls[[4]]
browser()
and the 3rd call is absent for foo(0). But y is not missing in the .local call,
because, hey, it has a default in the parent.
This also happens in plain function calls, e.g.
> f <- function(a=0)(function(a)missing(a))(a)
> f()
[1] FALSE
Also notice that it is not really "..." that is the trigger of this effect, but
the fact that the method has different arguments from the generic in any way:
You get the same situation with
> setMethod("foo", signature="numeric", function(x, y=0, z=1, ...){
+ if (missing(y))
+return("must give y for numeric")
+ y
+ })
> foo(0)
[1] 0
-pd
> On 18 Apr 2024, at 06:32 , Jeff Newmiller via R-help
> wrote:
>
> Fascinating. Or, well, not.
>
> Failing to use a method signature that is compatible with the generic is a
> no-no. So your bug seems to me to be outside the bounds of how R is supposed
> to be used. So don't do that.
>
> On April 17, 2024 4:25:38 PM PDT, "Boylan, Ross via R-help"
> wrote:
>> When a generic (S4) has an argument with a default followed by ...,
>> missing() doesn't seem to work if the method omits the ...
>> Sample---
>> foo <- function(x, y=0, ...){
>> "you are very generic"
>> }
>>
>> # no ... in function arguments
>> setMethod("foo", signature="character", function(x, y=0){
>> if (missing(y))
>> return("must give y for character")
>> y
>> })
>>
>> setMethod("foo", signature="numeric", function(x, y=0, ...){
>> if (missing(y))
>> return("must give y for numeric")
>> y
>> })
>>
>> print(foo("a")) #[1] 0
>> print(foo(0))#[1] "must give y for numeric"
>> --
>> It's the result for foo("a") I'm puzzled by, since missing(y) does not
>> evaluate to TRUE.
>>
>> Background
>> ==
>> The methods documentation has 2 points on which the above definitions may
>> fail.
>>
>> 1. The generic has regular arguments and ... arguments. But dotsMethods
>> docs say
>>> either the signature of the generic function is "..." only, or it
>>> does not contain "..."
>> Since the arguments in ... are not part of the signatures I think I'm OK,
>> but another reading is that
>> one just shouldn't mix ... and other arguments.
>>
>> 2. setMethod docs say
>>> The definition must be a function with the same formal arguments as the
>>> generic; however, setMethod() will handle methods that add arguments, if
>>> ... is a formal argument to the generic.
>> Since the initial definition has arguments x, y, ... and the first method
>> definition has only x, y, the arguments don't match. So maybe that's the
>> problem.
>>
>> I don't know if the fact that y has a default value matters.
>>
>> The real code has a function f that ordinarily requires an additional piece
>> of information, y, to compute a result. But for one class, the result
>> doesn't depend on y and so that argument may be omitted.
>>
>> Context
>> ==
>> R 4.3.3 on MS-Windows under RStudio 2023.12.1 build 402.
>>
>> Thanks for any insights.
>> Ross
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd@cbs.dk Priv: pda...@gmail.com
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
