Re: [R] .Rdata not loading
On Fri, 24 Dec 2021, Jeff Newmiller wrote:
The qsave/qread functions from the qs package are functionally
interchangeable with saveRDS/readRDS, but faster and create smaller files.
A simple
if ( file.exists( "obj1.qs" ) ) {
obj1 <- qread( "onj1.qs" )
} else {
obj1 <- compute_obj1()
qsave( obj1, "obj1.qs" )
}
can be used
Jeff,
I understand that with qs each R object needs to be in a separate .qs file
rather than having all of them in a single file such as .RData or .rda.
... but there are various caching packages [1] for ad-hoc use that
attempt to determine when a computation is out of date better than the
existence of a file (that you have to manually delete if you want to
recompute) but they may either fail to notice relevant changes or
recompute unnecessarily.
I'll look closely at caching-in-R.
I have in the last couple of months found that the targets package [2]
seems to excel at balancing these concerns and regularly use it now for
large or complicated analyses.
I read the targets users manual and vignettes and found that it's much too
complex for my simple needs. I consider my projects to be subjectively
small and uncomplicated. Each of my projects are unique and one-off.
Thanks and happy holiday,
Rich
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
On Fri, 24 Dec 2021, Rui Barradas wrote: Can section Adoption of this Library of Congress link https://www.loc.gov/preservation/digital/formats/fdd/fdd000470.shtml help? I'm really not sure. Rui, After reading that interesting page I don't see how it helps me. I think that re-running all my scripts, in order, is needed. As this is the first time I've started R and not have the saved workspace automatically loaded I'll keep saving the workspace prior to exiting. Should this happen again I'll change to using an alternative, probably quick sequense (qs). Having all scripts makes re-creating the workspace a small time investment. Happy holiday to you and yours, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
The qsave/qread functions from the qs package are functionally interchangeable
with saveRDS/readRDS, but faster and create smaller files. A simple
if ( file.exists( "obj1.qs" ) ) {
obj1 <- qread( "onj1.qs" )
} else {
obj1 <- compute_obj1()
qsave( obj1, "obj1.qs" )
}
can be used but there are various caching packages [1] for ad-hoc use that
attempt to determine when a computation is out of date better than the
existence of a file (that you have to manually delete if you want to recompute)
but they may either fail to notice relevant changes or recompute unnecessarily.
I have in the last couple of months found that the targets package [2] seems
to excel at balancing these concerns and regularly use it now for large or
complicated analyses. You can build the _targets.R file by hand or use a
targets markdown file to build it for you and let you document your process as
you go. Once all objects are computed and in the cache then you can build
reporting scripts that generate plots and tabular output that retrieve the data
as needed.
[1] https://joshuacook.netlify.app/post/caching-in-r/
[2] https://books.ropensci.org/targets/
On December 24, 2021 8:31:21 AM PST, Rich Shepard
wrote:
>On Fri, 24 Dec 2021, Adrian Dușa wrote:
>
>> Package admisc has a function called obj.rda(), which returns the names of
>> the objects from an .Rdata file. Not sure how it handles corrupt .Rdata
>> files, but should generally give an idea about what's inside.
>
>Adrian,
>
>Thank you. I know what dataframes and plots should be in there. I'll
>probably end up running all scripts again, in sequence, and reconstructing
>what's lost. And, I'm experimenting with qs (quick save) to understand what
>it does and how to use it.
>
>Regards,
>
>Rich
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
--
Sent from my phone. Please excuse my brevity.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
Hello, Can section Adoption of this Library of Congress link https://www.loc.gov/preservation/digital/formats/fdd/fdd000470.shtml help? I'm really not sure. Happy Holidays, Rui Barradas Às 16:31 de 24/12/21, Rich Shepard escreveu: On Fri, 24 Dec 2021, Adrian Dușa wrote: Package admisc has a function called obj.rda(), which returns the names of the objects from an .Rdata file. Not sure how it handles corrupt .Rdata files, but should generally give an idea about what's inside. Adrian, Thank you. I know what dataframes and plots should be in there. I'll probably end up running all scripts again, in sequence, and reconstructing what's lost. And, I'm experimenting with qs (quick save) to understand what it does and how to use it. Regards, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
On Fri, 24 Dec 2021, Adrian Dușa wrote: Package admisc has a function called obj.rda(), which returns the names of the objects from an .Rdata file. Not sure how it handles corrupt .Rdata files, but should generally give an idea about what's inside. Adrian, Thank you. I know what dataframes and plots should be in there. I'll probably end up running all scripts again, in sequence, and reconstructing what's lost. And, I'm experimenting with qs (quick save) to understand what it does and how to use it. Regards, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
On Fri, 24 Dec 2021 at 17:35, Rich Shepard wrote:
> On Fri, 24 Dec 2021, Rasmus Liland wrote:
>
> > If you want to look at Rdata-files in a quick way in the
> > terminal, use this little gem in your .zshrc.local:
> >
> > readrdata() { Rscript -e "options(width=$COLUMNS); load('$1');
> sapply(ls(), get, simplify=F)" | less }
>
> Rasmus,
>
> I use bash, not zsh. And running the readdata() command in R produces no
> output.
>
Package admisc has a function called obj.rda(), which returns the names of
the objects from an .Rdata file.
Not sure how it handles corrupt .Rdata files, but should generally give an
idea about what's inside.
Hth,
Adrian
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
On Fri, 24 Dec 2021, Rasmus Liland wrote:
If you want to look at Rdata-files in a quick way in the
terminal, use this little gem in your .zshrc.local:
readrdata() { Rscript -e "options(width=$COLUMNS); load('$1'); sapply(ls(), get,
simplify=F)" | less }
Rasmus,
I use bash, not zsh. And running the readdata() command in R produces no
output.
Thanks for the thought,
Rich
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
Dear Rich,
If you want to look at Rdata-files in a quick way in the
terminal, use this little gem in your .zshrc.local:
readrdata() { Rscript -e "options(width=$COLUMNS); load('$1'); sapply(ls(),
get, simplify=F)" | less }
Merry Christmas!
Best,
Rasmus
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
On Thu, 23 Dec 2021, Bill Dunlap wrote:
Three things you might try using R (and show the results in this email
thread):
Bill,
* load(verbose=TRUE, ".RData") # see how far it gets before stopping
load(verbose=TRUE, ".RData")
Loading objects:
all_turb_plot
disc_all
Error in load(verbose = TRUE, ".RData") :
ReadItem: unknown type 0, perhaps written by later version of R
* file.info(normalizePath(".RData")) # is this the file you think it is?
file.info(normalizePath(".RData"))
size
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
28074080
isdir
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
FALSE
mode
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
644
mtime
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
2021-12-22 15:32:59
ctime
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
2021-12-22 15:32:59
atime
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
2021-12-24 06:23:33
uid
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
1000
gid
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
100
uname
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
rshepard
grname
/home/rshepard/documents/monographs/regulatory-science/npdes-wqbel/analyses/.RData
users
* dput(readBin(".RData", what=raw(), n=100))
dput(readBin(".RData", what=raw(), n=100))
as.raw(c(0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x03, 0xec, 0xfd, 0x09, 0x9c, 0x24, 0x47, 0x79, 0xe7, 0x8d, 0x67,
0x9f, 0x73, 0x68, 0xa4, 0x19, 0x8d, 0xa4, 0xd1, 0x81, 0x84, 0x5a,
0x07, 0x58, 0x02, 0xd4, 0x74, 0xf7, 0xdc, 0x18, 0xd4, 0xc3, 0x21,
0x61, 0xec, 0xb1, 0x60, 0x41, 0xe2, 0xb2, 0xf0, 0xb8, 0x7a, 0xba,
0xa7, 0xd5, 0xa8, 0x2f, 0x77, 0xf7, 0x68, 0xa6, 0x05, 0x86, 0x59,
0xc0, 0x60, 0x03, 0xb6, 0x58, 0x63, 0x9b, 0xe3, 0xc5, 0x17, 0x5e,
0x1f, 0x8b, 0x4f, 0xfe, 0xde, 0x77, 0x7d, 0xbc, 0x3e, 0x64, 0xf0,
0x1a, 0x1b, 0x6c, 0x2f, 0xa7, 0x0d, 0x98, 0x43, 0x20, 0x83, 0x39,
0x0c, 0x6e, 0x63))
And a fourth thing to do:
* dput(tail(n=20, readBin(".RData", what=raw(), n=file.size(".RData"
dput(tail(n=20, readBin(".RData", what=raw(), n=file.size(".RData"
as.raw(c(0x5c, 0xae, 0x97, 0x5f, 0x1d, 0x9d, 0x2d, 0x1f, 0xad,
0xa8, 0xf5, 0x5f, 0x8f, 0x47, 0xdd, 0x56, 0x60, 0xd5, 0x70, 0x3c
))
I hope this explains why .RData contains only two files when there were
30 or more when I saved it.
Thanks,
Rich
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
And a fourth thing to do:
* dput(tail(n=20, readBin(".RData", what=raw(), n=file.size(".RData"
This can show if the file got truncated.
-Bill
On Thu, Dec 23, 2021 at 5:25 PM Bill Dunlap
wrote:
> Three things you might try using R (and show the results in this email
> thread):
>
> * load(verbose=TRUE, ".RData") # see how far it gets before stopping
> * file.info(normalizePath(".RData")) # is this the file you think it is?
> * dput(readBin(".RData", what=raw(), n=100))
>
> The last will print some hex numbers that others may be able to
> interpret. E.g., it may show that this .RData is not from a call to save()
> or save.image().
>
> -Bill
>
> On Thu, Dec 23, 2021 at 11:19 AM Rich Shepard
> wrote:
>
>> Each time I finish with a session I save the image. Today the saved image
>> did not load and manually running 'load('.RData') fails:
>> > load('.RData')
>> Error in load(".RData") :
>>ReadItem: unknown type 0, perhaps written by later version of R
>> >
>>
>> This has not happened before.
>>
>> Installed is R-4.1.2-x86_64-1_SBo and in the PWD I see:
>> $ ll .RData*
>> -rw-r--r-- 1 rshepard users 28074080 Dec 22 15:32 .RData
>> -rw-r--r-- 1 rshepard users 134 Nov 4 14:57 .RDataTmp
>> -rw-r--r-- 1 rshepard users0 Nov 4 14:57 .RDataTmp1
>>
>> What might be keeping .RData from loading? Is there a maximum size for
>> .RData? The /home partition has 270G available space so that's not
>> limiting.
>>
>> How do I load it?
>>
>> TIA,
>>
>> Rich
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
Three things you might try using R (and show the results in this email
thread):
* load(verbose=TRUE, ".RData") # see how far it gets before stopping
* file.info(normalizePath(".RData")) # is this the file you think it is?
* dput(readBin(".RData", what=raw(), n=100))
The last will print some hex numbers that others may be able to interpret.
E.g., it may show that this .RData is not from a call to save() or
save.image().
-Bill
On Thu, Dec 23, 2021 at 11:19 AM Rich Shepard
wrote:
> Each time I finish with a session I save the image. Today the saved image
> did not load and manually running 'load('.RData') fails:
> > load('.RData')
> Error in load(".RData") :
>ReadItem: unknown type 0, perhaps written by later version of R
> >
>
> This has not happened before.
>
> Installed is R-4.1.2-x86_64-1_SBo and in the PWD I see:
> $ ll .RData*
> -rw-r--r-- 1 rshepard users 28074080 Dec 22 15:32 .RData
> -rw-r--r-- 1 rshepard users 134 Nov 4 14:57 .RDataTmp
> -rw-r--r-- 1 rshepard users0 Nov 4 14:57 .RDataTmp1
>
> What might be keeping .RData from loading? Is there a maximum size for
> .RData? The /home partition has 270G available space so that's not
> limiting.
>
> How do I load it?
>
> TIA,
>
> Rich
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
On Thu, 23 Dec 2021, Jeff Newmiller wrote: This practice (saving and resuming from Rdata files) often ends badly this way. Objects that are "shared" in memory get saved as separate data and may not "fit" when re-loaded. This is why re-running from scratch should always be part of your workflow. There are caching approaches that track individual objects rather than entire workspaces such as the targets package or others or you can roll your own with saveRDS/readRDS or the qs package. Jeff, While I've used R for more than a decade I've not before had this experience. I'll set up a script to run all existing scripts, in order, and run it to be able to continue my project. I'll definitely look at the targets and qs packages as well as save-read/RDS. Thanks for pointing me in the right direction and teaching me a valuable lesson. Have safe and healthy holidays and stay well, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] .Rdata not loading
This practice (saving and resuming from Rdata files) often ends badly this way.
Objects that are "shared" in memory get saved as separate data and may not
"fit" when re-loaded. This is why re-running from scratch should always be part
of your workflow. There are caching approaches that track individual objects
rather than entire workspaces such as the targets package or others or you can
roll your own with saveRDS/readRDS or the qs package.
On December 23, 2021 11:18:45 AM PST, Rich Shepard
wrote:
>Each time I finish with a session I save the image. Today the saved image
>did not load and manually running 'load('.RData') fails:
>> load('.RData')
>Error in load(".RData") :
> ReadItem: unknown type 0, perhaps written by later version of R
>>
>
>This has not happened before.
>
>Installed is R-4.1.2-x86_64-1_SBo and in the PWD I see:
>$ ll .RData*
>-rw-r--r-- 1 rshepard users 28074080 Dec 22 15:32 .RData
>-rw-r--r-- 1 rshepard users 134 Nov 4 14:57 .RDataTmp
>-rw-r--r-- 1 rshepard users0 Nov 4 14:57 .RDataTmp1
>
>What might be keeping .RData from loading? Is there a maximum size for
>.RData? The /home partition has 270G available space so that's not limiting.
>
>How do I load it?
>
>TIA,
>
>Rich
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
--
Sent from my phone. Please excuse my brevity.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
