Re: [Rd] Excluding objects from save.image
On Thu, May 22, 2014 at 10:47 AM, Simon Urbanek wrote:
> On May 22, 2014, at 11:33 AM, Tim Keitt wrote:
>
> > On Thu, May 22, 2014 at 9:19 AM, Simon Urbanek <
> simon.urba...@r-project.org> wrote:
> > Tim,
> >
> > On May 21, 2014, at 9:54 PM, Tim Keitt wrote:
> >
> > > When dealing with object holding pointers that are only valid during a
> single session, it would be convenient to exclude them from being saved
> with the session image. (I am assuming that `quit` calls `save.image`
> before quitting or is it internal?) Currently they are saved with the
> pointer converted to NULL. Its a bit surprising for users not realizing the
> object is no longer valid.
> > >
> >
> > than's not surprising - it's actually very useful, because it tells you
> that the object has been restored. Typically, packages using external
> pointers use this to re-inilitialize the object from serialized data or
> other sources if possible.
> >
> > Automatically or with user intervention?
>
>
> Automatically - that is the point :).
>
Have an example? Do you save a promise?
>
>
> > > My thought is to designate a class name (perhaps "no.save") and
> exclude anything that inherits from that.
> >
> > Why would you want to not save it? It exists, so saving it makes it
> possible to decide on the behavior as needed - as opposed to not saving it
> and thus losing the information.
> >
> > That's a valid argument. I don't necessarily disagree, but it does
> depend on the situation and the author's intentions.
> >
> > My question was not about enforcing policy on authors, rather it was
> about whether this would add value for those that would like to use it for
> cases where they deem it a benefit. I see those as separate issues.
> >
> > Incidentally, I suggested many years ago hooks in load to make
> reinitialization possible. Is there some infrastructure for that in R? (And
> actually hooks on save would be quite useful as well eg one could
> substitute an object designed to reinitialize the original object on load.)
> >
>
> Load time is typically not an issue, since you can just do it lazily.
> (Incidentally, doing that eagerly at load time is the hard part since you'd
> have to load the packages necessary in the right order which is not always
> possible to determine from the serialization alone. It's a full can of
> worms, because R doesn't know anything about sharing and dependencies on
> the native side etc.).
>
> However, having a proper save hook for ext ptrs would be very useful. When
> I was asking for it few years ago, Luke mentioned there were issues - maybe
> it's something that could be re-visited.
>
I'd be interested. I can see it might be tricky deciding when to call the
hook.
THK
>
> Cheers,
> Simon
>
>
>
> > Cheers,
> > THK
> >
> >
> > Cheers,
> > Simon
> >
> >
> >
> > > I've hacked a quick implementation below.
> > > I did not issue a warning, but it would be a simple fix.
> > >
> > > Does anyone else see this as useful?
> > >
> > > THK
> > >
> > > --
> > > http://www.keittlab.org/
> > >
> > > .new.save.image = function (file = ".RData", version = NULL, ascii =
> FALSE,
> > > compress = !ascii,
> > >safe = TRUE, omit.no.save = TRUE) # added
> > > "omit.no.save" argument
> > > {
> > > if (!is.character(file) || file == "")
> > >stop("'file' must be non-empty string")
> > > opts <- getOption("save.image.defaults")
> > > if (is.null(opts))
> > >opts <- getOption("save.defaults")
> > > if (missing(safe) && !is.null(opts$safe))
> > >safe <- opts$safe
> > > if (missing(ascii) && !is.null(opts$ascii))
> > >ascii <- opts$ascii
> > > if (missing(compress) && !is.null(opts$compress))
> > >compress <- opts$compress
> > > if (missing(version))
> > >version <- opts$version
> > > if (safe) {
> > >outfile <- paste0(file, "Tmp")
> > >i <- 0
> > >while (file.exists(outfile)) {
> > > i <- i + 1
> > > outfile <- paste0(file, "Tmp", i)
> > >}
> > > }
> > > else outfile <- file
> > > on.exit(file.remove(outfile))
> > >
> > > ### Omit objects inheriting from "no.save" ###
> > >
> > > objns <- ls(envir = .GlobalEnv, all.names = TRUE)
> > >
> > > if ( omit.no.save ) ### make part of options?
> > >objns <- subset(objns, sapply(objns, function(x)
> > >{
> > > ! inherits(get(x, envir = .GlobalEnv), "no.save")
> > >}))
> > >
> > > save(list = objns, file = outfile,
> > > version = version, ascii = ascii, compress = compress,
> > > envir = .GlobalEnv, precheck = FALSE)
> > >
> > > ### End code changes ###
> > >
> > > if (safe)
> > >if (!file.rename(outfile, file)) {
> > > on.exit()
> > > stop(gettextf("image could not be renamed and is left in %s",
> > >outfile), domain = NA)
> > >}
> > > on.exit()
> > > }
> > >
> > > setClass("no.save")
> > > setClass("test.class", slots = c(s1 = "externalptr"))
> > > setIs("test.class", "no.save")
> > > reg.obj = "regular object"
> > > no.save
Re: [Rd] Excluding objects from save.image
On Thu, May 22, 2014 at 9:19 AM, Simon Urbanek
wrote:
> Tim,
>
> On May 21, 2014, at 9:54 PM, Tim Keitt wrote:
>
> > When dealing with object holding pointers that are only valid during a
> single session, it would be convenient to exclude them from being saved
> with the session image. (I am assuming that `quit` calls `save.image`
> before quitting or is it internal?) Currently they are saved with the
> pointer converted to NULL. Its a bit surprising for users not realizing the
> object is no longer valid.
> >
>
> than's not surprising - it's actually very useful, because it tells you
> that the object has been restored. Typically, packages using external
> pointers use this to re-inilitialize the object from serialized data or
> other sources if possible.
>
Automatically or with user intervention?
>
>
> > My thought is to designate a class name (perhaps "no.save") and exclude
> anything that inherits from that.
>
> Why would you want to not save it? It exists, so saving it makes it
> possible to decide on the behavior as needed - as opposed to not saving it
> and thus losing the information.
>
That's a valid argument. I don't necessarily disagree, but it does depend
on the situation and the author's intentions.
My question was not about enforcing policy on authors, rather it was about
whether this would add value for those that would like to use it for cases
where they deem it a benefit. I see those as separate issues.
Incidentally, I suggested many years ago hooks in load to make
reinitialization possible. Is there some infrastructure for that in R? (And
actually hooks on save would be quite useful as well eg one could
substitute an object designed to reinitialize the original object on load.)
Cheers,
THK
>
> Cheers,
> Simon
>
>
>
> > I've hacked a quick implementation below.
> > I did not issue a warning, but it would be a simple fix.
> >
> > Does anyone else see this as useful?
> >
> > THK
> >
> > --
> > http://www.keittlab.org/
> >
> > .new.save.image = function (file = ".RData", version = NULL, ascii =
> FALSE,
> > compress = !ascii,
> >safe = TRUE, omit.no.save = TRUE) # added
> > "omit.no.save" argument
> > {
> > if (!is.character(file) || file == "")
> >stop("'file' must be non-empty string")
> > opts <- getOption("save.image.defaults")
> > if (is.null(opts))
> >opts <- getOption("save.defaults")
> > if (missing(safe) && !is.null(opts$safe))
> >safe <- opts$safe
> > if (missing(ascii) && !is.null(opts$ascii))
> >ascii <- opts$ascii
> > if (missing(compress) && !is.null(opts$compress))
> >compress <- opts$compress
> > if (missing(version))
> >version <- opts$version
> > if (safe) {
> >outfile <- paste0(file, "Tmp")
> >i <- 0
> >while (file.exists(outfile)) {
> > i <- i + 1
> > outfile <- paste0(file, "Tmp", i)
> >}
> > }
> > else outfile <- file
> > on.exit(file.remove(outfile))
> >
> > ### Omit objects inheriting from "no.save" ###
> >
> > objns <- ls(envir = .GlobalEnv, all.names = TRUE)
> >
> > if ( omit.no.save ) ### make part of options?
> >objns <- subset(objns, sapply(objns, function(x)
> >{
> > ! inherits(get(x, envir = .GlobalEnv), "no.save")
> >}))
> >
> > save(list = objns, file = outfile,
> > version = version, ascii = ascii, compress = compress,
> > envir = .GlobalEnv, precheck = FALSE)
> >
> > ### End code changes ###
> >
> > if (safe)
> >if (!file.rename(outfile, file)) {
> > on.exit()
> > stop(gettextf("image could not be renamed and is left in %s",
> >outfile), domain = NA)
> >}
> > on.exit()
> > }
> >
> > setClass("no.save")
> > setClass("test.class", slots = c(s1 = "externalptr"))
> > setIs("test.class", "no.save")
> > reg.obj = "regular object"
> > no.save.obj = new("test.class")
> > .image.file = tempfile()
> > save.image(.image.file)
> > rm(reg.obj, no.save.obj)
> > load(.image.file)
> > print(ls())
> > unlink(.image.file)
> > .new.save.image(.image.file)
> > rm(reg.obj, no.save.obj)
> > load(.image.file)
> > print(ls())
> > unlink(.image.file)
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
>
--
http://www.keittlab.org/
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Excluding objects from save.image
On May 22, 2014, at 11:33 AM, Tim Keitt wrote:
> On Thu, May 22, 2014 at 9:19 AM, Simon Urbanek
> wrote:
> Tim,
>
> On May 21, 2014, at 9:54 PM, Tim Keitt wrote:
>
> > When dealing with object holding pointers that are only valid during a
> > single session, it would be convenient to exclude them from being saved
> > with the session image. (I am assuming that `quit` calls `save.image`
> > before quitting or is it internal?) Currently they are saved with the
> > pointer converted to NULL. Its a bit surprising for users not realizing the
> > object is no longer valid.
> >
>
> than's not surprising - it's actually very useful, because it tells you that
> the object has been restored. Typically, packages using external pointers use
> this to re-inilitialize the object from serialized data or other sources if
> possible.
>
> Automatically or with user intervention?
Automatically - that is the point :).
> > My thought is to designate a class name (perhaps "no.save") and exclude
> > anything that inherits from that.
>
> Why would you want to not save it? It exists, so saving it makes it possible
> to decide on the behavior as needed - as opposed to not saving it and thus
> losing the information.
>
> That's a valid argument. I don't necessarily disagree, but it does depend on
> the situation and the author's intentions.
>
> My question was not about enforcing policy on authors, rather it was about
> whether this would add value for those that would like to use it for cases
> where they deem it a benefit. I see those as separate issues.
>
> Incidentally, I suggested many years ago hooks in load to make
> reinitialization possible. Is there some infrastructure for that in R? (And
> actually hooks on save would be quite useful as well eg one could substitute
> an object designed to reinitialize the original object on load.)
>
Load time is typically not an issue, since you can just do it lazily.
(Incidentally, doing that eagerly at load time is the hard part since you'd
have to load the packages necessary in the right order which is not always
possible to determine from the serialization alone. It's a full can of worms,
because R doesn't know anything about sharing and dependencies on the native
side etc.).
However, having a proper save hook for ext ptrs would be very useful. When I
was asking for it few years ago, Luke mentioned there were issues - maybe it's
something that could be re-visited.
Cheers,
Simon
> Cheers,
> THK
>
>
> Cheers,
> Simon
>
>
>
> > I've hacked a quick implementation below.
> > I did not issue a warning, but it would be a simple fix.
> >
> > Does anyone else see this as useful?
> >
> > THK
> >
> > --
> > http://www.keittlab.org/
> >
> > .new.save.image = function (file = ".RData", version = NULL, ascii = FALSE,
> > compress = !ascii,
> >safe = TRUE, omit.no.save = TRUE) # added
> > "omit.no.save" argument
> > {
> > if (!is.character(file) || file == "")
> >stop("'file' must be non-empty string")
> > opts <- getOption("save.image.defaults")
> > if (is.null(opts))
> >opts <- getOption("save.defaults")
> > if (missing(safe) && !is.null(opts$safe))
> >safe <- opts$safe
> > if (missing(ascii) && !is.null(opts$ascii))
> >ascii <- opts$ascii
> > if (missing(compress) && !is.null(opts$compress))
> >compress <- opts$compress
> > if (missing(version))
> >version <- opts$version
> > if (safe) {
> >outfile <- paste0(file, "Tmp")
> >i <- 0
> >while (file.exists(outfile)) {
> > i <- i + 1
> > outfile <- paste0(file, "Tmp", i)
> >}
> > }
> > else outfile <- file
> > on.exit(file.remove(outfile))
> >
> > ### Omit objects inheriting from "no.save" ###
> >
> > objns <- ls(envir = .GlobalEnv, all.names = TRUE)
> >
> > if ( omit.no.save ) ### make part of options?
> >objns <- subset(objns, sapply(objns, function(x)
> >{
> > ! inherits(get(x, envir = .GlobalEnv), "no.save")
> >}))
> >
> > save(list = objns, file = outfile,
> > version = version, ascii = ascii, compress = compress,
> > envir = .GlobalEnv, precheck = FALSE)
> >
> > ### End code changes ###
> >
> > if (safe)
> >if (!file.rename(outfile, file)) {
> > on.exit()
> > stop(gettextf("image could not be renamed and is left in %s",
> >outfile), domain = NA)
> >}
> > on.exit()
> > }
> >
> > setClass("no.save")
> > setClass("test.class", slots = c(s1 = "externalptr"))
> > setIs("test.class", "no.save")
> > reg.obj = "regular object"
> > no.save.obj = new("test.class")
> > .image.file = tempfile()
> > save.image(.image.file)
> > rm(reg.obj, no.save.obj)
> > load(.image.file)
> > print(ls())
> > unlink(.image.file)
> > .new.save.image(.image.file)
> > rm(reg.obj, no.save.obj)
> > load(.image.file)
> > print(ls())
> > unlink(.image.file)
> >
> > [[alternative HTML version deleted]]
> >
> > __
> >
Re: [Rd] Excluding objects from save.image
Tim,
On May 21, 2014, at 9:54 PM, Tim Keitt wrote:
> When dealing with object holding pointers that are only valid during a single
> session, it would be convenient to exclude them from being saved with the
> session image. (I am assuming that `quit` calls `save.image` before quitting
> or is it internal?) Currently they are saved with the pointer converted to
> NULL. Its a bit surprising for users not realizing the object is no longer
> valid.
>
than's not surprising - it's actually very useful, because it tells you that
the object has been restored. Typically, packages using external pointers use
this to re-inilitialize the object from serialized data or other sources if
possible.
> My thought is to designate a class name (perhaps "no.save") and exclude
> anything that inherits from that.
Why would you want to not save it? It exists, so saving it makes it possible to
decide on the behavior as needed - as opposed to not saving it and thus losing
the information.
Cheers,
Simon
> I've hacked a quick implementation below.
> I did not issue a warning, but it would be a simple fix.
>
> Does anyone else see this as useful?
>
> THK
>
> --
> http://www.keittlab.org/
>
> .new.save.image = function (file = ".RData", version = NULL, ascii = FALSE,
> compress = !ascii,
>safe = TRUE, omit.no.save = TRUE) # added
> "omit.no.save" argument
> {
> if (!is.character(file) || file == "")
>stop("'file' must be non-empty string")
> opts <- getOption("save.image.defaults")
> if (is.null(opts))
>opts <- getOption("save.defaults")
> if (missing(safe) && !is.null(opts$safe))
>safe <- opts$safe
> if (missing(ascii) && !is.null(opts$ascii))
>ascii <- opts$ascii
> if (missing(compress) && !is.null(opts$compress))
>compress <- opts$compress
> if (missing(version))
>version <- opts$version
> if (safe) {
>outfile <- paste0(file, "Tmp")
>i <- 0
>while (file.exists(outfile)) {
> i <- i + 1
> outfile <- paste0(file, "Tmp", i)
>}
> }
> else outfile <- file
> on.exit(file.remove(outfile))
>
> ### Omit objects inheriting from "no.save" ###
>
> objns <- ls(envir = .GlobalEnv, all.names = TRUE)
>
> if ( omit.no.save ) ### make part of options?
>objns <- subset(objns, sapply(objns, function(x)
>{
> ! inherits(get(x, envir = .GlobalEnv), "no.save")
>}))
>
> save(list = objns, file = outfile,
> version = version, ascii = ascii, compress = compress,
> envir = .GlobalEnv, precheck = FALSE)
>
> ### End code changes ###
>
> if (safe)
>if (!file.rename(outfile, file)) {
> on.exit()
> stop(gettextf("image could not be renamed and is left in %s",
>outfile), domain = NA)
>}
> on.exit()
> }
>
> setClass("no.save")
> setClass("test.class", slots = c(s1 = "externalptr"))
> setIs("test.class", "no.save")
> reg.obj = "regular object"
> no.save.obj = new("test.class")
> .image.file = tempfile()
> save.image(.image.file)
> rm(reg.obj, no.save.obj)
> load(.image.file)
> print(ls())
> unlink(.image.file)
> .new.save.image(.image.file)
> rm(reg.obj, no.save.obj)
> load(.image.file)
> print(ls())
> unlink(.image.file)
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Excluding objects from save.image
When dealing with object holding pointers that are only valid during a
single session, it would be convenient to exclude them from being saved
with the session image. (I am assuming that `quit` calls `save.image`
before quitting or is it internal?) Currently they are saved with the
pointer converted to NULL. Its a bit surprising for users not realizing the
object is no longer valid.
My thought is to designate a class name (perhaps "no.save") and exclude
anything that inherits from that. I've hacked a quick implementation below.
I did not issue a warning, but it would be a simple fix.
Does anyone else see this as useful?
THK
--
http://www.keittlab.org/
.new.save.image = function (file = ".RData", version = NULL, ascii = FALSE,
compress = !ascii,
safe = TRUE, omit.no.save = TRUE) # added
"omit.no.save" argument
{
if (!is.character(file) || file == "")
stop("'file' must be non-empty string")
opts <- getOption("save.image.defaults")
if (is.null(opts))
opts <- getOption("save.defaults")
if (missing(safe) && !is.null(opts$safe))
safe <- opts$safe
if (missing(ascii) && !is.null(opts$ascii))
ascii <- opts$ascii
if (missing(compress) && !is.null(opts$compress))
compress <- opts$compress
if (missing(version))
version <- opts$version
if (safe) {
outfile <- paste0(file, "Tmp")
i <- 0
while (file.exists(outfile)) {
i <- i + 1
outfile <- paste0(file, "Tmp", i)
}
}
else outfile <- file
on.exit(file.remove(outfile))
### Omit objects inheriting from "no.save" ###
objns <- ls(envir = .GlobalEnv, all.names = TRUE)
if ( omit.no.save ) ### make part of options?
objns <- subset(objns, sapply(objns, function(x)
{
! inherits(get(x, envir = .GlobalEnv), "no.save")
}))
save(list = objns, file = outfile,
version = version, ascii = ascii, compress = compress,
envir = .GlobalEnv, precheck = FALSE)
### End code changes ###
if (safe)
if (!file.rename(outfile, file)) {
on.exit()
stop(gettextf("image could not be renamed and is left in %s",
outfile), domain = NA)
}
on.exit()
}
setClass("no.save")
setClass("test.class", slots = c(s1 = "externalptr"))
setIs("test.class", "no.save")
reg.obj = "regular object"
no.save.obj = new("test.class")
.image.file = tempfile()
save.image(.image.file)
rm(reg.obj, no.save.obj)
load(.image.file)
print(ls())
unlink(.image.file)
.new.save.image(.image.file)
rm(reg.obj, no.save.obj)
load(.image.file)
print(ls())
unlink(.image.file)
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
