[Bioc-devel] Issue with change in random sampling
Hello,
My package `clusterExperiment` has not changed but is hitting errors on the
devel branch. I’ve pinpointed it to the fact that a small dataset I am running
the tests on is randomly subsetted from a larger subset and is no longer
choosing the same observations. I have already in previous version corrected
the tests for the change in random number generation in R.4.0.x. I am wondering
if it is related to the changes in BiocParallel
(https://community-bioc.slack.com/archives/CEQ04GKEC/p1631903391030800?thread_ts=1631881095.027600=CEQ04GKEC
<https://community-bioc.slack.com/archives/CEQ04GKEC/p1631903391030800?thread_ts=1631881095.027600=CEQ04GKEC>).
It was unexpected for me that this would affect these results. My package
doesn’t use BiocParallel or depend on it. But it turns out the code in question
does make a call to BiocSingular to run a PCA, and BiocSingular does make calls
to BiocParallel. What is strange to me is that even if I don’t directly use the
results of runPCA, but simply make the call to runPCA before running the code
in question, the output of that code is changed. So this seems to me to
indicate that the sequence of random numbers is being globally affected by the
change, and not just internally to the results of calls to BiocParallel. I
didn’t realize this was the case from the above discussion — I thought it would
only affect output that directly relied on calls to BiocParallel — and I was
hoping someone could confirm that this is what is happening and/or give me
explicit way to check this is the source of my errors.
Here’s the basic setup. I have a setup file that sets up a lot of objects for
my tests (setup_create_objects.R). The relevant parts look something like this
(I’ve simplified it from what’s actually in the file so it more clearly shows
the progression):
data(simData)
suppressWarnings(RNGversion("3.5.0"))
set.seed(23)
… # bunch of code
clusterIds<- … # code that internally calls BiocSingular::runPCA
… # bunch of code
### sample 3 observations from each cluster:
whSamp<-unlist(tapply(1:ncol(simData),clusterIds,function(x){sample(x=x,size=3)}))
smSimData<-simData[1:20,whSamp]
This results in different values of clusterIds and thus different whSamp on the
release and the devel version.
The unexpected part was even if I add a line that manually overwrites
clusterIds to be the values of the vector `clusterIds` from the release version
(copied manually from running on a different computer that is not the devel
version) I don’t get the same result of whSamp (I still run the code for
`clusterIds`, so BiocSingular::runPCA is still being called). If, however, when
I manually feed the correct clusterIds on the devel version, I ALSO put in a
new call to `set.seed` in the line before calling whSamp then both the devel
and the release version give the same result, as I would expect. This makes me
think that that the random seed has been affected globally. Further, the second
entry of .Random.seed is not the same after running setup_create_objects.R on
the devel version as the new version.
Thanks,
Elizabeth Purdom
[[alternative HTML version deleted]]
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] Error when index duplicate rows in SummarizedExperiment -- is this a bug?
Hello,
I want to be able to index duplicate rows of an assay of a Summarized
Experiment (think bootstrapping), something like this:
assay(se[c(1,1,2,2),])
However this gives me an error when the assay contains a data.frame, rather
than a DataFrame
# > assay(se[c(1,1,2,2),]) #throws error
# Error in `.rowNamesDF<-`(x, value = value) :
# duplicate 'row.names' are not allowed
# In addition: Warning message:
# non-unique values when setting 'row.names': ‘A1’, ‘A2’
Here’s a simple example:
test <- data.frame(matrix(rnorm(100),ncol=5))
row.names(test) <- paste0("A",1:nrow(test))
se<-SummarizedExperiment(test)
I can pull duplicate rows of the original data.frame:
test[c(1,1,2,2),] # works
I can also index duplicate rows of the SummarizedExperiment
se[c(1,1,2,2),] #works
But I can’t then call `assay` on that object with the duplicated rows:
assay(se[c(1,1,2,2),]) #throws error
# > assay(se[c(1,1,2,2),])
# Error in `.rowNamesDF<-`(x, value = value) :
# duplicate 'row.names' are not allowed
# In addition: Warning message:
# non-unique values when setting 'row.names': ‘A1’, ‘A2’
Of course, I can do
assay(se)[c(1,1,2,2),]
because the underlying data.frame can be indexed that way, but then I am not
indexing the corresponding `rowData`, which is my goal in indexing `se`
directly, rather than the `assay`.
On the other hand, I don’t get this problem if the input object is a DataFrame
or matrix:
se<-SummarizedExperiment(DataFrame(test))
assay(se[c(1,1,2,2),]) #now it works
se<-SummarizedExperiment(data.matrix(test))
assay(se[c(1,1,2,2),]) #now it works
This seems like a bug, but I thought I’d check here. It seems, at a minimum,
unfortunate that you can call `se[c(1,1,2,2),]` but not
`assay(se[c(1,1,2,2),])`, especially given that the underlying `data.frame`
allows this call.
Thanks,
Elizabeth Purdom
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] How to extend nonstandardGenericFunction
No, sorry, it was also a problem with `Matrix` package. I was too quick in
checking it. I think upgrading my devtools and roxygen2 may have fixed it.
Thanks for helping me pinpoint where the problem was.
> On Oct 19, 2018, at 12:37 PM, Elizabeth Purdom
> wrote:
>
> Thanks Martin. And of course you’re right! The errors I was getting came from
> devtools functions— I didn’t think to check it outside of there and assumed
> it was a general problem with my package setup. But it's actually a problem
> in devtools.
>
> When I set up your example (thanks!) with Roxygen as follows:
>
> #' @import methods
> #' @importMethodsFrom phylobase nNodes
> .A = setClass("A", slots = c(x="numeric"))
>
> #' @export
> setMethod("nNodes", "A", function(x) "I'm an 'A’")
>
> Then I get the following errors with `document` and `load_all`
>
>> document()
> Updating PkgA documentation
> Loading PkgA
> Error in setMethod("nNodes", "A", function(x) "I'm an 'A'") :
> no existing definition for function ‘nNodes’
>
>> load_all()
> Loading PkgA
> Error in setMethod("nNodes", "A", function(x) "I'm an 'A'") :
> no existing definition for function ‘nNodes’
>
> But it does seem related to the `nonStandardGeneric`, since I don’t have this
> problem when I use the Matrix package instead, which creates
> `standardGeneric`:
>
> #' @import methods
> #' @importMethodsFrom Matrix isDiagonal
> .A = setClass("A", slots = c(x="numeric"))
>
> #' @export
> setMethod("isDiagonal", "A", function(object) "I'm an 'A'")
>
>> document()
> Updating PkgA documentation
> Loading PkgA
> Writing NAMESPACE
> Writing NAMESPACE
>
> I’ll submit the problem in devtools.
>
> I guess for the upcoming Bioconductor release I’ll switch the package to
> `Depends` to get my documentation and any final bugs, and at the last minute
> I’ll put it back to `Imports`.
>
> Thanks,
> Elizabeth
>
>> On Oct 19, 2018, at 12:03 PM, Martin Morgan wrote:
>>
>> That doesn't sound correct. I just created a simple package with
>>
>> DESCRIPTION
>>
>> Package: PkgA
>> Title: What the Package Does (one line, title case)
>> Version: 0.0.0.9000
>> Authors@R: person("First", "Last", email = "first.l...@example.com", role =
>> c("aut", "cre"))
>> Description: What the package does (one paragraph).
>> Depends: R (>= 3.5.1)
>> Imports: phylobase, methods
>> License: Artistic-2.0
>> Encoding: UTF-8
>> LazyData: true
>>
>> NAMESPACE
>>
>> import(methods)
>> importMethodsFrom(phylobase, "nNodes")
>> exportMethods("nNodes")
>>
>> and R/cls.R
>>
>> .A = setClass("A", slots = c(x="numeric"))
>> setMethod("nNodes", "A", function(x) "I'm an 'A'")
>>
>> This package builds and checks OK (complaining about documentation, but...)
>> and nNodes can be used.
>>
>> Do you have more information on the problem that you are seeing? Is it a
>> problem with roxygen2?
>>
>> Martin
>>
>>
>> On 10/19/18, 12:18 PM, "Bioc-devel on behalf of Elizabeth Purdom"
>>
>> wrote:
>>
>> Sorry, I copied the wrong code from phylobase (I copied the setMethod
>> code). Their setGeneric calls are in the following format:
>>
>> setGeneric("nNodes", function(x) {
>> standardGeneric("nNodes")
>> })
>>
>>
>>> On Oct 19, 2018, at 9:07 AM, Elizabeth Purdom
>>> wrote:
>>>
>>> Hello,
>>>
>>> In my package (`clusterExperiment`) I want to extend a S4 method from a
>>> package `phylobase` for my class. However, in the phylobase package, all of
>>> the `setGeneric` calls are in the form of
>>>
>>> setMethod("nTips", signature(x="phylo"),
>>> function(x) {
>>>Ntip(x)
>>> })
>>>
>>> Namely, they use braces in their function definitions, which I have just
>>> learned creates a `nonstandardGenericFunction` rather than a
>>> `standardGeneric`. Because of this I can’t just do `importMethodFrom` and
>>> then just extend the method. And if I try to create my own
>>> `standardGeneric` (e.g.
>>> https://stat.ethz.ch/R-manual/R-devel/library/methods/html/Methods_for_Nongenerics.html
>>>
&
Re: [Bioc-devel] How to extend nonstandardGenericFunction
Thanks Martin. And of course you’re right! The errors I was getting came from
devtools functions— I didn’t think to check it outside of there and assumed it
was a general problem with my package setup. But it's actually a problem in
devtools.
When I set up your example (thanks!) with Roxygen as follows:
#' @import methods
#' @importMethodsFrom phylobase nNodes
.A = setClass("A", slots = c(x="numeric"))
#' @export
setMethod("nNodes", "A", function(x) "I'm an 'A’")
Then I get the following errors with `document` and `load_all`
> document()
Updating PkgA documentation
Loading PkgA
Error in setMethod("nNodes", "A", function(x) "I'm an 'A'") :
no existing definition for function ‘nNodes’
> load_all()
Loading PkgA
Error in setMethod("nNodes", "A", function(x) "I'm an 'A'") :
no existing definition for function ‘nNodes’
But it does seem related to the `nonStandardGeneric`, since I don’t have this
problem when I use the Matrix package instead, which creates `standardGeneric`:
#' @import methods
#' @importMethodsFrom Matrix isDiagonal
.A = setClass("A", slots = c(x="numeric"))
#' @export
setMethod("isDiagonal", "A", function(object) "I'm an 'A'")
> document()
Updating PkgA documentation
Loading PkgA
Writing NAMESPACE
Writing NAMESPACE
I’ll submit the problem in devtools.
I guess for the upcoming Bioconductor release I’ll switch the package to
`Depends` to get my documentation and any final bugs, and at the last minute
I’ll put it back to `Imports`.
Thanks,
Elizabeth
> On Oct 19, 2018, at 12:03 PM, Martin Morgan wrote:
>
> That doesn't sound correct. I just created a simple package with
>
> DESCRIPTION
>
> Package: PkgA
> Title: What the Package Does (one line, title case)
> Version: 0.0.0.9000
> Authors@R: person("First", "Last", email = "first.l...@example.com", role =
> c("aut", "cre"))
> Description: What the package does (one paragraph).
> Depends: R (>= 3.5.1)
> Imports: phylobase, methods
> License: Artistic-2.0
> Encoding: UTF-8
> LazyData: true
>
> NAMESPACE
>
> import(methods)
> importMethodsFrom(phylobase, "nNodes")
> exportMethods("nNodes")
>
> and R/cls.R
>
> .A = setClass("A", slots = c(x="numeric"))
> setMethod("nNodes", "A", function(x) "I'm an 'A'")
>
> This package builds and checks OK (complaining about documentation, but...)
> and nNodes can be used.
>
> Do you have more information on the problem that you are seeing? Is it a
> problem with roxygen2?
>
> Martin
>
>
> On 10/19/18, 12:18 PM, "Bioc-devel on behalf of Elizabeth Purdom"
>
> wrote:
>
>Sorry, I copied the wrong code from phylobase (I copied the setMethod
> code). Their setGeneric calls are in the following format:
>
>setGeneric("nNodes", function(x) {
>standardGeneric("nNodes")
>})
>
>
>> On Oct 19, 2018, at 9:07 AM, Elizabeth Purdom
>> wrote:
>>
>> Hello,
>>
>> In my package (`clusterExperiment`) I want to extend a S4 method from a
>> package `phylobase` for my class. However, in the phylobase package, all of
>> the `setGeneric` calls are in the form of
>>
>> setMethod("nTips", signature(x="phylo"),
>> function(x) {
>> Ntip(x)
>> })
>>
>> Namely, they use braces in their function definitions, which I have just
>> learned creates a `nonstandardGenericFunction` rather than a
>> `standardGeneric`. Because of this I can’t just do `importMethodFrom` and
>> then just extend the method. And if I try to create my own `standardGeneric`
>> (e.g.
>> https://stat.ethz.ch/R-manual/R-devel/library/methods/html/Methods_for_Nongenerics.html
>>
>> <https://stat.ethz.ch/R-manual/R-devel/library/methods/html/Methods_for_Nongenerics.html>)
>> I can no longer call the function in the original `phylobase` package,
>> i.e. for the original class for which it is applied.
>>
>> My only solution appears to be if I put `phylobase` in the `Depends`
>> section, at which point I can then extend their generics. But this gives my
>> package the significant warning:
>>
>> " Warning: replacing previous import ‘phylobase::plot’ by ‘graphics::plot’
>> when loading ‘clusterExperiment’”
>>
>> My question is whether there is anything I can do on my end to either 1)
>> extend phylobase generics without a depends statement or 2) to get rid of
>> the warning about t
Re: [Bioc-devel] How to extend nonstandardGenericFunction
Sorry, I copied the wrong code from phylobase (I copied the setMethod code).
Their setGeneric calls are in the following format:
setGeneric("nNodes", function(x) {
standardGeneric("nNodes")
})
> On Oct 19, 2018, at 9:07 AM, Elizabeth Purdom
> wrote:
>
> Hello,
>
> In my package (`clusterExperiment`) I want to extend a S4 method from a
> package `phylobase` for my class. However, in the phylobase package, all of
> the `setGeneric` calls are in the form of
>
> setMethod("nTips", signature(x="phylo"),
> function(x) {
> Ntip(x)
> })
>
> Namely, they use braces in their function definitions, which I have just
> learned creates a `nonstandardGenericFunction` rather than a
> `standardGeneric`. Because of this I can’t just do `importMethodFrom` and
> then just extend the method. And if I try to create my own `standardGeneric`
> (e.g.
> https://stat.ethz.ch/R-manual/R-devel/library/methods/html/Methods_for_Nongenerics.html
>
> <https://stat.ethz.ch/R-manual/R-devel/library/methods/html/Methods_for_Nongenerics.html>)
> I can no longer call the function in the original `phylobase` package, i.e.
> for the original class for which it is applied.
>
> My only solution appears to be if I put `phylobase` in the `Depends` section,
> at which point I can then extend their generics. But this gives my package
> the significant warning:
>
> " Warning: replacing previous import ‘phylobase::plot’ by ‘graphics::plot’
> when loading ‘clusterExperiment’”
>
> My question is whether there is anything I can do on my end to either 1)
> extend phylobase generics without a depends statement or 2) to get rid of the
> warning about the imports (I have made sure to use `graphics::plot` in all of
> my functions’ calls to plot to make sure there is no problem with the import
> conflict)?
>
> Or is my only hope to have the developers of the package fix their calls?
> Personally, I was going to strip my setGeneric calls of braces after this
> experience, since I also have braces in my call but not for any good reason.
> Is there a good reason that setGenerics might have the braces in their calls?
> (Hadley’s Advanced R seems to say no, perhaps for this reason
> https://adv-r.hadley.nz/s4.html#generics-and-methods
> <https://adv-r.hadley.nz/s4.html#generics-and-methods>). Does using braces
> keep you from colliding with future developers or other packages?
>
> Thanks,
> Elizabeth
>
>
>
>
[[alternative HTML version deleted]]
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] How to extend nonstandardGenericFunction
Hello,
In my package (`clusterExperiment`) I want to extend a S4 method from a package
`phylobase` for my class. However, in the phylobase package, all of the
`setGeneric` calls are in the form of
setMethod("nTips", signature(x="phylo"),
function(x) {
Ntip(x)
})
Namely, they use braces in their function definitions, which I have just
learned creates a `nonstandardGenericFunction` rather than a `standardGeneric`.
Because of this I can’t just do `importMethodFrom` and then just extend the
method. And if I try to create my own `standardGeneric` (e.g.
https://stat.ethz.ch/R-manual/R-devel/library/methods/html/Methods_for_Nongenerics.html
)
I can no longer call the function in the original `phylobase` package, i.e.
for the original class for which it is applied.
My only solution appears to be if I put `phylobase` in the `Depends` section,
at which point I can then extend their generics. But this gives my package the
significant warning:
" Warning: replacing previous import ‘phylobase::plot’ by ‘graphics::plot’ when
loading ‘clusterExperiment’”
My question is whether there is anything I can do on my end to either 1) extend
phylobase generics without a depends statement or 2) to get rid of the warning
about the imports (I have made sure to use `graphics::plot` in all of my
functions’ calls to plot to make sure there is no problem with the import
conflict)?
Or is my only hope to have the developers of the package fix their calls?
Personally, I was going to strip my setGeneric calls of braces after this
experience, since I also have braces in my call but not for any good reason. Is
there a good reason that setGenerics might have the braces in their calls?
(Hadley’s Advanced R seems to say no, perhaps for this reason
https://adv-r.hadley.nz/s4.html#generics-and-methods
). Does using braces keep
you from colliding with future developers or other packages?
Thanks,
Elizabeth
[[alternative HTML version deleted]]
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Problem with setClassUnion and DelayedArray
Thanks so much Hervé! That appears to have solved the problem. And I will
definitely keep in mind for the future Michael’s suggestion of duck typing for
matrix-like objects.
All of the best,
Elizabeth
> On Oct 10, 2018, at 9:33 PM, Hervé Pagès wrote:
>
> Hi Elizabeth,
>
> I agree that the setClassUnion() warning is rather esoteric, especially
> the "consider setClassUnion()" part.
>
> library(DelayedArray)
> setClassUnion("matrixOrHDF5", c("matrix", "DelayedArray"))
> # Warning message:
> # subclass "DelayedArray1" of class "DelayedArray" is not local and
> # cannot be updated for new inheritance information; consider
>
> Furthermore, showClass("DelayedArray") reports the correct inheritance
> information:
>
> showClass("DelayedArray1")
> # Class "DelayedArray1" [package "DelayedArray"]
> #
> # Slots:
> #
> # Name:index delayed_opsseed
> # Class:listlist ANY
> #
> # Extends:
> # Class "DelayedArray", directly
> # Class "DelayedUnaryIsoOp", by class "DelayedArray", distance 2
> # Class "matrixOrHDF5", by class "DelayedArray", distance 2
> # Class "DelayedUnaryOp", by class "DelayedArray", distance 3
> # Class "DelayedOp", by class "DelayedArray", distance 4
> # Class "Array", by class "DelayedArray", distance 5
>
> As well as extends():
>
> extends("DelayedArray1")
> # [1] "DelayedArray1" "DelayedArray" "DelayedUnaryIsoOp"
> # [4] "matrixOrHDF5" "DelayedUnaryOp""DelayedOp"
> # [7] "Array"
>
> extends("DelayedArray", "matrixOrHDF5")
> # [1] TRUE
>
> extends("DelayedArray1", "DelayedArray")
> # [1] TRUE
>
> extends("DelayedArray1", "matrixOrHDF5")
> # [1] TRUE
>
> So it might just be a spurious warning :-/
>
> Anyway, I've exported the DelayedArray1 class in DelayedArray 0.7.48:
>
> https://github.com/Bioconductor/DelayedArray/commit/26061a9b28b87b8a3ee26b8b81ff3334b55115c1
>
> No more warning with this version:
>
> library(DelayedArray)
> setClassUnion("matrixOrHDF5", c("matrix", "DelayedArray"))
>
> Cheers,
> H.
>
>
> On 10/10/2018 11:51 AM, Elizabeth Purdom wrote:
>> Hello,
>> I am using `setClassUnion` in my package `clusterExperiment` in the
>> following code to allow for either matrix or DelayedArray:
>> setClassUnion("matrixOrHDF5",members=c("matrix", "DelayedArray"))
>> This causes the following warning in checking my package:
>> Warning: subclass "DelayedArray1" of class "DelayedArray" is not local and
>> cannot be updated for new inheritance information; consider setClassUnion()
>> I’ve gotten this warning in other settings, and I believe it is due to the
>> fact that setClassUnion works on the subclasses of the members, so if you
>> give the argument `members=c(“X”,”Y”)` and you haven’t imported into your
>> package all of the subclasses of “X” and “Y” it is warning you those
>> non-imported classes haven’t been dealt with (though if so, I’d say the
>> warning is awfully cryptic, especially since it says to use `setClassUnion`
>> as a solution). In my other cases, I have just gone ahead and imported all
>> of the subclasses from the package that defines the member classes and have
>> gotten rid of the message (in the past it hasn’t been so many). But
>> “DelayedArray1” is not an exported class of the DelayedArray package so that
>> is not an option here.
>> I have been just ignoring this warning, since I understand (I think) the
>> warning, I can’t do anything about it, and am not concerned about it since
>> this new class is only used internally by my function for the slot
>> definition. And I don’t think the user sees this generally. But given that
>> we’re coming up on a release I thought I would ask if there’s anything I can
>> do to get rid of this warning! Or can I go with my first instinct and safely
>> ignore it?
>> Thanks,
>> Elizabeth Purdom
>> ___
>> Bioc-devel@r-project.org mailing list
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel=DwIFaQ=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=4le20lkIbxVE8gFC4uH_tCGjq9qX1garrTomLOEFN6A=VePKILIDYgQk9KdF7u7hJQJLglF5ga8I6M5u99inEyo=
>
> --
> Hervé Pagès
>
> Program in Computational Biology
> Division of Public Health Sciences
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N, M1-B514
> P.O. Box 19024
> Seattle, WA 98109-1024
>
> E-mail: hpa...@fredhutch.org
> Phone: (206) 667-5791
> Fax:(206) 667-1319
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] Problem with setClassUnion and DelayedArray
Hello,
I am using `setClassUnion` in my package `clusterExperiment` in the following
code to allow for either matrix or DelayedArray:
setClassUnion("matrixOrHDF5",members=c("matrix", "DelayedArray"))
This causes the following warning in checking my package:
Warning: subclass "DelayedArray1" of class "DelayedArray" is not local and
cannot be updated for new inheritance information; consider setClassUnion()
I’ve gotten this warning in other settings, and I believe it is due to the fact
that setClassUnion works on the subclasses of the members, so if you give the
argument `members=c(“X”,”Y”)` and you haven’t imported into your package all of
the subclasses of “X” and “Y” it is warning you those non-imported classes
haven’t been dealt with (though if so, I’d say the warning is awfully cryptic,
especially since it says to use `setClassUnion` as a solution). In my other
cases, I have just gone ahead and imported all of the subclasses from the
package that defines the member classes and have gotten rid of the message (in
the past it hasn’t been so many). But “DelayedArray1” is not an exported class
of the DelayedArray package so that is not an option here.
I have been just ignoring this warning, since I understand (I think) the
warning, I can’t do anything about it, and am not concerned about it since this
new class is only used internally by my function for the slot definition. And I
don’t think the user sees this generally. But given that we’re coming up on a
release I thought I would ask if there’s anything I can do to get rid of this
warning! Or can I go with my first instinct and safely ignore it?
Thanks,
Elizabeth Purdom
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)
Thanks Hervé and Stephanie for your suggestions. I am really looking for a S4
methods solution however, given how my package is already set up. Also, I have
several functions that I need to adapt in this way, so it seems cleaner and
simpler to do the class union, which sounds like is not a problem — for
Stephanie’s solution, for each function I’d have to write 2 S4methods and an
internal function which feels more cluttered code to maintain for me. And it
sounds like there’s not a virtual class I could use instead so I am correct to
defining it myself.
In terms of the setClassUnion, I chose “DelayedArray” because I wanted to
capture HDF5Matrix and DelayedMatrix, but I now see that HDF5Matrix inherits
from DelayedMatrix. I must have missed that somehow.
Thanks,
Elizabeth
> On Apr 30, 2018, at 8:35 PM, Hervé Pagès <hpa...@fredhutch.org> wrote:
>
> The class union should probably be:
>
> setClassUnion("matrixOrDelayed", c("matrix", "DelayedMatrix"))
>
> i.e. use DelayedMatrix instead of DelayedArray.
>
> So in addition to the class union and to Stephanie's solution, which
> IMO are both valid solutions, you could also go for something like this:
>
> myNewRowMeans <- function(x,...)
> {
>if (length(dim(x)) != 2)
>stop("'x' must be a matrix-like object")
>...
> )
>
> that is, just a regular function that checks that 'x' is matrix-like
> based on its number of dimensions. If you really want to restrict to
> matrix and DelayedMatrix only, replace the test with
>
>if (!(is.matrix(x) || is(x, "DelayedMatrix")))
>stop("'x' must be a matrix or DelayedMatrix object")
>
> The difference being that now the function will reject matrix-like
> objects that are not matrix or DelayedMatrix objects (e.g. a Matrix
> derivative from the Matrix package).
>
> Cheers,
> H.
>
>
> On 04/30/2018 09:29 AM, Stephanie M. Gogarten wrote:
>> Rather than a class union, how about an internal function that is called by
>> the methods for both matrix and DelayedArray:
>> setGeneric("myNewRowMeans", function(x,...) {
>> standardGeneric("myNewRowMeans")})
>> #' @importFrom DelayedArray rowMeans
>> .myNewRowMeans <- function(x,...){
>> # a lot of code independent of x
>> print("This is a lot of code shared regardless of class of x\n")
>> # a lot of code that depends on x, but is dispatched by the functions
>> called
>> out<-rowMeans(x)
>> #a lot of code based on output of out
>> out<-out+1
>> return(out)
>> }
>> setMethod("myNewRowMeans",
>> signature = "matrix",
>> definition = function(x,...){
>> .myNewRowMeans(x,...)
>> }
>> )
>> setMethod("myNewRowMeans",
>> signature = "DelayedArray",
>> definition = function(x,...){
>> .myNewRowMeans(x,...)
>> }
>> )
>> On 4/30/18 9:10 AM, Tim Triche, Jr. wrote:
>>> But if you merge methods like that, the error method can be that much more
>>> difficult to identify. It took a couple of weeks to chase that bug down
>>> properly, and it ended up down to rowMeans2 vs rowMeans.
>>>
>>> I suppose the merged/abstracted method allows to centralize any such
>>> dispatch into one place and swap out ill-behaved methods once identified,
>>> so as long as DelayedArray/DelayedMatrixStats quirks are
>>> documented/understood, maybe it is better to create this union class?
>>>
>>> The Matrix/matrixStats/DelayedMatrix/DelayedMatrixStats situation has been
>>> "interesting" in practical terms, as seemingly simple abstractions appear
>>> to require more thought. That was my only point.
>>>
>>>
>>> --t
>>>
>>> On Mon, Apr 30, 2018 at 11:28 AM, Martin Morgan <
>>> martin.mor...@roswellpark.org> wrote:
>>>
>>>> But that issue will be fixed, so Tim's advice is inappropriate.
>>>>
>>>>
>>>> On 04/30/2018 10:42 AM, Tim Triche, Jr. wrote:
>>>>
>>>>> Don't do that. Seriously, just don't.
>>>>>
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Bioconductor_DelayedArray_issues_16=DwIDaQ=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=Rhy4i6H9xaY8HzWv9v_jhOnp5OyEpJcG52RP3nHorU8=olbErqY3_l7i45-WeTkaUNGalrQQr-7i59rhJVF6OGQ=
>>>>>
>>>>>
>>>>> --t
>>>>>
&
[Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)
Hello,
I am trying to extend my package to handle `HDF5Matrix` class ( or more
generally `DelayedArray`). I currently have S4 functions for `matrix` class.
Usually I have a method for `SummarizedExperiment`, which will call call the
method on `assay(x)` and I want the method to be able to deal with if
`assay(x)` is a `DelayedArray`.
Most of my functions, however, do not require separate code depending on
whether `x` is a `matrix` or `DelayedArray`. They are making use of existing
functions that will make that choice for me, e.g. rowMeans or subsetting. My
goal right now is compatibility, not cleverness, and I'm not creating HDF5
methods to handle other cases. (If something doesn't currently exist, then I
just enclose `x` with `data.matrix` or `as.matrix` and call the matrix into
memory — for cleanliness and ease in updating with appropriate methods in
future, I could make separate S4 functions for these specific tasks to
dispatch, but that's outside of the scope of my question). So for simplicity
assume I don't really need to dispatch *my code* -- that the methods I'm going
to use do that.
The natural solution for me seem to use `setClassUnion` and I was wondering if
such a virtual class already exists? Or is there a better way to handle this?
Here's a simple example, using `rowMeans` as my example:
```
setGeneric("myNewRowMeans", function(x,...) { standardGeneric("myNewRowMeans")})
setClassUnion("matrixOrDelayed",members=c("matrix", "DelayedArray"))
#' @importFrom DelayedArray rowMeans
setMethod("myNewRowMeans",
signature = "matrixOrDelayed",
definition = function(x,...){
# a lot of code independent of x
print("This is a lot of code shared regardless of class
of x\n")
# a lot of code that depends on x, but is dispatched by
the functions called
out<-rowMeans(x)
#a lot of code based on output of out
out<-out+1
return(out)
}
)
```
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] problem with class definitions between S4Vectors and RNeXML in using Summarized Experiment
gt;>>>
>>>>>> Biobase %c% ExpressionSet # a classRepresentation instance
>>>>>>
>>>>>>
>>>>>> is(1:5, Biobase %c% ExpressionSet) # FALSE
>>>>>>
>>>>>>
>>>>>> is(Biobase::ExpressionSet(), "ExpressionSet") # TRUE
>>>>>>
>>>>>>
>>>>>> is(Biobase::ExpressionSet(), Biobase %c% ExpressionSet) # TRUE
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 12, 2018 at 3:57 PM, Michael Lawrence
>>>>>> <lawrence.mich...@gene.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi Davide,
>>>>>>>
>>>>>>> We can get this fixed soon, but I was hoping to hear e.g. Herve's
>>>>>>> opinion first if he has one.
>>>>>>>
>>>>>>> Michael
>>>>>>>
>>>>>>> On Thu, Apr 12, 2018 at 12:53 PM, Davide Risso
>>>>>>> <dar2...@med.cornell.edu
>>>>>>>>
>>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Michael,
>>>>>>>>
>>>>>>>> Thanks for looking into this.
>>>>>>>>
>>>>>>>> Can you or someone with push permission to S4Vectors implement the
>>>>>>>> workaround that you mentioned?
>>>>>>>>
>>>>>>>> Happy to create a pull request on Github if that helps.
>>>>>>>>
>>>>>>>> We’re trying to solve this to fix the clusterExperiment package build
>>>>>>>> on
>>>>>>>> Bioc-devel.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Davide
>>>>>>>>
>>>>>>>>
>>>>>>>> On Apr 12, 2018, at 1:27 PM, Michael Lawrence
>>>>>>>> <lawrence.mich...@gene.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Yea it's basically
>>>>>>>>
>>>>>>>> library(S4Vectors)
>>>>>>>> library(RNeXML)
>>>>>>>> is(1:5, "Annotated")
>>>>>>>> # Found more than one class "Annotated" in cache; using the first,
>>>>>>>> from namespace 'S4Vectors'
>>>>>>>> # Also defined by ‘RNeXML’
>>>>>>>> # [1] FALSE
>>>>>>>>
>>>>>>>> But can be worked around:
>>>>>>>>
>>>>>>>> is(1:5, getClass("Annotated", where=getNamespace("S4Vectors"))
>>>>>>>>
>>>>>>>> # [1] FALSE
>>>>>>>>
>>>>>>>> Of course, using class objects instead of class names in every call
>>>>>>>> to
>>>>>>>> is() is not very palatable, but that's how it's done in all other
>>>>>>>> languages, as far as I know.
>>>>>>>>
>>>>>>>> There is an inconsistency between new() and is() when resolving the
>>>>>>>> class name. new() looks into the calling package's namespace, while
>>>>>>>> is() looks at the package for the class of the 'object'. The new()
>>>>>>>> approach seems sensible for that function, since packages should be
>>>>>>>> abstracting the construction of their objects with constructors. The
>>>>>>>> is() approach is broken though, because it's easy to imagine cases
>>>>>>>> like where some foreign object is passed to a function, and the
>>>>>>>> function checks the type with is().
>>>>>>>>
>>>>>>>> I can change is() to use the calling package as the fallback, so
>>>>>>>> DataFrame(1:5) no longer produces a message. But calling it from
>>>>>>>> another package, or global env, will still break, just like new().
>>>>>>>> How
>>>>>>>> does that sound?
>>>>>>>>
>>>>>>>> On the other hand, m
Re: [Bioc-devel] Problem with saveHDF5SummarizedExperiment in HDF5Array package
Yes, thanks. That also fixed it for me; only DelayedArray had a different
version number.
I have 0.17.42 of S4Vectors because there were also a fix in that package that
I need which are also not yet available from bioCLite (the question of
duplicated class names creating messages was breaking my package).
Elizabeth
> On Apr 17, 2018, at 1:19 PM, Vincent Carey <st...@channing.harvard.edu> wrote:
>
> Thanks Pete -- indeed installing fresh HDF5Array scotches the error. But the
> package
> version number is the same as Elizabeth's: HDF5Array_1.7.10 -- so is it new
> code,
> or just the act of reinstalling, perhaps with a different packageset in place
> than was
> present for the first HDF5Array installation, that solves the problem?
>
> On Tue, Apr 17, 2018 at 7:06 AM, Vincent Carey <st...@channing.harvard.edu
> <mailto:st...@channing.harvard.edu>> wrote:
> confirmed with
> R version 3.5.0 beta (2018-04-10 r74581) under macosx
>
> sHDF5S> ## Save 'se0' as an HDF5-based SummarizedExperiment object:
> sHDF5S> dir <- sub("file", "h5_se0_", tempfile())
>
> sHDF5S> h5_se0 <- saveHDF5SummarizedExperiment(se0, dir)
> Error: C stack usage 7969544 is too close to the limit
>
> I've been running into this a lot and in my case it seemed
> sufficient to "remake" old serialized objects. But this example
> is a good one. Somehow you have S4Vectors_0.17.42 while I am working with
> 0.17.41.
> On a linux system this error was not thrown.
>
>
> On Tue, Apr 17, 2018 at 6:14 AM, Elizabeth Purdom <epur...@stat.berkeley.edu
> <mailto:epur...@stat.berkeley.edu>> wrote:
> Hello,
>
> When I try to run the example code in the saveHDF5SummarizedExperiment
> function, I get the error "Error: C stack usage 7969416 is too close to the
> limit”. I am working with development R and have incorporated HDF5
> functionality in my package. I did so many weeks ago on earlier versions of
> the packages and didn’t use to get this error, but now my tests are failing,
> etc, since I can’t create a basic object.
>
> Perhaps I’m unknowingly using the wrong version or some other problem?
> Otherwise, I expect this is already known by authors since its their own
> example, but in that case I am also wondering if I should roll back to an
> earlier version for now, and if so which one so that I’m still reasonably
> current?
>
> Thanks,
> Elizabeth Purdom
>
> Following example from the help pages of saveHDF5SummarizedExperiment:
> > library(HDF5Array)
> > library(SummarizedExperiment)
> > nrows <- 200; ncols <- 6
> > counts <- matrix(runif(nrows * ncols, 1, 1e4), nrows)
> > colData <- DataFrame(Treatment=rep(c("ChIP", "Input"), 3),
> + row.names=LETTERS[1:6])
> > se0 <- SummarizedExperiment(assays=SimpleList(counts=counts),
> + colData=colData)
> > se0
> class: SummarizedExperiment
> dim: 200 6
> metadata(0):
> assays(1): counts
> rownames: NULL
> rowData names(0):
> colnames(6): A B ... E F
> colData names(1): Treatment
> >
> > ## Save 'se0' as an HDF5-based SummarizedExperiment object:
> > dir <- sub("file", "h5_se0_", tempfile())
> > h5_se0 <- saveHDF5SummarizedExperiment(se0, dir)
> Error: C stack usage 7969416 is too close to the limit
> #only showing part of traceback, because as expected by error, hitting some
> kind of loop
> > traceback()
> …..
> 28: nrow(x)
> 27: nrow(x)
> 26: dim(x)
> 25: dim(x)
> 24: nrow(x)
> 23: nrow(x)
> 22: dim(x)
> 21: dim(x)
> 20: nrow(x)
> 19: nrow(x)
> 18: dim(assay)
> 17: dim(assay)
> 16: FUN(X[[i]], ...)
> 15: lapply(as.list(X), match.fun(FUN), ...)
> 14: lapply(as.list(X), match.fun(FUN), ...)
> 13: lapply(X = X, FUN = FUN, ...)
> 12: lapply(X = X, FUN = FUN, ...)
> 11: sapply(assays, function(assay) dim(assay)[1:2])
> 10: sapply(assays, function(assay) dim(assay)[1:2])
> 9: valid.func(object)
> 8: validityMethod(as(object, superClass))
> 7: isTRUE(x)
> 6: anyStrings(validityMethod(as(object, superClass)))
> 5: validObject(ans)
> 4: `[[<-`(`*tmp*`, i, value = new("HDF5Matrix", seed = new("HDF5ArraySeed",
>filepath =
> "/private/var/folders/h4/xtpbyfq55qd3rc882bm4zfjwgn/T/RtmpKIQALa/h5_se0_7d29f927618/assays.h5",
>
>name = "assay001", dim = c(200L, 6L), first_val = 2481.95574347652,
>chunkdim = c(200L, 6L
> 3: `[[<-`(`*tmp*`, i, value = new("HDF5Matrix", seed = new("HDF5ArraySeed",
>filepath =
> "/private/var/
[Bioc-devel] Problem with saveHDF5SummarizedExperiment in HDF5Array package
Hello,
When I try to run the example code in the saveHDF5SummarizedExperiment
function, I get the error "Error: C stack usage 7969416 is too close to the
limit”. I am working with development R and have incorporated HDF5
functionality in my package. I did so many weeks ago on earlier versions of the
packages and didn’t use to get this error, but now my tests are failing, etc,
since I can’t create a basic object.
Perhaps I’m unknowingly using the wrong version or some other problem?
Otherwise, I expect this is already known by authors since its their own
example, but in that case I am also wondering if I should roll back to an
earlier version for now, and if so which one so that I’m still reasonably
current?
Thanks,
Elizabeth Purdom
Following example from the help pages of saveHDF5SummarizedExperiment:
> library(HDF5Array)
> library(SummarizedExperiment)
> nrows <- 200; ncols <- 6
> counts <- matrix(runif(nrows * ncols, 1, 1e4), nrows)
> colData <- DataFrame(Treatment=rep(c("ChIP", "Input"), 3),
+ row.names=LETTERS[1:6])
> se0 <- SummarizedExperiment(assays=SimpleList(counts=counts),
+ colData=colData)
> se0
class: SummarizedExperiment
dim: 200 6
metadata(0):
assays(1): counts
rownames: NULL
rowData names(0):
colnames(6): A B ... E F
colData names(1): Treatment
>
> ## Save 'se0' as an HDF5-based SummarizedExperiment object:
> dir <- sub("file", "h5_se0_", tempfile())
> h5_se0 <- saveHDF5SummarizedExperiment(se0, dir)
Error: C stack usage 7969416 is too close to the limit
#only showing part of traceback, because as expected by error, hitting some
kind of loop
> traceback()
…..
28: nrow(x)
27: nrow(x)
26: dim(x)
25: dim(x)
24: nrow(x)
23: nrow(x)
22: dim(x)
21: dim(x)
20: nrow(x)
19: nrow(x)
18: dim(assay)
17: dim(assay)
16: FUN(X[[i]], ...)
15: lapply(as.list(X), match.fun(FUN), ...)
14: lapply(as.list(X), match.fun(FUN), ...)
13: lapply(X = X, FUN = FUN, ...)
12: lapply(X = X, FUN = FUN, ...)
11: sapply(assays, function(assay) dim(assay)[1:2])
10: sapply(assays, function(assay) dim(assay)[1:2])
9: valid.func(object)
8: validityMethod(as(object, superClass))
7: isTRUE(x)
6: anyStrings(validityMethod(as(object, superClass)))
5: validObject(ans)
4: `[[<-`(`*tmp*`, i, value = new("HDF5Matrix", seed = new("HDF5ArraySeed",
filepath =
"/private/var/folders/h4/xtpbyfq55qd3rc882bm4zfjwgn/T/RtmpKIQALa/h5_se0_7d29f927618/assays.h5",
name = "assay001", dim = c(200L, 6L), first_val = 2481.95574347652,
chunkdim = c(200L, 6L
3: `[[<-`(`*tmp*`, i, value = new("HDF5Matrix", seed = new("HDF5ArraySeed",
filepath =
"/private/var/folders/h4/xtpbyfq55qd3rc882bm4zfjwgn/T/RtmpKIQALa/h5_se0_7d29f927618/assays.h5",
name = "assay001", dim = c(200L, 6L), first_val = 2481.95574347652,
chunkdim = c(200L, 6L
2: .write_h5_assays(x@assays, h5_path, chunkdim, level, verbose)
1: saveHDF5SummarizedExperiment(se0, dir)
> sessionInfo()
R Under development (unstable) (2018-03-22 r74446)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: OS X El Capitan 10.11.6
Matrix products: default
BLAS:
/Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK:
/Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel stats4stats graphics grDevices utils datasets
methods base
other attached packages:
[1] SummarizedExperiment_1.9.16 Biobase_2.39.2
GenomicRanges_1.31.23 GenomeInfoDb_1.15.5
[5] HDF5Array_1.7.10rhdf5_2.23.8
DelayedArray_0.5.30 BiocParallel_1.13.3
[9] IRanges_2.13.28 S4Vectors_0.17.42
BiocGenerics_0.25.3 matrixStats_0.53.1
loaded via a namespace (and not attached):
[1] lattice_0.20-35bitops_1.0-6 grid_3.5.0
zlibbioc_1.25.0XVector_0.19.9
[6] Matrix_1.2-14 Rhdf5lib_1.1.5 tools_3.5.0
RCurl_1.95-4.10compiler_3.5.0
[11] GenomeInfoDbData_1.1.0
>
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] problem with class definitions between S4Vectors and RNeXML in using Summarized Experiment
Just to follow up on my previous post. I am able to replicate the problem in the problem like in the github post from 2 years ago (https://github.com/epurdom/clusterExperiment/issues/66 <https://github.com/epurdom/clusterExperiment/issues/66>) only now it is not the SummarizedExperiment class but the SingleCellExperiment class that has the problem. [And I was incorrect, the problem does occur in development version 2018-03-22 r74446]. So this is actually a problem with the SingleCellExperiment package — sorry for the incorrect subject line. All of the best, Elizabeth > > library(SingleCellExperiment) > > SingleCellExperiment() > class: SingleCellExperiment > dim: 0 0 > metadata(0): > assays(0): > rownames: NULL > rowData names(0): > colnames: NULL > colData names(0): > reducedDimNames(0): > spikeNames(0): > > library(RNeXML) > Loading required package: ape > > > > SingleCellExperiment() > Found more than one class "Annotated" in cache; using the first, from > namespace 'S4Vectors' > Also defined by ‘RNeXML’ > Found more than one class "Annotated" in cache; using the first, from > namespace 'S4Vectors' > Also defined by ‘RNeXML’ > class: SingleCellExperiment > dim: 0 0 > metadata(0): > assays(0): > rownames: NULL > rowData names(0): > colnames: NULL > colData names(0): > reducedDimNames(0): > spikeNames(0): > > sessionInfo() > R Under development (unstable) (2018-03-22 r74446) > Platform: x86_64-apple-darwin15.6.0 (64-bit) > Running under: OS X El Capitan 10.11.6 > > Matrix products: default > BLAS: > /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib > LAPACK: > /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib > > locale: > [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 > > attached base packages: > [1] parallel stats4stats graphics grDevices utils datasets > methods base > > other attached packages: > [1] RNeXML_2.0.8ape_5.1 > SingleCellExperiment_1.1.2 > [4] SummarizedExperiment_1.9.16 DelayedArray_0.5.30 > BiocParallel_1.13.3 > [7] matrixStats_0.53.1 Biobase_2.39.2 > GenomicRanges_1.31.23 > [10] GenomeInfoDb_1.15.5 IRanges_2.13.28 > S4Vectors_0.17.41 > [13] BiocGenerics_0.25.3 > > loaded via a namespace (and not attached): > [1] Rcpp_0.12.16 pillar_1.2.1 bindr_0.1.1 > compiler_3.5.0 > [5] plyr_1.8.4 XVector_0.19.9 iterators_1.0.9 > bitops_1.0-6 > [9] tools_3.5.0zlibbioc_1.25.0uuid_0.1-2 > tibble_1.4.2 > [13] jsonlite_1.5 nlme_3.1-137 lattice_0.20-35 > pkgconfig_2.0.1 > [17] rlang_0.2.0Matrix_1.2-14 foreach_1.4.4 > crul_0.5.2 > [21] curl_3.2 bindrcpp_0.2.2 GenomeInfoDbData_1.1.0 > dplyr_0.7.4 > [25] httr_1.3.1 stringr_1.3.0 xml2_1.2.0 > grid_3.5.0 > [29] glue_1.2.0 reshape_0.8.7 data.table_1.10.4-3 > R6_2.2.2 > [33] XML_3.98-1.10 purrr_0.2.4tidyr_0.8.0 > reshape2_1.4.3 > [37] magrittr_1.5 codetools_0.2-15 assertthat_0.2.0 > bold_0.5.0 > [41] taxize_0.9.3 stringi_1.1.7 lazyeval_0.2.1 > RCurl_1.95-4.10 > [45] zoo_1.8-1 > On Apr 11, 2018, at 10:25 AM, Elizabeth Purdom <epur...@stat.berkeley.edu> > wrote: > > Hello, > > Our package clusterExperiment has suddenly started producing errors in the > bioconductor devel branch because our unit tests are failing, even though we > haven’t pushed any changes and they passed previously. We first noticed this > on April 5th. > > I believe the source of these errors likely do to calls to > SummarizedExperiment() creating the following messages due to conflicts in > ‘RNeXML’ and ’S4Vectors’ both defining class ‘Annotated': > >> Found more than one class "Annotated" in cache; using the first, from >> namespace 'S4Vectors' >> Also defined by ‘RNeXML’ > This is killing a vast number of our tests where we repeatedly use > ‘expect_silent’ calls in our unit tests and SummarizedExperiment calls > underlie everything. > > We had this message issue two years ago > (https://github.com/epurdom/clusterExperiment/issues/66 > <https://github.com/epurdom/clusterExperi
[Bioc-devel] problem with class definitions between S4Vectors and RNeXML in using Summarized Experiment
Hello, Our package clusterExperiment has suddenly started producing errors in the bioconductor devel branch because our unit tests are failing, even though we haven’t pushed any changes and they passed previously. We first noticed this on April 5th. I believe the source of these errors likely do to calls to SummarizedExperiment() creating the following messages due to conflicts in ‘RNeXML’ and ’S4Vectors’ both defining class ‘Annotated': > Found more than one class "Annotated" in cache; using the first, from > namespace 'S4Vectors' > Also defined by ‘RNeXML’ This is killing a vast number of our tests where we repeatedly use ‘expect_silent’ calls in our unit tests and SummarizedExperiment calls underlie everything. We had this message issue two years ago (https://github.com/epurdom/clusterExperiment/issues/66), when it appeared to be a problem with two definitions of the ‘Annotated’ class in two packages that are both dependencies of packages we call. At that time, Michael Lawrence posted that he would fix the problem, and it was then fixed in later versions of bioconductor/R. But it appears to be back. I am unfortunately unable to get the RNeXML package to compile from source on my computer with the current Mac OS X development binary which I just downloaded (2018-04-05 r74542), so I haven’t been able to completely redo the code that we presented in that earlier github issue to confirm it is the exact same problem. I am having to rely on the error reports/logs from both Bioconductor and TravisCI (e.g. 2018-04-07 r74551), where this message shows up everywhere and didn’t before. Thus I’m guessing that since they are the same messages from before that the source is again the call to SummarizedExperiment. I would note that in development version 2018-03-22 r74446, where I was able to install all of the packages, I was not getting these messages. Thanks, Elizabeth Purdom [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] problem in rbind with DelayedArray / HDF5Array package
Yes that does get around the problem, thanks! I would note, though, that for me at least, its a workaround and not ideal functionality, since to use it in my function means that I now I have to have two different code snippets depending on whether my input object (here `testhdf`) is a matrix or a HDF5Matrix (rather than it automatically using the class definitions to give a reasonable result which is what usually happens). Thanks, Elizabeth > On Apr 4, 2018, at 9:07 PM, Peter Hickey <peter.hic...@gmail.com> wrote: > > Does `rbind(testhdf, DelayedArray(testdata))` give you what you want? > > On Wed, 4 Apr 2018 at 14:58 Elizabeth Purdom <epur...@stat.berkeley.edu > <mailto:epur...@stat.berkeley.edu>> wrote: > Hello, > > I am trying to do a rbind a normal (in memory) matrix with a HDF5Matrix > object or DelayedArray object and I am hitting problems. I’m using the > development version of R and bioconductor (as of 2 weeks ago) and > HDF5Array_1.7.9, DelayedArray_0.5.23 — see sessionInfo at end of email. > > Basically if I apply rbind between my normal matrix and a HDF5Matrix, I get > an error that a method doesn't exist for a DataTable class. If I force my > HDF5Matrix object into a DelayedMatrix object using ‘as', I either get 1) a > warning that it doesn’t know how to select a method (even if I use > DelayedArray::rbind in my command) or 2) In addition to the warning, an error > that the column names don’t match, even though neither object has colnames > (they are NULL). Which of these I get depends on the order of the entries to > rbind — the warning-only version occurs if the DelayedMatrix object is first. > > The warning-only option version gives the correct answer, but I want to > understand how to avoid the warning, since I am using this in a package. > > It also seems like a bug that it would matter the order of the arguments, nor > why I need to manually manipulate the HDF5Matrix into a DelayedMatrix object > in order to do rbind. > > Thanks, > Elizabeth Purdom > > Here is my code setting up the objects: > > > testdata<-matrix(rnorm(1000),nrow=100,ncol=10) > > testhdf<-HDF5Array::writeHDF5Array(testdata, "./trash.h5") > > class(testhdf) > [1] "HDF5Matrix" > attr(,"package") > [1] “HDF5Array" > > Here are the errors/warnings I’m getting, some of which depend on the order > of the entries into rbind: > > > test1<-rbind(testhdf,testdata) #error is independent of order of entries > Error in rbind(...) : > missing 'rbind' method for DataTable class HDF5Matrix > > test2<-DelayedArray::rbind(as(testhdf,"DelayedMatrix"),testdata) > Warning message: > In methods:::.selectDotsMethod(classes, .MTable, .AllMTable) : > multiple direct matches: "DelayedMatrix", "DataFrame"; using the first of > these > > test3<-DelayedArray::rbind(testdata,as(testhdf,"DelayedMatrix")) > Error in rbind(...) : > column names for arg 2 do not match those of first arg > In addition: Warning message: > In methods:::.selectDotsMethod(classes, .MTable, .AllMTable) : > multiple direct matches: "DataFrame", "DelayedMatrix"; using the first of > these > > > colnames(testdata) > NULL > > colnames(as(testhdf,"DelayedMatrix")) > NULL > > Here is my sessionInfo: > > > sessionInfo() > R Under development (unstable) (2018-03-22 r74446) > Platform: x86_64-apple-darwin15.6.0 (64-bit) > Running under: OS X El Capitan 10.11.6 > > Matrix products: default > BLAS: > /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib > LAPACK: > /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib > > locale: > [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] HDF5Array_1.7.9 rhdf5_2.23.5DelayedArray_0.5.23 > BiocParallel_1.13.3 IRanges_2.13.28 > [6] S4Vectors_0.17.38 BiocGenerics_0.25.3 matrixStats_0.53.1 > > loaded via a namespace (and not attached): > [1] compiler_3.5.0 tools_3.5.0Rhdf5lib_1.1.5 > > ___ > Bioc-devel@r-project.org <mailto:Bioc-devel@r-project.org> mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel > <https://stat.ethz.ch/mailman/listinfo/bioc-devel> [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] problem in rbind with DelayedArray / HDF5Array package
Hello, I am trying to do a rbind a normal (in memory) matrix with a HDF5Matrix object or DelayedArray object and I am hitting problems. I’m using the development version of R and bioconductor (as of 2 weeks ago) and HDF5Array_1.7.9, DelayedArray_0.5.23 — see sessionInfo at end of email. Basically if I apply rbind between my normal matrix and a HDF5Matrix, I get an error that a method doesn't exist for a DataTable class. If I force my HDF5Matrix object into a DelayedMatrix object using ‘as', I either get 1) a warning that it doesn’t know how to select a method (even if I use DelayedArray::rbind in my command) or 2) In addition to the warning, an error that the column names don’t match, even though neither object has colnames (they are NULL). Which of these I get depends on the order of the entries to rbind — the warning-only version occurs if the DelayedMatrix object is first. The warning-only option version gives the correct answer, but I want to understand how to avoid the warning, since I am using this in a package. It also seems like a bug that it would matter the order of the arguments, nor why I need to manually manipulate the HDF5Matrix into a DelayedMatrix object in order to do rbind. Thanks, Elizabeth Purdom Here is my code setting up the objects: > testdata<-matrix(rnorm(1000),nrow=100,ncol=10) > testhdf<-HDF5Array::writeHDF5Array(testdata, "./trash.h5") > class(testhdf) [1] "HDF5Matrix" attr(,"package") [1] “HDF5Array" Here are the errors/warnings I’m getting, some of which depend on the order of the entries into rbind: > test1<-rbind(testhdf,testdata) #error is independent of order of entries Error in rbind(...) : missing 'rbind' method for DataTable class HDF5Matrix > test2<-DelayedArray::rbind(as(testhdf,"DelayedMatrix"),testdata) Warning message: In methods:::.selectDotsMethod(classes, .MTable, .AllMTable) : multiple direct matches: "DelayedMatrix", "DataFrame"; using the first of these > test3<-DelayedArray::rbind(testdata,as(testhdf,"DelayedMatrix")) Error in rbind(...) : column names for arg 2 do not match those of first arg In addition: Warning message: In methods:::.selectDotsMethod(classes, .MTable, .AllMTable) : multiple direct matches: "DataFrame", "DelayedMatrix"; using the first of these > colnames(testdata) NULL > colnames(as(testhdf,"DelayedMatrix")) NULL Here is my sessionInfo: > sessionInfo() R Under development (unstable) (2018-03-22 r74446) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: OS X El Capitan 10.11.6 Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] HDF5Array_1.7.9 rhdf5_2.23.5DelayedArray_0.5.23 BiocParallel_1.13.3 IRanges_2.13.28 [6] S4Vectors_0.17.38 BiocGenerics_0.25.3 matrixStats_0.53.1 loaded via a namespace (and not attached): [1] compiler_3.5.0 tools_3.5.0Rhdf5lib_1.1.5 ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] Warning that namespace of my package not available for my data object
Hello,
When I push my commits to my personal github and they run through TravisCI I
get the following warning when my package ‘clusterExperiment’ is build:
* looking to see if a ‘data/datalist’ file should be added
Warning: namespace ‘clusterExperiment’ is not available and has been replaced
by .GlobalEnv when processing object ‘rsecFluidigm’
(to be clear this warning is saying the namespace of MY package is not
available and the data object it refers to is part of MY package)
I do not get this warning when building on my laptop (where previous versions
of the package are already installed). The warning does not appear anywhere
that I see in R CMD CHECK on TravisCI, just on the build output— I just
discovered it today because I was digging around and reading the successful
build logs on Travis CI). Similarly on the bioconductor check it doesn’t make a
problem with the check (I've cleared bioconductor checks without realizing this
warning was there), but if I go further and read the log of the build on the
bioconductor, the warning is there as well. Given that this wasn’t picked up by
R CMD CHECK, is this something I need be worried about? (I expect the answer
will be yes!) And if so, I would appreciate advice on how to fix it.
I’ve already had problems with this object (‘rsecFluidigm’) breaking my build
when I first added it to the package, and so at that time I set lazyData=false
in my DESCRIPTION file. BEFORE I did this, I would get the following errors (on
my machine not just TravisCI):
** preparing package for lazy loading
Found more than one class "Annotated" in cache; using the first, from namespace
'S4Vectors'
Also defined by ‘RNeXML’
Error in get0(cname, where, inherits = inherits) :
invalid 'envir' argument
Error in setClassUnion("matrixOrMissing", members = c("matrix", "missing")) :
unable to create union class: could not set members "matrix"
Error in get0(cname, where, inherits = inherits) :
invalid 'envir' argument
Error in setClassUnion("matrixOrMissing", members = c("matrix", "missing")) :
unable to create union class: could not set members "matrix"
Error : unable to load R code in package ‘clusterExperiment’
ERROR: lazy loading failed for package ‘clusterExperiment’
* removing
‘/private/var/folders/h4/xtpbyfq55qd3rc882bm4zfjwgn/T/Rtmp5LNmph/Rinst1024b4c6db021/clusterExperiment’
I’ve never understood why adding this object to my package created these errors
on lazy load, but lazyData=false seemed to fix the problem, in the sense of
making these errors go away and building the package without problem on both my
machine and Travis CI. Except of course now I realize it’s been giving this
warning on Travis CI without me realizing it. “rsecFluidigm" is a object that
has a class that is created by my package. It’s sole reason for existence is so
that the vignette can be compiled quickly, rather than having to be recreated
while building the vignette (it takes somewhere from 1-5 minutes to recreate
depending on the machine, number of cores, etc.; the vignette has the code to
create it, but with eval=FALSE).
I experimented with also setting lazyLoad=false in my description, but that did
not change anything and I continued to get the warning above.
Here is my description file (minus the collate part):
Package: clusterExperiment
Title: Compare Clusterings for Single-Cell Sequencing
Version: 1.3.7
Description: Provides functionality for running and comparing many
different clusterings of single-cell sequencing data or other large mRNA
Expression data sets.
Authors@R: c(person("Elizabeth", "Purdom", email = "epur...@stat.berkeley.edu",
role = c("aut", "cre", "cph")),
person("Davide","Risso", role = "aut",email =
"risso.dav...@gmail.com"),
person("Marla", "Johnson", role = "ctb"))
Author: Elizabeth Purdom [aut, cre, cph], Davide Risso [aut], Marla Johnson
[ctb]
Maintainer: Elizabeth Purdom <epur...@stat.berkeley.edu>
BugReports: https://github.com/epurdom/clusterExperiment/issues
License: Artistic-2.0
Depends:
R (>= 3.4),
SummarizedExperiment
Imports:
methods,
NMF,
RColorBrewer,
ape,
phylobase,
cluster,
stats,
limma,
dendextend,
howmany,
locfdr,
matrixStats,
graphics,
parallel,
RSpectra,
kernlab,
stringr
Suggests:
BiocStyle,
knitr,
testthat,
scRNAseq,
MAST
VignetteBuilder: knitr
LazyData: false
RoxygenNote: 6.0.1
biocViews: Clustering, RNASeq, Sequencing, Software, SingleCell
Thanks for any help,
Elizabeth
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] How to recreate R CMD BUILD environment in interactive session?
Dear Martin,
Just for completeness, I figured out the discrepancy and solved my problem. In
my check, I check that the column names contain the expected names and I didn’t
want to make the order required in a certain way so I used sort -- but only of
one side because I naively assumed the other side would be fixed:
any(sort(colnames(object@merge_nodeMerge)) !=
c('Contrast','isMerged','mergeClusterId','Node’)
But the different environments are sorting differently!
In my normal interactive R session:
> sort(c("Contrast", "isMerged", "mergeClusterId", "Node"))
[1] "Contrast" "isMerged" "mergeClusterId" "Node"
In the build version of R however:
Browse[2]> sort(c("Contrast", "isMerged", "mergeClusterId", "Node"))
[1] "Contrast" "Node" "isMerged" "mergeClusterId"
Thank you very much for your help in getting an interactive session in the
build environment!
Elizabeth
> On Oct 23, 2017, at 4:35 PM, Martin Morgan <martin.mor...@roswellpark.org>
> wrote:
>
> On 10/23/2017 09:59 AM, Elizabeth Purdom wrote:
>>> On Oct 23, 2017, at 3:47 PM, Martin Morgan <martin.mor...@roswellpark.org
>>> <mailto:martin.mor...@roswellpark.org>> wrote:
>>>
>>> On 10/23/2017 09:26 AM, Elizabeth Purdom wrote:
>>>> Hello,
>>>> I am updating an existing package and I am getting an error in running my
>>>> vignette (and a similar error in an example in help pages) but ONLY when I
>>>> run R CMD BUILD. I can’t recreate the error in any session where I can
>>>> debug and figure out what is happening. So my question is how can I
>>>> recreate the exact environment of R CMD BUILD that runs the vignette but
>>>> in an interactive session so that I can figure out what is going on?
>>>> I have tried reproducing the error in other environments:
>>>> * Running R —vanilla interactively and trying the code manually
>>>> * running purl on my vignette to get pure R code and running just the R
>>>> code with R CMD BATCH —vanilla
>>>
>>> hint on the specific package and / or error message?
>>>
>>> My approach would be to install the package, Stangle / purl the vignette,
>>> and R -f vignette.R, then trim the vignette to a fast reproducible case.
>>> But it sounds like you're doing that...
>>>
>>> Martin
>>>
>> Yes, that is what I tried but did not get the error from the R code.
>> And I apologize, it’s the `clusterExperiment` package. My error was so
>> specific to the class created by my package that I didn’t think it would be
>> useful, but here is the relevant error message:
>> Quitting from lines 271-272 (clusterExperimentTutorial.Rmd)
>> Error: processing vignette 'clusterExperimentTutorial.Rmd' failed with
>> diagnostics:
>> invalid class "ClusterExperiment" object: merge_nodeMerge must have 4
>> columns and column names equal to:
>> 'Node','Contrast','isMerged','mergeClusterId'
>> Execution halted
>> I would note that my vignette calls an object that is saved as a data object
>> as part of my package to speed up compilation. But I experimented and you
>> can also switch it so that it creates the object from scratch and doesn’t
>> load the object, and it runs into the same error. There is a `LazyData:
>> false` in my DESCRIPTION File, because I was having problems with my R data
>> object, because it is of the class I make with my package, and without the
>> package loaded there was some problem loading it.
>
> I can reproduce the error with
>
> clusterExperiment/vignettes master$ R_DEFAULT_PACKAGES= LC_COLLATE=C R -f
> clusterExperimentTutorial.R
>
> leading to
>
> > ##
> > recallRSEC
> > rsecFluidigm<-RSEC(rsecFluidigm,isCount=TRUE,combineProportion=0.6,mergeMethod="JC",mergeCutoff=0.05)
> Error in validObject(.Object) :
> invalid class "ClusterExperiment" object: merge_nodeMerge must be data.frame
> with 4 columns and column names equal to:
> 'Node','Contrast','isMerged','mergeClusterId'
> Calls: RSEC ... .local -> new -> initialize -> initialize -> validObject
> Execution halted
>
> and then for work inside R
>
>
> clusterExperiment/vignettes$ R_DEFAULT_PACKAGES= LC_COLLATE=C R
> Bioconductor version 3.6 (BiocInstaller 1.27.7), ?biocLite for help
> > source("clusterExperimentTutorial.R", echo=TRUE, m
Re: [Bioc-devel] How to recreate R CMD BUILD environment in interactive session?
>
> I can reproduce the error with
>
> clusterExperiment/vignettes master$ R_DEFAULT_PACKAGES= LC_COLLATE=C R -f
> clusterExperimentTutorial.R
>
> leading to
>
> > ##
> > recallRSEC
> > rsecFluidigm<-RSEC(rsecFluidigm,isCount=TRUE,combineProportion=0.6,mergeMethod="JC",mergeCutoff=0.05)
> Error in validObject(.Object) :
> invalid class "ClusterExperiment" object: merge_nodeMerge must be data.frame
> with 4 columns and column names equal to:
> 'Node','Contrast','isMerged','mergeClusterId'
> Calls: RSEC ... .local -> new -> initialize -> initialize -> validObject
> Execution halted
>
> and then for work inside R
>
>
> clusterExperiment/vignettes$ R_DEFAULT_PACKAGES= LC_COLLATE=C R
> Bioconductor version 3.6 (BiocInstaller 1.27.7), ?biocLite for help
> > source("clusterExperimentTutorial.R", echo=TRUE, max=Inf)
>
> Does that set you down the right path?
>
> Martin
>
Yes, thank you!!! I can get it in the interactive session now.
Elizabeth
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] How to recreate R CMD BUILD environment in interactive session?
> On Oct 23, 2017, at 3:47 PM, Martin Morgan <martin.mor...@roswellpark.org> > wrote: > > On 10/23/2017 09:26 AM, Elizabeth Purdom wrote: >> Hello, >> I am updating an existing package and I am getting an error in running my >> vignette (and a similar error in an example in help pages) but ONLY when I >> run R CMD BUILD. I can’t recreate the error in any session where I can debug >> and figure out what is happening. So my question is how can I recreate the >> exact environment of R CMD BUILD that runs the vignette but in an >> interactive session so that I can figure out what is going on? >> I have tried reproducing the error in other environments: >> * Running R —vanilla interactively and trying the code manually >> * running purl on my vignette to get pure R code and running just the R code >> with R CMD BATCH —vanilla > > hint on the specific package and / or error message? > > My approach would be to install the package, Stangle / purl the vignette, and > R -f vignette.R, then trim the vignette to a fast reproducible case. But it > sounds like you're doing that... > > Martin > Yes, that is what I tried but did not get the error from the R code. And I apologize, it’s the `clusterExperiment` package. My error was so specific to the class created by my package that I didn’t think it would be useful, but here is the relevant error message: Quitting from lines 271-272 (clusterExperimentTutorial.Rmd) Error: processing vignette 'clusterExperimentTutorial.Rmd' failed with diagnostics: invalid class "ClusterExperiment" object: merge_nodeMerge must have 4 columns and column names equal to: 'Node','Contrast','isMerged','mergeClusterId' Execution halted I would note that my vignette calls an object that is saved as a data object as part of my package to speed up compilation. But I experimented and you can also switch it so that it creates the object from scratch and doesn’t load the object, and it runs into the same error. There is a `LazyData: false` in my DESCRIPTION File, because I was having problems with my R data object, because it is of the class I make with my package, and without the package loaded there was some problem loading it. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] How to recreate R CMD BUILD environment in interactive session?
Hello, I am updating an existing package and I am getting an error in running my vignette (and a similar error in an example in help pages) but ONLY when I run R CMD BUILD. I can’t recreate the error in any session where I can debug and figure out what is happening. So my question is how can I recreate the exact environment of R CMD BUILD that runs the vignette but in an interactive session so that I can figure out what is going on? I have tried reproducing the error in other environments: * Running R —vanilla interactively and trying the code manually * running purl on my vignette to get pure R code and running just the R code with R CMD BATCH —vanilla * Running R —vanilla interactively and compiling the vignette But all of the above runs the code without any error. Furthermore, for some reason TravisCI can build the package without problem nor does it hit any error in R CMD CHECK with TravisCI so for a long time I assumed it was some local file or environment on my machine and forgot about it. But now that I have pushed the changes to Bioconductor, they are getting the same error about the vignette not building. By the way, I can’t slowly take back my commands to see what broke it, because the change was a dramatic one involving redefining my S4 class, so a large number of things had to be adapted before any code would work and there’s no way to slowly roll that back. Most of the posts I have seen about debugging discrepancies between R CMD BUILD (or equivalently R CMD CHECK) seem to be about namespace issues with other packages. Of course I can’t be sure since I have a pretty terse error from R CMD BUILD, but the error that is given is my own error (it’s a check I wrote as part of the validity check when I updated the class) so it seems unclear how it would be a namespace error. Thanks for any suggestions, Elizabeth Purdom ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
