Re: [Bioc-devel] Use and Usability metrics / shields

2015-05-13 Thread Dan Tenenbaum 


- Original Message -
> From: "Henrik Bengtsson" 
> To: "COMMO Frederic" 
> Cc: bioc-devel@r-project.org
> Sent: Wednesday, May 13, 2015 12:28:54 PM
> Subject: Re: [Bioc-devel] Use and Usability metrics / shields
> 
> Sweet; you went live with the badges/shields, e.g.
> 
>   http://bioconductor.org/packages/release/bioc/html/affxparser.html
> 
> A positive side effect is that now there's a link from the package
> page to to the package's check results, which I always wanted :)
> 

That was there before (and still is, see the bottom of the Details section). 
But yes, it was not very visible.

Dan


> Thanks for adding this
> 
> /Henrik
> 
> 
> On Sun, May 10, 2015 at 11:39 AM, COMMO Frederic
>  wrote:
> > Dear Martin,
> >
> > All of these suggestions sound good.
> >
> > Wolfgang's suggestion regarding possible associated papers might be
> > also great.
> >
> > Another useful information would be to point to other publications
> > where a given package was used, and cited.
> > I don't know if it's technically possible, but it would be greatly
> > informative to know how frequently a package is used, and how it
> > performs, in real contexts.
> >
> > Frederic Commo
> > Bioinformatics, U981
> > Gustave Roussy
> >
> > 
> > De : Bioc-devel [bioc-devel-boun...@r-project.org] de la part de
> > Wolfgang Huber [whu...@embl.de]
> > Date d'envoi : samedi 9 mai 2015 19:57
> > À : Martin Morgan
> > Cc: bioc-devel@r-project.org
> > Objet : Re: [Bioc-devel] Use and Usability metrics / shields
> >
> > Dear Martin
> >
> > great idea.
> > "Current build status” could perhaps be wrapped with
> > "Cross-platform availability” into some sort of “Availability /
> > Accessibility”?
> >
> > I wonder how informative it would be to make metrics such as
> > (i) citations of the associated paper
> > (ii) full-text mentions e.g. in PubmedCentral
> > actually useful. (i) could be flawed if package and paper are
> > diverged; (ii) would require good disambiguation, e.g. like
> > bioNerDS http://www.biomedcentral.com/1471-2105/14/194 (or other
> > tools? not my expertise). Do we have someone with capabilities in
> > this area on this list?
> >
> > PS  Martin you’ll like Fig. 2 of their paper.
> >
> > Wolfgang
> >
> >
> >
> >
> >
> >> On May 9, 2015, at 19:15 GMT+2, Martin Morgan
> >>  wrote:
> >>
> >> Bioc developers!
> >>
> >> It's important that our users be able to identify packages that
> >> are suitable for their research question. Obviously a first step
> >> is to identify packages in the appropriate research domain, for
> >> instance through biocViews.
> >>
> >>  http://bioconductor.org/packages/release/
> >>
> >> We'd like to help users further prioritize their efforts by
> >> summarizing use and usability. Metrics include:
> >>
> >> - Cross-platform availability -- biocLite()-able from all or only
> >> some platforms
> >> - Support forum activity -- questions and comments / responses, 6
> >> month window
> >> - Download percentile -- top 5, 20, 50%, or 'available'
> >> - Current build status -- errors or warnings on some or all
> >> platforms
> >> - Developer activity -- commits in the last 6 months
> >> - Historical presence -- years in Bioconductor
> >>
> >> Obviously the metrics are imperfect, so constructive feedback
> >> welcome -- we think the above capture in a more-or-less objective
> >> and computable way the major axes influencing use and usability.
> >>
> >> We initially intend to prominently display 'shields' (small
> >> graphical icons) on package landing pages.
> >>
> >> Thanks in advance for your comments,
> >>
> >> Martin Morgan
> >> Bioconductor
> >> --
> >> Computational Biology / Fred Hutchinson Cancer Research Center
> >> 1100 Fairview Ave. N.
> >> PO Box 19024 Seattle, WA 98109
> >>
> >> Location: Arnold Building M1 B861
> >> Phone: (206) 667-2793
> >>
> >> ___
> >> Bioc-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
> > ___
> > Bioc-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
> > ___
> > Bioc-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> 
> ___
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> 

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Use and Usability metrics / shields

2015-05-13 Thread Henrik Bengtsson 
Sweet; you went live with the badges/shields, e.g.

  http://bioconductor.org/packages/release/bioc/html/affxparser.html

A positive side effect is that now there's a link from the package
page to to the package's check results, which I always wanted :)

Thanks for adding this

/Henrik


On Sun, May 10, 2015 at 11:39 AM, COMMO Frederic
 wrote:
> Dear Martin,
>
> All of these suggestions sound good.
>
> Wolfgang's suggestion regarding possible associated papers might be also 
> great.
>
> Another useful information would be to point to other publications where a 
> given package was used, and cited.
> I don't know if it's technically possible, but it would be greatly 
> informative to know how frequently a package is used, and how it performs, in 
> real contexts.
>
> Frederic Commo
> Bioinformatics, U981
> Gustave Roussy
>
> 
> De : Bioc-devel [bioc-devel-boun...@r-project.org] de la part de Wolfgang 
> Huber [whu...@embl.de]
> Date d'envoi : samedi 9 mai 2015 19:57
> À : Martin Morgan
> Cc: bioc-devel@r-project.org
> Objet : Re: [Bioc-devel] Use and Usability metrics / shields
>
> Dear Martin
>
> great idea.
> "Current build status” could perhaps be wrapped with "Cross-platform 
> availability” into some sort of “Availability / Accessibility”?
>
> I wonder how informative it would be to make metrics such as
> (i) citations of the associated paper
> (ii) full-text mentions e.g. in PubmedCentral
> actually useful. (i) could be flawed if package and paper are diverged; (ii) 
> would require good disambiguation, e.g. like bioNerDS 
> http://www.biomedcentral.com/1471-2105/14/194 (or other tools? not my 
> expertise). Do we have someone with capabilities in this area on this list?
>
> PS  Martin you’ll like Fig. 2 of their paper.
>
> Wolfgang
>
>
>
>
>
>> On May 9, 2015, at 19:15 GMT+2, Martin Morgan  wrote:
>>
>> Bioc developers!
>>
>> It's important that our users be able to identify packages that are suitable 
>> for their research question. Obviously a first step is to identify packages 
>> in the appropriate research domain, for instance through biocViews.
>>
>>  http://bioconductor.org/packages/release/
>>
>> We'd like to help users further prioritize their efforts by summarizing use 
>> and usability. Metrics include:
>>
>> - Cross-platform availability -- biocLite()-able from all or only some 
>> platforms
>> - Support forum activity -- questions and comments / responses, 6 month 
>> window
>> - Download percentile -- top 5, 20, 50%, or 'available'
>> - Current build status -- errors or warnings on some or all platforms
>> - Developer activity -- commits in the last 6 months
>> - Historical presence -- years in Bioconductor
>>
>> Obviously the metrics are imperfect, so constructive feedback welcome -- we 
>> think the above capture in a more-or-less objective and computable way the 
>> major axes influencing use and usability.
>>
>> We initially intend to prominently display 'shields' (small graphical icons) 
>> on package landing pages.
>>
>> Thanks in advance for your comments,
>>
>> Martin Morgan
>> Bioconductor
>> --
>> Computational Biology / Fred Hutchinson Cancer Research Center
>> 1100 Fairview Ave. N.
>> PO Box 19024 Seattle, WA 98109
>>
>> Location: Arnold Building M1 B861
>> Phone: (206) 667-2793
>>
>> ___
>> Bioc-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
> ___
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
> ___
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Use and Usability metrics / shields

2015-05-13 Thread Dan Tenenbaum 


- Original Message -
> From: "Martin Morgan" 
> To: bioc-devel@r-project.org
> Sent: Saturday, May 9, 2015 10:15:13 AM
> Subject: [Bioc-devel] Use and Usability metrics / shields
> 
> Bioc developers!
> 
> It's important that our users be able to identify packages that are
> suitable for
> their research question. Obviously a first step is to identify
> packages in the
> appropriate research domain, for instance through biocViews.
> 
>http://bioconductor.org/packages/release/
> 
> We'd like to help users further prioritize their efforts by
> summarizing use and
> usability. Metrics include:
> 
> - Cross-platform availability -- biocLite()-able from all or only
> some platforms
> - Support forum activity -- questions and comments / responses, 6
> month window
> - Download percentile -- top 5, 20, 50%, or 'available'
> - Current build status -- errors or warnings on some or all platforms
> - Developer activity -- commits in the last 6 months
> - Historical presence -- years in Bioconductor
> 
> Obviously the metrics are imperfect, so constructive feedback welcome
> -- we
> think the above capture in a more-or-less objective and computable
> way the major
> axes influencing use and usability.
> 
> We initially intend to prominently display 'shields' (small graphical
> icons) on
> package landing pages.

This has now been implemented. You can see it at e.g.

http://bioconductor.org/packages/release/bioc/html/BiocGenerics.html

The shields are also available for use externally, for example if you want to 
display build status on a README in Github. Just pull in the image from:

http://bioconductor.org/shields/build/release/bioc/BiocGenerics.svg

Note that devel has a different build shield:

http://bioconductor.org/shields/build/devel/bioc/BiocGenerics.svg

Thanks,
Dan

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] exptData(SummarizedExperiment)

2015-05-13 Thread Vincent Carey 
On Wed, May 13, 2015 at 1:48 PM, Kasper Daniel Hansen <
kasperdanielhan...@gmail.com> wrote:

> the original workflow idea was exactly that it should go beyond a single
> package.
>

My question here was not about workflow intent.  It was about the vehicle
for maintaining a workflow.  Some
are created as documents only, and a package is generated from the
document; others are
authored as packages and the workflow document is a vignette of the
enclosing package (I think).

Making a valid package is a bit more involved than making a computable
workflow document from which a
package is autogenerated.


> Having domain specific cores might be controversial since we often have
> multiple packages competing in the same domain.  To some extent the
> GenomicRanges/Biostring/etc/etc is a special case of this, where "everyone"
> is using these packages.  To me it sounds too much like an official
> endorsement of a specific combination of packages.  With workflows, all one
> is saying is that "you can use package A,B,C to accomplish tasks 1,2,3";
> there is no official endorsement of a "winner".  I think this is worth
> thinking about: I think the project does benefit from multiple attempts at
> achieving the same result and the resulting competition it creates.
>

Fair point.  Competing workflows are also quite possible.  There's a
tension between simplicity for the user and complexity/cacophony induced by
allowing multiple solutions in the system.  I think the user gains more
from the complex
system even if it is a bit harder to "use".


>
>
> Best,
> Kasper
>
> On Tue, May 12, 2015 at 11:26 PM, Vincent Carey <
> st...@channing.harvard.edu> wrote:
>
>> Agreed that the workflow vehicle should get more attention.  Do all
>> workflows correspond to packages?
>>
>> On Tue, May 12, 2015 at 7:31 PM, Michael Lawrence <
>> lawrence.mich...@gene.com
>> > wrote:
>>
>> > I like the idea of having multiple, domain-specific cores. Those could
>> also
>> > serve as a vehicle for high-level documentation, including the workflows
>> > but also more "cheat-sheet" and/or cookbook-style documentation. Rafa
>> has
>> > brought this up on the phone calls.
>> >
>> >
>> > On Tue, May 12, 2015 at 4:10 PM, Hervé Pagès 
>> wrote:
>> >
>> > > SummarizedExperiment was just an example. I agree it can be a
>> > > little challenging for end users to know where to find a particular
>> > > functionality but I'm not sure about using "meta" packages to address
>> > > that. At least I feel we should probably avoid creating new "meta"
>> > > packages out of the blue, with arbitrary limits and possibly endless
>> > > discussions about what exactly goes in them. Also I don't think there
>> > > is a single "core" but rather several domain-specific cores.
>> > >
>> > > What about using the existing workflow packages instead?
>> > > A workflow package (like the variants package here
>> > > http://bioconductor.org/help/workflows/variants/)
>> > > covers a specific domain and loading it should load the "core"
>> > > for that domain. Plus the user gets a great vignette as a bonus
>> > > to get started so it's not just an empty shell.
>> > >
>> > > There are probably some shortcomings with workflow packages
>> > > that would need to be addressed before they can serve as
>> > > convenient "meta" packages though e.g. they're treated too
>> > > differently from other BioC packages (e.g. they're not available
>> > > via biocLite() and don't show up under the biocViews tree here
>> > > http://bioconductor.org/packages/release/BiocViews.html).
>> > > Nothing that seems impossible to address though...
>> > >
>> > > H.
>> > >
>> > >
>> > > On 05/12/2015 03:22 PM, Michael Lawrence wrote:
>> > >
>> > >> It's more general than SummarizedExperiment. I think people would
>> > >> appreciate a simple way to load the core, without having to remember,
>> > >> for example, that VCF reading is in VariantAnnotation.
>> > >>
>> > >> On Mon, May 11, 2015 at 9:51 PM, Hervé Pagès > > >> > wrote:
>> > >>
>> > >> Hi Michael,
>> > >>
>> > >> On 05/11/2015 05:35 PM, Michael Lawrence wrote:
>> > >>
>> > >> Splitting stuff into different packages is good for
>> modularity,
>> > >> but
>> > >> tough on the mind of the user. What about having some sort of
>> > >> "meta"
>> > >> package that simply loads the core infrastructure packages?
>> > Named
>> > >> something simple like "Genomics" or "GenomicsCore".
>> > >>
>> > >>
>> > >> Don't know if we need this. For example, for all the
>> > >> SummarizedExperiment use cases I ran into, the end-user generally
>> > >> only needs to load the corresponding high-level package (DESeq2,
>> > >> VariantAnnotation, minfi, GenomicAlignments, etc...) and that
>> takes
>> > >> care of loading all the low-level infrastructure packages.
>> > >>
>> > >> H.
>> > >>
>> > >>
>> > >> On Mon, May 11, 2015 at 5:10 PM, Hervé Pagès
>> > >> mailto:hpa...@fredhut

Re: [Bioc-devel] exptData(SummarizedExperiment)

2015-05-13 Thread Tim Triche, Jr. 
I personally don't care as long as I have a cubbyhole for unstructured data
and the name of that cubbyhole doesn't change every few weeks

hence the patch that I sent about 12 hours after complaining :-)


Statistics is the grammar of science.
Karl Pearson 

On Wed, May 13, 2015 at 10:48 AM, Kasper Daniel Hansen <
kasperdanielhan...@gmail.com> wrote:

> the original workflow idea was exactly that it should go beyond a single
> package.
>
> Having domain specific cores might be controversial since we often have
> multiple packages competing in the same domain.  To some extent the
> GenomicRanges/Biostring/etc/etc is a special case of this, where "everyone"
> is using these packages.  To me it sounds too much like an official
> endorsement of a specific combination of packages.  With workflows, all one
> is saying is that "you can use package A,B,C to accomplish tasks 1,2,3";
> there is no official endorsement of a "winner".  I think this is worth
> thinking about: I think the project does benefit from multiple attempts at
> achieving the same result and the resulting competition it creates.
>
> Best,
> Kasper
>
> On Tue, May 12, 2015 at 11:26 PM, Vincent Carey <
> st...@channing.harvard.edu>
> wrote:
>
> > Agreed that the workflow vehicle should get more attention.  Do all
> > workflows correspond to packages?
> >
> > On Tue, May 12, 2015 at 7:31 PM, Michael Lawrence <
> > lawrence.mich...@gene.com
> > > wrote:
> >
> > > I like the idea of having multiple, domain-specific cores. Those could
> > also
> > > serve as a vehicle for high-level documentation, including the
> workflows
> > > but also more "cheat-sheet" and/or cookbook-style documentation. Rafa
> has
> > > brought this up on the phone calls.
> > >
> > >
> > > On Tue, May 12, 2015 at 4:10 PM, Hervé Pagès 
> > wrote:
> > >
> > > > SummarizedExperiment was just an example. I agree it can be a
> > > > little challenging for end users to know where to find a particular
> > > > functionality but I'm not sure about using "meta" packages to address
> > > > that. At least I feel we should probably avoid creating new "meta"
> > > > packages out of the blue, with arbitrary limits and possibly endless
> > > > discussions about what exactly goes in them. Also I don't think there
> > > > is a single "core" but rather several domain-specific cores.
> > > >
> > > > What about using the existing workflow packages instead?
> > > > A workflow package (like the variants package here
> > > > http://bioconductor.org/help/workflows/variants/)
> > > > covers a specific domain and loading it should load the "core"
> > > > for that domain. Plus the user gets a great vignette as a bonus
> > > > to get started so it's not just an empty shell.
> > > >
> > > > There are probably some shortcomings with workflow packages
> > > > that would need to be addressed before they can serve as
> > > > convenient "meta" packages though e.g. they're treated too
> > > > differently from other BioC packages (e.g. they're not available
> > > > via biocLite() and don't show up under the biocViews tree here
> > > > http://bioconductor.org/packages/release/BiocViews.html).
> > > > Nothing that seems impossible to address though...
> > > >
> > > > H.
> > > >
> > > >
> > > > On 05/12/2015 03:22 PM, Michael Lawrence wrote:
> > > >
> > > >> It's more general than SummarizedExperiment. I think people would
> > > >> appreciate a simple way to load the core, without having to
> remember,
> > > >> for example, that VCF reading is in VariantAnnotation.
> > > >>
> > > >> On Mon, May 11, 2015 at 9:51 PM, Hervé Pagès  > > >> > wrote:
> > > >>
> > > >> Hi Michael,
> > > >>
> > > >> On 05/11/2015 05:35 PM, Michael Lawrence wrote:
> > > >>
> > > >> Splitting stuff into different packages is good for
> > modularity,
> > > >> but
> > > >> tough on the mind of the user. What about having some sort
> of
> > > >> "meta"
> > > >> package that simply loads the core infrastructure packages?
> > > Named
> > > >> something simple like "Genomics" or "GenomicsCore".
> > > >>
> > > >>
> > > >> Don't know if we need this. For example, for all the
> > > >> SummarizedExperiment use cases I ran into, the end-user
> generally
> > > >> only needs to load the corresponding high-level package (DESeq2,
> > > >> VariantAnnotation, minfi, GenomicAlignments, etc...) and that
> > takes
> > > >> care of loading all the low-level infrastructure packages.
> > > >>
> > > >> H.
> > > >>
> > > >>
> > > >> On Mon, May 11, 2015 at 5:10 PM, Hervé Pagès
> > > >> mailto:hpa...@fredhutch.org>
> > > >>  >>>
> > > >> wrote:
> > > >>
> > > >>  Hi Tim,
> > > >>
> > > >>  The SummarizedExperiment class is being replaced with
> the
> > > >>  RangedSummarizedExperiment class from the new
> > > >

Re: [Bioc-devel] exptData(SummarizedExperiment)

2015-05-13 Thread Kasper Daniel Hansen 
the original workflow idea was exactly that it should go beyond a single
package.

Having domain specific cores might be controversial since we often have
multiple packages competing in the same domain.  To some extent the
GenomicRanges/Biostring/etc/etc is a special case of this, where "everyone"
is using these packages.  To me it sounds too much like an official
endorsement of a specific combination of packages.  With workflows, all one
is saying is that "you can use package A,B,C to accomplish tasks 1,2,3";
there is no official endorsement of a "winner".  I think this is worth
thinking about: I think the project does benefit from multiple attempts at
achieving the same result and the resulting competition it creates.

Best,
Kasper

On Tue, May 12, 2015 at 11:26 PM, Vincent Carey 
wrote:

> Agreed that the workflow vehicle should get more attention.  Do all
> workflows correspond to packages?
>
> On Tue, May 12, 2015 at 7:31 PM, Michael Lawrence <
> lawrence.mich...@gene.com
> > wrote:
>
> > I like the idea of having multiple, domain-specific cores. Those could
> also
> > serve as a vehicle for high-level documentation, including the workflows
> > but also more "cheat-sheet" and/or cookbook-style documentation. Rafa has
> > brought this up on the phone calls.
> >
> >
> > On Tue, May 12, 2015 at 4:10 PM, Hervé Pagès 
> wrote:
> >
> > > SummarizedExperiment was just an example. I agree it can be a
> > > little challenging for end users to know where to find a particular
> > > functionality but I'm not sure about using "meta" packages to address
> > > that. At least I feel we should probably avoid creating new "meta"
> > > packages out of the blue, with arbitrary limits and possibly endless
> > > discussions about what exactly goes in them. Also I don't think there
> > > is a single "core" but rather several domain-specific cores.
> > >
> > > What about using the existing workflow packages instead?
> > > A workflow package (like the variants package here
> > > http://bioconductor.org/help/workflows/variants/)
> > > covers a specific domain and loading it should load the "core"
> > > for that domain. Plus the user gets a great vignette as a bonus
> > > to get started so it's not just an empty shell.
> > >
> > > There are probably some shortcomings with workflow packages
> > > that would need to be addressed before they can serve as
> > > convenient "meta" packages though e.g. they're treated too
> > > differently from other BioC packages (e.g. they're not available
> > > via biocLite() and don't show up under the biocViews tree here
> > > http://bioconductor.org/packages/release/BiocViews.html).
> > > Nothing that seems impossible to address though...
> > >
> > > H.
> > >
> > >
> > > On 05/12/2015 03:22 PM, Michael Lawrence wrote:
> > >
> > >> It's more general than SummarizedExperiment. I think people would
> > >> appreciate a simple way to load the core, without having to remember,
> > >> for example, that VCF reading is in VariantAnnotation.
> > >>
> > >> On Mon, May 11, 2015 at 9:51 PM, Hervé Pagès  > >> > wrote:
> > >>
> > >> Hi Michael,
> > >>
> > >> On 05/11/2015 05:35 PM, Michael Lawrence wrote:
> > >>
> > >> Splitting stuff into different packages is good for
> modularity,
> > >> but
> > >> tough on the mind of the user. What about having some sort of
> > >> "meta"
> > >> package that simply loads the core infrastructure packages?
> > Named
> > >> something simple like "Genomics" or "GenomicsCore".
> > >>
> > >>
> > >> Don't know if we need this. For example, for all the
> > >> SummarizedExperiment use cases I ran into, the end-user generally
> > >> only needs to load the corresponding high-level package (DESeq2,
> > >> VariantAnnotation, minfi, GenomicAlignments, etc...) and that
> takes
> > >> care of loading all the low-level infrastructure packages.
> > >>
> > >> H.
> > >>
> > >>
> > >> On Mon, May 11, 2015 at 5:10 PM, Hervé Pagès
> > >> mailto:hpa...@fredhutch.org>
> > >> >>
> > >> wrote:
> > >>
> > >>  Hi Tim,
> > >>
> > >>  The SummarizedExperiment class is being replaced with the
> > >>  RangedSummarizedExperiment class from the new
> > >> SummarizedExperiment
> > >>  package. This is a work-in-progress and the name and
> > internal
> > >>  representation of the RangedSummarizedExperiment class
> are
> > >> not
> > >>  finalized yet. The main goal for now is to move all the
> > >>  SummarizedExperiment stuff from GenomicRanges to its own
> > >> package.
> > >>
> > >>  Anyway, metadata() is the replacement for exptData() on
> > >>  RangedSummarizedExperiment objects. It's on my list to
> add
> > >>  an exptData method for backward compatibility.
> > >>
> > >>  Cheers,
> > >>