Re: [R] Use case for HDF5 dataspace interface

[Michael Sumner]

Not sure about your questions  but I use rhdf5 to  read statistical daily
bins from remote sensing ocean colour products (L3bin)

https://github.com/AustralianAntarcticDivision/ocean_colour/blob/master/modis_daily_bins_init.R#L31

I'm working on database-like exploratory tools for HDF5 too, this is poorly
supported in R partly because  of patchy access to the underlying systems
libraries on CRAN. But,  ncdf4 is built cross platform and supports many
models (not L3bin,  because it lacks support  for groups and compound
types).

Cheers, Mike



On Thu, 3 Aug 2017, 18:39 Koustav Pal,  wrote:

> 1. This relates to the package *rhdf5* and its implementation of the HDF5
> dataspace interface. I am asking for an example of how other people who use
> this package make use of the HDF5 data space interface exposed by the
> library.
>
>
>
> Longer answer:
>
> As per my understanding, the dataspace interface exposes data locations
> within a dataspace, but even while retrieving data from an hdf5 file using
> methods implemented within rhdf5, the methods need to convert the requested
> data retrieval call to dataspace locations. So what is the usefulness in
> using the dataspace interface, and can I see this usefulness in a code
> example using the rhdf5 dataspace interface.
>
>
> 2. Not related to comp bio.
>
>
>
>
> ---
> Koustav Pal,
> PhD student in Computational Biology,
> Francesco Ferrari's group,
> IFOM - The FIRC Institute of Molecular Oncology
> Milan, Italy.
>
> On 1 August 2017 at 16:30, Bert Gunter  wrote:
>
> > 1. What does this have to do with R?
> >
> > 2. If it concerns computational biology, the Bioconductor Help list
> > may be a better place to post.
> >
> >
> > Cheers,
> > Bert
> >
> >
> > Bert Gunter
> >
> > "The trouble with having an open mind is that people keep coming along
> > and sticking things into it."
> > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
> >
> >
> > On Tue, Aug 1, 2017 at 2:28 AM, Koustav Pal 
> > wrote:
> > > This question is a clone of my stackoverflow question which never got
> > > answered (o_O). Therefore I am posting it here. I would really like
> some
> > > inputs if possible.
> > >
> > > I am currently building some applications which make use of HDF5 files.
> > >
> > > I have already taken a look at the hdfgroup website with regards to
> > > dataspace <
> https://support.hdfgroup.org/HDF5/doc/H5.user/Dataspaces.html
> > >
> > > and I think I understand the concept. But I am very much unable to
> > > understand it's real world use.
> > >
> > > Can someone please, tell me what is the dataspace interface supposed to
> > be
> > > used for?
> > >
> > > Currently, I think if I load a matrix of size 1.5M x 1.5M I may be able
> > to
> > > store dataspace coordinates and then retrieve that piece of data much
> > > faster. Is this correct?
> > >
> > > It would be great if you can provide some example use cases.
> > >
> > >
> > > Link to original question:
> > > https://stackoverflow.com/questions/44697599/hdf5-
> > dataspace-interface-what-does-it-do-and-what-is-its-real-world-applicati
> > >
> > > 
> > ---
> > > Koustav Pal,
> > > PhD student in Computational Biology,
> > > Francesco Ferrari's group,
> > > IFOM - The FIRC Institute of Molecular Oncology
> > > Milan, Italy.
> > >
> > > [[alternative HTML version deleted]]
> > >
> > > __
> > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide http://www.R-project.org/
> > posting-guide.html
> > > and provide commented, minimal, self-contained, reproducible code.
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
-- 
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Use case for HDF5 dataspace interface

[Koustav Pal]

1. This relates to the package *rhdf5* and its implementation of the HDF5
dataspace interface. I am asking for an example of how other people who use
this package make use of the HDF5 data space interface exposed by the
library.

Longer answer:

As per my understanding, the dataspace interface exposes data locations
within a dataspace, but even while retrieving data from an hdf5 file using
methods implemented within rhdf5, the methods need to convert the requested
data retrieval call to dataspace locations. So what is the usefulness in
using the dataspace interface, and can I see this usefulness in a code
example using the rhdf5 dataspace interface.


2. Not related to comp bio.



---
Koustav Pal,
PhD student in Computational Biology,
Francesco Ferrari's group,
IFOM - The FIRC Institute of Molecular Oncology
Milan, Italy.

On 1 August 2017 at 16:30, Bert Gunter  wrote:

> 1. What does this have to do with R?
>
> 2. If it concerns computational biology, the Bioconductor Help list
> may be a better place to post.
>
>
> Cheers,
> Bert
>
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along
> and sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Tue, Aug 1, 2017 at 2:28 AM, Koustav Pal 
> wrote:
> > This question is a clone of my stackoverflow question which never got
> > answered (o_O). Therefore I am posting it here. I would really like some
> > inputs if possible.
> >
> > I am currently building some applications which make use of HDF5 files.
> >
> > I have already taken a look at the hdfgroup website with regards to
> > dataspace  >
> > and I think I understand the concept. But I am very much unable to
> > understand it's real world use.
> >
> > Can someone please, tell me what is the dataspace interface supposed to
> be
> > used for?
> >
> > Currently, I think if I load a matrix of size 1.5M x 1.5M I may be able
> to
> > store dataspace coordinates and then retrieve that piece of data much
> > faster. Is this correct?
> >
> > It would be great if you can provide some example use cases.
> >
> >
> > Link to original question:
> > https://stackoverflow.com/questions/44697599/hdf5-
> dataspace-interface-what-does-it-do-and-what-is-its-real-world-applicati
> >
> > 
> ---
> > Koustav Pal,
> > PhD student in Computational Biology,
> > Francesco Ferrari's group,
> > IFOM - The FIRC Institute of Molecular Oncology
> > Milan, Italy.
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Use case for HDF5 dataspace interface

[Bert Gunter]

1. What does this have to do with R?

2. If it concerns computational biology, the Bioconductor Help list
may be a better place to post.


Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Aug 1, 2017 at 2:28 AM, Koustav Pal  wrote:
> This question is a clone of my stackoverflow question which never got
> answered (o_O). Therefore I am posting it here. I would really like some
> inputs if possible.
>
> I am currently building some applications which make use of HDF5 files.
>
> I have already taken a look at the hdfgroup website with regards to
> dataspace 
> and I think I understand the concept. But I am very much unable to
> understand it's real world use.
>
> Can someone please, tell me what is the dataspace interface supposed to be
> used for?
>
> Currently, I think if I load a matrix of size 1.5M x 1.5M I may be able to
> store dataspace coordinates and then retrieve that piece of data much
> faster. Is this correct?
>
> It would be great if you can provide some example use cases.
>
>
> Link to original question:
> https://stackoverflow.com/questions/44697599/hdf5-dataspace-interface-what-does-it-do-and-what-is-its-real-world-applicati
>
> ---
> Koustav Pal,
> PhD student in Computational Biology,
> Francesco Ferrari's group,
> IFOM - The FIRC Institute of Molecular Oncology
> Milan, Italy.
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Use case for HDF5 dataspace interface

[Koustav Pal]

This question is a clone of my stackoverflow question which never got
answered (o_O). Therefore I am posting it here. I would really like some
inputs if possible.

I am currently building some applications which make use of HDF5 files.

I have already taken a look at the hdfgroup website with regards to
dataspace 
and I think I understand the concept. But I am very much unable to
understand it's real world use.

Can someone please, tell me what is the dataspace interface supposed to be
used for?

Currently, I think if I load a matrix of size 1.5M x 1.5M I may be able to
store dataspace coordinates and then retrieve that piece of data much
faster. Is this correct?

It would be great if you can provide some example use cases.


Link to original question:
https://stackoverflow.com/questions/44697599/hdf5-dataspace-interface-what-does-it-do-and-what-is-its-real-world-applicati

---
Koustav Pal,
PhD student in Computational Biology,
Francesco Ferrari's group,
IFOM - The FIRC Institute of Molecular Oncology
Milan, Italy.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.