Hi Ray,
H5serv doesn’t work directly with S3 storage since the HDF5 library is
expecting a Posix file system. Using something like s3fuse
(https://github.com/s3fs-fuse/s3fs-fuse) may work for read-only access to HDF5
files stored as S3 objects, but I haven’t tried this.
Anyway, as it happens, we’re currently working on a project for NASA to
enable HDF Server to use object storage directly. The goal is to enable the
HDF api across mutii-PB datasets hosted on S3. We’re about 6 months into the 2
year project, but it’s looking quite promising.
I gave a talk at the summer ESIP meeting regarding this. You can see the
slides at: http://www.slideshare.net/HDFEOS/hdf-cloud-services. The first part
is a review of HDF Server progress to-date. Slide 22 on talks about the object
storage work.
Cheers,
John
From: Hdf-forum <[email protected]> on behalf of Ray
Courtney <[email protected]>
Reply-To: HDF Users Discussion List <[email protected]>
Date: Tuesday, February 14, 2017 at 12:39 PM
To: "[email protected]" <[email protected]>
Subject: [Hdf-forum] h5serv and S3
Hiya,
We have a repository of HDF5 files stored in Amazon S3 buckets, and we'd like
to use h5serv to work with these files.
I tried googling but didn't find any implementations of this.
Has anyone has any luck standing up h5serv in front of S3 storage instead of
the local data directory?
Any chance of this becoming a future enhancement for the h5serv project?
Thanks,
RAY
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
