[
https://issues.apache.org/jira/browse/HDFS-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544064#comment-14544064
]
Arpit Agarwal commented on HDFS-8392:
-------------------------------------
FsDatasetSpi is geared towards storing and retrieving files. In the object
store we want to be able to store and retrieve metadata containers and data
containers. Files may not be the best abstraction for these containers. For
these we'll introduce a StorageContainerDataset. We don't foresee a third
dataset type right now.
The DataNode already supports multiple block pools per storage volume and most
of the difficult work was done as part of the federation feature. It is
relatively straightforward to extend it to support the notion of a dataset per
block pool. So in a cluster running non-federated HDFS and Object store
services, the DataNodes would have two blockpools and two datasets, each
servicing one block pool.
Hope that's a little clearer. I intend to post a patch next week.
> DataNode support for multiple datasets
> --------------------------------------
>
> Key: HDFS-8392
> URL: https://issues.apache.org/jira/browse/HDFS-8392
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
>
> For HDFS-7240 we would like to share available DataNode storage across HDFS
> blocks and Ozone objects.
> The DataNode already supports sharing available storage across multiple block
> pool IDs for the federation feature. However all federated block pools use
> the same dataset implementation i.e. {{FsDatasetImpl}}.
> We can extend the DataNode to support multiple dataset implementations so the
> same storage space can be shared across one or more HDFS block pools and one
> or more Ozone block pools.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
