[
https://issues.apache.org/jira/browse/METAMODEL-220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kasper Sørensen updated METAMODEL-220:
--------------------------------------
Description:
We should offer a way to use other scheme's than "hdfs" for Hadoop FileSystem
based resources.
NOTE: This issue is blocked by/dependent on METAMODEL-219 because the file
system configuration would need to be loaded from configuration files.
I have heard of at least a few variants that supposedly support the same API:
* OpenStack Swift (swift://...)
* Amazon EMR FS (emrfs://...)
* MapR file system (maprfs://...)
And there are probably more.
Today "hdfs://" is hardcoded into HdfsResource and clearly it should be a field
which the user can select a different value for. I suggest to keep it simply as
a string and to also add a couple of constants in the class to make it easy for
users to pick between the usual candidates.
was:
We should offer a way to use other scheme's than "hdfs" for Hadoop FileSystem
based resources.
NOTE: This issue is blocked by/dependent on METAMODEL-219
I have heard of at least a few variants that supposedly support the same API:
* OpenStack Swift (swift://...)
* Amazon EMR FS (emrfs://...)
* MapR file system (maprfs://...)
And there are probably more.
Today "hdfs://" is hardcoded into HdfsResource and clearly it should be a field
which the user can select a different value for. I suggest to keep it simply as
a string and to also add a couple of constants in the class to make it easy for
users to pick between the usual candidates.
> Add dynamic scheme name to HdfsResource
> ---------------------------------------
>
> Key: METAMODEL-220
> URL: https://issues.apache.org/jira/browse/METAMODEL-220
> Project: Apache MetaModel
> Issue Type: Improvement
> Affects Versions: 4.4.1
> Reporter: Kasper Sørensen
>
> We should offer a way to use other scheme's than "hdfs" for Hadoop FileSystem
> based resources.
> NOTE: This issue is blocked by/dependent on METAMODEL-219 because the file
> system configuration would need to be loaded from configuration files.
> I have heard of at least a few variants that supposedly support the same API:
> * OpenStack Swift (swift://...)
> * Amazon EMR FS (emrfs://...)
> * MapR file system (maprfs://...)
> And there are probably more.
> Today "hdfs://" is hardcoded into HdfsResource and clearly it should be a
> field which the user can select a different value for. I suggest to keep it
> simply as a string and to also add a couple of constants in the class to make
> it easy for users to pick between the usual candidates.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
