[
https://issues.apache.org/jira/browse/METAMODEL-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051704#comment-15051704
]
ASF GitHub Bot commented on METAMODEL-220:
------------------------------------------
GitHub user kaspersorensen opened a pull request:
https://github.com/apache/metamodel/pull/79
HdfsResource with non-"hdfs" scheme
Suggested fix for METAMODEL-220.
I based this PR on the branch that is represented in PR #78, so please
review that one first. There is only the last commit in the branch that adds to
the work in #78.
In this PR I've added a scheme to the HdfsResource. To make the scheme
dynamic I also needed to improve the URI parsing, which anyways ended up being
a very good improvement.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kaspersorensen/metamodel
METAMODEL-220-hadoop-filesystem-other-schemes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metamodel/pull/79.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #79
----
commit 14b63cdc262d40b0d6469de6cfc12103aa6239e3
Author: Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Date: 2015-12-10T20:59:29Z
Moved HDFS resource stream classes to separate files.
commit 1e5dfe34d65cb9880a8ccd0e6adaa3a765640527
Author: Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Date: 2015-12-10T21:06:09Z
Improved FileHelper.safeClose(...) method by using AutoCloseable
... which is now super-interface for Closeable, Connection, Statement,
ResultSet and more.
commit 7b5fb0c09c5dac7af3a2f2ff0e2aef8e0bbf2012
Author: Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Date: 2015-12-10T21:24:39Z
METAMODEL-219: Added loading of core-site.xml and hdfs-site.xml
commit 28537a1a0f5d40f67a64f902cdf3a32f0dfd5d3f
Author: Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Date: 2015-12-10T21:45:42Z
METAMODEL-220: Added dynamic schemes to HdfsResource
----
> Add dynamic scheme name to HdfsResource
> ---------------------------------------
>
> Key: METAMODEL-220
> URL: https://issues.apache.org/jira/browse/METAMODEL-220
> Project: Apache MetaModel
> Issue Type: Improvement
> Affects Versions: 4.4.1
> Reporter: Kasper Sørensen
>
> We should offer a way to use other scheme's than "hdfs" for Hadoop FileSystem
> based resources.
> NOTE: This issue is blocked by/dependent on METAMODEL-219 because the file
> system configuration would need to be loaded from configuration files.
> I have heard of at least a few variants that supposedly support the same API:
> * OpenStack Swift (swift://...)
> * Amazon EMR FS (emrfs://...)
> * MapR file system (maprfs://...)
> And there are probably more.
> Today "hdfs://" is hardcoded into HdfsResource and clearly it should be a
> field which the user can select a different value for. I suggest to keep it
> simply as a string and to also add a couple of constants in the class to make
> it easy for users to pick between the usual candidates.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
