[
https://issues.apache.org/jira/browse/HDFS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162942#comment-13162942
]
Eli Collins commented on HDFS-2178:
-----------------------------------
@Tucu,
Looking good.
Sanjay's suggestion of having it be a pure proxy (forward operations to
webhdfs) makes a lot of sense to me, plan to do that?
I didn't look at param* and fsoperations* since I checked out a previous
version and they mostly look identical in the latest patch. It would be easier
to get a sense of all the concrete params and fsoperations classes lived in a
single file, perhaps as nested classes?
Where is webhdfs/httpfs interoperability tested? Seems like a good test would
be to send streams of operations through a tee to both and assert the final
contents of the file system are identical.
Config keys should live in DFSConfigKeys next to the WebHDFS ones, eg the
various "authentication.kerberos.keytab" defines.
Could webhdfs and httpfs share config keys? User-visible stuff like config we
should share asap so we don't break people when introducing the sharing. Feel
free to handle in a follow-on jira.
File a jira for code sharing with WebHDFS?
All the config keys should be covered in ServerSetup.apt.vm right? Is the info
there sufficient to setup a proxy w security enabled?
Nit: remove commented out code, eg TestWebHDFSFileSytem#operations
Nit: Why not re-use/update core's StringUtils instead of introducing a new one?
Thanks,
Eli
> Contributing Hoop to HDFS, replacement for HDFS proxy with read/write
> capabilities
> ----------------------------------------------------------------------------------
>
> Key: HDFS-2178
> URL: https://issues.apache.org/jira/browse/HDFS-2178
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.24.0, 0.23.1
>
> Attachments: HDFS-2178.patch, HDFS-2178.patch, HDFS-2178.patch,
> HDFS-2178.patch, HDFS-2178.patch, HDFSoverHTTP-API.html, HdfsHttpAPI.pdf
>
>
> We'd like to contribute Hoop to Hadoop HDFS as a replacement (an improvement)
> for HDFS Proxy.
> Hoop provides access to all Hadoop Distributed File System (HDFS) operations
> (read and write) over HTTP/S.
> The Hoop server component is a REST HTTP gateway to HDFS supporting all file
> system operations. It can be accessed using standard HTTP tools (i.e. curl
> and wget), HTTP libraries from different programing languages (i.e. Perl,
> Java Script) as well as using the Hoop client. The Hoop server component is a
> standard Java web-application and it has been implemented using Jersey
> (JAX-RS).
> The Hoop client component is an implementation of Hadoop FileSystem client
> that allows using the familiar Hadoop filesystem API to access HDFS data
> through a Hoop server.
> Repo: https://github.com/cloudera/hoop
> Docs: http://cloudera.github.com/hoop
> Blog: http://www.cloudera.com/blog/2011/07/hoop-hadoop-hdfs-over-http/
> Hoop is a Maven based project that depends on Hadoop HDFS and Alfredo (for
> Kerberos HTTP SPNEGO authentication).
> To make the integration easy, HDFS Mavenization (HDFS-2096) would have to be
> done first, as well as the Alfredo contribution (HADOOP-7119).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
