[
https://issues.apache.org/jira/browse/FLUME-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231810#comment-13231810
]
[email protected] commented on FLUME-1020:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4360/
-----------------------------------------------------------
(Updated 2012-03-17 02:30:51.378131)
Review request for Flume.
Changes
-------
Got this working. Please take a look.
For now, I am not throwing if authentication fails.
Also, based on speaking with folks familiar with HDFS, it turns out that in
order to communicate with a secure cluster one must have the hadoop config
directory on the classpath. This is due to some static variables being used to
keep track of settings & states related to the UserGroupInformation class. So,
I am looking for Hadoop environment variables in bin/flume-ng.
Tested this against a Kerberized Hadoop cluster running in a VM (Centos 6)
using MIT Kerberos on my laptop.
Summary
-------
This is an initial pass at an implementation of HDFS security. I think it will
probably work. Currently trying to get Kerberos to play nice with the cluster
on my VM though, so I haven't successfully tested it yet. It still works when
used on HDFS with security disabled. :)
The only thing I don't like is in configure() when authentication fails I throw
a FlumeException. I'll trace up and see how bad that would be but it seems
likely to break something. Just logging the error is kind of a bummer as well,
though ... need to ensure process() doesn't fill up the disk while spewing
copious error messages into the logs. Maybe this is a use case for some kind of
FatalException type thing.
This addresses bug FLUME-1020.
https://issues.apache.org/jira/browse/FLUME-1020
Diffs (updated)
-----
bin/flume-ng 0796a5b
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
da82f7e
Diff: https://reviews.apache.org/r/4360/diff
Testing
-------
Thanks,
Mike
> Implement Kerberos security for HDFS Sink
> -----------------------------------------
>
> Key: FLUME-1020
> URL: https://issues.apache.org/jira/browse/FLUME-1020
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Assignee: Mike Percy
>
> Make flume HDFS sink work with secure clusters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
