Wei-Chiu Chuang created FLUME-3146:
--------------------------------------
Summary: Use public API
HdfsDataOutputStream#getCurrentBlockReplication where applicable
Key: FLUME-3146
URL: https://issues.apache.org/jira/browse/FLUME-3146
Project: Flume
Issue Type: Bug
Reporter: Wei-Chiu Chuang
AbstractHDFSWrite#reflectGetNumCurrentReplicas uses a private HDFS API, which
causes it to emit an confusing error message:
{noformat}
2017-08-08 22:01:10,743 INFO org.apache.flume.sink.hdfs.AbstractHDFSWriter:
FileSystem's output stream doesn't support getNumCurrentReplicas; --HDFS-826
not available; fsOut=org.apache.hadoop.crypto.CryptoOutputStream;
err=java.lang.NoSuchMethodException:
org.apache.hadoop.crypto.CryptoOutputStream.getNumCurrentReplicas()
2017-08-08 22:02:11,086 ERROR org.apache.flume.sink.hdfs.AbstractHDFSWriter:
Error while trying to hflushOrSync!
{noformat}
This error message is emitted even with the most recent version of HDFS that
has HDFS-826 for sure.
Turns out that the relevant code is not valid with HDFS encryption. Instead of
using the current private (and deprecated)
getNumCurrentReplicas#getNumCurrentReplicas() API, it should use
HdfsDataOutputStream#getCurrentBlockReplication, which is a public API and
which considers the case of HDFS encryption.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
