[
https://issues.apache.org/jira/browse/FLUME-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870978#comment-15870978
]
ASF subversion and git services commented on FLUME-2427:
--------------------------------------------------------
Commit 9604c5ae1c9f2b266200889219b5604fe0e756e4 in flume's branch
refs/heads/trunk from [~wpwang]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=9604c5a ]
FLUME-2427. Be less verbose in isFileClosed() notification
Some versions of HDFS and implementations of Filesystem do not implement
the isFileClosed() method.
We perform a check for that method each time a new file is opened. This
patch lowers the severity of the message printed when we detect that the
method id not present from LOG.warn() to LOG.info().
We also remove the stack trace from the log message.
(Ping Wang via Mike Percy)
> java.lang.NoSuchMethodException and warning on HDFS (S3) sink
> --------------------------------------------------------------
>
> Key: FLUME-2427
> URL: https://issues.apache.org/jira/browse/FLUME-2427
> Project: Flume
> Issue Type: Question
> Reporter: Bijith Kumar
> Priority: Minor
> Attachments: FLUME-2427-0.patch
>
>
> The below warning and Exception is thrown every time a file is written to S3
> using HDFS sink. Looks like a jar mismatch to me. Tried latest hadoop and
> jets3 jars but didn't work
> 17 Jul 2014 23:30:18,159 INFO [hdfs-s3sink-engagements-call-runner-6]
> (org.apache.flume.sink.hdfs.AbstractHDFSWriter.reflectGetNumCurrentReplicas:184)
> - FileSystem's output stream doesn't support getNumCurrentReplicas;
> --HDFS-826 not available;
> fsOut=org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream;
> err=java.lang.NoSuchMethodException:
> org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.getNumCurrentReplicas()
> 17 Jul 2014 23:30:18,160 WARN
> [SinkRunner-PollingRunner-DefaultSinkProcessor]
> (org.apache.flume.sink.hdfs.BucketWriter.getRefIsClosed:210) - isFileClosed
> is not available in the version of HDFS being used. Flume will not attempt to
> close files if the close fails on the first attempt
> java.lang.NoSuchMethodException:
> org.apache.hadoop.fs.s3native.NativeS3FileSystem.isFileClosed(org.apache.hadoop.fs.Path)
> at java.lang.Class.getMethod(Class.java:1665)
> at
> org.apache.flume.sink.hdfs.BucketWriter.getRefIsClosed(BucketWriter.java:207)
> at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:295)
> at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:554)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:426)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:744)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
