[
https://issues.apache.org/jira/browse/FLUME-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870981#comment-15870981
]
ASF subversion and git services commented on FLUME-2427:
--------------------------------------------------------
Commit 630083228364f6cd5dc18839fbd4b167cb615b38 in flume's branch
refs/heads/trunk from [~wpwang]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=6300832 ]
FLUME-2427. Be less verbose in isFileClosed() notification
Some versions of HDFS and implementations of FileSystem do not implement
the isFileClosed() method.
We perform a check for that method each time a new file is opened. This
patch lowers the severity of the message printed when we detect that the
method is not present from LOG.warn() to LOG.info().
We also remove the stack trace from the log message.
(Ping Wang via Mike Percy)
> java.lang.NoSuchMethodException and warning on HDFS (S3) sink
> --------------------------------------------------------------
>
> Key: FLUME-2427
> URL: https://issues.apache.org/jira/browse/FLUME-2427
> Project: Flume
> Issue Type: Question
> Reporter: Bijith Kumar
> Assignee: Ping Wang
> Priority: Minor
> Fix For: v1.8.0
>
> Attachments: FLUME-2427-0.patch
>
>
> The below warning and Exception is thrown every time a file is written to S3
> using HDFS sink. Looks like a jar mismatch to me. Tried latest hadoop and
> jets3 jars but didn't work
> 17 Jul 2014 23:30:18,159 INFO [hdfs-s3sink-engagements-call-runner-6]
> (org.apache.flume.sink.hdfs.AbstractHDFSWriter.reflectGetNumCurrentReplicas:184)
> - FileSystem's output stream doesn't support getNumCurrentReplicas;
> --HDFS-826 not available;
> fsOut=org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream;
> err=java.lang.NoSuchMethodException:
> org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.getNumCurrentReplicas()
> 17 Jul 2014 23:30:18,160 WARN
> [SinkRunner-PollingRunner-DefaultSinkProcessor]
> (org.apache.flume.sink.hdfs.BucketWriter.getRefIsClosed:210) - isFileClosed
> is not available in the version of HDFS being used. Flume will not attempt to
> close files if the close fails on the first attempt
> java.lang.NoSuchMethodException:
> org.apache.hadoop.fs.s3native.NativeS3FileSystem.isFileClosed(org.apache.hadoop.fs.Path)
> at java.lang.Class.getMethod(Class.java:1665)
> at
> org.apache.flume.sink.hdfs.BucketWriter.getRefIsClosed(BucketWriter.java:207)
> at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:295)
> at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:554)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:426)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:744)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
