wangzheng created FLUME-3264:
--------------------------------
Summary: Null point using hdfs sink when flume startup faster than
hdfs
Key: FLUME-3264
URL: https://issues.apache.org/jira/browse/FLUME-3264
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: 1.8.0, 1.7.0
Environment: flume 1.8.0, hdfs 1.9.1.
Reporter: wangzheng
When flume starts up faster than hdfs, then hdfs sink will report NullPoint,
nothing will sink to hdfs. Patch have been tested in my dev env.
flume_1 | 2018-07-25 03:49:06,147
(SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN -
org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:582)] Caught
IOException writing to HDFSWriter (opt). Closing file
(hdfs://namenode:9000/flume/log00/201807/FlumeData.1532490542456.avro.tmp) and
rethrowing exception.
flume_1 | 2018-07-25 03:49:06,161
(SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO -
org.apache.flume.sink.hdfs.BucketWriter.close(BucketWriter.java:393)] Closing
hdfs://namenode:9000/flume/log00/201807/FlumeData.1532490542456.avro.tmp
flume_1 | 2018-07-25 03:49:06,164
(SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:447)]
process failed
flume_1 | java.lang.NullPointerException
flume_1 | at
org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187)
flume_1 | at
org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131)
flume_1 | at
org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327)
flume_1 | at
org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323)
flume_1 | at
org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701)
flume_1 | at
org.apache.flume.auth.SimpleAuthenticator.execute(SimpleAuthenticator.java:50)
flume_1 | at
org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:698)
flume_1 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
flume_1 | at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
flume_1 | at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
flume_1 | at java.lang.Thread.run(Thread.java:748)
flume_1 | 2018-07-25 03:49:06,164
(SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to
deliver event. Exception follows.
flume_1 | org.apache.flume.EventDeliveryException:
java.lang.NullPointerException
flume_1 | at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:451)
flume_1 | at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
flume_1 | at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
flume_1 | at java.lang.Thread.run(Thread.java:748)
flume_1 | Caused by: java.lang.NullPointerException
flume_1 | at
org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187)
flume_1 | at
org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131)
flume_1 | at
org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327)
flume_1 | at
org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323)
flume_1 | at
org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
