[
https://issues.apache.org/jira/browse/FLUME-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099933#comment-16099933
]
ASF subversion and git services commented on FLUME-2620:
--------------------------------------------------------
Commit 1e69fc7c29f104a2117a62de11cba9b2a2c740e1 in flume's branch
refs/heads/trunk from [~marcellhegedus]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=1e69fc7 ]
FLUME-2620. File Channel to support empty values in headers
Flume user guide does not specify whether a value in event header could be null
or not.
Given an external system generating events which header values can be null and
a user configures
Flume with Memory Channel then he will have no trouble.
Later on when the user changes Memory Channel to File Channel then Flume will
fail with NPE.
It is because FC is serializing events with protocol buffer and header values
are defined as
required in the proto file.
In this patch I have changed the value field to optional. However protocol
buffer does not have
a notation for null and setting a field to null raises NPE again. Added a null
check before
serialization to prevent this.
There is on caveat: When an optional field is not set, at deserialization it
will be set to a
default value: in this case it will be empty string.
Reviewers: Miklos Csanady
(Marcell Hegedus via Denes Arvay)
> File channel throws NullPointerException if a header value is null
> ------------------------------------------------------------------
>
> Key: FLUME-2620
> URL: https://issues.apache.org/jira/browse/FLUME-2620
> Project: Flume
> Issue Type: Bug
> Components: File Channel
> Reporter: Santiago M. Mola
> Assignee: Marcell Hegedus
> Attachments: FLUME-2620-0.patch, FLUME-2620-1.patch,
> FLUME-2620-2.patch, FLUME-2620-3.patch, FLUME-2620-4.patch,
> FLUME-2620-5.patch, FLUME-2620.patch, FLUME-2620.patch
>
>
> File channel throws NullPointerException if a header value is null.
> If this is intended, it should be reported correctly in the logs.
> Sample trace:
> org.apache.flume.ChannelException: Unable to put batch on required channel:
> FileChannel chan { dataDirs: [/var/lib/ingestion-csv/chan/data] }
> at
> org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
> at
> org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:236)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at
> org.apache.flume.channel.file.proto.ProtosFactory$FlumeEventHeader$Builder.setValue(ProtosFactory.java:7415)
> at org.apache.flume.channel.file.Put.writeProtos(Put.java:85)
> at
> org.apache.flume.channel.file.TransactionEventRecord.toByteBuffer(TransactionEventRecord.java:174)
> at org.apache.flume.channel.file.Log.put(Log.java:622)
> at
> org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doPut(FileChannel.java:469)
> at
> org.apache.flume.channel.BasicTransactionSemantics.put(BasicTransactionSemantics.java:93)
> at
> org.apache.flume.channel.BasicChannelSemantics.put(BasicChannelSemantics.java:80)
> at
> org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:189)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
