[
https://issues.apache.org/jira/browse/FLUME-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ralph Goers updated FLUME-3423:
-------------------------------
Fix Version/s: (was: 1.10.0)
> Taildir source will throw unnecessay exception when restart agent
> -----------------------------------------------------------------
>
> Key: FLUME-3423
> URL: https://issues.apache.org/jira/browse/FLUME-3423
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: 1.9.0
> Reporter: Jasper D
> Priority: Trivial
> Labels: patch
> Original Estimate: 5m
> Time Spent: 50m
> Remaining Estimate: 0h
>
> h2. *Phenomenon*
> When we use Taildir source, it will auto create an empty tail position file
> at start time. But if the filegroup directory is empty, it has no file to
> tail and then we restart agent, it will throw an EOFException. Although it
> has little effect on functionality, it will annoy users.
> h2. *Recur*
> This problem can esaily recur.
> We can use the example.conf from flume user guide directly.
> {code:java}
> //代码占位符
> a1.sources = r1
> a1.channels = c1
> a1.sources.r1.type = TAILDIR
> a1.sources.r1.channels = c1
> a1.sources.r1.positionFile = /var/log/flume/taildir_position.json
> a1.sources.r1.filegroups = f1
> a1.sources.r1.filegroups.f1 = /var/log/test1/*.log
> a1.sources.ri.maxBatchCount = 1000
> a1.channels.c1.type = memory
> a1.channels.c1.capacity = 1000
> a1.channels.c1.transactionCapacity = 100
> a1.sinks.k1.type = logger
> a1.sinks.k1.channel = c1{code}
> and start flume with it
> {code:java}
> //代码占位符
> bin/flume-ng agent --conf conf --conf-file conf/example.conf --name a1
> -Dflume.root.logger=INFO,console {code}
> then ctrl + c stop it.
> Just wait a moment, and then start flume again, then it will throw an
> exception like below.
> {code:java}
> //代码占位符
> ERROR [lifecycleSupervisor-1-1]
> (org.apache.flume.source.taildir.ReliableTaildirEventReader.loadPositionFile:147)
> - Failed loading positionFile: /var/log/flume/taildir_position.json
> java.io.EOFException: End of input at line 1 column 1
> at
> com.google.gson.stream.JsonReader.nextNonWhitespace(JsonReader.java:954)
> ~[gson-2.2.2.jar:?]
> at com.google.gson.stream.JsonReader.nextValue(JsonReader.java:771)
> ~[gson-2.2.2.jar:?]
> at com.google.gson.stream.JsonReader.peek(JsonReader.java:367)
> ~[gson-2.2.2.jar:?]
> at com.google.gson.stream.JsonReader.expect(JsonReader.java:337)
> ~[gson-2.2.2.jar:?]
> at com.google.gson.stream.JsonReader.beginArray(JsonReader.java:306)
> ~[gson-2.2.2.jar:?]
> at
> org.apache.flume.source.taildir.ReliableTaildirEventReader.loadPositionFile(ReliableTaildirEventReader.java:111)
> [flume-taildir-source-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at
> org.apache.flume.source.taildir.ReliableTaildirEventReader.<init>(ReliableTaildirEventReader.java:96)
> [flume-taildir-source-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at
> org.apache.flume.source.taildir.ReliableTaildirEventReader.<init>(ReliableTaildirEventReader.java:49)
> [flume-taildir-source-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at
> org.apache.flume.source.taildir.ReliableTaildirEventReader$Builder.build(ReliableTaildirEventReader.java:355)
> [flume-taildir-source-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at
> org.apache.flume.source.taildir.TaildirSource.start(TaildirSource.java:105)
> [flume-taildir-source-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at
> org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:71)
> [flume-ng-core-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
> [flume-ng-core-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0-292]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [?:1.8.0-292]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [?:1.8.0-292]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [?:1.8.0-292]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0-292]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0-292]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0-292] {code}
> h2. Fix
> We just add a condition in ReliableTaildirEventReader constructor.
> The pr/patch will be submitted as as shown below.
> Or let it still exist and using warning log instead of exception thrown is
> better?
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
