[
https://issues.apache.org/jira/browse/FLUME-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14957868#comment-14957868
]
Rollin Crittendon commented on FLUME-2812:
------------------------------------------
This is what we have, with machine names, ports, some things like that
redacted, for example, <ZOOKEEPER-SETTINGS>.
=============================
bridge.channels = bridge-channel
bridge.sources = spool-source kafka-source
bridge.sinks = to-conduit-sink
bridge.channels.bridge-channel.type = memory
bridge.channels.bridge-channel.capacity = 100000
bridge.channels.bridge-channel.transactionCapacity = 40000
bridge.channels.bridge-channel.byteCapacityBufferPercentage = 20
bridge.channels.bridge-channel.byteCapacity = 0
bridge.sources.spool-source.channels = bridge-channel
bridge.sources.spool-source.type = spooldir
bridge.sources.spool-source.spoolDir = <SPOOL-DIRECTORY-LOCATION>
bridge.sources.spool-source.fileHeader = false
bridge.sources.spool-source.deletePolicy = immediate
bridge.sources.spool-source.batchSize = 100
bridge.sources.spool-source.deserializer.maxLineLength = 100000
bridge.sources.kafka-source.channels = bridge-channel
bridge.sources.kafka-source.type = org.apache.flume.source.kafka.KafkaSource
bridge.sources.kafka-source.zookeeperConnect = <ZOOKEEPER-SETTINGS>
bridge.sources.kafka-source.groupId = <GROUP-ID>
bridge.sources.kafka-source.topic = <TOPIC>
bridge.sources.kafka-source.batchSize = 100
bridge.sinks.to-conduit-sink.channel = bridge-channel
bridge.sinks.to-conduit-sink.type = thrift
bridge.sinks.to-conduit-sink.hostname = <OTHER-AGENT-MACHINE-NAME>
bridge.sinks.to-conduit-sink.port = <OTHER-AGENT-MACHINE-PORT>
bridge.sinks.to-conduit-sink.batch-size = 1000
> Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor"
> java.lang.Error: Maximum permit count exceeded
> ------------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-2812
> URL: https://issues.apache.org/jira/browse/FLUME-2812
> Project: Flume
> Issue Type: Bug
> Components: Channel, Sinks+Sources
> Affects Versions: v1.6.0
> Environment: **OS INFO**
> CentOS release 6.6 (Final)
> Kernel \r on an \m
> **JAVA INFO**
> java version "1.8.0_40"
> Java(TM) SE Runtime Environment (build 1.8.0_40-b26)
> Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
> Reporter: Rollin Crittendon
> Priority: Critical
>
> We are finding that around after an hour or so of heavy processing of Flume
> data in an agent we are getting the following exception. This is after
> processing about 5-7 k lines/second during that time.
> The configuration of this agent is using a Kafka source, the one that comes
> with 1.6.0.
> It is also using a Memory channel, and a Thrift sink.
> =======
> Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor"
> java.lang.Error: Maximum permit count exceeded
> at
> java.util.concurrent.Semaphore$Sync.tryReleaseShared(Semaphore.java:192)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1341)
> at java.util.concurrent.Semaphore.release(Semaphore.java:609)
> at
> org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:147)
> at
> org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
> at
> org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:379)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:745)
> =======
> The above error is from standard error when running the Flume agent. The
> effect is that the "SinkRunner-PollingRunner-DefaultSinkProcessor" thread
> disappears from the agent, this can be seen on a JMX console.
> For us, this means that the Flume agent needs to get restarted. It is an
> error that is terminal in that instance of the Java process due to the thread
> disappearing as a result.
> It sounds like something in JDK 7+ got stricter?!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
