[ https://issues.apache.org/jira/browse/KAFKA-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054129#comment-16054129 ]
Hongyuan Li commented on KAFKA-5444: ------------------------------------ I think this problem is associate with direct buffer allocation, use new producer api may ecountered similar problem which may not hang. I thought we can solve this in two steps: 1、move {{XX:+DisableExplicitGC}} from default {{kafka-run-class.sh}} 2、use {{sun.misc.cleaner}} to clean the directbuffer or use {{netty}} {{PlatformDependent}}#{{freeDirectBuffer}} to clean the direct buffer. Is that right? > Producer.send() will hang 8+ hours > ---------------------------------- > > Key: KAFKA-5444 > URL: https://issues.apache.org/jira/browse/KAFKA-5444 > Project: Kafka > Issue Type: Bug > Reporter: Hongyuan Li > > Frequent kafka old Producer open and close with cause the server hang with > lots of error messages logged in the server.log .In my occasion,we may > frequent open and close kafka producer,the procedure just like the code below: > {code} > Producer producer = …… > producer.send(List<KeyedMessage> lists); > producer.close(); > {code} > the error is below: > {code} > 2017-06-13 00:00:00,084] ERROR Processor got uncaught exception. > (kafka.network.Processor) > java.lang.OutOfMemoryError: Direct buffer memory > at java.nio.Bits.reserveMemory(Bits.java:658) > at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) > at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) > at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174) > at sun.nio.ch.IOUtil.read(IOUtil.java:195) > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) > at > org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110) > at > org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97) > at > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71) > at > org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154) > at > org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135) > at > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:343) > at org.apache.kafka.common.network.Selector.poll(Selector.java:291) > at kafka.network.Processor.poll(SocketServer.scala:476) > at kafka.network.Processor.run(SocketServer.scala:416) > at java.lang.Thread.run(Thread.java:745) > {code} > seen from all existing logs, all error is repeats of error above. > Any good idea to solve this? -- This message was sent by Atlassian JIRA (v6.4.14#64029)