[ https://issues.apache.org/jira/browse/KAFKA-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Petrovsky updated KAFKA-5814: --------------------------------------- Description: Hello! We have in our production - kafka_2.12-0.10.2.1 - # java -version openjdk version "1.8.0_131" OpenJDK Runtime Environment (build 1.8.0_131-b12) OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode) And we had strange behavior with kafka which result to big outage: {code:java} авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: [2017-08-30 21:17:19,266] ERROR Processor got uncaught exception. (kafka.network.Processor) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: java.lang.OutOfMemoryError: Direct buffer memory авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.nio.Bits.reserveMemory(Bits.java:694) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at sun.nio.ch.IOUtil.read(IOUtil.java:195) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:355) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.Selector.poll(Selector.java:303) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at kafka.network.Processor.poll(SocketServer.scala:494) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at kafka.network.Processor.run(SocketServer.scala:432) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.lang.Thread.run(Thread.java:748) {code} Only kafka cluster restart helps to fix this problem. I wonder, why this problem have take a place, cause we running kafka with follow options Environment="KAFKA_HEAP_OPTS=-Xmx5G -Xms5G -XX:MaxDirectMemorySize=2G" was: Hello! We have in our production - kafka_2.12-0.10.2.1 - # java -version openjdk version "1.8.0_131" OpenJDK Runtime Environment (build 1.8.0_131-b12) OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode) And we had strange behavior with kafka with result to big outage: {code:java} авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: [2017-08-30 21:17:19,266] ERROR Processor got uncaught exception. (kafka.network.Processor) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: java.lang.OutOfMemoryError: Direct buffer memory авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.nio.Bits.reserveMemory(Bits.java:694) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at sun.nio.ch.IOUtil.read(IOUtil.java:195) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:355) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at org.apache.kafka.common.network.Selector.poll(Selector.java:303) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at kafka.network.Processor.poll(SocketServer.scala:494) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at kafka.network.Processor.run(SocketServer.scala:432) авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at java.lang.Thread.run(Thread.java:748) {code} Only kafka cluster restart helps to fix this problem. I wonder, why this problem have take a place, cause we running kafka with follow options Environment="KAFKA_HEAP_OPTS=-Xmx5G -Xms5G -XX:MaxDirectMemorySize=2G" > java.lang.OutOfMemoryError: Direct buffer memory > ------------------------------------------------ > > Key: KAFKA-5814 > URL: https://issues.apache.org/jira/browse/KAFKA-5814 > Project: Kafka > Issue Type: Bug > Components: core > Reporter: Alexander Petrovsky > > Hello! > We have in our production > - kafka_2.12-0.10.2.1 > - # java -version > openjdk version "1.8.0_131" > OpenJDK Runtime Environment (build 1.8.0_131-b12) > OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode) > And we had strange behavior with kafka which result to big outage: > {code:java} > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: [2017-08-30 > 21:17:19,266] ERROR Processor got uncaught exception. > (kafka.network.Processor) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: > java.lang.OutOfMemoryError: Direct buffer memory > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > java.nio.Bits.reserveMemory(Bits.java:694) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > sun.nio.ch.IOUtil.read(IOUtil.java:195) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:355) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > org.apache.kafka.common.network.Selector.poll(Selector.java:303) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > kafka.network.Processor.poll(SocketServer.scala:494) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > kafka.network.Processor.run(SocketServer.scala:432) > авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at > java.lang.Thread.run(Thread.java:748) > {code} > Only kafka cluster restart helps to fix this problem. > I wonder, why this problem have take a place, cause we running kafka with > follow options Environment="KAFKA_HEAP_OPTS=-Xmx5G -Xms5G > -XX:MaxDirectMemorySize=2G" -- This message was sent by Atlassian JIRA (v6.4.14#64029)