[jira] [Commented] (KAFKA-10360) Disabling JmxReporter registration
[ https://issues.apache.org/jira/browse/KAFKA-10360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284208#comment-17284208 ] Jaikiran Pai commented on KAFKA-10360: -- I think this would be a useful enhancement. We have been using Kafka for around 6 years now and during all these years we have only used these metrics maybe 2 or 3 times. For us these metrics don't add much value and we would like to optional disable them. I have meant to ask for this enhancement for a while now, but kept pushing it out. What prompted me to post today is that while looking at the JVM memory usage of one of our products, I noticed these entries: ``` 26: 710 34080 org.apache.kafka.common.metrics.stats.SampledStat$Sample 27: 1004 32128 org.apache.kafka.common.MetricName 28: 1004 32128 org.apache.kafka.common.metrics.KafkaMetric ``` These values may not be too high, but the mere fact they are there and aren't 0 isn't a good thing for us, since we don't need these metrics. > Disabling JmxReporter registration > --- > > Key: KAFKA-10360 > URL: https://issues.apache.org/jira/browse/KAFKA-10360 > Project: Kafka > Issue Type: New Feature > Components: clients >Reporter: Romain Quinio >Priority: Minor > > In Kafka client applications, JMX usage is often being replaced in favor of > frameworks like micrometer or microprofile-metrics. > It would be nice to be able to disable the JmxReporter that is today built-in > with KafkaProducer/KafkaConsumer/KafkaStreams > [https://github.com/apache/kafka/blob/783a6451f5f8c50dbe151caf5e76b74917690364/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L355-L357] > [https://github.com/apache/kafka/blob/ffdec02e25bb3be52ee5c06fe76d388303f6ea43/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L869-L871] > [https://github.com/apache/kafka/blob/42f46abb34a2b29993b1a8e6333a400a00227e30/streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java#L685-L687] > Example of issue in Quarkus: https://github.com/quarkusio/quarkus/issues/9799 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-4090) JVM runs into OOM if (Java) client uses a SSL port without setting the security protocol
[ https://issues.apache.org/jira/browse/KAFKA-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886003#comment-16886003 ] jaikiran pai commented on KAFKA-4090: - Hi [~akumar], I'm not working on a fix for this one. I just reported it and there was the mailing list discussion about it (linked in one of my comments above). Feel free to take it up. The kafka-dev mailing list would be a better place to discuss this change to get inputs from the Kafka developers. > JVM runs into OOM if (Java) client uses a SSL port without setting the > security protocol > > > Key: KAFKA-4090 > URL: https://issues.apache.org/jira/browse/KAFKA-4090 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.9.0.1, 0.10.0.1 >Reporter: jaikiran pai >Priority: Major > > Quoting from the mail thread that was sent to Kafka mailing list: > {quote} > We have been using Kafka 0.9.0.1 (server and Java client libraries). So far > we had been using it with plaintext transport but recently have been > considering upgrading to using SSL. It mostly works except that a > mis-configured producer (and even consumer) causes a hard to relate > OutOfMemory exception and thus causing the JVM in which the client is > running, to go into a bad state. We can consistently reproduce that OOM very > easily. We decided to check if this is something that is fixed in 0.10.0.1 so > upgraded one of our test systems to that version (both server and client > libraries) but still see the same issue. Here's how it can be easily > reproduced > 1. Enable SSL listener on the broker via server.properties, as per the Kafka > documentation > {code} > listeners=PLAINTEXT://:9092,SSL://:9093 > ssl.keystore.location= > ssl.keystore.password=pass > ssl.key.password=pass > ssl.truststore.location= > ssl.truststore.password=pass > {code} > 2. Start zookeeper and kafka server > 3. Create a "oom-test" topic (which will be used for these tests): > {code} > kafka-topics.sh --zookeeper localhost:2181 --create --topic oom-test > --partitions 1 --replication-factor 1 > {code} > 4. Create a simple producer which sends a single message to the topic via > Java (new producer) APIs: > {code} > public class OOMTest { > public static void main(final String[] args) throws Exception { > final Properties kafkaProducerConfigs = new Properties(); > // NOTE: Intentionally use a SSL port without specifying > security.protocol as SSL > > kafkaProducerConfigs.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, > "localhost:9093"); > > kafkaProducerConfigs.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, > StringSerializer.class.getName()); > > kafkaProducerConfigs.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, > StringSerializer.class.getName()); > try (KafkaProducer producer = new > KafkaProducer<>(kafkaProducerConfigs)) { > System.out.println("Created Kafka producer"); > final String topicName = "oom-test"; > final String message = "Hello OOM!"; > // send a message to the topic > final Future recordMetadataFuture = > producer.send(new ProducerRecord<>(topicName, message)); > final RecordMetadata sentRecordMetadata = > recordMetadataFuture.get(); > System.out.println("Sent message '" + message + "' to topic '" + > topicName + "'"); > } > System.out.println("Tests complete"); > } > } > {code} > Notice that the server URL is using a SSL endpoint localhost:9093 but isn't > specifying any of the other necessary SSL configs like security.protocol. > 5. For the sake of easily reproducing this issue run this class with a max > heap size of 256MB (-Xmx256M). Running this code throws up the following > OutOfMemoryError in one of the Sender threads: > {code} > 18:33:25,770 ERROR [KafkaThread] - Uncaught exception in > kafka-producer-network-thread | producer-1: > java.lang.OutOfMemoryError: Java heap space > at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57) > at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) > at > org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:93) > at > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71) > at > org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:153) > at > org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:134) > at org.apache.kafka.common.network.Selector.poll(Selector.java:286) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:256) > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216) > at
[jira] [Updated] (KAFKA-4090) JVM runs into OOM if (Java) client uses a SSL port without setting the security protocol
[ https://issues.apache.org/jira/browse/KAFKA-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-4090: Affects Version/s: 2.1.0 > JVM runs into OOM if (Java) client uses a SSL port without setting the > security protocol > > > Key: KAFKA-4090 > URL: https://issues.apache.org/jira/browse/KAFKA-4090 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.9.0.1, 0.10.0.1, 2.1.0 >Reporter: jaikiran pai >Priority: Major > > Quoting from the mail thread that was sent to Kafka mailing list: > {quote} > We have been using Kafka 0.9.0.1 (server and Java client libraries). So far > we had been using it with plaintext transport but recently have been > considering upgrading to using SSL. It mostly works except that a > mis-configured producer (and even consumer) causes a hard to relate > OutOfMemory exception and thus causing the JVM in which the client is > running, to go into a bad state. We can consistently reproduce that OOM very > easily. We decided to check if this is something that is fixed in 0.10.0.1 so > upgraded one of our test systems to that version (both server and client > libraries) but still see the same issue. Here's how it can be easily > reproduced > 1. Enable SSL listener on the broker via server.properties, as per the Kafka > documentation > {code} > listeners=PLAINTEXT://:9092,SSL://:9093 > ssl.keystore.location= > ssl.keystore.password=pass > ssl.key.password=pass > ssl.truststore.location= > ssl.truststore.password=pass > {code} > 2. Start zookeeper and kafka server > 3. Create a "oom-test" topic (which will be used for these tests): > {code} > kafka-topics.sh --zookeeper localhost:2181 --create --topic oom-test > --partitions 1 --replication-factor 1 > {code} > 4. Create a simple producer which sends a single message to the topic via > Java (new producer) APIs: > {code} > public class OOMTest { > public static void main(final String[] args) throws Exception { > final Properties kafkaProducerConfigs = new Properties(); > // NOTE: Intentionally use a SSL port without specifying > security.protocol as SSL > > kafkaProducerConfigs.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, > "localhost:9093"); > > kafkaProducerConfigs.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, > StringSerializer.class.getName()); > > kafkaProducerConfigs.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, > StringSerializer.class.getName()); > try (KafkaProducer producer = new > KafkaProducer<>(kafkaProducerConfigs)) { > System.out.println("Created Kafka producer"); > final String topicName = "oom-test"; > final String message = "Hello OOM!"; > // send a message to the topic > final Future recordMetadataFuture = > producer.send(new ProducerRecord<>(topicName, message)); > final RecordMetadata sentRecordMetadata = > recordMetadataFuture.get(); > System.out.println("Sent message '" + message + "' to topic '" + > topicName + "'"); > } > System.out.println("Tests complete"); > } > } > {code} > Notice that the server URL is using a SSL endpoint localhost:9093 but isn't > specifying any of the other necessary SSL configs like security.protocol. > 5. For the sake of easily reproducing this issue run this class with a max > heap size of 256MB (-Xmx256M). Running this code throws up the following > OutOfMemoryError in one of the Sender threads: > {code} > 18:33:25,770 ERROR [KafkaThread] - Uncaught exception in > kafka-producer-network-thread | producer-1: > java.lang.OutOfMemoryError: Java heap space > at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57) > at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) > at > org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:93) > at > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71) > at > org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:153) > at > org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:134) > at org.apache.kafka.common.network.Selector.poll(Selector.java:286) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:256) > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216) > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:128) > at java.lang.Thread.run(Thread.java:745) > {code} > Note that I set it to 256MB as heap size to easily reproduce it but this > isn't specific to that size. We have been able to reproduce it at even 516MB > and higher too. > This even happens
[jira] [Commented] (KAFKA-2561) Optionally support OpenSSL for SSL/TLS
[ https://issues.apache.org/jira/browse/KAFKA-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224784#comment-16224784 ] jaikiran pai commented on KAFKA-2561: - I just came across this JIRA, so I thought I will update it with my own recent experiments with OpenSSL (Java 8) and Kafka. For those interested, I got some performance numbers OpenSSL (backed by WildFly OpenSSL Java bindings) and have detailed them in my blog[1]. Later this week, I plan to rerun the same thing with Java 9 and see how it performs. [1] https://jaitechwriteups.blogspot.com/2017/10/kafka-with-openssl.html > Optionally support OpenSSL for SSL/TLS > --- > > Key: KAFKA-2561 > URL: https://issues.apache.org/jira/browse/KAFKA-2561 > Project: Kafka > Issue Type: New Feature > Components: security >Affects Versions: 0.9.0.0 >Reporter: Ismael Juma > > JDK's `SSLEngine` is unfortunately a bit slow (KAFKA-2431 covers this in more > detail). We should consider supporting OpenSSL for SSL/TLS. Initial > experiments on my laptop show that it performs a lot better: > {code} > start.time, end.time, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, > nMsg.sec, config > 2015-09-21 14:41:58:245, 2015-09-21 14:47:02:583, 28610.2295, 94.0081, > 3000, 98574.6111, Java 8u60/server auth JDK > SSLEngine/TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA > 2015-09-21 14:38:24:526, 2015-09-21 14:40:19:941, 28610.2295, 247.8900, > 3000, 259931.5514, Java 8u60/server auth > OpenSslEngine/TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 > 2015-09-21 14:49:03:062, 2015-09-21 14:50:27:764, 28610.2295, 337.7751, > 3000, 354182.9000, Java 8u60/plaintext > {code} > Extracting the throughput figures: > * JDK SSLEngine: 94 MB/s > * OpenSSL SSLEngine: 247 MB/s > * Plaintext: 337 MB/s (code from trunk, so no zero-copy due to KAFKA-2517) > In order to get these figures, I used Netty's `OpenSslEngine` by hacking > `SSLFactory` to use Netty's `SslContextBuilder` and made a few changes to > `SSLTransportLayer` in order to workaround differences in behaviour between > `OpenSslEngine` and JDK's SSLEngine (filed > https://github.com/netty/netty/issues/4235 and > https://github.com/netty/netty/issues/4238 upstream). -- This message was sent by Atlassian JIRA (v6.4.14#64029)