[jira] [Commented] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024593#comment-16024593 ] Konstantin Smirnov commented on KAFKA-5263: --- Thanks a lot for your fix! > kakfa-clients consume 100% CPU with manual partition assignment when network > connection is lost > --- > > Key: KAFKA-5263 > URL: https://issues.apache.org/jira/browse/KAFKA-5263 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1 >Reporter: Konstantin Smirnov >Assignee: Rajini Sivaram > Fix For: 0.11.0.0, 0.11.1.0 > > Attachments: cpu_consuming.log, cpu_consuming_profile.csv > > > Noticed that lose of the connection to Kafka broker leads kafka-clients to > consume 100% CPU. The bug only appears when the manual partition assignmet is > used. It appears since the version 0.10.1.0. The bug is quite similar to > KAFKA-1642. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Smirnov updated KAFKA-5263: -- Attachment: cpu_consuming_profile.csv Captured profile > kakfa-clients consume 100% CPU with manual partition assignment when network > connection is lost > --- > > Key: KAFKA-5263 > URL: https://issues.apache.org/jira/browse/KAFKA-5263 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1 >Reporter: Konstantin Smirnov > Attachments: cpu_consuming.log, cpu_consuming_profile.csv > > > Noticed that lose of the connection to Kafka broker leads kafka-clients to > consume 100% CPU. The bug only appears when the manual partition assignmet is > used. It appears since the version 0.10.1.0. The bug is quite similar to > KAFKA-1642. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Smirnov updated KAFKA-5263: -- Attachment: cpu_consuming.log Sample application output > kakfa-clients consume 100% CPU with manual partition assignment when network > connection is lost > --- > > Key: KAFKA-5263 > URL: https://issues.apache.org/jira/browse/KAFKA-5263 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1 >Reporter: Konstantin Smirnov > Attachments: cpu_consuming.log > > > Noticed that lose of the connection to Kafka broker leads kafka-clients to > consume 100% CPU. The bug only appears when the manual partition assignmet is > used. It appears since the version 0.10.1.0. The bug is quite similar to > KAFKA-1642. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Smirnov updated KAFKA-5263: -- Summary: kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost (was: kakfa-clients consume 100% CPU with manual partition assignment) > kakfa-clients consume 100% CPU with manual partition assignment when network > connection is lost > --- > > Key: KAFKA-5263 > URL: https://issues.apache.org/jira/browse/KAFKA-5263 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1 >Reporter: Konstantin Smirnov > > Noticed that lose of the connection to Kafka broker leads kafka-clients to > consume 100% CPU. The bug only appears when the manual partition assignmet is > used. It appears since the version 0.10.1.0. The bug is quite similar to > KAFKA-1642. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment
[
https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014226#comment-16014226
]
Konstantin Smirnov edited comment on KAFKA-5263 at 5/17/17 3:36 PM:
Sample code leading to the trouble:
{code}
public static void main(String[] args) {
System.out.println("Starting consumer");
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "test12345");
props.put("reconnect.backoff.ms", "1000");
props.put("retry.backoff.ms", "1000");
props.put("session.timeout.ms", "1");
props.put("key.deserializer",
"org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer",
"org.apache.kafka.common.serialization.StringDeserializer");
try (KafkaConsumer consumer = new
KafkaConsumer<>(props)) {
String topic = "test-topic";
//consumer.subscribe(Arrays.asList(topic));
List partitions = new ArrayList<>();
for (PartitionInfo partition : consumer.partitionsFor(topic)) {
partitions.add(new TopicPartition(topic,
partition.partition()));
}
consumer.assign(partitions);
for (;;) {
ConsumerRecords records = consumer.poll(1000);
if (!records.isEmpty()) {
System.out.println("Records aren't empty!");
}
records.forEach(System.out::println);
}
}
}
{code}
Steps to reproduce:
* Start Kafka broker
* Run the sample code
* Stop Kafka broker
was (Author: kosm):
Sample code leading to the trouble:
{code}
public static void main(String[] args) {
System.out.println("Starting consumer");
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "test12345");
props.put("reconnect.backoff.ms", "1000");
props.put("retry.backoff.ms", "1000");
props.put("session.timeout.ms", "1");
props.put("key.deserializer",
"org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer",
"org.apache.kafka.common.serialization.StringDeserializer");
try (KafkaConsumer consumer = new
KafkaConsumer<>(props)) {
String topic = "test-topic";
//consumer.subscribe(Arrays.asList(topic));
List partitions = new ArrayList<>();
for (PartitionInfo partition : consumer.partitionsFor(topic)) {
partitions.add(new TopicPartition(topic,
partition.partition()));
}
consumer.assign(partitions);
for (;;) {
ConsumerRecords records = consumer.poll(1000);
if (!records.isEmpty()) {
System.out.println("Records aren't empty!");
}
records.forEach(System.out::println);
}
}
}
{code}
> kakfa-clients consume 100% CPU with manual partition assignment
> ---
>
> Key: KAFKA-5263
> URL: https://issues.apache.org/jira/browse/KAFKA-5263
> Project: Kafka
> Issue Type: Bug
> Components: clients
>Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1
>Reporter: Konstantin Smirnov
>
> Noticed that lose of the connection to Kafka broker leads kafka-clients to
> consume 100% CPU. The bug only appears when the manual partition assignmet is
> used. It appears since the version 0.10.1.0. The bug is quite similar to
> KAFKA-1642.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment
[
https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014226#comment-16014226
]
Konstantin Smirnov commented on KAFKA-5263:
---
Sample code leading to the trouble:
{code}
public static void main(String[] args) {
System.out.println("Starting consumer");
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "test12345");
props.put("reconnect.backoff.ms", "1000");
props.put("retry.backoff.ms", "1000");
props.put("session.timeout.ms", "1");
props.put("key.deserializer",
"org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer",
"org.apache.kafka.common.serialization.StringDeserializer");
try (KafkaConsumer consumer = new
KafkaConsumer<>(props)) {
String topic = "test-topic";
//consumer.subscribe(Arrays.asList(topic));
List partitions = new ArrayList<>();
for (PartitionInfo partition : consumer.partitionsFor(topic)) {
partitions.add(new TopicPartition(topic,
partition.partition()));
}
consumer.assign(partitions);
for (;;) {
ConsumerRecords records = consumer.poll(1000);
if (!records.isEmpty()) {
System.out.println("Records aren't empty!");
}
records.forEach(System.out::println);
}
}
}
{code}
> kakfa-clients consume 100% CPU with manual partition assignment
> ---
>
> Key: KAFKA-5263
> URL: https://issues.apache.org/jira/browse/KAFKA-5263
> Project: Kafka
> Issue Type: Bug
> Components: clients
>Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1
>Reporter: Konstantin Smirnov
>
> Noticed that lose of the connection to Kafka broker leads kafka-clients to
> consume 100% CPU. The bug only appears when the manual partition assignmet is
> used. It appears since the version 0.10.1.0. The bug is quite similar to
> KAFKA-1642.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Smirnov updated KAFKA-5263: -- Affects Version/s: 0.10.1.0 0.10.1.1 Description: Noticed that lose of the connection to Kafka broker leads kafka-clients to consume 100% CPU. The bug only appears when the manual partition assignmet is used. It appears since the version 0.10.1.0. The bug is quite similar to KAFKA-1642. > kakfa-clients consume 100% CPU with manual partition assignment > --- > > Key: KAFKA-5263 > URL: https://issues.apache.org/jira/browse/KAFKA-5263 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1 >Reporter: Konstantin Smirnov > > Noticed that lose of the connection to Kafka broker leads kafka-clients to > consume 100% CPU. The bug only appears when the manual partition assignmet is > used. It appears since the version 0.10.1.0. The bug is quite similar to > KAFKA-1642. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment
Konstantin Smirnov created KAFKA-5263: - Summary: kakfa-clients consume 100% CPU with manual partition assignment Key: KAFKA-5263 URL: https://issues.apache.org/jira/browse/KAFKA-5263 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 0.10.2.1, 0.10.2.0 Reporter: Konstantin Smirnov -- This message was sent by Atlassian JIRA (v6.3.15#6346)
