[jira] [Comment Edited] (KAFKA-4966) Producer throw a NullPointerException under a network environment where packet loss and error packets exist.
[ https://issues.apache.org/jira/browse/KAFKA-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946408#comment-15946408 ] Bo Wang edited comment on KAFKA-4966 at 3/29/17 2:06 AM: - Hi, Ismael Juma. I used the 0.10.0.1 version and merged some other issues patches, and I found that the code have the problem, and the trunk is still the same, so I list the two versions. was (Author: wangbo23): Hi, Ismael Juma. I used the 0.10.0.1 version and merged some other issues patches, and I found that the code have the problem, and the trunk is still the same, so I list the two version. > Producer throw a NullPointerException under a network environment where > packet loss and error packets exist. > > > Key: KAFKA-4966 > URL: https://issues.apache.org/jira/browse/KAFKA-4966 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.10.0.1, 0.10.2.0 >Reporter: Bo Wang > > 2017-03-27 18:45:54,757 | ERROR | [kafka-producer-network-thread | > producer-1] | Uncaught error in request completion: | > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:284) > java.lang.NullPointerException > at > org.apache.kafka.clients.producer.internals.Sender.canRetry(Sender.java:342) > at > org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:310) > at > org.apache.kafka.clients.producer.internals.Sender.handleProduceResponse(Sender.java:286) > at > org.apache.kafka.clients.producer.internals.Sender.access$100(Sender.java:57) > at > org.apache.kafka.clients.producer.internals.Sender$1.onComplete(Sender.java:372) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:282) > at > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:241) > at > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:136) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KAFKA-4966) Producer throw a NullPointerException under a network environment where packet loss and error packets exist.
[ https://issues.apache.org/jira/browse/KAFKA-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-4966: --- Summary: Producer throw a NullPointerException under a network environment where packet loss and error packets exist. (was: Producer maybe throw a NullPointerException under a network environment where packet loss and error packets exist.) > Producer throw a NullPointerException under a network environment where > packet loss and error packets exist. > > > Key: KAFKA-4966 > URL: https://issues.apache.org/jira/browse/KAFKA-4966 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 0.10.0.1, 0.10.2.0 >Reporter: Bo Wang > > 2017-03-27 18:45:54,757 | ERROR | [kafka-producer-network-thread | > producer-1] | Uncaught error in request completion: | > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:284) > java.lang.NullPointerException > at > org.apache.kafka.clients.producer.internals.Sender.canRetry(Sender.java:342) > at > org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:310) > at > org.apache.kafka.clients.producer.internals.Sender.handleProduceResponse(Sender.java:286) > at > org.apache.kafka.clients.producer.internals.Sender.access$100(Sender.java:57) > at > org.apache.kafka.clients.producer.internals.Sender$1.onComplete(Sender.java:372) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:282) > at > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:241) > at > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:136) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KAFKA-4966) Producer maybe throw a NullPointerException under a network environment where packet loss and error packets exist.
Bo Wang created KAFKA-4966: -- Summary: Producer maybe throw a NullPointerException under a network environment where packet loss and error packets exist. Key: KAFKA-4966 URL: https://issues.apache.org/jira/browse/KAFKA-4966 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 0.10.2.0, 0.10.0.1 Reporter: Bo Wang 2017-03-27 18:45:54,757 | ERROR | [kafka-producer-network-thread | producer-1] | Uncaught error in request completion: | org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:284) java.lang.NullPointerException at org.apache.kafka.clients.producer.internals.Sender.canRetry(Sender.java:342) at org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:310) at org.apache.kafka.clients.producer.internals.Sender.handleProduceResponse(Sender.java:286) at org.apache.kafka.clients.producer.internals.Sender.access$100(Sender.java:57) at org.apache.kafka.clients.producer.internals.Sender$1.onComplete(Sender.java:372) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:282) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:241) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:136) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047899#comment-15047899 ] Bo Wang commented on KAFKA-2965: https://github.com/apache/kafka/pull/646 [~gwenshap] please review it. thanks. > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Priority: Minor > Labels: bug > Attachments: Kafka-2965.patch > > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2965: --- Labels: patch (was: ) Reviewer: (was: Neha Narkhede) Status: Patch Available (was: Open) > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Priority: Minor > Labels: patch > Attachments: Kafka-2965.patch > > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2965: --- Description: Two variables should be exchanged in KafkaController.scala as follows： val topicsForWhichPartitionReassignmentIsInProgress = controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) val topicsForWhichPreferredReplicaElectionIsInProgress = controllerContext.partitionsBeingReassigned.keySet.map(_.topic) val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | topicsForWhichPartitionReassignmentIsInProgress | topicsForWhichPreferredReplicaElectionIsInProgress Should change to: val topicsForWhichPreferredReplicaElectionIsInProgress = controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) val topicsForWhichPartitionReassignmentIsInProgress = controllerContext.partitionsBeingReassigned.keySet.map(_.topic) val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | topicsForWhichPartitionReassignmentIsInProgress | topicsForWhichPreferredReplicaElectionIsInProgress was:Two variables should be exchanged in KafkaController.scala as follows： > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Assignee: Neha Narkhede >Priority: Minor > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2965: --- Attachment: Kafka-2965.patch > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Assignee: Neha Narkhede >Priority: Minor > Attachments: Kafka-2965.patch > > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2965) Two variables should be exchanged.
Bo Wang created KAFKA-2965: -- Summary: Two variables should be exchanged. Key: KAFKA-2965 URL: https://issues.apache.org/jira/browse/KAFKA-2965 Project: Kafka Issue Type: Bug Components: controller Affects Versions: 0.9.0.0 Environment: NA Reporter: Bo Wang Assignee: Neha Narkhede Priority: Minor Two variables should be exchanged in KafkaController.scala as follows： -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047855#comment-15047855 ] Bo Wang commented on KAFKA-2965: According to the code, I think the name of the two variable needs to be exchanged. > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Assignee: Neha Narkhede >Priority: Minor > Fix For: 0.9.1.0 > > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2965: --- Reviewer: Neha Narkhede Fix Version/s: 0.9.1.0 Status: Patch Available (was: Open) > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Assignee: Neha Narkhede >Priority: Minor > Fix For: 0.9.1.0 > > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047873#comment-15047873 ] Bo Wang commented on KAFKA-2965: patch submit. please review it. thanks. > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Priority: Minor > Labels: bug > Attachments: Kafka-2965.patch > > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2965: --- Fix Version/s: (was: 0.9.1.0) > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Assignee: Neha Narkhede >Priority: Minor > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2965) Two variables should be exchanged.
[ https://issues.apache.org/jira/browse/KAFKA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2965: --- Status: Open (was: Patch Available) > Two variables should be exchanged. > -- > > Key: KAFKA-2965 > URL: https://issues.apache.org/jira/browse/KAFKA-2965 > Project: Kafka > Issue Type: Bug > Components: controller >Affects Versions: 0.9.0.0 > Environment: NA >Reporter: Bo Wang >Assignee: Neha Narkhede >Priority: Minor > Fix For: 0.9.1.0 > > > Two variables should be exchanged in KafkaController.scala as follows： > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress > Should change to: > val topicsForWhichPreferredReplicaElectionIsInProgress = > controllerContext.partitionsUndergoingPreferredReplicaElection.map(_.topic) > val topicsForWhichPartitionReassignmentIsInProgress = > controllerContext.partitionsBeingReassigned.keySet.map(_.topic) > val topicsIneligibleForDeletion = topicsWithReplicasOnDeadBrokers | > topicsForWhichPartitionReassignmentIsInProgress | > > topicsForWhichPreferredReplicaElectionIsInProgress -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1554) Corrupt index found on clean startup
[
https://issues.apache.org/jira/browse/KAFKA-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14956579#comment-14956579
]
Bo Wang commented on KAFKA-1554:
@ Jun Rao, @Mayuresh Gharat , I don't think this patch is available.
The method given in patch appears to be useless, because the sanityCheck() for
loop in advance, then logSegments is empty, so the code useless.
> Corrupt index found on clean startup
>
>
> Key: KAFKA-1554
> URL: https://issues.apache.org/jira/browse/KAFKA-1554
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1
> Environment: ubuntu 12.04, oracle jdk 1.7
>Reporter: Alexis Midon
>Assignee: Mayuresh Gharat
>Priority: Critical
> Fix For: 0.10.0.0
>
> Attachments: KAFKA-1554.patch
>
>
> On a clean start up, corrupted index files are found.
> After investigations, it appears that some pre-allocated index files are not
> "compacted" correctly and the end of the file is full of zeroes.
> As a result, on start up, the last relative offset is zero which yields an
> offset equal to the base offset.
> The workaround is to delete all index files of size 10MB (the size of the
> pre-allocated files), and restart. Index files will be re-created.
> {code}
> find $your_data_directory -size 10485760c -name *.index #-delete
> {code}
> This is issue might be related/similar to
> https://issues.apache.org/jira/browse/KAFKA-1112
> {code}
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,696
> INFO main kafka.server.KafkaServer.info - [Kafka Server 847605514], starting
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,698
> INFO main kafka.server.KafkaServer.info - [Kafka Server 847605514],
> Connecting to zookeeper on
> zk-main0.XXX:2181,zk-main1.XXX:2181,zk-main2.:2181/production/kafka/main
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,708
> INFO
> ZkClient-EventThread-14-zk-main0.XXX.com:2181,zk-main1.XXX.com:2181,zk-main2.XXX.com:2181,zk-main3.XXX.com:2181,zk-main4.XXX.com:2181/production/kafka/main
> org.I0Itec.zkclient.ZkEventThread.run - Starting ZkClient event thread.
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,714
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,714
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:host.name=i-6b948138.inst.aws.airbnb.com
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,714
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:java.version=1.7.0_55
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,715
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:java.vendor=Oracle Corporation
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,715
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:java.home=/usr/lib/jvm/jre-7-oracle-x64/jre
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,715
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:java.class.path=libs/snappy-java-1.0.5.jar:libs/scala-library-2.10.1.jar:libs/slf4j-api-1.7.2.jar:libs/jopt-simple-3.2.jar:libs/metrics-annotation-2.2.0.jar:libs/log4j-1.2.15.jar:libs/kafka_2.10-0.8.1.jar:libs/zkclient-0.3.jar:libs/zookeeper-3.3.4.jar:libs/metrics-core-2.2.0.jar
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,715
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,716
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:java.io.tmpdir=/tmp
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,716
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:java.compiler=
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,716
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:os.name=Linux
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,716
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:os.arch=amd64
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,717
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
> environment:os.version=3.2.0-61-virtual
> 2014-07-11T00:53:17+00:00 i-6b948138 local3.info 2014-07-11 - 00:53:17,717
> INFO main org.apache.zookeeper.ZooKeeper.logEnv - Client
>
[jira] [Created] (KAFKA-2560) Fatal error during KafkaServer startup because of Map failed error.
Bo Wang created KAFKA-2560: -- Summary: Fatal error during KafkaServer startup because of Map failed error. Key: KAFKA-2560 URL: https://issues.apache.org/jira/browse/KAFKA-2560 Project: Kafka Issue Type: Bug Components: log Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Assignee: Jay Kreps I have 3 kafka nodes, create 30 topics ,every topic has 100 pations, and replica factor is 2. Kafka server start failed, 2015-09-21 10:28:35,668 | INFO | pool-2-thread-1 | Recovering unflushed segment 0 in log testTopic_14-34. | kafka.utils.Logging$class.info(Logging.scala:68) 2015-09-21 10:28:35,942 | ERROR | main | There was an error in one of the threads during logs loading: java.io.IOException: Map failed | kafka.utils.Logging$class.error(Logging.scala:97) 2015-09-21 10:28:35,943 | INFO | pool-2-thread-5 | Recovering unflushed segment 0 in log testTopic_17-23. | kafka.utils.Logging$class.info(Logging.scala:68) 2015-09-21 10:28:35,944 | INFO | pool-2-thread-5 | Completed load of log testTopic_17-23 with log end offset 0 | kafka.utils.Logging$class.info(Logging.scala:68) 2015-09-21 10:28:35,945 | FATAL | main | [Kafka Server 54], Fatal error during KafkaServer startup. Prepare to shutdown | kafka.utils.Logging$class.fatal(Logging.scala:116) java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:907) at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:286) at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:276) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.log.OffsetIndex.resize(OffsetIndex.scala:276) at kafka.log.Log.loadSegments(Log.scala:179) at kafka.log.Log.(Log.scala:67) at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$7$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:142) at kafka.utils.Utils$$anon$1.run(Utils.scala:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.OutOfMemoryError: Map failed at sun.nio.ch.FileChannelImpl.map0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904) ... 13 more 2015-09-21 10:28:35,946 | INFO | pool-2-thread-5 | Recovering unflushed segment 0 in log testTopic_25-77. | kafka.utils.Logging$class.info(Logging.scala:68) 2015-09-21 10:28:35,949 | INFO | main | [Kafka Server 54], shutting down | kafka.utils.Logging$class.info(Logging.scala:68) Kafka server host's top infomation below: top - 17:16:23 up 53 min, 6 users, load average: 0.42, 0.99, 1.19 Tasks: 215 total, 1 running, 214 sleeping, 0 stopped, 0 zombie Cpu(s): 4.5%us, 2.4%sy, 0.0%ni, 92.9%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 40169M total, 6118M used,34050M free,9M buffers Swap:0M total,0M used,0M free, 431M cached -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2470) kafka-producer-perf-test.sh can't configure all to request-num-acks
[ https://issues.apache.org/jira/browse/KAFKA-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2470: --- Fix Version/s: 0.8.3 kafka-producer-perf-test.sh can't configure all to request-num-acks --- Key: KAFKA-2470 URL: https://issues.apache.org/jira/browse/KAFKA-2470 Project: Kafka Issue Type: Bug Components: clients, tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Fix For: 0.8.3 Original Estimate: 24h Remaining Estimate: 24h For New Producer API, kafka-producer-perf-test.sh can't configure all to request-num-acks : bin]# ./kafka-producer-perf-test.sh --topic test --broker-list host:port --messages 100 --message-size 200 --new-producer --sync --batch-size 1 --request-num-acks all Exception in thread main joptsimple.OptionArgumentConversionException: Cannot convert argument 'all' of option ['request-num-acks'] to class java.lang.Integer at joptsimple.ArgumentAcceptingOptionSpec.convert(ArgumentAcceptingOptionSpec.java:237) at joptsimple.OptionSet.valuesOf(OptionSet.java:226) at joptsimple.OptionSet.valueOf(OptionSet.java:170) at kafka.tools.ProducerPerformance$ProducerPerfConfig.init(ProducerPerformance.scala:146) at kafka.tools.ProducerPerformance$.main(ProducerPerformance.scala:42) at kafka.tools.ProducerPerformance.main(ProducerPerformance.scala) Caused by: joptsimple.internal.ReflectionException: java.lang.NumberFormatException: For input string: all at joptsimple.internal.Reflection.reflectionException(Reflection.java:136) at joptsimple.internal.Reflection.invoke(Reflection.java:123) at joptsimple.internal.MethodInvokingValueConverter.convert(MethodInvokingValueConverter.java:48) at joptsimple.ArgumentAcceptingOptionSpec.convert(ArgumentAcceptingOptionSpec.java:234) ... 5 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2470) kafka-producer-perf-test.sh can't configure all to request-num-acks
Bo Wang created KAFKA-2470: -- Summary: kafka-producer-perf-test.sh can't configure all to request-num-acks Key: KAFKA-2470 URL: https://issues.apache.org/jira/browse/KAFKA-2470 Project: Kafka Issue Type: Bug Components: clients, tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang For New Producer API, kafka-producer-perf-test.sh can't configure all to request-num-acks : bin]# ./kafka-producer-perf-test.sh --topic test --broker-list host:port --messages 100 --message-size 200 --new-producer --sync --batch-size 1 --request-num-acks all Exception in thread main joptsimple.OptionArgumentConversionException: Cannot convert argument 'all' of option ['request-num-acks'] to class java.lang.Integer at joptsimple.ArgumentAcceptingOptionSpec.convert(ArgumentAcceptingOptionSpec.java:237) at joptsimple.OptionSet.valuesOf(OptionSet.java:226) at joptsimple.OptionSet.valueOf(OptionSet.java:170) at kafka.tools.ProducerPerformance$ProducerPerfConfig.init(ProducerPerformance.scala:146) at kafka.tools.ProducerPerformance$.main(ProducerPerformance.scala:42) at kafka.tools.ProducerPerformance.main(ProducerPerformance.scala) Caused by: joptsimple.internal.ReflectionException: java.lang.NumberFormatException: For input string: all at joptsimple.internal.Reflection.reflectionException(Reflection.java:136) at joptsimple.internal.Reflection.invoke(Reflection.java:123) at joptsimple.internal.MethodInvokingValueConverter.convert(MethodInvokingValueConverter.java:48) at joptsimple.ArgumentAcceptingOptionSpec.convert(ArgumentAcceptingOptionSpec.java:234) ... 5 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2414) Running kafka-producer-perf-test.sh with --messages 10000000 --message-size 1000 --new-producer will get WARN Error in I/O.
[ https://issues.apache.org/jira/browse/KAFKA-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2414: --- Summary: Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will get WARN Error in I/O. (was: Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will got WARN Error in I/O.) Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will get WARN Error in I/O. Key: KAFKA-2414 URL: https://issues.apache.org/jira/browse/KAFKA-2414 Project: Kafka Issue Type: Bug Components: tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Labels: performance Original Estimate: 1m Remaining Estimate: 1m Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will got WARN Error in I/O： java.io.EOFException at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) at org.apache.kafka.common.network.Selector.poll(Selector.java:248) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2414) Running kafka-producer-perf-test.sh with --messages 10000000 --message-size 1000 --new-producer will got WARN Error in I/O.
[ https://issues.apache.org/jira/browse/KAFKA-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661164#comment-14661164 ] Bo Wang commented on KAFKA-2414: The exception sometimes occurs , how can I fix or avoid this problem? Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will got WARN Error in I/O. Key: KAFKA-2414 URL: https://issues.apache.org/jira/browse/KAFKA-2414 Project: Kafka Issue Type: Bug Components: tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Labels: performance Original Estimate: 1m Remaining Estimate: 1m Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will got WARN Error in I/O： java.io.EOFException at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) at org.apache.kafka.common.network.Selector.poll(Selector.java:248) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2414) Running kafka-producer-perf-test.sh with --messages 10000000 --message-size 1000 --new-producer will got WARN Error in I/O.
Bo Wang created KAFKA-2414: -- Summary: Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will got WARN Error in I/O. Key: KAFKA-2414 URL: https://issues.apache.org/jira/browse/KAFKA-2414 Project: Kafka Issue Type: Bug Components: tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Running kafka-producer-perf-test.sh with --messages 1000 --message-size 1000 --new-producer will got WARN Error in I/O： java.io.EOFException at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) at org.apache.kafka.common.network.Selector.poll(Selector.java:248) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2360) The kafka-consumer-perf-test.sh script help information print useless parameters.
[ https://issues.apache.org/jira/browse/KAFKA-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2360: --- Description: Run kafka-consumer-perf-test.sh --help to show help information, but found 3 parameters useless : --batch-size and --batch-size That is producer of parameters. bin]# ./kafka-consumer-perf-test.sh --help Missing required argument [topic] Option Description -- --- --batch-size Integer: sizeNumber of messages to write in a single batch. (default: 200) --compression-codec Integer: If set, messages are sent compressed supported codec: NoCompressionCodec (default: 0) as 0, GZIPCompressionCodec as 1, SnappyCompressionCodec as 2, LZ4CompressionCodec as 3 --date-format date format The date format to use for formatting the time field. See java.text. SimpleDateFormat for options. (default: -MM-dd HH:mm:ss:SSS) --fetch-size Integer: sizeThe amount of data to fetch in a single request. (default: 1048576) was: Run kafka-consumer-perf-test.sh --help to show help information, but found two parameters useless : --batch-size and --batch-size That is producer of parameters. bin]# ./kafka-consumer-perf-test.sh --help Missing required argument [topic] Option Description -- --- --batch-size Integer: sizeNumber of messages to write in a single batch. (default: 200) --compression-codec Integer: If set, messages are sent compressed supported codec: NoCompressionCodec (default: 0) as 0, GZIPCompressionCodec as 1, SnappyCompressionCodec as 2, LZ4CompressionCodec as 3 --date-format date format The date format to use for formatting the time field. See java.text. SimpleDateFormat for options. (default: -MM-dd HH:mm:ss:SSS) --fetch-size Integer: sizeThe amount of data to fetch in a single request. (default: 1048576) The kafka-consumer-perf-test.sh script help information print useless parameters. - Key: KAFKA-2360 URL: https://issues.apache.org/jira/browse/KAFKA-2360 Project: Kafka Issue Type: Bug Components: tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Priority: Minor Original Estimate: 24h Remaining Estimate: 24h Run kafka-consumer-perf-test.sh --help to show help information, but found 3 parameters useless : --batch-size and --batch-size That is producer of parameters. bin]# ./kafka-consumer-perf-test.sh --help Missing required argument [topic] Option Description -- --- --batch-size Integer: sizeNumber of messages to write in a single batch. (default: 200) --compression-codec Integer: If set, messages are sent compressed supported codec: NoCompressionCodec (default: 0) as 0, GZIPCompressionCodec as 1, SnappyCompressionCodec as 2, LZ4CompressionCodec as 3 --date-format date format The date format to use for formatting the time field. See java.text. SimpleDateFormat for options. (default: -MM-dd HH:mm:ss:SSS) --fetch-size Integer: sizeThe amount of data to fetch in a
[jira] [Updated] (KAFKA-2360) The kafka-consumer-perf-test.sh script help information print useless parameters.
[ https://issues.apache.org/jira/browse/KAFKA-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2360: --- Description: Run kafka-consumer-perf-test.sh --help to show help information, but found 3 parameters useless : --batch-size and --batch-size --messages That is producer of parameters. bin]# ./kafka-consumer-perf-test.sh --help Missing required argument [topic] Option Description -- --- --batch-size Integer: sizeNumber of messages to write in a single batch. (default: 200) --compression-codec Integer: If set, messages are sent compressed supported codec: NoCompressionCodec (default: 0) as 0, GZIPCompressionCodec as 1, SnappyCompressionCodec as 2, LZ4CompressionCodec as 3 --date-format date format The date format to use for formatting the time field. See java.text. SimpleDateFormat for options. (default: -MM-dd HH:mm:ss:SSS) --fetch-size Integer: sizeThe amount of data to fetch in a single request. (default: 1048576) --messages Long: countThe number of messages to send or consume (default: 9223372036854775807) was: Run kafka-consumer-perf-test.sh --help to show help information, but found 3 parameters useless : --batch-size and --batch-size That is producer of parameters. bin]# ./kafka-consumer-perf-test.sh --help Missing required argument [topic] Option Description -- --- --batch-size Integer: sizeNumber of messages to write in a single batch. (default: 200) --compression-codec Integer: If set, messages are sent compressed supported codec: NoCompressionCodec (default: 0) as 0, GZIPCompressionCodec as 1, SnappyCompressionCodec as 2, LZ4CompressionCodec as 3 --date-format date format The date format to use for formatting the time field. See java.text. SimpleDateFormat for options. (default: -MM-dd HH:mm:ss:SSS) --fetch-size Integer: sizeThe amount of data to fetch in a single request. (default: 1048576) The kafka-consumer-perf-test.sh script help information print useless parameters. - Key: KAFKA-2360 URL: https://issues.apache.org/jira/browse/KAFKA-2360 Project: Kafka Issue Type: Bug Components: tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Priority: Minor Original Estimate: 24h Remaining Estimate: 24h Run kafka-consumer-perf-test.sh --help to show help information, but found 3 parameters useless : --batch-size and --batch-size --messages That is producer of parameters. bin]# ./kafka-consumer-perf-test.sh --help Missing required argument [topic] Option Description -- --- --batch-size Integer: sizeNumber of messages to write in a single batch. (default: 200) --compression-codec Integer: If set, messages are sent compressed supported codec: NoCompressionCodec (default: 0) as 0, GZIPCompressionCodec as 1, SnappyCompressionCodec as 2, LZ4CompressionCodec as 3 --date-format date format The date format to use for formatting the time field. See java.text.
[jira] [Created] (KAFKA-2360) The kafka-consumer-perf-test.sh script help information print useless parameters.
Bo Wang created KAFKA-2360: -- Summary: The kafka-consumer-perf-test.sh script help information print useless parameters. Key: KAFKA-2360 URL: https://issues.apache.org/jira/browse/KAFKA-2360 Project: Kafka Issue Type: Bug Components: tools Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Priority: Minor Run kafka-consumer-perf-test.sh --help to show help information, but found two parameters useless : --batch-size and --batch-size That is producer of parameters. bin]# ./kafka-consumer-perf-test.sh --help Missing required argument [topic] Option Description -- --- --batch-size Integer: sizeNumber of messages to write in a single batch. (default: 200) --compression-codec Integer: If set, messages are sent compressed supported codec: NoCompressionCodec (default: 0) as 0, GZIPCompressionCodec as 1, SnappyCompressionCodec as 2, LZ4CompressionCodec as 3 --date-format date format The date format to use for formatting the time field. See java.text. SimpleDateFormat for options. (default: -MM-dd HH:mm:ss:SSS) --fetch-size Integer: sizeThe amount of data to fetch in a single request. (default: 1048576) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2361) Unit Test failed:testCleanShutdownAfterFailedStartup
Bo Wang created KAFKA-2361: -- Summary: Unit Test failed:testCleanShutdownAfterFailedStartup Key: KAFKA-2361 URL: https://issues.apache.org/jira/browse/KAFKA-2361 Project: Kafka Issue Type: Bug Reporter: Bo Wang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2361) Unit Test failed:testCleanShutdownAfterFailedStartup
[ https://issues.apache.org/jira/browse/KAFKA-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2361: --- Priority: Minor (was: Major) Unit Test failed:testCleanShutdownAfterFailedStartup Key: KAFKA-2361 URL: https://issues.apache.org/jira/browse/KAFKA-2361 Project: Kafka Issue Type: Bug Reporter: Bo Wang Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2361) Unit Test BUILD FAILED
[ https://issues.apache.org/jira/browse/KAFKA-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2361: --- Affects Version/s: 0.8.2.1 Environment: Linux Remaining Estimate: 48h Original Estimate: 48h Priority: Major (was: Minor) Description: 290 tests completed, 2 failed : kafka.api.ProducerFailureHandlingTest testNotEnoughReplicasAfterBrokerShutdown FAILED org.scalatest.junit.JUnitTestFailedError: Expected NotEnoughReplicasException when producing to topic with fewer brokers than min.insync.replicas at org.scalatest.junit.AssertionsForJUnit$class.newAssertionFailedException(AssertionsForJUnit.scala:101) at org.scalatest.junit.JUnit3Suite.newAssertionFailedException(JUnit3Suite.scala:149) at org.scalatest.Assertions$class.fail(Assertions.scala:711) at org.scalatest.junit.JUnit3Suite.fail(JUnit3Suite.scala:149) at kafka.api.ProducerFailureHandlingTest.testNotEnoughReplicasAfterBrokerShutdown(ProducerFailureHandlingTest.scala:355) kafka.server.ServerShutdownTest testCleanShutdownAfterFailedStartup FAILED org.scalatest.junit.JUnitTestFailedError: Expected KafkaServer setup to fail with connection exception but caught a different exception. at org.scalatest.junit.AssertionsForJUnit$class.newAssertionFailedException(AssertionsForJUnit.scala:101) at org.scalatest.junit.JUnit3Suite.newAssertionFailedException(JUnit3Suite.scala:149) at org.scalatest.Assertions$class.fail(Assertions.scala:711) at org.scalatest.junit.JUnit3Suite.fail(JUnit3Suite.scala:149) at kafka.server.ServerShutdownTest.testCleanShutdownAfterFailedStartup(ServerShutdownTest.scala:136) Component/s: unit tests Summary: Unit Test BUILD FAILED (was: Unit Test failed:testCleanShutdownAfterFailedStartup) Unit Test BUILD FAILED -- Key: KAFKA-2361 URL: https://issues.apache.org/jira/browse/KAFKA-2361 Project: Kafka Issue Type: Bug Components: unit tests Affects Versions: 0.8.2.1 Environment: Linux Reporter: Bo Wang Original Estimate: 48h Remaining Estimate: 48h 290 tests completed, 2 failed : kafka.api.ProducerFailureHandlingTest testNotEnoughReplicasAfterBrokerShutdown FAILED org.scalatest.junit.JUnitTestFailedError: Expected NotEnoughReplicasException when producing to topic with fewer brokers than min.insync.replicas at org.scalatest.junit.AssertionsForJUnit$class.newAssertionFailedException(AssertionsForJUnit.scala:101) at org.scalatest.junit.JUnit3Suite.newAssertionFailedException(JUnit3Suite.scala:149) at org.scalatest.Assertions$class.fail(Assertions.scala:711) at org.scalatest.junit.JUnit3Suite.fail(JUnit3Suite.scala:149) at kafka.api.ProducerFailureHandlingTest.testNotEnoughReplicasAfterBrokerShutdown(ProducerFailureHandlingTest.scala:355) kafka.server.ServerShutdownTest testCleanShutdownAfterFailedStartup FAILED org.scalatest.junit.JUnitTestFailedError: Expected KafkaServer setup to fail with connection exception but caught a different exception. at org.scalatest.junit.AssertionsForJUnit$class.newAssertionFailedException(AssertionsForJUnit.scala:101) at org.scalatest.junit.JUnit3Suite.newAssertionFailedException(JUnit3Suite.scala:149) at org.scalatest.Assertions$class.fail(Assertions.scala:711) at org.scalatest.junit.JUnit3Suite.fail(JUnit3Suite.scala:149) at kafka.server.ServerShutdownTest.testCleanShutdownAfterFailedStartup(ServerShutdownTest.scala:136) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1482) Transient test failures for kafka.admin.DeleteTopicTest
[ https://issues.apache.org/jira/browse/KAFKA-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14640080#comment-14640080 ] Bo Wang commented on KAFKA-1482: When I execute unit tests reported the same fault！ Transient test failures for kafka.admin.DeleteTopicTest --- Key: KAFKA-1482 URL: https://issues.apache.org/jira/browse/KAFKA-1482 Project: Kafka Issue Type: Bug Reporter: Guozhang Wang Assignee: Sriharsha Chintalapani Labels: newbie Fix For: 0.8.2.0 Attachments: kafka_delete_topic_test.log, kafka_tests.log A couple of test cases have timing related transient test failures: kafka.admin.DeleteTopicTest testPartitionReassignmentDuringDeleteTopic FAILED junit.framework.AssertionFailedError: Admin path /admin/delete_topic/test path not deleted even after a replica is restarted at junit.framework.Assert.fail(Assert.java:47) at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:578) at kafka.admin.DeleteTopicTest.verifyTopicDeletion(DeleteTopicTest.scala:333) at kafka.admin.DeleteTopicTest.testPartitionReassignmentDuringDeleteTopic(DeleteTopicTest.scala:197) kafka.admin.DeleteTopicTest testDeleteTopicDuringAddPartition FAILED junit.framework.AssertionFailedError: Replica logs not deleted after delete topic is complete at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at kafka.admin.DeleteTopicTest.verifyTopicDeletion(DeleteTopicTest.scala:338) at kafka.admin.DeleteTopicTest.testDeleteTopicDuringAddPartition(DeleteTopicTest.scala:216) kafka.admin.DeleteTopicTest testRequestHandlingDuringDeleteTopic FAILED org.scalatest.junit.JUnitTestFailedError: fails with exception at org.scalatest.junit.AssertionsForJUnit$class.newAssertionFailedException(AssertionsForJUnit.scala:102) at org.scalatest.junit.JUnit3Suite.newAssertionFailedException(JUnit3Suite.scala:142) at org.scalatest.Assertions$class.fail(Assertions.scala:664) at org.scalatest.junit.JUnit3Suite.fail(JUnit3Suite.scala:142) at kafka.admin.DeleteTopicTest.testRequestHandlingDuringDeleteTopic(DeleteTopicTest.scala:123) Caused by: org.scalatest.junit.JUnitTestFailedError: Test should fail because the topic is being deleted at org.scalatest.junit.AssertionsForJUnit$class.newAssertionFailedException(AssertionsForJUnit.scala:101) at org.scalatest.junit.JUnit3Suite.newAssertionFailedException(JUnit3Suite.scala:142) at org.scalatest.Assertions$class.fail(Assertions.scala:644) at org.scalatest.junit.JUnit3Suite.fail(JUnit3Suite.scala:142) at kafka.admin.DeleteTopicTest.testRequestHandlingDuringDeleteTopic(DeleteTopicTest.scala:120) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
[ https://issues.apache.org/jira/browse/KAFKA-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2254: --- Status: Patch Available (was: Open) I create a patch kafka-shell-script.patch of the trunk . The shell script should be optimized , even kafka-run-class.sh has a syntax error. -- Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Priority: Critical Labels: client-script, kafka-run-class.sh, shell-script Fix For: 0.8.2.1 Attachments: kafka-shell-script.patch Original Estimate: 24h Remaining Estimate: 24h kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
[ https://issues.apache.org/jira/browse/KAFKA-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2254: --- Priority: Major (was: Critical) The shell script should be optimized , even kafka-run-class.sh has a syntax error. -- Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Labels: client-script, kafka-run-class.sh, shell-script Fix For: 0.8.2.1 Attachments: kafka-shell-script.patch Original Estimate: 24h Remaining Estimate: 24h kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
[ https://issues.apache.org/jira/browse/KAFKA-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2254: --- Status: Patch Available (was: Open) The shell script should be optimized , even kafka-run-class.sh has a syntax error. -- Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Priority: Critical Labels: client-script, kafka-run-class.sh, shell-script Fix For: 0.8.2.1 Original Estimate: 24h Remaining Estimate: 24h kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
[ https://issues.apache.org/jira/browse/KAFKA-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2254: --- Status: Open (was: Patch Available) The shell script should be optimized , even kafka-run-class.sh has a syntax error. -- Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Priority: Critical Labels: client-script, kafka-run-class.sh, shell-script Fix For: 0.8.2.1 Original Estimate: 24h Remaining Estimate: 24h kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
[ https://issues.apache.org/jira/browse/KAFKA-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2254: --- Attachment: kafka-shell-script.patch The shell script should be optimized , even kafka-run-class.sh has a syntax error. -- Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Priority: Critical Labels: client-script, kafka-run-class.sh, shell-script Fix For: 0.8.2.1 Attachments: kafka-shell-script.patch Original Estimate: 24h Remaining Estimate: 24h kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
[ https://issues.apache.org/jira/browse/KAFKA-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578153#comment-14578153 ] Bo Wang commented on KAFKA-2254: The trunk have fixed kafka-run-class.sh syntax error. The shell script should be optimized , even kafka-run-class.sh has a syntax error. -- Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Labels: client-script, kafka-run-class.sh, shell-script Fix For: 0.8.2.1 Attachments: kafka-shell-script.patch Original Estimate: 24h Remaining Estimate: 24h kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
[ https://issues.apache.org/jira/browse/KAFKA-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated KAFKA-2254: --- Description: kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 was: kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors and warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 The shell script should be optimized , even kafka-run-class.sh has a syntax error. -- Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Priority: Critical Labels: client-script, kafka-run-class.sh, shell-script Fix For: 0.8.2.1 Original Estimate: 24h Remaining Estimate: 24h kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors 、 warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2254) The shell script should be optimized , even kafka-run-class.sh has a syntax error.
Bo Wang created KAFKA-2254: -- Summary: The shell script should be optimized , even kafka-run-class.sh has a syntax error. Key: KAFKA-2254 URL: https://issues.apache.org/jira/browse/KAFKA-2254 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Environment: linux Reporter: Bo Wang Priority: Critical Fix For: 0.8.2.1 kafka-run-class.sh 128 line has a syntax error(missing a space): 127-loggc) 128 if [ -z $KAFKA_GC_LOG_OPTS] ; then 129GC_LOG_ENABLED=true 130 fi And use the ShellCheck to check the shell scripts, the results shows some errors and warnings and notes： https://github.com/koalaman/shellcheck/wiki/SC2068 https://github.com/koalaman/shellcheck/wiki/Sc2046 https://github.com/koalaman/shellcheck/wiki/Sc2086 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-2122) Remove controller.message.queue.size Config
[ https://issues.apache.org/jira/browse/KAFKA-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502583#comment-14502583 ] Bo Wang edited comment on KAFKA-2122 at 4/20/15 10:27 AM: -- Which version this patch is based on? was (Author: wangbo23): Which version this patch is based on? I found KAFKA-2122_2015-04-19_12: 44: 41.patch is not based on 0.8.2.1? Remove controller.message.queue.size Config --- Key: KAFKA-2122 URL: https://issues.apache.org/jira/browse/KAFKA-2122 Project: Kafka Issue Type: Bug Reporter: Onur Karaman Assignee: Sriharsha Chintalapani Attachments: KAFKA-2122.patch, KAFKA-2122_2015-04-19_12:44:41.patch A deadlock can happen during a delete topic if controller.message.queue.size is overridden to a custom value. Details are here: https://issues.apache.org/jira/browse/KAFKA-2046?focusedCommentId=14380776page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14380776 Given that KAFKA-1993 is enabling delete topic by default, it would be unsafe to simultaneously allow a configurable controller.message.queue.size -- This message was sent by Atlassian JIRA (v6.3.4#6332)
