[jira] [Commented] (KAFKA-3736) Add http metrics reporter
[ https://issues.apache.org/jira/browse/KAFKA-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296063#comment-15296063 ] Adrian Muraru commented on KAFKA-3736: -- Ok, just noticed that "Monitoring" is considered public facing change so I'm going to create a KIP. For some reason I cannot add a new page on wiki here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals Do I need special perms? Thanks a lot. > Add http metrics reporter > - > > Key: KAFKA-3736 > URL: https://issues.apache.org/jira/browse/KAFKA-3736 > Project: Kafka > Issue Type: New Feature > Components: core >Reporter: Adrian Muraru > Fix For: 0.10.1.0 > > > The current builtin JMX metrics reporter is pretty heavy in terms of load and > collection. A new http lightweight reporter is proposed to expose the metrics > via a local http port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3736) Add http metrics reporter
[ https://issues.apache.org/jira/browse/KAFKA-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293377#comment-15293377 ] Adrian Muraru commented on KAFKA-3736: -- But is this change changing the public interfaces? > Add http metrics reporter > - > > Key: KAFKA-3736 > URL: https://issues.apache.org/jira/browse/KAFKA-3736 > Project: Kafka > Issue Type: New Feature > Components: core >Reporter: Adrian Muraru > Fix For: 0.10.1.0 > > > The current builtin JMX metrics reporter is pretty heavy in terms of load and > collection. A new http lightweight reporter is proposed to expose the metrics > via a local http port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3736) Add http metrics reporter
[ https://issues.apache.org/jira/browse/KAFKA-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293323#comment-15293323 ] Adrian Muraru commented on KAFKA-3736: -- I can add a KIP no problem but I was wondering if this is really a new feature given the fact the support is already there and there are two already reporters availbale: JMX and CSV. > Add http metrics reporter > - > > Key: KAFKA-3736 > URL: https://issues.apache.org/jira/browse/KAFKA-3736 > Project: Kafka > Issue Type: New Feature > Components: core >Reporter: Adrian Muraru > Fix For: 0.10.1.0 > > > The current builtin JMX metrics reporter is pretty heavy in terms of load and > collection. A new http lightweight reporter is proposed to expose the metrics > via a local http port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3736) Add http metrics reporter
[
https://issues.apache.org/jira/browse/KAFKA-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293322#comment-15293322
]
Adrian Muraru commented on KAFKA-3736:
--
Agree - let me see if I can extract this to a {{-metrics-reporter}} module.
> Add http metrics reporter
> -
>
> Key: KAFKA-3736
> URL: https://issues.apache.org/jira/browse/KAFKA-3736
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Reporter: Adrian Muraru
> Fix For: 0.10.1.0
>
>
> The current builtin JMX metrics reporter is pretty heavy in terms of load and
> collection. A new http lightweight reporter is proposed to expose the metrics
> via a local http port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-3733) Avoid long command lines by setting CLASSPATH in environment
[
https://issues.apache.org/jira/browse/KAFKA-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrian Muraru updated KAFKA-3733:
-
Fix Version/s: 0.10.0.0
Status: Patch Available (was: Open)
> Avoid long command lines by setting CLASSPATH in environment
>
>
> Key: KAFKA-3733
> URL: https://issues.apache.org/jira/browse/KAFKA-3733
> Project: Kafka
> Issue Type: Improvement
> Components: tools
>Reporter: Adrian Muraru
>Priority: Minor
> Fix For: 0.10.0.0
>
>
> {{kafka-run-class.sh}} sets the JVM classpath in the command line via {{-cp}}.
> This generates long command lines that gets trimmed by the shell in commands
> like ps, pgrep,etc.
> An alternative is to set the CLASSPATH in environment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-3736) Add http metrics reporter
[ https://issues.apache.org/jira/browse/KAFKA-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Muraru updated KAFKA-3736: - Fix Version/s: 0.10.0.0 Status: Patch Available (was: Open) > Add http metrics reporter > - > > Key: KAFKA-3736 > URL: https://issues.apache.org/jira/browse/KAFKA-3736 > Project: Kafka > Issue Type: New Feature > Components: core >Reporter: Adrian Muraru > Fix For: 0.10.0.0 > > > The current builtin JMX metrics reporter is pretty heavy in terms of load and > collection. A new http lightweight reporter is proposed to expose the metrics > via a local http port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3736) Add http metrics reporter
Adrian Muraru created KAFKA-3736: Summary: Add http metrics reporter Key: KAFKA-3736 URL: https://issues.apache.org/jira/browse/KAFKA-3736 Project: Kafka Issue Type: New Feature Components: core Reporter: Adrian Muraru The current builtin JMX metrics reporter is pretty heavy in terms of load and collection. A new http lightweight reporter is proposed to expose the metrics via a local http port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3733) Avoid long command lines by setting CLASSPATH in environment
Adrian Muraru created KAFKA-3733:
Summary: Avoid long command lines by setting CLASSPATH in
environment
Key: KAFKA-3733
URL: https://issues.apache.org/jira/browse/KAFKA-3733
Project: Kafka
Issue Type: Improvement
Components: tools
Reporter: Adrian Muraru
Priority: Minor
{{kafka-run-class.sh}} sets the JVM classpath in the command line via {{-cp}}.
This generates long command lines that gets trimmed by the shell in commands
like ps, pgrep,etc.
An alternative is to set the CLASSPATH in environment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1070) Auto-assign node id
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734938#comment-14734938 ] Adrian Muraru commented on KAFKA-1070: -- [~harsha_ch] I see KAFKA-1973 already fixed that, nw :) > Auto-assign node id > --- > > Key: KAFKA-1070 > URL: https://issues.apache.org/jira/browse/KAFKA-1070 > Project: Kafka > Issue Type: Bug >Reporter: Jay Kreps >Assignee: Sriharsha Chintalapani > Labels: usability > Fix For: 0.8.3 > > Attachments: KAFKA-1070.patch, KAFKA-1070_2014-07-19_16:06:13.patch, > KAFKA-1070_2014-07-22_11:34:18.patch, KAFKA-1070_2014-07-24_20:58:17.patch, > KAFKA-1070_2014-07-24_21:05:33.patch, KAFKA-1070_2014-08-21_10:26:20.patch, > KAFKA-1070_2014-11-20_10:50:04.patch, KAFKA-1070_2014-11-25_20:29:37.patch, > KAFKA-1070_2015-01-01_17:39:30.patch, KAFKA-1070_2015-01-12_10:46:54.patch, > KAFKA-1070_2015-01-12_18:30:17.patch > > > It would be nice to have Kafka brokers auto-assign node ids rather than > having that be a configuration. Having a configuration is irritating because > (1) you have to generate a custom config for each broker and (2) even though > it is in configuration, changing the node id can cause all kinds of bad > things to happen. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1070) Auto-assign node id
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731941#comment-14731941 ] Adrian Muraru commented on KAFKA-1070: -- Saw it, that's why I raised the flag here, the patch in trunk seems broken with the .orig file added. A new jira issue might be needed to fix it > Auto-assign node id > --- > > Key: KAFKA-1070 > URL: https://issues.apache.org/jira/browse/KAFKA-1070 > Project: Kafka > Issue Type: Bug >Reporter: Jay Kreps >Assignee: Sriharsha Chintalapani > Labels: usability > Fix For: 0.8.3 > > Attachments: KAFKA-1070.patch, KAFKA-1070_2014-07-19_16:06:13.patch, > KAFKA-1070_2014-07-22_11:34:18.patch, KAFKA-1070_2014-07-24_20:58:17.patch, > KAFKA-1070_2014-07-24_21:05:33.patch, KAFKA-1070_2014-08-21_10:26:20.patch, > KAFKA-1070_2014-11-20_10:50:04.patch, KAFKA-1070_2014-11-25_20:29:37.patch, > KAFKA-1070_2015-01-01_17:39:30.patch, KAFKA-1070_2015-01-12_10:46:54.patch, > KAFKA-1070_2015-01-12_18:30:17.patch > > > It would be nice to have Kafka brokers auto-assign node ids rather than > having that be a configuration. Having a configuration is irritating because > (1) you have to generate a custom config for each broker and (2) even though > it is in configuration, changing the node id can cause all kinds of bad > things to happen. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1070) Auto-assign node id
[
https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730763#comment-14730763
]
Adrian Muraru commented on KAFKA-1070:
--
[~nehanarkhede] For some reason this patch was not applied correctly in trunk
branch, I see an {{.orig}} file in the patch here:
https://github.com/apache/kafka/commit/b1b80860a01cc378cfada3549a3480f0773c3ff8#diff-d0716898c8daed9887ef83500ee0e16e
> Auto-assign node id
> ---
>
> Key: KAFKA-1070
> URL: https://issues.apache.org/jira/browse/KAFKA-1070
> Project: Kafka
> Issue Type: Bug
>Reporter: Jay Kreps
>Assignee: Sriharsha Chintalapani
> Labels: usability
> Fix For: 0.8.3
>
> Attachments: KAFKA-1070.patch, KAFKA-1070_2014-07-19_16:06:13.patch,
> KAFKA-1070_2014-07-22_11:34:18.patch, KAFKA-1070_2014-07-24_20:58:17.patch,
> KAFKA-1070_2014-07-24_21:05:33.patch, KAFKA-1070_2014-08-21_10:26:20.patch,
> KAFKA-1070_2014-11-20_10:50:04.patch, KAFKA-1070_2014-11-25_20:29:37.patch,
> KAFKA-1070_2015-01-01_17:39:30.patch, KAFKA-1070_2015-01-12_10:46:54.patch,
> KAFKA-1070_2015-01-12_18:30:17.patch
>
>
> It would be nice to have Kafka brokers auto-assign node ids rather than
> having that be a configuration. Having a configuration is irritating because
> (1) you have to generate a custom config for each broker and (2) even though
> it is in configuration, changing the node id can cause all kinds of bad
> things to happen.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-2237) zookeeper chroot path should be created on broker startup
[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569586#comment-14569586 ] Adrian Muraru commented on KAFKA-2237: -- Do you know by chance the JIRA id where this was implemented? zookeeper chroot path should be created on broker startup - Key: KAFKA-2237 URL: https://issues.apache.org/jira/browse/KAFKA-2237 Project: Kafka Issue Type: Bug Components: config Reporter: Adrian Muraru Right now when a chroot path is used to maintain zk metadata and not available at startup - broker fails to start. Would it make sense to have it automatically created by the broker itself if it doesn't exist? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2237) zookeeper chroot path should be created on broker startup
[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569583#comment-14569583 ] Adrian Muraru commented on KAFKA-2237: -- [~nehanarkhede] I'm on 0.8.2 branch zookeeper chroot path should be created on broker startup - Key: KAFKA-2237 URL: https://issues.apache.org/jira/browse/KAFKA-2237 Project: Kafka Issue Type: Bug Components: config Reporter: Adrian Muraru Right now when a chroot path is used to maintain zk metadata and not available at startup - broker fails to start. Would it make sense to have it automatically created by the broker itself if it doesn't exist? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2237) zookeeper chroot path should be created on broker startup
[ https://issues.apache.org/jira/browse/KAFKA-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569752#comment-14569752 ] Adrian Muraru commented on KAFKA-2237: -- False alarm, yeah on 0.8.2 it seems to work fine. It was 0.8.1 where this was seen. Let's close was Cannot Reproduce or smth. Thanks zookeeper chroot path should be created on broker startup - Key: KAFKA-2237 URL: https://issues.apache.org/jira/browse/KAFKA-2237 Project: Kafka Issue Type: Bug Components: config Reporter: Adrian Muraru Right now when a chroot path is used to maintain zk metadata and not available at startup - broker fails to start. Would it make sense to have it automatically created by the broker itself if it doesn't exist? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2237) zookeeper chroot path should be created on broker startup
Adrian Muraru created KAFKA-2237: Summary: zookeeper chroot path should be created on broker startup Key: KAFKA-2237 URL: https://issues.apache.org/jira/browse/KAFKA-2237 Project: Kafka Issue Type: Bug Components: config Reporter: Adrian Muraru Right now when a chroot path is used to maintain zk metadata and not available at startup - broker fails to start. Would it make sense to have it automatically created by the broker itself if it doesn't exist? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1465) kafka-reassign-partitions.sh fails when topic name contains dash/hyphen
[
https://issues.apache.org/jira/browse/KAFKA-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011125#comment-14011125
]
Adrian Muraru commented on KAFKA-1465:
--
[~junrao] Updated the description with the json content
kafka-reassign-partitions.sh fails when topic name contains dash/hyphen
---
Key: KAFKA-1465
URL: https://issues.apache.org/jira/browse/KAFKA-1465
Project: Kafka
Issue Type: Bug
Components: tools
Affects Versions: 0.8.0
Reporter: Adrian Muraru
{{./bin/kafka-reassign-partitions.sh --topics-to-move-json-file
~/rebalance-topic.json --broker-list 18,19 --zookeeper $ZK_QUORUM
--execute}}
{code}
Partitions reassignment failed due to Can't parse json string: null
kafka.common.KafkaException: Can't parse json string: null
at kafka.utils.Json$.liftedTree1$1(Json.scala:36)
at kafka.utils.Json$.parseFull(Json.scala:32)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:529)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:525)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at kafka.utils.ZkUtils$.getReplicaAssignmentForTopics(ZkUtils.scala:525)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:112)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: java.lang.NullPointerException
at
scala.util.parsing.combinator.lexical.Scanners$Scanner.init(Scanners.scala:52)
at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:71)
at scala.util.parsing.json.JSON$.parseFull(JSON.scala:85)
at kafka.utils.Json$.liftedTree1$1(Json.scala:33)
... 8 more
{code}
*rebalance-topic.json*
{code}
{topics:
[{topic: metrics-logs}],
version:1
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (KAFKA-1465) kafka-reassign-partitions.sh fails when topic name contains dash/hyphen
[
https://issues.apache.org/jira/browse/KAFKA-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrian Muraru updated KAFKA-1465:
-
Description:
{{./bin/kafka-reassign-partitions.sh --topics-to-move-json-file
~/rebalance-topic.json --broker-list 18,19 --zookeeper $ZK_QUORUM --execute}}
{code}
Partitions reassignment failed due to Can't parse json string: null
kafka.common.KafkaException: Can't parse json string: null
at kafka.utils.Json$.liftedTree1$1(Json.scala:36)
at kafka.utils.Json$.parseFull(Json.scala:32)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:529)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:525)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at kafka.utils.ZkUtils$.getReplicaAssignmentForTopics(ZkUtils.scala:525)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:112)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: java.lang.NullPointerException
at
scala.util.parsing.combinator.lexical.Scanners$Scanner.init(Scanners.scala:52)
at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:71)
at scala.util.parsing.json.JSON$.parseFull(JSON.scala:85)
at kafka.utils.Json$.liftedTree1$1(Json.scala:33)
... 8 more
{code}
*rebalance-topic.json*
{code}
{topics:
[{topic: metrics-logs}],
version:1
}
{code}
was:
{{./bin/kafka-reassign-partitions.sh --topics-to-move-json-file
~/rebalance-topic.json --broker-list 18,19 --zookeeper $ZK_QUORUM --execute}}
{code}
Partitions reassignment failed due to Can't parse json string: null
kafka.common.KafkaException: Can't parse json string: null
at kafka.utils.Json$.liftedTree1$1(Json.scala:36)
at kafka.utils.Json$.parseFull(Json.scala:32)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:529)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:525)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at kafka.utils.ZkUtils$.getReplicaAssignmentForTopics(ZkUtils.scala:525)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:112)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: java.lang.NullPointerException
at
scala.util.parsing.combinator.lexical.Scanners$Scanner.init(Scanners.scala:52)
at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:71)
at scala.util.parsing.json.JSON$.parseFull(JSON.scala:85)
at kafka.utils.Json$.liftedTree1$1(Json.scala:33)
... 8 more
{code}
kafka-reassign-partitions.sh fails when topic name contains dash/hyphen
---
Key: KAFKA-1465
URL: https://issues.apache.org/jira/browse/KAFKA-1465
Project: Kafka
Issue Type: Bug
Components: tools
Affects Versions: 0.8.0
Reporter: Adrian Muraru
{{./bin/kafka-reassign-partitions.sh --topics-to-move-json-file
~/rebalance-topic.json --broker-list 18,19 --zookeeper $ZK_QUORUM
--execute}}
{code}
Partitions reassignment failed due to Can't parse json string: null
kafka.common.KafkaException: Can't parse json string: null
at kafka.utils.Json$.liftedTree1$1(Json.scala:36)
at kafka.utils.Json$.parseFull(Json.scala:32)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:529)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:525)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at kafka.utils.ZkUtils$.getReplicaAssignmentForTopics(ZkUtils.scala:525)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:112)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: java.lang.NullPointerException
at
scala.util.parsing.combinator.lexical.Scanners$Scanner.init(Scanners.scala:52)
at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:71)
at scala.util.parsing.json.JSON$.parseFull(JSON.scala:85)
at kafka.utils.Json$.liftedTree1$1(Json.scala:33)
... 8 more
{code}
*rebalance-topic.json*
{code}
{topics:
[{topic: metrics-logs}],
version:1
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (KAFKA-1465) kafka-reassign-partitions.sh fails when topic name contains dash/hyphen
Adrian Muraru created KAFKA-1465:
Summary: kafka-reassign-partitions.sh fails when topic name
contains dash/hyphen
Key: KAFKA-1465
URL: https://issues.apache.org/jira/browse/KAFKA-1465
Project: Kafka
Issue Type: Bug
Components: tools
Affects Versions: 0.8.0
Reporter: Adrian Muraru
{{./bin/kafka-reassign-partitions.sh --topics-to-move-json-file
~/rebalance-topic.json --broker-list 18,19 --zookeeper $ZK_QUORUM --execute}}
{code}
Partitions reassignment failed due to Can't parse json string: null
kafka.common.KafkaException: Can't parse json string: null
at kafka.utils.Json$.liftedTree1$1(Json.scala:36)
at kafka.utils.Json$.parseFull(Json.scala:32)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:529)
at
kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:525)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at kafka.utils.ZkUtils$.getReplicaAssignmentForTopics(ZkUtils.scala:525)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:112)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: java.lang.NullPointerException
at
scala.util.parsing.combinator.lexical.Scanners$Scanner.init(Scanners.scala:52)
at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:71)
at scala.util.parsing.json.JSON$.parseFull(JSON.scala:85)
at kafka.utils.Json$.liftedTree1$1(Json.scala:33)
... 8 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1183) DefaultEventHandler causes unbalanced distribution of messages across partitions
[
https://issues.apache.org/jira/browse/KAFKA-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847812#comment-13847812
]
Adrian Muraru commented on KAFKA-1183:
--
[~junrao] Jun, what's the impact of lowering the metadata refresh interval to
say 10 seconds, this would solve the partition stickiness in the producer but
would that have other performance side effects? Reading the email thread on
KAFKA-1067 I see that creating a customised random partioner is not possible at
this point as it's not aware of AVAILABLE partitions. Thanks
DefaultEventHandler causes unbalanced distribution of messages across
partitions
Key: KAFKA-1183
URL: https://issues.apache.org/jira/browse/KAFKA-1183
Project: Kafka
Issue Type: Bug
Components: producer
Affects Versions: 0.8.0
Reporter: Dragos Dena
Assignee: Jun Rao
Fix For: 0.8.1, 0.9.0
Attachments: KAFKA-1183-trunk.patch
KAFKA-959 introduced an optimisation in {{DefaultEventHandler}} that was
supposed to have the effect of sending all messages from the same batch to a
single partition if no key is specified.
The problem is that the {{sendPartitionPerTopicCache}} cache, which holds the
current selected partition for each topic, isn't actually invalided at the
start or end of each batch.
The observed result is that, after the first request chooses a random
partition, all subsequent messages from that producer land in the same
partition. If you have a large number of producers, then it should be fine,
but if your producer count is comparable to the partition count, then it will
get unbalanced.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (KAFKA-1067) the default partitioner should be randomizing messages and a new partition for the meta refresh requirements created
[ https://issues.apache.org/jira/browse/KAFKA-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847848#comment-13847848 ] Adrian Muraru commented on KAFKA-1067: -- A more readable version of the email thread discussing this issue: http://markmail.org/thread/wbr2jni7ndimeoml Any news on updating the FAQ with the conclusion in that thread? the default partitioner should be randomizing messages and a new partition for the meta refresh requirements created Key: KAFKA-1067 URL: https://issues.apache.org/jira/browse/KAFKA-1067 Project: Kafka Issue Type: Bug Reporter: Joe Stein Fix For: 0.8.1 Details behind this http://mail-archives.apache.org/mod_mbox/kafka-dev/201310.mbox/%3CCAFbh0Q0aVh%2Bvqxfy7H-%2BMnRFBt6BnyoZk1LWBoMspwSmTqUKMg%40mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.1.4#6159)
