[jira] [Updated] (KAFKA-1992) Following KAFKA-1697, checkEnoughReplicasReachOffset doesn't need to get requiredAcks
[ https://issues.apache.org/jira/browse/KAFKA-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1992: Attachment: KAFKA-1992.patch > Following KAFKA-1697, checkEnoughReplicasReachOffset doesn't need to get > requiredAcks > - > > Key: KAFKA-1992 > URL: https://issues.apache.org/jira/browse/KAFKA-1992 > Project: Kafka > Issue Type: Bug >Reporter: Gwen Shapira > Attachments: KAFKA-1992.patch > > > Follow up from Jun's review: > "Should we just remove requiredAcks completely since > checkEnoughReplicasReachOffset() will only be called when requiredAcks is -1?" > Answer is: Yes, we should :) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1992) Following KAFKA-1697, checkEnoughReplicasReachOffset doesn't need to get requiredAcks
[ https://issues.apache.org/jira/browse/KAFKA-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1992: Assignee: Gwen Shapira Status: Patch Available (was: Open) > Following KAFKA-1697, checkEnoughReplicasReachOffset doesn't need to get > requiredAcks > - > > Key: KAFKA-1992 > URL: https://issues.apache.org/jira/browse/KAFKA-1992 > Project: Kafka > Issue Type: Bug >Reporter: Gwen Shapira >Assignee: Gwen Shapira > Attachments: KAFKA-1992.patch > > > Follow up from Jun's review: > "Should we just remove requiredAcks completely since > checkEnoughReplicasReachOffset() will only be called when requiredAcks is -1?" > Answer is: Yes, we should :) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-1990) Add unlimited time-based log retention
[ https://issues.apache.org/jira/browse/KAFKA-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Holoman reassigned KAFKA-1990: --- Assignee: Jeff Holoman > Add unlimited time-based log retention > -- > > Key: KAFKA-1990 > URL: https://issues.apache.org/jira/browse/KAFKA-1990 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps >Assignee: Jeff Holoman > Labels: newbie > > Currently you can set > log.retention.bytes = -1 > to disable size based retention (in fact that is the default). However, there > is no equivalent for time based retention. You can set time based retention > to something really big like 2147483647 hours, which in practical terms is > forever, but it is kind of silly to require this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31591: Patch for KAFKA-1992
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31591/#review74699 --- Looks good to me. - Jeff Holoman On March 1, 2015, 7:58 a.m., Gwen Shapira wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/31591/ > --- > > (Updated March 1, 2015, 7:58 a.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1992 > https://issues.apache.org/jira/browse/KAFKA-1992 > > > Repository: kafka > > > Description > --- > > remove unnecessary requiredAcks parameter and clean up few comments > > > Diffs > - > > core/src/main/scala/kafka/cluster/Partition.scala > c4bf48a801007ebe7497077d2018d6dffe1677d4 > core/src/main/scala/kafka/server/DelayedProduce.scala > 4d763bf05efb24a394662721292fc54d32467969 > > Diff: https://reviews.apache.org/r/31591/diff/ > > > Testing > --- > > > Thanks, > > Gwen Shapira > >
[jira] [Updated] (KAFKA-1809) Refactor brokers to allow listening on multiple ports and IPs
[ https://issues.apache.org/jira/browse/KAFKA-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1809: Attachment: KAFKA-1809.v3.patch Another rebase... > Refactor brokers to allow listening on multiple ports and IPs > -- > > Key: KAFKA-1809 > URL: https://issues.apache.org/jira/browse/KAFKA-1809 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Gwen Shapira >Assignee: Gwen Shapira > Attachments: KAFKA-1809.patch, KAFKA-1809.v1.patch, > KAFKA-1809.v2.patch, KAFKA-1809.v3.patch, > KAFKA-1809_2014-12-25_11:04:17.patch, KAFKA-1809_2014-12-27_12:03:35.patch, > KAFKA-1809_2014-12-27_12:22:56.patch, KAFKA-1809_2014-12-29_12:11:36.patch, > KAFKA-1809_2014-12-30_11:25:29.patch, KAFKA-1809_2014-12-30_18:48:46.patch, > KAFKA-1809_2015-01-05_14:23:57.patch, KAFKA-1809_2015-01-05_20:25:15.patch, > KAFKA-1809_2015-01-05_21:40:14.patch, KAFKA-1809_2015-01-06_11:46:22.patch, > KAFKA-1809_2015-01-13_18:16:21.patch, KAFKA-1809_2015-01-28_10:26:22.patch, > KAFKA-1809_2015-02-02_11:55:16.patch, KAFKA-1809_2015-02-03_10:45:31.patch, > KAFKA-1809_2015-02-03_10:46:42.patch, KAFKA-1809_2015-02-03_10:52:36.patch > > > The goal is to eventually support different security mechanisms on different > ports. > Currently brokers are defined as host+port pair, and this definition exists > throughout the code-base, therefore some refactoring is needed to support > multiple ports for a single broker. > The detailed design is here: > https://cwiki.apache.org/confluence/display/KAFKA/Multiple+Listeners+for+Kafka+Brokers -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1809) Refactor brokers to allow listening on multiple ports and IPs
[ https://issues.apache.org/jira/browse/KAFKA-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342466#comment-14342466 ] Gwen Shapira commented on KAFKA-1809: - [~junrao] - patch updated per your review. Will appreciate if you can take a look :) > Refactor brokers to allow listening on multiple ports and IPs > -- > > Key: KAFKA-1809 > URL: https://issues.apache.org/jira/browse/KAFKA-1809 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Gwen Shapira >Assignee: Gwen Shapira > Attachments: KAFKA-1809.patch, KAFKA-1809.v1.patch, > KAFKA-1809.v2.patch, KAFKA-1809.v3.patch, > KAFKA-1809_2014-12-25_11:04:17.patch, KAFKA-1809_2014-12-27_12:03:35.patch, > KAFKA-1809_2014-12-27_12:22:56.patch, KAFKA-1809_2014-12-29_12:11:36.patch, > KAFKA-1809_2014-12-30_11:25:29.patch, KAFKA-1809_2014-12-30_18:48:46.patch, > KAFKA-1809_2015-01-05_14:23:57.patch, KAFKA-1809_2015-01-05_20:25:15.patch, > KAFKA-1809_2015-01-05_21:40:14.patch, KAFKA-1809_2015-01-06_11:46:22.patch, > KAFKA-1809_2015-01-13_18:16:21.patch, KAFKA-1809_2015-01-28_10:26:22.patch, > KAFKA-1809_2015-02-02_11:55:16.patch, KAFKA-1809_2015-02-03_10:45:31.patch, > KAFKA-1809_2015-02-03_10:46:42.patch, KAFKA-1809_2015-02-03_10:52:36.patch > > > The goal is to eventually support different security mechanisms on different > ports. > Currently brokers are defined as host+port pair, and this definition exists > throughout the code-base, therefore some refactoring is needed to support > multiple ports for a single broker. > The detailed design is here: > https://cwiki.apache.org/confluence/display/KAFKA/Multiple+Listeners+for+Kafka+Brokers -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1993) Enable topic deletion as default
Gwen Shapira created KAFKA-1993: --- Summary: Enable topic deletion as default Key: KAFKA-1993 URL: https://issues.apache.org/jira/browse/KAFKA-1993 Project: Kafka Issue Type: Bug Reporter: Gwen Shapira Since topic deletion is now throughly tested and works as well as most Kafka features, we should enable it by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-330) Add delete topic support
[ https://issues.apache.org/jira/browse/KAFKA-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342487#comment-14342487 ] Gwen Shapira commented on KAFKA-330: created KAFKA-1993 to track this suggestion. > Add delete topic support > - > > Key: KAFKA-330 > URL: https://issues.apache.org/jira/browse/KAFKA-330 > Project: Kafka > Issue Type: New Feature > Components: controller, log, replication >Affects Versions: 0.8.0, 0.8.1 >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: features, project > Fix For: 0.8.1 > > Attachments: KAFKA-330.patch, KAFKA-330_2014-01-28_15:19:20.patch, > KAFKA-330_2014-01-28_22:01:16.patch, KAFKA-330_2014-01-31_14:19:14.patch, > KAFKA-330_2014-01-31_17:45:25.patch, KAFKA-330_2014-02-01_11:30:32.patch, > KAFKA-330_2014-02-01_14:58:31.patch, KAFKA-330_2014-02-05_09:31:30.patch, > KAFKA-330_2014-02-06_07:48:40.patch, KAFKA-330_2014-02-06_09:42:38.patch, > KAFKA-330_2014-02-06_10:29:31.patch, KAFKA-330_2014-02-06_11:37:48.patch, > KAFKA-330_2014-02-08_11:07:37.patch, kafka-330-v1.patch, kafka-330-v2.patch > > > One proposal of this API is here - > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+replication+detailed+design+V2#KafkareplicationdetaileddesignV2-Deletetopic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 31606: Patch for KAFKA-1416
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31606/ --- Review request for kafka. Bugs: KAFKA-1416 https://issues.apache.org/jira/browse/KAFKA-1416 Repository: kafka Description --- Initial Patch Diffs - core/src/test/scala/unit/kafka/consumer/ZookeeperConsumerConnectorTest.scala a17e8532c44aadf84b8da3a57bcc797a848b5020 core/src/test/scala/unit/kafka/integration/FetcherTest.scala 25845abbcad2e79f56f729e59239b738d3ddbc9d core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala ba3bcdcd1de9843e75e5395dff2fc31b39a5a9d5 core/src/test/scala/unit/kafka/javaapi/consumer/ZookeeperConsumerConnectorTest.scala d6248b09bb0f86ee7d3bd0ebce5b99135491453b core/src/test/scala/unit/kafka/metrics/MetricsTest.scala 111e4a26c1efb6f7c151ca9217dbe107c27ab617 core/src/test/scala/unit/kafka/utils/TestUtils.scala 6ce18076f6b5deb05b51c25be5bed9957e6b4339 Diff: https://reviews.apache.org/r/31606/diff/ Testing --- Thanks, Flutra Osmani
[jira] [Updated] (KAFKA-1416) Unify sendMessages/getMessages in unit tests
[ https://issues.apache.org/jira/browse/KAFKA-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flutra Osmani updated KAFKA-1416: - Attachment: KAFKA-1416.patch > Unify sendMessages/getMessages in unit tests > > > Key: KAFKA-1416 > URL: https://issues.apache.org/jira/browse/KAFKA-1416 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Flutra Osmani > Labels: newbie > Attachments: KAFKA-1416.patch > > > Multiple unit tests have its own internal function to send/get messages from > the brokers. For example: > sendMessages in ZookeeperConsumerConnectorTest > produceMessage in UncleanLeaderElectionTest > sendMessages in FetcherTest > etc > It is better to unify them in TestUtils. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1416) Unify sendMessages/getMessages in unit tests
[ https://issues.apache.org/jira/browse/KAFKA-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flutra Osmani updated KAFKA-1416: - Status: Patch Available (was: Open) > Unify sendMessages/getMessages in unit tests > > > Key: KAFKA-1416 > URL: https://issues.apache.org/jira/browse/KAFKA-1416 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Flutra Osmani > Labels: newbie > Attachments: KAFKA-1416.patch > > > Multiple unit tests have its own internal function to send/get messages from > the brokers. For example: > sendMessages in ZookeeperConsumerConnectorTest > produceMessage in UncleanLeaderElectionTest > sendMessages in FetcherTest > etc > It is better to unify them in TestUtils. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1416) Unify sendMessages/getMessages in unit tests
[ https://issues.apache.org/jira/browse/KAFKA-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342565#comment-14342565 ] Flutra Osmani commented on KAFKA-1416: -- Created reviewboard https://reviews.apache.org/r/31606/diff/ against branch origin/trunk > Unify sendMessages/getMessages in unit tests > > > Key: KAFKA-1416 > URL: https://issues.apache.org/jira/browse/KAFKA-1416 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Flutra Osmani > Labels: newbie > Attachments: KAFKA-1416.patch > > > Multiple unit tests have its own internal function to send/get messages from > the brokers. For example: > sendMessages in ZookeeperConsumerConnectorTest > produceMessage in UncleanLeaderElectionTest > sendMessages in FetcherTest > etc > It is better to unify them in TestUtils. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1416) Unify sendMessages/getMessages in unit tests
[ https://issues.apache.org/jira/browse/KAFKA-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342574#comment-14342574 ] Flutra Osmani commented on KAFKA-1416: -- Thank you. I just committed a patch. > Unify sendMessages/getMessages in unit tests > > > Key: KAFKA-1416 > URL: https://issues.apache.org/jira/browse/KAFKA-1416 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Flutra Osmani > Labels: newbie > Attachments: KAFKA-1416.patch > > > Multiple unit tests have its own internal function to send/get messages from > the brokers. For example: > sendMessages in ZookeeperConsumerConnectorTest > produceMessage in UncleanLeaderElectionTest > sendMessages in FetcherTest > etc > It is better to unify them in TestUtils. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31606: Patch for KAFKA-1416
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31606/ --- (Updated March 2, 2015, 1:25 a.m.) Review request for kafka. Bugs: KAFKA-1416 https://issues.apache.org/jira/browse/KAFKA-1416 Repository: kafka Description --- Initial Patch Diffs (updated) - core/src/test/scala/unit/kafka/consumer/ZookeeperConsumerConnectorTest.scala a17e8532c44aadf84b8da3a57bcc797a848b5020 core/src/test/scala/unit/kafka/integration/FetcherTest.scala 25845abbcad2e79f56f729e59239b738d3ddbc9d core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala ba3bcdcd1de9843e75e5395dff2fc31b39a5a9d5 core/src/test/scala/unit/kafka/javaapi/consumer/ZookeeperConsumerConnectorTest.scala d6248b09bb0f86ee7d3bd0ebce5b99135491453b core/src/test/scala/unit/kafka/metrics/MetricsTest.scala 111e4a26c1efb6f7c151ca9217dbe107c27ab617 core/src/test/scala/unit/kafka/utils/TestUtils.scala 6ce18076f6b5deb05b51c25be5bed9957e6b4339 Diff: https://reviews.apache.org/r/31606/diff/ Testing --- Thanks, Flutra Osmani
[jira] [Updated] (KAFKA-1416) Unify sendMessages/getMessages in unit tests
[ https://issues.apache.org/jira/browse/KAFKA-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flutra Osmani updated KAFKA-1416: - Attachment: KAFKA-1416_2015-03-01_17:24:55.patch > Unify sendMessages/getMessages in unit tests > > > Key: KAFKA-1416 > URL: https://issues.apache.org/jira/browse/KAFKA-1416 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Flutra Osmani > Labels: newbie > Attachments: KAFKA-1416.patch, KAFKA-1416_2015-03-01_17:24:55.patch > > > Multiple unit tests have its own internal function to send/get messages from > the brokers. For example: > sendMessages in ZookeeperConsumerConnectorTest > produceMessage in UncleanLeaderElectionTest > sendMessages in FetcherTest > etc > It is better to unify them in TestUtils. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1416) Unify sendMessages/getMessages in unit tests
[ https://issues.apache.org/jira/browse/KAFKA-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342592#comment-14342592 ] Flutra Osmani commented on KAFKA-1416: -- Updated reviewboard https://reviews.apache.org/r/31606/diff/ against branch origin/trunk > Unify sendMessages/getMessages in unit tests > > > Key: KAFKA-1416 > URL: https://issues.apache.org/jira/browse/KAFKA-1416 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Flutra Osmani > Labels: newbie > Attachments: KAFKA-1416.patch, KAFKA-1416_2015-03-01_17:24:55.patch > > > Multiple unit tests have its own internal function to send/get messages from > the brokers. For example: > sendMessages in ZookeeperConsumerConnectorTest > produceMessage in UncleanLeaderElectionTest > sendMessages in FetcherTest > etc > It is better to unify them in TestUtils. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1993) Enable topic deletion as default
[ https://issues.apache.org/jira/browse/KAFKA-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1993: Attachment: KAFKA-1993.patch This should do the job. > Enable topic deletion as default > > > Key: KAFKA-1993 > URL: https://issues.apache.org/jira/browse/KAFKA-1993 > Project: Kafka > Issue Type: Bug >Reporter: Gwen Shapira > Attachments: KAFKA-1993.patch > > > Since topic deletion is now throughly tested and works as well as most Kafka > features, we should enable it by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1993) Enable topic deletion as default
[ https://issues.apache.org/jira/browse/KAFKA-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1993: Assignee: Gwen Shapira Affects Version/s: 0.8.2.0 Status: Patch Available (was: Open) > Enable topic deletion as default > > > Key: KAFKA-1993 > URL: https://issues.apache.org/jira/browse/KAFKA-1993 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.2.0 >Reporter: Gwen Shapira >Assignee: Gwen Shapira > Attachments: KAFKA-1993.patch > > > Since topic deletion is now throughly tested and works as well as most Kafka > features, we should enable it by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
subscribe the dev channel
...
[jira] [Created] (KAFKA-1994) Evaluate performance effect of chroot check on Topic creation
Ashish K Singh created KAFKA-1994: - Summary: Evaluate performance effect of chroot check on Topic creation Key: KAFKA-1994 URL: https://issues.apache.org/jira/browse/KAFKA-1994 Project: Kafka Issue Type: Improvement Reporter: Ashish K Singh Assignee: Ashish K Singh KAFKA-1664 adds check for chroot while creating a node in ZK. ZkPath checks if namespace exists before trying to create a path in ZK. This raises a concern that checking namespace for each path creation might be unnecessary and can potentially make creations expensive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1664) Kafka does not properly parse multiple ZK nodes with non-root chroot
[
https://issues.apache.org/jira/browse/KAFKA-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342763#comment-14342763
]
Ashish K Singh commented on KAFKA-1664:
---
[~junrao] thanks for bringing up this point. Created KAFKA-1994 to track this.
> Kafka does not properly parse multiple ZK nodes with non-root chroot
>
>
> Key: KAFKA-1664
> URL: https://issues.apache.org/jira/browse/KAFKA-1664
> Project: Kafka
> Issue Type: Bug
> Components: clients
>Reporter: Ricky Saltzer
>Assignee: Ashish K Singh
>Priority: Minor
> Labels: newbie
> Attachments: KAFKA-1664.1.patch, KAFKA-1664.2.patch,
> KAFKA-1664.patch, KAFKA-1664_2015-01-29_10:26:20.patch,
> KAFKA-1664_2015-02-24_11:02:23.patch
>
>
> When using a non-root ZK directory for Kafka, if you specify multiple ZK
> servers, Kafka does not seem to properly parse the connection string.
> *Error*
> {code}
> [root@hodor-001 bin]# ./kafka-console-consumer.sh --zookeeper
> baelish-001.edh.cloudera.com:2181/kafka,baelish-002.edh.cloudera.com:2181/kafka,baelish-003.edh.cloudera.com:2181/kafka
> --topic test-topic
> [2014-10-01 15:31:04,629] ERROR Error processing message, stopping consumer:
> (kafka.consumer.ConsoleConsumer$)
> java.lang.IllegalArgumentException: Path length must be > 0
> at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:48)
> at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:766)
> at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
> at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
> at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:213)
> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
> at kafka.utils.ZkUtils$.createParentPath(ZkUtils.scala:245)
> at kafka.utils.ZkUtils$.createEphemeralPath(ZkUtils.scala:256)
> at
> kafka.utils.ZkUtils$.createEphemeralPathExpectConflict(ZkUtils.scala:268)
> at
> kafka.utils.ZkUtils$.createEphemeralPathExpectConflictHandleZKBug(ZkUtils.scala:306)
> at
> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$registerConsumerInZK(ZookeeperConsumerConnector.scala:226)
> at
> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.(ZookeeperConsumerConnector.scala:755)
> at
> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
> at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:196)
> at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
> {code}
> *Working*
> {code}
> [root@hodor-001 bin]# ./kafka-console-consumer.sh --zookeeper
> baelish-001.edh.cloudera.com:2181/kafka --topic test-topic
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1994) Evaluate performance effect of chroot check on Topic creation
[
https://issues.apache.org/jira/browse/KAFKA-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342769#comment-14342769
]
Ashish K Singh commented on KAFKA-1994:
---
Ran quick test of creating a topic with 1000 partitions with and without
KAFKA-1664's patch. Results are averaged over 100 iterations.
W/o patch (100 iterations):
real2m4.689s
user1m45.213s
sys 0m6.893s
user + sys = 1121.06 ms for each topic creation
W patch (100 iterations):
real2m4.327s
user1m45.497s
sys 0m7.004s
user + sys = 1125.01 ms for each topic creation
Effect due to the patch is ~0.35%. Below is the simple test I ran. Let me know
if you have a better test in mind.
{code}
time for i in `seq 1 100`; do bin/kafka-topics.sh --zookeeper localhost:2181
--create --topic topic$i --partitions 2000 --replication-factor 1; done
{code}
> Evaluate performance effect of chroot check on Topic creation
> -
>
> Key: KAFKA-1994
> URL: https://issues.apache.org/jira/browse/KAFKA-1994
> Project: Kafka
> Issue Type: Improvement
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
>
> KAFKA-1664 adds check for chroot while creating a node in ZK. ZkPath checks
> if namespace exists before trying to create a path in ZK. This raises a
> concern that checking namespace for each path creation might be unnecessary
> and can potentially make creations expensive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1994) Evaluate performance effect of chroot check on Topic creation
[ https://issues.apache.org/jira/browse/KAFKA-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342774#comment-14342774 ] Gwen Shapira commented on KAFKA-1994: - Thanks for checking. This looks good :) I think a 3-node zk cluster not on localhost will be more realistic for production. Can you try that? > Evaluate performance effect of chroot check on Topic creation > - > > Key: KAFKA-1994 > URL: https://issues.apache.org/jira/browse/KAFKA-1994 > Project: Kafka > Issue Type: Improvement >Reporter: Ashish K Singh >Assignee: Ashish K Singh > > KAFKA-1664 adds check for chroot while creating a node in ZK. ZkPath checks > if namespace exists before trying to create a path in ZK. This raises a > concern that checking namespace for each path creation might be unnecessary > and can potentially make creations expensive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1995) JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but hit Kafka)
Rekha Joshi created KAFKA-1995: -- Summary: JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but hit Kafka) Key: KAFKA-1995 URL: https://issues.apache.org/jira/browse/KAFKA-1995 Project: Kafka Issue Type: Wish Components: core Affects Versions: 0.8.3 Reporter: Rekha Joshi Kafka is a great alternative to JMS, providing high performance, throughput as scalable, distributed pub sub/commit log service. However there always exist traditional systems running on JMS. Rather than rewriting, it would be great if we just had an inbuilt JMSAdaptor/JMSProxy/JMSBridge by which client can speak JMS but hit Kafka behind-the-scene. Something like Chukwa's o.a.h.chukwa.datacollection.adaptor.jms.JMSAdaptor, which receives msg off JMS queue and transforms to a Chukwa chunk? I have come across folks talking of this need in past as well.Is it considered and/or part of the roadmap? http://grokbase.com/t/kafka/users/131cst8xpv/stomp-binding-for-kafka http://grokbase.com/t/kafka/users/148dm4247q/consuming-messages-from-kafka-and-pushing-on-to-a-jms-queue http://grokbase.com/t/kafka/users/143hjepbn2/request-kafka-zookeeper-jms-details Looking for inputs on correct way to approach this so to retain all good features of Kafka while still not rewriting entire application.Possible? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1995) JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but hit Kafka)
[ https://issues.apache.org/jira/browse/KAFKA-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342819#comment-14342819 ] Rekha Joshi commented on KAFKA-1995: [~jkreps] [~nehanarkhede] [~junrao] [~joestein] it would be great to know your thoughts on it.thanks. > JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but > hit Kafka) > > > Key: KAFKA-1995 > URL: https://issues.apache.org/jira/browse/KAFKA-1995 > Project: Kafka > Issue Type: Wish > Components: core >Affects Versions: 0.8.3 >Reporter: Rekha Joshi > > Kafka is a great alternative to JMS, providing high performance, throughput > as scalable, distributed pub sub/commit log service. > However there always exist traditional systems running on JMS. > Rather than rewriting, it would be great if we just had an inbuilt > JMSAdaptor/JMSProxy/JMSBridge by which client can speak JMS but hit Kafka > behind-the-scene. > Something like Chukwa's o.a.h.chukwa.datacollection.adaptor.jms.JMSAdaptor, > which receives msg off JMS queue and transforms to a Chukwa chunk? > I have come across folks talking of this need in past as well.Is it > considered and/or part of the roadmap? > http://grokbase.com/t/kafka/users/131cst8xpv/stomp-binding-for-kafka > http://grokbase.com/t/kafka/users/148dm4247q/consuming-messages-from-kafka-and-pushing-on-to-a-jms-queue > http://grokbase.com/t/kafka/users/143hjepbn2/request-kafka-zookeeper-jms-details > Looking for inputs on correct way to approach this so to retain all good > features of Kafka while still not rewriting entire application.Possible? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
