Re: Monitoring Producers at Large Scale

[Darion Yaphet]

Sorry I want to  know  you want to monitor kafka producers or kafka brokers
and zookeepers ?
It's seems you will want to monitor monitor Exceptions eg Leader Not Found,
Queue is full, resend fail  etc  are kafka cluster


2014-06-25 8:20 GMT+08:00 Bhavesh Mistry :

> We use Kafka as Transport Layer to transport application logs.  How do we
> monitor Producers at large scales about 6000 boxes x 4 topic per box so
> roughly 24000 producers (spread across multiple data center.. we have
> brokers per DC).  We do the monitoring based on logs.  I have tried
> intercepting logs via Log4J custom implementation which only intercept WARN
> and ERROR and FATAL events  org.apache.log4j.AppenderSkeleton append method
> which send its logs to brokers (This is working but after load testing it
> is causing deadlock some times between ProducerSendThread and Producer).
>
> I know there are JMX monitoring MBeans available which we can pull the
> data, but I would like to monitor Exceptions eg Leader Not Found, Queue is
> full, resend fail etc in Kafka Library.
>
> How does LinkedIn monitor the Producers ?
>
> Thanks,
>
> Bhavesh
>



-- 


long is the way and hard  that out of Hell leads up to light

Monitoring Producers at Large Scale

[Bhavesh Mistry]

We use Kafka as Transport Layer to transport application logs.  How do we
monitor Producers at large scales about 6000 boxes x 4 topic per box so
roughly 24000 producers (spread across multiple data center.. we have
brokers per DC).  We do the monitoring based on logs.  I have tried
intercepting logs via Log4J custom implementation which only intercept WARN
and ERROR and FATAL events  org.apache.log4j.AppenderSkeleton append method
which send its logs to brokers (This is working but after load testing it
is causing deadlock some times between ProducerSendThread and Producer).

I know there are JMX monitoring MBeans available which we can pull the
data, but I would like to monitor Exceptions eg Leader Not Found, Queue is
full, resend fail etc in Kafka Library.

How does LinkedIn monitor the Producers ?

Thanks,

Bhavesh

Request New Feature On Kafka 0.8.1

[Bhavesh Mistry]

Hi Kafka Dev Teeam,





I would like to request following features in Kafka Async Producers:



1) Ability to Inject the implementation of the Blocking  Queue  similar to
serialize and partition class etc…



I would like to inject following LIB for queue implementation.  Of course
wrapper upon Queue interface..

eg  http://lmax-exchange.github.io/disruptor/



2) Can we have configuration to send the data in Parallel to Brokers using
Pool Executor Service …etc or ability to inject Default Handlers in here
which will run on back ground threads



https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/producer/async/ProducerSendThread.scala



eg:  Our event contains a timestamp so it is ok to have data out of order
into partition due to parallel write to Brokers per partition...


We are looking to dump the messages parallel not build up queue while it
data is being send to brokers and the send thread will take time to
complete entire batch if we have round robin on Partitions ( eg message
count % number Of partitions).

Please let me know if there is any alternative..


Thanks,
Bhavesh

[jira] [Assigned] (KAFKA-1308) Publish jar of test utilities to Maven

[Jakob Homan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan reassigned KAFKA-1308:
--

Assignee: Jakob Homan

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Assignee: Jakob Homan
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Jakob Homan (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042818#comment-14042818
 ] 

Jakob Homan commented on KAFKA-1308:


I've committed this to 0.8.1.  I had already staged the jars, so I just went 
ahead and promoted the release to central as well.

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Joe Stein (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042804#comment-14042804
 ] 

Joe Stein commented on KAFKA-1308:
--

Sounds good, I will wait for the commit to the 0.8.1 branch and then push the 
jar to maven staging , then will post here again for your +1 before shipping to 
central 

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Jakob Homan (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042775#comment-14042775
 ] 

Jakob Homan commented on KAFKA-1308:


Yes, just KAFKA-1308-2.patch (and then manually remove the -SNAPSHOT) 
designation.  This patch has been +1'ed by Neha and I intend to commit it to 
trunk and the 0.8.1.1 branch.  

I've been using this patch to publish snapshots and testing those against 
Samza, so I'm confident the patch itself is good.  

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Jakob Homan (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042776#comment-14042776
 ] 

Jakob Homan commented on KAFKA-1308:


If no one is opposed, I'll go ahead and do it now.

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Neha Narkhede (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042755#comment-14042755
 ] 

Neha Narkhede commented on KAFKA-1308:
--

I'm also +1 on Jakob's suggestion. Joe, thanks for picking it up.

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Joe Stein (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042749#comment-14042749
 ] 

Joe Stein commented on KAFKA-1308:
--

which patch(s) would apply to the 0.8.1.1 tag? 

it looks like I would apply KAFKA-1308-8.1.1.patch and then 
kafka-1308_v2_0.8.1.patch

also I don't see the SNAPSHOT change in either patch it looks like only in 
KAFKA-1308-2.patch

just so we are all saying the same thing we are expecting

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Joe Stein (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042744#comment-14042744
 ] 

Joe Stein commented on KAFKA-1308:
--

I am +1 to Jakob's suggestion and I can do this in a couple of hours if others 
agree (not sure we need a vote I think we are good with just the commit and 
upload release what is there no code change)

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (KAFKA-1508) Scripts Break When Path Has Spaces

[Tim Olson (JIRA)]

Tim Olson created KAFKA-1508:


 Summary: Scripts Break When Path Has Spaces
 Key: KAFKA-1508
 URL: https://issues.apache.org/jira/browse/KAFKA-1508
 Project: Kafka
  Issue Type: Bug
  Components: tools
Affects Versions: 0.8.1.1
 Environment: Any *nix flavor where the full path name to the Kafka 
deployment contains spaces
Reporter: Tim Olson
Priority: Minor


All the shell scripts in {{bin}} use the idom
{{$(dirname $0)}}
but this produces the error
{{usage: dirname path}}
if the path contains spaces.  The correct way to get the dirname is to use:
{{"$(dirname "$0")"}}
and subsequently wrap the result in quotes when it is used.  For example, the 
file {{bin/kafka-run-class.sh}} should look like this starting line 23:
{code}
# BUGFIX: quotes added
base_dir="$(dirname "$0")/.."

# create logs directory
# BUGFIX: quotes added
LOG_DIR="$base_dir/logs"
if [ ! -d "$LOG_DIR" ]; then
# BUGFIX: quotes added around $LOG_DIR
mkdir "$LOG_DIR"
fi
# ...
# BUGFIX: quotes added
for file in "$base_dir/core/build/dependant-libs-${SCALA_VERSION}/*.jar";
do
  CLASSPATH=$CLASSPATH:"$file"
done
{code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Jakob Homan (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042700#comment-14042700
 ] 

Jakob Homan commented on KAFKA-1308:


We're still blocked on the Samza release even with this patch because the 
snapshots I'm publishing are being cleaned on a daily basis.  To get around 
this I would like to publish the 0.8.1.1 release to the main maven site using 
this patch:
* Use the rc1 tgz that was voted in as 0.8.1.1 as the release 
(https://people.apache.org/~joestein/kafka-0.8.1.1-candidate1/kafka-0.8.1.1-src.tgz)
* Apply the patch, remove the -SNAPSHOT
* Run ./gradlew uploadArchivesAll against the main repo, publishing the test 
jar and all of the original jars.

This is ASF-ok since this patch does not change anything about the code that 
was generated, just publishes an extra jar on top of what was previously 
published.  This will unblock us since we can then pull from the main repo.  

The alternative is to wait for Kafka to release a 0.8.1.2 (or whatever), which 
would add an extra week or more for the vote to run.  

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Neha Narkhede (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042455#comment-14042455
 ] 

Neha Narkhede commented on KAFKA-1308:
--

+1 on KAFKA-1308-v2.patch. Thanks Jakob!

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308-2.patch, KAFKA-1308-8.1.1.patch, 
> KAFKA-1308.patch, KAFKA-1308.patch, kafka-1308_v2_0.8.1.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1507) Using GetOffsetShell against non-existent topic creates the topic unintentionally

[Neha Narkhede (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neha Narkhede updated KAFKA-1507:
-

Labels: newbie  (was: )

> Using GetOffsetShell against non-existent topic creates the topic 
> unintentionally
> -
>
> Key: KAFKA-1507
> URL: https://issues.apache.org/jira/browse/KAFKA-1507
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
> Environment: centos
>Reporter: Luke Forehand
>Assignee: Sriharsha Chintalapani
>Priority: Minor
>  Labels: newbie
>
> A typo in using GetOffsetShell command can cause a
> topic to be created which cannot be deleted (because deletion is still in
> progress)
> ./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list
> kafka10:9092,kafka11:9092,kafka12:9092,kafka13:9092 --topic typo --time 1
> ./kafka-topics.sh --zookeeper stormqa1/kafka-prod --describe --topic typo
> Topic:typo  PartitionCount:8ReplicationFactor:1 Configs:
>  Topic: typo Partition: 0Leader: 10  Replicas: 10
>   Isr: 10
> ...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1506) Cancel "kafka-reassign-partitions" Job

[Paul Lung (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042423#comment-14042423
 ] 

Paul Lung commented on KAFKA-1506:
--

I’m not sure. Unfortunately I have already removed everything so I can’t
verify that now.

Paul Lung





> Cancel "kafka-reassign-partitions" Job
> --
>
> Key: KAFKA-1506
> URL: https://issues.apache.org/jira/browse/KAFKA-1506
> Project: Kafka
>  Issue Type: New Feature
>  Components: replication, tools
>Affects Versions: 0.8.1, 0.8.1.1
>Reporter: Paul Lung
>Assignee: Neha Narkhede
>
> I started a reassignment, and for some reason it just takes forever. However, 
> it won¹t let me start another reassignment job while this one is running. So 
> a tool to cancel a reassignment job is needed. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 22874: Fix KAFKA-1498

[Jay Kreps]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22874/#review46543
---



clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java


ReadyNodes?



clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java


nextReadyCheckDelayMs?


- Jay Kreps


On June 23, 2014, 5:54 p.m., Guozhang Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22874/
> ---
> 
> (Updated June 23, 2014, 5:54 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1498
> https://issues.apache.org/jira/browse/KAFKA-1498
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> 1. Use a size limit on the memory records to guard too-large message cases; 
> 2. Caller thread check partition readiness due to batch size upon append, and 
> only wake up sender when the appended partition is ready; 3. Sender thread 
> select time based on the partition readiness timeout and metadata timeout. 4. 
> Mirror maker to use one blocking queue per producer thread. 5. Other minor 
> fixes.
> 
> 
> Diffs
> -
> 
>   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
> 522881c972ca42ff4dfb6237a2db15b625334d7e 
>   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
> 00775abbcac850b0f2bb9a70b6fbc7cdf319bcf6 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
>  57bc285c20b5af8957bcc5322cd75c021a5af215 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java
>  1ed3c28b436d28381d9402896e32d16f2586c65e 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
> 6fb5b82dedb48d946d1ac1ec7a535bddfdc693fa 
>   clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
> 759f577eaf0e7d28a84926d4aa30f4ef0cb27bc2 
>   clients/src/test/java/org/apache/kafka/clients/NetworkClientTest.java 
> 6a3cdcc1f2542479f37bc339baca87464c01e84e 
>   clients/src/test/java/org/apache/kafka/clients/producer/MetadataTest.java 
> 8b4ac0f9a59b4f2e67e48e6d9b0d9fe340f77166 
>   
> clients/src/test/java/org/apache/kafka/clients/producer/RecordAccumulatorTest.java
>  93b58d02eac0f8ca28440e3e0ebea28ed3a7673c 
>   clients/src/test/java/org/apache/kafka/clients/producer/SenderTest.java 
> 5489acac6806b3ae5e6d568d401d5a20c86cac05 
>   core/src/main/scala/kafka/tools/MirrorMaker.scala 
> 763839157d9736f15110072bcae93fc7fdc33f55 
> 
> Diff: https://reviews.apache.org/r/22874/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Guozhang Wang
> 
>

Re: Review Request 22874: Fix KAFKA-1498

[Jay Kreps]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22874/#review46540
---



clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java


Should also be static/final



clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java


nextPartitionExpirationMs?


- Jay Kreps


On June 23, 2014, 5:54 p.m., Guozhang Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22874/
> ---
> 
> (Updated June 23, 2014, 5:54 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1498
> https://issues.apache.org/jira/browse/KAFKA-1498
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> 1. Use a size limit on the memory records to guard too-large message cases; 
> 2. Caller thread check partition readiness due to batch size upon append, and 
> only wake up sender when the appended partition is ready; 3. Sender thread 
> select time based on the partition readiness timeout and metadata timeout. 4. 
> Mirror maker to use one blocking queue per producer thread. 5. Other minor 
> fixes.
> 
> 
> Diffs
> -
> 
>   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
> 522881c972ca42ff4dfb6237a2db15b625334d7e 
>   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
> 00775abbcac850b0f2bb9a70b6fbc7cdf319bcf6 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
>  57bc285c20b5af8957bcc5322cd75c021a5af215 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java
>  1ed3c28b436d28381d9402896e32d16f2586c65e 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
> 6fb5b82dedb48d946d1ac1ec7a535bddfdc693fa 
>   clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
> 759f577eaf0e7d28a84926d4aa30f4ef0cb27bc2 
>   clients/src/test/java/org/apache/kafka/clients/NetworkClientTest.java 
> 6a3cdcc1f2542479f37bc339baca87464c01e84e 
>   clients/src/test/java/org/apache/kafka/clients/producer/MetadataTest.java 
> 8b4ac0f9a59b4f2e67e48e6d9b0d9fe340f77166 
>   
> clients/src/test/java/org/apache/kafka/clients/producer/RecordAccumulatorTest.java
>  93b58d02eac0f8ca28440e3e0ebea28ed3a7673c 
>   clients/src/test/java/org/apache/kafka/clients/producer/SenderTest.java 
> 5489acac6806b3ae5e6d568d401d5a20c86cac05 
>   core/src/main/scala/kafka/tools/MirrorMaker.scala 
> 763839157d9736f15110072bcae93fc7fdc33f55 
> 
> Diff: https://reviews.apache.org/r/22874/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Guozhang Wang
> 
>

Re: Review Request 22874: Fix KAFKA-1498

[Jay Kreps]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22874/#review46536
---



clients/src/main/java/org/apache/kafka/clients/NetworkClient.java


I think it is clear what this code does, so the doc is a little redundant. 
But what isn't explained is the rationale...I assume the idea is that we want 
to wake up if we need to do a metadata update.



clients/src/main/java/org/apache/kafka/clients/NetworkClient.java


This method is pretty odd, and this change makes it worse. Would it be 
better if we moved the time calculation out of this method. 



clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java


Should this be static and final?

Also, can we combine it with FutureRecordMetadata?


- Jay Kreps


On June 23, 2014, 5:54 p.m., Guozhang Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22874/
> ---
> 
> (Updated June 23, 2014, 5:54 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1498
> https://issues.apache.org/jira/browse/KAFKA-1498
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> 1. Use a size limit on the memory records to guard too-large message cases; 
> 2. Caller thread check partition readiness due to batch size upon append, and 
> only wake up sender when the appended partition is ready; 3. Sender thread 
> select time based on the partition readiness timeout and metadata timeout. 4. 
> Mirror maker to use one blocking queue per producer thread. 5. Other minor 
> fixes.
> 
> 
> Diffs
> -
> 
>   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
> 522881c972ca42ff4dfb6237a2db15b625334d7e 
>   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
> 00775abbcac850b0f2bb9a70b6fbc7cdf319bcf6 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
>  57bc285c20b5af8957bcc5322cd75c021a5af215 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java
>  1ed3c28b436d28381d9402896e32d16f2586c65e 
>   
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
> 6fb5b82dedb48d946d1ac1ec7a535bddfdc693fa 
>   clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
> 759f577eaf0e7d28a84926d4aa30f4ef0cb27bc2 
>   clients/src/test/java/org/apache/kafka/clients/NetworkClientTest.java 
> 6a3cdcc1f2542479f37bc339baca87464c01e84e 
>   clients/src/test/java/org/apache/kafka/clients/producer/MetadataTest.java 
> 8b4ac0f9a59b4f2e67e48e6d9b0d9fe340f77166 
>   
> clients/src/test/java/org/apache/kafka/clients/producer/RecordAccumulatorTest.java
>  93b58d02eac0f8ca28440e3e0ebea28ed3a7673c 
>   clients/src/test/java/org/apache/kafka/clients/producer/SenderTest.java 
> 5489acac6806b3ae5e6d568d401d5a20c86cac05 
>   core/src/main/scala/kafka/tools/MirrorMaker.scala 
> 763839157d9736f15110072bcae93fc7fdc33f55 
> 
> Diff: https://reviews.apache.org/r/22874/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Guozhang Wang
> 
>

[jira] [Assigned] (KAFKA-1507) Using GetOffsetShell against non-existent topic creates the topic unintentionally

[Sriharsha Chintalapani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani reassigned KAFKA-1507:
-

Assignee: Sriharsha Chintalapani

> Using GetOffsetShell against non-existent topic creates the topic 
> unintentionally
> -
>
> Key: KAFKA-1507
> URL: https://issues.apache.org/jira/browse/KAFKA-1507
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
> Environment: centos
>Reporter: Luke Forehand
>Assignee: Sriharsha Chintalapani
>Priority: Minor
>
> A typo in using GetOffsetShell command can cause a
> topic to be created which cannot be deleted (because deletion is still in
> progress)
> ./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list
> kafka10:9092,kafka11:9092,kafka12:9092,kafka13:9092 --topic typo --time 1
> ./kafka-topics.sh --zookeeper stormqa1/kafka-prod --describe --topic typo
> Topic:typo  PartitionCount:8ReplicationFactor:1 Configs:
>  Topic: typo Partition: 0Leader: 10  Replicas: 10
>   Isr: 10
> ...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (KAFKA-1507) Using GetOffsetShell against non-existent topic creates the topic unintentionally

[Luke Forehand (JIRA)]

Luke Forehand created KAFKA-1507:


 Summary: Using GetOffsetShell against non-existent topic creates 
the topic unintentionally
 Key: KAFKA-1507
 URL: https://issues.apache.org/jira/browse/KAFKA-1507
 Project: Kafka
  Issue Type: Bug
Affects Versions: 0.8.1.1
 Environment: centos
Reporter: Luke Forehand
Priority: Minor


A typo in using GetOffsetShell command can cause a
topic to be created which cannot be deleted (because deletion is still in
progress)

./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list
kafka10:9092,kafka11:9092,kafka12:9092,kafka13:9092 --topic typo --time 1

./kafka-topics.sh --zookeeper stormqa1/kafka-prod --describe --topic typo
Topic:typo  PartitionCount:8ReplicationFactor:1 Configs:
 Topic: typo Partition: 0Leader: 10  Replicas: 10
  Isr: 10
...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[Dong Lin (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042258#comment-14042258
 ] 

Dong Lin commented on KAFKA-1477:
-

Hey Rajasekar, I have submitted to this JIRA by mistake. Please ignore it
or remove unrelated patch/link. Sorry about it.


On Tue, Jun 24, 2014 at 8:18 AM, Rajasekar Elango (JIRA) 




-- 
Lin, Dong
Dept. of Computer & Information Science
University of Pennsylvania


> add authentication layer and initial JKS x509 implementation for brokers, 
> producers and consumer for network communication
> --
>
> Key: KAFKA-1477
> URL: https://issues.apache.org/jira/browse/KAFKA-1477
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Joe Stein
>Assignee: Ivan Lyutov
> Fix For: 0.8.2
>
> Attachments: KAFKA-1477-binary.patch, KAFKA-1477.patch, 
> KAFKA-1477_2014-06-02_16:59:40.patch, KAFKA-1477_2014-06-02_17:24:26.patch, 
> KAFKA-1477_2014-06-03_13:46:17.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[Dong Lin (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042255#comment-14042255
 ] 

Dong Lin commented on KAFKA-1477:
-

Rajasekar, I have submitted to this JIRA by mistake. Please ignore it or remove 
unrelated patch/link. Sorry about it.

> add authentication layer and initial JKS x509 implementation for brokers, 
> producers and consumer for network communication
> --
>
> Key: KAFKA-1477
> URL: https://issues.apache.org/jira/browse/KAFKA-1477
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Joe Stein
>Assignee: Ivan Lyutov
> Fix For: 0.8.2
>
> Attachments: KAFKA-1477-binary.patch, KAFKA-1477.patch, 
> KAFKA-1477.patch, KAFKA-1477.patch, KAFKA-1477_2014-06-02_16:59:40.patch, 
> KAFKA-1477_2014-06-02_17:24:26.patch, KAFKA-1477_2014-06-03_13:46:17.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[Dong Lin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dong Lin updated KAFKA-1477:


Attachment: (was: KAFKA-1477.patch)

> add authentication layer and initial JKS x509 implementation for brokers, 
> producers and consumer for network communication
> --
>
> Key: KAFKA-1477
> URL: https://issues.apache.org/jira/browse/KAFKA-1477
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Joe Stein
>Assignee: Ivan Lyutov
> Fix For: 0.8.2
>
> Attachments: KAFKA-1477-binary.patch, KAFKA-1477.patch, 
> KAFKA-1477_2014-06-02_16:59:40.patch, KAFKA-1477_2014-06-02_17:24:26.patch, 
> KAFKA-1477_2014-06-03_13:46:17.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[Dong Lin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dong Lin updated KAFKA-1477:


Attachment: (was: KAFKA-1477.patch)

> add authentication layer and initial JKS x509 implementation for brokers, 
> producers and consumer for network communication
> --
>
> Key: KAFKA-1477
> URL: https://issues.apache.org/jira/browse/KAFKA-1477
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Joe Stein
>Assignee: Ivan Lyutov
> Fix For: 0.8.2
>
> Attachments: KAFKA-1477-binary.patch, KAFKA-1477.patch, 
> KAFKA-1477_2014-06-02_16:59:40.patch, KAFKA-1477_2014-06-02_17:24:26.patch, 
> KAFKA-1477_2014-06-03_13:46:17.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Issue Comment Deleted] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[Dong Lin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dong Lin updated KAFKA-1477:


Comment: was deleted

(was: Created reviewboard  against branch transactional_messaging)

> add authentication layer and initial JKS x509 implementation for brokers, 
> producers and consumer for network communication
> --
>
> Key: KAFKA-1477
> URL: https://issues.apache.org/jira/browse/KAFKA-1477
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Joe Stein
>Assignee: Ivan Lyutov
> Fix For: 0.8.2
>
> Attachments: KAFKA-1477-binary.patch, KAFKA-1477.patch, 
> KAFKA-1477_2014-06-02_16:59:40.patch, KAFKA-1477_2014-06-02_17:24:26.patch, 
> KAFKA-1477_2014-06-03_13:46:17.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Issue Comment Deleted] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[Dong Lin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dong Lin updated KAFKA-1477:


Comment: was deleted

(was: Created reviewboard https://reviews.apache.org/r/22905/diff/
 against branch origin/transactional_messaging)

> add authentication layer and initial JKS x509 implementation for brokers, 
> producers and consumer for network communication
> --
>
> Key: KAFKA-1477
> URL: https://issues.apache.org/jira/browse/KAFKA-1477
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Joe Stein
>Assignee: Ivan Lyutov
> Fix For: 0.8.2
>
> Attachments: KAFKA-1477-binary.patch, KAFKA-1477.patch, 
> KAFKA-1477_2014-06-02_16:59:40.patch, KAFKA-1477_2014-06-02_17:24:26.patch, 
> KAFKA-1477_2014-06-03_13:46:17.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1504) Replica is unable to update ISR

[Guozhang Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042245#comment-14042245
 ] 

Guozhang Wang commented on KAFKA-1504:
--

This may be due to KAFKA-1382.

You can verify by checking if there is a ZK session timeout in the log.

> Replica is unable to update ISR
> ---
>
> Key: KAFKA-1504
> URL: https://issues.apache.org/jira/browse/KAFKA-1504
> Project: Kafka
>  Issue Type: Bug
>  Components: replication
>Affects Versions: 0.8.1
> Environment: Windows Server 2012; JDK 1.7.0
>Reporter: Jie Tong
>Assignee: Neha Narkhede
>
> We ran into this issue relatively frequent: replica cannot update ISR and 
> thus fall out of the ISR and result in the # of replicas in ISR < replication 
> factor. A restart of broker can mitigate the issue.
> Below is the error log:
> e,06/22/2014 
> 20:29:20,Logging,kafka.utils.Logging$class,SrcFile="Logging.scala" 
> SrcFunc="error" SrcLine="97" Pid="1952" Tid="8852" TS="0x01CF8E935200FDAF" 
> String1="Conditional update of path 
> /brokers/topics/RTUserCount/partitions/60/state with data 
> {'controller_epoch':44,'leader':3690877,'version':1,'leader_epoch':110,'isr':[3690877]}
>  and expected version 230 failed due to 
> org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = 
> BadVersion for /brokers/topics/RTUserCount/partitions/60/state"
> i,06/22/2014 
> 20:29:20,Logging,kafka.utils.Logging$class,SrcFile="Logging.scala" 
> SrcFunc="info" SrcLine="68" Pid="1952" Tid="8852" TS="0x01CF8E93520124BC" 
> String1="Partition [RTUserCount,60] on broker 3690877: Cached zkVersion [230] 
> not equal to that in zookeeper, skip updating ISR"



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1506) Cancel "kafka-reassign-partitions" Job

[Guozhang Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042249#comment-14042249
 ] 

Guozhang Wang commented on KAFKA-1506:
--

Any exceptions/errors you saw in the controller log?

> Cancel "kafka-reassign-partitions" Job
> --
>
> Key: KAFKA-1506
> URL: https://issues.apache.org/jira/browse/KAFKA-1506
> Project: Kafka
>  Issue Type: New Feature
>  Components: replication, tools
>Affects Versions: 0.8.1, 0.8.1.1
>Reporter: Paul Lung
>Assignee: Neha Narkhede
>
> I started a reassignment, and for some reason it just takes forever. However, 
> it won¹t let me start another reassignment job while this one is running. So 
> a tool to cancel a reassignment job is needed. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[Rajasekar Elango (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042234#comment-14042234
 ] 

Rajasekar Elango commented on KAFKA-1477:
-

[~lindong] It doesn't look like review https://reviews.apache.org/r/22905/diff/ 
commented on this jira is not related to this functionality. Can you double 
check?

> add authentication layer and initial JKS x509 implementation for brokers, 
> producers and consumer for network communication
> --
>
> Key: KAFKA-1477
> URL: https://issues.apache.org/jira/browse/KAFKA-1477
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Joe Stein
>Assignee: Ivan Lyutov
> Fix For: 0.8.2
>
> Attachments: KAFKA-1477-binary.patch, KAFKA-1477.patch, 
> KAFKA-1477.patch, KAFKA-1477.patch, KAFKA-1477_2014-06-02_16:59:40.patch, 
> KAFKA-1477_2014-06-02_17:24:26.patch, KAFKA-1477_2014-06-03_13:46:17.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Announcing Kafka Web Console v2.0.0

[Joe Stein]

Awesome Claude, thanks!

/***
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop 
/


On Tue, Jun 24, 2014 at 10:57 AM, Claude Mamo  wrote:

> Announcing the second major release of Kafka Web Console:
> https://github.com/claudemamo/kafka-web-console/releases/tag/v2.0.0.
> Highlights:
>
> - I've borrowed some ideas from Kafka Offset Monitor and added graphs to
> show the history of consumers offsets and lag as well as message throughput
>
> - Added partition leaders, log size, offset and lag
>
> - Bug fixes!
>
> Claude
>

Announcing Kafka Web Console v2.0.0

[Claude Mamo]

Announcing the second major release of Kafka Web Console:
https://github.com/claudemamo/kafka-web-console/releases/tag/v2.0.0.
Highlights:

- I've borrowed some ideas from Kafka Offset Monitor and added graphs to
show the history of consumers offsets and lag as well as message throughput

- Added partition leaders, log size, offset and lag

- Bug fixes!

Claude