Build failed in Jenkins: kafka-trunk-jdk7 #2040

See Changes: [me] HOTFIX: Fix unsafe dependence on class name in VerifiableClientJava -- [...truncated 331.23 KB...] kafka.log.LogTest > testCorruptIndexRebuild STARTED

[GitHub] kafka pull request #2736: HOTFIX: Fix unsafe dependence on class name in Ver...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2736 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2736: HOTFIX: Fix unsafe dependence on class name in Ver...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2736 HOTFIX: Fix unsafe dependence on class name in VerifiableClientJava You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka

Re: [VOTE] KIP-117: Add a public AdminClient API for Kafka admin operations

With binding +1 votes from Gwen Shapira, Sriram Subramanian, and Grant Henke, and a non-binding vote from Dong Lin, the vote passes. There were no +0 or -1 votes. As mentioned earlier, the interface will be unstable at first and we will continue to evolve it. thanks, Colin McCabe On Wed, Mar

Build failed in Jenkins: kafka-trunk-jdk7 #2039

See Changes: [ismael] KAFKA-4816; Message format changes for idempotent/transactional producer -- [...truncated 896.24 KB...]

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

Hey Jun, No.. the current approach describe in the KIP (see here ) also

Re: [DISCUSS] KIP-134: Delay initial consumer group rebalance

Hi Matthias, Yes, that was what I was thinking. We will keep delay it until either reaching the rebalance timeout or no new consumer joins in that small delay which is configured on the broker side. Thanks, Jiangjie (Becket) Qin On Fri, Mar 24, 2017 at 1:39 PM, Matthias J. Sax

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

Hi, Dong, We are just comparing whether it's better for the reassignment tool to send ChangeReplicaDirRequest (1) before or (2) after writing the reassignment path in ZK . In the case when all brokers are alive when the reassignment tool is run, (1) guarantees 100% that the new replicas will be

[jira] [Commented] (KAFKA-4900) Brokers stuck in controller re-election loop after failing to register metrics

[ https://issues.apache.org/jira/browse/KAFKA-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941296#comment-15941296 ] Onur Karaman commented on KAFKA-4900: - I haven't figured out some of the small details but I think I

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi, Mayuresh, Please find my replies inlined below. On Wed, Mar 22, 2017 at 8:07 PM, Mayuresh Gharat wrote: > Hi Jun, > > Please find the replies inline. > > One reason to have KafkaPrincipal in ACL is that we can extend it to > support group in the future. Have

[jira] [Commented] (KAFKA-4817) Implement idempotent producer

[ https://issues.apache.org/jira/browse/KAFKA-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941193#comment-15941193 ] Apurva Mehta commented on KAFKA-4817: - There is some subtlety in the idempotent producer semantics.

[jira] [Commented] (KAFKA-4815) Idempotent/transactional Producer Checklist (KIP-98)

[ https://issues.apache.org/jira/browse/KAFKA-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941166#comment-15941166 ] ASF GitHub Bot commented on KAFKA-4815: --- GitHub user apurvam opened a pull request:

[GitHub] kafka pull request #2735: KAFKA-4815 : Add idempotent producer semantics

GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/2735 KAFKA-4815 : Add idempotent producer semantics This is from the KIP-98 proposal. The main points of discussion surround the correctness logic, particularly the Log class where incoming

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

on logging - i think the best we can do is call principal.toString() and log that. the built-in one prints "type/name" but in some contexts that might not always make sense (using the certificate addr might make more sense if my security is cert-based) on string representation - again this is the

Re: [DISCUSS] KIP-134: Delay initial consumer group rebalance

@Becket: I am not sure, if I understand this correctly. Instead of applying a fixed delay, that starts when the first consumer of an (empty) group joins, you suggest to re-trigger/re-set the delay each time a new consumer joins? This sound like a good strategy to me, if the config is on the

Re: [DISCUSS] KIP-134: Delay initial consumer group rebalance

Thanks Damian, This KIP deals with the initial phase only. What about the cases when several consumers leave a group? Won't there be several expensive rebalances then as well? I'm wondering if it makes sense for the delay to hold anytime the "set" of consumers in a group changes, be it

Re: [DISCUSS] KIP-134: Delay initial consumer group rebalance

Thanks for the KIP, Damian. My two cents on this. It seems there are two things worth thinking here: 1. Better rebalance timing. We will try to rebalance only when all the consumers in a group have joined. The challenge would be someone has to define what does ALL consumers mean, it could either

[jira] [Resolved] (KAFKA-4816) Message format changes for idempotent/transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4816. Resolution: Fixed > Message format changes for idempotent/transactional producer >

[jira] [Commented] (KAFKA-4816) Message format changes for idempotent/transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941022#comment-15941022 ] ASF GitHub Bot commented on KAFKA-4816: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2614: KAFKA-4816: Message format changes for idempotent/...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2614 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4948) Failure in kafka.admin.DescribeConsumerGroupTest.testDescribeExistingGroupWithNoMembersWithNewConsumer

[ https://issues.apache.org/jira/browse/KAFKA-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940968#comment-15940968 ] ASF GitHub Bot commented on KAFKA-4948: --- GitHub user benstopford opened a pull request:

[GitHub] kafka pull request #2734: KAFKA-4948: Failure in kafka.admin.DescribeConsume...

GitHub user benstopford opened a pull request: https://github.com/apache/kafka/pull/2734 KAFKA-4948: Failure in kafka.admin.DescribeConsumerGroupTest... This test fails regularly when run on OSX (on my machine anyway). The fix decreases the number of partitions in the consumer

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

Michael, I don't think we "hand waved" this question, but as you mentioned anyway, without an implementation it's hard (or even impossible) to quantify the probability. Nevertheless, I'll try: First, I want to refer to the "Error Handling" section in the KIP:

Problem using connect to ingest log rotation

Hello, i'm trying to connect a log file in a rotation basis, all works as espected, but when the log file os deleted during the rotation, kafka stop listening the new file, althought it is called the same. Is there any workaround to manage this situation? Thanks in advance. Enviado desde mi

[jira] [Commented] (KAFKA-4943) SCRAM secret's should be better protected with Zookeeper ACLs

[ https://issues.apache.org/jira/browse/KAFKA-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940828#comment-15940828 ] ASF GitHub Bot commented on KAFKA-4943: --- GitHub user rajinisivaram opened a pull request:

[GitHub] kafka pull request #2733: KAFKA-4943: Make /config/users with SCRAM credenti...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/2733 KAFKA-4943: Make /config/users with SCRAM credentials not world-readable You can merge this pull request into a Git repository by running: $ git pull

[jira] [Created] (KAFKA-4948) Failure in kafka.admin.DescribeConsumerGroupTest.testDescribeExistingGroupWithNoMembersWithNewConsumer

Guozhang Wang created KAFKA-4948: Summary: Failure in kafka.admin.DescribeConsumerGroupTest.testDescribeExistingGroupWithNoMembersWithNewConsumer Key: KAFKA-4948 URL:

Re: [VOTE] KIP-82 Add Record Headers

Thanks Ismael for spotting from the discuss thread. Extra details documented on wiki. Cheers Mike From: isma...@gmail.com on behalf of Ismael Juma Sent: Friday, March 24, 2017 9:36:34 AM To: dev@kafka.apache.org

[jira] [Updated] (KAFKA-3992) InstanceAlreadyExistsException Error for Consumers Starting in Parallel

[ https://issues.apache.org/jira/browse/KAFKA-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Cook updated KAFKA-3992: -- Affects Version/s: 0.10.0.0 > InstanceAlreadyExistsException Error for Consumers Starting in

[jira] [Reopened] (KAFKA-3992) InstanceAlreadyExistsException Error for Consumers Starting in Parallel

[ https://issues.apache.org/jira/browse/KAFKA-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Cook reopened KAFKA-3992: --- I have moved to 0.10 of the Kafka Consumer Client and I am still seeing this issue. I believe

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

Thanks for the comments Matthias. I updated the KIP and PR. Cheers, Jeyhun On Fri, Mar 24, 2017 at 8:34 AM Eno Thereska wrote: > +1 (non-binding) > > Thanks > Eno > > > On 24 Mar 2017, at 03:37, Matthias J. Sax wrote: > > > > Thanks Jeyhun. > > >

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

Ah, I think the use of the word "regression" was misleading. Let me try to rephrase: At the moment it is unclear to me whether the scenario where we need to rebuild a state store from scratch will be as likely, a little bit more likely, much more likely, or very, very much more likely to happen

[jira] [Created] (KAFKA-4947) Include producing principal and host in messages

Fred Ross created KAFKA-4947: Summary: Include producing principal and host in messages Key: KAFKA-4947 URL: https://issues.apache.org/jira/browse/KAFKA-4947 Project: Kafka Issue Type: New

[jira] [Assigned] (KAFKA-4938) Creating a connector with missing name parameter throws a NullPointerException

[ https://issues.apache.org/jira/browse/KAFKA-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balint Molnar reassigned KAFKA-4938: Assignee: Balint Molnar > Creating a connector with missing name parameter throws a

[jira] [Work started] (KAFKA-4938) Creating a connector with missing name parameter throws a NullPointerException

[ https://issues.apache.org/jira/browse/KAFKA-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4938 started by Balint Molnar. > Creating a connector with missing name parameter throws a NullPointerException

[jira] [Comment Edited] (KAFKA-4943) SCRAM secret's should be better protected with Zookeeper ACLs

[ https://issues.apache.org/jira/browse/KAFKA-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940281#comment-15940281 ] Johan Ström edited comment on KAFKA-4943 at 3/24/17 1:09 PM: - Great, thanks!

Re: [VOTE] KIP-124: Request rate quotas

This vote has passed with four binding (Jun, Ismael, Becket, Guozhang) and two non-binding (Roger, me) +1s. Thank you all for the feedback and votes. I will update the KIP page and submit a PR. Regards, Rajini On Tue, Mar 21, 2017 at 5:13 PM, Guozhang Wang wrote: > +1.

[jira] [Commented] (KAFKA-4943) SCRAM secret's should be better protected with Zookeeper ACLs

[ https://issues.apache.org/jira/browse/KAFKA-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940281#comment-15940281 ] Johan Ström commented on KAFKA-4943: Great! > SCRAM secret's should be better protected with

[jira] [Commented] (KAFKA-4943) SCRAM secret's should be better protected with Zookeeper ACLs

[ https://issues.apache.org/jira/browse/KAFKA-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940275#comment-15940275 ] Rajini Sivaram commented on KAFKA-4943: --- [~jstrom] [~stephane.maa...@gmail.com] Will submit a PR to

[jira] [Assigned] (KAFKA-4943) SCRAM secret's should be better protected with Zookeeper ACLs

[ https://issues.apache.org/jira/browse/KAFKA-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram reassigned KAFKA-4943: - Assignee: Rajini Sivaram > SCRAM secret's should be better protected with Zookeeper ACLs

[jira] [Commented] (KAFKA-4855) Struct SchemaBuilder should not allow duplicate fields.

[ https://issues.apache.org/jira/browse/KAFKA-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940263#comment-15940263 ] ASF GitHub Bot commented on KAFKA-4855: --- GitHub user baluchicken opened a pull request:

[GitHub] kafka pull request #2732: KAFKA-4855 Struct SchemaBuilder should not allow d...

GitHub user baluchicken opened a pull request: https://github.com/apache/kafka/pull/2732 KAFKA-4855 Struct SchemaBuilder should not allow duplicate fields @ewencp can you please review. You can merge this pull request into a Git repository by running: $ git pull

[jira] [Assigned] (KAFKA-4855) Struct SchemaBuilder should not allow duplicate fields.

[ https://issues.apache.org/jira/browse/KAFKA-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balint Molnar reassigned KAFKA-4855: Assignee: Balint Molnar > Struct SchemaBuilder should not allow duplicate fields. >

[jira] [Work started] (KAFKA-4855) Struct SchemaBuilder should not allow duplicate fields.

[ https://issues.apache.org/jira/browse/KAFKA-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4855 started by Balint Molnar. > Struct SchemaBuilder should not allow duplicate fields. >

Re: [DISCUSS] KIP-134: Delay initial consumer group rebalance

Thanks for the feedback. Ewen: I'm happy to make it a client side config. Other than the protocol bump i think the effort is almost the same. Personally i see no other issues, but based on discussions with others this is what we came up with. True, it can probably be tested easily via an

[GitHub] kafka pull request #2731: MINOR: Make LeaderAndIsr immutable case class.

GitHub user jozi-k opened a pull request: https://github.com/apache/kafka/pull/2731 MINOR: Make LeaderAndIsr immutable case class. Continuation of #2531. Asking for review @guozhangwang and @ijuma You can merge this pull request into a Git repository by running: $ git pull

Re: [VOTE] KIP-82 Add Record Headers

Hi Michael, Can you please update the wiki page to cover the following 2 points that were discussed in the mailing list thread, but should be there too: 1. How to create an instance of a header to pass to the `add` method. You suggested doing it via the implementation class. If we do it that

[jira] [Created] (KAFKA-4946) Evaluation of kafka_mx4jenable is inverted

Ralph Weires created KAFKA-4946: --- Summary: Evaluation of kafka_mx4jenable is inverted Key: KAFKA-4946 URL: https://issues.apache.org/jira/browse/KAFKA-4946 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-82 Add Record Headers

Hi All, I would like to thank everyone for their votes, and all the feedback and comments. With +1 (non-binding) from myself. We have collected the following votes so far: Binding +1: 3 Jun Rao Jason Gustafson Joel Koshy Binding -1: 0 Non-Binding +1: 5 Radai Rosenblatt Onur Karaman

Re: [VOTE] KIP-82 Add Record Headers

Hi Jun, Thanks for your vote, I’ve updated the wiki to document this detail. Cheers Mike On 24/03/2017, 05:00, "Jun Rao" wrote: Hi, Ismael, Ok, that make sense. Hi, Michael, Could we document this in the wiki? +1 from me on the KIP. Thanks,

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

+1 (non-binding) Thanks Eno > On 24 Mar 2017, at 03:37, Matthias J. Sax wrote: > > Thanks Jeyhun. > > Can you also update the KIP accordingly. It must contain all changes to > public API. Thus, list all parameters that get deprecated and newly > added. And add a

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

Hey Jun, Thanks much for the response! I agree with you that if multiple replicas are created in the wrong directory, we may waste resource if either replicaMoveThread number is low or intra.broker.throttled.rate is slow. Then the question is whether the suggested approach increases the chance of

[jira] [Commented] (KAFKA-4943) SCRAM secret's should be better protected with Zookeeper ACLs

[ https://issues.apache.org/jira/browse/KAFKA-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939916#comment-15939916 ] Stephane Maarek commented on KAFKA-4943: [~rsivaram] if Zookeeper has only cdrwa by the kafka

[jira] [Commented] (KAFKA-4943) SCRAM secret's should be better protected with Zookeeper ACLs

[ https://issues.apache.org/jira/browse/KAFKA-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939888#comment-15939888 ] Johan Ström commented on KAFKA-4943: I agree, it won't be totally secure unless Zk is totally secure,