[jira] [Updated] (KAFKA-5145) Remove task close() call from closeNonAssignedSuspendedTasks method

2017-05-01 Thread Narendra Kumar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narendra Kumar updated KAFKA-5145: -- Reviewer: Guozhang Wang > Remove task close() call from closeNonAssignedSuspendedTasks method >

[jira] [Created] (KAFKA-5152) Kafka Streams keeps restoring state after shutdown is initiated during startup

2017-05-01 Thread JIRA
Xavier Léauté created KAFKA-5152: Summary: Kafka Streams keeps restoring state after shutdown is initiated during startup Key: KAFKA-5152 URL: https://issues.apache.org/jira/browse/KAFKA-5152

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

2017-05-01 Thread Konstantine Karantasis
Gwen, Randall thank you for your very insightful observations. I'm glad you find this first draft to be an adequate platform for discussion. I'll attempt replying to your comments in order. Gwen, I also debated exactly the same two options: a) interpreting absence of module path as a user's

[jira] [Commented] (KAFKA-4772) Exploit #peek to implement #print() and other methods

2017-05-01 Thread james chien (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992298#comment-15992298 ] james chien commented on KAFKA-4772: okay, times to resolve this :) :) > Exploit #peek to implement

[jira] [Commented] (KAFKA-4772) Exploit #peek to implement #print() and other methods

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992286#comment-15992286 ] Matthias J. Sax commented on KAFKA-4772: Please open a new PR. :) > Exploit #peek to implement

Build failed in Jenkins: kafka-trunk-jdk8 #1468

2017-05-01 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-5059 Follow-up: Remove broken locking and fix handleAddPartitions [ismael] MINOR: Some cleanups in the transactional producer [wangguoz] KAFKA-5136: move coordinatorEpoch from

Re: [DISCUSS] KIP 141 - ProducerRecordBuilder Interface

2017-05-01 Thread Jay Kreps
Hey Matthias, Yeah, I think we mostly agree. I think if we think the new way is a lot better, then I agree keeping it deprecated for a long while then removing is a viable path. To me this case just seemed like a minor thing so the 1+ years of having a deprecated way in common use and a new way

[jira] [Assigned] (KAFKA-5055) Kafka Streams skipped-records-rate sensor producing nonzero values even when FailOnInvalidTimestamp is used as extractor

2017-05-01 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-5055: Assignee: Davor Poldrugo (was: Guozhang Wang) > Kafka Streams skipped-records-rate sensor

[jira] [Commented] (KAFKA-5055) Kafka Streams skipped-records-rate sensor producing nonzero values even when FailOnInvalidTimestamp is used as extractor

2017-05-01 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991978#comment-15991978 ] Guozhang Wang commented on KAFKA-5055: -- [~dpoldrugo] Thanks! Added. > Kafka Streams

[jira] [Comment Edited] (KAFKA-4772) Exploit #peek to implement #print() and other methods

2017-05-01 Thread james chien (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991968#comment-15991968 ] james chien edited comment on KAFKA-4772 at 5/2/17 1:54 AM: Great !! I am a

Jenkins build is back to normal : kafka-trunk-jdk7 #2134

2017-05-01 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-4772) Exploit #peek to implement #print() and other methods

2017-05-01 Thread james chien (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991968#comment-15991968 ] james chien commented on KAFKA-4772: Great !! what should I do now? (keep going this comment

Throttle replicas based on client-id

2017-05-01 Thread Archie
Hi All, I am pretty new to kafka development and I am trying to limit the amount of traffic that a client can send/produce to the kafka cluster. Using quotas on client-id, I was able to limit the amount of traffic that can be produced by a client to a value very close to

Re: [DISCUSS] KIP-143: Controller Health Metrics

2017-05-01 Thread Jun Rao
Hi, Joel, 1. Yes, the long term goal is to migrate the metrics on the broker to kafka-metrics. Since many people are using Yammer reporters, we probably need to support a few popular ones in kafka-metrics before migrating. Until that happens, we probably want to stick with the Yammer metrics on

Build failed in Jenkins: kafka-trunk-jdk8 #1467

2017-05-01 Thread Apache Jenkins Server
See Changes: [ismael] KAFKA-3265; Add a public AdminClient API in Java (KIP-117) -- [...truncated 1.65 MB...] kafka.integration.SaslPlaintextTopicMetadataTest >

[jira] [Commented] (KAFKA-4667) Connect should create internal topics

2017-05-01 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991939#comment-15991939 ] Colin P. McCabe commented on KAFKA-4667: The Admin client has been merged :) KIP-133, which is

[jira] [Created] (KAFKA-5151) Refactor TransactionCoordinator in-memory structure and error handling logic

2017-05-01 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-5151: Summary: Refactor TransactionCoordinator in-memory structure and error handling logic Key: KAFKA-5151 URL: https://issues.apache.org/jira/browse/KAFKA-5151 Project:

[jira] [Commented] (KAFKA-5136) Move coordinatorEpoch from WriteTxnMarkerRequest to TxnMarkerEntry

2017-05-01 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991873#comment-15991873 ] ASF GitHub Bot commented on KAFKA-5136: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2925: KAFKA-5136: move coordinatorEpoch from WriteTxnMar...

2017-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2925 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2933: MINOR: Some cleanup in the transactional producer

2017-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2933 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: kafka-trunk-jdk7 #2133

2017-05-01 Thread Apache Jenkins Server
See Changes: [ismael] KAFKA-3265; Add a public AdminClient API in Java (KIP-117) -- [...truncated 846.82 KB...] kafka.server.DynamicConfigChangeTest >

[jira] [Updated] (KAFKA-5150) LZ4 decompression is 4-5x slower than Snappy on small batches / messages

2017-05-01 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté updated KAFKA-5150: - Affects Version/s: (was: 0.10.2.0) 0.10.2.1

[jira] [Commented] (KAFKA-5059) Implement Transactional Coordinator

2017-05-01 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991810#comment-15991810 ] ASF GitHub Bot commented on KAFKA-5059: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2934: KAFKA-5059: [Follow Up] remove broken locking. Fix...

2017-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2934 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-5150) LZ4 decompression is 4-5x slower than Snappy on small batches / messages

2017-05-01 Thread JIRA
Xavier Léauté created KAFKA-5150: Summary: LZ4 decompression is 4-5x slower than Snappy on small batches / messages Key: KAFKA-5150 URL: https://issues.apache.org/jira/browse/KAFKA-5150 Project:

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-05-01 Thread Jeyhun Karimov
Thanks for comments. The concerns makes sense. Although we can guard for immutable keys in current implementation (with few changes), I didn't consider backward compatibility. In this case 2 solutions come to my mind. In both cases, user accesses the key in Object type, as passing extra type

[jira] [Updated] (KAFKA-5126) Implement KIP-98 transactional methods in the MockProducer

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5126: --- Status: Patch Available (was: In Progress) > Implement KIP-98 transactional methods in the

[jira] [Commented] (KAFKA-5126) Implement KIP-98 transactional methods in the MockProducer

2017-05-01 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991779#comment-15991779 ] ASF GitHub Bot commented on KAFKA-5126: --- GitHub user mjsax opened a pull request:

[GitHub] kafka pull request #2951: KAFKA-5126: Implement KIP-98 transactional methods...

2017-05-01 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2951 KAFKA-5126: Implement KIP-98 transactional methods in the MockProducer You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka

[jira] [Resolved] (KAFKA-3265) Create Java Admin Client

2017-05-01 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3265. Resolution: Fixed > Create Java Admin Client > > > Key:

[GitHub] kafka pull request #2472: KAFKA-3265: Create Java Admin Client

2017-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2472 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3265) Create Java Admin Client

2017-05-01 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991774#comment-15991774 ] ASF GitHub Bot commented on KAFKA-3265: --- Github user asfgit closed the pull request at:

Jenkins build is back to normal : kafka-trunk-jdk8 #1466

2017-05-01 Thread Apache Jenkins Server
See

[jira] [Comment Edited] (KAFKA-4667) Connect should create internal topics

2017-05-01 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991697#comment-15991697 ] Randall Hauch edited comment on KAFKA-4667 at 5/1/17 10:47 PM: --- The

[jira] [Commented] (KAFKA-4667) Connect should create internal topics

2017-05-01 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991697#comment-15991697 ] Randall Hauch commented on KAFKA-4667: -- The AdminClient pull request is being added as part of [PR

Jenkins build is back to normal : kafka-trunk-jdk7 #2132

2017-05-01 Thread Apache Jenkins Server
See

[jira] [Updated] (KAFKA-5148) Add configurable compression block size to the broker

2017-05-01 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5148: --- Description: Similar to the discussion in KAFKA-3704, we should consider a configurable compression

[jira] [Commented] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

2017-05-01 Thread Shannon Carey (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991488#comment-15991488 ] Shannon Carey commented on KAFKA-4669: -- We encountered this exception with our 0.9.0.1 client too,

[DISCUSS] KIP-150 - Kafka-Streams Cogroup

2017-05-01 Thread Kyle Winkelman
Hello all, I have created KIP-150 to facilitate discussion about adding cogroup to the streams DSL. Please find the KIP here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-150+-+Kafka-Streams+Cogroup Please find my initial implementation here: https://github.com/KyleWinkelman/kafka

Build failed in Jenkins: kafka-trunk-jdk7 #2131

2017-05-01 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-4954; Request handler utilization quotas -- [...truncated 822.64 KB...] java.lang.OutOfMemoryError: unable to create new native thread

[jira] [Commented] (KAFKA-3877) Gradle compiler daemon exits with non-zero exit code while running LogOffsetTest

2017-05-01 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991444#comment-15991444 ] Colin P. McCabe commented on KAFKA-3877: As [~original-brownbear] commented, this is almost

[jira] [Commented] (KAFKA-5112) Trunk compatibility tests should test against 0.10.2

2017-05-01 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991418#comment-15991418 ] Colin P. McCabe commented on KAFKA-5112: This seems to be a duplicate of

[jira] [Assigned] (KAFKA-5112) Trunk compatibility tests should test against 0.10.2

2017-05-01 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe reassigned KAFKA-5112: -- Assignee: Ismael Juma (was: Colin P. McCabe) > Trunk compatibility tests should test

[jira] [Resolved] (KAFKA-4899) Fix findbugs warnings in kafka-core

2017-05-01 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe resolved KAFKA-4899. Resolution: Fixed Fix Version/s: 0.11.0.0 > Fix findbugs warnings in kafka-core >

[jira] [Commented] (KAFKA-5091) ReassignPartitionsCommand should protect against empty replica list assignment

2017-05-01 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991354#comment-15991354 ] ASF GitHub Bot commented on KAFKA-5091: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2904: KAFKA-5091: ReassignPartitionsCommand should...

2017-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2904 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5091) ReassignPartitionsCommand should protect against empty replica list assignment

2017-05-01 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-5091. Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 2904

[jira] [Commented] (KAFKA-5149) New producer hardcodes key and value serializers to ByteArray

2017-05-01 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991317#comment-15991317 ] Jun Rao commented on KAFKA-5149: This is similar to what's reported in

Re: [DISCUSS] KIP 141 - ProducerRecordBuilder Interface

2017-05-01 Thread Michael André Pearce
If it's a choice of either or. I would vote keep as is. At least then people can write their own api wrappers easily with not many lines of code, like the one supplied. Sent from my iPhone > On 1 May 2017, at 18:34, Matthias J. Sax wrote: > > Hi, > > I am personally

Re: [DISCUSS] KIP 145 - Expose Record Headers in Kafka Connect

2017-05-01 Thread Michael Pearce
Hi Gwen, Then intent here was to allow tools that perform similar role to mirror makers of replicating the messaging from one cluster to another. Eg like mirror make should just be taking and transferring the headers as is. We don't actually use this inside our company, so not exposing this

[jira] [Work started] (KAFKA-5149) New producer hardcodes key and value serializers to ByteArray

2017-05-01 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-5149 started by Bharat Viswanadham. - > New producer hardcodes key and value serializers to ByteArray >

[jira] [Assigned] (KAFKA-5149) New producer hardcodes key and value serializers to ByteArray

2017-05-01 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham reassigned KAFKA-5149: - Assignee: Bharat Viswanadham > New producer hardcodes key and value serializers

Build failed in Jenkins: kafka-trunk-jdk8 #1465

2017-05-01 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-4954; Request handler utilization quotas -- [...truncated 1.64 MB...] kafka.integration.SaslPlaintextTopicMetadataTest >

[jira] [Updated] (KAFKA-5149) New producer hardcodes key and value serializers to ByteArray

2017-05-01 Thread Yeva Byzek (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeva Byzek updated KAFKA-5149: -- Labels: newbie (was: ) > New producer hardcodes key and value serializers to ByteArray >

[jira] [Created] (KAFKA-5149) New producer hardcodes key and value serializers to ByteArray

2017-05-01 Thread Yeva Byzek (JIRA)
Yeva Byzek created KAFKA-5149: - Summary: New producer hardcodes key and value serializers to ByteArray Key: KAFKA-5149 URL: https://issues.apache.org/jira/browse/KAFKA-5149 Project: Kafka Issue

[jira] [Created] (KAFKA-5148) Add configurable compression block size to the broker

2017-05-01 Thread Dustin Cote (JIRA)
Dustin Cote created KAFKA-5148: -- Summary: Add configurable compression block size to the broker Key: KAFKA-5148 URL: https://issues.apache.org/jira/browse/KAFKA-5148 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-5147) KafkaProducer's TransactionManager needs a review on synchronization

2017-05-01 Thread Apurva Mehta (JIRA)
Apurva Mehta created KAFKA-5147: --- Summary: KafkaProducer's TransactionManager needs a review on synchronization Key: KAFKA-5147 URL: https://issues.apache.org/jira/browse/KAFKA-5147 Project: Kafka

[jira] [Work started] (KAFKA-5126) Implement KIP-98 transactional methods in the MockProducer

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-5126 started by Matthias J. Sax. -- > Implement KIP-98 transactional methods in the MockProducer >

[jira] [Assigned] (KAFKA-5126) Implement KIP-98 transactional methods in the MockProducer

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-5126: -- Assignee: Matthias J. Sax (was: Apurva Mehta) > Implement KIP-98 transactional

[jira] [Updated] (KAFKA-5055) Kafka Streams skipped-records-rate sensor producing nonzero values even when FailOnInvalidTimestamp is used as extractor

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5055: --- Status: Patch Available (was: Open) > Kafka Streams skipped-records-rate sensor producing

Re: [DISCUSS] KIP 141 - ProducerRecordBuilder Interface

2017-05-01 Thread Matthias J. Sax
Hi, I am personally not a big fan of providing two APIs to do the same thing. If we believe that one API is better than the other, we should indicate this by deprecating the old API IMHO. Just my two cents. -Matthias On 4/30/17 11:05 PM, Michael Pearce wrote: > See > >

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-05-01 Thread Matthias J. Sax
Jeyhun, thanks a lot for the KIP! I think there are two issues we need to address: (1) The KIP does not consider backward compatibility. Users did complain about this in past releases already, and as the user base grows, we should not break backward compatibility in future releases anymore.

Re: [DISCUSS] KIP-145: Classloading Isolation in Connect

2017-05-01 Thread Colin McCabe
+1 (non-binding) This would be a nice improvement. C. On Fri, Apr 28, 2017, at 11:03, Konstantine Karantasis wrote: > Hi everyone, > > we aim to address dependency conflicts in Kafka Connect soon by applying > class loading isolation. > > Feel free to take a look at KIP-145 here: > >

Re: [DISCUSS] KIP-144: Exponential backoff for broker reconnect attempts

2017-05-01 Thread Colin McCabe
Thanks for the KIP, Ismael & Dana! This could be pretty important for avoiding congestion collapse when there are a lot of clients. It seems like a good idea to keep the "ms" suffix, like we have with "reconnect.backoff.ms". So maybe we should use "reconnect.backoff.max.ms"? In general

Re: Kafka-Streams: Cogroup

2017-05-01 Thread Guozhang Wang
Kyle, What's your apache id? I can grant you the permission. Guozhang On Sat, Apr 29, 2017 at 7:33 AM, Kyle Winkelman wrote: > I don't seem to have permission. When logged in I can neither edit the > main page nor create an additional KIP. > > Thanks, > Kyle > > On

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

2017-05-01 Thread Randall Hauch
Very nice work, Konstantine. Conflicting dependencies of connectors is indeed a big issue that makes it hard to manage installed connectors. I do like Gwen's idea about removing the 'module.isolation.enabled' property. However, I would have anticipated always using classpath isolation for *only*

Re: Edit Permissions for KIP Wiki

2017-05-01 Thread Jun Rao
Hi, Kyle, Thanks for your interest. Just gave the permission to the wiki. Jun On Mon, May 1, 2017 at 4:43 AM, Kyle Winkelman wrote: > Hello, > > I was hoping to get the ability to edit the wiki so I can post a KIP. > > Username: winkelman.kyle > > Thanks, > Kyle >

Re: Support for Kafka Consumer over SASL_SSL

2017-05-01 Thread Michal Borowiecki
Hi Nixon, kafka.consumer.Consumer is the old consumer. When the documentation is referring to the new producer and consumer, it means those in the org.apache.kafka.clients.producer and org.apache.kafka.clients.consumer packages respectively. They are packaged in the kafka-clients-0.10.x.y.jar

[jira] [Commented] (KAFKA-4954) Implementation and unit tests for request handler thread utilization quota

2017-05-01 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991013#comment-15991013 ] ASF GitHub Bot commented on KAFKA-4954: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2744: KAFKA-4954: Request handler utilization quotas

2017-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2744 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-4954) Implementation and unit tests for request handler thread utilization quota

2017-05-01 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-4954: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 2744

[jira] [Commented] (KAFKA-5144) MinTimestampTracker uses confusing variable names

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990952#comment-15990952 ] Matthias J. Sax commented on KAFKA-5144: Makes sense. Thanks for your contribution! >

[jira] [Updated] (KAFKA-5144) MinTimestampTracker uses confusing variable names

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5144: --- Description: --When adding elements MinTimestampTracker removes all existing elements

[jira] [Updated] (KAFKA-5144) MinTimestampTracker uses confusing variable names

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5144: --- Description: --When adding elements MinTimestampTracker removes all existing elements

[jira] [Updated] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5144: --- Fix Version/s: 0.11.0.0 Status: Patch Available (was: Open) > MinTimestampTracker

[jira] [Updated] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5144: --- Priority: Trivial (was: Major) > MinTimestampTracker does not correctly add timestamps lower

[jira] [Updated] (KAFKA-5144) MinTimestampTracker uses confusing variable names

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5144: --- Summary: MinTimestampTracker uses confusing variable names (was: MinTimestampTracker does

[jira] [Updated] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

2017-05-01 Thread Mariam John (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mariam John updated KAFKA-5096: --- Reviewer: Matthias J. Sax Status: Patch Available (was: Open) > Only log invalid user configs

[jira] [Commented] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

2017-05-01 Thread Mariam John (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990902#comment-15990902 ] Mariam John commented on KAFKA-5096: [~mjsax] This is ready for review. All the tests passed.

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

2017-05-01 Thread Gwen Shapira
Hi Konstantine, Thank you so much for driving this! The connector classpath mess is driving me nuts (or worse, driving me to use Docker). I like the proposal for micro-benchmarks to test the context switching overhead. I have a difficult time figuring out the module.isolation.enabled.

Re: [DISCUSS] KIP 145 - Expose Record Headers in Kafka Connect

2017-05-01 Thread Gwen Shapira
Hi, I'm excited to see the community expanding Connect in this direction! Headers + Transforms == Fun message routing. I like how clean the proposal is, but I'm concerned that it kinda deviates from how Connect handles data elsewhere. Unlike Kafka, Connect doesn't look at all data as

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-05-01 Thread Mathieu Fenniak
Hi Jeyhun, I just want to add my voice that, I too, have wished for access to the record key during a mapValues or similar operation. On the other hand, the number of compile failures that would need to be fixed from this change is unfortunate. :-) But at least it would all be a pretty clear

Support for Kafka Consumer over SASL_SSL

2017-05-01 Thread Nixon Rodrigues
Hi Kafka dev team, I am using kafka_2.11-0.10.0.0.jar api for kafka consumer, i am facing issue while consuming from topics over SASL_SSL, getting below exception Consumer.createJavaConsumerConnector(new kafka.consumer.ConsumerConfig(consumerProperties)) As per documentation

[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-05-01 Thread Jeyhun Karimov
Dear community, I want to share KIP-149 [1] based on issues KAFKA-4218 [2], KAFKA-4726 [3], KAFKA-3745 [4]. The related PR can be found at [5]. I would like to get your comments. [1]

Edit Permissions for KIP Wiki

2017-05-01 Thread Kyle Winkelman
Hello, I was hoping to get the ability to edit the wiki so I can post a KIP. Username: winkelman.kyle Thanks, Kyle

[jira] [Updated] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2017-05-01 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-5146: Description: We currently have a compile-dependency on `connect-json`: {code} org.apache.kafka

[jira] [Updated] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2017-05-01 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-5146: Affects Version/s: 0.10.2.1 0.10.1.0 0.10.2.0 > Kafka

[jira] [Updated] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2017-05-01 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-5146: Description: We currently have a compile-dependency on `connect-json`: {code} org.apache.kafka

[jira] [Updated] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2017-05-01 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-5146: Description: We currently have a compile-dependency on `connect-json`: {code} org.apache.kafka

[jira] [Created] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2017-05-01 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-5146: --- Summary: Kafka Streams: remove compile dependency on connect-json Key: KAFKA-5146 URL: https://issues.apache.org/jira/browse/KAFKA-5146 Project: Kafka Issue

[jira] [Commented] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread Michal Borowiecki (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990683#comment-15990683 ] Michal Borowiecki commented on KAFKA-5144: -- I understand now, thanks a lot for the thorough

[jira] [Commented] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990682#comment-15990682 ] ASF GitHub Bot commented on KAFKA-5144: --- Github user mihbor closed the pull request at:

[GitHub] kafka pull request #2947: KAFKA-5144 added 2 test cases

2017-05-01 Thread mihbor
Github user mihbor closed the pull request at: https://github.com/apache/kafka/pull/2947 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Comment Edited] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990643#comment-15990643 ] Matthias J. Sax edited comment on KAFKA-5144 at 5/1/17 7:30 AM: --After

[jira] [Comment Edited] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990619#comment-15990619 ] Matthias J. Sax edited comment on KAFKA-5144 at 5/1/17 7:29 AM: It is

[jira] [Commented] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990643#comment-15990643 ] Matthias J. Sax commented on KAFKA-5144: After digging a little deeper, partition time get's

[jira] [Commented] (KAFKA-5144) MinTimestampTracker does not correctly add timestamps lower than the current max

2017-05-01 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990619#comment-15990619 ] Matthias J. Sax commented on KAFKA-5144: It is intended behavior. {{MinTimestampTracker}} tracks

Re: [DISCUSS] KIP 141 - ProducerRecordBuilder Interface

2017-05-01 Thread Michael Pearce
See https://hc.apache.org/httpcomponents-client-ga/tutorial/html/fluent.html Doesn't cause much issue over there where you have a fluent api wrapper for those who like that style, and the original more verbose api. Surely it would be better than removing a way of doing things that everyone has