Jenkins build is back to normal : kafka-trunk-jdk10 #302

See

Jenkins build is back to normal : kafka-1.1-jdk7 #167

See

Re: [Discuss] KIP-321: Add method to get TopicNameExtractor in TopologyDescription

I definitely agree with you on deprecating topics(). I also think changing the method names for consistency is reasonable, since there is no functionality change. Although, I can be convinced either way on this one. Best, Nishanth Pradeep On Fri, Jul 20, 2018 at 12:15 PM Matthias J. Sax wrote:

Build failed in Jenkins: kafka-trunk-jdk8 #2815

See Changes: [rajinisivaram] KAFKA-7185: Allow empty resource name when matching ACLs (#5400) -- [...truncated 429.66 KB...] kafka.utils.SchedulerTest >

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hi Ted, I do plan to start one. When is the appropriate time? My reasoning was that people would like to view the changes first On Fri, Jul 20, 2018, 6:21 PM Ted Yu wrote: > Hi, Stanislav: > Do you plan to start VOTE thread ? > > Cheers > > On Fri, Jul 20, 2018 at 6:11 PM Stanislav Kozlovski <

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hi, Stanislav: Do you plan to start VOTE thread ? Cheers On Fri, Jul 20, 2018 at 6:11 PM Stanislav Kozlovski wrote: > Hey group, > > I added a Pull Request for this KIP - here it is > https://github.com/apache/kafka/pull/5410 > Please take a look. > > Best, > Stanislav > > On Thu, Jul 5, 2018

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hey group, I added a Pull Request for this KIP - here it is https://github.com/apache/kafka/pull/5410 Please take a look. Best, Stanislav On Thu, Jul 5, 2018 at 11:06 AM Ismael Juma wrote: > Yes, the Scala consumers have been removed in 2.0.0, which simplifies some > of this. The following

Build failed in Jenkins: kafka-trunk-jdk8 #2814

See Changes: [github] KAFKA-5037 Follow-up: move Scala test to Java (#5399) [matthias] MINOR: fix upgrade docs for Streams (#5394) [jason] MINOR: Add thread dumps if broker node cannot be stopped (#5373)

Build failed in Jenkins: kafka-2.0-jdk8 #80

See Changes: [rajinisivaram] KAFKA-7185: Allow empty resource name when matching ACLs (#5400) -- [...truncated 2.48 MB...] org.apache.kafka.streams.TopologyTest >

Build failed in Jenkins: kafka-trunk-jdk10 #301

See Changes: [rajinisivaram] KAFKA-7185: Allow empty resource name when matching ACLs (#5400) -- [...truncated 1.54 MB...] kafka.security.auth.ZkAuthorizationTest >

Discussion: New components in JIRA?

I've been doing a little bit of component cleanup in JIRA.  What do people think of adding one or more of the following components? - logging: For any consumer/producer/broker logging (i.e. log4j). This should help disambiguate from the "log" component (i.e. Kafka messages). - mirrormaker:

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Hi Jason, The proposed API seems reasonable to me too. Could you please also update the wiki page ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-320%3A+Allow+fetchers+to+detect+and+handle+log+truncation) with a section say "workflow" on how the proposed API will be co-used with others

Build failed in Jenkins: kafka-2.0-jdk8 #79

See Changes: [github] MINOR: fix upgrade docs for Streams (#5394) -- [...truncated 435.49 KB...] kafka.server.epoch.LeaderEpochFileCacheTest >

[jira] [Resolved] (KAFKA-7185) getMatchingAcls throws StringIndexOutOfBoundsException for empty resource name

[ https://issues.apache.org/jira/browse/KAFKA-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-7185. --- Resolution: Fixed Reviewer: Ismael Juma Fix Version/s: 2.0.0 >

Build failed in Jenkins: kafka-1.1-jdk7 #166

See Changes: [jason] MINOR: Add thread dumps if broker node cannot be stopped (#5373) -- [...truncated 397.82 KB...] at java.io.FileOutputStream.writeBytes(Native

Build failed in Jenkins: kafka-trunk-jdk8 #2813

See Changes: [wangguoz] MINOR: Fix format violations streams scala tests (#5402) [jason] KAFKA-7141; Consumer group describe should include groups with no --

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

Thanks, +1 from me. On Fri, Jul 20, 2018 at 11:41 AM, Manikumar wrote: > Thanks for the feedback, sounds good to me. Updated the KIP. > > On Sat, Jul 21, 2018 at 12:04 AM Jason Gustafson > wrote: > > > After thinking about it, perhaps a better option is INVALID_REQUEST? I > > think this is

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

Thanks for the feedback, sounds good to me. Updated the KIP. On Sat, Jul 21, 2018 at 12:04 AM Jason Gustafson wrote: > After thinking about it, perhaps a better option is INVALID_REQUEST? I > think this is usually used for requests which are malformed in some way, so > it's a slight abuse, but

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

After thinking about it, perhaps a better option is INVALID_REQUEST? I think this is usually used for requests which are malformed in some way, so it's a slight abuse, but at least it suggests an incompatibility between what the client is requesting to do and what the server supports.

Jenkins build is back to normal : kafka-trunk-jdk10 #299

See

[jira] [Created] (KAFKA-7190) Under low traffic conditions purging repartition topics cause WARN statements about UNKNOWN_PRODUCER_ID

Bill Bejeck created KAFKA-7190: -- Summary: Under low traffic conditions purging repartition topics cause WARN statements about UNKNOWN_PRODUCER_ID Key: KAFKA-7190 URL:

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

I updated the KIP. https://cwiki.apache.org/confluence/display/KAFKA/KIP-339%3A+Create+a+new+IncrementalAlterConfigs+API Updates: * Use "incrementalAlterConfigs" rather than "modifyConfigs," for consistency with the other "alter" APIs. * Implement Magnus' idea of supporting "append" and

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

I agree with Jason's notion that > implicit use of the empty group.id to commit offsets is more likely to be causing users unexpected problems than actually providing a useful capability. I was initially confused that this is the behavior when investigating a new-ish JIRA issue

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

If we are looking into deprecating the empty group id, would it also make sense to have the same character restriction for it as that for topic names? We have stricter validation for topic names but none for group id and transaction id. I think we should (eventually) make character restriction the

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

Hi Jason, Yes, we can return UNKNOWN_SERVER_ERROR for older versions. This will give better developer experience. Updated the KIP. Thanks, On Fri, Jul 20, 2018 at 11:02 PM Jason Gustafson wrote: > Hey Manikumar, > > This looks good. Do we have to keep the current behavior when handling old >

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

Hey Manikumar, This looks good. Do we have to keep the current behavior when handling old DeleteTopics versions? I'm wondering if it would be better to return an UNKNOWN_ERROR (say) and let the client raise that to the user instead of just timing out. Thanks, Jason On Fri, Jul 20, 2018 at 10:00

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 to deprecate use of "" as group.id since it is odd to have a resource name that you cannot set ACLs for. Agree, we have to support older clients though. Thanks, Rajini On Fri, Jul 20, 2018 at 5:25 PM, Jason Gustafson wrote: > Hi Vahid, > > Sorry for getting to this so late. I think there

Re: [Discuss] KIP-321: Add method to get TopicNameExtractor in TopologyDescription

I would still deprecate existing `topics()` method. If users need a String, they can call `topicSet().toString()`. It's just a personal preference, because I believe it's good to keep the API "minimal". About renaming the other methods: I thinks it's a very small burden to deprecate the existing

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

Bumping this thread. On Tue, Jul 10, 2018 at 10:33 PM Manikumar wrote: > Waiting for one more binding vote to pass this minor KIP. Appreciate your > vote. > > On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska > wrote: > >> +1 (non binding) >> >> On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Ron, I saw that and decided that would be the best approach. The current ScramExtensions implementation uses a Map in the public credentials and I thought I would follow convention rather than introduce my own thing, but maybe this is best On Fri, Jul 20, 2018 at 8:39 AM Ron Dagostino wrote:

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

Hi Vahid, Sorry for getting to this so late. I think there are two things here: 1. The use of "" as a groupId has always been a dubious practice at best. We definitely ought to deprecate its use in the client. Perhaps in the next major release, we can remove support completely. However, since

[jira] [Resolved] (KAFKA-7081) Add describe all topics API to AdminClient

[ https://issues.apache.org/jira/browse/KAFKA-7081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-7081. -- Resolution: Won't Fix Closing this KIP in favor of adding filtering support to the Metadata API  

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Stanislav. I'm wondering if we should make SaslExtensions part of the public API. I mentioned this in my review of the PR, too (and tagged Rajini to get her input). If we add a Map to the Subject's public credentials we are basically making a public commitment that any Map associated with

Build failed in Jenkins: kafka-trunk-jdk8 #2812

See Changes: [github] MINOR: Fix transient test failure in SslTransportLayerTest (#5396) [github] KAFKA-3702: Change log level of SSL close_notify failure (#5397) --

[jira] [Created] (KAFKA-7189) Add a Merge Transformer for Kafka Connect

Abdelhamide EL ARIB created KAFKA-7189: -- Summary: Add a Merge Transformer for Kafka Connect Key: KAFKA-7189 URL: https://issues.apache.org/jira/browse/KAFKA-7189 Project: Kafka Issue

[jira] [Created] (KAFKA-7188) Avoid reverse DNS lookup in SASL channel builder

Rajini Sivaram created KAFKA-7188: - Summary: Avoid reverse DNS lookup in SASL channel builder Key: KAFKA-7188 URL: https://issues.apache.org/jira/browse/KAFKA-7188 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-2.0-jdk8 #78

See Changes: [rajinisivaram] MINOR: Fix transient test failure in SslTransportLayerTest (#5396) [rajinisivaram] KAFKA-3702: Change log level of SSL close_notify failure (#5397)

Re: [DISCUSSION] KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

> The approach where all the methods have a default implementation and the > user chooses to override one of them seems the most appealing to me given > the current state. sounds good. one Q: Which implementation is suitable for serialize()/deserialize()? maybe just throw exception? BTW,

Build failed in Jenkins: kafka-trunk-jdk10 #298

See Changes: [github] MINOR: Fix transient test failure in SslTransportLayerTest (#5396) [github] KAFKA-3702: Change log level of SSL close_notify failure (#5397) --

Build failed in Jenkins: kafka-trunk-jdk8 #2811

See Changes: [lindong28] KAFKA-7098; Improve accuracy of throttling by avoiding under-estimating -- [...truncated 2.63 MB...]

Re: [VOTE] KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde

> Part of the motivation for this KIP is to make these interfaces functional > interfaces. But I think that may not be desirable due to the method that > passes headers. So, it doesn't make sense to discuss two separate changes > to the same interfaces in isolation, we should figure out how we

Requesting review of MINOR PR, make regular expressions statically compiled to prevent constant recompilation.

Hi all, The PR: https://github.com/apache/kafka/pull/5168 This MINOR PR has been around for a while, guess I'll poke here a bit. Poking was suggested somewhere, I recall... What does the PR do? It makes statically compiled regular expressions to replace the current in-line regular expressions,

Re: [VOTE] KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde

Part of the motivation for this KIP is to make these interfaces functional interfaces. But I think that may not be desirable due to the method that passes headers. So, it doesn't make sense to discuss two separate changes to the same interfaces in isolation, we should figure out how we want them

Re: [VOTE] KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde

> The KIP needs 3 binding votes to pass. Thanks for the reminder. I will reopen the ballot box until we get 3 tickets. > I still think we should include the details of how things will look like > with the headers being passed to serializers/deserializers to ensure > things actually make sense

[jira] [Created] (KAFKA-7187) offsetsForTimes in MockConsumer implementation

Jing Chen created KAFKA-7187: Summary: offsetsForTimes in MockConsumer implementation Key: KAFKA-7187 URL: https://issues.apache.org/jira/browse/KAFKA-7187 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-344: The auto-generated client id should be passed to MetricsReporter

Looks good to me. On Fri, Jul 20, 2018 at 12:17 AM Kevin Lu wrote: > Hi All, > > This is the discussion thread for KIP-344 ( > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-344%3A+The+auto-generated+client+id+should+be+passed+to+MetricsReporter > ). > > Thanks! > > Regards, > Kevin >

Re: Contribution request

thank you On Thu, Jul 19, 2018 at 6:14 PM Matthias J. Sax wrote: > Done > > On 7/19/18 8:49 AM, András Katona wrote: > > Hello Team, > > > > Could you add me to the contributors on Apache Kafka project in JIRA, > > please? > > Username: akatona > > > > Thank you, > > Andras Katona > > > >

Re: [ANNOUNCE] Apache Kafka 1.1.1 Released

Thank you for great work! :) On Fri, Jul 20, 2018 at 11:25 AM Guozhang Wang wrote: > Dong, thanks for driving the release! > > On Thu, Jul 19, 2018 at 6:51 PM, Ismael Juma wrote: > > > Thank you for managing the release Dong! > > > > Ismael > > > > On Thu, 19 Jul 2018, 16:54 Dong Lin, wrote:

[DISCUSS] KIP-344: The auto-generated client id should be passed to MetricsReporter

Hi All, This is the discussion thread for KIP-344 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-344%3A+The+auto-generated+client+id+should+be+passed+to+MetricsReporter ). Thanks! Regards, Kevin

[jira] [Resolved] (KAFKA-7098) Improve accuracy of the log cleaner throttle rate

[ https://issues.apache.org/jira/browse/KAFKA-7098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-7098. - Resolution: Fixed > Improve accuracy of the log cleaner throttle rate >

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

Lucas and Mayuresh, Good idea. The correlation id should work. In the ControllerChannelManager, a request will be resent until a response is received. So if the controller to broker connection disconnects after controller sends R1_a, but before the response of R1_a is received, a disconnection