Build failed in Jenkins: kafka-trunk-jdk10 #354

See Changes: [wangguoz] KAFKA-6761: Construct Physical Plan using Graph, Reduce streams [harsha] MINOR: Fixed default streams state dir location. (#5441) --

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hey Guozhang, thanks for this detailed proposal! Quickly I want to clarify, where this registry id is generated? My understanding is that the registry id is a unique id provided by user correct? This way even if multiple consumers are using the same member id, we are fencing against polluting

Build failed in Jenkins: kafka-0.10.2-jdk7 #233

See Changes: [matthias] Bump version to 0.10.2.2 [jason] MINOR: Use kill_java_processes when killing ConsoleConsumer in system [jason] Revert "MINOR: Use kill_java_processes when killing ConsoleConsumer in

[jira] [Created] (KAFKA-7233) InMemoryKeyValueStore is not thread-safe for Interactive Queries

Hashan Gayasri Udugahapattuwa created KAFKA-7233: Summary: InMemoryKeyValueStore is not thread-safe for Interactive Queries Key: KAFKA-7233 URL: https://issues.apache.org/jira/browse/KAFKA-7233

Build failed in Jenkins: kafka-trunk-jdk8 #2853

See Changes: [harsha] MINOR: Fixed default streams state dir location. (#5441) -- [...truncated 425.38 KB...] kafka.api.SaslClientsWithInvalidCredentialsTest >

[DISCUSS] KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it

Hi, I would like to open a discussion thread on KIP-350: https://cwiki.apache.org/confluence/display/KAFKA/KIP-350%3A+Allow+kafka-topics.sh+to+take+brokerid+as+parameter+to+show+partitions+associated+with+it This is KIP is to add the optional brokerid parameter in kafka-topics.sh, to only get

[jira] [Created] (KAFKA-7232) Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it

Ratish Ravindran created KAFKA-7232: --- Summary: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it Key: KAFKA-7232 URL: https://issues.apache.org/jira/browse/KAFKA-7232

Re: [Vote] KIP-321: Update TopologyDescription to better represent Source and Sink Nodes

The only issue I see with this is that Sink#topic would also need to be Optional as was pointed out already. Since Sink#topic is a preexisting method, changing its return type would break backwards compatibility. On the other hand, it might be worth it to rip that bandaid now. Best, Nishanth

Build failed in Jenkins: kafka-trunk-jdk10 #353

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H20 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-0.10.2-jdk7 #232

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H27 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-trunk-jdk10 #352

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H31 (ubuntu xenial) in workspace

Re: Permission to create KIP

Thanks a lot. -Ratish On Thu, Aug 2, 2018 at 12:12 AM Matthias J. Sax wrote: > Done. :) > > -Matthias > > On 8/1/18 11:19 AM, Ratish Ravindran wrote: > > Userid is *rravindran* > > > > -Ratish > > > > On Wed, Aug 1, 2018 at 10:22 PM Matthias J. Sax > > wrote: > > > >> Can you provide you

Re: KAFKA-7137 - max time guarantees of compaction for GDPR et al

Thanks Jason. >From what we've seen so far adding a max time should be the easy part because there's already a block of checks for if compaction should be run. The active segment issue can be dealt with by specifying a time based segment. Triggering compaction is the harder part given it's not

Build failed in Jenkins: kafka-trunk-jdk10 #351

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H20 (ubuntu xenial) in workspace

Re: KAFKA-7137 - max time guarantees of compaction for GDPR et al

Hey Brett, I'd suggest going ahead with a KIP if you already have some ideas about it. One of the complications you may run into is that the active segment is not cleaned at present. I went ahead and gave you wiki permission. I assumed your username is brettrann, so let me know if it's

Build failed in Jenkins: kafka-0.10.2-jdk7 #231

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H26 (ubuntu xenial) in workspace

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

Hey Vahid, I talked with Colin offline. I think specifically he felt the version bump on the broker was overkill since the broker still has to support the empty group id for older versions. I had thought that eventually we would be able to remove those old versions, but it's true that this may

Re: KAFKA-7137 - max time guarantees of compaction for GDPR et al

Ping on this. We're prepared to do the work i'm just not sure of the right way to start. If nobody wants to discuss it now should we just write up a KIP? On Thu, Jul 19, 2018 at 10:57 AM Brett Rann wrote: > re: https://issues.apache.org/jira/browse/KAFKA-7137 > > My team is investigating what

Re: [DISCUSS] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hey Stanislav, Just to make sure I understood you right - you propose not exposing any new > exception types but rather the interface itself only? Yes, exactly. Our exception hierarchy is a bit of a mess to be honest. Interfaces are more flexible and here it simplifies the error handling.

Build failed in Jenkins: kafka-trunk-jdk10 #350

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H20 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-0.10.2-jdk7 #230

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H27 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-trunk-jdk10 #349

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H20 (ubuntu xenial) in workspace

Jenkins build is back to normal : kafka-0.11.0-jdk7 #394

See

Build failed in Jenkins: kafka-0.10.2-jdk7 #229

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H26 (ubuntu xenial) in workspace

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

If there is no additional feedback on this KIP I'll start a vote soon. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 07/18/2018 04:47 PM Subject:Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol The KIP is updated to follow the

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

Thanks Jason for responding to Colin's concerns. If there are no other comment / feedback / objection I'll start a vote soon. Thanks. --Vahid From: Jason Gustafson To: dev Date: 07/27/2018 10:38 AM Subject:Re: [DISCUSS] KIP-289: Improve the default group id behavior in

Build failed in Jenkins: kafka-trunk-jdk10 #348

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H20 (ubuntu xenial) in workspace

Jenkins build is back to normal : kafka-1.0-jdk7 #222

See

Build failed in Jenkins: kafka-0.10.2-jdk7 #228

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H26 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-trunk-jdk10 #347

See Changes: [wangguoz] MINOR: Remove 1 minute minimum segment interval (#5323) [wangguoz] KAFKA-7161: check invariant: oldValue is in the state (#5366) [github] MINOR: Producers should set delivery timeout

Build failed in Jenkins: kafka-trunk-jdk8 #2851

See Changes: [wangguoz] MINOR: Remove 1 minute minimum segment interval (#5323) [wangguoz] KAFKA-7161: check invariant: oldValue is in the state (#5366) [github] MINOR: Producers should set delivery timeout

Jenkins build is back to normal : kafka-2.0-jdk8 #97

See

Re: Discussion: New components in JIRA?

I haven't seen any comments.  Let me know if/when you add the new components.  Thanks. -Ray On 7/27/18 9:54 PM, Guozhang Wang wrote: Hello Ray, Any PMC member of the project can add more components in the JIRA system. If there is no objection in the next 72 hours I can just go ahead and add

[jira] [Created] (KAFKA-7231) NetworkClient.newClientRequest() ignores custom request timeout in favor of the default

Ron Dagostino created KAFKA-7231: Summary: NetworkClient.newClientRequest() ignores custom request timeout in favor of the default Key: KAFKA-7231 URL: https://issues.apache.org/jira/browse/KAFKA-7231

Re: Permission to create KIP

Done. :) -Matthias On 8/1/18 11:19 AM, Ratish Ravindran wrote: > Userid is *rravindran* > > -Ratish > > On Wed, Aug 1, 2018 at 10:22 PM Matthias J. Sax > wrote: > >> Can you provide you login ID so we can grant permission? >> >> (The screenshot was striped btw) >> >> -Matthias >> >> >> On

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

I’m a little confused about something. Is this KIP focused on log cleaner exceptions in general, or focused on log cleaner exceptions due to disk failures? Will max.uncleanable.partitions apply to all exceptions (including log cleaner logic errors) or will it apply to only disk I/o exceptions?

Re: Permission to create KIP

Userid is *rravindran* -Ratish On Wed, Aug 1, 2018 at 10:22 PM Matthias J. Sax wrote: > Can you provide you login ID so we can grant permission? > > (The screenshot was striped btw) > > -Matthias > > > On 7/31/18 9:56 PM, Ratish Ravindran wrote: > > > > >

Build failed in Jenkins: kafka-trunk-jdk8 #2850

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H20 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-trunk-jdk10 #346

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H30 (ubuntu xenial) in workspace

[jira] [Resolved] (KAFKA-7161) KTable Reduce should check for invalid conditions

[ https://issues.apache.org/jira/browse/KAFKA-7161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7161. -- Resolution: Fixed Fix Version/s: 2.1.0 > KTable Reduce should check for invalid

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hello Boyang, Thanks for the great summary. Personally I think 1) is still doable as we are not necessarily have to rely on source of truth to always fence "the right zombie"; in fact, my point is that as long as we can fence "someone" when there are zombies sharing the same member id so that

[jira] [Created] (KAFKA-7230) Empty Record created when producer failed due to RecordTooLargeException

Julien Fabre created KAFKA-7230: --- Summary: Empty Record created when producer failed due to RecordTooLargeException Key: KAFKA-7230 URL: https://issues.apache.org/jira/browse/KAFKA-7230 Project: Kafka

Re: [Vote] KIP-321: Update TopologyDescription to better represent Source and Sink Nodes

For source node, only one of `Set topicsSet` and `TopicPattern topicPattern()` will be specified by the user. Similarly for sink node, only one of `String` and `TopicNameExtractor` will be specified by the user. Although I've not seen Nishanth's updated PR, I think when it is not specified today

[jira] [Resolved] (KAFKA-7229) Failed to dynamically update kafka certificate in kafka 2.0.0

[ https://issues.apache.org/jira/browse/KAFKA-7229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Yang resolved KAFKA-7229. Resolution: Not A Problem > Failed to dynamically update kafka certificate in kafka 2.0.0 >

Re: Permission to create KIP

Can you provide you login ID so we can grant permission? (The screenshot was striped btw) -Matthias On 7/31/18 9:56 PM, Ratish Ravindran wrote: > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > When tried to click on Create KIP >

[jira] [Created] (KAFKA-7229) Failed to dynamically update kafka certificate in kafka 2.0.0

Yu Yang created KAFKA-7229: -- Summary: Failed to dynamically update kafka certificate in kafka 2.0.0 Key: KAFKA-7229 URL: https://issues.apache.org/jira/browse/KAFKA-7229 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Yes, good catch. Thank you, James! Best, Stanislav On Wed, Aug 1, 2018 at 5:05 PM James Cheng wrote: > Can you update the KIP to say what the default is for > max.uncleanable.partitions? > > -James > > Sent from my iPhone > > > On Jul 31, 2018, at 9:56 AM, Stanislav Kozlovski > wrote: > > > >

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Can you update the KIP to say what the default is for max.uncleanable.partitions? -James Sent from my iPhone > On Jul 31, 2018, at 9:56 AM, Stanislav Kozlovski > wrote: > > Hey group, > > I am planning on starting a voting thread tomorrow. Please do reply if you > feel there is anything

[jira] [Created] (KAFKA-7228) DeadLetterQueue throws a NullPointerException

Arjun Satish created KAFKA-7228: --- Summary: DeadLetterQueue throws a NullPointerException Key: KAFKA-7228 URL: https://issues.apache.org/jira/browse/KAFKA-7228 Project: Kafka Issue Type: Task

Re: [DISCUSS] KIP-332: Update AclCommand to use AdminClient API

Hi all, If there are no concerns, I will start the voting process soon. Thanks On Tue, Jul 31, 2018 at 9:08 AM Manikumar wrote: > Hi Colin, > > Yes, "--authorizer-properties" option is not required with > "--bootstrap-server" option. Updated the KIP. > > > Thanks, > > On Tue, Jul 31, 2018 at

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hey group, I just wanted to note that I have an implementation ready for review. Feel free to take a quick look and raise any concerns you might have in due time. I plan on starting the voting thread tomorrow. Best, Stanislav On Wed, Aug 1, 2018 at 10:01 AM Stanislav Kozlovski wrote: > Hey

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Thanks for everyone's (Guozhang, Jason, Mike, John and someone in case I'm missing) input here! Looks like we have got more interesting thoughts on this KIP. When I draft KIP-345, the original proposal is targeted at solving "static membership", which is trying to address 1/2/5/6 proposed by

[jira] [Created] (KAFKA-7227) Kafka cluster should has a default rack id if not set

Genmao Yu created KAFKA-7227: Summary: Kafka cluster should has a default rack id if not set Key: KAFKA-7227 URL: https://issues.apache.org/jira/browse/KAFKA-7227 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-7226) kafka-console-consumer.sh doesn't use security.protocol provided in config file

Alexandre GRIFFAUT created KAFKA-7226: - Summary: kafka-console-consumer.sh doesn't use security.protocol provided in config file Key: KAFKA-7226 URL: https://issues.apache.org/jira/browse/KAFKA-7226

Re: [Vote] KIP-321: Update TopologyDescription to better represent Source and Sink Nodes

Ewen - no as I don't believe they are never null. Whereas the topicNameExtractor method returns null if it is the default extractor or the extractor. So i think this would be better to be optional as it is optionally returning a TopicNameExtractor On Tue, 31 Jul 2018 at 23:01 Ewen

[jira] [Created] (KAFKA-7225) Kafka Connect ConfigProvider not invoked before validation

Nacho Munoz created KAFKA-7225: -- Summary: Kafka Connect ConfigProvider not invoked before validation Key: KAFKA-7225 URL: https://issues.apache.org/jira/browse/KAFKA-7225 Project: Kafka Issue

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hey Ray, * Yes, we'd have the logDir as a tag in the metric * The intention is to have Int.MaxValue as the maximum uncleanable partitions count * My idea is to store the marked logs (actually partitions) in memory instead of the ".skip" files to keep the change simple. I have also decided to omit

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

Hi All, I’ve updated the proposal to include the improvements suggested by Stephane. I have also submitted a PR to implement this functionality into Kafka. https://github.com/apache/kafka/pull/5438 I don’t have a benchmark against MirrorMaker yet, as I only currently have a local docker stack