Build failed in Jenkins: kafka-2.3-jdk8 #160

2020-01-06 Thread Apache Jenkins Server
See Changes: [wangguoz] HOTFIX: fix system test race condition (#7836) -- [...truncated 2.94 MB...] kafka.log.LogValidatorTest > testCompressedV1 STARTED

[jira] [Created] (KAFKA-9374) Worker can be disabled by blocked connectors

2020-01-06 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-9374: Summary: Worker can be disabled by blocked connectors Key: KAFKA-9374 URL: https://issues.apache.org/jira/browse/KAFKA-9374 Project: Kafka Issue Type: Bug

Build failed in Jenkins: kafka-2.4-jdk8 #121

2020-01-06 Thread Apache Jenkins Server
See Changes: [bill] MINOR: Fixes for adding new DSL naming page (#7899) -- [...truncated 5.48 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

2020-01-06 Thread Николай Ижиков
Hello, Rajini. Can you, please, clarify, what should be done? I can try to do tests by myself. > 6 янв. 2020 г., в 21:29, Rajini Sivaram написал(а): > > Hi Brajesh. > > No one is working on this yet, but will follow up with the Confluent tools > team to see when this can be done. > > On Mon,

Jenkins build is back to normal : kafka-trunk-jdk11 #1055

2020-01-06 Thread Apache Jenkins Server
See

[jira] [Created] (KAFKA-9373) Improve shutdown performance via lazy accessing the offset and time indices.

2020-01-06 Thread Adem Efe Gencer (Jira)
Adem Efe Gencer created KAFKA-9373: -- Summary: Improve shutdown performance via lazy accessing the offset and time indices. Key: KAFKA-9373 URL: https://issues.apache.org/jira/browse/KAFKA-9373

[jira] [Created] (KAFKA-9372) Add producer config to make topicExpiry configurable

2020-01-06 Thread Jiao Zhang (Jira)
Jiao Zhang created KAFKA-9372: - Summary: Add producer config to make topicExpiry configurable Key: KAFKA-9372 URL: https://issues.apache.org/jira/browse/KAFKA-9372 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-9371) Disk space is not released after Kafka clears data due to retention settings

2020-01-06 Thread Arda Savran (Jira)
Arda Savran created KAFKA-9371: -- Summary: Disk space is not released after Kafka clears data due to retention settings Key: KAFKA-9371 URL: https://issues.apache.org/jira/browse/KAFKA-9371 Project:

Re: [DISCUSS] KIP-551: Expose disk read and write metrics

2020-01-06 Thread Colin McCabe
On Tue, Dec 10, 2019, at 11:10, Magnus Edenhill wrote: > Hi Colin, > Hi Magnus, Thanks for taking a look. > aren't those counters (ever increasing), rather than gauges (fluctuating)? Since this is in the Kafka broker, we're using Yammer. This might be confusing, but Yammer's concept of a

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Colin McCabe
On Mon, Jan 6, 2020, at 14:40, Brian Byrne wrote: > So the performance of a metadata RPC that occurs every once every 10 > seconds should not be measured strictly in CPU cost, but rather the effect > on the 95-99%. The larger the request is, the more opportunity there is to > put a burst stress on

Build failed in Jenkins: kafka-trunk-jdk11 #1054

2020-01-06 Thread Apache Jenkins Server
See Changes: [github] MINOR: Propagate LogContext to channel builders and SASL authenticator -- [...truncated 2.79 MB...] org.apache.kafka.streams.TestTopicsTest >

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Brian Byrne
Hello all, Does anyone else have opinions on the issue of RPC frequency? Would it be better to remove the fetching of non-urgent topics altogether, so that the refreshes are contained in a larger batch? Thanks, Brian On Mon, Jan 6, 2020 at 2:40 PM Brian Byrne wrote: > > So the performance of

Jenkins build is back to normal : kafka-trunk-jdk11 #1053

2020-01-06 Thread Apache Jenkins Server
See

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Brian Byrne
So the performance of a metadata RPC that occurs every once every 10 seconds should not be measured strictly in CPU cost, but rather the effect on the 95-99%. The larger the request is, the more opportunity there is to put a burst stress on the producer and broker, and the larger the response

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-06 Thread Colin McCabe
On Fri, Dec 27, 2019, at 10:48, Ron Dagostino wrote: > Hi everyone. I would like to make the following changes to the KIP. > > MOTIVATION: > Include a statement that it will be difficult in the short term to > deprecate direct Zookeeper communication in kafka-configs.{sh, bat} (which > invoke

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Colin McCabe
On Mon, Jan 6, 2020, at 13:07, Brian Byrne wrote: > Hi Colin, > > Thanks again for the feedback! > > On Mon, Jan 6, 2020 at 12:07 PM Colin McCabe wrote: > > > Metadata requests don't (always) go to the controller, right? We should > > fix the wording in this section. > > > > You're correct,

[jira] [Created] (KAFKA-9370) Return UNKNOWN_TOPIC_OR_PARTITION if topic deletion is in progress

2020-01-06 Thread Vikas Singh (Jira)
Vikas Singh created KAFKA-9370: -- Summary: Return UNKNOWN_TOPIC_OR_PARTITION if topic deletion is in progress Key: KAFKA-9370 URL: https://issues.apache.org/jira/browse/KAFKA-9370 Project: Kafka

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Brian Byrne
Hi Colin, Thanks again for the feedback! On Mon, Jan 6, 2020 at 12:07 PM Colin McCabe wrote: > Metadata requests don't (always) go to the controller, right? We should > fix the wording in this section. > You're correct, s/controller/broker(s)/. I feel like "Proposed Changes" should come

[jira] [Created] (KAFKA-9369) Allow Consumers and Producers to Connect with User-Agent String

2020-01-06 Thread David Mollitor (Jira)
David Mollitor created KAFKA-9369: - Summary: Allow Consumers and Producers to Connect with User-Agent String Key: KAFKA-9369 URL: https://issues.apache.org/jira/browse/KAFKA-9369 Project: Kafka

Jenkins build is back to normal : kafka-trunk-jdk8 #4132

2020-01-06 Thread Apache Jenkins Server
See

Re: Kafka 2.4.0 & Mirror Maker 2.0 Error

2020-01-06 Thread Ryanne Dolan
I just downloaded the 2.4.0 release tarball and didn't run into any issues. Peter, Jamie, can one of you file a jira ticket if you are still seeing this? Thanks! Ryanne On Fri, Dec 27, 2019 at 12:04 PM Ryanne Dolan wrote: > Thanks Peter, I'll take a look. > > Ryanne > > On Fri, Dec 27, 2019,

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Colin McCabe
Hi Brian, Thanks for continuing to work on this. It looks good overall. It's probably better to keep the discussion on this thread rather than going back and forth between this and the DISCUSS thread. The KIP states: > For (1), an RPC is generated every time an uncached topic's metadata

[jira] [Created] (KAFKA-9368) Preserve stream-time across rebalances/restarts

2020-01-06 Thread Sophie Blee-Goldman (Jira)
Sophie Blee-Goldman created KAFKA-9368: -- Summary: Preserve stream-time across rebalances/restarts Key: KAFKA-9368 URL: https://issues.apache.org/jira/browse/KAFKA-9368 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk11 #1052

2020-01-06 Thread Apache Jenkins Server
See Changes: [github] MINOR: Fix failing test case in TransactionLogTest (#7895) -- [...truncated 2.79 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest >

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

2020-01-06 Thread Rajini Sivaram
Hi Brajesh. No one is working on this yet, but will follow up with the Confluent tools team to see when this can be done. On Mon, Jan 6, 2020 at 3:29 PM Brajesh Kumar wrote: > Hello Rajini, > > What is the plan to run system tests using JDK 11? Is someone working on > this? > > On Mon, Jan 6,

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Lucas Bradstreet
+1 (non-binding) On Thu, Jan 2, 2020 at 11:15 AM Brian Byrne wrote: > Hello all, > > After further discussion and improvements, I'd like to reinstate the voting > process. > > The updated KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-526 >

[jira] [Resolved] (KAFKA-9345) Deploy Mirror Maker 2.0 and dynamically modify the topic whitelist through REST API

2020-01-06 Thread Ryanne Dolan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryanne Dolan resolved KAFKA-9345. - Resolution: Information Provided > Deploy Mirror Maker 2.0 and dynamically modify the topic

Build failed in Jenkins: kafka-trunk-jdk11 #1051

2020-01-06 Thread Apache Jenkins Server
See Changes: [manikumar.reddy] MINOR: Remove compilation warnings (#7888) -- [...truncated 5.65 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

[jira] [Created] (KAFKA-9367) CRC failure

2020-01-06 Thread Shivangi Singh (Jira)
Shivangi Singh created KAFKA-9367: - Summary: CRC failure Key: KAFKA-9367 URL: https://issues.apache.org/jira/browse/KAFKA-9367 Project: Kafka Issue Type: Bug Components: core

Build failed in Jenkins: kafka-trunk-jdk8 #4131

2020-01-06 Thread Apache Jenkins Server
See Changes: [manikumar.reddy] MINOR: Remove compilation warnings (#7888) -- [...truncated 5.63 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

2020-01-06 Thread Brajesh Kumar
Hello Rajini, What is the plan to run system tests using JDK 11? Is someone working on this? On Mon, Jan 6, 2020 at 3:00 PM Rajini Sivaram wrote: > Hi Nikolay, > > We can leave the KIP open and restart the discussion once system tests are > running. > > Thanks, > > Rajini > > On Mon, Jan 6,

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

2020-01-06 Thread Rajini Sivaram
Hi Nikolay, We can leave the KIP open and restart the discussion once system tests are running. Thanks, Rajini On Mon, Jan 6, 2020 at 2:46 PM Николай Ижиков wrote: > Hello, Rajini. > > Thanks, for the feedback. > > Should I mark this KIP as declined? > Or just wait for the system tests

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

2020-01-06 Thread Николай Ижиков
Hello, Rajini. Thanks, for the feedback. Should I mark this KIP as declined? Or just wait for the system tests results? > 6 янв. 2020 г., в 17:26, Rajini Sivaram написал(а): > > Hi Nikolay, > > Thanks for the KIP. We currently run system tests using JDK 8 and hence we > don't yet have full

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

2020-01-06 Thread Rajini Sivaram
Hi Nikolay, Thanks for the KIP. We currently run system tests using JDK 8 and hence we don't yet have full system test results with TLS 1.3 which requires JDK 11. We should wait until that is done before enabling TLS1.3 by default. Regards, Rajini On Mon, Dec 30, 2019 at 5:36 AM Николай

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Stanislav Kozlovski
+1 (non-binding) Thanks for the KIP, Brian! On Thu, Jan 2, 2020 at 7:15 PM Brian Byrne wrote: > Hello all, > > After further discussion and improvements, I'd like to reinstate the voting > process. > > The updated KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-526 >

Re: [DISCUSS] KIP-552: Add interface to handle unused config

2020-01-06 Thread Stanislav Kozlovski
Hey Artur, Perhaps changing the log level to DEBUG is the simplest approach. I wonder if other people know what the motivation behind the WARN log was? I'm struggling to think up of a scenario where I'd like to see unused values printed in anything above DEBUG. Best, Stanislav On Mon, Dec 30,