[VOTE] 2.5.0 RC1

Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 2.5.0. The first release candidate included an erroneous NOTICE file, so another RC was needed to fix that. This is a major release of Kafka which includes many new features,

Build failed in Jenkins: kafka-trunk-jdk11 #1224

See Changes: [github] KAFKA-9176: Do not update limit offset if we are in RESTORE_ACTIVE mode -- [...truncated 2.91 MB...]

Build failed in Jenkins: kafka-trunk-jdk8 #4305

See Changes: [github] KAFKA-9176: Do not update limit offset if we are in RESTORE_ACTIVE mode -- [...truncated 2.89 MB...]

Re: [Vote] KIP-569: Update DescribeConfigsResponse to include additional metadata information

+1 Looks great. Thanks for the proposal, Shailesh. Gwen Shapira Engineering Manager | Confluent 650.450.2760 | @gwenshap Follow us: Twitter | blog On Mon, Mar 09, 2020 at 6:00 AM, Shailesh Panwar < span...@confluent.io > wrote: > > > > Hi All, > I would like to start a vote on KIP-569:

[jira] [Created] (KAFKA-9689) Automatic broker version detection to initialize stream client

Boyang Chen created KAFKA-9689: -- Summary: Automatic broker version detection to initialize stream client Key: KAFKA-9689 URL: https://issues.apache.org/jira/browse/KAFKA-9689 Project: Kafka

[jira] [Resolved] (KAFKA-9176) Flaky test failure: OptimizedKTableIntegrationTest.shouldApplyUpdatesToStandbyStore

[ https://issues.apache.org/jira/browse/KAFKA-9176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-9176. -- Fix Version/s: 2.6.0 Resolution: Fixed > Flaky test failure: >

[jira] [Resolved] (KAFKA-3702) SslTransportLayer.close() does not shutdown gracefully

[ https://issues.apache.org/jira/browse/KAFKA-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3702. Fix Version/s: 2.0.0 Resolution: Fixed > SslTransportLayer.close() does not shutdown

Re: Can I ask protocol questions here?

Hi Chris I think it’s fine to ask it here. I’m not aware of any rules against it. Adam > On Mar 9, 2020, at 10:43 AM, Chris Done wrote: > > Hi all, > > I'm writing a Kafka client at the protocol level and was wondering whether > here, or the users@ mailing list was more appropriate for

Re: [DISCUSS] Apache Kafka 2.5.0 release

Hi Ismael, I understand your reasoning. I thought, since it is a broken feature, I better let this thread know about it. Best, Bruno On Mon, Mar 9, 2020 at 10:00 PM Ismael Juma wrote: > > Hi Bruno, > > 2.4.0 has been out for months. If this was not noticed since then, I don't > think it

Re: Subject: [VOTE] 2.4.1 RC0

Is this a blocker given that it's been like this for months and no-one noticed? 2.4.1 seemingly has all the votes needed for the release. Why not go ahead with it. When KAFKA-9675 is merged, it can be included in the next release. Ismael On Mon, Mar 9, 2020 at 8:43 PM Bill Bejeck wrote: >

Re: [DISCUSS] Apache Kafka 2.5.0 release

Hi Bruno, 2.4.0 has been out for months. If this was not noticed since then, I don't think it qualifies as a blocker for 2.5.0. Ismael On Mon, Mar 9, 2020 at 8:44 PM Bruno Cadonna wrote: > Hi David, > > A bug report was filed that can be considered a blocker. Basically, > with this bug all

Re: [DISCUSS] Apache Kafka 2.5.0 release

Hi David, A bug report was filed that can be considered a blocker. Basically, with this bug all RocksDB metrics reported by Streams are constant zero. The bug is there since 2.4., so it is not a regression, but a broken feature. Here is the ticket:

Re: Subject: [VOTE] 2.4.1 RC0

Thanks to everyone for voting. A new blocker has surfaced https://issues.apache.org/jira/browse/KAFKA-9675, so I'll do another RC soon. Thanks again. Bill On Mon, Mar 9, 2020 at 1:35 PM Levani Kokhreidze wrote: > +1 non-binding. > > - Built from source > - Ran unit tests. All passed. > -

Build failed in Jenkins: kafka-trunk-jdk11 #1223

See Changes: [github] MINOR: Remove throttling logic from RecordAccumulator (#7195) -- [...truncated 2.90 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest

Build failed in Jenkins: kafka-trunk-jdk8 #4304

See Changes: [github] MINOR: Remove throttling logic from RecordAccumulator (#7195) -- [...truncated 2.89 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: 回复：回复：[Vote] KIP-571: Add option to force remove members in StreamsResetter

Hey Feyman, 1) Regarding point 2 in your last email, if I understand correctly you propose to change the current behavior of the reset tool when --force is not specified, and wait for (up to) the session timeout for all members to be removed. I'm not sure we should change this, especially now

[jira] [Created] (KAFKA-9688) kafka-topic.sh should show KIP-455 adding and removing replicas

Colin McCabe created KAFKA-9688: --- Summary: kafka-topic.sh should show KIP-455 adding and removing replicas Key: KAFKA-9688 URL: https://issues.apache.org/jira/browse/KAFKA-9688 Project: Kafka

Build failed in Jenkins: kafka-2.3-jdk8 #183

See Changes: [github] KAFKA-8245: Fix Flaky Test -- [...truncated 2.95 MB...] kafka.log.LogCleanerTest > testPartialSegmentClean STARTED kafka.log.LogCleanerTest >

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

Hi Colin, Thanks for the suggestion. Using a dash does seem reasonable. I can make that change. On Mon, Mar 9, 2020 at 12:35 PM Colin McCabe wrote: > > Hi Aneel, > > Thanks for the KIP. I like the idea. > > You mention that "input from STDIN can be used instead of a file on disk." > The

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

Hi David, Is the expected behavior that the keys are deleted without checking the values? Let's say I had this file new.properties: a=1 b=2 And ran: bin/kafka-configs --bootstrap-server localhost:9092 \ --entity-type brokers --entity-default \ --alter --add-config-file new.properties It

Jenkins build is back to normal : kafka-2.2-jdk8 #33

See

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

Hi Colin, Yes, you're right. This is weird but convenient because you don't have to duplicate the "keys". I was thinking about the kubernetes API which allows to create a Pod based on a file and allows to delete it as well with the same file. I have always found this convenient, especially when

Can I ask protocol questions here?

Hi all, I'm writing a Kafka client at the protocol level and was wondering whether here, or the users@ mailing list was more appropriate for questions of that nature? I looked on the web site, but didn't see clarification on this point. I'll start a fresh thread if here is indeed the correct

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

Hi Aneel, Thanks for the KIP. I like the idea. You mention that "input from STDIN can be used instead of a file on disk." The example given in the KIP seems to suggest that the command defaults to reading from STDIN if no argument is given to --add-config-file. I would argue against this

Re: Subject: [VOTE] 2.4.1 RC0

+1 non-binding. - Built from source - Ran unit tests. All passed. - Quickstart passed. Looking forward upgrading to 2.4.1 Regards, Levani On Mon, 9 Mar 2020, 17:11 Sean Glover, wrote: > +1 (non-binding). I built from source and ran the unit test suite > successfully. > > Thanks for running

[jira] [Created] (KAFKA-9687) Improve Trogdor workload exit logging

Colin McCabe created KAFKA-9687: --- Summary: Improve Trogdor workload exit logging Key: KAFKA-9687 URL: https://issues.apache.org/jira/browse/KAFKA-9687 Project: Kafka Issue Type: Improvement

Re: Question about log flusher real frequency

Hi Alexandre, Thank you for your quick answer. I want to monitor it cause I'm trying to find out the reason why our existing Kafka cluster is configured to flush data every10 milliseconds! (people who configured it are not available anymore to answer). As that value seems really low to me, I

Re: Question about log flusher real frequency

Hi Fares, On Linux kernels, you can use the property "dirty_writeback_centisecs" [1] to configure the period between executions of kswapd, which does this "sync" job. The period is usually set to 30 seconds. There are few exceptions where Kafka explicitly forces a sync (via the force() method

Question about log flusher real frequency

Hello, By default, both log.flush.interval.ms and log.flush.interval.messages are set to Long.MAX_VALUE. As I understand, it makes Kafka flush log to disk (fsync) only depends on file system. Is there any simple way to monitor that frequency ? Is there a rule of thumb to estimate that value

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

Hi Aneel, Thank you for the KIP. I agree that managing complex configurations is not easy with the current tool. Having the possibility to use properties file sounds quite handy to me. It makes it easier to edit and to reuse base configurations. I wonder if we should also add a

Re: Subject: [VOTE] 2.4.1 RC0

+1 (non-binding). I built from source and ran the unit test suite successfully. Thanks for running this release. I'm looking forward to upgrading to 2.4.1. Sean On Mon, Mar 9, 2020 at 8:07 AM Mickael Maison wrote: > Thanks for running the release! > +1 (binding) > > - Verified signatures >

Build failed in Jenkins: kafka-2.2-jdk8-old #215

See Changes: [github] KAFKA-8245: Fix Flaky Test -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on

[jira] [Resolved] (KAFKA-8245) Flaky Test DeleteConsumerGroupsTest#testDeleteCmdAllGroups

[ https://issues.apache.org/jira/browse/KAFKA-8245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-8245. Resolution: Fixed > Flaky Test DeleteConsumerGroupsTest#testDeleteCmdAllGroups >

[Vote] KIP-569: Update DescribeConfigsResponse to include additional metadata information

Hi All, I would like to start a vote on KIP-569: Update DescribeConfigsResponse to include additional metadata information The KIP is here:

Re: Subject: [VOTE] 2.4.1 RC0

Thanks for running the release! +1 (binding) - Verified signatures - Built from source - Ran unit tests, all passed - Ran through quickstart steps, all worked On Mon, Mar 9, 2020 at 11:04 AM Tom Bentley wrote: > > +1 (non-binding) > > Built from source, all unit tests passed. > > Thanks Bill. >

[jira] [Created] (KAFKA-9686) MockConsumer#endOffsets should be idempotent

Chia-Ping Tsai created KAFKA-9686: - Summary: MockConsumer#endOffsets should be idempotent Key: KAFKA-9686 URL: https://issues.apache.org/jira/browse/KAFKA-9686 Project: Kafka Issue Type: Bug

Re: Subject: [VOTE] 2.4.1 RC0

+1 (non-binding) Built from source, all unit tests passed. Thanks Bill. On Mon, Mar 9, 2020 at 3:44 AM Gwen Shapira wrote: > +1 (binding) > > Verified signatures, built jars from source, quickstart passed and local > unit tests all passed. > > Thank you for the release Bill! > > Gwen Shapira

ConfigProvider notification

Colin asked that discussion about the future of the ConfigProvider notification mechanism happen on the list (rather than in https://issues.apache.org/jira/browse/KAFKA-9635 where I originally asked). The background that I've pieced together is that the subscribe family of methods were originally

[jira] [Resolved] (KAFKA-9684) Add support for SNI names in SSL request

[ https://issues.apache.org/jira/browse/KAFKA-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Badai Aqrandista resolved KAFKA-9684. - Resolution: Not A Problem Java 7 should include SNI names by default. > Add support for

[jira] [Created] (KAFKA-9685) Solve Set concatenation perf issue in AclAuthorizer

Jiao Zhang created KAFKA-9685: - Summary: Solve Set concatenation perf issue in AclAuthorizer Key: KAFKA-9685 URL: https://issues.apache.org/jira/browse/KAFKA-9685 Project: Kafka Issue Type: