Build failed in Jenkins: kafka-2.3-jdk8 #148

See Changes: [jason] KAFKA-9251; Describing a non consumer group with the Admin API hangs -- [...truncated 2.93 MB...]

Build failed in Jenkins: kafka-2.4-jdk8 #96

See Changes: [rhauch] KAFKA-9184: Redundant task creation and periodic rebalances after zombie -- [...truncated 2.70 MB...]

[jira] [Created] (KAFKA-9270) KafkaStream crash on offset commit failure

Rohan Kulkarni created KAFKA-9270: - Summary: KafkaStream crash on offset commit failure Key: KAFKA-9270 URL: https://issues.apache.org/jira/browse/KAFKA-9270 Project: Kafka Issue Type: Bug

Re: [VOTE] 2.4.0 RC2

Hi all, We merged all the blockers and released a new release candidate RC3. Hopefully this will be a final RC for 2.4 release. Thanks, On Thu, Dec 5, 2019 at 6:56 AM Israel Ekpo wrote: > Thanks @Randall for the update. I will run the tests on RC3 when it becomes > available. > > On Wed,

[VOTE] 2.4.0 RC3

Hello Kafka users, developers and client-developers, This is the fourth candidate for release of Apache Kafka 2.4.0. This release includes many new features, including: - Allow consumers to fetch from closest replica - Support for incremental cooperative rebalancing to the consumer rebalance

Re: [DISCUSS] KIP-501 Avoid out-of-sync or offline partitions when follower fetch requests not processed in time

Hi Jason, As Satish said just increase replica max lag will not work in this case. Just before a disk dies the reads becomes really slow and its hard to estimate how much this is, as we noticed range is pretty wide. Overall it doesn't make sense to knock good replicas out of just because

Re: Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

Hi Team, We are also facing slowness in producing messages for some of topics. Kafka acknowledgement is taking more than 1 seconds sometimes. Normally it takes 1 to 2 milliseconds. Do you think, this exception leads to slow acknowledge? Your help is much appreciated, we are facing this issue in

Build failed in Jenkins: kafka-trunk-jdk11 #1002

See Changes: [jason] KAFKA-9190; Close connections with expired authentication sessions [cmccabe] MINOR: Adds entity-specific flags to ConfigCommand per KIP-543. (#7667)

[jira] [Resolved] (KAFKA-9123) Add system test with large number of partitions

[ https://issues.apache.org/jira/browse/KAFKA-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-9123. -- Resolution: Fixed > Add system test with large number of partitions >

Re: Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

I once encountered the same problem when I installed 2 versions of Kafka with the same config, and startup version A Kafka firstly and shutdown it, and misoperated to start version B Kafka secondly and write some data to it and shutdown it, and then startup version A Kafka. I think it's OK to

Build failed in Jenkins: kafka-2.4-jdk8 #95

See Changes: [jason] KAFKA-9190; Close connections with expired authentication sessions [bill] MINOR: Convert Stream-StreamJoin Integration Test to TTD (#7752) [jason] KAFKA-9251; Describing a non consumer group

[jira] [Resolved] (KAFKA-9184) Redundant task creation and periodic rebalances after zombie worker rejoins the group

[ https://issues.apache.org/jira/browse/KAFKA-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-9184. -- Reviewer: Randall Hauch Resolution: Fixed > Redundant task creation and periodic

[jenkins problem]

I always receive these kinds of messages about jenkins failed, and most prs of Kafka build failed even just a trivial change，can someone inspects it. [image: image.png] [image: image.png]

[jira] [Created] (KAFKA-9269) Warn if security.protocol does not match security configuration

Richard Wise created KAFKA-9269: --- Summary: Warn if security.protocol does not match security configuration Key: KAFKA-9269 URL: https://issues.apache.org/jira/browse/KAFKA-9269 Project: Kafka

Re: [VOTE] 2.4.0 RC2

Thanks @Randall for the update. I will run the tests on RC3 when it becomes available. On Wed, Dec 4, 2019 at 12:42 PM Randall Hauch wrote: > Ah, I merged https://issues.apache.org/jira/browse/KAFKA-9258 last night > and forgot to send an email to this thread. > > Thanks! > > Randall > > On

Re: [VOTE] KIP-545 support automated consumer offset sync across clusters in MM 2.0

Bump. We've got 2 non-binding votes so far. On Wed, Nov 13, 2019 at 3:32 PM Ning Zhang wrote: > My current plan is to implement this in "MirrorCheckpointTask" > > On 2019/11/02 03:30:11, Xu Jianhai wrote: > > I think this kip will implement a task in sinkTask ? right? > > > > On Sat, Nov 2,

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

Hi Stanislav & Viktor, Kafka has historically delegated partition reassignment to external tools. For example, the choice of which partitions to reassign is determined by an external tool. The choice of what throttle to use is also set externally. So I think that putting this into the

Build failed in Jenkins: kafka-2.2-jdk8 #10

See Changes: [jason] KAFKA-9190; Close connections with expired authentication sessions [jason] HOTFIX: Compile error in SocketServerTest -- [...truncated 2.73 MB...]

Build failed in Jenkins: kafka-2.4-jdk8 #94

See Changes: [ismael] KAFKA-9265: Fix kafka.log.Log instance leak on log deletion (#7773) -- [...truncated 5.46 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi, Harsha, Thanks for the reply. 40/41. I am curious which block storages you have tested. S3 seems to be one of the popular block stores. The concerns that I have with pull based approach are the following. (a) Cost: S3 list object requests cost $0.005 per 1000 requests. If you have 100,000

Re: SSLAuthentication Exception JMX

Hi Tyler, One way to find the metrics exported by Kafka is to use a tool like jconsole. best, Colin On Wed, Dec 4, 2019, at 13:30, Tyler Lubeck wrote: > Wow, thank you. That covers everything I need. If I can take up a bit more > of your time, can you tell me how you found that? > > Thanks, >

[jira] [Resolved] (KAFKA-9251) Describing a non consumer group with the Admin API hangs forever

[ https://issues.apache.org/jira/browse/KAFKA-9251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9251. Resolution: Fixed > Describing a non consumer group with the Admin API hangs forever >

Re: SSLAuthentication Exception JMX

Wow, thank you. That covers everything I need. If I can take up a bit more of your time, can you tell me how you found that? Thanks, Tyler Lubeck (813) 469 - 1499 www.TylerLubeck.com www.linkedin.com/in/tylerlubeck/ On Tue, Dec 3, 2019 at 3:57 AM Rajini Sivaram

Build failed in Jenkins: kafka-trunk-jdk11 #1001

See Changes: [ismael] KAFKA-9265: Fix kafka.log.Log instance leak on log deletion (#7773) -- [...truncated 5.58 MB...] org.apache.kafka.streams.test.OutputVerifierTest

Jenkins build is back to normal : kafka-trunk-jdk8 #4083

See

Re: Broker Interceptors

Hi Thomas, Thanks for your interest in KIP-388. As Ignacio and Radai have mentioned, this is our (LinkedIn's) implementation of KIP-388. The implementation and deployment of this broker-side observer has been

Build failed in Jenkins: kafka-2.2-jdk8-old #191

See Changes: [jason] KAFKA-9190; Close connections with expired authentication sessions [jason] HOTFIX: Compile error in SocketServerTest -- Started by an SCM change

[jira] [Resolved] (KAFKA-9190) Server leaves connections with expired authentication sessions open

[ https://issues.apache.org/jira/browse/KAFKA-9190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9190. Fix Version/s: 2.3.2 2.2.3 2.4.0 Resolution:

Re: [VOTE] 2.4.0 RC2

Ah, I merged https://issues.apache.org/jira/browse/KAFKA-9258 last night and forgot to send an email to this thread. Thanks! Randall On Wed, Dec 4, 2019 at 10:42 AM Eric Lalonde wrote: > > I have also shared a pull request for the changes I have made to the test > > script > > > >

Re: [VOTE] 2.4.0 RC2

> I have also shared a pull request for the changes I have made to the test > script > > https://github.com/elalonde/kafka/pull/2 Thanks, this has been merged.

[jira] [Created] (KAFKA-9268) Follow-on: Streams Threads may die from recoverable errors with EOS enabled

John Roesler created KAFKA-9268: --- Summary: Follow-on: Streams Threads may die from recoverable errors with EOS enabled Key: KAFKA-9268 URL: https://issues.apache.org/jira/browse/KAFKA-9268 Project:

RE: Broker Interceptors

Thanks for the responses. I did worry about the challenge of exposing a vast number of internal classes with general interceptor framework. A less general solution more along the lines of the producer/consumer interceptors on the client would satisfy the majority of use cases. If we are smart,

[jira] [Created] (KAFKA-9267) ZkSecurityMigrator should not create /controller node

NanerLee created KAFKA-9267: --- Summary: ZkSecurityMigrator should not create /controller node Key: KAFKA-9267 URL: https://issues.apache.org/jira/browse/KAFKA-9267 Project: Kafka Issue Type: Bug

Fwd: Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

-- Forwarded message - From: Upendra Yadav Date: Tue, Dec 3, 2019 at 5:28 PM Subject: Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

Reporting every existing SSLContext's certificate expiration dates?

Hi all, recently we ran into a bug which was supposedly fixed in https://issues.apache.org/jira/browse/KAFKA-8190 (we were running Kafka 2.1.1). We use SSL for interbroker communication as well as for communication with clients. The new certificate was rendered under the same filename, which

Jenkins build is back to normal : kafka-2.3-jdk8 #145

See

[jira] [Created] (KAFKA-9266) KafkaConsumer manual assignment does not reset group assignment

G G created KAFKA-9266: -- Summary: KafkaConsumer manual assignment does not reset group assignment Key: KAFKA-9266 URL: https://issues.apache.org/jira/browse/KAFKA-9266 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

Hey all, I spoke offline with Viktor regarding this KIP and intentions on starting a vote soon, so I made another pass on it. I have some comments: >From our past discussion: > I reiterated on the current algorithm and indeed it would change the order of replicas in ZK but wouldn't do a leader

[jira] [Resolved] (KAFKA-9265) kafka.log.Log instances are leaking on log delete

[ https://issues.apache.org/jira/browse/KAFKA-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-9265. Fix Version/s: 2.3.2 2.4.0 Resolution: Fixed > kafka.log.Log instances

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

Thanks Viktor. How do we intend to handle the case where a broker loses its disk and has to catch up from the beginning? Ismael On Wed, Dec 4, 2019, 4:31 AM Viktor Somogyi-Vass wrote: > Thanks for the notice Ismael, KAFKA-4313 fixed this issue indeed. I've > updated the KIP. > > Viktor > > On

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

Thanks for the notice Ismael, KAFKA-4313 fixed this issue indeed. I've updated the KIP. Viktor On Tue, Dec 3, 2019 at 3:28 PM Ismael Juma wrote: > Hi Viktor, > > The KIP states: > > "KIP-73 > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas > > > added > quotas

Jenkins build is back to normal : kafka-trunk-jdk11 #1000

See

Build failed in Jenkins: kafka-trunk-jdk8 #4082

See Changes: [john] KAFKA-9231: Streams Threads may die from recoverable errors with EOS -- [...truncated 8.32 MB...]

Build failed in Jenkins: kafka-2.3-jdk8 #144

See Changes: [github] KAFKA-9073: check assignment in requestFailed to avoid NPE (#7630) -- [...truncated 2.96 MB...] kafka.controller.ControllerIntegrationTest >