Re: Is there an equivalent of a "synchronous subscribe"?

Yes, there is no such "synchronous subscribe" method in Kafka. Consumer group will start to work after `poll` is called. Inside Kafka, there are util methods like `awaitAssignment` or `awaitRebalance` to wait for the initialization completed for tests, which is checking with `poll` You can refer

Re: Is there an equivalent of a "synchronous subscribe"?

Faced with the same problem, we've done the following: - Find and save the latest offset(s) using consumer.endOffsets() - consumer.poll() and process records in a while loop until you've processed up through the saved offsets Notice that it doesn't matter how long the poll duration is: you've

Re: [ANNOUNCE] Apache Kafka 3.1.0

Must spark3, kafka3, scala3, python3 work together if my project used these stacks? Thanks On Tue, Jan 25, 2022 at 1:04 AM David Jacot wrote: > The Apache Kafka community is pleased to announce the release for > Apache Kafka 3.1.0. > > It is a major release that includes many new features,

Is there an equivalent of a "synchronous subscribe"?

Hey all! I am actually not sure whether this is a bug report or a feature request, maybe you can help me understand whether I am missing the obvious. I am using Apache Kafka in one of my Java projects (using apache-kafka-clients-3.1.0). In an integration test, I would like to check that (after

Re: [ANNOUNCE] Apache Kafka 3.1.0

Thanks David for driving the release! @Deepti, no, this release doesn't include the log4j2 upgrade feature. Luke On Tue, Jan 25, 2022 at 11:43 AM Gwen Shapira wrote: > Exciting! Thanks for driving the release, David. > > On Mon, Jan 24, 2022 at 9:04 AM David Jacot wrote: > > > > The Apache

Re: [ANNOUNCE] Apache Kafka 3.1.0

Exciting! Thanks for driving the release, David. On Mon, Jan 24, 2022 at 9:04 AM David Jacot wrote: > > The Apache Kafka community is pleased to announce the release for > Apache Kafka 3.1.0. > > It is a major release that includes many new features, including: > > * Apache Kafka supports Java

Repartition Stateful Kafka Streams App

I am looking for documentation covering the process and/or gotchas of repartitioning (increasing the number of partitions) of an input topic used in a stateful (using the Processor API) Kafka Streams app. I haven't found a concise document saying what the process & gotchas are, but AFAICT

Re: Log4j 1.2

In general you can delete log4j1.jar Replace with log4jcore_2.17.1.jar And log4japi_2.17.1.jar Ed On Monday, January 24, 2022, Men Lim wrote: > Is there a write out of the steps that need to be taken? > > On Mon, Jan 24, 2022 at 10:36 AM Edward Capriolo > wrote: > > > Explained in another

Re: Log4j 1.2

Is there a write out of the steps that need to be taken? On Mon, Jan 24, 2022 at 10:36 AM Edward Capriolo wrote: > Explained in another thread log4j api is separate from implementation. Its > possible to remove log4j 1.2 jars from classpath and upgrade to log4j > 2.17.1 without changing a line

Re: [ANNOUNCE] Apache Kafka 3.1.0

Thanks David for driving this release! On Mon, Jan 24, 2022 at 6:07 PM Deepti Sharma S wrote: > > Congratulations! > > One thing would like to confirm, if we have upgraded the Log4J version from > 1.x to 2.x in this release? > > > Regards, > Deepti Sharma > PMP® & ITIL > > > -Original

Re: Log4j 1.2

Explained in another thread log4j api is separate from implementation. Its possible to remove log4j 1.2 jars from classpath and upgrade to log4j 2.17.1 without changing a line of code in kafka. On Monday, January 10, 2022, Tauzell, Dave wrote: > Thanks. Those KIPs show that there is a fair

[ANNOUNCE] Apache Kafka 3.1.0

The Apache Kafka community is pleased to announce the release for Apache Kafka 3.1.0. It is a major release that includes many new features, including: * Apache Kafka supports Java 17 * The FetchRequest supports Topic IDs (KIP-516) * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768) * Add

RE: [ANNOUNCE] Apache Kafka 3.1.0

Congratulations! One thing would like to confirm, if we have upgraded the Log4J version from 1.x to 2.x in this release? Regards, Deepti Sharma PMP® & ITIL -Original Message- From: David Jacot Sent: Monday, January 24, 2022 10:34 PM To: annou...@apache.org; d...@kafka.apache.org;

Re: Discuss: outage impact of not updating kafka clients to a version that has KAFKA-9893

On Sun, Jan 23, 2022 at 7:51 AM Edward Capriolo wrote: > Hello, > The bulk of this thread is to discuss > [KAFKA-9893] Configurable TCP connection timeout and improve the initial > metadata fetch - ASF JIRA (apache.org) > . IMHO t it should be >