Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2063

[Apache Jenkins Server]

See 

Re: [VOTE] KIP-759: Unneeded repartition canceling

[Sophie Blee-Goldman]

+1 (binding)

thanks Shay!

On Wed, Aug 2, 2023 at 1:19 AM Bruno Cadonna  wrote:

> Hi,
>
> +1 (binding)
>
> Thanks for the KIP!
>
> Best,
> Bruno
>
> On 8/2/23 1:19 AM, Bill Bejeck wrote:
> > I caught up on the discussion thread and the KIP LGTM.
> >
> > +1(binding)
> >
> > On Tue, Aug 1, 2023 at 3:07 PM Walker Carlson
> 
> > wrote:
> >
> >> +1 (binding)
> >>
> >> On Mon, Jul 31, 2023 at 10:43 PM Matthias J. Sax 
> wrote:
> >>
> >>> +1 (binding)
> >>>
> >>> On 7/11/23 11:16 AM, Shay Lin wrote:
>  Hi all,
> 
>  I'd like to call a vote on KIP-759: Unneeded repartition canceling
>  The KIP has been under discussion for quite some time(two years). This
> >>> is a
>  valuable optimization for advanced users. I hope we can push this
> >> toward
>  the finish line this time.
> 
>  Link to the KIP:
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-759%3A+Unneeded+repartition+canceling
> 
>  Best,
>  Shay
> 
> >>>
> >>
> >
>

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

[Guozhang Wang]

Thanks Florin for the writeup,

One quick thing I'd like to bring up is that in KIP-149
(https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner)
we introduced ValueJoinerWithKey which is aimed to enhance
ValueJoiner. It would have a benefit for this KIP such that
implementers can distinguish "null-key" v.s. "not-null-key but
null-value" scenarios.

Hence I'd suggest we also include the semantic changes with
ValueJoinerWithKey, which can help distinguish these two scenarios,
and also document that if users apply ValueJoiner only, they may not
have this benefit, and hence we suggest users to use the former.


Guozhang

On Mon, Jul 31, 2023 at 12:11 PM Florin Akermann
 wrote:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-962%3A+Relax+non-null+key+requirement+in+Kafka+Streams

[jira] [Resolved] (KAFKA-13562) Mirror Maker 2 Negative Offsets

[Greg Harris (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-13562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Harris resolved KAFKA-13562.
-
Fix Version/s: 3.3.3
   3.4.1
   3.5.0
   Resolution: Fixed

> Mirror Maker 2 Negative Offsets
> ---
>
> Key: KAFKA-13562
> URL: https://issues.apache.org/jira/browse/KAFKA-13562
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect, mirrormaker
>Affects Versions: 2.8.1
>Reporter: David Urton
>Priority: Minor
> Fix For: 3.3.3, 3.4.1, 3.5.0
>
>
> We recently started a migration from a traditional self-managed Apache Kafka 
> cluster to Strimzi. We enabled group offset sync and something I'm struggling 
> to fully understand is why the mirrored group has a current-offset matching 
> (or close to matching) the source offset, but then the log-end-offset 
> essentially starts over resulting in a negative lag. Is this just aesthetic? 
> The lag is slowly moving closer to 0 but some of our negative lags are very 
> "large". Several hundred million. I verified all the data has moved over, but 
> I'm struggling on this lag issue.
> Any help clearing this up is much appreciated!
> Source cluster is 2.4.1 destination is 2.8.1. Detailed info in the Github URL.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-8597) Give access to the Dead Letter Queue APIs to Kafka Connect Developers

[Greg Harris (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Harris resolved KAFKA-8597.

Fix Version/s: 2.6.0
   Resolution: Fixed

> Give access to the Dead Letter Queue APIs to Kafka Connect Developers
> -
>
> Key: KAFKA-8597
> URL: https://issues.apache.org/jira/browse/KAFKA-8597
> Project: Kafka
>  Issue Type: Improvement
>  Components: KafkaConnect
>Reporter: Andrea Santurbano
>Priority: Major
>  Labels: needs-kip
> Fix For: 2.6.0
>
>
> Would be cool to have the chance to have access to the DLQ APIs in order to 
> enable us (developers) to use that.
> For instance, if someone uses JSON as message format with no schema and it's 
> trying to import some data into a table, and the JSON contains a null value 
> for a NON-NULL table field, so we want to move that event to the DLQ.
> Thanks a lot!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-8754) Connect API: Expose the list of available transformations

[Greg Harris (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Harris resolved KAFKA-8754.

Fix Version/s: 3.2.0
   Resolution: Fixed

> Connect API: Expose the list of available transformations
> -
>
> Key: KAFKA-8754
> URL: https://issues.apache.org/jira/browse/KAFKA-8754
> Project: Kafka
>  Issue Type: Wish
>  Components: KafkaConnect
>Reporter: Stéphane Derosiaux
>Priority: Minor
> Fix For: 3.2.0
>
> Attachments: image-2019-08-05-22-45-01-527.png
>
>
> The API of Kafka Connect exposes the available connectors through: 
> /connector-plugins/.
> It would be useful to have another API method to expose the list of available 
> transformations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-15300) Include remotelog size in complete log size and also add local log size and remote log size separately in kafka-log-dirs tool.

[Satish Duggana (Jira)]

Satish Duggana created KAFKA-15300:
--

 Summary: Include remotelog size in complete log size and also add 
local log size and remote log size separately in kafka-log-dirs tool. 
 Key: KAFKA-15300
 URL: https://issues.apache.org/jira/browse/KAFKA-15300
 Project: Kafka
  Issue Type: Task
  Components: core
Reporter: Satish Duggana


Include remotelog size in complete log size and also add local log size and 
remote log size separately in kafka-log-dirs tool. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE] KIP-955: Add stream-table join on foreign key

[Igor Fomenko]

Hi All,

I would like to call a Vote on KIP-955: Add stream-table join on foreign
key.
KIP link is here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-955%3A+Add+stream-table+join+on+foreign+key

Mostly I would like to get consensus on the need for this new API to Kafka
Streams. The actual design of course will still be open to discussions
until this API is implemented.

Thank you,

Igor

Re: [VOTE] KIP-949: Add flag to enable the usage of topic separator in MM2 DefaultReplicationPolicy

[Omnia Ibrahim]

Thanks, Mickael.
Am closing the voting now with 3 binding votes (from Chris, Greg and
Mickale) and 1 non-binding (from Federico).

On Wed, Aug 2, 2023 at 2:52 PM Mickael Maison 
wrote:

> +1 (binding)
>
> Thanks for the KIP
>
> On Tue, Aug 1, 2023 at 1:26 PM Omnia Ibrahim 
> wrote:
> >
> > Thanks for the binding vote, Greg, We now need one extra binding vote to
> > get this KIP accepted.
> >
> > On Tue, Jul 25, 2023 at 8:10 PM Greg Harris  >
> > wrote:
> >
> > > Hey Omnia,
> > >
> > > Thanks for the KIP!
> > >
> > > I think that MM2 is responsible for providing an upgrade path for
> > > users, even if it isn't backwards-compatible by default due to a
> > > mistake.
> > > The non-configuration-based strategies I could think of aren't viable
> > > due to the danger of inferring the incorrect topic name, and inherent
> > > complexity which makes them hard to backport.
> > > I also support the decision to backport this to 3.1 - 3.5, so that MM2
> > > users can upgrade in minor version increments after those patch
> > > releases go out.
> > >
> > > I'm +1 (binding).
> > >
> > > Thanks,
> > > Greg
> > >
> > > On Mon, Jul 24, 2023 at 7:21 AM Omnia Ibrahim  >
> > > wrote:
> > > >
> > > > Hi Chris, I updated the KIP to address your feedback. Thanks for the
> > > vote.
> > > >
> > > > On Mon, Jul 24, 2023 at 1:30 PM Chris Egerton
> 
> > > > wrote:
> > > >
> > > > > Hi Omnia,
> > > > >
> > > > > I think there's a few clarifications that should still be made on
> the
> > > KIP,
> > > > > but assuming these are agreeable, I'm +1 (binding)
> > > > >
> > > > > - In the description for the
> > > > > replication.policy.internal.topic.separator.enabled property (in
> the
> > > > > "Public Interfaces" section), we should specify that it affects
> only
> > > the
> > > > > checkpoints and offset syncs topic
> > > > > - We can remove the code snippet from the "Proposed Changes"
> section
> > > (right
> > > > > now it's a little buggy; there's two different implementations for
> the
> > > same
> > > > > "internalSuffix" method, and there are references to an
> > > "internalSeparator"
> > > > > method but no implementation for it); since we don't usually
> require
> > > > > specific code changes in KIPs, I think as long as we can describe
> the
> > > > > changes we're proposing in the "Public Interfaces" section, that
> > > should be
> > > > > enough for this KIP
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Chris
> > > > >
> > > > > On Mon, Jul 24, 2023 at 2:04 AM Federico Valeri <
> fedeval...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1 (non binding)
> > > > > >
> > > > > > Thanks
> > > > > > Fede
> > > > > >
> > > > > >
> > > > > > On Sun, Jul 23, 2023 at 6:30 PM Omnia Ibrahim <
> > > o.g.h.ibra...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi everyone,
> > > > > > > I would like to open a vote for KIP-949. The proposal is here
> > > > > > >
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy
> > > > > > > .
> > > > > > > <
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy
> > > > > > >
> > > > > > >
> > > > > > > Thanks
> > > > > > > Omnia
> > > > > >
> > > > >
> > >
>

[jira] [Created] (KAFKA-15299) Support left stream-table join on foreign key

[Igor Fomenko (Jira)]

Igor Fomenko created KAFKA-15299:


 Summary: Support left stream-table join on foreign key
 Key: KAFKA-15299
 URL: https://issues.apache.org/jira/browse/KAFKA-15299
 Project: Kafka
  Issue Type: New Feature
  Components: streams
Reporter: Igor Fomenko
Assignee: Igor Fomenko


KIP-955: 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-955%3A+Add+stream-table+join+on+foreign+key

Currently in Kafka Streams DSL, KStream to KTable joins could only be performed 
with the keys. However in practice it is often required to join the messages in 
Kafka topics using message field as a "foreign key" with the following pattern: 
 

 
streamX.leftJoin(tableY, RecordTableY::getForegnKey, 
joiner).to("output-topic-name")
 
The left loin on foreign key operation will result in a stream of messages from 
two topics joined on foreign key where each output message is produced for each 
event on the input stream.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] KIP-949: Add flag to enable the usage of topic separator in MM2 DefaultReplicationPolicy

[Mickael Maison]

+1 (binding)

Thanks for the KIP

On Tue, Aug 1, 2023 at 1:26 PM Omnia Ibrahim  wrote:
>
> Thanks for the binding vote, Greg, We now need one extra binding vote to
> get this KIP accepted.
>
> On Tue, Jul 25, 2023 at 8:10 PM Greg Harris 
> wrote:
>
> > Hey Omnia,
> >
> > Thanks for the KIP!
> >
> > I think that MM2 is responsible for providing an upgrade path for
> > users, even if it isn't backwards-compatible by default due to a
> > mistake.
> > The non-configuration-based strategies I could think of aren't viable
> > due to the danger of inferring the incorrect topic name, and inherent
> > complexity which makes them hard to backport.
> > I also support the decision to backport this to 3.1 - 3.5, so that MM2
> > users can upgrade in minor version increments after those patch
> > releases go out.
> >
> > I'm +1 (binding).
> >
> > Thanks,
> > Greg
> >
> > On Mon, Jul 24, 2023 at 7:21 AM Omnia Ibrahim 
> > wrote:
> > >
> > > Hi Chris, I updated the KIP to address your feedback. Thanks for the
> > vote.
> > >
> > > On Mon, Jul 24, 2023 at 1:30 PM Chris Egerton 
> > > wrote:
> > >
> > > > Hi Omnia,
> > > >
> > > > I think there's a few clarifications that should still be made on the
> > KIP,
> > > > but assuming these are agreeable, I'm +1 (binding)
> > > >
> > > > - In the description for the
> > > > replication.policy.internal.topic.separator.enabled property (in the
> > > > "Public Interfaces" section), we should specify that it affects only
> > the
> > > > checkpoints and offset syncs topic
> > > > - We can remove the code snippet from the "Proposed Changes" section
> > (right
> > > > now it's a little buggy; there's two different implementations for the
> > same
> > > > "internalSuffix" method, and there are references to an
> > "internalSeparator"
> > > > method but no implementation for it); since we don't usually require
> > > > specific code changes in KIPs, I think as long as we can describe the
> > > > changes we're proposing in the "Public Interfaces" section, that
> > should be
> > > > enough for this KIP
> > > >
> > > > Cheers,
> > > >
> > > > Chris
> > > >
> > > > On Mon, Jul 24, 2023 at 2:04 AM Federico Valeri 
> > > > wrote:
> > > >
> > > > > +1 (non binding)
> > > > >
> > > > > Thanks
> > > > > Fede
> > > > >
> > > > >
> > > > > On Sun, Jul 23, 2023 at 6:30 PM Omnia Ibrahim <
> > o.g.h.ibra...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Hi everyone,
> > > > > > I would like to open a vote for KIP-949. The proposal is here
> > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy
> > > > > > .
> > > > > > <
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy
> > > > > >
> > > > > >
> > > > > > Thanks
> > > > > > Omnia
> > > > >
> > > >
> >

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2062

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-15298) Disable DeleteRecords on Tiered Storage topics

[Christo Lolov (Jira)]

Christo Lolov created KAFKA-15298:
-

 Summary: Disable DeleteRecords on Tiered Storage topics
 Key: KAFKA-15298
 URL: https://issues.apache.org/jira/browse/KAFKA-15298
 Project: Kafka
  Issue Type: Sub-task
Reporter: Christo Lolov
Assignee: Christo Lolov


Currently the DeleteRecords API does not work with Tiered Storage. We should 
ensure that this is reflected in the responses that clients get when trying to 
use the API with tiered topics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-15297) Cache flush order might not be topological order

[Bruno Cadonna (Jira)]

Bruno Cadonna created KAFKA-15297:
-

 Summary: Cache flush order might not be topological order 
 Key: KAFKA-15297
 URL: https://issues.apache.org/jira/browse/KAFKA-15297
 Project: Kafka
  Issue Type: Bug
  Components: streams
Affects Versions: 3.4.0
Reporter: Bruno Cadonna
 Attachments: minimal_example.png

The flush order of the state store caches in Kafka Streams might not correspond 
to the topological order of the state stores in the topology. The order depends 
on how the processors and state stores are added to the topology. 
In some cases downstream state stores might be flushed before upstream state 
stores. That means, that during a commit records in upstream caches might end 
up in downstream caches that have already been flushed during the same commit. 
If a crash happens at that point, those records in the downstream caches are 
lost. Those records are lost for two reasons:

1. Records in caches are only changelogged after they are flushed from the 
cache. However, the downstream caches have already been flushed and they will 
not be flushed again during the same commit.

2. The offsets of the input records that caused the records that now are 
blocked in the downstream caches are committed during the same commit and so 
they will not be re-processed after the crash.

An example for a topology where the flush order of the caches is wrong is the 
following:




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE] KIP-940: Broker extension point for validating record contents at produce time

[Adrian Preston]

Hello all,



Edo and I would like to call for a vote on KIP-940:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-940%3A+Broker+extension+point+for+validating+record+contents+at+produce+time



Thanks,

Adrian

Unless otherwise stated above:

IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

[jira] [Created] (KAFKA-15296) Allow committing offsets for Dropped records via SMTs

[Sagar Rao (Jira)]

Sagar Rao created KAFKA-15296:
-

 Summary: Allow committing offsets for Dropped records via SMTs
 Key: KAFKA-15296
 URL: https://issues.apache.org/jira/browse/KAFKA-15296
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Reporter: Sagar Rao
Assignee: Sagar Rao


Currently the connect Runtime doesn't commit the offsets of records which have 
been dropped due to SMT. This can lead to issues if the dropped record's 
partition reflects a source partition and the connector depends upon the 
committed offsets to make progress. In such cases, the connector might just 
stall. We should enable committing offsets for dropped records as well. Note 
that today if a record is dropped because exactly-once support is enabled and 
the connector chose to abort the batch containing the record, then its offset 
is still committed. So there already exists a discrepancy in the way the 
runtime treats these dropped records.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

[Sagar]

Attaching the KIP link for reference:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records

Thanks!
Sagar.

On Wed, Aug 2, 2023 at 4:37 PM Sagar  wrote:

> Hi All,
>
> Calling a Vote on KIP-910 [1]. I feel we have converged to a reasonable
> design. Ofcourse I am open to any feedback/suggestions and would address
> them.
>
> Thanks!
> Sagar.
>

[VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

[Sagar]

Hi All,

Calling a Vote on KIP-910 [1]. I feel we have converged to a reasonable
design. Ofcourse I am open to any feedback/suggestions and would address
them.

Thanks!
Sagar.

Re: Debugging Jenkins test failures

[Divij Vaidya]

No. We started using gradle analysis starting July 12th. Prior to
that, the only data that we have is coming from Apache CI which AFAIK
doesn't have a per-test history view -
https://ci-builds.apache.org/job/Kafka/job/kafka/job/trunk/

--
Divij Vaidya



On Wed, Aug 2, 2023 at 1:04 AM Kirk True  wrote:
>
> Hi Divij,
>
> Thanks for the pointer to Gradle Enterprise! That’s exactly what I was 
> looking for.
>
> Did we track builds before July 12? I see only tiny blips of failures on the 
> 90-day view.
>
> Thanks,
> Kirk
>
> > On Jul 26, 2023, at 2:08 AM, Divij Vaidya  wrote:
> >
> > Hi Kirk
> >
> > I have been using this new tool to analyze the trends of test
> > failures: 
> > https://ge.apache.org/scans/tests?search.relativeStartTime=P28D=kafka=Europe/Berlin
> > and general build failures:
> > https://ge.apache.org/scans/failures?search.relativeStartTime=P28D=kafka=Europe/Berlin
> >
> > About the classes of build failure, if we look at the last 28 days, I
> > do not observe an increasing trend. The top causes of failure are:
> > (link [2])
> > 1. Failures due to checkstyle (193 builds)
> > 2. Timeout waiting to lock cache. It is currently in-use by another
> > Gradle instance.
> > 3. Compilation failures (116 builds)
> > 4. "Gradle Test Executor" finished with a non-zero exit value. Process
> > 'Gradle Test Executor 180' finished with non-zero exit value 1
> >
> > #4 is caused by a test failure that causes a crash of the Gradle
> > process. To debug this, I usually go to complete test output and try
> > to figure out which was the last test that 'Gradle Test Executor 180'
> > was running. As an example, consider
> > https://ge.apache.org/s/luizhogirob4e. We observe that this fails for
> > PR-14094. Now, we need to see the complete system out. To find that, I
> > will go to Kafka PR builder at
> > https://ci-builds.apache.org/job/Kafka/job/kafka-pr/view/change-requests/
> > and find the build page for PR-14094. That page is
> > https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14094/.
> > Next, find last failed build at
> > https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14094/lastFailedBuild/
> > , observe that we have a failure for "Gradle Test Executor 177", click
> > on view as plain text (it takes a long time to load), find what the
> > GradleTest Executor was doing. In this case, it failed with the
> > following error. I strongly believe that it is due to
> > https://github.com/apache/kafka/pull/13572 but unfortunately, this was
> > reverted and never fixed after that. Perhaps you might want to re
> >
> > Gradle Test Run :core:integrationTest > Gradle Test Executor 177 >
> > ProducerFailureHandlingTest > testTooLargeRecordWithAckZero() STARTED
> >
> >> Task :clients:integrationTest FAILED
> > org.gradle.internal.remote.internal.ConnectException: Could not
> > connect to server [bd7b0504-7491-43f8-a716-513adb302c92 port:43321,
> > addresses:[/127.0.0.1]]. Tried addresses: [/127.0.0.1].
> > at 
> > org.gradle.internal.remote.internal.inet.TcpOutgoingConnector.connect(TcpOutgoingConnector.java:67)
> > at 
> > org.gradle.internal.remote.internal.hub.MessageHubBackedClient.getConnection(MessageHubBackedClient.java:36)
> > at 
> > org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:103)
> > at 
> > org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
> > at 
> > worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
> > at 
> > worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
> > Caused by: java.net.ConnectException: Connection refused
> > at java.base/sun.nio.ch.Net.pollConnect(Native Method)
> > at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
> > at 
> > java.base/sun.nio.ch.SocketChannelImpl.finishTimedConnect(SocketChannelImpl.java:1141)
> > at 
> > java.base/sun.nio.ch.SocketChannelImpl.blockingConnect(SocketChannelImpl.java:1183)
> > at java.base/sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:98)
> > at 
> > org.gradle.internal.remote.internal.inet.TcpOutgoingConnector.tryConnect(TcpOutgoingConnector.java:81)
> > at 
> > org.gradle.internal.remote.internal.inet.TcpOutgoingConnector.connect(TcpOutgoingConnector.java:54)
> > ... 5 more
> >
> >
> >
> >
> > About the classes of test failure problems, if we look at the last 28
> > days, the following tests are the biggest culprits. If we fix just
> > these two, our CI would be in a much better shape. (link [1])
> > 1. https://issues.apache.org/jira/browse/KAFKA-15197 (this test passes
> > only 53% of the time)
> > 2. https://issues.apache.org/jira/browse/KAFKA-15052 (this test passes
> > only 49% of the time)
> >
> >
> > [1] 
> > https://ge.apache.org/scans/tests?search.relativeStartTime=P28D=kafka=Europe/Berlin
> > [2] 
> > 

[jira] [Created] (KAFKA-15295) Add config validation when remote storage is enabled on a topic

[Kamal Chandraprakash (Jira)]

Kamal Chandraprakash created KAFKA-15295:


 Summary: Add config validation when remote storage is enabled on a 
topic
 Key: KAFKA-15295
 URL: https://issues.apache.org/jira/browse/KAFKA-15295
 Project: Kafka
  Issue Type: Task
Reporter: Kamal Chandraprakash
Assignee: Kamal Chandraprakash


If system level remote storage is not enabled, then enabling remote storage on 
a topic should throw exception while validating the configs. 

See https://github.com/apache/kafka/pull/14114#discussion_r1280372441 for more 
details



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2061

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 395709 lines...]
Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerInner[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerOuter[caching enabled = false] STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerOuter[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerLeft[caching enabled = false] STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerLeft[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testOuterInner[caching enabled = false] STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testOuterInner[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testOuterOuter[caching enabled = false] STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testOuterOuter[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerWithRightVersionedOnly[caching enabled = false] STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerWithRightVersionedOnly[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testLeftWithLeftVersionedOnly[caching enabled = false] STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testLeftWithLeftVersionedOnly[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerWithLeftVersionedOnly[caching enabled = false] STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TableTableJoinIntegrationTest > [caching enabled = false] > 
testInnerWithLeftVersionedOnly[caching enabled = false] PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TaskAssignorIntegrationTest > shouldProperlyConfigureTheAssignor STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TaskAssignorIntegrationTest > shouldProperlyConfigureTheAssignor PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TaskMetadataIntegrationTest > shouldReportCorrectEndOffsetInformation STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TaskMetadataIntegrationTest > shouldReportCorrectEndOffsetInformation PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TaskMetadataIntegrationTest > shouldReportCorrectCommittedOffsetInformation 
STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
TaskMetadataIntegrationTest > shouldReportCorrectCommittedOffsetInformation 
PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
EmitOnChangeIntegrationTest > shouldEmitSameRecordAfterFailover() STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
EmitOnChangeIntegrationTest > shouldEmitSameRecordAfterFailover() PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
HighAvailabilityTaskAssignorIntegrationTest > 
shouldScaleOutWithWarmupTasksAndPersistentStores(TestInfo) STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
HighAvailabilityTaskAssignorIntegrationTest > 
shouldScaleOutWithWarmupTasksAndPersistentStores(TestInfo) PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
HighAvailabilityTaskAssignorIntegrationTest > 
shouldScaleOutWithWarmupTasksAndInMemoryStores(TestInfo) STARTED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 
HighAvailabilityTaskAssignorIntegrationTest > 
shouldScaleOutWithWarmupTasksAndInMemoryStores(TestInfo) PASSED

Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > 

Re: [VOTE] KIP-759: Unneeded repartition canceling

[Bruno Cadonna]

Hi,

+1 (binding)

Thanks for the KIP!

Best,
Bruno

On 8/2/23 1:19 AM, Bill Bejeck wrote:

I caught up on the discussion thread and the KIP LGTM.

+1(binding)

On Tue, Aug 1, 2023 at 3:07 PM Walker Carlson 
wrote:


+1 (binding)

On Mon, Jul 31, 2023 at 10:43 PM Matthias J. Sax  wrote:


+1 (binding)

On 7/11/23 11:16 AM, Shay Lin wrote:

Hi all,

I'd like to call a vote on KIP-759: Unneeded repartition canceling
The KIP has been under discussion for quite some time(two years). This

is a

valuable optimization for advanced users. I hope we can push this

toward

the finish line this time.

Link to the KIP:




https://cwiki.apache.org/confluence/display/KAFKA/KIP-759%3A+Unneeded+repartition+canceling


Best,
Shay

[jira] [Created] (KAFKA-15294) Make remote storage related configs as public (i.e. non-internal)

[Luke Chen (Jira)]

Luke Chen created KAFKA-15294:
-

 Summary: Make remote storage related configs as public (i.e. 
non-internal)
 Key: KAFKA-15294
 URL: https://issues.apache.org/jira/browse/KAFKA-15294
 Project: Kafka
  Issue Type: Sub-task
Reporter: Luke Chen
 Fix For: 3.6.0


We should publish all the remote storage related configs in v3.6.0. It can be 
verified by:

 
{code:java}
./gradlew releaseTarGz

# The build output is stored in 
./core/build/distributions/kafka_2.13-3.x.x-site-docs.tgz. Untar the file 
verify it{code}
{{}}

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-15293) Update metrics doc to add tiered storage metrics

[Abhijeet Kumar (Jira)]

Abhijeet Kumar created KAFKA-15293:
--

 Summary: Update metrics doc to add tiered storage metrics
 Key: KAFKA-15293
 URL: https://issues.apache.org/jira/browse/KAFKA-15293
 Project: Kafka
  Issue Type: Sub-task
Reporter: Abhijeet Kumar






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [kafka-site] netstratum-labs opened a new pull request, #534: Netstratum info updated

[via GitHub]

netstratum-labs opened a new pull request, #534:
URL: https://github.com/apache/kafka-site/pull/534

   Added Netstratum information and how we use Kafka in our products.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org