[jira] [Resolved] (KAFKA-14827) Support for StandardAuthorizer in Benchmark

[Purshotam Chauhan (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Purshotam Chauhan resolved KAFKA-14827.
---
Fix Version/s: 3.5.0
 Reviewer: Manikumar Reddy
   Resolution: Fixed

> Support for StandardAuthorizer in Benchmark
> ---
>
> Key: KAFKA-14827
> URL: https://issues.apache.org/jira/browse/KAFKA-14827
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Purshotam Chauhan
>Assignee: Purshotam Chauhan
>Priority: Major
> Fix For: 3.5.0
>
>
> Support for StandardAuthorizer in Benchmark



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.4 #113

[Apache Jenkins Server]

See 

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.1 #145

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 368564 lines...]
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[2023-04-21T02:51:07.580Z] 
[2023-04-21T02:51:07.580Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldKStreamGlobalKTableLeftJoin[exactly_once] PASSED
[2023-04-21T02:51:07.580Z] 
[2023-04-21T02:51:07.580Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldSkipOverAbortedMessagesOnRestore[exactly_once] STARTED
[2023-04-21T02:51:11.109Z] 
[2023-04-21T02:51:11.109Z] 
org.apache.kafka.streams.integration.EosIntegrationTest > 
shouldCommitCorrectOffsetIfInputTopicIsTransactional[exactly_once_v2] PASSED
[2023-04-21T02:51:11.109Z] 
[2023-04-21T02:51:11.109Z] 
org.apache.kafka.streams.integration.EosIntegrationTest > 
shouldBeAbleToRunWithTwoSubtopologies[exactly_once_v2] STARTED
[2023-04-21T02:51:14.122Z] 
[2023-04-21T02:51:14.122Z] 
org.apache.kafka.streams.integration.EosIntegrationTest > 
shouldBeAbleToRunWithTwoSubtopologies[exactly_once_v2] PASSED
[2023-04-21T02:51:14.122Z] 
[2023-04-21T02:51:14.122Z] 
org.apache.kafka.streams.integration.EosIntegrationTest > 
shouldNotViolateEosIfOneTaskGetsFencedUsingIsolatedAppInstances[exactly_once_v2]
 STARTED
[2023-04-21T02:51:15.069Z] 
[2023-04-21T02:51:15.069Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldSkipOverAbortedMessagesOnRestore[exactly_once] PASSED
[2023-04-21T02:51:15.069Z] 
[2023-04-21T02:51:15.069Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldNotRestoreAbortedMessages[exactly_once] STARTED
[2023-04-21T02:51:18.374Z] 
[2023-04-21T02:51:18.374Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldNotRestoreAbortedMessages[exactly_once] PASSED
[2023-04-21T02:51:18.374Z] 
[2023-04-21T02:51:18.374Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldRestoreTransactionalMessages[exactly_once] STARTED
[2023-04-21T02:51:21.502Z] 
[2023-04-21T02:51:21.502Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldRestoreTransactionalMessages[exactly_once] PASSED
[2023-04-21T02:51:21.502Z] 
[2023-04-21T02:51:21.502Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldSkipOverTxMarkersOnRestore[exactly_once] STARTED
[2023-04-21T02:51:28.944Z] 
[2023-04-21T02:51:28.944Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldSkipOverTxMarkersOnRestore[exactly_once] PASSED
[2023-04-21T02:51:28.944Z] 
[2023-04-21T02:51:28.944Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldKStreamGlobalKTableJoin[exactly_once] STARTED
[2023-04-21T02:51:32.158Z] 
[2023-04-21T02:51:32.158Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldKStreamGlobalKTableJoin[exactly_once] PASSED
[2023-04-21T02:51:32.158Z] 
[2023-04-21T02:51:32.158Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldKStreamGlobalKTableLeftJoin[exactly_once_v2] STARTED
[2023-04-21T02:51:36.258Z] 
[2023-04-21T02:51:36.258Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldKStreamGlobalKTableLeftJoin[exactly_once_v2] PASSED
[2023-04-21T02:51:36.258Z] 
[2023-04-21T02:51:36.258Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldSkipOverAbortedMessagesOnRestore[exactly_once_v2] STARTED
[2023-04-21T02:51:44.327Z] 
[2023-04-21T02:51:44.327Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldSkipOverAbortedMessagesOnRestore[exactly_once_v2] PASSED
[2023-04-21T02:51:44.327Z] 
[2023-04-21T02:51:44.327Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldNotRestoreAbortedMessages[exactly_once_v2] STARTED
[2023-04-21T02:51:46.513Z] 
[2023-04-21T02:51:46.513Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldNotRestoreAbortedMessages[exactly_once_v2] PASSED
[2023-04-21T02:51:46.513Z] 
[2023-04-21T02:51:46.513Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldRestoreTransactionalMessages[exactly_once_v2] STARTED
[2023-04-21T02:51:49.841Z] 
[2023-04-21T02:51:49.841Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldRestoreTransactionalMessages[exactly_once_v2] PASSED
[2023-04-21T02:51:49.841Z] 
[2023-04-21T02:51:49.841Z] 
org.apache.kafka.streams.integration.GlobalKTableEOSIntegrationTest > 
shouldSkipOverTxMarkersOnRestore[exactly_once_v2] STARTED
[2023-04-21T02:51:49.841Z] 
[2023-04-21T02:51:49.841Z] 
org.apache.kafka.streams.integration.EosIntegrationTest > 

Re: [VOTE] KIP-902: Upgrade Zookeeper to 3.8.1

[Colin McCabe]

On Tue, Apr 18, 2023, at 09:01, Christo Lolov wrote:
> Thank you all for the suggestions for improvements on the KIP and the
> resulting discussions!
>
> Since the voting has been opened for ~2 months and I have received 3
> binding +1 (Colin, Ismael, Mickael) and 1 non-binding +1 (Divij) I will
> move the KIP to accepted and update the associated pull request so that it
> is ready to merge in trunk as soon as 3.5 has been made public.

Hi Christo,

Sounds good. We will also want to update the documentation in that PR.

>
> Re: Colin,
>
> What I meant to demonstrate by putting the Kafka clients is the upgrade
> path for people using the Kafka command line tools and the --zookeeper flag
> which was marked as deprecated in Kafka 2.4. I am happy to remove it if you
> think it is unnecessary or amend it so that it becomes clearer. Please let
> me know what you think!

Yes, please remove it.

thanks,
Colin

>
> Best,
> Christo
>
> On Mon, 17 Apr 2023 at 16:30, Mickael Maison 
> wrote:
>
>> Hi Christo,
>>
>> +1 (binding)
>>
>> Thanks for the KIP
>>
>> On Fri, Apr 14, 2023 at 7:32 PM Colin McCabe  wrote:
>> >
>> > On Sun, Apr 9, 2023, at 19:17, Ismael Juma wrote:
>> > >
>> > > On Sun, Apr 9, 2023 at 4:53 PM Colin McCabe 
>> wrote:
>> > >
>> > >> We are going to deprecate ZK mode soon. So if this is indeed a
>> requirement
>> > >> (no deprecated software in prod), perhaps those users will have to
>> move to
>> > >> KRaft mode. (Independently of what we decide here)
>> > >>
>> > >
>> > > Not sure where "no deprecated software in prod" is coming from. The
>> concern
>> > > is regarding end-of-life software - i.e. software that no longer
>> receives
>> > > security fixes. If we don't upgrade beyond 3.6.x, we'll be in a tough
>> > > position when a CVE is fixed only in ZooKeeper 3.7.x, 3.8.x, etc. If
>> it's a
>> > > serious security problem, then it's likely that an additional release
>> of
>> > > ZooKeeper 3.6.x might be released. But the more likely case is that a
>> > > library dependency will have a CVE that will trigger the compliance
>> checks
>> > > from enterprise users, but not warrant another ZooKeeper 3.6.x release.
>> >
>> > Hi Ismael,
>> >
>> > Fair enough. There is a difference between deprecated and unsupported.
>> ZK 3.6.x is unsupported which is worse than deprecated, since it means it
>> will not be updated.
>> >
>> > Overall, I agree with you that we're going to have to move to the new
>> version of ZK. This fits in with the overall timeline of one more year of
>> Kafka releases supporting ZK. If Apache Kafka 4.0 is April 2024, we'll need
>> to be getting security updates for ZK during this time.
>> >
>> > On Wed, Apr 12, 2023, at 08:45, Christo Lolov wrote:
>> > > Hello Colin,
>> > >
>> > > Thank you for the response!
>> > >
>> > > 1. I have attached the compatibility matrix in the KIP under the
>> section
>> > > Compatibility, Deprecation, and Migration Plan.
>> >
>> > Hi Christo,
>> >
>> > Thanks for attaching the matrix to the KIP.
>> >
>> > I don't understand why Kafka clients are part of this matrix. The Kafka
>> client doesn't use ZK directly. (Well, certain very ancient pre-1.0 Kafka
>> clients did, but that was a long time ago). So let's remove this.
>> >
>> > If I understand this correctly, the main documentation that will be
>> needed is for pre-2.4 Kafka releases. Assuming they keep everything "stock"
>> (which in my experience most users do), the net-net is that pre-2.4
>> releases need to make an extra hop through a post-2.4, pre-3.6 release. We
>> will have to document that as prominently as we can.
>> >
>> > I am +1 for this with the proviso that we do it in 3.6. We should update
>> the version as soon as we can post-3.5 so that any bugs shake out as soon
>> as possible.
>> >
>> > best,
>> > Colin
>>

Re: KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum

[Colin McCabe]

On Wed, Apr 19, 2023, at 20:56, Philip Nee wrote:
> Hey Colin,
>
> I still need to finish reading and understanding the KIP, but I have a
> couple of comments despite being ignorant of most of the KRaft stuff.
> (Sorry!)
>
> Firstly, does it make sense to create an extension of the current
> AdminClient only to handle these specific KRaft use cases? It seems
> cumbersome to have two sets of bootstrap configurations to make the
> AdminClient generic enough to handle these specific cases, instead, maybe
> it is more obvious (to me) to just extend the AdminClient. What I'm
> thinking is KraftAdminClient which continuously uses *bootstrap.servers*,
> but make this class only serves the Kraft controllers APIs.

Hi Philip,

Thanks for taking a look.

We would not want to create a new Admin client class in order to communicate 
directly with the controllers. The RPCs accepted by the controllers have the 
same format as the those accepted by the brokers. There is no difference in 
what is sent over the wire or the data structures that are used in the client.

I know you mentioned you haven't had time to read all the KRaft stuff (and 
there is a lot, I understand). But this is one area that would probably be 
clarified if you were able to read at least KIP-500 and KIP-590. RPCs sent to 
the broker are forwarded to the controller (mostly) without modification.

>
> Secondly, if we want to continue with the design, I'm not yet sure why we
> can't continue using the *bootstrap.servers*? I assume when the client gets
> the metadata, it should know who it is talking to. I'm just reconsidering
> your alternative again.
>
> A bad idea: Why don't we continue using *bootstrap.servers* but have a
> separated config like *kraft.controller* = true/false. I feel like most
> users might not know what is a controller and causes some mistakes down the
> road.
>

Well, you called it a bad idea, and I can't help but agree! :)

I think "the user might not know what a controller is" is a good sign that they 
shouldn't be interacting with the controller. Like many AdminClient APIs, this 
one is intended for use by administrators only. Most users indeed should not 
need to know what a controller is or interact with it directly. If they do 
interact it should be very clear that they are doing so. The 
--controller-bootstrap flag makes it very clear, as does the separate 
configuration.

Let me give an example of the kind of problems that arise if you want to reuse 
bootstrap.servers for this purpose.

If the user grasb a 3.4 Kafka AdminClient and set bootstrap.servers to a set of 
controller servers, and set direct.to.controller to true, the unknown (in 3.4) 
configuration will be ignored, and a normal metadata request will be sent 
without the direct to controller flag. In that case it will give back an error. 
Confusing, right?

Using controller.servers clarifies this situation because the 3.4 client will 
not support that config, and will complain about the lack of bootstrap.servers.

Actually, the situation could get even more confusing than what I described 
since some older preproduction versions of the KRaft controller did implement 
the METADATA RPC. So if you send them a METADATA request without any special 
information, it's not clear what you'll get.  Indeed, it would be dependent on 
the client version and the controller version.

The bottom line is that reusing the bootstrap.servers configuration here is not 
a good idea.

best,
Colin

> Thanks,
> P
>
> On Wed, Apr 19, 2023 at 2:18 PM Colin McCabe  wrote:
>
>> Hi all,
>>
>> I wrote a short KIP about allowing AdminClient to talk directly with the
>> KRaft controller quorum. Check it out here:
>>
>> https://cwiki.apache.org/confluence/x/Owo0Dw
>>
>> best,
>> Colin
>>

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1785

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 561841 lines...]
[2023-04-20T22:40:02.096Z] 
[2023-04-20T22:40:02.096Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > GlobalThreadShutDownOrderTest > 
shouldFinishGlobalStoreOperationOnShutDown() STARTED
[2023-04-20T22:40:08.098Z] 
[2023-04-20T22:40:08.098Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > GlobalThreadShutDownOrderTest > 
shouldFinishGlobalStoreOperationOnShutDown() PASSED
[2023-04-20T22:40:09.292Z] 
[2023-04-20T22:40:09.292Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFailStopped() STARTED
[2023-04-20T22:40:09.292Z] 
[2023-04-20T22:40:09.292Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFailStopped() PASSED
[2023-04-20T22:40:09.292Z] 
[2023-04-20T22:40:09.292Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > 
shouldNotRequireQueryHandler(TestInfo) STARTED
[2023-04-20T22:40:10.400Z] 
[2023-04-20T22:40:10.400Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > 
shouldNotRequireQueryHandler(TestInfo) PASSED
[2023-04-20T22:40:10.400Z] 
[2023-04-20T22:40:10.400Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFailNotStarted() STARTED
[2023-04-20T22:40:10.400Z] 
[2023-04-20T22:40:10.400Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFailNotStarted() PASSED
[2023-04-20T22:40:10.400Z] 
[2023-04-20T22:40:10.400Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFetchFromPartition() STARTED
[2023-04-20T22:40:13.160Z] 
[2023-04-20T22:40:13.160Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFetchFromPartition() PASSED
[2023-04-20T22:40:13.160Z] 
[2023-04-20T22:40:13.160Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > 
shouldFetchExplicitlyFromAllPartitions() STARTED
[2023-04-20T22:40:15.263Z] 
[2023-04-20T22:40:15.263Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > 
shouldFetchExplicitlyFromAllPartitions() PASSED
[2023-04-20T22:40:15.263Z] 
[2023-04-20T22:40:15.263Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFailUnknownStore() STARTED
[2023-04-20T22:40:15.263Z] 
[2023-04-20T22:40:15.263Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldFailUnknownStore() PASSED
[2023-04-20T22:40:15.263Z] 
[2023-04-20T22:40:15.263Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldRejectNonRunningActive() STARTED
[2023-04-20T22:40:18.096Z] 
[2023-04-20T22:40:18.096Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > IQv2IntegrationTest > shouldRejectNonRunningActive() PASSED
[2023-04-20T22:40:21.118Z] 
[2023-04-20T22:40:21.118Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > InternalTopicIntegrationTest > 
shouldCompactTopicsForKeyValueStoreChangelogs() STARTED
[2023-04-20T22:40:22.890Z] 
[2023-04-20T22:40:22.890Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > InternalTopicIntegrationTest > 
shouldCompactTopicsForKeyValueStoreChangelogs() PASSED
[2023-04-20T22:40:22.890Z] 
[2023-04-20T22:40:22.890Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > InternalTopicIntegrationTest > 
shouldGetToRunningWithWindowedTableInFKJ() STARTED
[2023-04-20T22:40:25.872Z] 
[2023-04-20T22:40:25.872Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > InternalTopicIntegrationTest > 
shouldGetToRunningWithWindowedTableInFKJ() PASSED
[2023-04-20T22:40:25.872Z] 
[2023-04-20T22:40:25.872Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > InternalTopicIntegrationTest > 
shouldCompactAndDeleteTopicsForWindowStoreChangelogs() STARTED
[2023-04-20T22:40:27.094Z] 
[2023-04-20T22:40:27.094Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > InternalTopicIntegrationTest > 
shouldCompactAndDeleteTopicsForWindowStoreChangelogs() PASSED
[2023-04-20T22:40:28.029Z] 
[2023-04-20T22:40:28.029Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > KStreamAggregationIntegrationTest > 
shouldAggregateSlidingWindows(TestInfo) STARTED
[2023-04-20T22:40:33.342Z] 
[2023-04-20T22:40:33.342Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 182 > KStreamAggregationIntegrationTest > 
shouldAggregateSlidingWindows(TestInfo) PASSED
[2023-04-20T22:40:33.342Z] 
[2023-04-20T22:40:33.342Z] Gradle Test Run :streams:integrationTest > 

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

[Philip Nee]

Hey Jason,

Thanks for the response. I agree with your suggestions. Just to clarify, we
want the timeout, bootstrap.resolve.timeout.ms to only bound the DNS
resolution time, and we can throw a fatal, BootstrapResolutionException (so
not connection exception anymore) afterward.

I think that aligns with the goal of this KIP.

P

On Thu, Apr 20, 2023 at 9:23 AM Jason Gustafson 
wrote:

> Hey Philip,
>
> Yeah, I see your point. Here is the challenge I'm considering. Today, the
> client does not forward connection-related errors to the application. It is
> debatable whether it should, but that is the behavior that applications
> expect today. The only case of a fatal error is DNS resolution and the
> application must handle it because they cannot even construct the client.
> Once we have this KIP, we will introduce a separate fatal failure mechanism
> through the `BootstrapConnectionException`. And we will use it not only for
> DNS failures, but general bootstrap connection failures . The problem is
> that existing applications will not know that this should be treated as a
> fatal error. So they may continue to retry. Does it help in this situation
> to poison the client so that it cannot be used? I'm not sure. Perhaps it
> would reduce the risk a bit if we change `bootstrap.connection.timeout.ms`
> to `bootstrap.resolve.timeout.ms` or something like that? Then we retain
> the current behavior if DNS succeeds, but connections fail. I know I'm the
> one that suggested generalizing the configuration, but I feel some
> hesitation after thinking about it more.
>
> Thanks,
> Jason
>
> On Wed, Apr 19, 2023 at 8:10 PM Philip Nee  wrote:
>
> > Hey Jason,
> >
> > Thanks for your review.  I think if we make it a retriable error, does it
> > make sense to have a configurable timeout still? as we expect the user to
> > continue to retry anyway.
> >
> > I'm considering the case of bad configuration. If the user retries the
> > error, then we rely on the error/warning to alert the user.  In this
> case,
> > maybe we continue using the proposed behavior, i.e. warn on each poll
> after
> > the timeout period.
> >
> > If you agree that a configuration is needed, maybe we can call this
> > *bootstrap.auto.retry.ms
> >  *instead, to indicate a configurable
> > period of automatic retry. What do you think?
> >
> > Cheers,
> > P
> >
> > On Wed, Apr 19, 2023 at 7:17 PM Jason Gustafson
>  > >
> > wrote:
> >
> > > Hey Phillip,
> > >
> > > The KIP looks good. 5 minutes seems like a reasonable tradeoff. I do
> > wonder
> > > if it is necessary to treat bootstrap timeout as a fatal error though.
> It
> > > seems possible that the exception might be caught by handlers in
> existing
> > > applications which may not expect that the client needs to be
> restarted.
> > > Perhaps it would be safer to make it retriable? As long as the
> > application
> > > continues trying to use the client, we could continue trying to reach
> the
> > > bootstrap servers perhaps? That would be closer to behavior today which
> > > only treats DNS resolution failures as fatal. What do you think?
> > >
> > > Best,
> > > Jason
> > >
> > > On Mon, Apr 10, 2023 at 1:53 PM Philip Nee 
> wrote:
> > >
> > > > Thanks, everyone: I'm starting a vote today.  Here's the recap for
> some
> > > of
> > > > the questions:
> > > >
> > > > John: I changed the proposal to throw a non-retriable exception after
> > the
> > > > timeout elapses. I feel it might be necessary to poison the client
> > after
> > > > retry expires, as it might indicate a real issue.
> > > > Ismael: The proposal is to add a configuration for the retry and it
> > will
> > > > throw a non-retriable exception after the time expires.
> > > > Chris: Addressed some unclarity that you mentioned, and a new API
> won't
> > > be
> > > > introduced in this KIP.  Maybe up for future discussion.
> > > > Jason: I'm proposing adding a timeout config and a bootstrap
> exception
> > > per
> > > > your suggestion.
> > > > Kirk: I'm proposing throwing a non-retriable exception in the network
> > > > client. See previous comment.
> > > >
> > > >
> > > > On Mon, Feb 27, 2023 at 9:36 AM Chris Egerton
>  > >
> > > > wrote:
> > > >
> > > > > Hi Philip,
> > > > >
> > > > > Yeah,  "DNS resolution should occur..." seems like a better fit. 
> > > > >
> > > > > One other question I have is whether we should expose some kind of
> > > public
> > > > > API for performing preflight validation of the bootstrap URLs. If
> we
> > > > change
> > > > > the behavior of a client configured with a silly typo (e.g.,
> > > > > "loclahost instead of localhost") from failing in the constructor
> to
> > > > > failing with a retriable exception, this might lead some client
> > > > > applications to handle that failure by, well, retrying. For
> > reference,
> > > > this
> > > > > is exactly what we do in Kafka Connect right now; see [1] and [2].
> > IMO
> > > > it'd
> > > > > be nice to be able to opt into keeping the 

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1784

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 472147 lines...]
[2023-04-20T19:55:49.792Z] 
[2023-04-20T19:55:49.792Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > GlobalThreadShutDownOrderTest > 
shouldFinishGlobalStoreOperationOnShutDown() STARTED
[2023-04-20T19:55:55.121Z] 
[2023-04-20T19:55:55.121Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > GlobalThreadShutDownOrderTest > 
shouldFinishGlobalStoreOperationOnShutDown() PASSED
[2023-04-20T19:55:57.088Z] 
[2023-04-20T19:55:57.088Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFailStopped() STARTED
[2023-04-20T19:55:57.088Z] 
[2023-04-20T19:55:57.088Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFailStopped() PASSED
[2023-04-20T19:55:57.088Z] 
[2023-04-20T19:55:57.088Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > 
shouldNotRequireQueryHandler(TestInfo) STARTED
[2023-04-20T19:55:58.275Z] 
[2023-04-20T19:55:58.275Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > 
shouldNotRequireQueryHandler(TestInfo) PASSED
[2023-04-20T19:55:58.275Z] 
[2023-04-20T19:55:58.275Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFailNotStarted() STARTED
[2023-04-20T19:55:58.275Z] 
[2023-04-20T19:55:58.275Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFailNotStarted() PASSED
[2023-04-20T19:55:58.275Z] 
[2023-04-20T19:55:58.275Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFetchFromPartition() STARTED
[2023-04-20T19:56:01.026Z] 
[2023-04-20T19:56:01.026Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFetchFromPartition() PASSED
[2023-04-20T19:56:01.026Z] 
[2023-04-20T19:56:01.026Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > 
shouldFetchExplicitlyFromAllPartitions() STARTED
[2023-04-20T19:56:03.156Z] 
[2023-04-20T19:56:03.156Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > 
shouldFetchExplicitlyFromAllPartitions() PASSED
[2023-04-20T19:56:03.156Z] 
[2023-04-20T19:56:03.156Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFailUnknownStore() STARTED
[2023-04-20T19:56:03.156Z] 
[2023-04-20T19:56:03.156Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldFailUnknownStore() PASSED
[2023-04-20T19:56:03.156Z] 
[2023-04-20T19:56:03.156Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldRejectNonRunningActive() STARTED
[2023-04-20T19:56:05.408Z] 
[2023-04-20T19:56:05.408Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > IQv2IntegrationTest > shouldRejectNonRunningActive() PASSED
[2023-04-20T19:56:07.496Z] 
[2023-04-20T19:56:07.496Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > InternalTopicIntegrationTest > 
shouldCompactTopicsForKeyValueStoreChangelogs() STARTED
[2023-04-20T19:56:09.845Z] 
[2023-04-20T19:56:09.845Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > InternalTopicIntegrationTest > 
shouldCompactTopicsForKeyValueStoreChangelogs() PASSED
[2023-04-20T19:56:09.845Z] 
[2023-04-20T19:56:09.845Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > InternalTopicIntegrationTest > 
shouldGetToRunningWithWindowedTableInFKJ() STARTED
[2023-04-20T19:56:13.422Z] 
[2023-04-20T19:56:13.422Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > InternalTopicIntegrationTest > 
shouldGetToRunningWithWindowedTableInFKJ() PASSED
[2023-04-20T19:56:13.422Z] 
[2023-04-20T19:56:13.422Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > InternalTopicIntegrationTest > 
shouldCompactAndDeleteTopicsForWindowStoreChangelogs() STARTED
[2023-04-20T19:56:14.791Z] 
[2023-04-20T19:56:14.791Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > InternalTopicIntegrationTest > 
shouldCompactAndDeleteTopicsForWindowStoreChangelogs() PASSED
[2023-04-20T19:56:17.028Z] 
[2023-04-20T19:56:17.028Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > KStreamAggregationIntegrationTest > 
shouldAggregateSlidingWindows(TestInfo) STARTED
[2023-04-20T19:56:23.233Z] 
[2023-04-20T19:56:23.233Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 184 > KStreamAggregationIntegrationTest > 
shouldAggregateSlidingWindows(TestInfo) PASSED
[2023-04-20T19:56:23.233Z] 
[2023-04-20T19:56:23.233Z] Gradle Test Run :streams:integrationTest > 

Re: [DISCUSS] Apache Kafka 3.5.0 release

[Mickael Maison]

Hi Ron,

Yes feel free to merge that fix. Thanks for letting me know!

Mickael

On Thu, Apr 20, 2023 at 8:15 PM Ron Dagostino  wrote:
>
> Hi Mickael.  I would like to merge
> https://github.com/apache/kafka/pull/13532 (KAFKA-14887: No shutdown
> for ZK session expiration in feature processing) to the 3.5 branch.
> It is a very small and focused fix that can cause unexpected broker
> shutdowns when there is instability in the connectivity to ZooKeeper.
> The risk is very low.
>
> Ron
>
>
> On Tue, Apr 18, 2023 at 9:57 AM Mickael Maison  
> wrote:
> >
> > Hi David,
> >
> > Thanks for the update. I've marked KAFKA-14869 as fixed in 3.5.0, I
> > guess you'll only resolve this ticket once you merge the backports to
> > earlier branches. The ticket will have to be resolved to run the
> > release but that should leave you enough time.
> >
> > Thanks,
> > Mickael
> >
> > On Tue, Apr 18, 2023 at 3:42 PM David Jacot  
> > wrote:
> > >
> > > Hi Mickael,
> > >
> > > FYI - I just merged the two PRs for KIP-915 to trunk/3.5. We are all good.
> > >
> > > Cheers,
> > > David
> > >
> > > On Mon, Apr 17, 2023 at 5:10 PM Mickael Maison 
> > > wrote:
> > >
> > > > Hi Chris,
> > > >
> > > > I was looking at that just now! As you said, the PRs merged provide
> > > > some functionality so I think it's fine to deliver the KIP across 2
> > > > releases.
> > > > I left a comment in https://issues.apache.org/jira/browse/KAFKA-14876
> > > > to document what's in 3.5.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > >
> > > > On Mon, Apr 17, 2023 at 5:05 PM Chris Egerton 
> > > > wrote:
> > > > >
> > > > > Hi Mickael,
> > > > >
> > > > > It looks like we missed the feature freeze cutoff for part but not 
> > > > > all of
> > > > > KIP-875 [1]. The features we've been able to merge so far are the new
> > > > > STOPPED state for connectors [2] and the API for reading offsets [3]. 
> > > > > The
> > > > > features we have not been able to merge yet are the APIs for altering 
> > > > > and
> > > > > resetting offsets.
> > > > >
> > > > > The already-merged features are useful on their own and I believe it
> > > > should
> > > > > be acceptable to release them separately from the not-yet-merged ones,
> > > > but
> > > > > wanted to double-check with you that it's alright to split this KIP
> > > > across
> > > > > two or more releases, starting with 3.5.0.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Chris
> > > > >
> > > > > [1] -
> > > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect
> > > > > [2] - https://github.com/apache/kafka/pull/13424
> > > > > [3] - https://github.com/apache/kafka/pull/13434
> > > > >
> > > > > On Fri, Apr 14, 2023 at 10:13 AM Matthias J. Sax 
> > > > wrote:
> > > > >
> > > > > > Thanks a lot!
> > > > > >
> > > > > > On 4/14/23 5:32 AM, Mickael Maison wrote:
> > > > > > > Hi Matthias,
> > > > > > >
> > > > > > > I merged the PR before cutting the 3.5 branch.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mickael
> > > > > > >
> > > > > > > On Fri, Apr 14, 2023 at 2:31 PM Mickael Maison <
> > > > mickael.mai...@gmail.com>
> > > > > > wrote:
> > > > > > >>
> > > > > > >> Hi David,
> > > > > > >>
> > > > > > >> I've created the 3.5 branch. Feel free to cherry pick these 2
> > > > commits
> > > > > > >> when they are ready.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Mickael
> > > > > > >>
> > > > > > >> On Fri, Apr 14, 2023 at 11:23 AM Satish Duggana
> > > > > > >>  wrote:
> > > > > > >>>
> > > > > > >>> Thanks Luke for helping with the reviews and adding a few tests 
> > > > > > >>> in
> > > > a
> > > > > > >>> couple of PRs.
> > > > > > >>>
> > > > > > >>> Hi Mickael,
> > > > > > >>> I raised 3 PRs recently for tiered storage, one is merged. The
> > > > other 2
> > > > > > >>> PRs are in the critical path of non-tiered storage changes also.
> > > > > > >>> Especially, in consumer fetch and retention cleanup paths. These
> > > > need
> > > > > > >>> to be thoroughly reviewed and avoid any regressions in that 
> > > > > > >>> area.
> > > > We
> > > > > > >>> should merge them to the trunk as soon as possible to make it
> > > > easier
> > > > > > >>> to work on follow-up PRs. IMO, we can avoid merging these PRs in
> > > > 3.5
> > > > > > >>> just before the release without baking for a longer duration. We
> > > > can
> > > > > > >>> take a call on this later after the reviews are done.
> > > > > > >>>
> > > > > > >>> Many of the individual functionalities related to tiered storage
> > > > like
> > > > > > >>> default topic based RLMM implementation, enhanced follower fetch
> > > > > > >>> protocol implementation for tiered storage, copying remote log
> > > > > > >>> segments are merged.
> > > > > > >>> There are 2 PRs for consumer fetch for remote record reads, 
> > > > > > >>> remote
> > > > > > >>> retention cleanup and topic deletion functionality are under
> > > > review.
> > > > > > >>>
> > > > > > >>> I do not think it can 

[jira] [Resolved] (KAFKA-14904) Flaky Test kafka.api.TransactionsBounceTest.testWithGroupId()

[Justine Olshan (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Justine Olshan resolved KAFKA-14904.

Resolution: Fixed

> Flaky Test  kafka.api.TransactionsBounceTest.testWithGroupId()
> --
>
> Key: KAFKA-14904
> URL: https://issues.apache.org/jira/browse/KAFKA-14904
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.5.0
>Reporter: Justine Olshan
>Assignee: Justine Olshan
>Priority: Blocker
>
> After merging KAFKA-14561 I noticed this test still occasionally failed via 
> org.apache.kafka.common.errors.TimeoutException: Timeout expired after 
> 6ms while awaiting EndTxn(true)
> I will investigate the cause. 
> Note: This error occurs when we are waiting for the transaction to be 
> committed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Reopened] (KAFKA-14884) Include check transaction is still ongoing right before append

[Justine Olshan (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Justine Olshan reopened KAFKA-14884:


I'm confused by all my blockers 臘‍♀️

> Include check transaction is still ongoing right before append 
> ---
>
> Key: KAFKA-14884
> URL: https://issues.apache.org/jira/browse/KAFKA-14884
> Project: Kafka
>  Issue Type: Sub-task
>Affects Versions: 3.5.0
>Reporter: Justine Olshan
>Assignee: Justine Olshan
>Priority: Blocker
>
> Even after checking via AddPartitionsToTxn, the transaction could be aborted 
> after the response. We can add one more check before appending.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Apache Kafka 3.5.0 release

[Ron Dagostino]

Hi Mickael.  I would like to merge
https://github.com/apache/kafka/pull/13532 (KAFKA-14887: No shutdown
for ZK session expiration in feature processing) to the 3.5 branch.
It is a very small and focused fix that can cause unexpected broker
shutdowns when there is instability in the connectivity to ZooKeeper.
The risk is very low.

Ron


On Tue, Apr 18, 2023 at 9:57 AM Mickael Maison  wrote:
>
> Hi David,
>
> Thanks for the update. I've marked KAFKA-14869 as fixed in 3.5.0, I
> guess you'll only resolve this ticket once you merge the backports to
> earlier branches. The ticket will have to be resolved to run the
> release but that should leave you enough time.
>
> Thanks,
> Mickael
>
> On Tue, Apr 18, 2023 at 3:42 PM David Jacot  
> wrote:
> >
> > Hi Mickael,
> >
> > FYI - I just merged the two PRs for KIP-915 to trunk/3.5. We are all good.
> >
> > Cheers,
> > David
> >
> > On Mon, Apr 17, 2023 at 5:10 PM Mickael Maison 
> > wrote:
> >
> > > Hi Chris,
> > >
> > > I was looking at that just now! As you said, the PRs merged provide
> > > some functionality so I think it's fine to deliver the KIP across 2
> > > releases.
> > > I left a comment in https://issues.apache.org/jira/browse/KAFKA-14876
> > > to document what's in 3.5.
> > >
> > > Thanks,
> > > Mickael
> > >
> > >
> > > On Mon, Apr 17, 2023 at 5:05 PM Chris Egerton 
> > > wrote:
> > > >
> > > > Hi Mickael,
> > > >
> > > > It looks like we missed the feature freeze cutoff for part but not all 
> > > > of
> > > > KIP-875 [1]. The features we've been able to merge so far are the new
> > > > STOPPED state for connectors [2] and the API for reading offsets [3]. 
> > > > The
> > > > features we have not been able to merge yet are the APIs for altering 
> > > > and
> > > > resetting offsets.
> > > >
> > > > The already-merged features are useful on their own and I believe it
> > > should
> > > > be acceptable to release them separately from the not-yet-merged ones,
> > > but
> > > > wanted to double-check with you that it's alright to split this KIP
> > > across
> > > > two or more releases, starting with 3.5.0.
> > > >
> > > > Cheers,
> > > >
> > > > Chris
> > > >
> > > > [1] -
> > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect
> > > > [2] - https://github.com/apache/kafka/pull/13424
> > > > [3] - https://github.com/apache/kafka/pull/13434
> > > >
> > > > On Fri, Apr 14, 2023 at 10:13 AM Matthias J. Sax 
> > > wrote:
> > > >
> > > > > Thanks a lot!
> > > > >
> > > > > On 4/14/23 5:32 AM, Mickael Maison wrote:
> > > > > > Hi Matthias,
> > > > > >
> > > > > > I merged the PR before cutting the 3.5 branch.
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > > >
> > > > > > On Fri, Apr 14, 2023 at 2:31 PM Mickael Maison <
> > > mickael.mai...@gmail.com>
> > > > > wrote:
> > > > > >>
> > > > > >> Hi David,
> > > > > >>
> > > > > >> I've created the 3.5 branch. Feel free to cherry pick these 2
> > > commits
> > > > > >> when they are ready.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Mickael
> > > > > >>
> > > > > >> On Fri, Apr 14, 2023 at 11:23 AM Satish Duggana
> > > > > >>  wrote:
> > > > > >>>
> > > > > >>> Thanks Luke for helping with the reviews and adding a few tests in
> > > a
> > > > > >>> couple of PRs.
> > > > > >>>
> > > > > >>> Hi Mickael,
> > > > > >>> I raised 3 PRs recently for tiered storage, one is merged. The
> > > other 2
> > > > > >>> PRs are in the critical path of non-tiered storage changes also.
> > > > > >>> Especially, in consumer fetch and retention cleanup paths. These
> > > need
> > > > > >>> to be thoroughly reviewed and avoid any regressions in that area.
> > > We
> > > > > >>> should merge them to the trunk as soon as possible to make it
> > > easier
> > > > > >>> to work on follow-up PRs. IMO, we can avoid merging these PRs in
> > > 3.5
> > > > > >>> just before the release without baking for a longer duration. We
> > > can
> > > > > >>> take a call on this later after the reviews are done.
> > > > > >>>
> > > > > >>> Many of the individual functionalities related to tiered storage
> > > like
> > > > > >>> default topic based RLMM implementation, enhanced follower fetch
> > > > > >>> protocol implementation for tiered storage, copying remote log
> > > > > >>> segments are merged.
> > > > > >>> There are 2 PRs for consumer fetch for remote record reads, remote
> > > > > >>> retention cleanup and topic deletion functionality are under
> > > review.
> > > > > >>>
> > > > > >>> I do not think it can be considered as an early access review even
> > > > > >>> with the 2 PRs in review. Luke and I synced up and agreed on the
> > > same.
> > > > > >>> Most of the recent functionality is added with a few unit tests. 
> > > > > >>> We
> > > > > >>> plan to have follow-up PRs on the immediate pending items and also
> > > > > >>> raise PRs in the next few weeks on unit tests, integration test
> > > > > >>> framework and several integration tests for many of these

Re: Adding reviewers with Github actions

[Justine Olshan]

I've tried the script, but it's not quite complete.
I've had issues finding folks -- if they haven't reviewed in kafka, we can
not find an email for them. I also had some issues with finding folks who
had reviewed before.

Right now, my strategy is to use GitHub to search previous commits for
folks' emails, but that isn't the most optimal solution -- especially if
the reviewer has no public email.
I do think it is useful to have in the commit though, so if anyone has some
ideas on how to improve, I'd be happy to hear.

Justine

On Wed, Apr 19, 2023 at 6:53 AM Ismael Juma  wrote:

> It's a lot more convenient to have it in the commit than having to follow
> links, etc.
>
> David Arthur also wrote a script to help with this step, I believe.
>
> Ismael
>
> On Tue, Apr 18, 2023, 9:29 AM Divij Vaidya 
> wrote:
>
> > Do we even need a manual attribution for a reviewer in the commit
> message?
> > GitHub automatically marks the folks as "reviewers" who have used the
> > "review-changes" button on the top left corner and left feedback. GitHub
> > also has searchability for such reviews done by a particular person using
> > the following link:
> >
> > https://github.com/search?q=is%3Apr+reviewed-by%3A
> >
> +repo%3Aapache%2Fkafka+repo%3Aapache%2Fkafka-site=issues
> >
> > (replace  with the GitHub username)
> >
> > --
> > Divij Vaidya
> >
> >
> >
> > On Tue, Apr 18, 2023 at 4:09 PM Viktor Somogyi-Vass
> >  wrote:
> >
> > > I'm not that familiar with Actions either, it just seemed like a tool
> for
> > > this purpose. :)
> > > I Did some digging and what I have in mind is that on pull request
> review
> > > it can trigger a workflow:
> > >
> > >
> >
> https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_review
> > >
> > > We could in theory use Github CLI to edit the description of the PR
> when
> > > someone gives a review (or we could perhaps enable this to simply
> comment
> > > too):
> > >
> > >
> >
> https://docs.github.com/en/actions/using-workflows/using-github-cli-in-workflows
> > >
> > > So the action definition would look something like this below. Note
> that
> > > the "run" part is very basic, it's just here for the idea. We'll
> probably
> > > need a shell script instead of that line to format it better. But the
> > point
> > > is that it edits the PR and adds the reviewer:
> > >
> > > name: Add revieweron:
> > >   issues:
> > > types:
> > >   - pull_request_reviewjobs:
> > >   comment:
> > > runs-on: ubuntu-latest
> > > steps:  - run: gh pr edit $PR_ID --title "$PR_TITLE" --body
> > > "$PR_BODY\n\nReviewers: $SENDER"
> > > env:
> > >   GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
> > >   PR_ID: ${{ github.event.pull_request.id }}
> > >   PR_TITLE: ${{ github.event.pull_request.title }}
> > >   PR_BODY: ${{ github.event.pull_request.body }}
> > >   SENDER: ${{ github.event.sender }}
> > >
> > > I'll take a look if I can try this out one my fork and get back if it
> > leads
> > > to anything.
> > >
> > > Viktor
> > >
> > > On Tue, Apr 18, 2023 at 10:12 AM Josep Prat
>  > >
> > > wrote:
> > >
> > > > Hi all,
> > > > Unless I miss something, wouldn't this GitHub action either amend the
> > > > commit (breaking signature if any) or directly do the commit itself
> > > > (meaning the action would be the one squashing and merging and not
> the
> > > > maintainer anymore)?
> > > >
> > > > Let me know if I'm missing something or if there are some nice hidden
> > > > tricks in GitHub that I didn't know :)
> > > >
> > > > Best,
> > > > On Tue, Apr 18, 2023 at 9:48 AM Viktor Somogyi-Vass
> > > >  wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Unfortunately I forgot to add myself as a reviewer *again *on a PR
> > when
> > > > > merging. Shame on me.
> > > > > However I was thinking about looking into Github actions whether we
> > can
> > > > > automate this process or at least prevent PRs from merging that
> don't
> > > > have
> > > > > "reviewers" in the description.
> > > > >
> > > > > Has anyone ever looked at it, is it worth chasing this or does
> anyone
> > > > know
> > > > > anything that'd prevent us from using it?
> > > > >
> > > > > Viktor
> > > > >
> > > >
> > > >
> > > > --
> > > > [image: Aiven] 
> > > >
> > > > *Josep Prat*
> > > > Open Source Engineering Director, *Aiven*
> > > > josep.p...@aiven.io   |   +491715557497
> > > > aiven.io    |   <
> > > https://www.facebook.com/aivencloud
> > > > >
> > > >      <
> > > > https://twitter.com/aiven_io>
> > > > *Aiven Deutschland GmbH*
> > > > Alexanderufer 3-7, 10117 Berlin
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > >
> >
>

[jira] [Resolved] (KAFKA-14884) Include check transaction is still ongoing right before append

[Justine Olshan (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Justine Olshan resolved KAFKA-14884.

Resolution: Fixed

> Include check transaction is still ongoing right before append 
> ---
>
> Key: KAFKA-14884
> URL: https://issues.apache.org/jira/browse/KAFKA-14884
> Project: Kafka
>  Issue Type: Sub-task
>Affects Versions: 3.5.0
>Reporter: Justine Olshan
>Assignee: Justine Olshan
>Priority: Blocker
>
> Even after checking via AddPartitionsToTxn, the transaction could be aborted 
> after the response. We can add one more check before appending.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

[Jason Gustafson]

Hey Philip,

Yeah, I see your point. Here is the challenge I'm considering. Today, the
client does not forward connection-related errors to the application. It is
debatable whether it should, but that is the behavior that applications
expect today. The only case of a fatal error is DNS resolution and the
application must handle it because they cannot even construct the client.
Once we have this KIP, we will introduce a separate fatal failure mechanism
through the `BootstrapConnectionException`. And we will use it not only for
DNS failures, but general bootstrap connection failures . The problem is
that existing applications will not know that this should be treated as a
fatal error. So they may continue to retry. Does it help in this situation
to poison the client so that it cannot be used? I'm not sure. Perhaps it
would reduce the risk a bit if we change `bootstrap.connection.timeout.ms`
to `bootstrap.resolve.timeout.ms` or something like that? Then we retain
the current behavior if DNS succeeds, but connections fail. I know I'm the
one that suggested generalizing the configuration, but I feel some
hesitation after thinking about it more.

Thanks,
Jason

On Wed, Apr 19, 2023 at 8:10 PM Philip Nee  wrote:

> Hey Jason,
>
> Thanks for your review.  I think if we make it a retriable error, does it
> make sense to have a configurable timeout still? as we expect the user to
> continue to retry anyway.
>
> I'm considering the case of bad configuration. If the user retries the
> error, then we rely on the error/warning to alert the user.  In this case,
> maybe we continue using the proposed behavior, i.e. warn on each poll after
> the timeout period.
>
> If you agree that a configuration is needed, maybe we can call this
> *bootstrap.auto.retry.ms
>  *instead, to indicate a configurable
> period of automatic retry. What do you think?
>
> Cheers,
> P
>
> On Wed, Apr 19, 2023 at 7:17 PM Jason Gustafson  >
> wrote:
>
> > Hey Phillip,
> >
> > The KIP looks good. 5 minutes seems like a reasonable tradeoff. I do
> wonder
> > if it is necessary to treat bootstrap timeout as a fatal error though. It
> > seems possible that the exception might be caught by handlers in existing
> > applications which may not expect that the client needs to be restarted.
> > Perhaps it would be safer to make it retriable? As long as the
> application
> > continues trying to use the client, we could continue trying to reach the
> > bootstrap servers perhaps? That would be closer to behavior today which
> > only treats DNS resolution failures as fatal. What do you think?
> >
> > Best,
> > Jason
> >
> > On Mon, Apr 10, 2023 at 1:53 PM Philip Nee  wrote:
> >
> > > Thanks, everyone: I'm starting a vote today.  Here's the recap for some
> > of
> > > the questions:
> > >
> > > John: I changed the proposal to throw a non-retriable exception after
> the
> > > timeout elapses. I feel it might be necessary to poison the client
> after
> > > retry expires, as it might indicate a real issue.
> > > Ismael: The proposal is to add a configuration for the retry and it
> will
> > > throw a non-retriable exception after the time expires.
> > > Chris: Addressed some unclarity that you mentioned, and a new API won't
> > be
> > > introduced in this KIP.  Maybe up for future discussion.
> > > Jason: I'm proposing adding a timeout config and a bootstrap exception
> > per
> > > your suggestion.
> > > Kirk: I'm proposing throwing a non-retriable exception in the network
> > > client. See previous comment.
> > >
> > >
> > > On Mon, Feb 27, 2023 at 9:36 AM Chris Egerton  >
> > > wrote:
> > >
> > > > Hi Philip,
> > > >
> > > > Yeah,  "DNS resolution should occur..." seems like a better fit. 
> > > >
> > > > One other question I have is whether we should expose some kind of
> > public
> > > > API for performing preflight validation of the bootstrap URLs. If we
> > > change
> > > > the behavior of a client configured with a silly typo (e.g.,
> > > > "loclahost instead of localhost") from failing in the constructor to
> > > > failing with a retriable exception, this might lead some client
> > > > applications to handle that failure by, well, retrying. For
> reference,
> > > this
> > > > is exactly what we do in Kafka Connect right now; see [1] and [2].
> IMO
> > > it'd
> > > > be nice to be able to opt into keeping the current behavior so that
> > > > projects like Connect could still do preflight checks of the
> > > > bootstrap.servers property for connectors before starting them, and
> > > report
> > > > any issues by failing fast instead of continuously writing
> > warning/error
> > > > messages to their logs.
> > > >
> > > > I'm not sure about where this new API could go, but a few options
> might
> > > be:
> > > >
> > > > - Expose a public variant of the existing ClientUtils class
> > > > - Add static methods to the ConsumerConfig, ProducerConfig, and
> > > > AdminClientConfig classes
> > > > - Add those same static methods to the 

Re: [DISCUSS] OpenJDK CRaC support

[Josep Prat]

Hi Radim,
You should have now permissions to create a KIP.

Best,

On Thu, Apr 20, 2023 at 2:22 PM Radim Vansa  wrote:

> Hello,
>
> upon filing a PR [1] with some initial support for OpenJDK CRaC [2][3] I
> was directed here to raise a KIP (I don't have the permissions in
> wiki/JIRA to create the KIP page yet, though).
>
> In a nutshell, CRaC intends to provide a way to checkpoint (snapshot)
> and persist a running Java application and later restore it, possibly on
> a different computer. This can be used to significantly speed up the
> boot process (from seconds or minutes to tens of milliseconds), live
> replication or migration of the heated up application. This is not
> entirely transparent to the application; the application can register
> for notification when this is happening, and sometime has to assist with
> that to prevent unexpected state after restore - e.g. close network
> connections and files.
>
> CRaC is not integrated yet into the mainline JDK; JEP is being prepared,
> and users are welcome to try out our builds. However even when this gets
> into JDK we can't expect users jump onto the latest release immediately;
> therefore we provide a facade package org.crac [4] that delegates to the
> implementation, if it is present in the running JDK, or provides a no-op
> implementation.
>
> With or without the implementation, the support for CRaC in the
> application should be designed to have a minimal impact on performance
> (few extra objects, some volatile reads...). On the other hand the
> checkpoint operation itself can be non-trivial in this matter. Therefore
> the main consideration should be about the maintenance costs - keeping a
> small JAR in dependencies and some extra code in networking and
> persistence.
>
> The support for CRaC does not have to be all-in for all components -
> maybe it does not make sense to snapshot a Broker. My PR was for Kafka
> Clients because the open network connections need to be handled in a web
> application (in my case I am enabling CRaC in Quarkus Superheros [5]
> demo). The PR does not handle all possible client-side uses; as I am not
> familiar with Kafka I follow the whack-a-mole strategy.
>
> It is possible that the C/R could be handled in a different layer, e.g.
> in Quarkus integration code. However our intent is to push the changes
> as low in the technology stack as possible, to provide the best fanout
> to users without duplicating maintenance efforts. Also having the
> support higher up can be fragile and break encapsulation.
>
> Thank you for your consideration, I hope that you'll appreciate our
> attempt to innovate the Java ecosystem.
>
> Radim Vansa
>
> PS: I'd appreciate if someone could give me the permissions on wiki to
> create a proper KIP! Username: rvansa (both Confluence and JIRA).
>
> [1] https://github.com/apache/kafka/pull/13619
>
> [2] https://wiki.openjdk.org/display/crac
>
> [3] https://github.com/openjdk/crac
>
> [4] https://github.com/CRaC/org.crac
>
> [5] https://quarkus.io/quarkus-workshops/super-heroes/
>
>

-- 
[image: Aiven] 

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.p...@aiven.io   |   +491715557497
aiven.io    |   
     
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1783

[Apache Jenkins Server]

See 

[DISCUSS] OpenJDK CRaC support

[Radim Vansa]

Hello,

upon filing a PR [1] with some initial support for OpenJDK CRaC [2][3] I 
was directed here to raise a KIP (I don't have the permissions in 
wiki/JIRA to create the KIP page yet, though).


In a nutshell, CRaC intends to provide a way to checkpoint (snapshot) 
and persist a running Java application and later restore it, possibly on 
a different computer. This can be used to significantly speed up the 
boot process (from seconds or minutes to tens of milliseconds), live 
replication or migration of the heated up application. This is not 
entirely transparent to the application; the application can register 
for notification when this is happening, and sometime has to assist with 
that to prevent unexpected state after restore - e.g. close network 
connections and files.


CRaC is not integrated yet into the mainline JDK; JEP is being prepared, 
and users are welcome to try out our builds. However even when this gets 
into JDK we can't expect users jump onto the latest release immediately; 
therefore we provide a facade package org.crac [4] that delegates to the 
implementation, if it is present in the running JDK, or provides a no-op 
implementation.


With or without the implementation, the support for CRaC in the 
application should be designed to have a minimal impact on performance 
(few extra objects, some volatile reads...). On the other hand the 
checkpoint operation itself can be non-trivial in this matter. Therefore 
the main consideration should be about the maintenance costs - keeping a 
small JAR in dependencies and some extra code in networking and persistence.


The support for CRaC does not have to be all-in for all components - 
maybe it does not make sense to snapshot a Broker. My PR was for Kafka 
Clients because the open network connections need to be handled in a web 
application (in my case I am enabling CRaC in Quarkus Superheros [5] 
demo). The PR does not handle all possible client-side uses; as I am not 
familiar with Kafka I follow the whack-a-mole strategy.


It is possible that the C/R could be handled in a different layer, e.g. 
in Quarkus integration code. However our intent is to push the changes 
as low in the technology stack as possible, to provide the best fanout 
to users without duplicating maintenance efforts. Also having the 
support higher up can be fragile and break encapsulation.


Thank you for your consideration, I hope that you'll appreciate our 
attempt to innovate the Java ecosystem.


Radim Vansa

PS: I'd appreciate if someone could give me the permissions on wiki to 
create a proper KIP! Username: rvansa (both Confluence and JIRA).


[1] https://github.com/apache/kafka/pull/13619

[2] https://wiki.openjdk.org/display/crac

[3] https://github.com/openjdk/crac

[4] https://github.com/CRaC/org.crac

[5] https://quarkus.io/quarkus-workshops/super-heroes/

[jira] [Created] (KAFKA-14925) The website shouldn't load external resources

[Mickael Maison (Jira)]

Mickael Maison created KAFKA-14925:
--

 Summary: The website shouldn't load external resources
 Key: KAFKA-14925
 URL: https://issues.apache.org/jira/browse/KAFKA-14925
 Project: Kafka
  Issue Type: Improvement
  Components: website
Reporter: Mickael Maison


In includes/_header.htm, we load a resource from fontawesome.com



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1782

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 284866 lines...]
[2023-04-20T10:14:27.435Z] > Task :raft:compileTestJava UP-TO-DATE
[2023-04-20T10:14:27.435Z] > Task :raft:testClasses UP-TO-DATE
[2023-04-20T10:14:28.356Z] > Task :connect:json:compileTestJava UP-TO-DATE
[2023-04-20T10:14:28.356Z] > Task :connect:json:testClasses UP-TO-DATE
[2023-04-20T10:14:28.356Z] > Task 
:streams:generateMetadataFileForMavenJavaPublication
[2023-04-20T10:14:28.356Z] > Task :group-coordinator:compileTestJava UP-TO-DATE
[2023-04-20T10:14:28.356Z] > Task :group-coordinator:testClasses UP-TO-DATE
[2023-04-20T10:14:28.356Z] > Task :connect:json:testJar
[2023-04-20T10:14:28.356Z] > Task :connect:json:testSrcJar
[2023-04-20T10:14:28.356Z] > Task :metadata:compileTestJava UP-TO-DATE
[2023-04-20T10:14:28.356Z] > Task :metadata:testClasses UP-TO-DATE
[2023-04-20T10:14:28.356Z] > Task 
:clients:generateMetadataFileForMavenJavaPublication
[2023-04-20T10:14:31.105Z] 
[2023-04-20T10:14:31.105Z] > Task :connect:api:javadoc
[2023-04-20T10:14:31.105Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk_2/connect/api/src/main/java/org/apache/kafka/connect/source/SourceRecord.java:44:
 warning - Tag @link: reference not found: org.apache.kafka.connect.data
[2023-04-20T10:14:32.021Z] 1 warning
[2023-04-20T10:14:32.937Z] 
[2023-04-20T10:14:32.937Z] > Task :connect:api:copyDependantLibs UP-TO-DATE
[2023-04-20T10:14:32.937Z] > Task :connect:api:jar UP-TO-DATE
[2023-04-20T10:14:32.937Z] > Task 
:connect:api:generateMetadataFileForMavenJavaPublication
[2023-04-20T10:14:32.937Z] > Task :connect:json:copyDependantLibs UP-TO-DATE
[2023-04-20T10:14:32.937Z] > Task :connect:json:jar UP-TO-DATE
[2023-04-20T10:14:32.937Z] > Task 
:connect:json:generateMetadataFileForMavenJavaPublication
[2023-04-20T10:14:32.937Z] > Task 
:connect:json:publishMavenJavaPublicationToMavenLocal
[2023-04-20T10:14:32.937Z] > Task :connect:json:publishToMavenLocal
[2023-04-20T10:14:32.937Z] > Task :connect:api:javadocJar
[2023-04-20T10:14:32.937Z] > Task :connect:api:compileTestJava UP-TO-DATE
[2023-04-20T10:14:32.937Z] > Task :connect:api:testClasses UP-TO-DATE
[2023-04-20T10:14:32.937Z] > Task :connect:api:testJar
[2023-04-20T10:14:32.937Z] > Task :connect:api:testSrcJar
[2023-04-20T10:14:32.937Z] > Task 
:connect:api:publishMavenJavaPublicationToMavenLocal
[2023-04-20T10:14:32.937Z] > Task :connect:api:publishToMavenLocal
[2023-04-20T10:14:36.446Z] > Task :streams:javadoc
[2023-04-20T10:14:36.446Z] > Task :streams:javadocJar
[2023-04-20T10:14:37.362Z] 
[2023-04-20T10:14:37.362Z] > Task :clients:javadoc
[2023-04-20T10:14:37.362Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk_2/clients/src/main/java/org/apache/kafka/clients/admin/ScramMechanism.java:32:
 warning - Tag @see: missing final '>': "https://cwiki.apache.org/confluence/display/KAFKA/KIP-554%3A+Add+Broker-side+SCRAM+Config+API;>KIP-554:
 Add Broker-side SCRAM Config API
[2023-04-20T10:14:37.362Z] 
[2023-04-20T10:14:37.362Z]  This code is duplicated in 
org.apache.kafka.common.security.scram.internals.ScramMechanism.
[2023-04-20T10:14:37.362Z]  The type field in both files must match and must 
not change. The type field
[2023-04-20T10:14:37.362Z]  is used both for passing ScramCredentialUpsertion 
and for the internal
[2023-04-20T10:14:37.362Z]  UserScramCredentialRecord. Do not change the type 
field."
[2023-04-20T10:14:39.078Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk_2/clients/src/main/java/org/apache/kafka/common/security/oauthbearer/secured/package-info.java:21:
 warning - Tag @link: reference not found: 
org.apache.kafka.common.security.oauthbearer
[2023-04-20T10:14:39.078Z] 2 warnings
[2023-04-20T10:14:39.994Z] 
[2023-04-20T10:14:39.994Z] > Task :clients:javadocJar
[2023-04-20T10:14:41.709Z] > Task :clients:srcJar
[2023-04-20T10:14:41.709Z] > Task :clients:testJar
[2023-04-20T10:14:42.625Z] > Task :clients:testSrcJar
[2023-04-20T10:14:42.625Z] > Task 
:clients:publishMavenJavaPublicationToMavenLocal
[2023-04-20T10:14:42.625Z] > Task :clients:publishToMavenLocal
[2023-04-20T10:14:58.923Z] > Task :core:compileScala
[2023-04-20T10:16:32.529Z] > Task :core:classes
[2023-04-20T10:16:32.529Z] > Task :core:compileTestJava NO-SOURCE
[2023-04-20T10:17:02.701Z] > Task :core:compileTestScala
[2023-04-20T10:18:36.608Z] > Task :core:testClasses
[2023-04-20T10:18:36.608Z] > Task :streams:compileTestJava UP-TO-DATE
[2023-04-20T10:18:36.608Z] > Task :streams:testClasses UP-TO-DATE
[2023-04-20T10:18:36.608Z] > Task :streams:testJar
[2023-04-20T10:18:36.608Z] > Task :streams:testSrcJar
[2023-04-20T10:18:36.608Z] > Task 
:streams:publishMavenJavaPublicationToMavenLocal
[2023-04-20T10:18:36.608Z] > Task :streams:publishToMavenLocal
[2023-04-20T10:18:36.608Z] 
[2023-04-20T10:18:36.608Z] Deprecated Gradle features were used in this build, 
making it incompatible with Gradle 9.0.

Re: [DISCUSS] Re-visit end of life policy

[Divij Vaidya]

Thank you Matthias for your comments.

I agree with you that the decision should be driven based on strong
community ask as it introduces a significant overhead on the maintainers. I
was hoping that more folks (users of Kafka) would contribute to this thread
with their opinion but perhaps, I need to find alternative ways to get data
about Kafka version usage in the wild. Given the effort of migrating major
versions (2.x to 3.x), I am actually surprised that we don't hear more
often from the users about the community's 12 month EOL policy.

I will get back on this thread once I have more data to support the
proposal.

--
Divij Vaidya



On Thu, Apr 20, 2023 at 3:52 AM Matthias J. Sax  wrote:

> While I understand the desire, I tend to agree with Ismael.
>
> In general, it's a significant amount of work not just to do the actual
> releases, but also the cherry-pick bug-fixed to older branches. Code
> diverges very quickly, and a clean cherry-pick is usually only possible
> for one or two branches. And it's not just simple conflicts that are
> easy to resolve, but it often even implies to do a full new fix, if the
> corresponding code was refactored, what is more often the case than one
> might think.
>
> If there is no very strong ask from the community, I would rather let
> committer spent their time reviewing PRs instead and help contributors
> to get the work merged.
>
> Just my 2ct.
>
> -Matthias
>
>
> On 4/13/23 2:52 PM, Ismael Juma wrote:
> > Clarification below.
> >
> > I did not understand your point about maintenance expense to ensure
> >> compatibility. I am confused because, IMO, irrespective of our bug fix
> >> support duration for minor versions, we should ensure that all prior
> minor
> >> versions are compatible. Hence, increasing the support duration to 24
> >> months will not add more expense than today to ensure compatibility.
> >>
> >
> > No, I am not saying that. I am saying that there is no reason not to
> > upgrade from one minor release to another since we provide full
> > compatibility between minor releases. The expensive part is that we
> release
> > 3 times a year, so you have to support 6 releases at any given point in
> > time. More importantly, you have to validate all these releases, handle
> any
> > additional bugs and so on. When it comes to the CVE stuff, you also have
> to
> > deal with cases where a project you depend on forces an upgrade to a
> > release with compatibility impact and so on. Having seen this first hand,
> > it's a significant amount of work.
> >
> > Ismael
> >
>

[jira] [Created] (KAFKA-14924) Kafka DOAP file has an error

[Claude Warren (Jira)]

Claude Warren created KAFKA-14924:
-

 Summary: Kafka DOAP file has an error
 Key: KAFKA-14924
 URL: https://issues.apache.org/jira/browse/KAFKA-14924
 Project: Kafka
  Issue Type: Bug
  Components: documentation
Affects Versions: 3.3.2
Reporter: Claude Warren


The DOAP file [1] as listed in [2] has the error:

[line: 25, col: 6 ] \{E201} The attributes on this property element, are not 
permitted with any content; expecting end element tag.

[1] 
https://gitbox.apache.org/repos/asf?p=kafka.git;a=blob_plain;f=doap_Kafka.rdf;hb=HEAD
 
[2] 
https://svn.apache.org/repos/asf/comdev/projects.apache.org/trunk/data/projects.xml



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1781

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 468010 lines...]
[2023-04-20T06:00:19.630Z] > Task 
:clients:generateMetadataFileForMavenJavaPublication
[2023-04-20T06:00:22.285Z] 
[2023-04-20T06:00:22.285Z] > Task :connect:api:javadoc
[2023-04-20T06:00:22.285Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk_2/connect/api/src/main/java/org/apache/kafka/connect/source/SourceRecord.java:44:
 warning - Tag @link: reference not found: org.apache.kafka.connect.data
[2023-04-20T06:00:24.029Z] 1 warning
[2023-04-20T06:00:24.960Z] 
[2023-04-20T06:00:24.960Z] > Task :connect:api:copyDependantLibs UP-TO-DATE
[2023-04-20T06:00:24.960Z] > Task :connect:api:jar UP-TO-DATE
[2023-04-20T06:00:24.960Z] > Task 
:connect:api:generateMetadataFileForMavenJavaPublication
[2023-04-20T06:00:24.960Z] > Task :connect:json:copyDependantLibs UP-TO-DATE
[2023-04-20T06:00:24.960Z] > Task :connect:json:jar UP-TO-DATE
[2023-04-20T06:00:24.960Z] > Task 
:connect:json:generateMetadataFileForMavenJavaPublication
[2023-04-20T06:00:24.960Z] > Task :connect:api:javadocJar
[2023-04-20T06:00:24.960Z] > Task 
:connect:json:publishMavenJavaPublicationToMavenLocal
[2023-04-20T06:00:24.960Z] > Task :connect:json:publishToMavenLocal
[2023-04-20T06:00:24.960Z] > Task :connect:api:compileTestJava UP-TO-DATE
[2023-04-20T06:00:24.960Z] > Task :connect:api:testClasses UP-TO-DATE
[2023-04-20T06:00:24.960Z] > Task :connect:api:testJar
[2023-04-20T06:00:24.960Z] > Task :connect:api:testSrcJar
[2023-04-20T06:00:24.960Z] > Task 
:connect:api:publishMavenJavaPublicationToMavenLocal
[2023-04-20T06:00:24.960Z] > Task :connect:api:publishToMavenLocal
[2023-04-20T06:00:27.915Z] > Task :streams:javadoc
[2023-04-20T06:00:27.915Z] > Task :streams:javadocJar
[2023-04-20T06:00:30.108Z] 
[2023-04-20T06:00:30.108Z] > Task :clients:javadoc
[2023-04-20T06:00:30.108Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk_2/clients/src/main/java/org/apache/kafka/clients/admin/ScramMechanism.java:32:
 warning - Tag @see: missing final '>': "https://cwiki.apache.org/confluence/display/KAFKA/KIP-554%3A+Add+Broker-side+SCRAM+Config+API;>KIP-554:
 Add Broker-side SCRAM Config API
[2023-04-20T06:00:30.108Z] 
[2023-04-20T06:00:30.108Z]  This code is duplicated in 
org.apache.kafka.common.security.scram.internals.ScramMechanism.
[2023-04-20T06:00:30.108Z]  The type field in both files must match and must 
not change. The type field
[2023-04-20T06:00:30.108Z]  is used both for passing ScramCredentialUpsertion 
and for the internal
[2023-04-20T06:00:30.108Z]  UserScramCredentialRecord. Do not change the type 
field."
[2023-04-20T06:00:31.041Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk_2/clients/src/main/java/org/apache/kafka/common/security/oauthbearer/secured/package-info.java:21:
 warning - Tag @link: reference not found: 
org.apache.kafka.common.security.oauthbearer
[2023-04-20T06:00:31.974Z] 2 warnings
[2023-04-20T06:00:32.912Z] 
[2023-04-20T06:00:32.912Z] > Task :clients:javadocJar
[2023-04-20T06:00:33.843Z] > Task :clients:srcJar
[2023-04-20T06:00:35.084Z] > Task :clients:testJar
[2023-04-20T06:00:35.084Z] > Task :clients:testSrcJar
[2023-04-20T06:00:35.084Z] > Task 
:clients:publishMavenJavaPublicationToMavenLocal
[2023-04-20T06:00:35.084Z] > Task :clients:publishToMavenLocal
[2023-04-20T06:00:50.102Z] > Task :core:compileScala
[2023-04-20T06:02:27.299Z] > Task :core:classes
[2023-04-20T06:02:27.299Z] > Task :core:compileTestJava NO-SOURCE
[2023-04-20T06:02:58.897Z] > Task :core:compileTestScala
[2023-04-20T06:04:34.417Z] > Task :core:testClasses
[2023-04-20T06:04:34.417Z] > Task :streams:compileTestJava UP-TO-DATE
[2023-04-20T06:04:34.417Z] > Task :streams:testClasses UP-TO-DATE
[2023-04-20T06:04:34.417Z] > Task :streams:testJar
[2023-04-20T06:04:34.417Z] > Task :streams:testSrcJar
[2023-04-20T06:04:34.417Z] > Task 
:streams:publishMavenJavaPublicationToMavenLocal
[2023-04-20T06:04:34.417Z] > Task :streams:publishToMavenLocal
[2023-04-20T06:04:34.417Z] 
[2023-04-20T06:04:34.417Z] Deprecated Gradle features were used in this build, 
making it incompatible with Gradle 9.0.
[2023-04-20T06:04:34.417Z] 
[2023-04-20T06:04:34.417Z] You can use '--warning-mode all' to show the 
individual deprecation warnings and determine if they come from your own 
scripts or plugins.
[2023-04-20T06:04:34.417Z] 
[2023-04-20T06:04:34.417Z] See 
https://docs.gradle.org/8.0.2/userguide/command_line_interface.html#sec:command_line_warnings
[2023-04-20T06:04:34.417Z] 
[2023-04-20T06:04:34.417Z] BUILD SUCCESSFUL in 4m 35s
[2023-04-20T06:04:34.417Z] 89 actionable tasks: 33 executed, 56 up-to-date
[Pipeline] sh
[2023-04-20T06:04:37.187Z] + grep ^version= gradle.properties
[2023-04-20T06:04:37.187Z] + cut -d= -f 2
[Pipeline] dir
[2023-04-20T06:04:37.878Z] Running in 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk_2/streams/quickstart
[Pipeline] {
[Pipeline] sh

[jira] [Created] (KAFKA-14923) Upgrade io.netty_netty-codec for CVE-2022-41881

[Vikash Mishra (Jira)]

Vikash Mishra created KAFKA-14923:
-

 Summary: Upgrade io.netty_netty-codec for CVE-2022-41881
 Key: KAFKA-14923
 URL: https://issues.apache.org/jira/browse/KAFKA-14923
 Project: Kafka
  Issue Type: Task
Affects Versions: 3.3.2, 3.4.0
Reporter: Vikash Mishra


Currently used io.netty_netty-codec version 4.1.78 has high severity CVE: [NVD 
- CVE-2022-41881 (nist.gov)|https://nvd.nist.gov/vuln/detail/CVE-2022-41881]

Fix was patched in version 4.1.86.Final. As we have higher stable version 
4.1.91.Final available we should upgrade to same to fix mentioned CVE.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)