Re: Need Contributer Access (JIRAID- ganeshsahu)

[Luke Chen]

Hi Ganesh,

Your account is all set.

Thank you.
Luke

On Fri, Mar 3, 2023 at 12:26 PM Ganesh S  wrote:

> Hello Team,i have got my JIRA ID created.
> Would like to know if I need an additional contributor access to start
> contributing.
> I would appreciate it if you could inform me if yes.
>
> My Jira ID :  ganeshsahu
>
>
> Regards
> Ganesh
>

AW: Problems with Kafka Connector Base classes

[Michael Vodep]

Hi

Thanks a lot - it's clear now.

Maybe the behavior of what happens if

* no config is returned
* more config > maxTasks is returned

Can be added to API docu 

http://localhost:8083/connectors?expand=info=status

If here the task factory also reflects some "WAITING_FOR_TASK_CONFIG" - that 
would be awesome 

Thanks
Michael

-Ursprüngliche Nachricht-
Von: Chris Egerton 
Gesendet: Donnerstag, 2. März 2023 17:33
An: dev@kafka.apache.org
Betreff: Re: Problems with Kafka Connector Base classes

[You don't often get email from chr...@aiven.io.invalid. Learn why this is 
important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Michael,

The purpose of the Connector::taskConfigs method is to specify (implicitly) the 
number of tasks that need to be run for the connector, and (explicitly) how 
each task should be configured. The former is provided implicitly by the size 
of the list returned from that method; for each element in that list, one task 
will be brought up for the connector, regardless of whether the map for that 
element is actually just empty.

The reason your for-loop was necessary is that, although we do distinguish 
between an empty list of task configs and a non-empty list, we do not 
distinguish on a per-element basis between an empty task config (i.e., the 
Map inside the task configs list) and a non-empty task config. 
So, by adding the for-loop, you signaled to the runtime that you wanted the 
number of tasks brought up for your connector to be `maxTasks`.

We choose to allow connectors to return an empty list of task configs to 
facilitate the use case of Connector implementations that may decide not to 
generate any tasks, which could be useful if, for example, there are no tables 
that match a given allow/block list to be read from in the database at the 
moment. This is also why we do not fail connectors when they generate an empty 
set of task configs (or throw an exception in some API call that the connector 
may make of the runtime).

I hope this helps clear things up for you.

Cheers,

Chris

On Thu, Mar 2, 2023 at 11:14 AM Michael Vodep < michael.vo...@bearingpoint.com> 
wrote:

> Hi
>
> I'm using:  org.apache.kafka:connect-api:3.4.0
>
> I have a simple connector:
>
> ==
> public class SimpleSinkConnector extends SinkConnector {
>
> @Override
> public List> taskConfigs(int maxTasks) {
> ArrayList> configs = new ArrayList<>();
>
> for (int i = 0; i < maxTasks; i++) {
> configs.add(new HashMap<>());
> }
>
> return configs;
> }
> ==
>
> I don't have config params (POC). So, for testing purposes I just
> returned the ArrayList. It took me now 2 hours to figure out why no
> Task was created. At the end of the day: I added the for loop - then it 
> worked.
> Reproduced the behavior several times.
>
> Can anybody conform this strange behavior? I would expect an exception
> if something is expected ...
>
> Thanks
> Michael
> 
> BearingPoint GmbH
> Sitz: Wien
> Firmenbuchgericht: Handelsgericht Wien
> Firmenbuchnummer: FN 175524z
>
> The information in this email is confidential and may be legally
> privileged. If you are not the intended recipient of this message, any
> review, disclosure, copying, distribution, retention, or any action
> taken or omitted to be taken in reliance on it is prohibited and may be 
> unlawful.
> If you are not the intended recipient, please reply to or forward a
> copy of this message to the sender and delete the message, any
> attachments, and any copies thereof from your system.
>

 BearingPoint GmbH
Sitz: Wien
Firmenbuchgericht: Handelsgericht Wien
Firmenbuchnummer: FN 175524z


The information in this email is confidential and may be legally privileged. If 
you are not the intended recipient of this message, any review, disclosure, 
copying, distribution, retention, or any action taken or omitted to be taken in 
reliance on it is prohibited and may be unlawful. If you are not the intended 
recipient, please reply to or forward a copy of this message to the sender and 
delete the message, any attachments, and any copies thereof from your system.

Re: [VOTE] KIP-882: Kafka Connect REST API timeout improvements

[Yash Mayya]

Hi Ashwin,

Thanks for the feedback. I intentionally went with the choice of
an unambiguous milliseconds unit and this would be clearly documented in
the public Kafka Connect documentation. I think supporting multiple time
units might lead to unnecessary confusion (what happens when no unit is
specified; what if an alternate short form for a time unit is used etc.).
Furthermore, the worker configuration for max request timeout will also
only accept millisecond values so this will be consistent across both the
places.

However, it's definitely not too late to incorporate the change and I'd be
happy to do so if there are others from the community who share the same
opinion!

Thanks,
Yash

On Thu, Mar 2, 2023 at 12:18 PM Ashwin  wrote:

> Thanks for the KIP Yash - +1 (non-binding)
>
> One nitpick -
> In one of the responses Chris had mentioned that the timeout param could be
> like 'timeout=10s'.
> The KIP seems to favour a millisecond timeout value and has an unwieldy
> value in the example - `POST /connectors?timeout=12`
>
> I like the idea of having smaller timeout values and the unit as part of
> the value eg: '2m', '10s' or '500ms'.
> Is it too late to incorporate this change?
>
> Thanks,
> Ashwin
>
>
> On Wed, Mar 1, 2023 at 6:32 PM Yash Mayya  wrote:
>
> > Hi all,
> >
> > I'd like to call for a vote on the (hopefully) straightforward KIP-882
> > which adds support for configuring request timeouts on Kafka Connect REST
> > APIs via query parameters along with a couple of related small
> > improvements.
> >
> > KIP -
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-882%3A+Kafka+Connect+REST+API+timeout+improvements
> >
> > Discussion thread -
> > https://lists.apache.org/thread/cygy115qmwpc3nj5omnj0crws2dw8nor
> >
> > Thanks,
> > Yash
> >
>

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

[Yash Mayya]

Hi Greg,

Thanks for taking a look!

1. I believe Chris suggested the use of a query parameter above as we
already have precedents for using query parameters to configure per request
behavior in Kafka Connect (the restart connectors API and the get
connectors API for instance). Also, the limited choice of endpoints
targeted is intentional (see my reply to the next point).

2. I intentionally targeted just the three listed endpoints where
synchronous connector config validations come into the picture. This is
because of the legitimate cases where config validation for specific
connector plugins might exceed the default request timeout in edge case
scenarios (outlined in the KIP's motivation section). Other Connect REST
endpoints shouldn't be taking longer than the default 90 second request
timeout; if they do so, it would either be indicative of a bug in the
Connect framework or a cluster health issue - neither of which should be
covered up by manually setting a longer request timeout.

3. 4. I think changing the config validation behavior would be a backward
incompatible change and I wanted to avoid that in this particular KIP.
There are multiple programmatic UIs out there which rely on the current
synchronous config validation behavior and breaking the existing contract
would definitely require a larger discussion.

Thanks,
Yash

On Fri, Mar 3, 2023 at 12:04 AM Greg Harris 
wrote:

> Hey Yash,
>
> Thanks for the KIP, and sorry for the late review.
>
> 1. Have you considered a HTTP header to provide the client-configurable
> timeout? A header might more naturally extend to all of the other endpoints
> in the future, rather than duplicating the query parameter across
> endpoints.
>
> 2. I understand that this change is targeted at just long duration
> Connector::validation calls, is that due to voluntary scope constraints?
> Implementing configurable timeouts for all endpoints in a uniform way could
> be desirable, even if the default timeout will work for nearly all of the
> other calls.
>
> 3. Did you consider adding asynchronous validation as a user-facing
> feature? I think that relying on the synchronous validation results in a
> single HTTP request is a bit of an anti-pattern, and one that we've
> inherited from the original REST design. It seems useful when using the
> REST API by hand, but seems to be a liability when used in environments
> with an external management layer.
>
> 4. Perhaps rather than allowing synchronous calls to Connector:validate to
> increase in duration, we should provide a way for connectors to surface
> validation problems later in their lifecycle. Currently there is the
> ConnectorContext::raiseError that transitions the task to FAILED, perhaps a
> similar API could asynchronously emit re-validation results. We've also had
> a problem with long start() duration for the same reasons as validate().
>
> I understand if you want to keep this KIP as tightly focused as possible,
> but i'm worried that it is addressing the symptom and not the problem. I
> want to make sure that this change is impactful and isn't obsoleted by a
> later improvement.
>
> Thanks,
> Greg
>
>
> On Wed, Mar 1, 2023 at 5:07 AM Yash Mayya  wrote:
>
> > Hi all,
> >
> > Thanks for all the feedback and discussion. I've renamed the KIP title to
> > "Kafka Connect REST API timeout improvements" since we're introducing a
> > couple of improvements (cancelling create / update connector requests
> when
> > config validations timeout and avoiding double config validations in
> > distributed mode) along with making the request timeouts configurable.
> The
> > new KIP link is
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-882%3A+Kafka+Connect+REST+API+timeout+improvements
> > and I've called for a vote for the KIP here -
> > https://lists.apache.org/thread/mgx8lczx2f57pk7x3vh0nqk00s79grgp.
> >
> > Thanks,
> > Yash
> >
> > On Sat, Nov 5, 2022 at 11:42 PM Sagar  wrote:
> >
> > > Hey Yash,
> > >
> > > Thanks for the explanation. I think it should be fine to delegate the
> > > validation directly to the leader.
> > >
> > > Thanks!
> > > Sagar.
> > >
> > > On Sat, Nov 5, 2022 at 10:42 AM Yash Mayya 
> wrote:
> > >
> > > > Hi Sagar,
> > > >
> > > > Thanks for chiming in!
> > > >
> > > > > Having said that, why does the worker forward to the
> > > > > leader? I am thinking if the worker can perform the validation on
> > it's
> > > > own,
> > > > > we could let it do the validation instead of forwarding everything
> to
> > > the
> > > > > leader
> > > >
> > > > Only the leader is allowed to perform writes to the config topic, so
> > any
> > > > request that requires a config topic write ends up being forwarded to
> > the
> > > > leader. The `POST /connectors` and `PUT
> /connectors/{connector}/config`
> > > > endpoints call `Herder::putConnectorConfig` which internally does a
> > > config
> > > > validation first before initiating another herder request to write to
> > the
> > > > config topic (in 

Need Contributer Access (JIRAID- ganeshsahu)

[Ganesh S]

Hello Team,i have got my JIRA ID created.
Would like to know if I need an additional contributor access to start
contributing.
I would appreciate it if you could inform me if yes.

My Jira ID :  ganeshsahu


Regards
Ganesh

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1642

[Apache Jenkins Server]

See 

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1641

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 447921 lines...]
[2023-03-02T22:29:54.956Z] > Task :clients:compileTestJava UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :clients:testClasses UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :server-common:compileTestJava UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :server-common:testClasses UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :connect:json:compileTestJava UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :connect:json:testClasses UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :storage:api:compileTestJava UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :storage:api:testClasses UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task 
:streams:generateMetadataFileForMavenJavaPublication
[2023-03-02T22:29:54.956Z] > Task :raft:compileTestJava UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :raft:testClasses UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :connect:json:testJar
[2023-03-02T22:29:54.956Z] > Task :group-coordinator:compileTestJava UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :group-coordinator:testClasses UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :connect:json:testSrcJar
[2023-03-02T22:29:54.956Z] > Task :metadata:compileTestJava UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task :metadata:testClasses UP-TO-DATE
[2023-03-02T22:29:54.956Z] > Task 
:clients:generateMetadataFileForMavenJavaPublication
[2023-03-02T22:29:57.628Z] 
[2023-03-02T22:29:57.628Z] > Task :connect:api:javadoc
[2023-03-02T22:29:57.628Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk/connect/api/src/main/java/org/apache/kafka/connect/source/SourceRecord.java:44:
 warning - Tag @link: reference not found: org.apache.kafka.connect.data
[2023-03-02T22:30:00.367Z] 1 warning
[2023-03-02T22:30:00.367Z] 
[2023-03-02T22:30:00.367Z] > Task :connect:api:copyDependantLibs UP-TO-DATE
[2023-03-02T22:30:00.367Z] > Task :connect:api:jar UP-TO-DATE
[2023-03-02T22:30:00.367Z] > Task 
:connect:api:generateMetadataFileForMavenJavaPublication
[2023-03-02T22:30:00.367Z] > Task :connect:json:copyDependantLibs UP-TO-DATE
[2023-03-02T22:30:00.367Z] > Task :connect:json:jar UP-TO-DATE
[2023-03-02T22:30:00.367Z] > Task 
:connect:json:generateMetadataFileForMavenJavaPublication
[2023-03-02T22:30:00.367Z] > Task :connect:api:javadocJar
[2023-03-02T22:30:00.367Z] > Task 
:connect:json:publishMavenJavaPublicationToMavenLocal
[2023-03-02T22:30:00.367Z] > Task :connect:json:publishToMavenLocal
[2023-03-02T22:30:00.367Z] > Task :connect:api:compileTestJava UP-TO-DATE
[2023-03-02T22:30:00.367Z] > Task :connect:api:testClasses UP-TO-DATE
[2023-03-02T22:30:00.367Z] > Task :connect:api:testJar
[2023-03-02T22:30:00.367Z] > Task :connect:api:testSrcJar
[2023-03-02T22:30:00.367Z] > Task 
:connect:api:publishMavenJavaPublicationToMavenLocal
[2023-03-02T22:30:00.367Z] > Task :connect:api:publishToMavenLocal
[2023-03-02T22:30:02.996Z] > Task :streams:javadoc
[2023-03-02T22:30:02.996Z] > Task :streams:javadocJar
[2023-03-02T22:30:06.743Z] 
[2023-03-02T22:30:06.743Z] > Task :clients:javadoc
[2023-03-02T22:30:06.743Z] 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk/clients/src/main/java/org/apache/kafka/common/security/oauthbearer/secured/package-info.java:21:
 warning - Tag @link: reference not found: 
org.apache.kafka.common.security.oauthbearer
[2023-03-02T22:30:07.679Z] 1 warning
[2023-03-02T22:30:08.615Z] 
[2023-03-02T22:30:08.615Z] > Task :clients:javadocJar
[2023-03-02T22:30:09.553Z] > Task :clients:srcJar
[2023-03-02T22:30:10.490Z] > Task :clients:testJar
[2023-03-02T22:30:10.490Z] > Task :clients:testSrcJar
[2023-03-02T22:30:10.490Z] > Task 
:clients:publishMavenJavaPublicationToMavenLocal
[2023-03-02T22:30:10.490Z] > Task :clients:publishToMavenLocal
[2023-03-02T22:30:24.743Z] > Task :core:compileScala
[2023-03-02T22:31:59.132Z] > Task :core:classes
[2023-03-02T22:31:59.132Z] > Task :core:compileTestJava NO-SOURCE
[2023-03-02T22:32:29.319Z] > Task :core:compileTestScala
[2023-03-02T22:34:03.395Z] > Task :core:testClasses
[2023-03-02T22:34:03.395Z] > Task :streams:compileTestJava UP-TO-DATE
[2023-03-02T22:34:03.395Z] > Task :streams:testClasses UP-TO-DATE
[2023-03-02T22:34:03.395Z] > Task :streams:testJar
[2023-03-02T22:34:03.395Z] > Task :streams:testSrcJar
[2023-03-02T22:34:03.395Z] > Task 
:streams:publishMavenJavaPublicationToMavenLocal
[2023-03-02T22:34:03.395Z] > Task :streams:publishToMavenLocal
[2023-03-02T22:34:03.395Z] 
[2023-03-02T22:34:03.395Z] Deprecated Gradle features were used in this build, 
making it incompatible with Gradle 9.0.
[2023-03-02T22:34:03.395Z] 
[2023-03-02T22:34:03.395Z] You can use '--warning-mode all' to show the 
individual deprecation warnings and determine if they come from your own 
scripts or plugins.
[2023-03-02T22:34:03.395Z] 
[2023-03-02T22:34:03.395Z] See 
https://docs.gradle.org/8.0.1/userguide/command_line_interface.html#sec:command_line_warnings

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1640

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 450254 lines...]
[2023-03-02T20:14:59.468Z] > Task :streams:test-utils:compileJava UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :server-common:compileTestJava UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :server-common:testClasses UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :connect:json:compileTestJava UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :connect:json:testClasses UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :storage:api:compileTestJava UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :storage:api:testClasses UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :raft:compileTestJava UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :raft:testClasses UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :group-coordinator:compileTestJava UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :group-coordinator:testClasses UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :connect:json:testJar
[2023-03-02T20:14:59.468Z] > Task :connect:json:testSrcJar
[2023-03-02T20:14:59.468Z] > Task :metadata:compileTestJava UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :metadata:testClasses UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task :streams:jar UP-TO-DATE
[2023-03-02T20:14:59.468Z] > Task 
:streams:generateMetadataFileForMavenJavaPublication
[2023-03-02T20:14:59.468Z] > Task 
:clients:generateMetadataFileForMavenJavaPublication
[2023-03-02T20:15:02.478Z] 
[2023-03-02T20:15:02.478Z] > Task :connect:api:javadoc
[2023-03-02T20:15:02.478Z] 
/home/jenkins/workspace/Kafka_kafka_trunk/connect/api/src/main/java/org/apache/kafka/connect/source/SourceRecord.java:44:
 warning - Tag @link: reference not found: org.apache.kafka.connect.data
[2023-03-02T20:15:03.572Z] 1 warning
[2023-03-02T20:15:04.567Z] 
[2023-03-02T20:15:04.567Z] > Task :connect:api:copyDependantLibs UP-TO-DATE
[2023-03-02T20:15:04.567Z] > Task :connect:api:jar UP-TO-DATE
[2023-03-02T20:15:04.567Z] > Task 
:connect:api:generateMetadataFileForMavenJavaPublication
[2023-03-02T20:15:04.567Z] > Task :connect:json:copyDependantLibs UP-TO-DATE
[2023-03-02T20:15:04.567Z] > Task :connect:json:jar UP-TO-DATE
[2023-03-02T20:15:04.567Z] > Task 
:connect:json:generateMetadataFileForMavenJavaPublication
[2023-03-02T20:15:04.567Z] > Task :connect:api:javadocJar
[2023-03-02T20:15:04.567Z] > Task :connect:api:compileTestJava UP-TO-DATE
[2023-03-02T20:15:04.567Z] > Task :connect:api:testClasses UP-TO-DATE
[2023-03-02T20:15:04.567Z] > Task 
:connect:json:publishMavenJavaPublicationToMavenLocal
[2023-03-02T20:15:04.567Z] > Task :connect:json:publishToMavenLocal
[2023-03-02T20:15:05.555Z] > Task :connect:api:testJar
[2023-03-02T20:15:05.555Z] > Task :connect:api:testSrcJar
[2023-03-02T20:15:05.555Z] > Task 
:connect:api:publishMavenJavaPublicationToMavenLocal
[2023-03-02T20:15:05.555Z] > Task :connect:api:publishToMavenLocal
[2023-03-02T20:15:06.709Z] > Task :streams:javadoc
[2023-03-02T20:15:06.709Z] > Task :streams:javadocJar
[2023-03-02T20:15:09.003Z] 
[2023-03-02T20:15:09.003Z] > Task :clients:javadoc
[2023-03-02T20:15:09.003Z] 
/home/jenkins/workspace/Kafka_kafka_trunk/clients/src/main/java/org/apache/kafka/common/security/oauthbearer/secured/package-info.java:21:
 warning - Tag @link: reference not found: 
org.apache.kafka.common.security.oauthbearer
[2023-03-02T20:15:09.003Z] 1 warning
[2023-03-02T20:15:10.078Z] 
[2023-03-02T20:15:10.078Z] > Task :clients:javadocJar
[2023-03-02T20:15:11.154Z] > Task :clients:srcJar
[2023-03-02T20:15:11.154Z] > Task :clients:testJar
[2023-03-02T20:15:12.220Z] > Task :clients:testSrcJar
[2023-03-02T20:15:12.221Z] > Task 
:clients:publishMavenJavaPublicationToMavenLocal
[2023-03-02T20:15:12.221Z] > Task :clients:publishToMavenLocal
[2023-03-02T20:15:29.169Z] > Task :core:compileScala
[2023-03-02T20:16:28.717Z] > Task :core:classes
[2023-03-02T20:16:28.717Z] > Task :core:compileTestJava NO-SOURCE
[2023-03-02T20:17:01.372Z] > Task :core:compileTestScala
[2023-03-02T20:17:40.544Z] > Task :core:testClasses
[2023-03-02T20:17:40.544Z] > Task :streams:compileTestJava UP-TO-DATE
[2023-03-02T20:17:40.544Z] > Task :streams:testClasses UP-TO-DATE
[2023-03-02T20:17:40.544Z] > Task :streams:testJar
[2023-03-02T20:17:40.544Z] > Task :streams:testSrcJar
[2023-03-02T20:17:40.544Z] > Task 
:streams:publishMavenJavaPublicationToMavenLocal
[2023-03-02T20:17:40.544Z] > Task :streams:publishToMavenLocal
[2023-03-02T20:17:40.544Z] 
[2023-03-02T20:17:40.544Z] Deprecated Gradle features were used in this build, 
making it incompatible with Gradle 9.0.
[2023-03-02T20:17:40.544Z] 
[2023-03-02T20:17:40.544Z] You can use '--warning-mode all' to show the 
individual deprecation warnings and determine if they come from your own 
scripts or plugins.
[2023-03-02T20:17:40.544Z] 
[2023-03-02T20:17:40.544Z] See 
https://docs.gradle.org/8.0.1/userguide/command_line_interface.html#sec:command_line_warnings
[2023-03-02T20:17:40.544Z] 

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.3 #160

[Apache Jenkins Server]

See 

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

[Greg Harris]

Hey Yash,

Thanks for the KIP, and sorry for the late review.

1. Have you considered a HTTP header to provide the client-configurable
timeout? A header might more naturally extend to all of the other endpoints
in the future, rather than duplicating the query parameter across endpoints.

2. I understand that this change is targeted at just long duration
Connector::validation calls, is that due to voluntary scope constraints?
Implementing configurable timeouts for all endpoints in a uniform way could
be desirable, even if the default timeout will work for nearly all of the
other calls.

3. Did you consider adding asynchronous validation as a user-facing
feature? I think that relying on the synchronous validation results in a
single HTTP request is a bit of an anti-pattern, and one that we've
inherited from the original REST design. It seems useful when using the
REST API by hand, but seems to be a liability when used in environments
with an external management layer.

4. Perhaps rather than allowing synchronous calls to Connector:validate to
increase in duration, we should provide a way for connectors to surface
validation problems later in their lifecycle. Currently there is the
ConnectorContext::raiseError that transitions the task to FAILED, perhaps a
similar API could asynchronously emit re-validation results. We've also had
a problem with long start() duration for the same reasons as validate().

I understand if you want to keep this KIP as tightly focused as possible,
but i'm worried that it is addressing the symptom and not the problem. I
want to make sure that this change is impactful and isn't obsoleted by a
later improvement.

Thanks,
Greg


On Wed, Mar 1, 2023 at 5:07 AM Yash Mayya  wrote:

> Hi all,
>
> Thanks for all the feedback and discussion. I've renamed the KIP title to
> "Kafka Connect REST API timeout improvements" since we're introducing a
> couple of improvements (cancelling create / update connector requests when
> config validations timeout and avoiding double config validations in
> distributed mode) along with making the request timeouts configurable. The
> new KIP link is
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-882%3A+Kafka+Connect+REST+API+timeout+improvements
> and I've called for a vote for the KIP here -
> https://lists.apache.org/thread/mgx8lczx2f57pk7x3vh0nqk00s79grgp.
>
> Thanks,
> Yash
>
> On Sat, Nov 5, 2022 at 11:42 PM Sagar  wrote:
>
> > Hey Yash,
> >
> > Thanks for the explanation. I think it should be fine to delegate the
> > validation directly to the leader.
> >
> > Thanks!
> > Sagar.
> >
> > On Sat, Nov 5, 2022 at 10:42 AM Yash Mayya  wrote:
> >
> > > Hi Sagar,
> > >
> > > Thanks for chiming in!
> > >
> > > > Having said that, why does the worker forward to the
> > > > leader? I am thinking if the worker can perform the validation on
> it's
> > > own,
> > > > we could let it do the validation instead of forwarding everything to
> > the
> > > > leader
> > >
> > > Only the leader is allowed to perform writes to the config topic, so
> any
> > > request that requires a config topic write ends up being forwarded to
> the
> > > leader. The `POST /connectors` and `PUT /connectors/{connector}/config`
> > > endpoints call `Herder::putConnectorConfig` which internally does a
> > config
> > > validation first before initiating another herder request to write to
> the
> > > config topic (in distributed mode). If we want to allow the first
> worker
> > to
> > > do the config validation, and then forward the request to the leader
> just
> > > for the write to the config topic, we'd either need something like a
> skip
> > > validations query parameter on the endpoint like Chris talks about
> above
> > or
> > > else a new internal only endpoint that just does a write to the config
> > > topic without any prior config validation. However, we agreed that this
> > > optimization doesn't really seem necessary for now and can be done
> later
> > if
> > > deemed useful. I'd be happy to hear differing thoughts if any, however.
> > >
> > > > I think a bound is certainly needed but IMO it shouldn't go beyond
> > > > 10 mins considering this is just validation
> > >
> > > Yeah, I agree that this seems like a fair value - I've elected to go
> > with a
> > > default value of 10 minutes for the proposed worker configuration that
> > sets
> > > an upper bound for the timeout query parameter.
> > >
> > > Thanks,
> > > Yash
> > >
> > > On Sat, Nov 5, 2022 at 10:30 AM Yash Mayya 
> wrote:
> > >
> > > > Hi Chris,
> > > >
> > > > Thanks again for your feedback. I think a worker configuration for
> the
> > > > upper bound makes sense - I initially thought we could hardcode it
> > (just
> > > > like the current request timeout is), but there's no reason to set
> > > another
> > > > artificial bound that isn't user configurable which is exactly what
> > we're
> > > > trying to change here in the first place. I've updated the KIP based
> on
> > > all
> > > > our discussion above.
> > 

Re: Problems with Kafka Connector Base classes

[Chris Egerton]

Hi Michael,

The purpose of the Connector::taskConfigs method is to specify (implicitly)
the number of tasks that need to be run for the connector, and (explicitly)
how each task should be configured. The former is provided implicitly by
the size of the list returned from that method; for each element in that
list, one task will be brought up for the connector, regardless of whether
the map for that element is actually just empty.

The reason your for-loop was necessary is that, although we do distinguish
between an empty list of task configs and a non-empty list, we do not
distinguish on a per-element basis between an empty task config (i.e., the
Map inside the task configs list) and a non-empty task
config. So, by adding the for-loop, you signaled to the runtime that you
wanted the number of tasks brought up for your connector to be `maxTasks`.

We choose to allow connectors to return an empty list of task configs to
facilitate the use case of Connector implementations that may decide not to
generate any tasks, which could be useful if, for example, there are no
tables that match a given allow/block list to be read from in the database
at the moment. This is also why we do not fail connectors when they
generate an empty set of task configs (or throw an exception in some API
call that the connector may make of the runtime).

I hope this helps clear things up for you.

Cheers,

Chris

On Thu, Mar 2, 2023 at 11:14 AM Michael Vodep <
michael.vo...@bearingpoint.com> wrote:

> Hi
>
> I'm using:  org.apache.kafka:connect-api:3.4.0
>
> I have a simple connector:
>
> ==
> public class SimpleSinkConnector extends SinkConnector {
>
> @Override
> public List> taskConfigs(int maxTasks) {
> ArrayList> configs = new ArrayList<>();
>
> for (int i = 0; i < maxTasks; i++) {
> configs.add(new HashMap<>());
> }
>
> return configs;
> }
> ==
>
> I don't have config params (POC). So, for testing purposes I just returned
> the ArrayList. It took me now 2 hours to figure out why no Task was
> created. At the end of the day: I added the for loop - then it worked.
> Reproduced the behavior several times.
>
> Can anybody conform this strange behavior? I would expect an exception if
> something is expected ...
>
> Thanks
> Michael
> 
> BearingPoint GmbH
> Sitz: Wien
> Firmenbuchgericht: Handelsgericht Wien
> Firmenbuchnummer: FN 175524z
>
> The information in this email is confidential and may be legally
> privileged. If you are not the intended recipient of this message, any
> review, disclosure, copying, distribution, retention, or any action taken
> or omitted to be taken in reliance on it is prohibited and may be unlawful.
> If you are not the intended recipient, please reply to or forward a copy of
> this message to the sender and delete the message, any attachments, and any
> copies thereof from your system.
>

Problems with Kafka Connector Base classes

[Michael Vodep]

Hi

I'm using:  org.apache.kafka:connect-api:3.4.0

I have a simple connector:

==
public class SimpleSinkConnector extends SinkConnector {

@Override
public List> taskConfigs(int maxTasks) {
ArrayList> configs = new ArrayList<>();

for (int i = 0; i < maxTasks; i++) {
configs.add(new HashMap<>());
}

return configs;
}
==

I don't have config params (POC). So, for testing purposes I just returned the 
ArrayList. It took me now 2 hours to figure out why no Task was created. At the 
end of the day: I added the for loop - then it worked. Reproduced the behavior 
several times.

Can anybody conform this strange behavior? I would expect an exception if 
something is expected ...

Thanks
Michael

BearingPoint GmbH
Sitz: Wien
Firmenbuchgericht: Handelsgericht Wien
Firmenbuchnummer: FN 175524z

The information in this email is confidential and may be legally privileged. If 
you are not the intended recipient of this message, any review, disclosure, 
copying, distribution, retention, or any action taken or omitted to be taken in 
reliance on it is prohibited and may be unlawful. If you are not the intended 
recipient, please reply to or forward a copy of this message to the sender and 
delete the message, any attachments, and any copies thereof from your system.

Re: [VOTE] KIP-907: Add Boolean Serde to public interface

[Lucas Brutschy]

+1 (non-binding)

Thank you!

On Wed, Mar 1, 2023 at 9:03 PM Matthias J. Sax  wrote:
>
> +1 (binding)
>
> Thanks for the KIP!
>
> On 3/1/23 10:59 AM, Walker Carlson wrote:
> > +1 Binding
> >
> > On Mon, Feb 27, 2023 at 1:46 PM Chia-Ping Tsai  wrote:
> >
> >> +1 (binding)
> >>
> >

Re: [DISCUSS] KIP-902: Upgrade Zookeeper to 3.8.1

[Ismael Juma]

Thanks for the KIP. I think the following is a little confusing since it
doesn't make it clear the the ZooKeeper deployment is separate from Kafka,
Kafka only includes the ZooKeeper libraries. I think it would be useful to
explain the upgrade process for someone running Apache Kafka 2.3 and
ZooKeeper 3.4 (the hardest case) and the same for someone running Apache
Kafka 2.4 and ZooKeeper 3.5.

Also, it's worth clarifying that we actually still test direct kafka
upgrades from 0.8.2 to 3.4. In practice, we have distinguished "providing
updates" versus "allowing direct upgrades from". Apache Kafka 4.0 will
change this since you will have to upgrade to a bridge release before
upgrading to 4.0, but that's a new development.

"Users who use Kafka clusters with Zookeeper clients older than 3.5.x won't
be able to communicate with a Zookeeper cluster using 3.8.1. As mentioned
in the accompanying JIRA ticket Apache Kafka has been using Zookeeper 3.5.x
since version 2.4 so versions above and including it should be safe for
this upgrade. It is acceptable to break compatibility with Apache Kafka
versions prior to 2.4 as they are considered beyond their end of life and
are not maintained. (source: Time Based Release Plan#WhatIsOurEOLPolicy)."

Ismael

On Wed, Feb 15, 2023 at 1:47 AM Christo Lolov 
wrote:

> Hello!
>
> I would like to start a discussion for KIP-902: Upgrade Zookeeper to
> 3.8.1. The Zookeeper version currently used in Kafka reached its end of
> life in December 2022. You can find the KIP at
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784
>
> Thanks in advance for the reviews.
>
> Christo

i have question to kafka error

[SeokYoung Jang]

hi~ 

i have question. 

i installed kafka in azure kubernetes services. 

but occur is error. 

-- error --
{"thread_name":"controller-event-thread","msg":"[ReplicaStateMachine 
controllerId=0] Triggering offline replica state 
changes","level":"INFO","time":"2023-02-27T00:09:39.590Z"}
{"thread_name":"kafka-log-cleaner-thread-0","msg":"Failed to clean up log for 
__consumer_offsets-4 in dir /opt/kafka/data/topics due to 
IOException","level":"ERROR","time":"2023-02-27T00:09:39.905Z"}
java.nio.file.FileSystemException: 
/opt/kafka/data/topics/__consumer_offsets-4/.log.cleaned: 
Operation not permitted
at 
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
at 
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at 
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at 
java.base/sun.nio.fs.UnixFileAttributeViews$Basic.setTimes(UnixFileAttributeViews.java:125)
at java.base/java.nio.file.Files.setLastModifiedTime(Files.java:2355)
at kafka.log.LogSegment.lastModified_$eq(LogSegment.scala:650)
at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:609)
at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:538)
at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:537)
at scala.collection.immutable.List.foreach(List.scala:333)
at kafka.log.Cleaner.doClean(LogCleaner.scala:537)
at kafka.log.Cleaner.clean(LogCleaner.scala:511)
at kafka.log.LogCleaner$CleanerThread.cleanLog(LogCleaner.scala:380)
at 
kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:352)
at 
kafka.log.LogCleaner$CleanerThread.tryCleanFilthiestLog(LogCleaner.scala:332)
at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:321)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
{"thread_name":"kafka-log-cleaner-thread-0","msg":"Failed to clean up log for 
__consumer_offsets-4 in dir /opt/kafka/data/topics due to 
IOException","level":"ERROR","time":"2023-02-27T00:09:39.905Z"}
java.nio.file.FileSystemException: 
/opt/kafka/data/topics/__consumer_offsets-4/.log.cleaned: 
Operation not permitted
at 
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
at 
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at 
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at 
java.base/sun.nio.fs.UnixFileAttributeViews$Basic.setTimes(UnixFileAttributeViews.java:125)
at java.base/java.nio.file.Files.setLastModifiedTime(Files.java:2355)
at kafka.log.LogSegment.lastModified_$eq(LogSegment.scala:650)
at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:609)
at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:538)
at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:537)
at scala.collection.immutable.List.foreach(List.scala:333)
at kafka.log.Cleaner.doClean(LogCleaner.scala:537)
at kafka.log.Cleaner.clean(LogCleaner.scala:511)
at kafka.log.LogCleaner$CleanerThread.cleanLog(LogCleaner.scala:380)
at 
kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:352)
at 
kafka.log.LogCleaner$CleanerThread.tryCleanFilthiestLog(LogCleaner.scala:332)
at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:321)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
{"thread_name":"LogDirFailureHandler","msg":"[ReplicaManager broker=0] Stopping 
serving replicas in dir 
/opt/kafka/data/topics","level":"WARN","time":"2023-02-27T00:09:39.910Z"}
{"thread_name":"LogDirFailureHandler","msg":"[ReplicaManager broker=0] Stopping 
serving replicas in dir 
/opt/kafka/data/topics","level":"WARN","time":"2023-02-27T00:09:39.910Z"}
{"thread_name":"LogDirFailureHandler","msg":"[ReplicaManager broker=0] Broker 0 
stopped fetcher for partitions  and stopped moving logs for partitions  because 
they are in the failed log directory 
/opt/kafka/data/topics.","level":"WARN","time":"2023-02-27T00:09:39.920Z"}
{"thread_name":"LogDirFailureHandler","msg":"[ReplicaManager broker=0] Broker 0 
stopped fetcher for partitions  and stopped moving logs for partitions  because 
they are in the failed log directory 
/opt/kafka/data/topics.","level":"WARN","time":"2023-02-27T00:09:39.920Z"}
{"thread_name":"LogDirFailureHandler","msg":"Stopping serving logs in dir 
/opt/kafka/data/topics","level":"WARN","time":"2023-02-27T00:09:39.921Z"}
{"thread_name":"LogDirFailureHandler","msg":"Stopping serving logs in dir 
/opt/kafka/data/topics","level":"WARN","time":"2023-02-27T00:09:39.921Z"}
{"thread_name":"LogDirFailureHandler","msg":"Shutdown broker because all log 
dirs in /opt/kafka/data/topics have 

[jira] [Created] (KAFKA-14774) the removed listeners should not be reconfigurable

[Chia-Ping Tsai (Jira)]

Chia-Ping Tsai created KAFKA-14774:
--

 Summary: the removed listeners should not be reconfigurable
 Key: KAFKA-14774
 URL: https://issues.apache.org/jira/browse/KAFKA-14774
 Project: Kafka
  Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai


Users can alter broker configuration to remove specify listeners. However, the 
removed listeners are NOT removed from `reconfigurables` list. It can result in 
the idle processors if users increases the network threads subsequently.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1639

[Apache Jenkins Server]

See 

KAFKA-1194

[Hamze HAMZE]

Dear Community,

I am currently using Apache kafka on Windows and I have stumbled upon the
problem
".timeindex ->
C:\Users\Administrator\Downloads\kafka\bin\windows\..\..\data\kafka\test1-3\.timeindex.deleted:
The process cannot access the file because it is being used by another
process."

That is the reason why I did some research and I found the following pull
request addressing this problem https://github.com/apache/kafka/pull/12331.
I am not a Java Developer and also not the one who has created this
proposal. But maybe someone has a solution or can continue on
migrating/helping with this.

Thank you in advance for your help.
Best Regards

-- 
*Hamze HAMZE*
Trainee/Apprentice/VIE
+49 (0) 90926032877
hamze.ha...@valeo.com

Valeo Schalter und Sensoren GmbH
Valeostrasse 1 - 86650 - Wemding (GERMANY)
[image: valeo] 
[image: linkedin]  [image: twitter]
 [image: youtube]
 [image: facebook]
 [image: instagram]


Sitz der Gesellschaft: 74321 Bietigheim-Bissingen
Handelsregister: Amtsgericht Stuttgart - HRB 301795
Vorsitzender des Aufsichtsrates: Dr. Andreas Heinrich
Geschäftsführer: Stiv Michael Smudja, Martin Mandry, Pierre-Yves Veltois

*This e-mail message is intended for the internal use of the intended
recipient(s) only.
The information contained herein is confidential/privileged. Its
disclosure or reproduction is strictly prohibited.
If you are not the intended recipient, please inform the sender
immediately, do not disclose it internally or to third parties and
destroy it.

In the course of our business relationship and for business purposes
only, Valeo may need to process some of your personal data.
For more information, please refer to the Valeo Data Protection
Statement and Privacy notice available on Valeo.com
*

-- 
*This e-mail message is intended for the internal use of the intended 
recipient(s) only.
The information contained herein is 
confidential/privileged. Its disclosure or reproduction is strictly 
prohibited.
If you are not the intended recipient, please inform the sender 
immediately, do not disclose it internally or to third parties and destroy 
it.

In the course of our business relationship and for business purposes 
only, Valeo may need to process some of your personal data. 
For more 
information, please refer to the Valeo Data Protection Statement and 
Privacy notice available on Valeo.com 
*

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1638

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 520943 lines...]
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testControllerManagementMethods() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testControllerManagementMethods() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testTopicAssignmentMethods() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testTopicAssignmentMethods() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testConnectionViaNettyClient() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testConnectionViaNettyClient() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testPropagateIsrChanges() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testPropagateIsrChanges() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testControllerEpochMethods() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testControllerEpochMethods() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testDeleteRecursive() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testDeleteRecursive() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testGetTopicPartitionStates() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testGetTopicPartitionStates() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testCreateConfigChangeNotification() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testCreateConfigChangeNotification() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testDelegationTokenMethods() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > KafkaZkClientTest > testDelegationTokenMethods() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testUpdateExistingPartitions() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testUpdateExistingPartitions() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testEmptyWrite() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testEmptyWrite() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testExistingKRaftControllerClaim() 
STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testExistingKRaftControllerClaim() PASSED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.472Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testReadAndWriteProducerId() STARTED
[2023-03-02T11:24:25.472Z] 
[2023-03-02T11:24:25.473Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > testReadAndWriteProducerId() PASSED
[2023-03-02T11:24:25.473Z] 
[2023-03-02T11:24:25.473Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 173 > ZkMigrationClientTest > 

[DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

[Sagar]

Hi All,

I wanted to create a discussion thread for KIP-910:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records

Thanks!
Sagar.

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1637

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 538533 lines...]
[2023-03-02T08:28:38.254Z] 
[2023-03-02T08:28:38.254Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testAclMethods() STARTED
[2023-03-02T08:28:39.181Z] 
[2023-03-02T08:28:39.181Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testAclMethods() PASSED
[2023-03-02T08:28:39.181Z] 
[2023-03-02T08:28:39.181Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testCreateSequentialPersistentPath() STARTED
[2023-03-02T08:28:39.181Z] 
[2023-03-02T08:28:39.181Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testCreateSequentialPersistentPath() PASSED
[2023-03-02T08:28:39.181Z] 
[2023-03-02T08:28:39.181Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testConditionalUpdatePath() STARTED
[2023-03-02T08:28:39.181Z] 
[2023-03-02T08:28:39.181Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testConditionalUpdatePath() PASSED
[2023-03-02T08:28:39.181Z] 
[2023-03-02T08:28:39.181Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testGetAllTopicsInClusterTriggersWatch() 
STARTED
[2023-03-02T08:28:40.108Z] 
[2023-03-02T08:28:40.108Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testGetAllTopicsInClusterTriggersWatch() 
PASSED
[2023-03-02T08:28:40.108Z] 
[2023-03-02T08:28:40.108Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testDeleteTopicZNode() STARTED
[2023-03-02T08:28:40.108Z] 
[2023-03-02T08:28:40.108Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testDeleteTopicZNode() PASSED
[2023-03-02T08:28:40.108Z] 
[2023-03-02T08:28:40.108Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testDeletePath() STARTED
[2023-03-02T08:28:40.108Z] 
[2023-03-02T08:28:40.108Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testDeletePath() PASSED
[2023-03-02T08:28:40.108Z] 
[2023-03-02T08:28:40.108Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testGetBrokerMethods() STARTED
[2023-03-02T08:28:41.034Z] 
[2023-03-02T08:28:41.034Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testGetBrokerMethods() PASSED
[2023-03-02T08:28:41.034Z] 
[2023-03-02T08:28:41.034Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testJuteMaxBufffer() STARTED
[2023-03-02T08:28:41.961Z] 
[2023-03-02T08:28:41.961Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testJuteMaxBufffer() PASSED
[2023-03-02T08:28:41.961Z] 
[2023-03-02T08:28:41.961Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testCreateTokenChangeNotification() STARTED
[2023-03-02T08:28:41.961Z] 
[2023-03-02T08:28:41.961Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testCreateTokenChangeNotification() PASSED
[2023-03-02T08:28:41.961Z] 
[2023-03-02T08:28:41.961Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testGetTopicsAndPartitions() STARTED
[2023-03-02T08:28:41.961Z] 
[2023-03-02T08:28:41.961Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testGetTopicsAndPartitions() PASSED
[2023-03-02T08:28:41.961Z] 
[2023-03-02T08:28:41.961Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testChroot(boolean) > 
kafka.zk.KafkaZkClientTest.testChroot(boolean)[1] STARTED
[2023-03-02T08:28:42.888Z] 
[2023-03-02T08:28:42.888Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testChroot(boolean) > 
kafka.zk.KafkaZkClientTest.testChroot(boolean)[1] PASSED
[2023-03-02T08:28:42.888Z] 
[2023-03-02T08:28:42.888Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testChroot(boolean) > 
kafka.zk.KafkaZkClientTest.testChroot(boolean)[2] STARTED
[2023-03-02T08:28:43.814Z] 
[2023-03-02T08:28:43.814Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testChroot(boolean) > 
kafka.zk.KafkaZkClientTest.testChroot(boolean)[2] PASSED
[2023-03-02T08:28:43.814Z] 
[2023-03-02T08:28:43.814Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testRegisterBrokerInfo() STARTED
[2023-03-02T08:28:43.814Z] 
[2023-03-02T08:28:43.814Z] Gradle Test Run :core:integrationTest > Gradle Test 
Executor 175 > KafkaZkClientTest > testRegisterBrokerInfo() PASSED