Re: Permission to contribute to Apache Kafka

2024-05-22 Thread Yash Mayya
Hi Frédérik, I've granted you the necessary permissions. Let me know if something doesn't work as expected. Cheers, Yash On Wed, May 22, 2024 at 1:38 PM Frédérik Rouleau wrote: > Hi, > As I now have my wiki Id: frouleau and my Jira Id: fred-ro, can I have the > permission to contribute to KIP

Re: [ANNOUNCE] New committer: Igor Soarez

2024-04-24 Thread Yash Mayya
Congratulations Igor! On Wed, 24 Apr, 2024, 23:36 Colin McCabe, wrote: > Hi all, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer, Igor > Soarez. > > Igor has been a Kafka contributor since 2019. In addition to being a > regular contributor and reviewer, he has made

Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

2024-04-13 Thread Yash Mayya
Congrats Greg! On Sun, 14 Apr, 2024, 05:56 Randall Hauch, wrote: > Congratulations, Greg! > > On Sat, Apr 13, 2024 at 6:36 PM Luke Chen wrote: > > > Congrats, Greg! > > > > On Sun, Apr 14, 2024 at 7:05 AM Viktor Somogyi-Vass > > wrote: > > > > > Congrats Greg! :) > > > > > > On Sun, Apr 14,

Re: [VOTE] KIP-477: Add PATCH method for connector config in Connect REST API

2024-04-12 Thread Yash Mayya
Hi Ivan, Thanks for reviving this KIP, I think it will be a useful addition to Connect! +1 (binding) Cheers, Yash On Tue, Apr 9, 2024 at 4:23 AM Knowles Atchison Jr wrote: > +1 (non binding) > > On Mon, Apr 8, 2024, 3:30 PM Chris Egerton > wrote: > > > Thanks Ivan! +1 (binding) from me. > >

Re: Permission to assign tickets in Jira

2024-03-27 Thread Yash Mayya
Hi Pavel, I've granted you the necessary permissions. Thanks for your interest in contributing to the Apache Kafka project! Cheers, Yash On Tue, Mar 26, 2024 at 11:32 PM Pavel Pozdeev wrote: > > Hi Team, > > Would it be possible to get a permission to assign tickets in Jira? > I've got a Jira

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Yash Mayya
Congratulations Christo! On Tue, Mar 26, 2024 at 5:34 PM Luke Chen wrote: > Hi, Everyone, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer: > Christo Lolov. > > Christo has been a Kafka contributor since 2021. He has made over 50 > commits. He authored KIP-902, KIP-963,

Re: [VOTE] KIP-995: Allow users to specify initial offsets while creating connectors

2024-03-05 Thread Yash Mayya
at > > if initial_offsets is set in the ConnectorCreate request, we will return > a > > new REST entity (ConnectorInfoWithInitialOffsetsResponse ?) which will > be a > > child class of ConnectorInfo. > > > > ( > > > > > https://github.com/apache/kafka/blo

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2024-01-30 Thread Yash Mayya
Hi Sagar, Thanks for the KIP and apologies for the extremely long delay here! I think we could do with some wordsmithing on the Javadoc for the new `SourceTask::updateOffsets` method but that can be taken care of in the PR. +1 (binding) Thanks, Yash On Wed, Nov 15, 2023 at 11:43 PM Sagar

[jira] [Created] (KAFKA-16196) Cast transform doesn't handle invalid whole value casts gracefully

2024-01-25 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-16196: -- Summary: Cast transform doesn't handle invalid whole value casts gracefully Key: KAFKA-16196 URL: https://issues.apache.org/jira/browse/KAFKA-16196 Project: Kafka

Re: [VOTE] KIP-995: Allow users to specify initial offsets while creating connectors

2024-01-17 Thread Yash Mayya
Hi Ashwin, Thanks for the KIP. > If Connect runtime encounters an error in any of these steps, > it will cleanup (if required) and return an error response Could you please elaborate on the cleanup steps? For instance, if we encounter an error after wiping existing offsets but before writing

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-03 Thread Yash Mayya
Hi Chris, +1 (binding), thanks for the KIP. Based on discussion in other threads, it looks like the community is aligned with having a 3.8 release before the 4.0 release so we should be able to remove the 'tasks.max.enforce' connector property in 4.0 (we'd discussed potentially having to live

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-28 Thread Yash Mayya
Congratulations Divij! On Wed, Dec 27, 2023 at 5:15 PM Luke Chen wrote: > Hi, Everyone, > > Divij has been a Kafka committer since June, 2023. He has remained very > active and instructive in the community since becoming a committer. It's my > pleasure to announce that Divij is now a member of

Re: Requesting permission to contribute to Apache Kafka

2023-12-14 Thread Yash Mayya
Hi Roman, I've granted the required permissions to your accounts. Cheers, Yash On Fri, Dec 15, 2023 at 12:12 PM Роман Бондарь wrote: > Hi all, > > Please add me as a contributor to kafka project > > JIRA username: rbond > Wiki username: rbond > GitHub username: gitrbond > > Thank you, > Roman

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-22 Thread Yash Mayya
Hi Chris, Thanks for the well written and comprehensive KIP! Given that we're already past the KIP freeze deadline for 3.7.0 ( https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.7.0) and there may not be a 3.8.0 release before the 4.0.0 release, would we then be forced to punt the

[jira] [Created] (KAFKA-15888) DistributedHerder log context should not use the same client ID for each Connect worker by default

2023-11-22 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15888: -- Summary: DistributedHerder log context should not use the same client ID for each Connect worker by default Key: KAFKA-15888 URL: https://issues.apache.org/jira/browse/KAFKA-15888

Re: Requesting permission to contribute to Apache Kafka.

2023-11-07 Thread Yash Mayya
Hi Afshin, I've granted you the necessary permissions. Thanks for your interest in contributing to Apache Kafka! Cheers, Yash On Tue, Nov 7, 2023 at 3:52 PM Afshin Moazami wrote: > Hi, > I would like to request permission to contribute to Apache Kafka. > wiki ID: amoazami > Jira ID: afshing >

Re: Requesting permission to contribute to Apache Kafka

2023-11-02 Thread Yash Mayya
Hi Vedarth, I've granted you the necessary permissions. Thanks for your interest in contributing to Apache Kafka! Cheers, Yash On Thu, Nov 2, 2023 at 1:22 PM Vedarth Sharma wrote: > Hi, > > Please grant me permission to contribute to KIPs and assign jira tickets to > myself. > > wiki id and

Re: [VOTE] KIP-980: Allow creating connectors in a stopped state

2023-10-16 Thread Yash Mayya
Hi all, Thanks for participating in the discussion and voting! KIP-980 has been accepted with the following +1 votes: - Chris Egerton (binding) - Knowles Atchison Jr (non-binding) - Greg Harris (binding) - Mickael Maison (binding) - Yash Mayya (binding) The target release

Re: [VOTE] KIP-980: Allow creating connectors in a stopped state

2023-10-16 Thread Yash Mayya
3, 7:57 AM Chris Egerton > wrote: > > > > > Thanks for the KIP, Yash! > > > > > > +1 (binding) > > > > > > On Mon, Oct 9, 2023, 01:12 Yash Mayya wrote: > > > > > > > Hi all, > > > > > > &g

[jira] [Created] (KAFKA-15570) Add unit tests for MemoryConfigBackingStore

2023-10-10 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15570: -- Summary: Add unit tests for MemoryConfigBackingStore Key: KAFKA-15570 URL: https://issues.apache.org/jira/browse/KAFKA-15570 Project: Kafka Issue Type: Test

[VOTE] KIP-980: Allow creating connectors in a stopped state

2023-10-08 Thread Yash Mayya
Hi all, I'd like to start a vote on KIP-980 which proposes allowing the creation of connectors in a stopped (or paused) state. KIP - https://cwiki.apache.org/confluence/display/KAFKA/KIP-980%3A+Allow+creating+connectors+in+a+stopped+state Discussion Thread -

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-08 Thread Yash Mayya
Hi Chris, Thanks for the KIP! +1 (binding) Yash On Fri, Oct 6, 2023 at 9:54 PM Greg Harris wrote: > Hey Chris, > > Thanks for the KIP! > I think that preserving the ephemeral nature of the logging change is > the right choice here, and using the config topic for intra-cluster > broadcast is

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

2023-10-06 Thread Yash Mayya
Chris Egerton wrote: > Hi Yash, > > Yeah, I think just hardcoding with JSON-first, properties-second is fine. > IMO it's worth calling this out explicitly in the KIP. > > Apart from that, no further comments. LGTM, thanks for the KIP! > > Cheers, > > Chris > > On Th

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

2023-10-05 Thread Yash Mayya
nks! Looks great. > > 6. (Addressed) Awesome, great to hear. The point about laggy connector > startup is very convincing; my paranoia is satiated. > > > Cheers, > > Chris > > On Wed, Oct 4, 2023 at 5:35 AM Yash Mayya wrote: > > > Hi Chris, > > > >

[jira] [Resolved] (KAFKA-15547) Thread leak in MirrorMakerConfigTest#testClientConfigProperties

2023-10-04 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya resolved KAFKA-15547. Fix Version/s: 3.7.0 Resolution: Fixed > Thread leak in MirrorMakerConfigT

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

2023-10-04 Thread Yash Mayya
ing target state to the config topic and then > rely on existing logic to simply refuse to generate task configs for the > newly-created connector? Is there any chance that the lack of task configs > (as opposed to an empty list of task configs) in the config topic for a > connector that ex

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

2023-10-03 Thread Yash Mayya
> > > [1] - > > https://github.com/apache/kafka/blob/dcd8c7d05f2f22f2d815405e7ab3ad7439669239/connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java#L234-L236 > > [2] - https://kafka.apache.org/35/javadoc/index.html?overview-summary.html >

[jira] [Resolved] (KAFKA-15177) MirrorMaker 2 should implement the alterOffsets KIP-875 API

2023-09-27 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya resolved KAFKA-15177. Fix Version/s: 3.6.0 Resolution: Fixed > MirrorMaker 2 should implement the alterOffs

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

2023-09-21 Thread Yash Mayya
onnect worker handle the optional > “state” field in config topic ? > > Thanks, > Ashwin > > > > > On Sun, Sep 17, 2023 at 11:09 PM Yash Mayya wrote: > > > Hi all, > > > > I'd like to begin discussion on a KIP to allow creating connectors in a > &

[DISCUSS] KIP-980: Allow creating connectors in a stopped state

2023-09-17 Thread Yash Mayya
Hi all, I'd like to begin discussion on a KIP to allow creating connectors in a stopped state - https://cwiki.apache.org/confluence/display/KAFKA/KIP-980%3A+Allow+creating+connectors+in+a+stopped+state Thanks, Yash

[jira] [Created] (KAFKA-15470) Allow creating connectors in a stopped state

2023-09-15 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15470: -- Summary: Allow creating connectors in a stopped state Key: KAFKA-15470 URL: https://issues.apache.org/jira/browse/KAFKA-15470 Project: Kafka Issue Type: New

[jira] [Resolved] (KAFKA-14067) Sink connector override.consumer.group.id can conflict with worker group.id

2023-09-12 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya resolved KAFKA-14067. Resolution: Fixed > Sink connector override.consumer.group.id can conflict with worker group

Re: [VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-09-08 Thread Yash Mayya
for the KIP! > > Mickael > > On Wed, Sep 6, 2023 at 6:05 PM Greg Harris > wrote: > > > > Hey Yash, > > > > +1(binding) > > > > Thanks for the KIP! > > Greg > > > > On Wed, Sep 6, 2023 at 6:59 AM Yash Mayya wrote: > > > > > &

Re: [VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-09-06 Thread Yash Mayya
Hi all, I just wanted to bump up this vote thread. Thanks to everyone who's voted so far - we have 1 binding +1 vote and 3 non-binding +1 votes so far. Thanks, Yash On Wed, Aug 30, 2023 at 11:14 PM Sagar wrote: > +1 (non - binding). > > Thanks ! > Sagar. > > On Wed, 30 Aug 2023 at 11:09 PM,

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-06 Thread Yash Mayya
Hi Chris, Thanks for the clarification on the last modified timestamp tracking here and on the KIP, things look good to me now. On the persistence front, I hadn't considered the interplay between levels set by the log level REST APIs and those set by the log4j configuration files, and the user

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-04 Thread Yash Mayya
nt for the fact that at worker startup we'll still have loggers with non-null log levels - the root logger and any other named loggers which have explicit log levels configured in the log4j properties. On Mon, Sep 4, 2023 at 12:00 PM Yash Mayya wrote: > Hi Chris, > > Thanks for the KIP,

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-04 Thread Yash Mayya
Hi Chris, Thanks for the KIP, this looks like a really useful addition to Kafka Connect's log level REST APIs! I have a few questions and comments: > If no modifications to the namespace have > been made since the worker was started, > they will be null If no modifications to a logging

[VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-08-30 Thread Yash Mayya
Hi all, This is the vote thread for KIP-970 which proposes deprecating (in the Apache Kafka 3.7 release) and eventually removing (in the next major Apache Kafka release - 4.0) Connect's redundant task configurations endpoint. KIP -

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-29 Thread Yash Mayya
Hi Sagar, > The size of offsets topic can be controlled by > setting appropriate topic retention values and > that is a standard practice in Kafka Kafka Connect enforces the `cleanup.policy` configuration for the offsets topic to be "compact" only (references - [1], [2]), so the topic retention

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-22 Thread Yash Mayya
other thread. The > current mechanism proposed, even though gets invoked multiple times, keeps > things simpler to reason about. > > Let me know how things look now. If it's all looking ok, I would go ahead > and create a Vote thread for the same. > > Thanks! > Sagar. > > O

[DISCUSS] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-08-22 Thread Yash Mayya
Hi all, I'd like to start a discussion thread for this KIP - https://cwiki.apache.org/confluence/display/KAFKA/KIP-970%3A+Deprecate+and+remove+Connect%27s+redundant+task+configurations+endpoint . It proposes the deprecation and eventual removal of Kafka Connect's redundant task configurations

[jira] [Created] (KAFKA-15387) Deprecate and remove Connect's duplicate task configurations retrieval endpoint

2023-08-21 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15387: -- Summary: Deprecate and remove Connect's duplicate task configurations retrieval endpoint Key: KAFKA-15387 URL: https://issues.apache.org/jira/browse/KAFKA-15387 Project

[jira] [Created] (KAFKA-15377) GET /connectors/{connector}/tasks-config endpoint exposes externalized secret values

2023-08-18 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15377: -- Summary: GET /connectors/{connector}/tasks-config endpoint exposes externalized secret values Key: KAFKA-15377 URL: https://issues.apache.org/jira/browse/KAFKA-15377

Re: Apache Kafka 3.6.0 release

2023-07-26 Thread Yash Mayya
Hi Hector, KIP-959 actually still requires 2 more binding votes to be accepted ( https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals). The non-binding votes from people who aren't committers (including myself) don't count towards the required lazy majority. Thanks, Yash On

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

2023-07-25 Thread Yash Mayya
Hi Hector, Thanks for the KIP! +1 (non-binding) Thanks, Yash On Tue, Jul 25, 2023 at 11:01 PM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Thanks for the KIP. As you say, not that controversial. > > +1 (non-binding) > > Thanks, > Andrew > > > On 25 Jul 2023, at 18:22, Hector

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-07-25 Thread Yash Mayya
so frequently when > >> exactly-once support is disabled? Wouldn't it be simpler both for our > >> implementation and for connector developers if we only invoked it > directly > >> before committing offsets, instead of potentially several times between > >> offset com

[jira] [Created] (KAFKA-15238) Connect workers can be disabled by DLQ related stuck admin client calls

2023-07-24 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15238: -- Summary: Connect workers can be disabled by DLQ related stuck admin client calls Key: KAFKA-15238 URL: https://issues.apache.org/jira/browse/KAFKA-15238 Project: Kafka

[jira] [Created] (KAFKA-15216) InternalSinkRecord::newRecord method ignores the headers argument

2023-07-19 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15216: -- Summary: InternalSinkRecord::newRecord method ignores the headers argument Key: KAFKA-15216 URL: https://issues.apache.org/jira/browse/KAFKA-15216 Project: Kafka

[jira] [Created] (KAFKA-15182) Normalize offsets before invoking SourceConnector::alterOffsets

2023-07-12 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15182: -- Summary: Normalize offsets before invoking SourceConnector::alterOffsets Key: KAFKA-15182 URL: https://issues.apache.org/jira/browse/KAFKA-15182 Project: Kafka

[jira] [Created] (KAFKA-15179) Add integration tests for the FileStream Sink and Source connectors

2023-07-11 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15179: -- Summary: Add integration tests for the FileStream Sink and Source connectors Key: KAFKA-15179 URL: https://issues.apache.org/jira/browse/KAFKA-15179 Project: Kafka

[jira] [Created] (KAFKA-15177) MirrorMaker 2 should implement the alterOffsets KIP-875 API

2023-07-11 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15177: -- Summary: MirrorMaker 2 should implement the alterOffsets KIP-875 API Key: KAFKA-15177 URL: https://issues.apache.org/jira/browse/KAFKA-15177 Project: Kafka

Re: [ANNOUNCE] New committer: Greg Harris

2023-07-10 Thread Yash Mayya
Congrats Greg! On Mon, Jul 10, 2023 at 9:15 PM Chris Egerton wrote: > Hi all, > > The PMC for Apache Kafka has invited Greg Harris to become a committer, and > we are happy to announce that he has accepted! > > Greg has been contributing to Kafka since 2019. He has made over 50 commits > mostly

Re: [VOTE] KIP-882: Kafka Connect REST API timeout improvements

2023-07-10 Thread Yash Mayya
with a public discussion and vote. Thanks, Yash On Wed, Mar 1, 2023 at 6:32 PM Yash Mayya wrote: > Hi all, > > I'd like to call for a vote on the (hopefully) straightforward KIP-882 > which adds support for configuring request timeouts on Kafka Connect REST > APIs via query p

Re: Apache Kafka 3.6.0 release

2023-07-06 Thread Yash Mayya
Hi Satish, KIP-793 [1] just passed voting and we should be able to wrap up the implementation in time for the 3.6.0 feature freeze. Could we add it to the release plan? [1] - https://cwiki.apache.org/confluence/display/KAFKA/KIP-793%3A+Allow+sink+connectors+to+be+used+with+topic-mutating+SMTs

Re: [VOTE] KIP-793: Allow sink connectors to be used with topic-mutating SMTs

2023-07-06 Thread Yash Mayya
; > On Mon, Jul 3, 2023 at 7:19 AM Chris Egerton > > wrote: > > > > > > > > Hi Yash, > > > > > > > > Thanks for the KIP! +1 (binding) > > > > > > > > Cheers, > > > > > > > > Chris > > > > >

Re: [DISCUSS] KIP-943: Add independent "offset.storage.segment.bytes" for connect-distributed.properties

2023-07-06 Thread Yash Mayya
quot; (via the existing KIP-605 based mechanism). On Thu, Jul 6, 2023 at 6:04 PM Yash Mayya wrote: > Hi hudeqi, > > Thanks for the KIP! Just to clarify - since KIP-605 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-605%3A+Expand+Connect+Worker+Internal+Topic+Settings) > a

Re: [DISCUSS] KIP-943: Add independent "offset.storage.segment.bytes" for connect-distributed.properties

2023-07-06 Thread Yash Mayya
Hi hudeqi, Thanks for the KIP! Just to clarify - since KIP-605 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-605%3A+Expand+Connect+Worker+Internal+Topic+Settings) already allows configuring "segment.bytes" for the Connect cluster's offsets topic via a worker configuration

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-07-04 Thread Yash Mayya
fluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records > >>> with a newer approach which shelves the need for an explicit topic. > >>> > >>> Please review again and let me know what you think. > >>>

[jira] [Created] (KAFKA-15145) AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions

2023-07-04 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15145: -- Summary: AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions Key: KAFKA-15145 URL: https://issues.apache.org/jira/browse/KAFKA-15145

[VOTE] KIP-793: Allow sink connectors to be used with topic-mutating SMTs

2023-07-03 Thread Yash Mayya
Hi all, I'd like to start a vote on KIP-793 which enables sink connector implementations to be used with SMTs that mutate the topic / partition / offset information of a record. KIP -

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-07-03 Thread Yash Mayya
nkTask::preCommit. > > [1] - > > https://github.com/apache/kafka/blob/12be344fdd3b20f338ccab87933b89049ce202a4/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SubmittedRecords.java > > Cheers, > > Chris > > On Wed, Jun 21, 2023 at 9:46 AM Yash Mayya wro

[jira] [Created] (KAFKA-15121) FileStreamSourceConnector and FileStreamSinkConnector should implement KIP-875 APIs

2023-06-26 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15121: -- Summary: FileStreamSourceConnector and FileStreamSinkConnector should implement KIP-875 APIs Key: KAFKA-15121 URL: https://issues.apache.org/jira/browse/KAFKA-15121

[jira] [Created] (KAFKA-15113) Gracefully handle cases where a sink connector's admin and consumer client config overrides target different Kafka clusters

2023-06-22 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-15113: -- Summary: Gracefully handle cases where a sink connector's admin and consumer client config overrides target different Kafka clusters Key: KAFKA-15113 URL: https://issues.apache.org

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-06-21 Thread Yash Mayya
etting the KIP go dormant, I'd happily choose the > former. > > Thanks Yash and Greg for the discussion so far, and apologies for the wall > of text. Looking forward to your thoughts. > > Cheers, > > Chris > > [1] - > > https://kafka.apache.org/34/javadoc/org/apache

Re: [ANNOUNCE] New committer: Divij Vaidya

2023-06-13 Thread Yash Mayya
Congratulations Divij! On Tue, Jun 13, 2023 at 9:20 PM Bruno Cadonna wrote: > Hi all, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer > Divij Vaidya. > > Divij's major contributions are: > > GDPR compliance enforcement of kafka-site - >

Re: [VOTE] 3.4.1 RC3

2023-06-06 Thread Yash Mayya
Hi Prem, You can follow the instructions here if you wish to unsubscribe from one or more mailing lists - https://kafka.apache.org/contact On Tue, Jun 6, 2023 at 2:44 PM Prem Sagar wrote: > Please remove my mail ID from the database. > > > Warm Regards > K Prem Sagar > Sr.Manager -

[jira] [Resolved] (KAFKA-14956) Flaky test org.apache.kafka.connect.integration.OffsetsApiIntegrationTest#testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted

2023-05-29 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya resolved KAFKA-14956. Resolution: Fixed > Flaky test > org.apache.kafka.connect.integration.OffsetsApiIntegrati

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-05-24 Thread Yash Mayya
gt; > > > > > Just a quick update: after merging the changes to asf.yaml, I > received > > a > > > > notification that the list is limited to only 10 people, not 20 as > the > > > > documentation states. > > > > > > > > Here is t

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-05-24 Thread Yash Mayya
es to asf.yaml, I received a > > notification that the list is limited to only 10 people, not 20 as the > > documentation states. > > > > Here is the list of folks who will now be able to triage PRs and trigger > > builds: Victoria Xia, Greg Harris, Divij Vaidya, Lucas B

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-05-12 Thread Yash Mayya
ortlog). Congratulations to our new collaborators! > > Victoria Xia, Greg Harris, Divij Vaidya, Lucas Brutschy, Yash Mayya, > Philip Nee, vamossagar12,, Christo Lolov, Federico Valeri, andymg3, > RivenSun, Kirk True, Matthew de Detrich, Akhilesh C, Alyssa Huang, Artem > Livshits, Gan

[jira] [Created] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14974: -- Summary: Restore backward compatibility in KafkaBasedLog Key: KAFKA-14974 URL: https://issues.apache.org/jira/browse/KAFKA-14974 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-14933) Document Kafka Connect's log level REST APIs added in KIP-495

2023-04-25 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14933: -- Summary: Document Kafka Connect's log level REST APIs added in KIP-495 Key: KAFKA-14933 URL: https://issues.apache.org/jira/browse/KAFKA-14933 Project: Kafka

Re: [ANNOUNCE] New PMC chair: Mickael Maison

2023-04-24 Thread Yash Mayya
Congratulations Mickael! On Fri, Apr 21, 2023 at 8:39 PM Jun Rao wrote: > Hi, everyone, > > After more than 10 years, I am stepping down as the PMC chair of Apache > Kafka. We now have a new chair Mickael Maison, who has been a PMC member > since 2020. I plan to continue to contribute to Apache

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-04-24 Thread Yash Mayya
Hi Sagar, Thanks for the KIP! I have a few questions and comments: 1) I agree with Chris' point about the separation of a connector heartbeat mechanism and allowing source connectors to generate offsets without producing data. What is the purpose of the heartbeat topic here and are there any

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-04-23 Thread Yash Mayya
ntation of metadata book-keeping which > provides a reasonable system of virtual coordinates, it just ended up > equivalent to hydrating intermediate topics to compute a consistent record > ordering. I wasn't convinced by calling it "book-keeping" since i've seen >

[jira] [Created] (KAFKA-14910) Consider cancelling ongoing alter connector offsets requests when the connector is resumed

2023-04-14 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14910: -- Summary: Consider cancelling ongoing alter connector offsets requests when the connector is resumed Key: KAFKA-14910 URL: https://issues.apache.org/jira/browse/KAFKA-14910

Re: [VOTE] KIP-875: First-class offsets support in Kafka Connect

2023-04-12 Thread Yash Mayya
fire-and-forget the request and, if new tasks are started for the connector > in the meantime, trust that the request will get automatically fenced out > without doing any more work). > > Cheers, > > Chris > > On Wed, Apr 12, 2023 at 1:01 PM Yash Mayya wrote: > > >

Re: [VOTE] KIP-875: First-class offsets support in Kafka Connect

2023-04-12 Thread Yash Mayya
>> > >> >> > >> Best, >> > >> >> > >> On Wed, Mar 1, 2023 at 12:16 PM Tom Bentley >> > wrote: >> > >> >> > >> > Hi Chris, >> > >> > >> > >> > Thanks for the K

[jira] [Created] (KAFKA-14876) Public documentation for new Kafka Connect offset management REST APIs

2023-04-02 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14876: -- Summary: Public documentation for new Kafka Connect offset management REST APIs Key: KAFKA-14876 URL: https://issues.apache.org/jira/browse/KAFKA-14876 Project: Kafka

[jira] [Created] (KAFKA-14844) Kafka Connect's OffsetBackingStore interface should handle (de)serialization and connector namespacing

2023-03-24 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14844: -- Summary: Kafka Connect's OffsetBackingStore interface should handle (de)serialization and connector namespacing Key: KAFKA-14844 URL: https://issues.apache.org/jira/browse/KAFKA

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

2023-03-19 Thread Yash Mayya
er). > But as it stands, I don't think I see the value-add of configurable > timeouts once there is another solution which targets the validate call > duration specifically, and I'm concerned that configurable timeouts would > be made irrelevant in that case. > > Thanks, >

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-03-12 Thread Yash Mayya
deriving all of the above, I think that's a reasonable > tradeoff to make. > > Thanks, > Greg > > On Tue, Feb 21, 2023 at 10:17 AM Chris Egerton > wrote: > > > Hi Yash, > > > > We'll probably want to make a few tweaks to the Javadocs for the new > > meth

Re: [ANNOUNCE] New Kafka PMC Member: Chris Egerton

2023-03-09 Thread Yash Mayya
Congratulations Chris! On Thu, Mar 9, 2023, 23:42 Jun Rao wrote: > Hi, Everyone, > > Chris Egerton has been a Kafka committer since July 2022. He has been very > instrumental to the community since becoming a committer. It's my pleasure > to announce that Chris is now a member of Kafka PMC. > >

Re: [ANNOUNCE] New Kafka PMC Member: David Arthur

2023-03-09 Thread Yash Mayya
Congrats David! On Thu, Mar 9, 2023, 23:42 Jun Rao wrote: > Hi, Everyone, > > David Arthur has been a Kafka committer since 2013. He has been very > instrumental to the community since becoming a committer. It's my pleasure > to announce that David is now a member of Kafka PMC. > >

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

2023-03-08 Thread Yash Mayya
traint. > Again, I'm not necessarily saying that you must implement such an > asynchronous validation scheme in this KIP, but we should consider if that > is a more extensible solution. If we decided to implement configurable > synchronous timeouts now, how would that complement an asynchronous API i

Re: [VOTE] KIP-882: Kafka Connect REST API timeout improvements

2023-03-02 Thread Yash Mayya
gt; Is it too late to incorporate this change? > > Thanks, > Ashwin > > > On Wed, Mar 1, 2023 at 6:32 PM Yash Mayya wrote: > > > Hi all, > > > > I'd like to call for a vote on the (hopefully) straightforward KIP-882 > > which adds support for configur

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

2023-03-02 Thread Yash Mayya
t to keep this KIP as tightly focused as possible, > but i'm worried that it is addressing the symptom and not the problem. I > want to make sure that this change is impactful and isn't obsoleted by a > later improvement. > > Thanks, > Greg > > > On Wed, Mar 1, 2023 at 5:

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

2023-03-01 Thread Yash Mayya
org/thread/mgx8lczx2f57pk7x3vh0nqk00s79grgp. Thanks, Yash On Sat, Nov 5, 2022 at 11:42 PM Sagar wrote: > Hey Yash, > > Thanks for the explanation. I think it should be fine to delegate the > validation directly to the leader. > > Thanks! > Sagar. > > On Sat, Nov 5, 2022 at 10:42 AM Yash Mayya

[VOTE] KIP-882: Kafka Connect REST API timeout improvements

2023-03-01 Thread Yash Mayya
Hi all, I'd like to call for a vote on the (hopefully) straightforward KIP-882 which adds support for configuring request timeouts on Kafka Connect REST APIs via query parameters along with a couple of related small improvements. KIP -

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-02-21 Thread Yash Mayya
k class has not overridden the > multi-arg variant. But I think this is going a bit too far and would prefer > to keep things simple(r) for now. > > Cheers, > > Chris > > > On Sun, Feb 19, 2023 at 2:34 AM Yash Mayya wrote: > > > Hi Chris, > > > &g

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-02-18 Thread Yash Mayya
e everything looks good enough before opening a vote > thread. > > Finally, I think you make a convincing case for a time-based eviction > policy. I wasn't thinking about the fairly common SMT pattern of deriving a > topic name from, e.g., a record field or header. > > Cheers

[jira] [Created] (KAFKA-14732) Use an exponential backoff retry mechanism while reconfiguring connector tasks

2023-02-18 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14732: -- Summary: Use an exponential backoff retry mechanism while reconfiguring connector tasks Key: KAFKA-14732 URL: https://issues.apache.org/jira/browse/KAFKA-14732 Project

Re: [ANNOUNCE] New committer: Lucas Bradstreet

2023-02-16 Thread Yash Mayya
Congratulations Lucas! On Fri, Feb 17, 2023 at 3:25 AM Jun Rao wrote: > Hi, Everyone, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer Lucas > Bradstreet. > > Lucas has been a long time Kafka contributor since Oct. 2018. He has been > extremely valuable for Kafka on both

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-02-14 Thread Yash Mayya
nteed resource leaks or > failure to obey API contract in any cases. > > Cheers, > > Chris > > On Mon, Feb 13, 2023 at 10:54 AM Yash Mayya wrote: > > > Hi Chris, > > > > > especially if connectors are intentionally designed around > > > original t

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-02-13 Thread Yash Mayya
ourselves into a corner with the KIP, we give ourselves leeway to > tweak it in the future if necessary without filing another KIP or > introducing a pluggable interface. > > Cheers, > > Chris > > On Thu, Feb 9, 2023 at 7:39 AM Yash Mayya wrote: > > > Hi

Re: [DISCUSS] KIP-793: Sink Connectors: Support topic-mutating SMTs for async connectors (preCommit users)

2023-02-09 Thread Yash Mayya
afka/connect/sink/SinkTask.html#open(java.util.Collection) > [2] - > > https://kafka.apache.org/33/javadoc/org/apache/kafka/connect/sink/SinkTask.html#close(java.util.Collection) > > Cheers, > > Chris > > On Sat, Nov 5, 2022 at 5:46 AM Yash Mayya wrote: > > > Hi Chr

Re: [DISCUSS] KIP-877: Mechanism for plugins and connectors to register metrics

2023-02-03 Thread Yash Mayya
> > > > > Thanks, > > > Mickael > > > > > > > > > On Thu, Jan 26, 2023 at 3:54 PM Mickael Maison < > mickael.mai...@gmail.com> > > wrote: > > > > > > > > Hi Chris, > > > > > > > > 1) I update

Re: [VOTE] KIP-875: First-class offsets support in Kafka Connect

2023-01-24 Thread Yash Mayya
Hi Chris, I'm +1 (non-binding). Thanks again for proposing this extremely valuable addition to Kafka Connect! Thanks, Yash On Thu, Jan 19, 2023 at 12:11 AM Chris Egerton wrote: > Hi all, > > I'd like to call for a vote on KIP-875, which adds support for viewing and > manipulating the offsets

Re: [DISCUSS] KIP-877: Mechanism for plugins and connectors to register metrics

2023-01-24 Thread Yash Mayya
; a list of all plugin types and add that to the KIP. > > > > Thanks, > > Mickael > > > > > > > > On Tue, Dec 27, 2022 at 4:54 PM Chris Egerton > wrote: > > > > > > Hi Yash, > > > > > > Yes, a default no-op is ex

Re: [ANNOUNCE] New committer: Walker Carlson

2023-01-17 Thread Yash Mayya
Congrats, Walker! On Wed, Jan 18, 2023 at 3:27 AM Matthias J. Sax wrote: > Dear community, > > I am pleased to announce Walker Carlson as a new Kafka committer. > > Walker has been contributing to Apache Kafka since November 2019. He > made various contributions including the following KIPs. >

  1   2   >