Two potential bugs in Flink ML

[Komal M]

Hi Flink Dev Team,
I have two possible bugs to report for Flink ML Iteration.
Flink v1.17.2
Flink ML v2.3.0
Java 11

Bug # 1
Implementing a UDF KeyedRichCoProcessFunction or CoFlatMapFunction inside 
IterationBody yields a “java.lang.ClassCastException: 
org.apache.flink.iteration.IterationRecord cannot be cast to class 
org.apache.flink.api.java.tuple.Tuple” error. For reference, I do not get any 
error when applying  .keyBy().flatMap()on the streams individually inside the 
iteration body.

Exception in thread "main" 
org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at 
org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
….
at 
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by 
NoRestartBackoffTimeStrategy
at 
org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:139)
…
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
... 5 more
Caused by: java.lang.ClassCastException: class 
org.apache.flink.iteration.IterationRecord cannot be cast to class 
org.apache.flink.api.java.tuple.Tuple 
(org.apache.flink.iteration.IterationRecord and 
org.apache.flink.api.java.tuple.Tuple are in unnamed module of loader 'app')
at 
org.apache.flink.api.java.typeutils.runtime.TupleComparator.extractKeys(TupleComparator.java:148)
at 
org.apache.flink.streaming.util.keys.KeySelectorUtil$ComparableKeySelector.getKey(KeySelectorUtil.java:195)
at 
org.apache.flink.streaming.util.keys.KeySelectorUtil$ComparableKeySelector.getKey(KeySelectorUtil.java:168)
at 
org.apache.flink.streaming.api.operators.AbstractStreamOperator.setKeyContextElement(AbstractStreamOperator.java:502)
at 
org.apache.flink.streaming.api.operators.AbstractStreamOperator.setKeyContextElement1(AbstractStreamOperator.java:478)
at 
org.apache.flink.iteration.operator.allround.AbstractAllRoundWrapperOperator.setKeyContextElement1(AbstractAllRoundWrapperOperator.java:203)
at 
org.apache.flink.streaming.runtime.io.RecordProcessorUtils.lambda$getRecordProcessor1$1(RecordProcessorUtils.java:87)
at 
org.apache.flink.streaming.runtime.io.StreamTwoInputProcessorFactory$StreamTaskNetworkOutput.emitRecord(StreamTwoInputProcessorFactory.java:254)
at 
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
at 
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
at 
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at 
org.apache.flink.streaming.runtime.io.StreamMultipleInputProcessor.processInput(StreamMultipleInputProcessor.java:85)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550)
at 
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788)
at 
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952)
at 
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931)
at 
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.base/java.lang.Thread.run(Thread.java:829)



Potential Bug # 2

The onEpochWatermarkIncremented method is never invoked when the 
IterationListener interface is implemented by a UDF inside the iterationBody.



// method is invoked from within IterationBody

public class ComputeML2 extends KeyedProcessFunction, Tuple2> implements IterationListener>  {

// this method is never invoked, getting no output

@Override

public void onEpochWatermarkIncremented(int epochWaterMark, 
IterationListener.Context context, Collector> 
collector) throws Exception {

collector.collect(Tuple2.of(epochWaterMark,"epoch"));  //Bug: no output

}




@Override

public void onIterationTerminated(IterationListener.Context context, 
Collector> collector) throws Exception {

}


@Override

public void processElement(Tuple2 integerStringTuple2, 
KeyedProcessFunction, Tuple2>.Context context, Collector> collector) throws 
Exception {

// some processing here

}


}




Let me know if I should submit 

[jira] [Created] (FLINK-35020) Model Catalog implementation in Hive etc

[Hao Li (Jira)]

Hao Li created FLINK-35020:
--

 Summary: Model Catalog implementation in Hive etc
 Key: FLINK-35020
 URL: https://issues.apache.org/jira/browse/FLINK-35020
 Project: Flink
  Issue Type: Sub-task
Reporter: Hao Li






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35019) Support show create model syntax

[Hao Li (Jira)]

Hao Li created FLINK-35019:
--

 Summary: Support show create model syntax
 Key: FLINK-35019
 URL: https://issues.apache.org/jira/browse/FLINK-35019
 Project: Flink
  Issue Type: Sub-task
Reporter: Hao Li


show options in addition to input/output schema



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35017) ML_PREDICT function

[Hao Li (Jira)]

Hao Li created FLINK-35017:
--

 Summary: ML_PREDICT function
 Key: FLINK-35017
 URL: https://issues.apache.org/jira/browse/FLINK-35017
 Project: Flink
  Issue Type: Sub-task
Reporter: Hao Li






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35018) ML_EVALUATE function

[Hao Li (Jira)]

Hao Li created FLINK-35018:
--

 Summary: ML_EVALUATE function
 Key: FLINK-35018
 URL: https://issues.apache.org/jira/browse/FLINK-35018
 Project: Flink
  Issue Type: Sub-task
Reporter: Hao Li






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35016) Catalog changes for model CRUD

[Hao Li (Jira)]

Hao Li created FLINK-35016:
--

 Summary: Catalog changes for model CRUD
 Key: FLINK-35016
 URL: https://issues.apache.org/jira/browse/FLINK-35016
 Project: Flink
  Issue Type: Sub-task
Reporter: Hao Li






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35015) Flink Parquet Reader doesn't honor parquet configuration

[Zhenqiu Huang (Jira)]

Zhenqiu Huang created FLINK-35015:
-

 Summary: Flink Parquet Reader doesn't honor parquet configuration
 Key: FLINK-35015
 URL: https://issues.apache.org/jira/browse/FLINK-35015
 Project: Flink
  Issue Type: Bug
  Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Affects Versions: 1.18.1, 1.19.0, 1.17.2
Reporter: Zhenqiu Huang
 Fix For: 1.20.0


For example, To access parquet files in legacy standard, users to need to use 
READ_INT96_AS_FIXED flag to read deprecated INT96 columns. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35014) SqlNode to operation conversion for models

[Hao Li (Jira)]

Hao Li created FLINK-35014:
--

 Summary: SqlNode to operation conversion for models
 Key: FLINK-35014
 URL: https://issues.apache.org/jira/browse/FLINK-35014
 Project: Flink
  Issue Type: Sub-task
Reporter: Hao Li






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35013) Support temporary model

[Hao Li (Jira)]

Hao Li created FLINK-35013:
--

 Summary: Support temporary model
 Key: FLINK-35013
 URL: https://issues.apache.org/jira/browse/FLINK-35013
 Project: Flink
  Issue Type: Sub-task
Reporter: Hao Li






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35012) ChangelogNormalizeRestoreTest.testRestore failure

[Ryan Skraba (Jira)]

Ryan Skraba created FLINK-35012:
---

 Summary: ChangelogNormalizeRestoreTest.testRestore failure
 Key: FLINK-35012
 URL: https://issues.apache.org/jira/browse/FLINK-35012
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Planner
Affects Versions: 1.20.0
Reporter: Ryan Skraba


https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58716&view=logs&j=0c940707-2659-5648-cbe6-a1ad63045f0a&t=075c2716-8010-5565-fe08-3c4bb45824a4&l=11921

{code}
Apr 03 22:57:43 22:57:43.159 [ERROR] Failures: 
Apr 03 22:57:43 22:57:43.160 [ERROR]   
ChangelogNormalizeRestoreTest>RestoreTestBase.testRestore:337 
Apr 03 22:57:43 Expecting actual:
Apr 03 22:57:43   ["+I[two, 2, b]",
Apr 03 22:57:43 "+I[one, 1, a]",
Apr 03 22:57:43 "+I[three, 3, c]",
Apr 03 22:57:43 "-U[one, 1, a]",
Apr 03 22:57:43 "+U[one, 1, aa]",
Apr 03 22:57:43 "-U[three, 3, c]",
Apr 03 22:57:43 "+U[three, 3, cc]",
Apr 03 22:57:43 "-D[two, 2, b]",
Apr 03 22:57:43 "+I[four, 4, d]",
Apr 03 22:57:43 "+I[five, 5, e]",
Apr 03 22:57:43 "-U[four, 4, d]",
Apr 03 22:57:43 "+U[four, 4, dd]"]
Apr 03 22:57:43 to contain exactly in any order:
Apr 03 22:57:43   ["+I[one, 1, a]",
Apr 03 22:57:43 "+I[two, 2, b]",
Apr 03 22:57:43 "-U[one, 1, a]",
Apr 03 22:57:43 "+U[one, 1, aa]",
Apr 03 22:57:43 "+I[three, 3, c]",
Apr 03 22:57:43 "-D[two, 2, b]",
Apr 03 22:57:43 "-U[three, 3, c]",
Apr 03 22:57:43 "+U[three, 3, cc]",
Apr 03 22:57:43 "+I[four, 4, d]",
Apr 03 22:57:43 "+I[five, 5, e]",
Apr 03 22:57:43 "-U[four, 4, d]",
Apr 03 22:57:43 "+U[four, 4, dd]",
Apr 03 22:57:43 "+I[six, 6, f]",
Apr 03 22:57:43 "-D[six, 6, f]"]
Apr 03 22:57:43 but could not find the following elements:
Apr 03 22:57:43   ["+I[six, 6, f]", "-D[six, 6, f]"]
Apr 03 22:57:43 
{code}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35011) The change in visibility of MockDeserializationSchema cause compilation failure in kafka connector

[Jiabao Sun (Jira)]

Jiabao Sun created FLINK-35011:
--

 Summary: The change in visibility of MockDeserializationSchema 
cause compilation failure in kafka connector
 Key: FLINK-35011
 URL: https://issues.apache.org/jira/browse/FLINK-35011
 Project: Flink
  Issue Type: Bug
  Components: Tests
Affects Versions: 1.20.0
Reporter: Jiabao Sun
Assignee: Jiabao Sun
 Fix For: 1.20.0


Flink Kafka connector can't compile with 1.20-SNAPSHOT, see 
https://github.com/apache/flink-connector-kafka/actions/runs/8553981349/job/23438292087?pr=90#step:15:165

Error message is:

{code}
Error:  Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile 
(default-testCompile) on project flink-connector-kafka: Compilation failure
Error:  
/home/runner/work/flink-connector-kafka/flink-connector-kafka/flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumerBaseTest.java:[60,39]
 org.apache.flink.streaming.util.MockDeserializationSchema is not public in 
org.apache.flink.streaming.util; cannot be accessed from outside package
{code}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[RESULT][VOTE] FLIP-437: Support ML Models in Flink SQL

[Hao Li]

Hi Dev,

I'm happy to announce that FLIP-437: Support ML Models in Flink SQL [1] has
been accepted with 7 approving votes (6 binding) [2]

Timo Walther (binding)
Jark Wu (binding)
Yu Chen (non-binding)
Piotr Nowojski (binding)
Leonard Xu (binding)
Martijn Visser (binding)
David Moravek (binding)

Thanks,
Hao

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-437%3A+Support+ML+Models+in+Flink+SQL
[2] https://lists.apache.org/thread/gw1hfnqb05mwrstbtw43yh5tllrscgn6

Re: [VOTE] FLIP-437: Support ML Models in Flink SQL

[Hao Li]

Hi Dev,

Thanks all for voting. I'm closing the vote and the result will be posted
in a separate email.

Thanks,
Hao

On Wed, Apr 3, 2024 at 10:24 AM Hao Li  wrote:

> Thanks David Radley and David Moravek for the comments. I'll reply in the
> discussion thread.
>
> Hao
>
> On Wed, Apr 3, 2024 at 5:45 AM David Morávek  wrote:
>
>> +1 (binding)
>>
>> My only suggestion would be to move Catalog changes into a separate
>> interface to allow us to begin with lower stability guarantees. Existing
>> Catalogs would be able to opt-in by implementing it. It's a minor thing
>> though, overall the FLIP is solid and the direction is pretty exciting.
>>
>> Best,
>> D.
>>
>> On Wed, Apr 3, 2024 at 2:31 AM David Radley 
>> wrote:
>>
>> > Hi Hao,
>> > I don’t think this counts as an objection, I have some comments. I
>> should
>> > have put this on the discussion thread earlier but have just got to
>> this.
>> > - I suggest we can put a model version in the model resource. Versions
>> are
>> > notoriously difficult to add later; I don’t think we want to proliferate
>> > differently named models as a model mutates. We may want to work with
>> > non-latest models.
>> > - I see that the model name is the unique identifier. I realise this
>> would
>> > move away from the Oracle syntax – so may not be feasible short term;
>> but I
>> > wonder if we can have:
>> >  - a uuid as the main identifier and the model name as an attribute.
>> > or
>> >  - a namespace (or something like a system of origin)
>> > to help organise models with the same name.
>> > - does the model have an owner? I assume that Flink model resource is
>> the
>> > master of the model? I imagine in the future that a model that comes in
>> via
>> > a new connector could be kept up to date with the external model and
>> would
>> > not be allowed to be changed by anything other than the connector.
>> >
>> >Kind regards, David.
>> >
>> > From: Hao Li 
>> > Date: Friday, 29 March 2024 at 16:30
>> > To: dev@flink.apache.org 
>> > Subject: [EXTERNAL] [VOTE] FLIP-437: Support ML Models in Flink SQL
>> > Hi devs,
>> >
>> > I'd like to start a vote on the FLIP-437: Support ML Models in Flink
>> > SQL [1]. The discussion thread is here [2].
>> >
>> > The vote will be open for at least 72 hours unless there is an
>> objection or
>> > insufficient votes.
>> >
>> > [1]
>> >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-437%3A+Support+ML+Models+in+Flink+SQL
>> >
>> > [2] https://lists.apache.org/thread/9z94m2bv4w265xb5l2mrnh4lf9m28ccn
>> >
>> > Thanks,
>> > Hao
>> >
>> > Unless otherwise stated above:
>> >
>> > IBM United Kingdom Limited
>> > Registered in England and Wales with number 741598
>> > Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
>> >
>>
>

[jira] [Created] (FLINK-35010) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.0 for Flink Mongodb connector

[Zhongqiang Gong (Jira)]

Zhongqiang Gong created FLINK-35010:
---

 Summary: Bump org.apache.commons:commons-compress from 1.25.0 to 
1.26.0 for Flink Mongodb connector
 Key: FLINK-35010
 URL: https://issues.apache.org/jira/browse/FLINK-35010
 Project: Flink
  Issue Type: Technical Debt
  Components: Connectors / MongoDB
Reporter: Zhongqiang Gong






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35009) Change on getTransitivePredecessors breaks connectors

[Martijn Visser (Jira)]

Martijn Visser created FLINK-35009:
--

 Summary: Change on getTransitivePredecessors breaks connectors
 Key: FLINK-35009
 URL: https://issues.apache.org/jira/browse/FLINK-35009
 Project: Flink
  Issue Type: Bug
  Components: API / Core, Connectors / Kafka
Affects Versions: 1.18.2, 1.20.0, 1.19.1
Reporter: Martijn Visser


{code:java}
Error:  Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile 
(default-testCompile) on project flink-connector-kafka: Compilation failure: 
Compilation failure: 
Error:  
/home/runner/work/flink-connector-kafka/flink-connector-kafka/flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/testutils/DataGenerators.java:[214,24]
 
org.apache.flink.streaming.connectors.kafka.testutils.DataGenerators.InfiniteStringsGenerator.MockTransformation
 is not abstract and does not override abstract method 
getTransitivePredecessorsInternal() in org.apache.flink.api.dag.Transformation
Error:  
/home/runner/work/flink-connector-kafka/flink-connector-kafka/flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/testutils/DataGenerators.java:[220,44]
 getTransitivePredecessors() in 
org.apache.flink.streaming.connectors.kafka.testutils.DataGenerators.InfiniteStringsGenerator.MockTransformation
 cannot override getTransitivePredecessors() in 
org.apache.flink.api.dag.Transformation
Error:overridden method is final
{code}

Example: 
https://github.com/apache/flink-connector-kafka/actions/runs/8494349338/job/23269406762#step:15:167





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35008) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.0 for Flink Kafka connector

[Martijn Visser (Jira)]

Martijn Visser created FLINK-35008:
--

 Summary: Bump org.apache.commons:commons-compress from 1.25.0 to 
1.26.0 for Flink Kafka connector
 Key: FLINK-35008
 URL: https://issues.apache.org/jira/browse/FLINK-35008
 Project: Flink
  Issue Type: Technical Debt
  Components: Connectors / Kafka
Reporter: Martijn Visser
Assignee: Martijn Visser






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [FYI] The Azure CI for PRs is currently not triggered

[lorenzo affetti]

The CI is back to normal since this morning.
Thanks to the collaboration with Robert and Mathias.

For more information, see the issue link in Matthias message.



On Thu, Apr 4, 2024, 08:23 Matthias Pohl 
wrote:

> Hi everyone,
> just for your information: The Azure CI for PRs is currently not working.
> This started to happen on Tuesday (April 2 at around 7pm (CEST)).
> FLINK-34999 [1] covers the issue.
>
> We're expecting the issue to be gone by today. But in the meantime, these
> are the things you can do:
> 1. Wait for FLINK-34999 to be fixed before merging your PR.
> 2. Check the GHA workflow run for the PR and commit in your fork (and share
> the link in your PR for documentation).
> 3. Azure Pipelines CI is still triggered for pushes to master and the
> release branches [2], i.e. if you decide to merge, monitor these builds
> closely.
>
> That said, option (1), i.e. waiting for FLINK-34999 to be fixed, is still
> the preferred way considering that we have Azure Pipelines defined as our
> ground of truth for now and that the issue is going to be fixed today,
> hopefully. Additionally, merging a change without a CI run isn't the best
> option, either. But I still want to be transparent about your options.
>
> Matthias
>
> [1] https://issues.apache.org/jira/browse/FLINK-34999
> [2]
>
> https://dev.azure.com/apache-flink/apache-flink/_build?definitionId=1&_a=summary
>
> --
>
> [image: Aiven] 
>
> *Matthias Pohl*
> Opensource Software Engineer, *Aiven*
> matthias.p...@aiven.io|  +49 170 9869525
> aiven.io    |    >
>      <
> https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
>

[jira] [Created] (FLINK-35007) Update Flink Kafka connector to support 1.19 and test 1.20-SNAPSHOT

[Martijn Visser (Jira)]

Martijn Visser created FLINK-35007:
--

 Summary: Update Flink Kafka connector to support 1.19 and test 
1.20-SNAPSHOT
 Key: FLINK-35007
 URL: https://issues.apache.org/jira/browse/FLINK-35007
 Project: Flink
  Issue Type: Technical Debt
  Components: Connectors / Kafka
Reporter: Martijn Visser
Assignee: Martijn Visser






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35006) Use try with-resource for StandaloneAutoscalerExecutor

[Kirill Plugatarev (Jira)]

Kirill Plugatarev created FLINK-35006:
-

 Summary: Use try with-resource for StandaloneAutoscalerExecutor
 Key: FLINK-35006
 URL: https://issues.apache.org/jira/browse/FLINK-35006
 Project: Flink
  Issue Type: Improvement
  Components: Kubernetes Operator
Reporter: Kirill Plugatarev






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Externalized Google Cloud Connectors

[Martijn Visser]

Hi Lorenzo,

Bahir is retired, see the homepage. It plays no role (anymore).

>  This, unfortunately, is the tradeoff for developing the connectors
outside of Apache in exchange for development velocity.

I understand that. It can be considered to develop the connectors outside
of the Flink project, in order to achieve development velocity. We've seen
a similar thing happen with the CDC connectors, before that was ultimately
donated to the Flink project. However, there are no guarantees that
external contributions are considered when evaluating committers, because
there's no visibility for the PMC on these external contributions.

Best regards,

Martijn

On Wed, Apr 3, 2024 at 3:26 PM 
wrote:

> @Leonard @Martijn
> Following up on @Claire question, what is the role of Bahir (
> https://bahir.apache.org/) in this scenario?
>
> I am also trying to understand how connectors fir in the Flink project
> scenario :)
>
> Thank you,
> Lorenzo
> On Apr 2, 2024 at 06:13 +0200, Leonard Xu , wrote:
> > Hey, Claire
> >
> > Thanks starting this discussion, all flink external connector repos are
> sub-projects of Apache Flink, including
> https://github.com/apache/flink-connector-aws.
> >
> > Creating a flink external connector repo named flink-connectors-gcp as
> sub-project of Apache Beam is not a good idea from my side.
> >
> > > Currently, we have no Flink committers on our team. We are actively
> > > involved in the Apache Beam community and have a number of ASF members
> on
> > > the team.
> >
> > Not having Flink committer should not be a strong reason in this case,
> Flink community welcome contributors to contribute and maintain the
> connectors, as a contributor, through continuous connector development and
> maintenance work in the community, you will also have the opportunity to
> become a Committer.
> >
> > Best,
> > Leonard
> >
> >
> > > 2024年2月14日 上午12:24，Claire McCarthy 
> 写道：
> > >
> > > Hi Devs!
> > >
> > > I’d like to kick off a discussion on setting up a repo for a new fleet
> of
> > > Google Cloud connectors.
> > >
> > > A bit of context:
> > >
> > > -
> > >
> > > We have a team of Google engineers who are looking to build/maintain
> > > 5-10 GCP connectors for Flink.
> > > -
> > >
> > > We are wondering if it would make sense to host our connectors under
> the
> > > ASF umbrella following a similar repo structure as AWS (
> > > https://github.com/apache/flink-connector-aws). In our case:
> > > apache/flink-connectors-gcp.
> > > -
> > >
> > > Currently, we have no Flink committers on our team. We are actively
> > > involved in the Apache Beam community and have a number of ASF members
> on
> > > the team.
> > >
> > >
> > > We saw that one of the original motivations for externalizing
> connectors
> > > was to encourage more activity and contributions around connectors by
> > > easing the contribution overhead. We understand that the decision was
> > > ultimately made to host the externalized connector repos under the ASF
> > > organization. For the same reasons (release infra, quality assurance,
> > > integration with the community, etc.), we would like all GCP
> connectors to
> > > live under the ASF organization.
> > >
> > > We want to ask the Flink community what you all think of this idea, and
> > > what would be the best way for us to go about contributing something
> like
> > > this. We are excited to contribute and want to learn and follow your
> > > practices.
> > >
> > > A specific issue we know of is that our changes need approval from
> Flink
> > > committers. Do you have a suggestion for how best to go about a new
> > > contribution like ours from a team that does not have committers? Is it
> > > possible, for example, to partner with a committer (or a small cohort)
> for
> > > tight engagement? We also know about ASF voting and release process,
> but
> > > that doesn't seem to be as much of a potential hurdle.
> > >
> > > Huge thanks in advance for sharing your thoughts!
> > >
> > >
> > > Claire
> >
>