[jira] [Created] (FLINK-31878) Fix the wrong name of PauseOrResumeSplitsTask#toString in connector fetcher

[Yuxin Tan (Jira)]

Yuxin Tan created FLINK-31878:
-

 Summary: Fix the wrong name of PauseOrResumeSplitsTask#toString in 
connector fetcher 
 Key: FLINK-31878
 URL: https://issues.apache.org/jira/browse/FLINK-31878
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Common
Reporter: Yuxin Tan
Assignee: Yuxin Tan
 Fix For: 1.18.0


The class name PauseOrResumeSplitsTask#toString is now right. Users will be 
very confused when calling the toString method of the class. So we should fix 
it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] FLIP-304: Pluggable Failure Enrichers

[weijie guo]

+1 (binding)

Best regards,

Weijie


Zhu Zhu  于2023年4月21日周五 11:03写道：

> +1 (binding)
>
> Thanks,
> Zhu
>
> Anton Kalashnikov  于2023年4月20日周四 20:03写道：
> >
> > +1 (binding)
> >
> >
> > Thanks for this FLIP Panos, LGTM.
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> > On 20.04.23 13:44, Roman Khachatryan wrote:
> > > +1 (binding)
> > >
> > > The FLIP LGTM, thanks Panos!
> > >
> > > Regards,
> > > Roman
> > >
> > >
> > > On Thu, Apr 20, 2023 at 1:33 PM Hong Teoh  wrote:
> > >
> > >> +1 (non-binding)
> > >>
> > >> Thank you for driving this effort, Panagiotis.
> > >>
> > >> Regards,
> > >> Hong
> > >>
> > >>
> > >>> On 20 Apr 2023, at 12:16, David Morávek  wrote:
> > >>>
> > >>> Thanks for the update!
> > >>>
> > >>> +1 (binding)
> > >>>
> > >>> Best,
> > >>> D.
> > >>>
> > >>> On Thu, Apr 20, 2023 at 9:50 AM Piotr Nowojski  >
> > >> wrote:
> >  Hi,
> > 
> >  I see that the FLIP has been updated, thanks Panos!
> > 
> >  +1 (binding)
> > 
> >  Best,
> >  Piotrek
> > 
> >  śr., 19 kwi 2023 o 13:49 Piotr Nowojski 
> >  napisał(a):
> > 
> > > +1 to what David wrote. I think we need to update the FLIP and
> extend
> > >> the
> > > voting?
> > >
> > > Piotrek
> > >
> > > śr., 19 kwi 2023 o 09:06 David Morávek 
> napisał(a):
> > >
> > >> Hi Panos,
> > >>
> > >> It seems that most recent discussions (e.g. changing the
> semantics of
> >  the
> > >> config option) are not reflected in the FLIP. Can you please
> >  double-check
> > >> that this is the correct version?
> > >>
> > >> Best,
> > >> D.
> > >>
> > >>
> > >> On Mon, Apr 17, 2023 at 9:24 AM Panagiotis Garefalakis <
> >  pga...@apache.org
> > >> wrote:
> > >>
> > >>> Hello everyone,
> > >>>
> > >>> I want to start the vote for FLIP-304: Pluggable Failure
> Enrichers
> > >> [1]
> > >> --
> > >>> discussed as part of [2].
> > >>>
> > >>> FLIP-304 introduces a pluggable interface allowing users to add
> > >> custom
> > >>> logic and enrich failures with custom metadata labels.
> > >>>
> > >>> The vote will last for at least 72 hours (Thursday, 20th of April
> >  2023,
> > >>> 12:30 PST) unless there is an objection or insufficient votes.
> > >>>
> > >>> [1]
> > >>>
> > >>>
> > >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-304%3A+Pluggable+Failure+Enrichers
> > >>> [2]
> https://lists.apache.org/thread/zs9n9p8d7tyvnq4yyxhc8zvq1k2c1hvs
> > >>>
> > >>>
> > >>> Cheers,
> > >>> Panagiotis
> > >>>
> > >>
>

Re: [VOTE] Release flink-connector-kafka 3.0.0 for Flink 1.17, release candidate #2

[Maximilian Michels]

Thanks for driving the release Gordon!

-Max

On Fri, Apr 21, 2023 at 12:33 AM Tzu-Li (Gordon) Tai
 wrote:
>
> We have unanimously approved this release.
>
> There are 6 approving votes, 3 of which are binding:
>
> * Alexander Sorokoumov
> * Martijn Visser (binding)
> * Tzu-Li (Gordon) Tai (binding)
> * Danny Cranmer (binding)
> * Ahmed Hamdy
> * Mason Chen
>
> Thanks so much everyone for testing and voting! I will now finalize the
> release.
>
> Thanks,
> Gordon
>
> On Thu, Apr 20, 2023 at 3:04 PM Mason Chen  wrote:
>
> > +1 (non-binding)
> >
> > * Verified hashes and signatures
> > * Verified no binaries
> > * Verified LICENSE and NOTICE files, pointing to 2023 as well
> > * Verified poms point to 3.0.0
> > * Reviewed web PR
> > * Built from source
> > * Verified git tag
> >
> > Best,
> > Mason
> >
> > On Thu, Apr 20, 2023 at 10:04 AM Ahmed Hamdy  wrote:
> >
> > > +1 (non-binding)
> > >
> > > - Release notes look good.
> > > - verified signatures and checksums are correct.
> > > - Verified no binaries in source archive.
> > > - Built from source
> > > - Approved Web PR (no comments if we are supporting 1.17+)
> > > Best Regards
> > > Ahmed
> > >
> > > On Thu, 20 Apr 2023 at 17:08, Danny Cranmer 
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > - +1 on skipping 1.16
> > > > - Release notes look ok
> > > > - Verified signature/hashes of source archive
> > > > - Verified there are no binaries in the source archive
> > > > - Built from source
> > > > - Contents of Maven repo look good
> > > > - Verified NOTICE files
> > > > - Tag exists in Github
> > > > - Reviewed web PR (looks good apart from the open comment from Martijn)
> > > >
> > > >
> > > > On Tue, Apr 18, 2023 at 6:38 PM Tzu-Li (Gordon) Tai <
> > tzuli...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > - Checked hashes and signatures
> > > > > - Built from source mvn clean install -Pcheck-convergence
> > > > > -Dflink.version=1.17.0
> > > > > - Eyeballed NOTICE license files
> > > > > - Started a Flink 1.17.0 cluster + Kafka 3.2.3 cluster, submitted a
> > SQL
> > > > > statement using the Kafka connector under exactly-once mode.
> > > > Checkpointing
> > > > > and restoring works, with or without throughput on the Kafka topic.
> > > > >
> > > > > Thanks,
> > > > > Gordon
> > > > >
> > > > > On Fri, Apr 14, 2023 at 2:13 AM Martijn Visser <
> > > martijnvis...@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > - Validated hashes
> > > > > > - Verified signature
> > > > > > - Verified that no binaries exist in the source archive
> > > > > > - Build the source with Maven via mvn clean install
> > > -Pcheck-convergence
> > > > > > -Dflink.version=1.17.0
> > > > > > - Verified licenses
> > > > > > - Verified web PR
> > > > > > - Started a cluster and the Flink SQL client, successfully read and
> > > > wrote
> > > > > > with the Kafka connector to Confluent Cloud with AVRO and Schema
> > > > Registry
> > > > > > enabled
> > > > > >
> > > > > > On Fri, Apr 14, 2023 at 12:24 AM Alexander Sorokoumov
> > > > > >  wrote:
> > > > > >
> > > > > > > +1 (nb).
> > > > > > >
> > > > > > > Checked:
> > > > > > >
> > > > > > >- checksums are correct
> > > > > > >- source code builds (JDK 8+11)
> > > > > > >- release notes are correct
> > > > > > >
> > > > > > >
> > > > > > > Best,
> > > > > > > Alex
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Apr 12, 2023 at 5:07 PM Tzu-Li (Gordon) Tai <
> > > > > tzuli...@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > A few important remarks about this release candidate:
> > > > > > > >
> > > > > > > > - As mentioned in the previous voting thread of RC1 [1], we've
> > > > > decided
> > > > > > to
> > > > > > > > skip releasing a version of the externalized Flink Kafka
> > > Connector
> > > > > > > matching
> > > > > > > > with Flink 1.16.x since the original vote thread stalled, and
> > > > > meanwhile
> > > > > > > > we've already completed externalizing all Kafka connector code
> > as
> > > > of
> > > > > > > Flink
> > > > > > > > 1.17.0.
> > > > > > > >
> > > > > > > > - As such, this RC is basically identical to the Kafka
> > connector
> > > > code
> > > > > > > > bundled with the Flink 1.17.0 release, PLUS a few critical
> > fixes
> > > > for
> > > > > > > > exactly-once violations, namely FLINK-31305, FLINK-31363, and
> > > > > > FLINK-31620
> > > > > > > > (please see release notes [2]).
> > > > > > > >
> > > > > > > > - As part of preparing this RC, I've also deleted the original
> > > v3.0
> > > > > > > branch
> > > > > > > > and re-named the v4.0 branch to replace it instead.
> > Effectively,
> > > > this
> > > > > > > > resets the versioning numbers for the externalized Flink Kafka
> > > > > > Connector
> > > > > > > > code repository, so that this first release of the repo starts
> > > from
> > > > > > > v3.0.0.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Gordon
> > > > > > > 

Off for a week

[Etienne Chauchot]

Hi all,

Just to let you know, I'll be off and unresponsive for a week starting 
tonight.


Best

Etienne

Re: [DISCUSS] FLIP-287: Extend Sink#InitContext to expose ExecutionConfig and JobID

[Zhu Zhu]

Good point! @Gordon
Introducing an `InitContext#createSerializer(TypeInformation)` looks a
better option to me, so we do not need to introduce an unmodifiable
`ExecutionConfig` at this moment.

Hope that we can make `ExecutionConfig` a read-only interface in
Flink 2.0. It is exposed in `RuntimeContext` to user functions already,
while mutating the values at runtime is actually an undefined behavior.

Thanks,
Zhu

Tzu-Li (Gordon) Tai  于2023年4月18日周二 01:02写道：
>
> Hi,
>
> Sorry for chiming in late.
>
> I'm not so sure that exposing ExecutionConfig / ReadExecutionConfig
> directly through Sink#InitContext is the right thing to do.
>
> 1. A lot of the read-only getter methods on ExecutionConfig are irrelevant
> for sinks. Expanding the scope of the InitContext interface with so many
> irrelevant methods is probably going to make writing unit tests a pain.
>
> 2. There's actually a few getter methods on `InitContext` that have
> duplicate/redundant info for what ExecutionConfig exposes. For example,
> InitContext#getNumberOfParallelSubtasks and InitContext#getAttemptNumber
> currently exist and it can be confusing if users find 2 sources of that
> information (either via the `InitContext` and via the wrapped
> `ExecutionConfig`).
>
> All in all, it feels like `Sink#InitContext` was introduced initially as a
> means to selectively only expose certain information to sinks.
>
> It looks like right now, the only requirement is that some sinks require 1)
> isObjectReuseEnabled, and 2) TypeSerializer for the input type. Would it
> make sense to follow the original intent and only selectively expose these?
> For 1), we can just add a new method to `InitContext` and forward the
> information from `ExecutionConfig` accessible at the operator level.
> For 2), would it make sense to create the serializer at the operator level
> and then provide it through `InitContext`?
>
> Thanks,
> Gordon
>
> On Mon, Apr 17, 2023 at 8:23 AM Zhu Zhu  wrote:
>
> > We can let the `InitContext` return `ExecutionConfig` in the interface.
> > However, a `ReadableExecutionConfig` implementation should be returned
> > so that exceptions will be thrown if users tries to modify the
> > `ExecutionConfig`.
> >
> > We can rework all the setters of `ExecutionConfig` to internally invoke a
> > `setConfiguration(...)` method. Then the `ReadableExecutionConfig` can
> > just override that method. But pay attention to a few exceptional
> > setters, i.e. those for globalJobParameters and serializers.
> >
> > We should also explicitly state in the documentation of
> > `InitContext #getExecutionConfig()`, that the returned `ExecutionConfig`
> > is unmodifiable.
> >
> > Thanks,
> > Zhu
> >
> > João Boto  于2023年4月17日周一 16:51写道：
> > >
> > > Hi Zhu,
> > >
> > > Thanks for you time for reviewing this.
> > >
> > > Extending ´ExecutionConfig´ will allow to modify the values in the
> > config (this is what we want to prevent with Option2)
> > >
> > > To extend the ExecutionConfig is not simpler to do Option1 (expose
> > ExecutionConfig directly).
> > >
> > > Regards
> > >
> > >
> > >
> > > On 2023/04/03 09:42:28 Zhu Zhu wrote:
> > > > Hi João,
> > > >
> > > > Thanks for creating this FLIP!
> > > > I'm overall +1 for it to unblock the migration of sinks to SinkV2.
> > > >
> > > > Yet I think it's better to let the `ReadableExecutionConfig` extend
> > > > `ExecutionConfig`, because otherwise we have to introduce a new method
> > > > `TypeInformation#createSerializer(ReadableExecutionConfig)`. The new
> > > > method may require every `TypeInformation` to implement it, including
> > > > Flink built-in ones and custom ones, otherwise exceptions will happen.
> > > > That goal, however, is pretty hard to achieve.
> > > >
> > > > Thanks,
> > > > Zhu
> > > >
> > > > João Boto  于2023年2月28日周二 23:34写道：
> > > > >
> > > > > I have update the FLIP with the 2 options that we have discussed..
> > > > >
> > > > > Option 1: Expose ExecutionConfig directly on InitContext
> > > > > this have a minimal impact as we only have to expose the new methods
> > > > >
> > > > > Option 2: Expose ReadableExecutionConfig on InitContext
> > > > > with this option we have more impact as we need to add a new method
> > to TypeInformation and change all implementations (current exists 72
> > implementations)
> > > > >
> > > > > Waiting for feedback or concerns about the two options
> > > >
> >

Re: [VOTE] FLIP-304: Pluggable Failure Enrichers

[Weihua Hu]

+1 (non-binding)

Best,
Weihua


On Fri, Apr 21, 2023 at 4:00 PM weijie guo 
wrote:

> +1 (binding)
>
> Best regards,
>
> Weijie
>
>
> Zhu Zhu  于2023年4月21日周五 11:03写道：
>
> > +1 (binding)
> >
> > Thanks,
> > Zhu
> >
> > Anton Kalashnikov  于2023年4月20日周四 20:03写道：
> > >
> > > +1 (binding)
> > >
> > >
> > > Thanks for this FLIP Panos, LGTM.
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > > On 20.04.23 13:44, Roman Khachatryan wrote:
> > > > +1 (binding)
> > > >
> > > > The FLIP LGTM, thanks Panos!
> > > >
> > > > Regards,
> > > > Roman
> > > >
> > > >
> > > > On Thu, Apr 20, 2023 at 1:33 PM Hong Teoh 
> wrote:
> > > >
> > > >> +1 (non-binding)
> > > >>
> > > >> Thank you for driving this effort, Panagiotis.
> > > >>
> > > >> Regards,
> > > >> Hong
> > > >>
> > > >>
> > > >>> On 20 Apr 2023, at 12:16, David Morávek  wrote:
> > > >>>
> > > >>> Thanks for the update!
> > > >>>
> > > >>> +1 (binding)
> > > >>>
> > > >>> Best,
> > > >>> D.
> > > >>>
> > > >>> On Thu, Apr 20, 2023 at 9:50 AM Piotr Nowojski <
> pnowoj...@apache.org
> > >
> > > >> wrote:
> > >  Hi,
> > > 
> > >  I see that the FLIP has been updated, thanks Panos!
> > > 
> > >  +1 (binding)
> > > 
> > >  Best,
> > >  Piotrek
> > > 
> > >  śr., 19 kwi 2023 o 13:49 Piotr Nowojski  >
> > >  napisał(a):
> > > 
> > > > +1 to what David wrote. I think we need to update the FLIP and
> > extend
> > > >> the
> > > > voting?
> > > >
> > > > Piotrek
> > > >
> > > > śr., 19 kwi 2023 o 09:06 David Morávek 
> > napisał(a):
> > > >
> > > >> Hi Panos,
> > > >>
> > > >> It seems that most recent discussions (e.g. changing the
> > semantics of
> > >  the
> > > >> config option) are not reflected in the FLIP. Can you please
> > >  double-check
> > > >> that this is the correct version?
> > > >>
> > > >> Best,
> > > >> D.
> > > >>
> > > >>
> > > >> On Mon, Apr 17, 2023 at 9:24 AM Panagiotis Garefalakis <
> > >  pga...@apache.org
> > > >> wrote:
> > > >>
> > > >>> Hello everyone,
> > > >>>
> > > >>> I want to start the vote for FLIP-304: Pluggable Failure
> > Enrichers
> > > >> [1]
> > > >> --
> > > >>> discussed as part of [2].
> > > >>>
> > > >>> FLIP-304 introduces a pluggable interface allowing users to add
> > > >> custom
> > > >>> logic and enrich failures with custom metadata labels.
> > > >>>
> > > >>> The vote will last for at least 72 hours (Thursday, 20th of
> April
> > >  2023,
> > > >>> 12:30 PST) unless there is an objection or insufficient votes.
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>>
> > > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-304%3A+Pluggable+Failure+Enrichers
> > > >>> [2]
> > https://lists.apache.org/thread/zs9n9p8d7tyvnq4yyxhc8zvq1k2c1hvs
> > > >>>
> > > >>>
> > > >>> Cheers,
> > > >>> Panagiotis
> > > >>>
> > > >>
> >
>

[ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Jark Wu]

Hi everyone,

We are thrilled to announce that Leonard Xu has joined the Flink PMC!

Leonard has been an active member of the Apache Flink community for many
years and became a committer in Nov 2021. He has been involved in various
areas of the project, from code contributions to community building. His
contributions are mainly focused on Flink SQL and connectors, especially
leading the flink-cdc-connectors project to receive 3.8+K GitHub stars. He
authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs (e.g.,
FLIP-132, FLIP-162). He has participated in plenty of discussions in the
dev mailing list, answering questions about 500+ threads in the
user/user-zh mailing list. Besides that, he is community minded, such as
being the release manager of 1.17, verifying releases, managing release
syncs, etc.

Congratulations and welcome Leonard!

Best,
Jark (on behalf of the Flink PMC)

[ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Jark Wu]

Hi everyone,

We are thrilled to announce that Qingsheng Ren has joined the Flink PMC!

Qingsheng has been contributing to Apache Flink for a long time. He is the
core contributor and maintainer of the Kafka connector and
flink-cdc-connectors, bringing users stability and ease of use in both
projects. He drove discussions and implementations in FLIP-221, FLIP-288,
and the connector testing framework. He is continuously helping with the
expansion of the Flink community and has given several talks about Flink
connectors at many conferences, such as Flink Forward Global and Flink
Forward Asia. Besides that, he is willing to help a lot in the community
work, such as being the release manager for both 1.17 and 1.18, verifying
releases, and answering questions on the mailing list.

Congratulations and welcome Qingsheng!

Best,
Jark (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Jiabao Sun]

Congratulations, Leonard.

Best,
Jiabao


> 2023年4月21日 下午7:47，Jark Wu  写道：
> 
> Hi everyone,
> 
> We are thrilled to announce that Leonard Xu has joined the Flink PMC!
> 
> Leonard has been an active member of the Apache Flink community for many
> years and became a committer in Nov 2021. He has been involved in various
> areas of the project, from code contributions to community building. His
> contributions are mainly focused on Flink SQL and connectors, especially
> leading the flink-cdc-connectors project to receive 3.8+K GitHub stars. He
> authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs (e.g.,
> FLIP-132, FLIP-162). He has participated in plenty of discussions in the
> dev mailing list, answering questions about 500+ threads in the
> user/user-zh mailing list. Besides that, he is community minded, such as
> being the release manager of 1.17, verifying releases, managing release
> syncs, etc.
> 
> Congratulations and welcome Leonard!
> 
> Best,
> Jark (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Jiabao Sun]

Congratulations, Qingsheng.

Best,
Jiabao


> 2023年4月21日 下午7:50，Jark Wu  写道：
> 
> Hi everyone,
> 
> We are thrilled to announce that Qingsheng Ren has joined the Flink PMC!
> 
> Qingsheng has been contributing to Apache Flink for a long time. He is the
> core contributor and maintainer of the Kafka connector and
> flink-cdc-connectors, bringing users stability and ease of use in both
> projects. He drove discussions and implementations in FLIP-221, FLIP-288,
> and the connector testing framework. He is continuously helping with the
> expansion of the Flink community and has given several talks about Flink
> connectors at many conferences, such as Flink Forward Global and Flink
> Forward Asia. Besides that, he is willing to help a lot in the community
> work, such as being the release manager for both 1.17 and 1.18, verifying
> releases, and answering questions on the mailing list.
> 
> Congratulations and welcome Qingsheng!
> 
> Best,
> Jark (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[yuxia]

Congratulations, Leonard.

Best regards,
Yuxia

- 原始邮件 -
发件人: "Jiabao Sun" 
收件人: "dev" , "Jark Wu" 
发送时间: 星期五, 2023年 4 月 21日 下午 7:52:08
主题: Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

Congratulations, Leonard.

Best,
Jiabao


> 2023年4月21日 下午7:47，Jark Wu  写道：
> 
> Hi everyone,
> 
> We are thrilled to announce that Leonard Xu has joined the Flink PMC!
> 
> Leonard has been an active member of the Apache Flink community for many
> years and became a committer in Nov 2021. He has been involved in various
> areas of the project, from code contributions to community building. His
> contributions are mainly focused on Flink SQL and connectors, especially
> leading the flink-cdc-connectors project to receive 3.8+K GitHub stars. He
> authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs (e.g.,
> FLIP-132, FLIP-162). He has participated in plenty of discussions in the
> dev mailing list, answering questions about 500+ threads in the
> user/user-zh mailing list. Besides that, he is community minded, such as
> being the release manager of 1.17, verifying releases, managing release
> syncs, etc.
> 
> Congratulations and welcome Leonard!
> 
> Best,
> Jark (on behalf of the Flink PMC)

Re:[ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Mang Zhang]

Congratulations, Leonard.


--

Best regards,
Mang Zhang





At 2023-04-21 19:47:52, "Jark Wu"  wrote:
>Hi everyone,
>
>We are thrilled to announce that Leonard Xu has joined the Flink PMC!
>
>Leonard has been an active member of the Apache Flink community for many
>years and became a committer in Nov 2021. He has been involved in various
>areas of the project, from code contributions to community building. His
>contributions are mainly focused on Flink SQL and connectors, especially
>leading the flink-cdc-connectors project to receive 3.8+K GitHub stars. He
>authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs (e.g.,
>FLIP-132, FLIP-162). He has participated in plenty of discussions in the
>dev mailing list, answering questions about 500+ threads in the
>user/user-zh mailing list. Besides that, he is community minded, such as
>being the release manager of 1.17, verifying releases, managing release
>syncs, etc.
>
>Congratulations and welcome Leonard!
>
>Best,
>Jark (on behalf of the Flink PMC)

Re:[ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Mang Zhang]

Congratulations, Qingsheng.





--

Best regards,
Mang Zhang





At 2023-04-21 19:50:02, "Jark Wu"  wrote:
>Hi everyone,
>
>We are thrilled to announce that Qingsheng Ren has joined the Flink PMC!
>
>Qingsheng has been contributing to Apache Flink for a long time. He is the
>core contributor and maintainer of the Kafka connector and
>flink-cdc-connectors, bringing users stability and ease of use in both
>projects. He drove discussions and implementations in FLIP-221, FLIP-288,
>and the connector testing framework. He is continuously helping with the
>expansion of the Flink community and has given several talks about Flink
>connectors at many conferences, such as Flink Forward Global and Flink
>Forward Asia. Besides that, he is willing to help a lot in the community
>work, such as being the release manager for both 1.17 and 1.18, verifying
>releases, and answering questions on the mailing list.
>
>Congratulations and welcome Qingsheng!
>
>Best,
>Jark (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Feng Jin]

Congratulations, Leonard



Best,
Feng Jin

On Fri, Apr 21, 2023 at 8:38 PM Mang Zhang  wrote:

> Congratulations, Leonard.
>
>
> --
>
> Best regards,
> Mang Zhang
>
>
>
>
>
> At 2023-04-21 19:47:52, "Jark Wu"  wrote:
> >Hi everyone,
> >
> >We are thrilled to announce that Leonard Xu has joined the Flink PMC!
> >
> >Leonard has been an active member of the Apache Flink community for many
> >years and became a committer in Nov 2021. He has been involved in various
> >areas of the project, from code contributions to community building. His
> >contributions are mainly focused on Flink SQL and connectors, especially
> >leading the flink-cdc-connectors project to receive 3.8+K GitHub stars. He
> >authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs (e.g.,
> >FLIP-132, FLIP-162). He has participated in plenty of discussions in the
> >dev mailing list, answering questions about 500+ threads in the
> >user/user-zh mailing list. Besides that, he is community minded, such as
> >being the release manager of 1.17, verifying releases, managing release
> >syncs, etc.
> >
> >Congratulations and welcome Leonard!
> >
> >Best,
> >Jark (on behalf of the Flink PMC)
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Feng Jin]

Congratulations, Qingsheng



Best,
Feng Jin

On Fri, Apr 21, 2023 at 8:39 PM Mang Zhang  wrote:

> Congratulations, Qingsheng.
>
>
>
>
>
> --
>
> Best regards,
> Mang Zhang
>
>
>
>
>
> At 2023-04-21 19:50:02, "Jark Wu"  wrote:
> >Hi everyone,
> >
> >We are thrilled to announce that Qingsheng Ren has joined the Flink PMC!
> >
> >Qingsheng has been contributing to Apache Flink for a long time. He is the
> >core contributor and maintainer of the Kafka connector and
> >flink-cdc-connectors, bringing users stability and ease of use in both
> >projects. He drove discussions and implementations in FLIP-221, FLIP-288,
> >and the connector testing framework. He is continuously helping with the
> >expansion of the Flink community and has given several talks about Flink
> >connectors at many conferences, such as Flink Forward Global and Flink
> >Forward Asia. Besides that, he is willing to help a lot in the community
> >work, such as being the release manager for both 1.17 and 1.18, verifying
> >releases, and answering questions on the mailing list.
> >
> >Congratulations and welcome Qingsheng!
> >
> >Best,
> >Jark (on behalf of the Flink PMC)
>

Re: [DISCUSS] FLIP-287: Extend Sink#InitContext to expose ExecutionConfig and JobID

[Tzu-Li (Gordon) Tai]

Do we have to introduce `InitContext#createSerializer(TypeInformation)`
which returns TypeSerializer, or is it sufficient to only provide
`InitContext#createInputSerializer()` which returns TypeSerializer?

I had the impression that buffering sinks like JDBC only need the
latter. @Joao, could you confirm?

If that's the case, +1 to adding the following method signatures to
InitContext:
* TypeSerializer createInputSerializer()
* boolean isObjectReuseEnabled()

Thanks,
Gordon

On Fri, Apr 21, 2023 at 3:04 AM Zhu Zhu  wrote:

> Good point! @Gordon
> Introducing an `InitContext#createSerializer(TypeInformation)` looks a
> better option to me, so we do not need to introduce an unmodifiable
> `ExecutionConfig` at this moment.
>
> Hope that we can make `ExecutionConfig` a read-only interface in
> Flink 2.0. It is exposed in `RuntimeContext` to user functions already,
> while mutating the values at runtime is actually an undefined behavior.
>
> Thanks,
> Zhu
>
> Tzu-Li (Gordon) Tai  于2023年4月18日周二 01:02写道：
> >
> > Hi,
> >
> > Sorry for chiming in late.
> >
> > I'm not so sure that exposing ExecutionConfig / ReadExecutionConfig
> > directly through Sink#InitContext is the right thing to do.
> >
> > 1. A lot of the read-only getter methods on ExecutionConfig are
> irrelevant
> > for sinks. Expanding the scope of the InitContext interface with so many
> > irrelevant methods is probably going to make writing unit tests a pain.
> >
> > 2. There's actually a few getter methods on `InitContext` that have
> > duplicate/redundant info for what ExecutionConfig exposes. For example,
> > InitContext#getNumberOfParallelSubtasks and InitContext#getAttemptNumber
> > currently exist and it can be confusing if users find 2 sources of that
> > information (either via the `InitContext` and via the wrapped
> > `ExecutionConfig`).
> >
> > All in all, it feels like `Sink#InitContext` was introduced initially as
> a
> > means to selectively only expose certain information to sinks.
> >
> > It looks like right now, the only requirement is that some sinks require
> 1)
> > isObjectReuseEnabled, and 2) TypeSerializer for the input type. Would it
> > make sense to follow the original intent and only selectively expose
> these?
> > For 1), we can just add a new method to `InitContext` and forward the
> > information from `ExecutionConfig` accessible at the operator level.
> > For 2), would it make sense to create the serializer at the operator
> level
> > and then provide it through `InitContext`?
> >
> > Thanks,
> > Gordon
> >
> > On Mon, Apr 17, 2023 at 8:23 AM Zhu Zhu  wrote:
> >
> > > We can let the `InitContext` return `ExecutionConfig` in the interface.
> > > However, a `ReadableExecutionConfig` implementation should be returned
> > > so that exceptions will be thrown if users tries to modify the
> > > `ExecutionConfig`.
> > >
> > > We can rework all the setters of `ExecutionConfig` to internally
> invoke a
> > > `setConfiguration(...)` method. Then the `ReadableExecutionConfig` can
> > > just override that method. But pay attention to a few exceptional
> > > setters, i.e. those for globalJobParameters and serializers.
> > >
> > > We should also explicitly state in the documentation of
> > > `InitContext #getExecutionConfig()`, that the returned
> `ExecutionConfig`
> > > is unmodifiable.
> > >
> > > Thanks,
> > > Zhu
> > >
> > > João Boto  于2023年4月17日周一 16:51写道：
> > > >
> > > > Hi Zhu,
> > > >
> > > > Thanks for you time for reviewing this.
> > > >
> > > > Extending ´ExecutionConfig´ will allow to modify the values in the
> > > config (this is what we want to prevent with Option2)
> > > >
> > > > To extend the ExecutionConfig is not simpler to do Option1 (expose
> > > ExecutionConfig directly).
> > > >
> > > > Regards
> > > >
> > > >
> > > >
> > > > On 2023/04/03 09:42:28 Zhu Zhu wrote:
> > > > > Hi João,
> > > > >
> > > > > Thanks for creating this FLIP!
> > > > > I'm overall +1 for it to unblock the migration of sinks to SinkV2.
> > > > >
> > > > > Yet I think it's better to let the `ReadableExecutionConfig` extend
> > > > > `ExecutionConfig`, because otherwise we have to introduce a new
> method
> > > > > `TypeInformation#createSerializer(ReadableExecutionConfig)`. The
> new
> > > > > method may require every `TypeInformation` to implement it,
> including
> > > > > Flink built-in ones and custom ones, otherwise exceptions will
> happen.
> > > > > That goal, however, is pretty hard to achieve.
> > > > >
> > > > > Thanks,
> > > > > Zhu
> > > > >
> > > > > João Boto  于2023年2月28日周二 23:34写道：
> > > > > >
> > > > > > I have update the FLIP with the 2 options that we have
> discussed..
> > > > > >
> > > > > > Option 1: Expose ExecutionConfig directly on InitContext
> > > > > > this have a minimal impact as we only have to expose the new
> methods
> > > > > >
> > > > > > Option 2: Expose ReadableExecutionConfig on InitContext
> > > > > > with this option we have more impact as we need to add a new
> method
> > > to TypeInformation and change a

Re: [DISCUSS] FLIP-288:Enable Dynamic Partition Discovery by Default in Kafka Source

[Tzu-Li (Gordon) Tai]

> I have already modified FLIP-288 to provide a
newDiscoveryOffsetsInitializer in the KafkaSourceBuilder and
KafkaSourceEnumerator. Users can use
KafkaSourceBuilder#setNewDiscoveryOffsets to change the strategy for new
partitions.

Thanks for addressing my comment Hongshun.

> Considering these reasons and facts, I’m +1 to only use EARLIEST for  new
discovered partitions.

Sounds good to me.


Overall, +1 to this proposal in principle (I'll formally vote on the vote
thread as well).

Thanks,
Gordon

On Tue, Apr 18, 2023 at 9:12 PM Leonard Xu  wrote:

> Thanks Hongshun for deeper analysis of the existing KafkaSource
> implementation details, Cool!
> There’s no specific use case to use a future TIMESTAMP and SPECIFIC-OFFSET
> for new discovered partitions
>  The existing SpecifiedOffsetsInitializer will use the EARLIEST offset for
> unspecified partitions as well as new discovered partitions
>  The existing TimestampOffsetsInitializer will use the LATEST offset for
> future timestamp, the  LATEST offset is similar to  EARLIEST offset for new
> discovered partitions  in this case,  and EARLIEST is safer as it covers
> all records.
> Considering these reasons and facts, I’m +1 to only use EARLIEST for  new
> discovered partitions.
>
> The updated FLIP looks good to me, we can start a vote thread soon if
> there are no new divergences.
>
> Best,
> Leonard
>
> > On Apr 18, 2023, at 4:58 PM, Hongshun Wang 
> wrote:
> >
> > Hi Shammon,
> >
> > Thank you for your advice.I have carefully considered whether to show
> this
> > in SQL DDL. Therefore, I carefully studied whether it is feasible
> Recently
> >
> > However,  after reading the corresponding code more thoroughly, it
> appears
> > that SpecifiedOffsetsInitializer and TimestampOffsetsInitializer do not
> > work as we initially thought. Finally, I have decided to only use
> > "EARLIEST" instead of allowing the user to make a free choice.
> >
> > Now, let me show my new understanding.
> >
> > The actual work of SpecifiedOffsetsInitializer and
> > TimestampOffsetsInitializer:
> >
> >
> >   - *SpecifiedOffsetsInitializer*: Use *Specified offset* for specified
> >   partitions while use *EARLIEST* for unspecified partitions. Specified
> >   partitions offset should be less than the latest offset, otherwise it
> will
> >   start from the *EARLIEST*.
> >   - *TimestampOffsetsInitializer*: Initialize the offsets based on a
> >   timestamp. If the message meeting the requirement of the timestamp
> have not
> >   been produced to Kafka yet, just use the *LATEST* offset.
> >
> > So, some problems will occur when new partition use
> > SpecifiedOffsetsInitializer or TimestampOffsetsInitializer. You can find
> > more information in the "Rejected Alternatives" section of Flip-288,
> which
> > includes details of the code and process of deductive reasoning.
> > All these problems can be reproducible in the current version. The reason
> > why they haven't been exposed is probably because users usually set the
> > existing specified offset or timestamp, so it appears as earliest in
> > production.
> >
> > WDYT?
> > CC：Ruan, Shammon, Gordon, Leonard and Qingsheng.
> >
> > Yours
> >
> > Hongshun
> >
> >
> >
> >
> > On Fri, Apr 14, 2023 at 5:48 PM Shammon FY  wrote:
> >
> >> Hi Hongshun
> >>
> >> Thanks for updating the FLIP, it totally sounds good to me.
> >>
> >> I just have one comment: How does sql job set new discovery offsets
> >> initializer?
> >> I found `DataStream` jobs can set different offsets initializers for new
> >> discovery partitions in `KafkaSourceBuilder.setNewDiscoveryOffsets`. Do
> SQL
> >> jobs need to support this feature?
> >>
> >> Best,
> >> Shammon FY
> >>
> >> On Wed, Apr 12, 2023 at 2:27 PM Hongshun Wang 
> >> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> I have already modified FLIP-288 to provide a
> >>> newDiscoveryOffsetsInitializer in the KafkaSourceBuilder and
> >>> KafkaSourceEnumerator. Users can use
> >>> KafkaSourceBuilder#setNewDiscoveryOffsets to change the strategy for
> new
> >>> partitions.
> >>>
> >>> Surely, enabling the partition discovery strategy by default and
> >> modifying
> >>> the offset strategy for new partitions should be brought to the user's
> >>> attention. Therefore, it will be explained in the 1.18 release notes.
> >>>
> >>> WDYT?CC, Ruan, Shammon, Gordon and Leonard.
> >>>
> >>>
> >>> Best,
> >>>
> >>> Hongshun
> >>>
> >>> On Fri, Mar 31, 2023 at 2:56 PM Hongshun Wang  >
> >>> wrote:
> >>>
>  Hi everyone,
>  Thanks for your participation.
> 
>  @Gordon, I looked at the several questions you raised:
> 
>    1. Should we use the firstDiscovery flag or two separate
>    OffsetsInitializers? Actually, I have considered later. If we follow
>    my initial idea, we can provide a default earliest
> >> OffsetsInitializer
>    for a new partition. However, According to @Shammon's suggestion,
> >>> different
>    startup OffsetsInitializers correspond to different post-startup
>    Offset

Re: [VOTE] FLIP-288: Enable Dynamic Partition Discovery by Default in Kafka Source

[Tzu-Li (Gordon) Tai]

+1

On Thu, Apr 20, 2023 at 11:52 PM Hongshun Wang 
wrote:

> Dear Flink Developers,
>
>
> Thank you for providing feedback on FLIP-288: Enable Dynamic Partition
> Discovery by Default in Kafka Source[1] on the discussion thread[2].
>
> The goal of the FLIP is to enable partition discovery by default and set
> EARLIEST offset strategy for later discovered partitions.
>
>
> I am initiating a vote for this FLIP. The vote will be open for at least 72
> hours, unless there is an objection or insufficient votes.
>
>
> [1]: [
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-288%3A+Enable+Dynamic+Partition+Discovery+by+Default+in+Kafka+Source](https://cwiki.apache.org/confluence/display/FLINK/FLIP-288%3A+Enable+Dynamic+Partition+Discovery+by+Default+in+Kafka+Source)
> 
> [2]: [
>
> https://lists.apache.org/thread/581f2xq5d1tlwc8gcr27gwkp3zp0wrg6](https://lists.apache.org/thread/581f2xq5d1tlwc8gcr27gwkp3zp0wrg6)
>
>
> Best regards,
> Hongshun
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[David Morávek]

Congratulations, Leonard, well deserved!

Best,
D.

On Fri 21. 4. 2023 at 16:40, Feng Jin  wrote:

> Congratulations, Leonard
>
>
> 
> Best,
> Feng Jin
>
> On Fri, Apr 21, 2023 at 8:38 PM Mang Zhang  wrote:
>
> > Congratulations, Leonard.
> >
> >
> > --
> >
> > Best regards,
> > Mang Zhang
> >
> >
> >
> >
> >
> > At 2023-04-21 19:47:52, "Jark Wu"  wrote:
> > >Hi everyone,
> > >
> > >We are thrilled to announce that Leonard Xu has joined the Flink PMC!
> > >
> > >Leonard has been an active member of the Apache Flink community for many
> > >years and became a committer in Nov 2021. He has been involved in
> various
> > >areas of the project, from code contributions to community building. His
> > >contributions are mainly focused on Flink SQL and connectors, especially
> > >leading the flink-cdc-connectors project to receive 3.8+K GitHub stars.
> He
> > >authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs (e.g.,
> > >FLIP-132, FLIP-162). He has participated in plenty of discussions in the
> > >dev mailing list, answering questions about 500+ threads in the
> > >user/user-zh mailing list. Besides that, he is community minded, such as
> > >being the release manager of 1.17, verifying releases, managing release
> > >syncs, etc.
> > >
> > >Congratulations and welcome Leonard!
> > >
> > >Best,
> > >Jark (on behalf of the Flink PMC)
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[David Morávek]

Congratulations, Qingsheng, well deserved!

Best,
D.

On Fri 21. 4. 2023 at 16:41, Feng Jin  wrote:

> Congratulations, Qingsheng
>
>
> 
> Best,
> Feng Jin
>
> On Fri, Apr 21, 2023 at 8:39 PM Mang Zhang  wrote:
>
> > Congratulations, Qingsheng.
> >
> >
> >
> >
> >
> > --
> >
> > Best regards,
> > Mang Zhang
> >
> >
> >
> >
> >
> > At 2023-04-21 19:50:02, "Jark Wu"  wrote:
> > >Hi everyone,
> > >
> > >We are thrilled to announce that Qingsheng Ren has joined the Flink PMC!
> > >
> > >Qingsheng has been contributing to Apache Flink for a long time. He is
> the
> > >core contributor and maintainer of the Kafka connector and
> > >flink-cdc-connectors, bringing users stability and ease of use in both
> > >projects. He drove discussions and implementations in FLIP-221,
> FLIP-288,
> > >and the connector testing framework. He is continuously helping with the
> > >expansion of the Flink community and has given several talks about Flink
> > >connectors at many conferences, such as Flink Forward Global and Flink
> > >Forward Asia. Besides that, he is willing to help a lot in the community
> > >work, such as being the release manager for both 1.17 and 1.18,
> verifying
> > >releases, and answering questions on the mailing list.
> > >
> > >Congratulations and welcome Qingsheng!
> > >
> > >Best,
> > >Jark (on behalf of the Flink PMC)
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Samrat Deb]

congratulations !

On Fri, 21 Apr 2023 at 9:45 PM, David Morávek  wrote:

> Congratulations, Qingsheng, well deserved!
>
> Best,
> D.
>
> On Fri 21. 4. 2023 at 16:41, Feng Jin  wrote:
>
> > Congratulations, Qingsheng
> >
> >
> > 
> > Best,
> > Feng Jin
> >
> > On Fri, Apr 21, 2023 at 8:39 PM Mang Zhang  wrote:
> >
> > > Congratulations, Qingsheng.
> > >
> > >
> > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Mang Zhang
> > >
> > >
> > >
> > >
> > >
> > > At 2023-04-21 19:50:02, "Jark Wu"  wrote:
> > > >Hi everyone,
> > > >
> > > >We are thrilled to announce that Qingsheng Ren has joined the Flink
> PMC!
> > > >
> > > >Qingsheng has been contributing to Apache Flink for a long time. He is
> > the
> > > >core contributor and maintainer of the Kafka connector and
> > > >flink-cdc-connectors, bringing users stability and ease of use in both
> > > >projects. He drove discussions and implementations in FLIP-221,
> > FLIP-288,
> > > >and the connector testing framework. He is continuously helping with
> the
> > > >expansion of the Flink community and has given several talks about
> Flink
> > > >connectors at many conferences, such as Flink Forward Global and Flink
> > > >Forward Asia. Besides that, he is willing to help a lot in the
> community
> > > >work, such as being the release manager for both 1.17 and 1.18,
> > verifying
> > > >releases, and answering questions on the mailing list.
> > > >
> > > >Congratulations and welcome Qingsheng!
> > > >
> > > >Best,
> > > >Jark (on behalf of the Flink PMC)
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Samrat Deb]

congratulations

On Fri, 21 Apr 2023 at 9:44 PM, David Morávek  wrote:

> Congratulations, Leonard, well deserved!
>
> Best,
> D.
>
> On Fri 21. 4. 2023 at 16:40, Feng Jin  wrote:
>
> > Congratulations, Leonard
> >
> >
> > 
> > Best,
> > Feng Jin
> >
> > On Fri, Apr 21, 2023 at 8:38 PM Mang Zhang  wrote:
> >
> > > Congratulations, Leonard.
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Mang Zhang
> > >
> > >
> > >
> > >
> > >
> > > At 2023-04-21 19:47:52, "Jark Wu"  wrote:
> > > >Hi everyone,
> > > >
> > > >We are thrilled to announce that Leonard Xu has joined the Flink PMC!
> > > >
> > > >Leonard has been an active member of the Apache Flink community for
> many
> > > >years and became a committer in Nov 2021. He has been involved in
> > various
> > > >areas of the project, from code contributions to community building.
> His
> > > >contributions are mainly focused on Flink SQL and connectors,
> especially
> > > >leading the flink-cdc-connectors project to receive 3.8+K GitHub
> stars.
> > He
> > > >authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs
> (e.g.,
> > > >FLIP-132, FLIP-162). He has participated in plenty of discussions in
> the
> > > >dev mailing list, answering questions about 500+ threads in the
> > > >user/user-zh mailing list. Besides that, he is community minded, such
> as
> > > >being the release manager of 1.17, verifying releases, managing
> release
> > > >syncs, etc.
> > > >
> > > >Congratulations and welcome Leonard!
> > > >
> > > >Best,
> > > >Jark (on behalf of the Flink PMC)
> > >
> >
>

[jira] [Created] (FLINK-31879) org.apache.avro.util.Utf8 cannot be serialized with avro when used in state

[Feroze Daud (Jira)]

Feroze Daud created FLINK-31879:
---

 Summary: org.apache.avro.util.Utf8 cannot be serialized with avro 
when used in state 
 Key: FLINK-31879
 URL: https://issues.apache.org/jira/browse/FLINK-31879
 Project: Flink
  Issue Type: Bug
  Components: API / Type Serialization System
Reporter: Feroze Daud


Scenario:

Write a flink app that reads avro messages from a kafka topic.

The avro pojos are generated with _org.apache.avro.util.Utf8_ type instead of 
_java.lang.String_

When this happens, Flink logs an error message as follows:
{noformat}
Class class org.apache.avro.util.Utf8 cannot be used as a POJO type because not 
all fields are valid POJO fields, and must be processed as GenericType. Please 
read the Flink documentation on "Data Types & Serialization" for details of the 
effect on performance. {noformat}
 

This is problematic because `Utf8` is designed to be a fast 
serialized/deserialized type for Avro. But since it is not inheriting from 
SpecificRecordBase, it seems as if it gets handled by Kryo serializer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Ahmed Hamdy]

Congratulations Leonard.
Best Regards
Ahmed

On Fri, 21 Apr 2023 at 17:23, Samrat Deb  wrote:

> congratulations
>
> On Fri, 21 Apr 2023 at 9:44 PM, David Morávek  wrote:
>
> > Congratulations, Leonard, well deserved!
> >
> > Best,
> > D.
> >
> > On Fri 21. 4. 2023 at 16:40, Feng Jin  wrote:
> >
> > > Congratulations, Leonard
> > >
> > >
> > > 
> > > Best,
> > > Feng Jin
> > >
> > > On Fri, Apr 21, 2023 at 8:38 PM Mang Zhang  wrote:
> > >
> > > > Congratulations, Leonard.
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Mang Zhang
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > At 2023-04-21 19:47:52, "Jark Wu"  wrote:
> > > > >Hi everyone,
> > > > >
> > > > >We are thrilled to announce that Leonard Xu has joined the Flink
> PMC!
> > > > >
> > > > >Leonard has been an active member of the Apache Flink community for
> > many
> > > > >years and became a committer in Nov 2021. He has been involved in
> > > various
> > > > >areas of the project, from code contributions to community building.
> > His
> > > > >contributions are mainly focused on Flink SQL and connectors,
> > especially
> > > > >leading the flink-cdc-connectors project to receive 3.8+K GitHub
> > stars.
> > > He
> > > > >authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs
> > (e.g.,
> > > > >FLIP-132, FLIP-162). He has participated in plenty of discussions in
> > the
> > > > >dev mailing list, answering questions about 500+ threads in the
> > > > >user/user-zh mailing list. Besides that, he is community minded,
> such
> > as
> > > > >being the release manager of 1.17, verifying releases, managing
> > release
> > > > >syncs, etc.
> > > > >
> > > > >Congratulations and welcome Leonard!
> > > > >
> > > > >Best,
> > > > >Jark (on behalf of the Flink PMC)
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Ahmed Hamdy]

Congratulations Qingsheng.
Best regards
Ahmed

On Fri, 21 Apr 2023 at 17:22, Samrat Deb  wrote:

> congratulations !
>
> On Fri, 21 Apr 2023 at 9:45 PM, David Morávek  wrote:
>
> > Congratulations, Qingsheng, well deserved!
> >
> > Best,
> > D.
> >
> > On Fri 21. 4. 2023 at 16:41, Feng Jin  wrote:
> >
> > > Congratulations, Qingsheng
> > >
> > >
> > > 
> > > Best,
> > > Feng Jin
> > >
> > > On Fri, Apr 21, 2023 at 8:39 PM Mang Zhang  wrote:
> > >
> > > > Congratulations, Qingsheng.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Mang Zhang
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > At 2023-04-21 19:50:02, "Jark Wu"  wrote:
> > > > >Hi everyone,
> > > > >
> > > > >We are thrilled to announce that Qingsheng Ren has joined the Flink
> > PMC!
> > > > >
> > > > >Qingsheng has been contributing to Apache Flink for a long time. He
> is
> > > the
> > > > >core contributor and maintainer of the Kafka connector and
> > > > >flink-cdc-connectors, bringing users stability and ease of use in
> both
> > > > >projects. He drove discussions and implementations in FLIP-221,
> > > FLIP-288,
> > > > >and the connector testing framework. He is continuously helping with
> > the
> > > > >expansion of the Flink community and has given several talks about
> > Flink
> > > > >connectors at many conferences, such as Flink Forward Global and
> Flink
> > > > >Forward Asia. Besides that, he is willing to help a lot in the
> > community
> > > > >work, such as being the release manager for both 1.17 and 1.18,
> > > verifying
> > > > >releases, and answering questions on the mailing list.
> > > > >
> > > > >Congratulations and welcome Qingsheng!
> > > > >
> > > > >Best,
> > > > >Jark (on behalf of the Flink PMC)
> > > >
> > >
> >
>

[DISCUSS] FLINK-31873: Add setMaxParallelism to the DataStreamSink Class

[eric xiao]

Hi there devs,

I would like to start a discussion thread for FLINK-31873[1].

We are in the processing of enabling Flink reactive mode as the default
scheduling mode. While reading configuration docs [2] (I believe it was
also mentioned during one of the training sessions during Flink Forward
2022), one can/should replace all setParallelism calls with
setMaxParallelism when migrating to reactive mode.

This currently isn't possible on a sink in a Flink pipeline as we do not
expose a setMaxParallelism on the DataStreamSink class [3]. The underlying
Transformation class does have both a setMaxParallelism and setParallelism
function defined [4], but only setParallelism is offered in the
DataStreamSink class.

I believe adding setMaxParallelism would be beneficial for not just flink
reactive mode, both modes of running of a flink pipeline (non reactive
mode, flink auto scaling).

Best,

Eric Xiao

[1] https://issues.apache.org/jira/browse/FLINK-31873
[2]
https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/deployment/elastic_scaling/#configuration
[3]
https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/DataStreamSink.java
[4]
https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/dag/Transformation.java#L248-L285

[jira] [Created] (FLINK-31880) Bad Test in OrcColumnarRowSplitReaderTest

[Kurt Ostfeld (Jira)]

Kurt Ostfeld created FLINK-31880:


 Summary: Bad Test in OrcColumnarRowSplitReaderTest
 Key: FLINK-31880
 URL: https://issues.apache.org/jira/browse/FLINK-31880
 Project: Flink
  Issue Type: Bug
  Components: Connectors / ORC, Formats (JSON, Avro, Parquet, ORC, 
SequenceFile)
Reporter: Kurt Ostfeld


This is a development issue with, what looks like a buggy unit test.
 
I tried to build Flink with a clean copy of the repository and I get:
 
```
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] OrcColumnarRowSplitReaderTest.testReadFileWithTypes:365
expected: "1969-12-31"
but was: "1970-01-01"
[INFO]
[ERROR] Tests run: 26, Failures: 1, Errors: 0, Skipped: 0
```
 
I see the test is testing Date data types with `new Date(562423)` which is 9 
minutes and 22 seconds after the epoch time, which is 1970-01-01 UTC time, or 
when I run that on my laptop in CST timezone, I get `Wed Dec 31 18:09:22 CST 
1969`.
 
I have a simple pull request ready which fixes this issue and uses the Java 8 
LocalDate API instead which avoids time zones entirely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] FLIP-304: Pluggable Failure Enrichers

[Panagiotis Garefalakis]

Thanks for the replies everyone!
Closing this vote now and following up with the result in a separate email.

Panagiotis

On Fri, Apr 21, 2023 at 4:23 AM Weihua Hu  wrote:

> +1 (non-binding)
>
> Best,
> Weihua
>
>
> On Fri, Apr 21, 2023 at 4:00 PM weijie guo 
> wrote:
>
> > +1 (binding)
> >
> > Best regards,
> >
> > Weijie
> >
> >
> > Zhu Zhu  于2023年4月21日周五 11:03写道：
> >
> > > +1 (binding)
> > >
> > > Thanks,
> > > Zhu
> > >
> > > Anton Kalashnikov  于2023年4月20日周四 20:03写道：
> > > >
> > > > +1 (binding)
> > > >
> > > >
> > > > Thanks for this FLIP Panos, LGTM.
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > > On 20.04.23 13:44, Roman Khachatryan wrote:
> > > > > +1 (binding)
> > > > >
> > > > > The FLIP LGTM, thanks Panos!
> > > > >
> > > > > Regards,
> > > > > Roman
> > > > >
> > > > >
> > > > > On Thu, Apr 20, 2023 at 1:33 PM Hong Teoh 
> > wrote:
> > > > >
> > > > >> +1 (non-binding)
> > > > >>
> > > > >> Thank you for driving this effort, Panagiotis.
> > > > >>
> > > > >> Regards,
> > > > >> Hong
> > > > >>
> > > > >>
> > > > >>> On 20 Apr 2023, at 12:16, David Morávek  wrote:
> > > > >>>
> > > > >>> Thanks for the update!
> > > > >>>
> > > > >>> +1 (binding)
> > > > >>>
> > > > >>> Best,
> > > > >>> D.
> > > > >>>
> > > > >>> On Thu, Apr 20, 2023 at 9:50 AM Piotr Nowojski <
> > pnowoj...@apache.org
> > > >
> > > > >> wrote:
> > > >  Hi,
> > > > 
> > > >  I see that the FLIP has been updated, thanks Panos!
> > > > 
> > > >  +1 (binding)
> > > > 
> > > >  Best,
> > > >  Piotrek
> > > > 
> > > >  śr., 19 kwi 2023 o 13:49 Piotr Nowojski <
> piotr.nowoj...@gmail.com
> > >
> > > >  napisał(a):
> > > > 
> > > > > +1 to what David wrote. I think we need to update the FLIP and
> > > extend
> > > > >> the
> > > > > voting?
> > > > >
> > > > > Piotrek
> > > > >
> > > > > śr., 19 kwi 2023 o 09:06 David Morávek 
> > > napisał(a):
> > > > >
> > > > >> Hi Panos,
> > > > >>
> > > > >> It seems that most recent discussions (e.g. changing the
> > > semantics of
> > > >  the
> > > > >> config option) are not reflected in the FLIP. Can you please
> > > >  double-check
> > > > >> that this is the correct version?
> > > > >>
> > > > >> Best,
> > > > >> D.
> > > > >>
> > > > >>
> > > > >> On Mon, Apr 17, 2023 at 9:24 AM Panagiotis Garefalakis <
> > > >  pga...@apache.org
> > > > >> wrote:
> > > > >>
> > > > >>> Hello everyone,
> > > > >>>
> > > > >>> I want to start the vote for FLIP-304: Pluggable Failure
> > > Enrichers
> > > > >> [1]
> > > > >> --
> > > > >>> discussed as part of [2].
> > > > >>>
> > > > >>> FLIP-304 introduces a pluggable interface allowing users to
> add
> > > > >> custom
> > > > >>> logic and enrich failures with custom metadata labels.
> > > > >>>
> > > > >>> The vote will last for at least 72 hours (Thursday, 20th of
> > April
> > > >  2023,
> > > > >>> 12:30 PST) unless there is an objection or insufficient
> votes.
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > > > >>>
> > > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-304%3A+Pluggable+Failure+Enrichers
> > > > >>> [2]
> > > https://lists.apache.org/thread/zs9n9p8d7tyvnq4yyxhc8zvq1k2c1hvs
> > > > >>>
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Panagiotis
> > > > >>>
> > > > >>
> > >
> >
>

[RESULT][VOTE] FLIP-304: Pluggable Failure Enrichers

[Panagiotis Garefalakis]

Happy to announce that FLIP-304 has been approved unanimously!
Voting included 8 votes, out of which 6 were binding -- and no
disapproving votes.

* Piotr Nowojski (binding)
* David Morávek (binding)
* Hong Teoh
* Roman Khachatryan (binding)
* Anton Kalashnikov (binding)
* Zhu Zhu (binding)
* Weijie Guo (binding)
* Weihua Hu

Thanks all for participating!

Cheers,
Panagiotis

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Panagiotis Garefalakis]

Congrats Leonard!

On Fri, Apr 21, 2023 at 11:19 AM Ahmed Hamdy  wrote:

> Congratulations Leonard.
> Best Regards
> Ahmed
>
> On Fri, 21 Apr 2023 at 17:23, Samrat Deb  wrote:
>
> > congratulations
> >
> > On Fri, 21 Apr 2023 at 9:44 PM, David Morávek  wrote:
> >
> > > Congratulations, Leonard, well deserved!
> > >
> > > Best,
> > > D.
> > >
> > > On Fri 21. 4. 2023 at 16:40, Feng Jin  wrote:
> > >
> > > > Congratulations, Leonard
> > > >
> > > >
> > > > 
> > > > Best,
> > > > Feng Jin
> > > >
> > > > On Fri, Apr 21, 2023 at 8:38 PM Mang Zhang 
> wrote:
> > > >
> > > > > Congratulations, Leonard.
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best regards,
> > > > > Mang Zhang
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > At 2023-04-21 19:47:52, "Jark Wu"  wrote:
> > > > > >Hi everyone,
> > > > > >
> > > > > >We are thrilled to announce that Leonard Xu has joined the Flink
> > PMC!
> > > > > >
> > > > > >Leonard has been an active member of the Apache Flink community
> for
> > > many
> > > > > >years and became a committer in Nov 2021. He has been involved in
> > > > various
> > > > > >areas of the project, from code contributions to community
> building.
> > > His
> > > > > >contributions are mainly focused on Flink SQL and connectors,
> > > especially
> > > > > >leading the flink-cdc-connectors project to receive 3.8+K GitHub
> > > stars.
> > > > He
> > > > > >authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs
> > > (e.g.,
> > > > > >FLIP-132, FLIP-162). He has participated in plenty of discussions
> in
> > > the
> > > > > >dev mailing list, answering questions about 500+ threads in the
> > > > > >user/user-zh mailing list. Besides that, he is community minded,
> > such
> > > as
> > > > > >being the release manager of 1.17, verifying releases, managing
> > > release
> > > > > >syncs, etc.
> > > > > >
> > > > > >Congratulations and welcome Leonard!
> > > > > >
> > > > > >Best,
> > > > > >Jark (on behalf of the Flink PMC)
> > > > >
> > > >
> > >
> >
>

[ANNOUNCE] Apache Flink Kafka Connectors 3.0.0 released

[Tzu-Li (Gordon) Tai]

The Apache Flink community is very happy to announce the release of Apache
Flink Kafka Connectors 3.0.0. This release is compatible with the Apache
Flink 1.17.x release series.

Apache Flink® is an open-source stream processing framework for
distributed, high-performing, always-available, and accurate data streaming
applications.

The release is available for download at:
https://flink.apache.org/downloads.html

The full release notes are available in Jira:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352577

We would like to thank all contributors of the Apache Flink community who
made this release possible!

Regards,
Gordon

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Yuxin Tan]

Congratulations, Qingsheng!

Best,
Yuxin


Ahmed Hamdy  于2023年4月22日周六 02:20写道：

> Congratulations Qingsheng.
> Best regards
> Ahmed
>
> On Fri, 21 Apr 2023 at 17:22, Samrat Deb  wrote:
>
> > congratulations !
> >
> > On Fri, 21 Apr 2023 at 9:45 PM, David Morávek  wrote:
> >
> > > Congratulations, Qingsheng, well deserved!
> > >
> > > Best,
> > > D.
> > >
> > > On Fri 21. 4. 2023 at 16:41, Feng Jin  wrote:
> > >
> > > > Congratulations, Qingsheng
> > > >
> > > >
> > > > 
> > > > Best,
> > > > Feng Jin
> > > >
> > > > On Fri, Apr 21, 2023 at 8:39 PM Mang Zhang 
> wrote:
> > > >
> > > > > Congratulations, Qingsheng.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best regards,
> > > > > Mang Zhang
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > At 2023-04-21 19:50:02, "Jark Wu"  wrote:
> > > > > >Hi everyone,
> > > > > >
> > > > > >We are thrilled to announce that Qingsheng Ren has joined the
> Flink
> > > PMC!
> > > > > >
> > > > > >Qingsheng has been contributing to Apache Flink for a long time.
> He
> > is
> > > > the
> > > > > >core contributor and maintainer of the Kafka connector and
> > > > > >flink-cdc-connectors, bringing users stability and ease of use in
> > both
> > > > > >projects. He drove discussions and implementations in FLIP-221,
> > > > FLIP-288,
> > > > > >and the connector testing framework. He is continuously helping
> with
> > > the
> > > > > >expansion of the Flink community and has given several talks about
> > > Flink
> > > > > >connectors at many conferences, such as Flink Forward Global and
> > Flink
> > > > > >Forward Asia. Besides that, he is willing to help a lot in the
> > > community
> > > > > >work, such as being the release manager for both 1.17 and 1.18,
> > > > verifying
> > > > > >releases, and answering questions on the mailing list.
> > > > > >
> > > > > >Congratulations and welcome Qingsheng!
> > > > > >
> > > > > >Best,
> > > > > >Jark (on behalf of the Flink PMC)
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Yuxin Tan]

Congratulations, Leonard!

Best,
Yuxin


Panagiotis Garefalakis  于2023年4月22日周六 08:08写道：

> Congrats Leonard!
>
> On Fri, Apr 21, 2023 at 11:19 AM Ahmed Hamdy  wrote:
>
> > Congratulations Leonard.
> > Best Regards
> > Ahmed
> >
> > On Fri, 21 Apr 2023 at 17:23, Samrat Deb  wrote:
> >
> > > congratulations
> > >
> > > On Fri, 21 Apr 2023 at 9:44 PM, David Morávek  wrote:
> > >
> > > > Congratulations, Leonard, well deserved!
> > > >
> > > > Best,
> > > > D.
> > > >
> > > > On Fri 21. 4. 2023 at 16:40, Feng Jin  wrote:
> > > >
> > > > > Congratulations, Leonard
> > > > >
> > > > >
> > > > > 
> > > > > Best,
> > > > > Feng Jin
> > > > >
> > > > > On Fri, Apr 21, 2023 at 8:38 PM Mang Zhang 
> > wrote:
> > > > >
> > > > > > Congratulations, Leonard.
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best regards,
> > > > > > Mang Zhang
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > At 2023-04-21 19:47:52, "Jark Wu"  wrote:
> > > > > > >Hi everyone,
> > > > > > >
> > > > > > >We are thrilled to announce that Leonard Xu has joined the Flink
> > > PMC!
> > > > > > >
> > > > > > >Leonard has been an active member of the Apache Flink community
> > for
> > > > many
> > > > > > >years and became a committer in Nov 2021. He has been involved
> in
> > > > > various
> > > > > > >areas of the project, from code contributions to community
> > building.
> > > > His
> > > > > > >contributions are mainly focused on Flink SQL and connectors,
> > > > especially
> > > > > > >leading the flink-cdc-connectors project to receive 3.8+K GitHub
> > > > stars.
> > > > > He
> > > > > > >authored 150+ PRs, and reviewed 250+ PRs, and drove several
> FLIPs
> > > > (e.g.,
> > > > > > >FLIP-132, FLIP-162). He has participated in plenty of
> discussions
> > in
> > > > the
> > > > > > >dev mailing list, answering questions about 500+ threads in the
> > > > > > >user/user-zh mailing list. Besides that, he is community minded,
> > > such
> > > > as
> > > > > > >being the release manager of 1.17, verifying releases, managing
> > > > release
> > > > > > >syncs, etc.
> > > > > > >
> > > > > > >Congratulations and welcome Leonard!
> > > > > > >
> > > > > > >Best,
> > > > > > >Jark (on behalf of the Flink PMC)
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] FLIP-288: Enable Dynamic Partition Discovery by Default in Kafka Source

[Leonard Xu]

+1(binding)

Best,
Leonard

> On Apr 21, 2023, at 2:50 PM, Hongshun Wang  wrote:
> 
> Dear Flink Developers,
> 
> 
> Thank you for providing feedback on FLIP-288: Enable Dynamic Partition
> Discovery by Default in Kafka Source[1] on the discussion thread[2].
> 
> The goal of the FLIP is to enable partition discovery by default and set
> EARLIEST offset strategy for later discovered partitions.
> 
> 
> I am initiating a vote for this FLIP. The vote will be open for at least 72
> hours, unless there is an objection or insufficient votes.
> 
> 
> [1]: [
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-288%3A+Enable+Dynamic+Partition+Discovery+by+Default+in+Kafka+Source](https://cwiki.apache.org/confluence/display/FLINK/FLIP-288%3A+Enable+Dynamic+Partition+Discovery+by+Default+in+Kafka+Source)
> [2]: [
> https://lists.apache.org/thread/581f2xq5d1tlwc8gcr27gwkp3zp0wrg6](https://lists.apache.org/thread/581f2xq5d1tlwc8gcr27gwkp3zp0wrg6)
> 
> 
> Best regards,
> Hongshun