Re:[ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

[Yuepeng Pan]

Congratulations !


Best,
Yuepeng Pan

At 2024-06-04 14:45:45, "Xintong Song"  wrote:
>Hi everyone,
>
>On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined
>the Flink PMC!
>
>Weijie has been an active member of the Apache Flink community for many
>years. He has made significant contributions in many components, including
>runtime, shuffle, sdk, connectors, etc. He has driven / participated in
>many FLIPs, authored and reviewed hundreds of PRs, been consistently active
>on mailing lists, and also helped with release management of 1.20 and
>several other bugfix releases.
>
>Congratulations and welcome Weijie!
>
>Best,
>
>Xintong (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

[Junrui Lee]

Congratulations Weijie!

Best,
Junrui

Xintong Song  于2024年6月4日周二 14:46写道：

> Hi everyone,
>
> On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined
> the Flink PMC!
>
> Weijie has been an active member of the Apache Flink community for many
> years. He has made significant contributions in many components, including
> runtime, shuffle, sdk, connectors, etc. He has driven / participated in
> many FLIPs, authored and reviewed hundreds of PRs, been consistently active
> on mailing lists, and also helped with release management of 1.20 and
> several other bugfix releases.
>
> Congratulations and welcome Weijie!
>
> Best,
>
> Xintong (on behalf of the Flink PMC)
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

[Rui Fan]

Congratulations Weijie, well deserved!

Best,
Rui

On Tue, Jun 4, 2024 at 2:47 PM Xintong Song  wrote:

> Hi everyone,
>
> On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined
> the Flink PMC!
>
> Weijie has been an active member of the Apache Flink community for many
> years. He has made significant contributions in many components, including
> runtime, shuffle, sdk, connectors, etc. He has driven / participated in
> many FLIPs, authored and reviewed hundreds of PRs, been consistently active
> on mailing lists, and also helped with release management of 1.20 and
> several other bugfix releases.
>
> Congratulations and welcome Weijie!
>
> Best,
>
> Xintong (on behalf of the Flink PMC)
>

[ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

[Xintong Song]

Hi everyone,

On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined
the Flink PMC!

Weijie has been an active member of the Apache Flink community for many
years. He has made significant contributions in many components, including
runtime, shuffle, sdk, connectors, etc. He has driven / participated in
many FLIPs, authored and reviewed hundreds of PRs, been consistently active
on mailing lists, and also helped with release management of 1.20 and
several other bugfix releases.

Congratulations and welcome Weijie!

Best,

Xintong (on behalf of the Flink PMC)

Re: [VOTE] Release flink-connector-jdbc v3.2.0, release candidate #2

[gongzhongqiang]

+1 (non-binding)

- Validated checksum hash and signature.
- Confirmed that no binaries exist in the source archive.
- Built the source with JDK 8.
- Verified the web PR.
- Ensured the JAR is built by JDK 8.

Best,
Zhongqiang Gong

Danny Cranmer  于2024年4月18日周四 18:20写道：

> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 3.2.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> This release supports Flink 1.18 and 1.19.
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2],
> which are signed with the key with fingerprint 125FD8DB [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag v3.2.0-rc1 [5],
> * website pull request listing the new release [6].
> * CI run of tag [7].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Danny
>
> [1]
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353143
> [2]
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-jdbc-3.2.0-rc2
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4]
> https://repository.apache.org/content/repositories/orgapacheflink-1718/
> [5] https://github.com/apache/flink-connector-jdbc/releases/tag/v3.2.0-rc2
> [6] https://github.com/apache/flink-web/pull/734
> [7] https://github.com/apache/flink-connector-jdbc/actions/runs/8736019099
>

[jira] [Created] (FLINK-35516) Update the Experimental annotation to PublicEvolving for files connector

[RocMarshal (Jira)]

RocMarshal created FLINK-35516:
--

 Summary: Update the Experimental annotation to PublicEvolving for 
files connector 
 Key: FLINK-35516
 URL: https://issues.apache.org/jira/browse/FLINK-35516
 Project: Flink
  Issue Type: Technical Debt
  Components: Connectors / FileSystem
Reporter: RocMarshal


as described in https://issues.apache.org/jira/browse/FLINK-35496
We should update the annotations for the stable APIs in files connector.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] FLIP-XXX Add K8S conditions to Flink CRD

[Lajith Koova]

Thank you  Thomas for the valuable feedback.  I have included couple of
screenshots in the Proposal and included screenshots of how  status will
looks like  if we have a Condition type other than Ready.

Thanks
Lajith K

On Mon, Jun 3, 2024 at 6:48 AM Thomas Weise  wrote:

> Thanks for the proposal. As I understand it the idea is to make the status
> of a Flink deployment more accessible to standard k8s tooling, which would
> be a nice improvement and further strengthen the k8s native experience!
>
> Regarding the FLIP document's overall structure: Before diving into the
> implementation details, can we please expand the intro with the
> motivation/rationale for this change? A few examples of the audience that
> would benefit from this change. Examples of tools that would pick up the
> condition and how that would look like (link or screenshot if you have it).
>
> Regarding multiple conditions: +1 for not commingling reconciliation status
> and job status. It would make the resulting condition confusing. I believe
> what the user would expect under "Ready" is the representation of the job
> status. We can then add another separate condition as suggested, however
> can the FLIP document also outline if/how conditions other than "Ready"
> would appear in the generic k8s tooling?
>
> Thanks,
> Thomas
>
>
>
> On Fri, May 31, 2024 at 10:37 AM David Radley 
> wrote:
>
> > Hi Mate and Gyula,
> > Thank you very much for your clarifications; it is clearer for me now. I
> > agree that a reconciliation condition would be useful – maybe reconciled
> > instead of ready for the boolean, so it is very explicit.
> >
> > Your suggestion of a job related readiness condition related to it’s
> > health would be useful; you suggest it be user configurable – this seems
> > closer to a liveliness / readiness probe.
> >
> > Kind regards, David.
> >
> > From: Mate Czagany 
> > Date: Thursday, 30 May 2024 at 10:39
> > To: dev@flink.apache.org 
> > Cc: morh...@apache.org 
> > Subject: [EXTERNAL] Re: [DISCUSS] FLIP-XXX Add K8S conditions to Flink
> CRD
> > Hi,
> >
> > I would definitely keep this as a FLIP. Not all FLIPs have to be big
> > changes, and this format makes it easier for others to chime in and
> follow.
> >
> > I am not a Kubernetes expert, but my understanding is that we don't have
> to
> > follow any strict convention for the type names in the conditions, e.g.
> > "Ready" or "Error". And as Gyula said it doesn't add too much value in
> the
> > currently proposed way, it might even be confusing for users who have not
> > read this email thread or FLIP because "Ready" might suggest that the job
> > is running and is healthy. So my suggestion is the same as Gyulas, to
> have
> > more explicit type names instead of just "Ready" and "Error". However
> > "ClusterReady" sounds weird in case of FlinkSessionJobs.
> >
> > Regarding appending to the conditions field: if I understand the FLIP
> > correctly, we would allow multiple elements of the same type to exist in
> > the conditions list if the message and reason fields are different. From
> > the Kubernetes documentation it seems like the correct way would be to
> use
> > the "type" field as the map key and merge the fields [1].
> >
> >
> > [1]
> >
> >
> https://github.com/kubernetes/kubernetes/blob/bce55b94cdc3a4592749aa919c591fa7df7453eb/staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go#L1528
> >
> > Best regards,
> > Mate
> >
> > Gyula Fóra  ezt írta (időpont: 2024. máj. 30., Cs,
> > 10:53):
> >
> > > David,
> > >
> > > The problem is exactly that ResourceLifecycleStates do not correspond
> to
> > > specific Job statuses (JobReady condition) in most cases. Let me give
> > you a
> > > concrete example:
> > >
> > > ResourceLifecycleState.STABLE means that app/job defined in the spec
> has
> > > been successfully deployed and was observed running, and this spec is
> now
> > > considered to be stable (won't be rolled back). Once a resource
> > > (FlinkDeployment) reached STABLE state, it won't change unless the user
> > > changes the spec. At the same time, this doesn't really say anything
> > about
> > > job health/readiness at any given future time. 10 minutes later the job
> > can
> > > go in an unrecoverable failure loop and never reach a running status,
> the
> > > ResourceLifecycleState will remain STABLE.
> > >
> > > This is actually not a problem with the ResourceLifecycleState but more
> > > with the understanding of it. It's called ResourceLifecycleState and
> not
> > > JobState exactly because it refers to the upgrade/rollback/suspend etc
> > > lifecycle of the FlinkDeployment/FlinkSessionJob resource and not the
> > > underlying flink job itself.
> > >
> > > But this is a crucial detail here that we need to consider otherwise
> the
> > > "Ready" condition that we may create will be practically useless.
> > >
> > > This is the reason why @morh...@apache.org  and
> > > I suggest separating this to at least 2 independent conditions. One
> could
> > > be the Up

Re: [VOTE] Release flink-connector-jdbc v3.2.0, release candidate #2

[Yuepeng Pan]

Thanks Leonard for the checking and Rui for the discussion.

I cancel the '-1 (non-binding)' vote.
And +1(non-binding) for the vote thread.

- Verified hash
- Test a few of usage demo on mysql-5.7
- build from source code with jdk-1.8 on MacOS.

Best regards,
Yuepeng Pan.

On 2024/06/03 14:46:35 Leonard Xu wrote:
> 
> > -1 (non-binding)
> > blocked by https://issues.apache.org/jira/browse/FLINK-35496.
> 
> 
> I just replied in https://issues.apache.org/jira/browse/FLINK-35496 
> , I think the API 
> annotation understanding is incorrect in this case, hope to see your comments
> 
> Best,
> Leonard
> 
> 
> 
> 
> > Best, 
> > Yuepeng Pan
> > 
> > 
> > On 2024/05/22 12:13:51 Leonard Xu wrote:
> >> +1 (binding)
> >> 
> >> - verified signatures
> >> - verified hashsums
> >> - built from source code with java 1.8 succeeded
> >> - checked Github release tag 
> >> - checked release notes
> >> - reviewed the web PR
> >> 
> >> Best,
> >> Leonard
> >> 
> >>> 2024年4月21日 下午9:42，Hang Ruan  写道：
> >>> 
> >>> +1 (non-binding)
> >>> 
> >>> - Validated checksum hash
> >>> - Verified signature
> >>> - Verified that no binaries exist in the source archive
> >>> - Build the source with Maven and jdk8
> >>> - Verified web PR
> >>> - Check that the jar is built by jdk8
> >>> 
> >>> Best,
> >>> Hang
> >>> 
> >>> Ahmed Hamdy  于2024年4月18日周四 21:37写道：
> >>> 
>  +1 (non-binding)
>  
>  - Verified Checksums and hashes
>  - Verified Signatures
>  - No binaries in source
>  - Build source
>  - Github tag exists
>  - Reviewed Web PR
>  
>  
>  Best Regards
>  Ahmed Hamdy
>  
>  
>  On Thu, 18 Apr 2024 at 11:22, Danny Cranmer 
>  wrote:
>  
> > Sorry for typos:
> > 
> >> Please review and vote on the release candidate #1 for the version
>  3.2.0,
> > as follows:
> > Should be "release candidate #2"
> > 
> >> * source code tag v3.2.0-rc1 [5],
> > Should be "source code tag v3.2.0-rc2"
> > 
> > Thanks,
> > Danny
> > 
> > On Thu, Apr 18, 2024 at 11:19 AM Danny Cranmer 
> > wrote:
> > 
> >> Hi everyone,
> >> 
> >> Please review and vote on the release candidate #1 for the version
>  3.2.0,
> >> as follows:
> >> [ ] +1, Approve the release
> >> [ ] -1, Do not approve the release (please provide specific comments)
> >> 
> >> This release supports Flink 1.18 and 1.19.
> >> 
> >> The complete staging area is available for your review, which includes:
> >> * JIRA release notes [1],
> >> * the official Apache source release to be deployed to dist.apache.org
> >> [2], which are signed with the key with fingerprint 125FD8DB [3],
> >> * all artifacts to be deployed to the Maven Central Repository [4],
> >> * source code tag v3.2.0-rc1 [5],
> >> * website pull request listing the new release [6].
> >> * CI run of tag [7].
> >> 
> >> The vote will be open for at least 72 hours. It is adopted by majority
> >> approval, with at least 3 PMC affirmative votes.
> >> 
> >> Thanks,
> >> Danny
> >> 
> >> [1]
> >> 
> > 
>  https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353143
> >> [2]
> >> 
> > 
>  https://dist.apache.org/repos/dist/dev/flink/flink-connector-jdbc-3.2.0-rc2
> >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> >> [4]
> >> 
>  https://repository.apache.org/content/repositories/orgapacheflink-1718/
> >> [5]
> > https://github.com/apache/flink-connector-jdbc/releases/tag/v3.2.0-rc2
> >> [6] https://github.com/apache/flink-web/pull/734
> >> [7]
> > https://github.com/apache/flink-connector-jdbc/actions/runs/8736019099
> >> 
> > 
>  
> >> 
> >> 
> 
> 

Re: [DISCUSS] Add Flink CDC Channel to Apache Flink Slack Workspace

[Leonard Xu]

I’ve created flink-cdc channel in Apache Flink Slack Workspace via 
https://issues.apache.org/jira/browse/FLINK-35514 


Best,
Leonard

> 2024年5月29日 下午9:53，Ahmed Hamdy  写道：
> 
> Thanks Zhongqiang, +1 for sure.
> Best Regards
> Ahmed Hamdy
> 
> 
> On Wed, 29 May 2024 at 13:48, ConradJam  wrote:
> 
>> +1 best
>> 
>> Hang Ruan  于2024年5月29日周三 11:28写道：
>> 
>>> Hi, zhongqiang.
>>> 
>>> Thanks for the proposal. +1 for it.
>>> 
>>> Best,
>>> Hang
>>> 
>>> Leonard Xu  于2024年5月28日周二 11:58写道：
>>> 
 
 Thanks Zhongqiang for the proposal, we need the Channel and I should
>> have
 been created it but not yet, +1 from my side.
 
 Best,
 Leonard
 
> 2024年5月28日 上午11:54，gongzhongqiang  写道：
> 
> Hi devs,
> 
> I would like to propose adding a dedicated Flink CDC channel to the
 Apache
> Flink Slack workspace.
> 
> Creating a channel focused on Flink CDC will help community members
 easily
> find previous discussions
> and target new discussions and questions to the correct place. Flink
>>> CDC
 is
> a sufficiently distinct component
> within the Apache Flink ecosystem, and having a dedicated channel
>> will
 make
> it viable and useful for
> those specifically working with or interested in this technology.
> 
> Looking forward to your feedback and support on this proposal.
> 
> 
> Best,
> Zhongqiang Gong
 
 
>>> 
>> 
>> 
>> --
>> Best
>> 
>> ConradJam
>> 

[jira] [Created] (FLINK-35515) Upgrade hive version to 4.0.0

[vikasap (Jira)]

vikasap created FLINK-35515:
---

 Summary: Upgrade hive version to 4.0.0
 Key: FLINK-35515
 URL: https://issues.apache.org/jira/browse/FLINK-35515
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / Hive
Affects Versions: 1.18.1
Reporter: vikasap
 Fix For: 1.18.2


Hive version 4.0.0 was released recently. However none of the major flink 
versions will work with this. Filing this so that major flink version's 
flink-sql and table api will be able to work with the new version of hive 
metastore.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35514) Add Flink CDC Channel to Apache Flink Slack Workspace

[Zhongqiang Gong (Jira)]

Zhongqiang Gong created FLINK-35514:
---

 Summary: Add Flink CDC Channel to Apache Flink Slack Workspace
 Key: FLINK-35514
 URL: https://issues.apache.org/jira/browse/FLINK-35514
 Project: Flink
  Issue Type: Bug
Reporter: Zhongqiang Gong


DISCUSS thread : 
https://lists.apache.org/thread/gqzrs3c0j9k7c5m9m5k2slomgorrqrwf



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Merge "flink run" and "flink run-application" in Flink 2.0

[Jeyhun Karimov]

Hi Ferenc,

Thanks for the proposal. +1 for it! This FLIP will improve the user
experience.

Regards,
Jeyhun





On Mon, Jun 3, 2024 at 1:50 PM Ferenc Csaky 
wrote:

> Hi Hang,
>
> Thank you for your inputs, both points make sense, updated the
> FLIP according to them.
>
> Best,
> Ferenc
>
>
>
>
> On Friday, 31 May 2024 at 04:31, Hang Ruan  wrote:
>
> >
> >
> > Hi, Ferenc.
> >
> > +1 for this proposal. This FLIP will help to make the CLI clearer for
> users.
> >
> > I think we should better add an example in the FLIP about how to use the
> > application mode with the new CLI.
> > Besides that, we need to add some new tests for this change instead of
> only
> > using the existed tests.
> >
> > Best,
> > Hang
> >
> > Mate Czagany czmat...@gmail.com 于2024年5月29日周三 19:57写道：
> >
> > > Hi Ferenc,
> > >
> > > Thanks for the FLIP, +1 from me for the proposal. I think these changes
> > > would be a great solution to all the confusion that comes from these
> two
> > > action parameters.
> > >
> > > Best regards,
> > > Mate
> > >
> > > Ferenc Csaky ferenc.cs...@pm.me.invalid ezt írta (időpont: 2024. máj.
> > > 28., K, 16:13):
> > >
> > > > Thank you Xintong for your input.
> > > >
> > > > I prepared a FLIP for this change [1], looking forward for any
> > > > other opinions.
> > > >
> > > > Thanks,
> > > > Ferenc
> > > >
> > > > [1]
> > >
> > >
> https://docs.google.com/document/d/1EX74rFp9bMKdfoGkz1ASOM6Ibw32rRxIadX72zs2zoY/edit?usp=sharing
> > >
> > > > On Friday, 17 May 2024 at 07:04, Xintong Song tonysong...@gmail.com
> > > > wrote:
> > > >
> > > > > AFAIK, the main purpose of having `run-application` was to make
> sure
> > > > > the user is aware that application mode is used, which executes the
> > > > > main
> > > > > method of the user program in JM rather than in client. This was
> > > > > important
> > > > > at the time application mode was first introduced, but maybe not
> that
> > > > > important anymore, given that per-job mode is deprecated and likely
> > > > > removed
> > > > > in 2.0. Therefore, +1 for the proposal.
> > > > >
> > > > > Best,
> > > > >
> > > > > Xintong
> > > > >
> > > > > On Thu, May 16, 2024 at 11:35 PM Ferenc Csaky
> > > > > ferenc.cs...@pm.me.invalid
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hello devs,
> > > > > >
> > > > > > I saw quite some examples when customers were confused about
> run, and
> > > > > > run-
> > > > > > application in the Flink CLI and I was wondering about the
> necessity
> > > > > > of
> > > > > > deploying
> > > > > > Application Mode (AM) jobs with a different command, than
> Session and
> > > > > > Per-Job mode jobs.
> > > > > >
> > > > > > I can see a point that YarnDeploymentTarget [1] and
> > > > > > KubernetesDeploymentTarget
> > > > > > [2] are part of their own maven modules and not known in
> > > > > > flink-clients,
> > > > > > so the
> > > > > > deployment mode validations are happening during cluster
> deployment
> > > > > > in
> > > > > > their specific
> > > > > > ClusterDescriptor implementation [3]. Although these are
> > > > > > implementation
> > > > > > details that
> > > > > > IMO should not define user-facing APIs.
> > > > > >
> > > > > > The command line setup is the same for both run and
> run-application,
> > > > > > so
> > > > > > I think there
> > > > > > is a quite simple way to achieve a unified flink run experience,
> but
> > > > > > I
> > > > > > might missed
> > > > > > something so I would appreciate any inputs on this topic.
> > > > > >
> > > > > > Based on my assumptions I think it would be possible to
> deprecate the
> > > > > > run-
> > > > > > application in Flink 1.20 and remove it completely in Flink 2.0.
> I
> > > > > > already put together a
> > > > > > PoC [4], and I was able to deploy AM jobs like this:
> > > > > >
> > > > > > flink run --target kubernetes-application ...
> > > > > >
> > > > > > If others also agree with this, I would be happy to open a FLIP.
> > > > > > WDYT?
> > > > > >
> > > > > > Thanks,
> > > > > > Ferenc
> > > > > >
> > > > > > [1]
> > >
> > >
> https://github.com/apache/flink/blob/master/flink-yarn/src/main/java/org/apache/flink/yarn/configuration/YarnDeploymentTarget.java
> > >
> > > > > > [2]
> > >
> > >
> https://github.com/apache/flink/blob/master/flink-kubernetes/src/main/java/org/apache/flink/kubernetes/configuration/KubernetesDeploymentTarget.java
> > >
> > > > > > [3]
> > >
> > >
> https://github.com/apache/flink/blob/48e5a39c9558083afa7589d2d8b054b625f61ee9/flink-kubernetes/src/main/java/org/apache/flink/kubernetes/KubernetesClusterDescriptor.java#L206
> > >
> > > > > > [4]
> > >
> > >
> https://github.com/ferenc-csaky/flink/commit/40b3e1b998c7a4273eaaff71d9162c9f1ee039c0
>

[jira] [Created] (FLINK-35513) ArtifactFetchManager unit tests are failing

[Elphas Toringepi (Jira)]

Elphas Toringepi created FLINK-35513:


 Summary: ArtifactFetchManager unit tests are failing
 Key: FLINK-35513
 URL: https://issues.apache.org/jira/browse/FLINK-35513
 Project: Flink
  Issue Type: Bug
  Components: Client / Job Submission
Affects Versions: 1.19.0
Reporter: Elphas Toringepi


The following unit tests in ArtifactFetchManagerTest are failing

 * testFileSystemFetchWithAdditionalUri()

 * testHttpFetch()
 * testMixedArtifactFetch()

 
{code:java}
Test 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.testFileSystemFetchWithAdditionalUri[testFileSystemFetchWithAdditionalUri()]
 failed with:
java.lang.AssertionError: 
Expecting actual not to be empty
 at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.getFlinkClientsJar(ArtifactFetchManagerTest.java:251)
 at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.testFileSystemFetchWithAdditionalUri(ArtifactFetchManagerTest.java:104)
 ...


Test 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.testMixedArtifactFetch[testMixedArtifactFetch()]
 failed with:
java.lang.AssertionError: 
Expecting actual not to be empty
 at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.getFlinkClientsJar(ArtifactFetchManagerTest.java:251)
 at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.testMixedArtifactFetch(ArtifactFetchManagerTest.java:149)
 ...


Test 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.testHttpFetch[testHttpFetch()]
 failed with:
java.lang.AssertionError: 
Expecting actual not to be empty
 at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.getFlinkClientsJar(ArtifactFetchManagerTest.java:251)
 at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.testHttpFetch(ArtifactFetchManagerTest.java:124)
 ... 
{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35512) ArtifactFetchManagerTest unit tests fail

[Hong Liang Teoh (Jira)]

Hong Liang Teoh created FLINK-35512:
---

 Summary: ArtifactFetchManagerTest unit tests fail
 Key: FLINK-35512
 URL: https://issues.apache.org/jira/browse/FLINK-35512
 Project: Flink
  Issue Type: Technical Debt
Affects Versions: 1.19.1
Reporter: Hong Liang Teoh
 Fix For: 1.19.1


The below three tests from *ArtifactFetchManagerTest* seem to fail consistently:
 * ArtifactFetchManagerTest.testFileSystemFetchWithAdditionalUri
 * ArtifactFetchManagerTest.testMixedArtifactFetch
 * ArtifactFetchManagerTest.testHttpFetch

The error printed is
{code:java}
java.lang.AssertionError:
Expecting actual not to be empty
    at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.getFlinkClientsJar(ArtifactFetchManagerTest.java:248)
    at 
org.apache.flink.client.program.artifact.ArtifactFetchManagerTest.testMixedArtifactFetch(ArtifactFetchManagerTest.java:146)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at 
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at 
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) 
{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release flink-connector-jdbc v3.2.0, release candidate #2

[Leonard Xu]

> -1 (non-binding)
> blocked by https://issues.apache.org/jira/browse/FLINK-35496.


I just replied in https://issues.apache.org/jira/browse/FLINK-35496 
, I think the API annotation 
understanding is incorrect in this case, hope to see your comments

Best,
Leonard




> Best, 
> Yuepeng Pan
> 
> 
> On 2024/05/22 12:13:51 Leonard Xu wrote:
>> +1 (binding)
>> 
>> - verified signatures
>> - verified hashsums
>> - built from source code with java 1.8 succeeded
>> - checked Github release tag 
>> - checked release notes
>> - reviewed the web PR
>> 
>> Best,
>> Leonard
>> 
>>> 2024年4月21日 下午9:42，Hang Ruan  写道：
>>> 
>>> +1 (non-binding)
>>> 
>>> - Validated checksum hash
>>> - Verified signature
>>> - Verified that no binaries exist in the source archive
>>> - Build the source with Maven and jdk8
>>> - Verified web PR
>>> - Check that the jar is built by jdk8
>>> 
>>> Best,
>>> Hang
>>> 
>>> Ahmed Hamdy  于2024年4月18日周四 21:37写道：
>>> 
 +1 (non-binding)
 
 - Verified Checksums and hashes
 - Verified Signatures
 - No binaries in source
 - Build source
 - Github tag exists
 - Reviewed Web PR
 
 
 Best Regards
 Ahmed Hamdy
 
 
 On Thu, 18 Apr 2024 at 11:22, Danny Cranmer 
 wrote:
 
> Sorry for typos:
> 
>> Please review and vote on the release candidate #1 for the version
 3.2.0,
> as follows:
> Should be "release candidate #2"
> 
>> * source code tag v3.2.0-rc1 [5],
> Should be "source code tag v3.2.0-rc2"
> 
> Thanks,
> Danny
> 
> On Thu, Apr 18, 2024 at 11:19 AM Danny Cranmer 
> wrote:
> 
>> Hi everyone,
>> 
>> Please review and vote on the release candidate #1 for the version
 3.2.0,
>> as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>> 
>> This release supports Flink 1.18 and 1.19.
>> 
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org
>> [2], which are signed with the key with fingerprint 125FD8DB [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag v3.2.0-rc1 [5],
>> * website pull request listing the new release [6].
>> * CI run of tag [7].
>> 
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>> 
>> Thanks,
>> Danny
>> 
>> [1]
>> 
> 
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353143
>> [2]
>> 
> 
 https://dist.apache.org/repos/dist/dev/flink/flink-connector-jdbc-3.2.0-rc2
>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>> [4]
>> 
 https://repository.apache.org/content/repositories/orgapacheflink-1718/
>> [5]
> https://github.com/apache/flink-connector-jdbc/releases/tag/v3.2.0-rc2
>> [6] https://github.com/apache/flink-web/pull/734
>> [7]
> https://github.com/apache/flink-connector-jdbc/actions/runs/8736019099
>> 
> 
 
>> 
>> 

[jira] [Created] (FLINK-35511) Enhance metrics. The incremental and full phases count the number of records separately

[melin (Jira)]

melin created FLINK-35511:
-

 Summary: Enhance metrics. The incremental and full phases count 
the number of records separately
 Key: FLINK-35511
 URL: https://issues.apache.org/jira/browse/FLINK-35511
 Project: Flink
  Issue Type: Improvement
  Components: Flink CDC
Affects Versions: cdc-3.2.0
Reporter: melin
 Attachments: image-2024-06-03-22-06-38-591.png

dataworks 实时同步的metric 信息

!image-2024-06-03-22-06-38-591.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35510) Implement basic incremental checkpoint for ForStStateBackend

[Feifan Wang (Jira)]

Feifan Wang created FLINK-35510:
---

 Summary: Implement basic incremental checkpoint for 
ForStStateBackend
 Key: FLINK-35510
 URL: https://issues.apache.org/jira/browse/FLINK-35510
 Project: Flink
  Issue Type: New Feature
  Components: Runtime / State Backends
Reporter: Feifan Wang


Use low DB api implement a basic incremental checkpoint for ForStStatebackend, 
follow steps:
 # db.disableFileDeletions()
 # db.getLiveFiles(true)
 # db.entableFileDeletes(false)

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Merge "flink run" and "flink run-application" in Flink 2.0

[Ferenc Csaky]

Hi Hang,

Thank you for your inputs, both points make sense, updated the
FLIP according to them.

Best,
Ferenc




On Friday, 31 May 2024 at 04:31, Hang Ruan  wrote:

> 
> 
> Hi, Ferenc.
> 
> +1 for this proposal. This FLIP will help to make the CLI clearer for users.
> 
> I think we should better add an example in the FLIP about how to use the
> application mode with the new CLI.
> Besides that, we need to add some new tests for this change instead of only
> using the existed tests.
> 
> Best,
> Hang
> 
> Mate Czagany czmat...@gmail.com 于2024年5月29日周三 19:57写道：
> 
> > Hi Ferenc,
> > 
> > Thanks for the FLIP, +1 from me for the proposal. I think these changes
> > would be a great solution to all the confusion that comes from these two
> > action parameters.
> > 
> > Best regards,
> > Mate
> > 
> > Ferenc Csaky ferenc.cs...@pm.me.invalid ezt írta (időpont: 2024. máj.
> > 28., K, 16:13):
> > 
> > > Thank you Xintong for your input.
> > > 
> > > I prepared a FLIP for this change [1], looking forward for any
> > > other opinions.
> > > 
> > > Thanks,
> > > Ferenc
> > > 
> > > [1]
> > 
> > https://docs.google.com/document/d/1EX74rFp9bMKdfoGkz1ASOM6Ibw32rRxIadX72zs2zoY/edit?usp=sharing
> > 
> > > On Friday, 17 May 2024 at 07:04, Xintong Song tonysong...@gmail.com
> > > wrote:
> > > 
> > > > AFAIK, the main purpose of having `run-application` was to make sure
> > > > the user is aware that application mode is used, which executes the
> > > > main
> > > > method of the user program in JM rather than in client. This was
> > > > important
> > > > at the time application mode was first introduced, but maybe not that
> > > > important anymore, given that per-job mode is deprecated and likely
> > > > removed
> > > > in 2.0. Therefore, +1 for the proposal.
> > > > 
> > > > Best,
> > > > 
> > > > Xintong
> > > > 
> > > > On Thu, May 16, 2024 at 11:35 PM Ferenc Csaky
> > > > ferenc.cs...@pm.me.invalid
> > > > 
> > > > wrote:
> > > > 
> > > > > Hello devs,
> > > > > 
> > > > > I saw quite some examples when customers were confused about run, and
> > > > > run-
> > > > > application in the Flink CLI and I was wondering about the necessity
> > > > > of
> > > > > deploying
> > > > > Application Mode (AM) jobs with a different command, than Session and
> > > > > Per-Job mode jobs.
> > > > > 
> > > > > I can see a point that YarnDeploymentTarget [1] and
> > > > > KubernetesDeploymentTarget
> > > > > [2] are part of their own maven modules and not known in
> > > > > flink-clients,
> > > > > so the
> > > > > deployment mode validations are happening during cluster deployment
> > > > > in
> > > > > their specific
> > > > > ClusterDescriptor implementation [3]. Although these are
> > > > > implementation
> > > > > details that
> > > > > IMO should not define user-facing APIs.
> > > > > 
> > > > > The command line setup is the same for both run and run-application,
> > > > > so
> > > > > I think there
> > > > > is a quite simple way to achieve a unified flink run experience, but
> > > > > I
> > > > > might missed
> > > > > something so I would appreciate any inputs on this topic.
> > > > > 
> > > > > Based on my assumptions I think it would be possible to deprecate the
> > > > > run-
> > > > > application in Flink 1.20 and remove it completely in Flink 2.0. I
> > > > > already put together a
> > > > > PoC [4], and I was able to deploy AM jobs like this:
> > > > > 
> > > > > flink run --target kubernetes-application ...
> > > > > 
> > > > > If others also agree with this, I would be happy to open a FLIP.
> > > > > WDYT?
> > > > > 
> > > > > Thanks,
> > > > > Ferenc
> > > > > 
> > > > > [1]
> > 
> > https://github.com/apache/flink/blob/master/flink-yarn/src/main/java/org/apache/flink/yarn/configuration/YarnDeploymentTarget.java
> > 
> > > > > [2]
> > 
> > https://github.com/apache/flink/blob/master/flink-kubernetes/src/main/java/org/apache/flink/kubernetes/configuration/KubernetesDeploymentTarget.java
> > 
> > > > > [3]
> > 
> > https://github.com/apache/flink/blob/48e5a39c9558083afa7589d2d8b054b625f61ee9/flink-kubernetes/src/main/java/org/apache/flink/kubernetes/KubernetesClusterDescriptor.java#L206
> > 
> > > > > [4]
> > 
> > https://github.com/ferenc-csaky/flink/commit/40b3e1b998c7a4273eaaff71d9162c9f1ee039c0

[jira] [Created] (FLINK-35509) Slack community invite link has expired

[Ufuk Celebi (Jira)]

Ufuk Celebi created FLINK-35509:
---

 Summary: Slack community invite link has expired
 Key: FLINK-35509
 URL: https://issues.apache.org/jira/browse/FLINK-35509
 Project: Flink
  Issue Type: Bug
  Components: Project Website
Reporter: Ufuk Celebi


The Slack invite link on the website has expired.

I've generated a new invite link without expiration here: 
[https://join.slack.com/t/apache-flink/shared_invite/zt-2k0fdioxx-D0kTYYLh3pPjMu5IItqx3Q]

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Slack Invite

[Robert Metzger]

I will update the Flink website.

On Thu, May 30, 2024 at 10:08 AM gongzhongqiang 
wrote:

> Hi,
> The invite  link :
>
> https://join.slack.com/t/apache-flink/shared_invite/zt-2jtsd06wy-31q_aELVkdc4dHsx0GMhOQ
>
> Best,
> Zhongqiang Gong
>
> Nelson de Menezes Neto  于2024年5月30日周四 15:01写道：
>
> > Hey guys!
> >
> > I want to join the slack community but the invite has expired..
> > Can u send me a new one?
> >
>

Re: [DISCUSS] Flink CDC 3.2 Release Planning

[Leonard Xu]

Hey, qingsheng

Thanks for driving the 3.2 release forward, it involved more contributions and 
work than we plan at the beginning.
I’d like to help the release management as one release manager, feel free to 
ping me if you need any help from my side.

Best,
Leonard

> 2024年6月3日 下午4:21，Qingsheng Ren  写道：
> 
> Hi devs,
> 
> Considering the current timeline and my workload, it would be great to
> have someone help with managing this release cycle of Flink CDC. Feel
> free to contact me if you are interested!
> 
> Best,
> Qingsheng
> 
> On Mon, May 27, 2024 at 7:39 PM Leonard Xu  wrote:
>> 
>> Hey, Xiqian
>> 
>>> In 3.2 KO plannings, it is expected to add some strongly demanded feature 
>>> to meet YAML pipeline job users’ needs [1]. However, 0 out of 10 3.2 
>>> feature tickets have been completed till now, and it’s very unlikely for us 
>>> to catch up with planned feature freeze (May 25th) and release deadline 
>>> (June 1st) [2].
>>> 
>>> I hereby suggest postponing release schedule by 2 weeks, extending feature 
>>> freeze day to June 8th and release day to June 15th. Considering its wide 
>>> affect, please leave your comments & concerns about this suggestion.
>> 
>> Thanks for the proposal, +1 for extending 2 weeks as the 3.1 release 
>> requires more work than expected.
>> 
>> 
>> Best,
>> Leonard
>> 
>>> 
>>> Regards,
>>> Xiqian
>>> 
>>> [1] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=607
>>> [2] 
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794651
>>> 
>>> 
>>> De : Peter Huang 
>>> Date : jeudi, 9 mai 2024 à 23:31
>>> À : dev@flink.apache.org 
>>> Objet : Re: [DISCUSS] Flink CDC 3.2 Release Planning
>>> Thanks Qingsheng for driving the release!
>>> 
>>> +1. I also would like to provide some help on CDC 3.2.
>>> 
>>> 
>>> Best Regards
>>> Peter Huang
>>> 
>>> On Thu, May 9, 2024 at 3:21 AM Xiqian YU  wrote:
>>> 
 Thanks Qingsheng for driving the release!
 
 +1. Would love to provide my help on CDC 3.2.
 
 Regards,
 Xiqian
 
 De : Hang Ruan 
 Date : jeudi, 9 mai 2024 à 17:50
 À : dev@flink.apache.org 
 Objet : Re: [DISCUSS] Flink CDC 3.2 Release Planning
 Thanks Qinsheng for driving.
 
 I would like to provide some helps for this verison too. +1.
 
 Best,
 Hang
 
 Hongshun Wang  于2024年5月9日周四 14:16写道：
 
> Thanks Qinsheng for driving,
> +1 from my side.
> 
> Besi,
> Hongshun
> 
> On Wed, May 8, 2024 at 11:41 PM Leonard Xu  wrote:
> 
>> +1 for the proposal code freeze date and RM candidate.
>> 
>> Best,
>> Leonard
>> 
>>> 2024年5月8日 下午10:27，gongzhongqiang  写道：
>>> 
>>> Hi Qingsheng
>>> 
>>> Thank you for driving the release.
>>> Agree with the goal and I'm willing to help.
>>> 
>>> Best,
>>> Zhongqiang Gong
>>> 
>>> Qingsheng Ren  于2024年5月8日周三 14:22写道：
>>> 
 Hi devs,
 
 As we are in the midst of the release voting process for Flink CDC
>> 3.1.0, I
 think it's a good time to kick off the upcoming Flink CDC 3.2
 release
 cycle.
 
 In this release cycle I would like to focus on the stability of
 Flink
>> CDC,
 especially for the newly introduced YAML-based data integration
 framework. To ensure we can iterate and improve swiftly, I propose
 to
>> make
 3.2 a relatively short release cycle, targeting a feature freeze by
> May
>> 24,
 2024.
 
 For developers that are interested in participating and contributing
> new
 features in this release cycle, please feel free to list your
 planning
 features in the wiki page [1].
 
 I'm happy to volunteer as a release manager and of course open to
 work
 together with someone on this.
 
 What do you think?
 
 Best,
 Qingsheng
 
 [1]
 
> https://cwiki.apache.org/confluence/display/FLINK/Flink+CDC+3.2+Release
 
>> 
>> 
> 
 
>> 

Re: [DISCUSS] Flink CDC 3.2 Release Planning

[Qingsheng Ren]

Hi devs,

Considering the current timeline and my workload, it would be great to
have someone help with managing this release cycle of Flink CDC. Feel
free to contact me if you are interested!

Best,
Qingsheng

On Mon, May 27, 2024 at 7:39 PM Leonard Xu  wrote:
>
> Hey, Xiqian
>
> > In 3.2 KO plannings, it is expected to add some strongly demanded feature 
> > to meet YAML pipeline job users’ needs [1]. However, 0 out of 10 3.2 
> > feature tickets have been completed till now, and it’s very unlikely for us 
> > to catch up with planned feature freeze (May 25th) and release deadline 
> > (June 1st) [2].
> >
> > I hereby suggest postponing release schedule by 2 weeks, extending feature 
> > freeze day to June 8th and release day to June 15th. Considering its wide 
> > affect, please leave your comments & concerns about this suggestion.
>
> Thanks for the proposal, +1 for extending 2 weeks as the 3.1 release requires 
> more work than expected.
>
>
> Best,
> Leonard
>
> >
> > Regards,
> > Xiqian
> >
> > [1] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=607
> > [2] 
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794651
> >
> >
> > De : Peter Huang 
> > Date : jeudi, 9 mai 2024 à 23:31
> > À : dev@flink.apache.org 
> > Objet : Re: [DISCUSS] Flink CDC 3.2 Release Planning
> > Thanks Qingsheng for driving the release!
> >
> > +1. I also would like to provide some help on CDC 3.2.
> >
> >
> > Best Regards
> > Peter Huang
> >
> > On Thu, May 9, 2024 at 3:21 AM Xiqian YU  wrote:
> >
> >> Thanks Qingsheng for driving the release!
> >>
> >> +1. Would love to provide my help on CDC 3.2.
> >>
> >> Regards,
> >> Xiqian
> >>
> >> De : Hang Ruan 
> >> Date : jeudi, 9 mai 2024 à 17:50
> >> À : dev@flink.apache.org 
> >> Objet : Re: [DISCUSS] Flink CDC 3.2 Release Planning
> >> Thanks Qinsheng for driving.
> >>
> >> I would like to provide some helps for this verison too. +1.
> >>
> >> Best,
> >> Hang
> >>
> >> Hongshun Wang  于2024年5月9日周四 14:16写道：
> >>
> >>> Thanks Qinsheng for driving,
> >>> +1 from my side.
> >>>
> >>> Besi,
> >>> Hongshun
> >>>
> >>> On Wed, May 8, 2024 at 11:41 PM Leonard Xu  wrote:
> >>>
>  +1 for the proposal code freeze date and RM candidate.
> 
>  Best,
>  Leonard
> 
> > 2024年5月8日 下午10:27，gongzhongqiang  写道：
> >
> > Hi Qingsheng
> >
> > Thank you for driving the release.
> > Agree with the goal and I'm willing to help.
> >
> > Best,
> > Zhongqiang Gong
> >
> > Qingsheng Ren  于2024年5月8日周三 14:22写道：
> >
> >> Hi devs,
> >>
> >> As we are in the midst of the release voting process for Flink CDC
>  3.1.0, I
> >> think it's a good time to kick off the upcoming Flink CDC 3.2
> >> release
> >> cycle.
> >>
> >> In this release cycle I would like to focus on the stability of
> >> Flink
>  CDC,
> >> especially for the newly introduced YAML-based data integration
> >> framework. To ensure we can iterate and improve swiftly, I propose
> >> to
>  make
> >> 3.2 a relatively short release cycle, targeting a feature freeze by
> >>> May
>  24,
> >> 2024.
> >>
> >> For developers that are interested in participating and contributing
> >>> new
> >> features in this release cycle, please feel free to list your
> >> planning
> >> features in the wiki page [1].
> >>
> >> I'm happy to volunteer as a release manager and of course open to
> >> work
> >> together with someone on this.
> >>
> >> What do you think?
> >>
> >> Best,
> >> Qingsheng
> >>
> >> [1]
> >>
> >>> https://cwiki.apache.org/confluence/display/FLINK/Flink+CDC+3.2+Release
> >>
> 
> 
> >>>
> >>
>

[jira] [Created] (FLINK-35508) Use OceanBase LTS version Docker image in testing

[He Wang (Jira)]

He Wang created FLINK-35508:
---

 Summary: Use OceanBase LTS version Docker image in testing
 Key: FLINK-35508
 URL: https://issues.apache.org/jira/browse/FLINK-35508
 Project: Flink
  Issue Type: Improvement
  Components: Flink CDC
Reporter: He Wang






--
This message was sent by Atlassian Jira
(v8.20.10#820010)