Re: [DISCUSS] Releasing Flink 1.14.3

Chesnay Schepler Tue, 11 Jan 2022 04:38:57 -0800

The assumption is that everyone that works enough on Flink to volunteeras a RM already has a working azure setup, in which case there isn't anyadditional setup overhead.


We may improve this in the future though.


On 11/01/2022 05:53, Thomas Weise wrote:

Thank you Xingbo. I meanwhile also got my Azure pipeline working and
was able to build the artifacts. Although in general it would be nice
if not every release volunteer had to set up their separate Azure
environment.

Martijn,

The release is staged, except for the website PR:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351075&projectId=12315522
https://dist.apache.org/repos/dist/dev/flink/flink-1.14.3-rc1/
https://repository.apache.org/content/repositories/orgapacheflink-1481/
https://github.com/apache/flink/releases/tag/release-1.14.3-rc1

Would you like to create the website PR and start a VOTE?

(If not, I can look into that tomorrow.)

Cheers,
Thomas



On Sun, Jan 9, 2022 at 9:17 PM Xingbo Huang <hxbks...@gmail.com> wrote:

Hi Thomas,

Since multiple wheel packages with different python versions for mac and
linux are generated, building locally requires you have multiple machines
with different os and Python environments. I have triggered the wheel
package build of release-1.14.3-rc1 in my private Azure[1] and you can
download the wheels after building successfully.

[1]
https://dev.azure.com/hxbks2ks/FLINK-TEST/_build/results?buildId=1704&view=results

Best,
Xingbo

Thomas Weise <t...@apache.org> 于2022年1月10日周一 11:12写道：

Hi Martijn,

I started building the release artifacts. The Maven part is ready.
Currently blocked on the Azure build for the PyFlink wheel packages.

I had to submit a "Azure DevOps Parallelism Request" and that might
take a couple of days.

Does someone have the steps to build the wheels locally?
Alternatively, if someone can build them on their existing setup and
point me to the result, that would speed up things as well.

The release branch:
https://github.com/apache/flink/tree/release-1.14.3-rc1

Thanks,
Thomas

On Thu, Jan 6, 2022 at 9:14 PM Martijn Visser <mart...@ververica.com>
wrote:

Hi Thomas,

Thanks for volunteering! There was no volunteer yet, so would be great if
you could help out.

Best regards,

Martijn

Op vr 7 jan. 2022 om 01:54 schreef Thomas Weise <t...@apache.org>

Hi Martijn,

Thanks for preparing the release. Did a volunteer check in with you?
If not, I would like to take this up.

Thomas

On Mon, Dec 27, 2021 at 7:11 AM Martijn Visser <mart...@ververica.com>
wrote:

Thank you all! That means that there's currently no more blocker to

start

with the Flink 1.14.3 release.

The only thing that's needed is a committer that's willing to follow

the

release process [1] Any volunteers?

Best regards,

Martijn

[1]

https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release

On Mon, 27 Dec 2021 at 03:17, Qingsheng Ren <renqs...@gmail.com>

wrote:

Hi Martjin,

FLINK-25132 has been merged to master and release-1.14.

Thanks for your work for releasing 1.14.3!

Cheers,

Qingsheng Ren

On Dec 26, 2021, at 3:46 PM, Konstantin Knauf <kna...@apache.org

wrote:

Hi Martijn,

FLINK-25375 is merged to release-1.14.

Cheers,

Konstantin

On Wed, Dec 22, 2021 at 12:02 PM David Morávek <d...@apache.org>

wrote:

Hi Martijn, FLINK-25271 has been merged to 1.14 branch.

Best,
D.

On Wed, Dec 22, 2021 at 7:27 AM 任庆盛 <renqs...@gmail.com> wrote:

Hi Martjin,

Thanks for the effort on Flink 1.14.3. FLINK-25132 has been

merged

on

master and is waiting for CI on release-1.14. I think it can be

closed

today.

Cheers,

Qingsheng Ren

On Dec 21, 2021, at 6:26 PM, Martijn Visser <

mart...@ververica.com>

wrote:

Hi everyone,

I'm restarting this thread [1] with a new subject, given that

Flink

1.14.1 was a (cancelled) emergency release for the Log4j

update and

we've

released Flink 1.14.2 as an emergency release for Log4j updates

[2].

To give an update, this is the current blocker for Flink

1.14.3:

* https://issues.apache.org/jira/browse/FLINK-25132 -

KafkaSource

cannot work with object-reusing DeserializationSchema -> @
renqs...@gmail.com can you provide an ETA for this ticket?

There are two critical tickets open for Flink 1.14.3. That

means

that

if

the above ticket is resolved, these two will not block the

release. If

we

can merge them in before the above ticket is completed, that's

bonus.

* https://issues.apache.org/jira/browse/FLINK-25199 -

fromValues

does

not emit final MAX watermark -> @Marios Trivyzas any update or

thoughts

on

this?

* https://issues.apache.org/jira/browse/FLINK-25227 -

Comparing

the

equality of the same (boxed) numeric values returns false ->

@Caizhi

Weng

any update or thoughts on this?

Best regards,

Martijn

[1]

https://lists.apache.org/thread/r0xhs9x01k8hnm0hyq2kk4ptrhkzgdw9

[2]

https://flink.apache.org/news/2021/12/16/log4j-patch-releases.html

On Thu, 9 Dec 2021 at 17:21, David Morávek <d...@apache.org>

wrote:

Hi Martijn, I've just opened a backport PR [1] for FLINK-23946

[2].

[1] https://github.com/apache/flink/pull/18066
[2] https://issues.apache.org/jira/browse/FLINK-23946

Best,
D.

On Thu, Dec 9, 2021 at 4:59 PM Fabian Paul <fp...@apache.org>

wrote:

Actually I meant

https://issues.apache.org/jira/browse/FLINK-25126

sorry for the confusion.

On Thu, Dec 9, 2021 at 4:55 PM Fabian Paul <fp...@apache.org>

wrote:

Hi Martijn,

I just opened the backport for
https://issues.apache.org/jira/browse/FLINK-25132. The

changes

are

already approved I only wait for a green Azure build.

Best,
Fabian

On Thu, Dec 9, 2021 at 4:01 PM Martijn Visser <

mart...@ververica.com

wrote:

Hi all,

Thanks for the fixes Jingsong and Zhu!

That means that we still have the following tickets open:

* https://issues.apache.org/jira/browse/FLINK-23946 -

Application

mode

fails fatally when being shut down -> A PR is there, just

pending a

review.

* https://issues.apache.org/jira/browse/FLINK-25126 - Kafka

connector tries

to commit aborted transaction in batch mode -> I believe

this is

pending a

backport, correct @fp...@apache.org <fp...@apache.org> ?
* https://issues.apache.org/jira/browse/FLINK-25132 -

KafkaSource

cannot

work with object-reusing DeserializationSchema -> @

renqs...@gmail.com

<renqs...@gmail.com> can you provide an ETA for this

ticket?

* https://issues.apache.org/jira/browse/FLINK-25199 -

fromValues

does not

emit final MAX watermark -> @Marios Trivyzas <

mat...@gmail.com>

can

you

provide an ETA for this ticket?
* https://issues.apache.org/jira/browse/FLINK-25227 -

Comparing

the

equality of the same (boxed) numeric values returns false ->

@Caizhi

Weng

<tsreape...@gmail.com> mentioned that a fix is planned for

today/tomorrow.

I am wondering if this is indeed a blocker for 1.14.1, but

given

that there

are still some blockers waiting to be merged we could

probably

include it.

Best regards,

Martijn

On Thu, 9 Dec 2021 at 07:31, Caizhi Weng <

tsreape...@gmail.com>

wrote:

Hi devs!

Sorry for the interruptions, but I just found an issue [1]

(which

I think

is a blocking one) in every Flink version, including Flink

1.14.1.

For Flink < 1.15, this issue will cause incorrect result

when

user

cast

two strings to numerics and compare the numerics.

I'm planning for a quick fix today or tomorrow.

[1] https://issues.apache.org/jira/browse/FLINK-25227

Zhu Zhu <reed...@gmail.com> 于2021年12月9日周四 10:48写道：

update: backport of FLINK-19142 is done

Thanks,
Zhu

Zhu Zhu <reed...@gmail.com> 于2021年12月8日周三 19:35写道：

Hi Martijn,

I'd like to backport the fix of FLINK-19142 to 1.14.1.
The backport is in progress.
Will update it here when it is done.

Thanks,
Zhu

Jingsong Li <jingsongl...@gmail.com> 于2021年12月8日周三

10:33写道：

Hi Martijn,

We just created a cherry-pick pull-request for
https://issues.apache.org/jira/browse/FLINK-20370
We could finish it as soon as possible.

Best,
Jingsong

On Fri, Dec 3, 2021 at 10:25 PM Fabian Paul <

fp...@apache.org>

wrote:

I just opened a PR for
https://issues.apache.org/jira/browse/FLINK-25126 I'll

expect to

merge

it sometime next week.

Best,
Fabian

On Fri, Dec 3, 2021 at 10:49 AM Martijn Visser <

mart...@ververica.com>

wrote:

Hi all,

Just a status update on the open blockers for 1.14.1:
* https://issues.apache.org/jira/browse/FLINK-22113 -

UniqueKey

constraint is lost with multiple sources join in SQL ->

believe most

review comments have been fixed and it's just the final

review

remarks

before it's ready.

* https://issues.apache.org/jira/browse/FLINK-23946 -

Application

mode fails fatally when being shut down -> @David

Morávek

can

you

provide

an update?

* https://issues.apache.org/jira/browse/FLINK-25022 -

ClassLoader

leak with ThreadLocals on the JM when submitting a job

through

the

REST API

-> I think this is just pending on a merge to master and

then

creating

backport?

* https://issues.apache.org/jira/browse/FLINK-25126 -

Kafka

connector tries to commit aborted transaction in batch

mode

->

This is

new blocker. @fp...@apache.org can you give an update?

* https://issues.apache.org/jira/browse/FLINK-25132 -

KafkaSource

cannot work with object-reusing DeserializationSchema ->

There's a PR

that's being reviewed and then needs a backport.

It would be great if we can finish all these blockers

next

week to

start a release. Do the assignees think that's

realistic?

Best regards,

Martijn




On Thu, 2 Dec 2021 at 14:25, Marios Trivyzas <

mat...@gmail.com>

wrote:

https://issues.apache.org/jira/browse/FLINK-22113

will

be merged

today (most probably)

On Mon, Nov 29, 2021 at 10:15 AM Martijn Visser <

mart...@ververica.com> wrote:

Thanks all for the updates! To summarize, these are

open

tickets

that are considered blockers for Flink 1.14.1:

* https://issues.apache.org/jira/browse/FLINK-22113

UniqueKey

constraint is lost with multiple sources join in SQL ->

@Marios

Trivyzas

can you give an estimate when you expect this to be

resolved?

* https://issues.apache.org/jira/browse/FLINK-23946

Application

mode fails fatally when being shut down -> A patch is

being

prepared.

@David Morávek do you have an estimate when this patch

will

be

there?

* https://issues.apache.org/jira/browse/FLINK-24596

Bugs in

sink.buffer-flush before upsert-kafka -> @

fp...@apache.org

has

provided

a PR is there, so I suspect it would take a couple of

days

before this

is

merged.

* https://issues.apache.org/jira/browse/FLINK-25022

ClassLoader

leak with ThreadLocals on the JM when submitting a job

through

the

REST API

-> @Chesnay Schepler has provided a PR, so I suspect it

would

also just

take a couple of days before this is merged.

Is there anyone who can help me with creating the

actual

release

when these tickets are resolved

Best regards,

Martijn


On Fri, 26 Nov 2021 at 12:08, Chesnay Schepler <

ches...@apache.org>

wrote:

FLINK-25022: I will open a PR later today, and it

should be

easy to

backport.
FLINK-25027: Unlikely to make it for 1.14.1; I also

wouldn't

consider it

a blocker

On 24/11/2021 19:40, Martijn Visser wrote:

Hi all,

I would like to start a discussion on releasing

Flink

1.14.1.

Flink 1.14

was released on the 29th of September [1] and so

far

issues

have been

resolved, including multiple blockers and critical

priorities

[2].

There are currently 169 open tickets which

contain a

fixVersion

for 1.14.1

[3]. I'm including the ones that are currently

marked

as

critical or a

blocker to verify if these should be included in

Flink 1.14.1.

It would be

great if those that are assigned or working on one

or

more of

these tickets

can give an update on its status.

*

https://issues.apache.org/jira/browse/FLINK-24543

Zookeeper

connection

issue causes inconsistent state in Flink -> I

think

this

depends

on the

outcome of dropping Zookeeper 3.4 as was proposed

on

the Dev

mailing list

https://issues.apache.org/jira/browse/FLINK-25027

- Allow

GC

of a

finished job's JobMaster before the slot timeout

is

reached

https://issues.apache.org/jira/browse/FLINK-25022

ClassLoader leak with

ThreadLocals on the JM when submitting a job

through

the REST

API

https://issues.apache.org/jira/browse/FLINK-24789

IllegalStateException

with CheckpointCleaner being closed already
*

https://issues.apache.org/jira/browse/FLINK-24328

- Long

term

fix for

receiving new buffer size before network reader

configured ->

I'm not sure

if this would end up in Flink 1.14.1, I think it's

more likely

that it

would be Flink 1.15. Anton/Dawid, could you

confirm

this?

https://issues.apache.org/jira/browse/FLINK-23946

Application mode

fails fatally when being shut down -> This depends

on

https://issues.apache.org/jira/browse/FLINK-24038

and I don't

see much

happening there, so I also expect that this would

move to

Flink

1.15.

David, could you confirm?
*

https://issues.apache.org/jira/browse/FLINK-22113

UniqueKey

constraint

is lost with multiple sources join in SQL
*

https://issues.apache.org/jira/browse/FLINK-21788

- Throw

PartitionNotFoundException if the partition file

has

been lost

for blocking

shuffle -> I'm also expecting that this would move

to

Flink

1.15, can you

confirm Yingjie ?

There are quite some other tickets that I've

excluded

from

this

list,

because they are either test instabilities or are

not

depending

on a Flink

release to be resolved.

Note: there are quite a few test instabilities in

the

list and

help on

those is always appreciated. You can check all

unassigned

tickets

instabilities in Jira [4].

Are there any other open tickets that we should

wait

for? Is

there a PMC

member who would like to manage the release? I'm

more

than

happy

to help

with monitoring the status of the tickets.

Best regards,

Martijn

[1]

https://flink.apache.org/news/2021/09/29/release-1.14.0.html

[2]

https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC

[3]

https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC

[4]

https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20AND%20labels%20%3D%20test-stability%20AND%20assignee%20in%20(EMPTY)%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC

Martijn Visser | Product Manager

mart...@ververica.com

<https://www.ververica.com/>


Follow us @VervericaData

--

Join Flink Forward <https://flink-forward.org/> -

The Apache

Flink

Conference

Stream Processing | Event Driven | Real Time


--
Marios



--
Best, Jingsong Lee


--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

--

Martijn Visser | Product Manager

mart...@ververica.com

<https://www.ververica.com/>


Follow us @VervericaData

--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

Re: [DISCUSS] Releasing Flink 1.14.3

Reply via email to