Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Yang Wang]

Congratulations Yuan!

Best,
Yang

XING JIN  于2021年7月8日周四 上午11:46写道：

> Congratulations Yuan~!
>
> Roc Marshal  于2021年7月8日周四 上午11:28写道：
>
> > Congratulations, Yuan!
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > At 2021-07-08 01:21:40, "Yu Li"  wrote:
> > >Hi all,
> > >
> > >On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> > >committer.
> > >
> > >Yuan has been an active contributor for more than two years, with code
> > >contributions on multiple components including kafka connectors,
> > >checkpointing, state backends, etc. Besides, she has been actively
> > involved
> > >in community activities such as helping manage releases, discussing
> > >questions on dev@list, supporting users and giving talks at
> conferences.
> > >
> > >Please join me in congratulating Yuan for becoming a Flink committer!
> > >
> > >Cheers,
> > >Yu
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[XING JIN]

Congratulations Yuan~!

Roc Marshal  于2021年7月8日周四 上午11:28写道：

> Congratulations, Yuan!
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> At 2021-07-08 01:21:40, "Yu Li"  wrote:
> >Hi all,
> >
> >On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> >committer.
> >
> >Yuan has been an active contributor for more than two years, with code
> >contributions on multiple components including kafka connectors,
> >checkpointing, state backends, etc. Besides, she has been actively
> involved
> >in community activities such as helping manage releases, discussing
> >questions on dev@list, supporting users and giving talks at conferences.
> >
> >Please join me in congratulating Yuan for becoming a Flink committer!
> >
> >Cheers,
> >Yu
>

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[XING JIN]

Congratulations Yang Wang!

Austin Cawley-Edwards  于2021年7月7日周三 下午9:29写道：

> Congrats Yang!
>
> On Wed, Jul 7, 2021 at 7:26 AM Jingsong Li  wrote:
>
> > Congratulations, Yang!
> >
> > Best,
> > Jingsong
> >
> > On Wed, Jul 7, 2021 at 6:43 PM Arvid Heise  wrote:
> >
> > > Congratulations!
> > >
> > > On Wed, Jul 7, 2021 at 12:17 PM godfrey he 
> wrote:
> > >
> > > > Congratulations, Yang!
> > > >
> > > > Best,
> > > > Godfrey
> > > >
> > > > Lijie Wang  于2021年7月7日周三 下午5:59写道：
> > > >
> > > > > Congratulations Yang!
> > > > >
> > > > > Till Rohrmann  于2021年7月7日周三 下午5:29写道：
> > > > >
> > > > > > Congratulations, Yang!
> > > > > >
> > > > > > Cheers,
> > > > > > Till
> > > > > >
> > > > > > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan <
> ro...@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Congrats!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Roman
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren <
> renqs...@gmail.com
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > Congratulations Yang!
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best Regards,
> > > > > > > >
> > > > > > > > Qingsheng Ren
> > > > > > > > Email: renqs...@gmail.com
> > > > > > > > On Jul 7, 2021, 2:26 PM +0800, Rui Li  >,
> > > > > wrote:
> > > > > > > > > Congratulations Yang ~
> > > > > > > > >
> > > > > > > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li <
> > > libenc...@apache.org>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Congratulations!
> > > > > > > > > >
> > > > > > > > > > Peter Huang  于2021年7月7日周三
> > > > 下午12:54写道：
> > > > > > > > > >
> > > > > > > > > > > Congratulations, Yang.
> > > > > > > > > > >
> > > > > > > > > > > Best Regards
> > > > > > > > > > > Peter Huang
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu <
> > > > dian0511...@gmail.com>
> > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Congratulations, Yang,
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > > Dian
> > > > > > > > > > > >
> > > > > > > > > > > > > 2021年7月7日 上午10:46，Jary Zhen 
> 写道：
> > > > > > > > > > > > >
> > > > > > > > > > > > > Congratulations, Yang Wang.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best
> > > > > > > > > > > > > Jary
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yun Gao 
> 于2021年7月7日周三
> > > > > > 上午10:38写道：
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Congratulations Yang!
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Best,
> > > > > > > > > > > > > > Yun
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > >
> > --
> > > > > > > > > > > > > > Sender:Jark Wu
> > > > > > > > > > > > > > Date:2021/07/07 10:20:27
> > > > > > > > > > > > > > Recipient:dev
> > > > > > > > > > > > > > Cc:Yang Wang; <
> > > > > > > wangyang0...@apache.org>
> > > > > > > > > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer -
> > > Yang
> > > > > Wang
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Congratulations Yang Wang!
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Best,
> > > > > > > > > > > > > > Jark
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song <
> > > > > > > tonysong...@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi everyone,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On behalf of the PMC, I'm very happy to
> announce
> > > Yang
> > > > > > Wang
> > > > > > > as a new
> > > > > > > > > > > > Flink
> > > > > > > > > > > > > > > committer.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Yang has been a very active contributor for
> more
> > > than
> > > > > two
> > > > > > > years,
> > > > > > > > > > > mainly
> > > > > > > > > > > > > > > focusing on Flink's deployment components.
> He's a
> > > > main
> > > > > > > contributor
> > > > > > > > > > > and
> > > > > > > > > > > > > > > maintainer of Flink's native Kubernetes
> > deployment
> > > > and
> > > > > > > native
> > > > > > > > > > > > Kubernetes
> > > > > > > > > > > > > > > HA. He's also very active on the mailing lists,
> > > > > > > participating in
> > > > > > > > > > > > > > > discussions and helping with user questions.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Please join me in congratulating Yang Wang for
> > > > > becoming a
> > > > > > > Flink
> > > > > > > > > > > > > > committer!
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thank you~
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Xintong Song
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > 

[jira] [Created] (FLINK-23304) More detail information in sql validate exception

[YING HOU (Jira)]

YING HOU created FLINK-23304:


 Summary: More detail information in sql validate exception
 Key: FLINK-23304
 URL: https://issues.apache.org/jira/browse/FLINK-23304
 Project: Flink
  Issue Type: Improvement
  Components: Table SQL / API, Table SQL / Planner
Affects Versions: 1.13.1, 1.13.0
Reporter: YING HOU
 Fix For: 1.13.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Lifecycle of ShuffleMaster and its Relationship with JobMaster and PartitionTracker

[Yingjie Cao]

Hi,

Thanks for the reply.

@Guowei
I agree that we can move forward step by step and start from the most
important part. Apart from the two points mentioned in your reply,
initializing and shutting down some external resources gracefully is also
important which is a reason for the open/close method.
About the cluster partitions and the ShuffleMasterContext, I agree that we
can postpone handling the cluster partitions because we need to do more to
support it, for ShuffleMasterContext, I think we still need it even we do
not support the cluster partitions in the first step. Currently, the
shuffle master can only access the cluster configuration, except that, I
think we also need need the ability of handling the fatal errors occurring
in the  ShuffleMaster gracefully by propagate the errors to the framework.
By introducing the ShuffleMasterContext, we can give ShuffleMaster the
ability to access both the  cluster configuration and the fatal error
handler. Instead of passing these components directly to the ShuffleMaster,
a ShuffleMasterContext interface can keep compatibility easily in the
future. Even we add some new method in the future, we can offer default
empty implementation in the interface which can keep compatibility.
About the JobShuffleContext::getConfiguration/listPartitions methods, I
agree that we can remove them in the first step and we can add them back
latter. As mentioned above, we can easily keep compatibility based on the
Context interface.

@Till
I totally agree that we should support that different jobs use different
shuffle services and the proposed solution will support this use case
eventually.

Best,
Yingjie

Till Rohrmann  于2021年7月7日周三 下午8:15写道：

> One quick comment: When developing the ShuffleService abstraction we also
> thought that different jobs might want to use different ShuffleServices
> depending on their workload (e.g. batch vs. streaming workload). So
> ideally, the chosen solution here can also support this use case
> eventually.
>
> Cheers,
> Till
>
> On Wed, Jul 7, 2021 at 12:50 PM Guowei Ma  wrote:
>
> > Hi,
> > Thank Yingjie for initiating this discussion. What I understand that the
> > document[1] actually mainly discusses two issues:
> > 1. ShuffleMaster should be at the cluster level instead of the job level
> > 2. ShuffleMaster should notify PartitionTracker that some data has been
> > lost
> >
> > Relatively speaking, I think the second problem is more serious. Because
> > for external or remote batch shuffling services, after the machine
> storing
> > shuffled data goes offline, PartitionTracker needs to be notified in time
> > to avoid repeated failures of the job. Therefore, it is hoped that when
> > shuffle data goes offline due to a machine error, ShuffleMaster can
> notify
> > the PartitionTracker in time. This requires ShuffleMaster to notify the
> > PartitionTracker with a handle such as JobShuffleContext.
> >
> > So how to pass JobShuffleContext to ShuffleMaster? There are two options:
> > 1. After ShuffleMaster is created, pass JobShuffleContext to
> ShuffleMaster,
> > such as ShuffleMaster::register(JobShuffleContext)
> > 2. Pass ShuffleServiceFactory when creating ShuffleMaster, such as
> > ShuffleServiceFatroctry.createcreateShuffleMaster(JobShuffleContext).
> >
> > Which one to choose is actually related to issue 1. Because if
> > ShuffleMaster is a cluster level, you should choose option 1, otherwise,
> > choose option 2. I think ShuffleMaster should be at the cluster level,
> for
> > example, because we don't need to maintain a ShuffleMaster for each job
> in
> > a SessionCluster; in addition, this ShuffleMaster should also be used by
> > RM's PartitionTracker in the future. Therefore, I think Option 1 is more
> > appropriate.
> >
> > To sum up, we may give priority to solving problem 2, while taking into
> > account that ShuffleMaster should be a cluster-level component.
> Therefore,
> > I think we could ignore the date ShuffleMasterContext at the beginning;
> at
> > the same time, JobShuffleContext::getConfiguration/listPartitions should
> > not be needed.
> >
> > [1]
> >
> >
> https://docs.google.com/document/d/1_cHoapNbx_fJ7ZNraSqw4ZK1hMRiWWJDITuSZrdMDDs/edit
> >
> > Best,
> > Guowei
> >
> >
> > On Fri, Jun 11, 2021 at 4:15 PM Yingjie Cao 
> > wrote:
> >
> > > Hi devs,
> > >
> > > I'd like to start a discussion about "Lifecycle of ShuffleMaster and
> its
> > > Relationship with JobMaster and PartitionTracker". (These are things we
> > > found when moving our external shuffle to the pluggable shuffle service
> > > framework.)
> > >
> > > The mail client may fail to display the right format. If so, please
> refer
> > > to this document:
> > >
> > >
> >
> https://docs.google.com/document/d/1_cHoapNbx_fJ7ZNraSqw4ZK1hMRiWWJDITuSZrdMDDs/edit?usp=sharing
> > > .
> > > Lifecycle of ShuffleMaster
> > >
> > > Currently, the lifecycle of ShuffleMaster seems unclear.  The
> > > ShuffleServiceFactory is loaded for each JobMaster instance and then
> > > 

Re:[ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Roc Marshal]

Congratulations, Yuan!















At 2021-07-08 01:21:40, "Yu Li"  wrote:
>Hi all,
>
>On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
>committer.
>
>Yuan has been an active contributor for more than two years, with code
>contributions on multiple components including kafka connectors,
>checkpointing, state backends, etc. Besides, she has been actively involved
>in community activities such as helping manage releases, discussing
>questions on dev@list, supporting users and giving talks at conferences.
>
>Please join me in congratulating Yuan for becoming a Flink committer!
>
>Cheers,
>Yu

Re: Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[JING ZHANG]

Congratulations, Yuan!

Rui Li  于2021年7月8日周四 上午10:52写道：

> Congratulations Yuan!
>
> On Thu, Jul 8, 2021 at 10:51 AM Yun Gao 
> wrote:
>
> > Congratulations Yuan!
> >
> > Best,
> > Yun
> >
> >
> >
> >  --Original Mail --
> > Sender:Zhilong Hong 
> > Send Date:Thu Jul 8 10:50:36 2021
> > Recipients:dev (dev@flink.apache.org) 
> > Subject:Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei
> > Congratulations, Yuan!
> >
> > Best,
> > Zhilong
> >
> > On Thu, Jul 8, 2021 at 1:22 AM Yu Li  wrote:
> >
> > > Hi all,
> > >
> > > On behalf of the PMC, I’m very happy to announce Yuan Mei as a new
> Flink
> > > committer.
> > >
> > > Yuan has been an active contributor for more than two years, with code
> > > contributions on multiple components including kafka connectors,
> > > checkpointing, state backends, etc. Besides, she has been actively
> > involved
> > > in community activities such as helping manage releases, discussing
> > > questions on dev@list, supporting users and giving talks at
> conferences.
> > >
> > > Please join me in congratulating Yuan for becoming a Flink committer!
> > >
> > > Cheers,
> > > Yu
> > >
> >
>
>
> --
> Best regards!
> Rui Li
>

Re: Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Rui Li]

Congratulations Yuan!

On Thu, Jul 8, 2021 at 10:51 AM Yun Gao 
wrote:

> Congratulations Yuan!
>
> Best,
> Yun
>
>
>
>  --Original Mail --
> Sender:Zhilong Hong 
> Send Date:Thu Jul 8 10:50:36 2021
> Recipients:dev (dev@flink.apache.org) 
> Subject:Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei
> Congratulations, Yuan!
>
> Best,
> Zhilong
>
> On Thu, Jul 8, 2021 at 1:22 AM Yu Li  wrote:
>
> > Hi all,
> >
> > On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> > committer.
> >
> > Yuan has been an active contributor for more than two years, with code
> > contributions on multiple components including kafka connectors,
> > checkpointing, state backends, etc. Besides, she has been actively
> involved
> > in community activities such as helping manage releases, discussing
> > questions on dev@list, supporting users and giving talks at conferences.
> >
> > Please join me in congratulating Yuan for becoming a Flink committer!
> >
> > Cheers,
> > Yu
> >
>


-- 
Best regards!
Rui Li

Re: Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Yun Gao]

Congratulations Yuan!

Best,
Yun



 --Original Mail --
Sender:Zhilong Hong 
Send Date:Thu Jul 8 10:50:36 2021
Recipients:dev (dev@flink.apache.org) 
Subject:Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei
Congratulations, Yuan!

Best,
Zhilong

On Thu, Jul 8, 2021 at 1:22 AM Yu Li  wrote:

> Hi all,
>
> On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> committer.
>
> Yuan has been an active contributor for more than two years, with code
> contributions on multiple components including kafka connectors,
> checkpointing, state backends, etc. Besides, she has been actively involved
> in community activities such as helping manage releases, discussing
> questions on dev@list, supporting users and giving talks at conferences.
>
> Please join me in congratulating Yuan for becoming a Flink committer!
>
> Cheers,
> Yu
>

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Zhilong Hong]

Congratulations, Yuan!

Best,
Zhilong

On Thu, Jul 8, 2021 at 1:22 AM Yu Li  wrote:

> Hi all,
>
> On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> committer.
>
> Yuan has been an active contributor for more than two years, with code
> contributions on multiple components including kafka connectors,
> checkpointing, state backends, etc. Besides, she has been actively involved
> in community activities such as helping manage releases, discussing
> questions on dev@list, supporting users and giving talks at conferences.
>
> Please join me in congratulating Yuan for becoming a Flink committer!
>
> Cheers,
> Yu
>

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Lijie Wang]

Congratulations, Yuan.

Zakelly Lan  于2021年7月8日周四 上午10:16写道：

> Congratulations, Yuan.
>
> Best,
> Zakelly
>
> On Thu, Jul 8, 2021 at 10:14 AM Yun Tang  wrote:
>
> > Congratulations, Yuan
> >
> > Best
> > Yun Tang
> > 
> > From: Yu Li 
> > Sent: Thursday, July 8, 2021 1:21
> > To: dev ; Yuan Mei ;
> > yuan...@apache.org 
> > Subject: [ANNOUNCE] New Apache Flink Committer - Yuan Mei
> >
> > Hi all,
> >
> > On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> > committer.
> >
> > Yuan has been an active contributor for more than two years, with code
> > contributions on multiple components including kafka connectors,
> > checkpointing, state backends, etc. Besides, she has been actively
> involved
> > in community activities such as helping manage releases, discussing
> > questions on dev@list, supporting users and giving talks at conferences.
> >
> > Please join me in congratulating Yuan for becoming a Flink committer!
> >
> > Cheers,
> > Yu
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Zakelly Lan]

Congratulations, Yuan.

Best,
Zakelly

On Thu, Jul 8, 2021 at 10:14 AM Yun Tang  wrote:

> Congratulations, Yuan
>
> Best
> Yun Tang
> 
> From: Yu Li 
> Sent: Thursday, July 8, 2021 1:21
> To: dev ; Yuan Mei ;
> yuan...@apache.org 
> Subject: [ANNOUNCE] New Apache Flink Committer - Yuan Mei
>
> Hi all,
>
> On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> committer.
>
> Yuan has been an active contributor for more than two years, with code
> contributions on multiple components including kafka connectors,
> checkpointing, state backends, etc. Besides, she has been actively involved
> in community activities such as helping manage releases, discussing
> questions on dev@list, supporting users and giving talks at conferences.
>
> Please join me in congratulating Yuan for becoming a Flink committer!
>
> Cheers,
> Yu
>

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Yun Tang]

Congratulations, Yuan

Best
Yun Tang

From: Yu Li 
Sent: Thursday, July 8, 2021 1:21
To: dev ; Yuan Mei ; 
yuan...@apache.org 
Subject: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

Hi all,

On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
committer.

Yuan has been an active contributor for more than two years, with code
contributions on multiple components including kafka connectors,
checkpointing, state backends, etc. Besides, she has been actively involved
in community activities such as helping manage releases, discussing
questions on dev@list, supporting users and giving talks at conferences.

Please join me in congratulating Yuan for becoming a Flink committer!

Cheers,
Yu

[jira] [Created] (FLINK-23303) org.apache.calcite.rex.RexLiteral cannot be cast to org.apache.calcite.rex.RexCall

[xiaojin.wy (Jira)]

xiaojin.wy created FLINK-23303:
--

 Summary: org.apache.calcite.rex.RexLiteral cannot be cast to 
org.apache.calcite.rex.RexCall
 Key: FLINK-23303
 URL: https://issues.apache.org/jira/browse/FLINK-23303
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Planner
Affects Versions: 1.14.0
Reporter: xiaojin.wy


CREATE TABLE database5_t1(
`c0` SMALLINT , `c1` INTEGER , `c2` SMALLINT
) WITH (
 'connector' = 'filesystem',
 'format' = 'testcsv',
 'path' = '$resultPath11'
)

INSERT INTO database5_t1(c0, c1, c2) VALUES(cast(-21957 as SMALLINT), 
1094690065, cast(16917 as SMALLINT))

SELECT database5_t1.c0 AS ref0 FROM database5_t1 WHERE (FALSE) NOT IN (((NOT 
CAST ((database5_t1.c0) AS BOOLEAN))) = (database5_t1.c0))

*After excuting the sql above, you will get the errors:
*

java.lang.ClassCastException: org.apache.calcite.rex.RexLiteral cannot be cast 
to org.apache.calcite.rex.RexCall

at 
org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:478)
at 
org.apache.flink.table.planner.plan.utils.RexNodeToExpressionConverter.visitCall(RexNodeExtractor.scala:367)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:174)
at 
org.apache.flink.table.planner.plan.utils.RexNodeExtractor$$anonfun$extractConjunctiveConditions$1.apply(RexNodeExtractor.scala:138)
at 
org.apache.flink.table.planner.plan.utils.RexNodeExtractor$$anonfun$extractConjunctiveConditions$1.apply(RexNodeExtractor.scala:137)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at 
org.apache.flink.table.planner.plan.utils.RexNodeExtractor$.extractConjunctiveConditions(RexNodeExtractor.scala:137)
at 
org.apache.flink.table.planner.plan.utils.RexNodeExtractor.extractConjunctiveConditions(RexNodeExtractor.scala)
at 
org.apache.flink.table.planner.plan.rules.logical.PushFilterIntoSourceScanRuleBase.extractPredicates(PushFilterIntoSourceScanRuleBase.java:145)
at 
org.apache.flink.table.planner.plan.rules.logical.PushFilterIntoTableSourceScanRule.pushFilterIntoScan(PushFilterIntoTableSourceScanRule.java:81)
at 
org.apache.flink.table.planner.plan.rules.logical.PushFilterIntoTableSourceScanRule.onMatch(PushFilterIntoTableSourceScanRule.java:70)
at 
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
at 
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
at 
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:243)
at 
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
at 
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:202)
at 
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:189)
at 
org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:69)
at 
org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:87)
at 
org.apache.flink.table.planner.plan.optimize.program.FlinkGroupProgram$$anonfun$optimize$1$$anonfun$apply$1.apply(FlinkGroupProgram.scala:63)
at 
org.apache.flink.table.planner.plan.optimize.program.FlinkGroupProgram$$anonfun$optimize$1$$anonfun$apply$1.apply(FlinkGroupProgram.scala:60)
at 
scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
at 
scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at 
scala.collection.TraversableOnce$class.foldLeft(TraversableOnce.scala:157)
at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
at 
org.apache.flink.table.planner.plan.optimize.program.FlinkGroupProgram$$anonfun$optimize$1.apply(FlinkGroupProgram.scala:60)
at 
org.apache.flink.table.planner.plan.optimize.program.FlinkGroupProgram$$anonfun$optimize$1.apply(FlinkGroupProgram.scala:55)
at 
scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
at 
scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
at 

[jira] [Created] (FLINK-23302) Precondition failed building CheckpointMetrics.

[Kyle Weaver (Jira)]

Kyle Weaver created FLINK-23302:
---

 Summary: Precondition failed building CheckpointMetrics.
 Key: FLINK-23302
 URL: https://issues.apache.org/jira/browse/FLINK-23302
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Checkpointing
Reporter: Kyle Weaver


Beam has a flaky test using Flink savepoints. It looks like 
alignmentDurationNanos is less than -1, which shouldn't be possible. As far as 
I know clients (like Beam) don't have any control over this value, so my best 
guess is that it's a bug in Flink.

See 
https://issues.apache.org/jira/browse/BEAM-10955?focusedCommentId=17376928=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17376928
 for context.

The failing test is here: 
[https://github.com/apache/beam/blob/b401d23dfc2a487ae5775164a7834952391ff4fa/runners/flink/src/test/java/org/apache/beam/runners/flink/FlinkSavepointTest.java#L146]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

[Xintong Song]

Thanks, Etienne. Comments addressed.

Thank you~

Xintong Song



On Wed, Jul 7, 2021 at 6:41 PM Etienne Chauchot 
wrote:

> Hi,
>
> Thanks Xintong for the very good doc ! I added 2 comments to it.
>
> Best,
>
> Etienne
>
> On 02/07/2021 05:57, Xintong Song wrote:
> > Thanks all for the positive feedback.
> >
> > I have updated the wiki page [1], and will send an announcement in a
> > separate thread, to draw more committers' attention.
> >
> > Moreover, I've opened FLINK-23212 where we can continue with the
> discussion
> > around pure documentation changing PRs.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> > [1]
> https://cwiki.apache.org/confluence/display/FLINK/Merging+Pull+Requests
> >
> > On Wed, Jun 30, 2021 at 5:26 PM Xintong Song 
> wrote:
> >
> >> I second Tison's opinion.
> >>
> >> Similar to how we skip docs_404_check for PRs that do not touch the
> >> documentation, we can skip other stages for PRs that only contain
> >> documentation changes.
> >>
> >> In addition to making merging documentation PRs easier, we can also
> reduce
> >> the workload on CI workers. Especially during the last days of a release
> >> cycle, which is usually the most busy time for the CI workers, and is
> also
> >> where most documentation efforts take place.
> >>
> >> Thank you~
> >>
> >> Xintong Song
> >>
> >>
> >>
> >> On Wed, Jun 30, 2021 at 3:56 PM Till Rohrmann 
> >> wrote:
> >>
> >>> I think you are right Tison that docs are a special case and they only
> >>> require flink-docs to pass. What I am wondering is how much of a
> problem
> >>> this will be (assuming that we have a decent build stability). The more
> >>> exceptions we add, the harder it will be to properly follow the
> >>> guidelines.
> >>> Maybe we can observe how many docs PRs get delayed/not merged because
> of
> >>> this and then revisit this discussion if needed.
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Wed, Jun 30, 2021 at 8:30 AM tison  wrote:
> >>>
>  Hi,
> 
>  There are a number of PRs modifying docs only, but we still require
> all
>  tests passed on that.
> 
>  It is a good proposal we avoid merge PR with "unrelated" failure, but
> >>> can
>  we improve the case where the contributor only works for docs?
> 
>  For example, base on the file change set, run doc tests only.
> 
>  Best,
>  tison.
> 
> 
>  godfrey he  于2021年6月30日周三 下午2:17写道：
> 
> > +1 for the proposal. Thanks Xintong!
> >
> > Best,
> > Godfrey
> >
> >
> >
> > Rui Li  于2021年6月30日周三 上午11:36写道：
> >
> >> Thanks Xintong. +1 to the proposal.
> >>
> >> On Tue, Jun 29, 2021 at 11:05 AM 刘建刚 
>  wrote:
> >>> +1 for the proposal. Since the test time is long and environment
> >>> may
> >> vary,
> >>> unstable tests are really annoying for developers. The solution is
> >> welcome.
> >>> Best
> >>> liujiangang
> >>>
> >>> Jingsong Li  于2021年6月29日周二 上午10:31写道：
> >>>
>  +1 Thanks Xintong for the update!
> 
>  Best,
>  Jingsong
> 
>  On Mon, Jun 28, 2021 at 6:44 PM Till Rohrmann <
>  trohrm...@apache.org>
>  wrote:
> 
> > +1, thanks for updating the guidelines Xintong!
> >
> > Cheers,
> > Till
> >
> > On Mon, Jun 28, 2021 at 11:49 AM Yangze Guo <
> >>> karma...@gmail.com>
> >>> wrote:
> >> +1
> >>
> >> Thanks Xintong for drafting this doc.
> >>
> >> Best,
> >> Yangze Guo
> >>
> >> On Mon, Jun 28, 2021 at 5:42 PM JING ZHANG <
>  beyond1...@gmail.com
>  wrote:
> >>> Thanks Xintong for giving detailed documentation.
> >>>
> >>> The best practice for handling test failure is very
> >>> detailed,
> >> it's
> >>> a
> > good
> >>> guidelines document with clear action steps.
> >>>
> >>> +1 to Xintong's proposal.
> >>>
> >>> Xintong Song  于2021年6月28日周一
> >>> 下午4:07写道：
>  Thanks all for the discussion.
> 
>  Based on the opinions so far, I've drafted the new
>  guidelines
> >>> [1],
> > as a
>  potential replacement of the original wiki page [2].
> 
>  Hopefully this draft has covered the most opinions
>  discussed
> >> and
> >> consensus
>  made in this discussion thread.
> 
>  Looking forward to your feedback.
> 
>  Thank you~
> 
>  Xintong Song
> 
> 
>  [1]
> 
> 
> >>>
> https://docs.google.com/document/d/1uUbxbgbGErBXtmEjhwVhBWG3i6nhQ0LXs96OlntEYnU/edit?usp=sharing
>  [2]
> 
> 
> https://cwiki.apache.org/confluence/display/FLINK/Merging+Pull+Requests
> 
> 
>  On 

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Yangze Guo]

Congrats, Yuan!
Best,
Yangze Guo

On Thu, Jul 8, 2021 at 9:49 AM Qingsheng Ren  wrote:
>
> Congratulations Yuan!
>
> --
> Best Regards,
>
> Qingsheng Ren
> Email: renqs...@gmail.com
> 2021年7月8日 +0800 09:45 Xintong Song ，写道：
> > Congratulations Yuan~!
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Thu, Jul 8, 2021 at 4:39 AM Till Rohrmann  wrote:
> >
> > > Congratulations Yuan.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Jul 7, 2021 at 7:22 PM Yu Li  wrote:
> > >
> > > > Hi all,
> > > >
> > > > On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> > > > committer.
> > > >
> > > > Yuan has been an active contributor for more than two years, with code
> > > > contributions on multiple components including kafka connectors,
> > > > checkpointing, state backends, etc. Besides, she has been actively
> > > involved
> > > > in community activities such as helping manage releases, discussing
> > > > questions on dev@list, supporting users and giving talks at conferences.
> > > >
> > > > Please join me in congratulating Yuan for becoming a Flink committer!
> > > >
> > > > Cheers,
> > > > Yu
> > > >
> > >

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Leonard Xu]

Congratulations Yuan.

Best,
Leonard
> 在 2021年7月8日，09:49，Qingsheng Ren  写道：
> 
> Congratulations Yuan!
> 
> --
> Best Regards,
> 
> Qingsheng Ren
> Email: renqs...@gmail.com
> 2021年7月8日 +0800 09:45 Xintong Song ，写道：
>> Congratulations Yuan~!
>> 
>> Thank you~
>> 
>> Xintong Song
>> 
>> 
>> 
>> On Thu, Jul 8, 2021 at 4:39 AM Till Rohrmann  wrote:
>> 
>>> Congratulations Yuan.
>>> 
>>> Cheers,
>>> Till
>>> 
>>> On Wed, Jul 7, 2021 at 7:22 PM Yu Li  wrote:
>>> 
 Hi all,
 
 On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
 committer.
 
 Yuan has been an active contributor for more than two years, with code
 contributions on multiple components including kafka connectors,
 checkpointing, state backends, etc. Besides, she has been actively
>>> involved
 in community activities such as helping manage releases, discussing
 questions on dev@list, supporting users and giving talks at conferences.
 
 Please join me in congratulating Yuan for becoming a Flink committer!
 
 Cheers,
 Yu
 
>>> 

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Qingsheng Ren]

Congratulations Yuan!

--
Best Regards,

Qingsheng Ren
Email: renqs...@gmail.com
2021年7月8日 +0800 09:45 Xintong Song ，写道：
> Congratulations Yuan~!
>
> Thank you~
>
> Xintong Song
>
>
>
> On Thu, Jul 8, 2021 at 4:39 AM Till Rohrmann  wrote:
>
> > Congratulations Yuan.
> >
> > Cheers,
> > Till
> >
> > On Wed, Jul 7, 2021 at 7:22 PM Yu Li  wrote:
> >
> > > Hi all,
> > >
> > > On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> > > committer.
> > >
> > > Yuan has been an active contributor for more than two years, with code
> > > contributions on multiple components including kafka connectors,
> > > checkpointing, state backends, etc. Besides, she has been actively
> > involved
> > > in community activities such as helping manage releases, discussing
> > > questions on dev@list, supporting users and giving talks at conferences.
> > >
> > > Please join me in congratulating Yuan for becoming a Flink committer!
> > >
> > > Cheers,
> > > Yu
> > >
> >

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Xintong Song]

Congratulations Yuan~!

Thank you~

Xintong Song



On Thu, Jul 8, 2021 at 4:39 AM Till Rohrmann  wrote:

> Congratulations Yuan.
>
> Cheers,
> Till
>
> On Wed, Jul 7, 2021 at 7:22 PM Yu Li  wrote:
>
> > Hi all,
> >
> > On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> > committer.
> >
> > Yuan has been an active contributor for more than two years, with code
> > contributions on multiple components including kafka connectors,
> > checkpointing, state backends, etc. Besides, she has been actively
> involved
> > in community activities such as helping manage releases, discussing
> > questions on dev@list, supporting users and giving talks at conferences.
> >
> > Please join me in congratulating Yuan for becoming a Flink committer!
> >
> > Cheers,
> > Yu
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Till Rohrmann]

Congratulations Yuan.

Cheers,
Till

On Wed, Jul 7, 2021 at 7:22 PM Yu Li  wrote:

> Hi all,
>
> On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
> committer.
>
> Yuan has been an active contributor for more than two years, with code
> contributions on multiple components including kafka connectors,
> checkpointing, state backends, etc. Besides, she has been actively involved
> in community activities such as helping manage releases, discussing
> questions on dev@list, supporting users and giving talks at conferences.
>
> Please join me in congratulating Yuan for becoming a Flink committer!
>
> Cheers,
> Yu
>

[jira] [Created] (FLINK-23301) StateFun HTTP Ingress

[Stephan Ewen (Jira)]

Stephan Ewen created FLINK-23301:


 Summary: StateFun HTTP Ingress
 Key: FLINK-23301
 URL: https://issues.apache.org/jira/browse/FLINK-23301
 Project: Flink
  Issue Type: Sub-task
  Components: Stateful Functions
Reporter: Stephan Ewen


The HTTP ingress would start an HTTP Server at a specified port.

The HTTP server would only handle _POST_ requests. The target function is 
represented by the path to which the request is made, the message contents is 
the body of the POST request.

The following example would send an empty message to the function with the 
address \{{namespace='example', type='greeter', id='Igal'}}.

{code}
curl -X POST http://statefun-ingress:/in/example/greeter/Igal

POST /in/example/greeter/Igal HTTP/1.1
Content-Length: 0
{code}

The example below would send empty message of type 'statefun/string' to the 
function with the address \{{namespace='example', type='greeter', id='Elisa'}} 
and the message contents\{{"{numTimesToGreet: 5}"}}.

curl -X POST -H "Content-Type: text/plain; charset=UTF-8" -d 
"\{numTimesToGreet: 5}" http://statefun-ingress:/in/example/greeter/Elisa

POST /in/example/greeter/Elisa HTTP/1.1
Content-Type: text/plain; charset=UTF-8
Content-Length: 20

{numTimesToGreet: 5}
{code}


h3. Data Types

The content type (mime type) specified in the request header of the HTTP 
request will be directly mapped to the statefun types.
For example, a \{{Content-Type: io.statefun.tyes/int}} will set the type of the 
message to \{{io.statefun.tyes/int}}.

As a special case, we map the content type \{{text/plain}} to 
\{{io.statefun.tyes/string}}, to make simple cases and examples work more 
seamlessly.

The following examples would send a message to a function that expectes a 
ProtoBuf encoded type \{{Greeting}} registerd in StateFun as 
\{{example/greeting}}.

{code}
> curl -X POST -H "Content-Type: text/plain; charset=UTF-8" -d 
> "\{numTimesToGreet: 5}"

> CONTENTS=`echo 'numTimesToGreet: 5' | ./protoc --encode Greeting 
> example_types.proto`
> echo $CONTENTS | curl -X POST -H "Content-Type: example/greeting" 
> --data-binary @- http://statefun-ingress:/in/example/greeter/Bobby
{code}


h3. Sender and Responses

We want to support round-trip-style interactions, meaning posting a request and 
receiving a response.
Given the async messaging nature of StateFun, this might not be necessarily in 
one HTTP request which immediately gives you the corresponsing response. 
Instead, it can be in issuing (POST) a request to the HTTP ingress and polling 
(GET) a response from an associated HTTP Egress.

To support these kind of patterns, the HTTP ingress will assign a random 
request correlation ID in the HTTP response.
Furthermore, the ingress will optionally set the \{{sender()}} field of the 
created message to reference a configured associated egress.

The ingress config woud add an entry referencing the egress (like 
\{{'paired_egress_name: httpout'}}).

{code}
> curl -X POST -i http://statefun-ingress:/in/example/greeter/Igal
POST /in/example/greeter/Igal HTTP/1.1
Content-Length: 0

HTTP/1.1 200 OK
StateFun-Request-Correlation-ID: 8acb377c-fc5e-4bdb-b2cc-eddb5992b7b5
Content-Length: 0
{code}

The created message would have no body, but would have the \{{sender() = 
{{egress/httpout/8acb377c-fc5e-4bdb-b2cc-eddb5992b7b5}}.

_Note: We would need to extend the message address scheme to be able to 
reference egresses.
The egress itself can grab the correlation ID from the ID part of the address, 
because the HTTP egres doesn't use that field (in fact, no egress currently 
interprets the ID)._

h3. Singleton Instantiation

To avoid port conflicts, we need to do a singleton instantiation per JVM.
This can be achieved by using a statically referenced context to hold the 
instantiated servier and a reference to the

In the future, we can look into extending this to avoid setup/teardown when 
operators are cancelled for recovery.
The server would then live as long as the StateFun application (job) lives (or 
more precisely, as long as the slot lives, which is the duration that the 
TaskManager is associated with the StateFun deployment - typically the entire 
lifetime).

To achieve that, we would tear down the server in a [shutdown hook of the 
user-code 
classloader](https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/functions/RuntimeContext.java#L145).
 Instead of letting the first source set up the server, the first source would 
register its output as the stream for the server to push messages to.


h3. Configuration parameters

- Bind host (default 0.0.0.0)
 - Bind port (default )
 - Path (default "in") (for the path in the URL 
\{{http(s)://:}})

- Egress pair name, for setting the egress that replies should go to.

- To setup SSL for the connection, we add similar settings as for Flink's REST 

[ANNOUNCE] New Apache Flink Committer - Yuan Mei

[Yu Li]

Hi all,

On behalf of the PMC, I’m very happy to announce Yuan Mei as a new Flink
committer.

Yuan has been an active contributor for more than two years, with code
contributions on multiple components including kafka connectors,
checkpointing, state backends, etc. Besides, she has been actively involved
in community activities such as helping manage releases, discussing
questions on dev@list, supporting users and giving talks at conferences.

Please join me in congratulating Yuan for becoming a Flink committer!

Cheers,
Yu

Re: [DISCUSS] Incrementally deprecating the DataSet API

[Timo Walther]

Hi Etienne,

sorry for the late reply due to my vacation last week.

Regarding: "support of aggregations in batch mode for DataStream API 
[...] is there a plan to solve it before the actual drop of DataSet API"


Just to clarify it again: we will not drop the DataSet API any time 
soon. So users will have enough time to update their pipelines. There 
are a couple of features missing to fully switch to DataStream API in 
batch mode. Thanks for opening an issue, this is helpful for us to 
gradually remove those barriers. They don't need to have a "Blocker" 
priority in JIRA for now.


But aggregations is a good example where we should discuss if it would 
be easier to simply switch to Table API for that. Table API has a lot of 
aggregation optimizations and can work on binary data. Also joins should 
be easier in Table API. DataStream API can be a very low-level API in 
the near future and most use cases (esp. the batch ones) should be 
possible in Table API.


Regarding: "Is it needed to port these Avro enhancements to new 
DataStream connectors (add a new equivalent of 
ParquetColumnarRowInputFormat but for Avro)"


We should definitely not loose functionality. The same functionality 
should be present in the new connectors. The questions is rather whether 
we need to offer a DataStream API connector or if a Table API connector 
would be nicer to use (also nicely integrated with catalogs).


So a user can use a simple CREATE TABLE statement to configure the 
connector; an easier abstraction is almost not possible. With 
`tableEnv.toDataStream(table)` you can then continue in DataStream API 
if there is still a need for it.


Regarding: "there are parquet bugs still open on deprecated parquet 
connector"


Yes, bugs should still be fixed in 1.13.

Regarading: "I've been doing TPCDS benchmarks with Flink lately"

Great to hear that :-)

Did you also see the recent discussion? A TPC-DS benchmark can further 
be improved by providing statistics. Maybe this is helpful to you:


https://lists.apache.org/thread.html/ra383c23f230ab8e7fa16ec64b4f277c267d6358d55cc8a0edc77bb63%40%3Cuser.flink.apache.org%3E

I will prepare a blog post shortly.

Regards,
Timo



On 06.07.21 15:05, Etienne Chauchot wrote:

Hi all,

Any comments ?

cheers,

Etienne

On 25/06/2021 15:09, Etienne Chauchot wrote:

Hi everyone,

@Timo, my comments are inline for steps 2, 4 and 5, please tell me 
what you think.


Best

Etienne


On 23/06/2021 15:27, Chesnay Schepler wrote:
If we want to publicize this plan more shouldn't we have a rough 
timeline for when 2.0 is on the table?


On 6/23/2021 2:44 PM, Stephan Ewen wrote:

Thanks for writing this up, this also reflects my understanding.

I think a blog post would be nice, ideally with an explicit call for
feedback so we learn about user concerns.
A blog post has a lot more reach than an ML thread.

Best,
Stephan


On Wed, Jun 23, 2021 at 12:23 PM Timo Walther  
wrote:



Hi everyone,

I'm sending this email to make sure everyone is on the same page about
slowly deprecating the DataSet API.

There have been a few thoughts mentioned in presentations, offline
discussions, and JIRA issues. However, I have observed that there are
still some concerns or different opinions on what steps are 
necessary to

implement this change.

Let me summarize some of the steps and assumpations and let's have a
discussion about it:

Step 1: Introduce a batch mode for Table API (FLIP-32)
[DONE in 1.9]

Step 2: Introduce a batch mode for DataStream API (FLIP-134)
[DONE in 1.12]



I've been using DataSet API and I tested migrating to DataStream + 
batch mode.


I opened this (1) ticket regarding the support of aggregations in 
batch mode for DataStream API. It seems that join operation (at least) 
does not work in batch mode even though I managed to implement a join 
using low level KeyedCoProcessFunction (thanks Seth, for the pointer !).


=> Should it be considered a blocker ? Is there a plan to solve it 
before the actual drop of DataSet API ? Maybe in step 6 ?


[1] https://issues.apache.org/jira/browse/FLINK-22587




Step 3: Soft deprecate DataSet API (FLIP-131)
[DONE in 1.12]

We updated the documentation recently to make this deprecation even 
more

visible. There is a dedicated `(Legacy)` label right next to the menu
item now.

We won't deprecate concrete classes of the API with a @Deprecated
annotation to avoid extensive warnings in logs until then.

Step 4: Drop the legacy SQL connectors and formats (FLINK-14437)
[DONE in 1.14]

We dropped code for ORC, Parque, and HBase formats that were only used
by DataSet API users. The removed classes had no documentation and 
were

not annotated with one of our API stability annotations.

The old functionality should be available through the new sources and
sinks for Table API and DataStream API. If not, we should bring them
into a shape that they can be a full replacement.

DataSet users are encouraged to either upgrade the API or use Flink
1.13. Users can either 

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Austin Cawley-Edwards]

Congrats Yang!

On Wed, Jul 7, 2021 at 7:26 AM Jingsong Li  wrote:

> Congratulations, Yang!
>
> Best,
> Jingsong
>
> On Wed, Jul 7, 2021 at 6:43 PM Arvid Heise  wrote:
>
> > Congratulations!
> >
> > On Wed, Jul 7, 2021 at 12:17 PM godfrey he  wrote:
> >
> > > Congratulations, Yang!
> > >
> > > Best,
> > > Godfrey
> > >
> > > Lijie Wang  于2021年7月7日周三 下午5:59写道：
> > >
> > > > Congratulations Yang!
> > > >
> > > > Till Rohrmann  于2021年7月7日周三 下午5:29写道：
> > > >
> > > > > Congratulations, Yang!
> > > > >
> > > > > Cheers,
> > > > > Till
> > > > >
> > > > > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan  >
> > > > wrote:
> > > > >
> > > > > > Congrats!
> > > > > >
> > > > > > Regards,
> > > > > > Roman
> > > > > >
> > > > > >
> > > > > > On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren  >
> > > > wrote:
> > > > > > >
> > > > > > > Congratulations Yang!
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards,
> > > > > > >
> > > > > > > Qingsheng Ren
> > > > > > > Email: renqs...@gmail.com
> > > > > > > On Jul 7, 2021, 2:26 PM +0800, Rui Li ,
> > > > wrote:
> > > > > > > > Congratulations Yang ~
> > > > > > > >
> > > > > > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li <
> > libenc...@apache.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Congratulations!
> > > > > > > > >
> > > > > > > > > Peter Huang  于2021年7月7日周三
> > > 下午12:54写道：
> > > > > > > > >
> > > > > > > > > > Congratulations, Yang.
> > > > > > > > > >
> > > > > > > > > > Best Regards
> > > > > > > > > > Peter Huang
> > > > > > > > > >
> > > > > > > > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu <
> > > dian0511...@gmail.com>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Congratulations, Yang,
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Dian
> > > > > > > > > > >
> > > > > > > > > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > > > > > > > > >
> > > > > > > > > > > > Congratulations, Yang Wang.
> > > > > > > > > > > >
> > > > > > > > > > > > Best
> > > > > > > > > > > > Jary
> > > > > > > > > > > >
> > > > > > > > > > > > Yun Gao  于2021年7月7日周三
> > > > > 上午10:38写道：
> > > > > > > > > > > >
> > > > > > > > > > > > > Congratulations Yang!
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Yun
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > >
> --
> > > > > > > > > > > > > Sender:Jark Wu
> > > > > > > > > > > > > Date:2021/07/07 10:20:27
> > > > > > > > > > > > > Recipient:dev
> > > > > > > > > > > > > Cc:Yang Wang; <
> > > > > > wangyang0...@apache.org>
> > > > > > > > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer -
> > Yang
> > > > Wang
> > > > > > > > > > > > >
> > > > > > > > > > > > > Congratulations Yang Wang!
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Jark
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song <
> > > > > > tonysong...@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi everyone,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On behalf of the PMC, I'm very happy to announce
> > Yang
> > > > > Wang
> > > > > > as a new
> > > > > > > > > > > Flink
> > > > > > > > > > > > > > committer.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Yang has been a very active contributor for more
> > than
> > > > two
> > > > > > years,
> > > > > > > > > > mainly
> > > > > > > > > > > > > > focusing on Flink's deployment components. He's a
> > > main
> > > > > > contributor
> > > > > > > > > > and
> > > > > > > > > > > > > > maintainer of Flink's native Kubernetes
> deployment
> > > and
> > > > > > native
> > > > > > > > > > > Kubernetes
> > > > > > > > > > > > > > HA. He's also very active on the mailing lists,
> > > > > > participating in
> > > > > > > > > > > > > > discussions and helping with user questions.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Please join me in congratulating Yang Wang for
> > > > becoming a
> > > > > > Flink
> > > > > > > > > > > > > committer!
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thank you~
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Xintong Song
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Benchao Li
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards!
> > > > > > > > Rui Li
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
> Best, Jingsong Lee
>

Re: [DISCUSS] Lifecycle of ShuffleMaster and its Relationship with JobMaster and PartitionTracker

[Till Rohrmann]

One quick comment: When developing the ShuffleService abstraction we also
thought that different jobs might want to use different ShuffleServices
depending on their workload (e.g. batch vs. streaming workload). So
ideally, the chosen solution here can also support this use case eventually.

Cheers,
Till

On Wed, Jul 7, 2021 at 12:50 PM Guowei Ma  wrote:

> Hi,
> Thank Yingjie for initiating this discussion. What I understand that the
> document[1] actually mainly discusses two issues:
> 1. ShuffleMaster should be at the cluster level instead of the job level
> 2. ShuffleMaster should notify PartitionTracker that some data has been
> lost
>
> Relatively speaking, I think the second problem is more serious. Because
> for external or remote batch shuffling services, after the machine storing
> shuffled data goes offline, PartitionTracker needs to be notified in time
> to avoid repeated failures of the job. Therefore, it is hoped that when
> shuffle data goes offline due to a machine error, ShuffleMaster can notify
> the PartitionTracker in time. This requires ShuffleMaster to notify the
> PartitionTracker with a handle such as JobShuffleContext.
>
> So how to pass JobShuffleContext to ShuffleMaster? There are two options:
> 1. After ShuffleMaster is created, pass JobShuffleContext to ShuffleMaster,
> such as ShuffleMaster::register(JobShuffleContext)
> 2. Pass ShuffleServiceFactory when creating ShuffleMaster, such as
> ShuffleServiceFatroctry.createcreateShuffleMaster(JobShuffleContext).
>
> Which one to choose is actually related to issue 1. Because if
> ShuffleMaster is a cluster level, you should choose option 1, otherwise,
> choose option 2. I think ShuffleMaster should be at the cluster level, for
> example, because we don't need to maintain a ShuffleMaster for each job in
> a SessionCluster; in addition, this ShuffleMaster should also be used by
> RM's PartitionTracker in the future. Therefore, I think Option 1 is more
> appropriate.
>
> To sum up, we may give priority to solving problem 2, while taking into
> account that ShuffleMaster should be a cluster-level component. Therefore,
> I think we could ignore the date ShuffleMasterContext at the beginning; at
> the same time, JobShuffleContext::getConfiguration/listPartitions should
> not be needed.
>
> [1]
>
> https://docs.google.com/document/d/1_cHoapNbx_fJ7ZNraSqw4ZK1hMRiWWJDITuSZrdMDDs/edit
>
> Best,
> Guowei
>
>
> On Fri, Jun 11, 2021 at 4:15 PM Yingjie Cao 
> wrote:
>
> > Hi devs,
> >
> > I'd like to start a discussion about "Lifecycle of ShuffleMaster and its
> > Relationship with JobMaster and PartitionTracker". (These are things we
> > found when moving our external shuffle to the pluggable shuffle service
> > framework.)
> >
> > The mail client may fail to display the right format. If so, please refer
> > to this document:
> >
> >
> https://docs.google.com/document/d/1_cHoapNbx_fJ7ZNraSqw4ZK1hMRiWWJDITuSZrdMDDs/edit?usp=sharing
> > .
> > Lifecycle of ShuffleMaster
> >
> > Currently, the lifecycle of ShuffleMaster seems unclear.  The
> > ShuffleServiceFactory is loaded for each JobMaster instance and then
> > ShuffleServiceFactory#createShuffleMaster will be called to create a
> > ShuffleMaser instance. However, the default NettyShuffleServiceFactory
> > always returns the same ShuffleMaser singleton instance for all jobs.
> Based
> > on the current implementation, the lifecycle of ShuffleMaster seems open
> > and depends on the shuffle plugin themselves. However, at the TM side,
> > the ShuffleEnvironment
> > is a part of the TaskManagerServices whose lifecycle is decoupled with
> jobs
> > which is more like a service. It means there is also an inconsistency
> > between the TM side and the JM side.
> >
> > From my understanding, the reason for this is that the pluggable shuffle
> > framework is still not completely finished yet, for example, there is a
> > follow up umbrella ticket  FLINK-19551
> >  for the pluggable
> > shuffle service framework and in its subtasks, there is one task (
> > FLINK-12731 ) which
> > aims
> > to load shuffle plugin with the PluginManager. I think this can solve the
> > issue mentioned above. After the corresponding factory  loaded by the
> > PluginManager, all ShuffleMaster instances can be stored in a map indexed
> > by the corresponding factory class name  which can be shared by all jobs.
> > After that, the ShuffleMaster becomes a cluster level service which is
> > consistent with the ShuffleEnvironment at the TM side.
> >
> > As a summary, we propose to finish FLINK-12731
> >  and make the shuffle
> > service a real cluster level service first. Furthermore, we add two
> > lifecycle methods to the ShuffleMaster interface, including start and
> > close responding
> > for initialization (for example, contacting the external system) and
> > graceful shutdown 

Re: [VOTE] Release 1.13.2, release candidate #1

[Yun Tang]

Since FLINK-23233 brings concerns to many guys, and this release candidate #1 
had not received enough binding +1 from PMCs, I think we could cancel this RC 
and wait for that problem resolved.

Best
Yun Tang

From: Till Rohrmann 
Sent: Wednesday, July 7, 2021 17:21
To: dev 
Cc: Xintong Song 
Subject: Re: [VOTE] Release 1.13.2, release candidate #1

I think FLINK-23233 sounds like a very serious problem to me. We either
continue releasing 1.13.2 and do an immediate 1.13.3 afterwards or we
cancel this RC and fix the problem and then resume the 1.13.2 release.

Cheers,
Till

On Wed, Jul 7, 2021 at 5:49 AM JING ZHANG  wrote:

> Thanks Xintong for bringing the alarm, +1 on Xintong's proposal to hold the
> vote until deciding what to do with FLINK-23233.
>
> BTW, I made a typo in the previous voting. I mean +1 (non-binding) instead
> of +1 (binding). I'm very sorry for the confusion.
>
> +1 (non-binding).
>
> 1. Reviewed website pull request
> 2. Built from source code flink-1.13.2-src.tgz
> <
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-src.tgz
> >
>  succeeded
> 3. Started a local Flink cluster, ran the WordCount example, WebUI looks
> good,  no suspicious output/log
> 4. Started cluster and run some e2e sql queries using SQL Client, query
> result is expected.
> 5. Repeat Step 3 and 4 with flink-1.13.2-bin-scala_2.11.tgz
> <
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-bin-scala_2.11.tgz
> >
>
> Best regards,
> JING ZHANG
>
> Xintong Song  于2021年7月7日周三 上午11:09写道：
>
> > Hi everyone,
> >
> > We find a bug that may cause data loss in a rare condition, which IMHO
> is a
> > release blocker. Please see FLINK-23233 [1] for details.
> >
> > Given that this bug is not newly introduced but already exists in 1.13.0
> &
> > 1.13.1 releases, I would not replace my +1 with a -1 immediately.
> Instead,
> > I would like to ask the release manager for holding this vote until
> > deciding what to do with FLINK-23233.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-23233
> >
> > On Wed, Jul 7, 2021 at 12:09 AM JING ZHANG  wrote:
> >
> > > +1 (binding)
> > >
> > > 1. Reviewed website pull request
> > > 2. Built from source code flink-1.13.2-src.tgz
> > > <
> > >
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-src.tgz
> > > >
> > > succeeded
> > > 3. Started a local Flink cluster, ran the WordCount example, WebUI
> looks
> > > good,  no suspicious output/log
> > > 4. Started cluster and run some e2e sql queries using SQL Client, query
> > > result is expected.
> > > 5. Repeat Step 3 and 4 with flink-1.13.2-bin-scala_2.11.tgz
> > > <
> > >
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-bin-scala_2.11.tgz
> > > >
> > >
> > > Best regards,
> > > JING ZHANG
> > >
> > >
> > > Zakelly Lan  于2021年7月5日周一 下午6:06写道：
> > >
> > > > +1 (non-binding)
> > > >
> > > > - built from sources
> > > > - run streaming job of wordcount
> > > > - web-ui looks good
> > > > - checkpoint and restore looks good
> > > >
> > > > Best,
> > > > Zakelly
> > > >
> > > > On Mon, Jul 5, 2021 at 2:40 PM Jingsong Li 
> > > wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > - Verified checksums and signatures
> > > > > - Built from sources
> > > > > - run table example jobs
> > > > > - web-ui looks good
> > > > > - sql-client looks good
> > > > >
> > > > > I think we should update the unresolved JIRAs in [1] to 1.13.3.
> > > > >
> > > > > And we should check resolved JIRAs in [2], commits of some are not
> in
> > > the
> > > > > 1.13.2. We should exclude them. For example FLINK-23196 FLINK-23166
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.13.2%20AND%20status%20not%20in%20(Closed%2C%20Resolved)%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12350218==12315522
> > > > >
> > > > > Best,
> > > > > Jingsong
> > > > >
> > > > > On Mon, Jul 5, 2021 at 2:27 PM Xingbo Huang 
> > > wrote:
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > - Verified checksums and signatures
> > > > > > - Built from sources
> > > > > > - Verified Python wheel package contents
> > > > > > - Pip install Python wheel package in Mac
> > > > > > - Run Python UDF job in Python shell
> > > > > >
> > > > > > Best,
> > > > > > Xingbo
> > > > > >
> > > > > > Yangze Guo  于2021年7月5日周一 上午11:17写道：
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > - built from sources
> > > > > > > - run example jobs with standalone and yarn.
> > > > > > > - check TaskManager's rest API from the JM master and its
> > standby,
> > > > > > > everything looks good
> > > > > > >
> > > > > > > Best,
> > > > > > > Yangze Guo
> > > > > > >
> > > 

Re: [VOTE] Release 1.12.5, release candidate #1

[Jingsong Li]

Hi all,

Thanks Xiongtong, Leonard, Yang, JING for the voting. Thanks Till for the
information.

+1 for canceling this RC. That should also give us the chance to fix
FLINK-23233.

Best,
Jingsong

On Wed, Jul 7, 2021 at 5:29 PM Till Rohrmann  wrote:

> Hi folks, are we sure that FLINK-23233 [1] does not affect the 1.12 release
> branch. I think this problem was introduced with FLINK-21996 [2] which is
> part of release-1.12. Hence, we might either fix this problem and cancel
> this RC or we need to create a fast 1.12.6 afterwards.
>
> [1] https://issues.apache.org/jira/browse/FLINK-23233
> [2] https://issues.apache.org/jira/browse/FLINK-21996
>
> Cheers,
> Till
>
> On Wed, Jul 7, 2021 at 7:43 AM JING ZHANG  wrote:
>
> > +1 (non-binding)
> >
> > 1. built from source code flink-1.12.5-src.tgz
> > <
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-1.12.5-rc1/flink-1.12.5-src.tgz
> > >
> > succeeded
> > 2. Started a local Flink cluster, ran the WordCount example, WebUI looks
> > good,  no suspicious output/log
> > 3. started cluster and run some e2e sql queries using SQL Client, query
> > result is expected.
> > 4. Repeat Step 2 and 3 with flink-1.12.5-bin-scala_2.11.tgz
> > <
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-1.12.5-rc1/flink-1.12.5-bin-scala_2.11.tgz
> > >
> >
> > Best,
> > JING ZHANG
> >
> > Yang Wang  于2021年7月7日周三 上午11:36写道：
> >
> > > +1 (non-binding)
> > >
> > > - verified checksums & signatures
> > > - start a session cluster and verify HA data cleanup once job reached
> to
> > > globally terminal state, FLINK-20695
> > > - start a local standalone cluster, check the webUI good and JM/TM
> > > without suspicious logs
> > >
> > >
> > > Best,
> > > Yang
> > >
> > > Leonard Xu  于2021年7月7日周三 上午10:52写道：
> > >
> > > > +1 (non-binding)
> > > >
> > > > - verified signatures and hashsums
> > > > - built from source code with scala 2.11 succeeded
> > > > - checked all denpendency artifacts are 1.12.5
> > > > - started a cluster, ran a wordcount job, the result is expected
> > > > - started SQL Client, ran a simple query, the result is expected
> > > > - reviewed the web PR, left one minor name comment
> > > >
> > > > Best,
> > > > Leonard
> > > >
> > > > > 在 2021年7月6日，10:02，Xintong Song  写道：
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > - verified checksums & signatures
> > > > > - built from sources
> > > > > - run example jobs with standalone and native k8s deployments
> > > > >
> > > > > Thank you~
> > > > >
> > > > > Xintong Song
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Jul 5, 2021 at 11:18 AM Jingsong Li <
> jingsongl...@gmail.com>
> > > > wrote:
> > > > >
> > > > >> Hi everyone,
> > > > >>
> > > > >> Please review and vote on the release candidate #1 for the version
> > > > 1.12.5,
> > > > >> as follows:
> > > > >> [ ] +1, Approve the release
> > > > >> [ ] -1, Do not approve the release (please provide specific
> > comments)
> > > > >>
> > > > >> The complete staging area is available for your review, which
> > > includes:
> > > > >> * JIRA release notes [1],
> > > > >> * the official Apache source release and binary convenience
> releases
> > > to
> > > > be
> > > > >> deployed to dist.apache.org [2], which are signed with the key
> with
> > > > >> fingerprint FBB83C0A4FFB9CA8 [3],
> > > > >> * all artifacts to be deployed to the Maven Central Repository
> [4],
> > > > >> * source code tag "release-1.12.5-rc1" [5],
> > > > >> * website pull request listing the new release and adding
> > announcement
> > > > blog
> > > > >> post [6].
> > > > >>
> > > > >> The vote will be open for at least 72 hours. It is adopted by
> > majority
> > > > >> approval, with at least 3 PMC affirmative votes.
> > > > >>
> > > > >> Best,
> > > > >> Jingsong Lee
> > > > >>
> > > > >> [1]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12350166
> > > > >> [2]
> https://dist.apache.org/repos/dist/dev/flink/flink-1.12.5-rc1/
> > > > >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > > >> [4]
> > > >
> https://repository.apache.org/content/repositories/orgapacheflink-1430
> > > > >> [5]
> https://github.com/apache/flink/releases/tag/release-1.12.5-rc1
> > > > >> [6] https://github.com/apache/flink-web/pull/455
> > > > >>
> > > >
> > > >
> > >
> >
>


-- 
Best, Jingsong Lee

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Jingsong Li]

Congratulations, Yang!

Best,
Jingsong

On Wed, Jul 7, 2021 at 6:43 PM Arvid Heise  wrote:

> Congratulations!
>
> On Wed, Jul 7, 2021 at 12:17 PM godfrey he  wrote:
>
> > Congratulations, Yang!
> >
> > Best,
> > Godfrey
> >
> > Lijie Wang  于2021年7月7日周三 下午5:59写道：
> >
> > > Congratulations Yang!
> > >
> > > Till Rohrmann  于2021年7月7日周三 下午5:29写道：
> > >
> > > > Congratulations, Yang!
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan 
> > > wrote:
> > > >
> > > > > Congrats!
> > > > >
> > > > > Regards,
> > > > > Roman
> > > > >
> > > > >
> > > > > On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren 
> > > wrote:
> > > > > >
> > > > > > Congratulations Yang!
> > > > > >
> > > > > > --
> > > > > > Best Regards,
> > > > > >
> > > > > > Qingsheng Ren
> > > > > > Email: renqs...@gmail.com
> > > > > > On Jul 7, 2021, 2:26 PM +0800, Rui Li ,
> > > wrote:
> > > > > > > Congratulations Yang ~
> > > > > > >
> > > > > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li <
> libenc...@apache.org>
> > > > > wrote:
> > > > > > >
> > > > > > > > Congratulations!
> > > > > > > >
> > > > > > > > Peter Huang  于2021年7月7日周三
> > 下午12:54写道：
> > > > > > > >
> > > > > > > > > Congratulations, Yang.
> > > > > > > > >
> > > > > > > > > Best Regards
> > > > > > > > > Peter Huang
> > > > > > > > >
> > > > > > > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu <
> > dian0511...@gmail.com>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Congratulations, Yang,
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Dian
> > > > > > > > > >
> > > > > > > > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > > > > > > > >
> > > > > > > > > > > Congratulations, Yang Wang.
> > > > > > > > > > >
> > > > > > > > > > > Best
> > > > > > > > > > > Jary
> > > > > > > > > > >
> > > > > > > > > > > Yun Gao  于2021年7月7日周三
> > > > 上午10:38写道：
> > > > > > > > > > >
> > > > > > > > > > > > Congratulations Yang!
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Yun
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > --
> > > > > > > > > > > > Sender:Jark Wu
> > > > > > > > > > > > Date:2021/07/07 10:20:27
> > > > > > > > > > > > Recipient:dev
> > > > > > > > > > > > Cc:Yang Wang; <
> > > > > wangyang0...@apache.org>
> > > > > > > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer -
> Yang
> > > Wang
> > > > > > > > > > > >
> > > > > > > > > > > > Congratulations Yang Wang!
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Jark
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song <
> > > > > tonysong...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi everyone,
> > > > > > > > > > > > >
> > > > > > > > > > > > > On behalf of the PMC, I'm very happy to announce
> Yang
> > > > Wang
> > > > > as a new
> > > > > > > > > > Flink
> > > > > > > > > > > > > committer.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yang has been a very active contributor for more
> than
> > > two
> > > > > years,
> > > > > > > > > mainly
> > > > > > > > > > > > > focusing on Flink's deployment components. He's a
> > main
> > > > > contributor
> > > > > > > > > and
> > > > > > > > > > > > > maintainer of Flink's native Kubernetes deployment
> > and
> > > > > native
> > > > > > > > > > Kubernetes
> > > > > > > > > > > > > HA. He's also very active on the mailing lists,
> > > > > participating in
> > > > > > > > > > > > > discussions and helping with user questions.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Please join me in congratulating Yang Wang for
> > > becoming a
> > > > > Flink
> > > > > > > > > > > > committer!
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thank you~
> > > > > > > > > > > > >
> > > > > > > > > > > > > Xintong Song
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Benchao Li
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards!
> > > > > > > Rui Li
> > > > >
> > > >
> > >
> >
>


-- 
Best, Jingsong Lee

Re: [ANNOUNCE] New PMC member: Guowei Ma

[Jingsong Li]

Congratulations, Guowei!

Best,
Jingsong

On Wed, Jul 7, 2021 at 6:36 PM Arvid Heise  wrote:

> Congratulations!
>
> On Wed, Jul 7, 2021 at 11:30 AM Till Rohrmann 
> wrote:
>
> > Congratulations, Guowei!
> >
> > Cheers,
> > Till
> >
> > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan 
> wrote:
> >
> > > Congratulations!
> > >
> > > Regards,
> > > Roman
> > >
> > > On Wed, Jul 7, 2021 at 8:24 AM Rui Li  wrote:
> > > >
> > > > Congratulations Guowei!
> > > >
> > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> > wrote:
> > > >
> > > > > Congratulations!
> > > > >
> > > > > Dian Fu  于2021年7月7日周三 下午12:46写道：
> > > > >
> > > > > > Congratulations, Guowei!
> > > > > >
> > > > > > Regards,
> > > > > > Dian
> > > > > >
> > > > > > > 2021年7月7日 上午10:37，Yun Gao  写道：
> > > > > > >
> > > > > > > Congratulations Guowei!
> > > > > > >
> > > > > > >
> > > > > > > Best,
> > > > > > > Yun
> > > > > > >
> > > > > > >
> > > > > > >
> > --
> > > > > > > Sender:JING ZHANG
> > > > > > > Date:2021/07/07 10:33:51
> > > > > > > Recipient:dev
> > > > > > > Theme:Re: [ANNOUNCE] New PMC member: Guowei Ma
> > > > > > >
> > > > > > > Congratulations,  Guowei Ma!
> > > > > > >
> > > > > > > Best regards,
> > > > > > > JING ZHANG
> > > > > > >
> > > > > > > Zakelly Lan  于2021年7月7日周三 上午10:30写道：
> > > > > > >
> > > > > > >> Congratulations, Guowei!
> > > > > > >>
> > > > > > >> Best,
> > > > > > >> Zakelly
> > > > > > >>
> > > > > > >> On Wed, Jul 7, 2021 at 10:24 AM tison 
> > > wrote:
> > > > > > >>
> > > > > > >>> Congrats! NB.
> > > > > > >>>
> > > > > > >>> Best,
> > > > > > >>> tison.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Jark Wu  于2021年7月7日周三 上午10:20写道：
> > > > > > >>>
> > > > > >  Congratulations Guowei!
> > > > > > 
> > > > > >  Best,
> > > > > >  Jark
> > > > > > 
> > > > > >  On Wed, 7 Jul 2021 at 09:54, XING JIN <
> > jinxing.co...@gmail.com>
> > > > > > wrote:
> > > > > > 
> > > > > > > Congratulations, Guowei~ !
> > > > > > >
> > > > > > > Best,
> > > > > > > Jin
> > > > > > >
> > > > > > > Xintong Song  于2021年7月7日周三
> 上午9:37写道：
> > > > > > >
> > > > > > >> Congratulations, Guowei~!
> > > > > > >>
> > > > > > >> Thank you~
> > > > > > >>
> > > > > > >> Xintong Song
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, Jul 7, 2021 at 9:31 AM Qingsheng Ren <
> > > renqs...@gmail.com>
> > > > > >  wrote:
> > > > > > >>
> > > > > > >>> Congratulations Guowei!
> > > > > > >>>
> > > > > > >>> --
> > > > > > >>> Best Regards,
> > > > > > >>>
> > > > > > >>> Qingsheng Ren
> > > > > > >>> Email: renqs...@gmail.com
> > > > > > >>> 2021年7月7日 +0800 09:30 Leonard Xu ，写道：
> > > > > >  Congratulations! Guowei Ma
> > > > > > 
> > > > > >  Best,
> > > > > >  Leonard
> > > > > > 
> > > > > > > ÔÚ 2021Äê7ÔÂ6ÈÕ£¬21:56£¬Kurt Young 
> > > дµÀ£º
> > > > > > >
> > > > > > > Hi all!
> > > > > > >
> > > > > > > I'm very happy to announce that Guowei Ma has joined
> the
> > > > > > >> Flink
> > > > > >  PMC!
> > > > > > >
> > > > > > > Congratulations and welcome Guowei!
> > > > > > >
> > > > > > > Best,
> > > > > > > Kurt
> > > > > > 
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > 
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best,
> > > > > Benchao Li
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards!
> > > > Rui Li
> > >
> >
>


-- 
Best, Jingsong Lee

Re: [DISCUSS] Lifecycle of ShuffleMaster and its Relationship with JobMaster and PartitionTracker

[Guowei Ma]

Hi,
Thank Yingjie for initiating this discussion. What I understand that the
document[1] actually mainly discusses two issues:
1. ShuffleMaster should be at the cluster level instead of the job level
2. ShuffleMaster should notify PartitionTracker that some data has been lost

Relatively speaking, I think the second problem is more serious. Because
for external or remote batch shuffling services, after the machine storing
shuffled data goes offline, PartitionTracker needs to be notified in time
to avoid repeated failures of the job. Therefore, it is hoped that when
shuffle data goes offline due to a machine error, ShuffleMaster can notify
the PartitionTracker in time. This requires ShuffleMaster to notify the
PartitionTracker with a handle such as JobShuffleContext.

So how to pass JobShuffleContext to ShuffleMaster? There are two options:
1. After ShuffleMaster is created, pass JobShuffleContext to ShuffleMaster,
such as ShuffleMaster::register(JobShuffleContext)
2. Pass ShuffleServiceFactory when creating ShuffleMaster, such as
ShuffleServiceFatroctry.createcreateShuffleMaster(JobShuffleContext).

Which one to choose is actually related to issue 1. Because if
ShuffleMaster is a cluster level, you should choose option 1, otherwise,
choose option 2. I think ShuffleMaster should be at the cluster level, for
example, because we don't need to maintain a ShuffleMaster for each job in
a SessionCluster; in addition, this ShuffleMaster should also be used by
RM's PartitionTracker in the future. Therefore, I think Option 1 is more
appropriate.

To sum up, we may give priority to solving problem 2, while taking into
account that ShuffleMaster should be a cluster-level component. Therefore,
I think we could ignore the date ShuffleMasterContext at the beginning; at
the same time, JobShuffleContext::getConfiguration/listPartitions should
not be needed.

[1]
https://docs.google.com/document/d/1_cHoapNbx_fJ7ZNraSqw4ZK1hMRiWWJDITuSZrdMDDs/edit

Best,
Guowei


On Fri, Jun 11, 2021 at 4:15 PM Yingjie Cao  wrote:

> Hi devs,
>
> I'd like to start a discussion about "Lifecycle of ShuffleMaster and its
> Relationship with JobMaster and PartitionTracker". (These are things we
> found when moving our external shuffle to the pluggable shuffle service
> framework.)
>
> The mail client may fail to display the right format. If so, please refer
> to this document:
>
> https://docs.google.com/document/d/1_cHoapNbx_fJ7ZNraSqw4ZK1hMRiWWJDITuSZrdMDDs/edit?usp=sharing
> .
> Lifecycle of ShuffleMaster
>
> Currently, the lifecycle of ShuffleMaster seems unclear.  The
> ShuffleServiceFactory is loaded for each JobMaster instance and then
> ShuffleServiceFactory#createShuffleMaster will be called to create a
> ShuffleMaser instance. However, the default NettyShuffleServiceFactory
> always returns the same ShuffleMaser singleton instance for all jobs. Based
> on the current implementation, the lifecycle of ShuffleMaster seems open
> and depends on the shuffle plugin themselves. However, at the TM side,
> the ShuffleEnvironment
> is a part of the TaskManagerServices whose lifecycle is decoupled with jobs
> which is more like a service. It means there is also an inconsistency
> between the TM side and the JM side.
>
> From my understanding, the reason for this is that the pluggable shuffle
> framework is still not completely finished yet, for example, there is a
> follow up umbrella ticket  FLINK-19551
>  for the pluggable
> shuffle service framework and in its subtasks, there is one task (
> FLINK-12731 ) which
> aims
> to load shuffle plugin with the PluginManager. I think this can solve the
> issue mentioned above. After the corresponding factory  loaded by the
> PluginManager, all ShuffleMaster instances can be stored in a map indexed
> by the corresponding factory class name  which can be shared by all jobs.
> After that, the ShuffleMaster becomes a cluster level service which is
> consistent with the ShuffleEnvironment at the TM side.
>
> As a summary, we propose to finish FLINK-12731
>  and make the shuffle
> service a real cluster level service first. Furthermore, we add two
> lifecycle methods to the ShuffleMaster interface, including start and
> close responding
> for initialization (for example, contacting the external system) and
> graceful shutdown (for example, releasing the resources) respectively
> (these methods already exist in the ShuffleEnvironment interface at the TM
> side). What do you think?
> Relationship of ShuffleMaster & JobMaster
>
> Currently, JobMaster holds the reference to the corresponding ShuffleMaster
> and it can register partitions (allocate ShuffleDescriptor from) to
> ShuffleMaster
> by the registerPartitionWithProducer method. To support use cases like
> allocating external resources when a job starts and releasing all allocated
> resources when a job 

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

[Etienne Chauchot]

Hi,

Thanks Xintong for the very good doc ! I added 2 comments to it.

Best,

Etienne

On 02/07/2021 05:57, Xintong Song wrote:

Thanks all for the positive feedback.

I have updated the wiki page [1], and will send an announcement in a
separate thread, to draw more committers' attention.

Moreover, I've opened FLINK-23212 where we can continue with the discussion
around pure documentation changing PRs.

Thank you~

Xintong Song


[1] https://cwiki.apache.org/confluence/display/FLINK/Merging+Pull+Requests

On Wed, Jun 30, 2021 at 5:26 PM Xintong Song  wrote:


I second Tison's opinion.

Similar to how we skip docs_404_check for PRs that do not touch the
documentation, we can skip other stages for PRs that only contain
documentation changes.

In addition to making merging documentation PRs easier, we can also reduce
the workload on CI workers. Especially during the last days of a release
cycle, which is usually the most busy time for the CI workers, and is also
where most documentation efforts take place.

Thank you~

Xintong Song



On Wed, Jun 30, 2021 at 3:56 PM Till Rohrmann 
wrote:


I think you are right Tison that docs are a special case and they only
require flink-docs to pass. What I am wondering is how much of a problem
this will be (assuming that we have a decent build stability). The more
exceptions we add, the harder it will be to properly follow the
guidelines.
Maybe we can observe how many docs PRs get delayed/not merged because of
this and then revisit this discussion if needed.

Cheers,
Till

On Wed, Jun 30, 2021 at 8:30 AM tison  wrote:


Hi,

There are a number of PRs modifying docs only, but we still require all
tests passed on that.

It is a good proposal we avoid merge PR with "unrelated" failure, but

can

we improve the case where the contributor only works for docs?

For example, base on the file change set, run doc tests only.

Best,
tison.


godfrey he  于2021年6月30日周三 下午2:17写道：


+1 for the proposal. Thanks Xintong!

Best,
Godfrey



Rui Li  于2021年6月30日周三 上午11:36写道：


Thanks Xintong. +1 to the proposal.

On Tue, Jun 29, 2021 at 11:05 AM 刘建刚 

wrote:

+1 for the proposal. Since the test time is long and environment

may

vary,

unstable tests are really annoying for developers. The solution is

welcome.

Best
liujiangang

Jingsong Li  于2021年6月29日周二 上午10:31写道：


+1 Thanks Xintong for the update!

Best,
Jingsong

On Mon, Jun 28, 2021 at 6:44 PM Till Rohrmann <

trohrm...@apache.org>

wrote:


+1, thanks for updating the guidelines Xintong!

Cheers,
Till

On Mon, Jun 28, 2021 at 11:49 AM Yangze Guo <

karma...@gmail.com>

wrote:

+1

Thanks Xintong for drafting this doc.

Best,
Yangze Guo

On Mon, Jun 28, 2021 at 5:42 PM JING ZHANG <

beyond1...@gmail.com

wrote:

Thanks Xintong for giving detailed documentation.

The best practice for handling test failure is very

detailed,

it's

a

good

guidelines document with clear action steps.

+1 to Xintong's proposal.

Xintong Song  于2021年6月28日周一

下午4:07写道：

Thanks all for the discussion.

Based on the opinions so far, I've drafted the new

guidelines

[1],

as a

potential replacement of the original wiki page [2].

Hopefully this draft has covered the most opinions

discussed

and

consensus

made in this discussion thread.

Looking forward to your feedback.

Thank you~

Xintong Song


[1]



https://docs.google.com/document/d/1uUbxbgbGErBXtmEjhwVhBWG3i6nhQ0LXs96OlntEYnU/edit?usp=sharing

[2]


https://cwiki.apache.org/confluence/display/FLINK/Merging+Pull+Requests



On Fri, Jun 25, 2021 at 10:40 PM Piotr Nowojski <

pnowoj...@apache.org>

wrote:


Thanks for the clarification Till. +1 for what you

have

written.

Piotrek

pt., 25 cze 2021 o 16:00 Till Rohrmann <

trohrm...@apache.org

napisał(a):

One quick note for clarification. I don't have

anything

against

builds

running on your personal Azure account and this is

not

what I

understood

under "local environment". For me "local

environment"

means

that

someone

runs the test locally on his machine and then says

that

the

tests have passed locally.

I do agree that there might be a conflict of

interests

if a

PR

author

disables tests. Here I would argue that we don't

have

malignant

committers

which means that every committer will probably first

check

the

respective

ticket for how often the test failed. Then I guess

the

next

step

would

be

to discuss on the ticket whether to disable it or

not.

And

finally,

after

reaching a consensus, it will be disabled. If we see

someone

abusing

this

policy, then we can still think about how to guard

against

it.

But,

honestly, I have very rarely seen such a case. I am

also

ok

to

pull in

the

release manager to make the final call if this

resolves

concerns.

Cheers,
Till

On Fri, Jun 25, 2021 at 9:07 AM Piotr Nowojski <

pnowoj...@apache.org>

wrote:


+1 for the general idea, however I have concerns

about

a

couple

of

details.

I would first try to not introduce the exception

for

local

builds.

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Arvid Heise]

Congratulations!

On Wed, Jul 7, 2021 at 12:17 PM godfrey he  wrote:

> Congratulations, Yang!
>
> Best,
> Godfrey
>
> Lijie Wang  于2021年7月7日周三 下午5:59写道：
>
> > Congratulations Yang!
> >
> > Till Rohrmann  于2021年7月7日周三 下午5:29写道：
> >
> > > Congratulations, Yang!
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan 
> > wrote:
> > >
> > > > Congrats!
> > > >
> > > > Regards,
> > > > Roman
> > > >
> > > >
> > > > On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren 
> > wrote:
> > > > >
> > > > > Congratulations Yang!
> > > > >
> > > > > --
> > > > > Best Regards,
> > > > >
> > > > > Qingsheng Ren
> > > > > Email: renqs...@gmail.com
> > > > > On Jul 7, 2021, 2:26 PM +0800, Rui Li ,
> > wrote:
> > > > > > Congratulations Yang ~
> > > > > >
> > > > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> > > > wrote:
> > > > > >
> > > > > > > Congratulations!
> > > > > > >
> > > > > > > Peter Huang  于2021年7月7日周三
> 下午12:54写道：
> > > > > > >
> > > > > > > > Congratulations, Yang.
> > > > > > > >
> > > > > > > > Best Regards
> > > > > > > > Peter Huang
> > > > > > > >
> > > > > > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu <
> dian0511...@gmail.com>
> > > > wrote:
> > > > > > > >
> > > > > > > > > Congratulations, Yang,
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Dian
> > > > > > > > >
> > > > > > > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > > > > > > >
> > > > > > > > > > Congratulations, Yang Wang.
> > > > > > > > > >
> > > > > > > > > > Best
> > > > > > > > > > Jary
> > > > > > > > > >
> > > > > > > > > > Yun Gao  于2021年7月7日周三
> > > 上午10:38写道：
> > > > > > > > > >
> > > > > > > > > > > Congratulations Yang!
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Yun
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > --
> > > > > > > > > > > Sender:Jark Wu
> > > > > > > > > > > Date:2021/07/07 10:20:27
> > > > > > > > > > > Recipient:dev
> > > > > > > > > > > Cc:Yang Wang; <
> > > > wangyang0...@apache.org>
> > > > > > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yang
> > Wang
> > > > > > > > > > >
> > > > > > > > > > > Congratulations Yang Wang!
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Jark
> > > > > > > > > > >
> > > > > > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song <
> > > > tonysong...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi everyone,
> > > > > > > > > > > >
> > > > > > > > > > > > On behalf of the PMC, I'm very happy to announce Yang
> > > Wang
> > > > as a new
> > > > > > > > > Flink
> > > > > > > > > > > > committer.
> > > > > > > > > > > >
> > > > > > > > > > > > Yang has been a very active contributor for more than
> > two
> > > > years,
> > > > > > > > mainly
> > > > > > > > > > > > focusing on Flink's deployment components. He's a
> main
> > > > contributor
> > > > > > > > and
> > > > > > > > > > > > maintainer of Flink's native Kubernetes deployment
> and
> > > > native
> > > > > > > > > Kubernetes
> > > > > > > > > > > > HA. He's also very active on the mailing lists,
> > > > participating in
> > > > > > > > > > > > discussions and helping with user questions.
> > > > > > > > > > > >
> > > > > > > > > > > > Please join me in congratulating Yang Wang for
> > becoming a
> > > > Flink
> > > > > > > > > > > committer!
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you~
> > > > > > > > > > > >
> > > > > > > > > > > > Xintong Song
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Best,
> > > > > > > Benchao Li
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards!
> > > > > > Rui Li
> > > >
> > >
> >
>

Re: [ANNOUNCE] New PMC member: Guowei Ma

[Arvid Heise]

Congratulations!

On Wed, Jul 7, 2021 at 11:30 AM Till Rohrmann  wrote:

> Congratulations, Guowei!
>
> Cheers,
> Till
>
> On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan  wrote:
>
> > Congratulations!
> >
> > Regards,
> > Roman
> >
> > On Wed, Jul 7, 2021 at 8:24 AM Rui Li  wrote:
> > >
> > > Congratulations Guowei!
> > >
> > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> wrote:
> > >
> > > > Congratulations!
> > > >
> > > > Dian Fu  于2021年7月7日周三 下午12:46写道：
> > > >
> > > > > Congratulations, Guowei!
> > > > >
> > > > > Regards,
> > > > > Dian
> > > > >
> > > > > > 2021年7月7日 上午10:37，Yun Gao  写道：
> > > > > >
> > > > > > Congratulations Guowei!
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > > Yun
> > > > > >
> > > > > >
> > > > > >
> --
> > > > > > Sender:JING ZHANG
> > > > > > Date:2021/07/07 10:33:51
> > > > > > Recipient:dev
> > > > > > Theme:Re: [ANNOUNCE] New PMC member: Guowei Ma
> > > > > >
> > > > > > Congratulations,  Guowei Ma!
> > > > > >
> > > > > > Best regards,
> > > > > > JING ZHANG
> > > > > >
> > > > > > Zakelly Lan  于2021年7月7日周三 上午10:30写道：
> > > > > >
> > > > > >> Congratulations, Guowei!
> > > > > >>
> > > > > >> Best,
> > > > > >> Zakelly
> > > > > >>
> > > > > >> On Wed, Jul 7, 2021 at 10:24 AM tison 
> > wrote:
> > > > > >>
> > > > > >>> Congrats! NB.
> > > > > >>>
> > > > > >>> Best,
> > > > > >>> tison.
> > > > > >>>
> > > > > >>>
> > > > > >>> Jark Wu  于2021年7月7日周三 上午10:20写道：
> > > > > >>>
> > > > >  Congratulations Guowei!
> > > > > 
> > > > >  Best,
> > > > >  Jark
> > > > > 
> > > > >  On Wed, 7 Jul 2021 at 09:54, XING JIN <
> jinxing.co...@gmail.com>
> > > > > wrote:
> > > > > 
> > > > > > Congratulations, Guowei~ !
> > > > > >
> > > > > > Best,
> > > > > > Jin
> > > > > >
> > > > > > Xintong Song  于2021年7月7日周三 上午9:37写道：
> > > > > >
> > > > > >> Congratulations, Guowei~!
> > > > > >>
> > > > > >> Thank you~
> > > > > >>
> > > > > >> Xintong Song
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Jul 7, 2021 at 9:31 AM Qingsheng Ren <
> > renqs...@gmail.com>
> > > > >  wrote:
> > > > > >>
> > > > > >>> Congratulations Guowei!
> > > > > >>>
> > > > > >>> --
> > > > > >>> Best Regards,
> > > > > >>>
> > > > > >>> Qingsheng Ren
> > > > > >>> Email: renqs...@gmail.com
> > > > > >>> 2021年7月7日 +0800 09:30 Leonard Xu ，写道：
> > > > >  Congratulations! Guowei Ma
> > > > > 
> > > > >  Best,
> > > > >  Leonard
> > > > > 
> > > > > > ÔÚ 2021Äê7ÔÂ6ÈÕ£¬21:56£¬Kurt Young 
> > дµÀ£º
> > > > > >
> > > > > > Hi all!
> > > > > >
> > > > > > I'm very happy to announce that Guowei Ma has joined the
> > > > > >> Flink
> > > > >  PMC!
> > > > > >
> > > > > > Congratulations and welcome Guowei!
> > > > > >
> > > > > > Best,
> > > > > > Kurt
> > > > > 
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > 
> > > > > >>>
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Best,
> > > > Benchao Li
> > > >
> > >
> > >
> > > --
> > > Best regards!
> > > Rui Li
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[godfrey he]

Congratulations, Yang!

Best,
Godfrey

Lijie Wang  于2021年7月7日周三 下午5:59写道：

> Congratulations Yang!
>
> Till Rohrmann  于2021年7月7日周三 下午5:29写道：
>
> > Congratulations, Yang!
> >
> > Cheers,
> > Till
> >
> > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan 
> wrote:
> >
> > > Congrats!
> > >
> > > Regards,
> > > Roman
> > >
> > >
> > > On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren 
> wrote:
> > > >
> > > > Congratulations Yang!
> > > >
> > > > --
> > > > Best Regards,
> > > >
> > > > Qingsheng Ren
> > > > Email: renqs...@gmail.com
> > > > On Jul 7, 2021, 2:26 PM +0800, Rui Li ,
> wrote:
> > > > > Congratulations Yang ~
> > > > >
> > > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> > > wrote:
> > > > >
> > > > > > Congratulations!
> > > > > >
> > > > > > Peter Huang  于2021年7月7日周三 下午12:54写道：
> > > > > >
> > > > > > > Congratulations, Yang.
> > > > > > >
> > > > > > > Best Regards
> > > > > > > Peter Huang
> > > > > > >
> > > > > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu 
> > > wrote:
> > > > > > >
> > > > > > > > Congratulations, Yang,
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Dian
> > > > > > > >
> > > > > > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > > > > > >
> > > > > > > > > Congratulations, Yang Wang.
> > > > > > > > >
> > > > > > > > > Best
> > > > > > > > > Jary
> > > > > > > > >
> > > > > > > > > Yun Gao  于2021年7月7日周三
> > 上午10:38写道：
> > > > > > > > >
> > > > > > > > > > Congratulations Yang!
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Yun
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > --
> > > > > > > > > > Sender:Jark Wu
> > > > > > > > > > Date:2021/07/07 10:20:27
> > > > > > > > > > Recipient:dev
> > > > > > > > > > Cc:Yang Wang; <
> > > wangyang0...@apache.org>
> > > > > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yang
> Wang
> > > > > > > > > >
> > > > > > > > > > Congratulations Yang Wang!
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Jark
> > > > > > > > > >
> > > > > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song <
> > > tonysong...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi everyone,
> > > > > > > > > > >
> > > > > > > > > > > On behalf of the PMC, I'm very happy to announce Yang
> > Wang
> > > as a new
> > > > > > > > Flink
> > > > > > > > > > > committer.
> > > > > > > > > > >
> > > > > > > > > > > Yang has been a very active contributor for more than
> two
> > > years,
> > > > > > > mainly
> > > > > > > > > > > focusing on Flink's deployment components. He's a main
> > > contributor
> > > > > > > and
> > > > > > > > > > > maintainer of Flink's native Kubernetes deployment and
> > > native
> > > > > > > > Kubernetes
> > > > > > > > > > > HA. He's also very active on the mailing lists,
> > > participating in
> > > > > > > > > > > discussions and helping with user questions.
> > > > > > > > > > >
> > > > > > > > > > > Please join me in congratulating Yang Wang for
> becoming a
> > > Flink
> > > > > > > > > > committer!
> > > > > > > > > > >
> > > > > > > > > > > Thank you~
> > > > > > > > > > >
> > > > > > > > > > > Xintong Song
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best,
> > > > > > Benchao Li
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards!
> > > > > Rui Li
> > >
> >
>

Re: [ANNOUNCE] New PMC member: Guowei Ma

[godfrey he]

Congratulations, Guowei!

Best,
Godfrey

Lijie Wang  于2021年7月7日周三 下午5:57写道：

> Congratulations, Guowei Ma!
>
> Till Rohrmann  于2021年7月7日周三 下午5:30写道：
>
> > Congratulations, Guowei!
> >
> > Cheers,
> > Till
> >
> > On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan 
> wrote:
> >
> > > Congratulations!
> > >
> > > Regards,
> > > Roman
> > >
> > > On Wed, Jul 7, 2021 at 8:24 AM Rui Li  wrote:
> > > >
> > > > Congratulations Guowei!
> > > >
> > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> > wrote:
> > > >
> > > > > Congratulations!
> > > > >
> > > > > Dian Fu  于2021年7月7日周三 下午12:46写道：
> > > > >
> > > > > > Congratulations, Guowei!
> > > > > >
> > > > > > Regards,
> > > > > > Dian
> > > > > >
> > > > > > > 2021年7月7日 上午10:37，Yun Gao  写道：
> > > > > > >
> > > > > > > Congratulations Guowei!
> > > > > > >
> > > > > > >
> > > > > > > Best,
> > > > > > > Yun
> > > > > > >
> > > > > > >
> > > > > > >
> > --
> > > > > > > Sender:JING ZHANG
> > > > > > > Date:2021/07/07 10:33:51
> > > > > > > Recipient:dev
> > > > > > > Theme:Re: [ANNOUNCE] New PMC member: Guowei Ma
> > > > > > >
> > > > > > > Congratulations,  Guowei Ma!
> > > > > > >
> > > > > > > Best regards,
> > > > > > > JING ZHANG
> > > > > > >
> > > > > > > Zakelly Lan  于2021年7月7日周三 上午10:30写道：
> > > > > > >
> > > > > > >> Congratulations, Guowei!
> > > > > > >>
> > > > > > >> Best,
> > > > > > >> Zakelly
> > > > > > >>
> > > > > > >> On Wed, Jul 7, 2021 at 10:24 AM tison 
> > > wrote:
> > > > > > >>
> > > > > > >>> Congrats! NB.
> > > > > > >>>
> > > > > > >>> Best,
> > > > > > >>> tison.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Jark Wu  于2021年7月7日周三 上午10:20写道：
> > > > > > >>>
> > > > > >  Congratulations Guowei!
> > > > > > 
> > > > > >  Best,
> > > > > >  Jark
> > > > > > 
> > > > > >  On Wed, 7 Jul 2021 at 09:54, XING JIN <
> > jinxing.co...@gmail.com>
> > > > > > wrote:
> > > > > > 
> > > > > > > Congratulations, Guowei~ !
> > > > > > >
> > > > > > > Best,
> > > > > > > Jin
> > > > > > >
> > > > > > > Xintong Song  于2021年7月7日周三
> 上午9:37写道：
> > > > > > >
> > > > > > >> Congratulations, Guowei~!
> > > > > > >>
> > > > > > >> Thank you~
> > > > > > >>
> > > > > > >> Xintong Song
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, Jul 7, 2021 at 9:31 AM Qingsheng Ren <
> > > renqs...@gmail.com>
> > > > > >  wrote:
> > > > > > >>
> > > > > > >>> Congratulations Guowei!
> > > > > > >>>
> > > > > > >>> --
> > > > > > >>> Best Regards,
> > > > > > >>>
> > > > > > >>> Qingsheng Ren
> > > > > > >>> Email: renqs...@gmail.com
> > > > > > >>> 2021年7月7日 +0800 09:30 Leonard Xu ，写道：
> > > > > >  Congratulations! Guowei Ma
> > > > > > 
> > > > > >  Best,
> > > > > >  Leonard
> > > > > > 
> > > > > > > ÔÚ 2021Äê7ÔÂ6ÈÕ£¬21:56£¬Kurt Young 
> > > дµÀ£º
> > > > > > >
> > > > > > > Hi all!
> > > > > > >
> > > > > > > I'm very happy to announce that Guowei Ma has joined
> the
> > > > > > >> Flink
> > > > > >  PMC!
> > > > > > >
> > > > > > > Congratulations and welcome Guowei!
> > > > > > >
> > > > > > > Best,
> > > > > > > Kurt
> > > > > > 
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > 
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best,
> > > > > Benchao Li
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards!
> > > > Rui Li
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Lijie Wang]

Congratulations Yang!

Till Rohrmann  于2021年7月7日周三 下午5:29写道：

> Congratulations, Yang!
>
> Cheers,
> Till
>
> On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan  wrote:
>
> > Congrats!
> >
> > Regards,
> > Roman
> >
> >
> > On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren  wrote:
> > >
> > > Congratulations Yang!
> > >
> > > --
> > > Best Regards,
> > >
> > > Qingsheng Ren
> > > Email: renqs...@gmail.com
> > > On Jul 7, 2021, 2:26 PM +0800, Rui Li , wrote:
> > > > Congratulations Yang ~
> > > >
> > > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> > wrote:
> > > >
> > > > > Congratulations!
> > > > >
> > > > > Peter Huang  于2021年7月7日周三 下午12:54写道：
> > > > >
> > > > > > Congratulations, Yang.
> > > > > >
> > > > > > Best Regards
> > > > > > Peter Huang
> > > > > >
> > > > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu 
> > wrote:
> > > > > >
> > > > > > > Congratulations, Yang,
> > > > > > >
> > > > > > > Regards,
> > > > > > > Dian
> > > > > > >
> > > > > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > > > > >
> > > > > > > > Congratulations, Yang Wang.
> > > > > > > >
> > > > > > > > Best
> > > > > > > > Jary
> > > > > > > >
> > > > > > > > Yun Gao  于2021年7月7日周三
> 上午10:38写道：
> > > > > > > >
> > > > > > > > > Congratulations Yang!
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Yun
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > --
> > > > > > > > > Sender:Jark Wu
> > > > > > > > > Date:2021/07/07 10:20:27
> > > > > > > > > Recipient:dev
> > > > > > > > > Cc:Yang Wang; <
> > wangyang0...@apache.org>
> > > > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang
> > > > > > > > >
> > > > > > > > > Congratulations Yang Wang!
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Jark
> > > > > > > > >
> > > > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song <
> > tonysong...@gmail.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi everyone,
> > > > > > > > > >
> > > > > > > > > > On behalf of the PMC, I'm very happy to announce Yang
> Wang
> > as a new
> > > > > > > Flink
> > > > > > > > > > committer.
> > > > > > > > > >
> > > > > > > > > > Yang has been a very active contributor for more than two
> > years,
> > > > > > mainly
> > > > > > > > > > focusing on Flink's deployment components. He's a main
> > contributor
> > > > > > and
> > > > > > > > > > maintainer of Flink's native Kubernetes deployment and
> > native
> > > > > > > Kubernetes
> > > > > > > > > > HA. He's also very active on the mailing lists,
> > participating in
> > > > > > > > > > discussions and helping with user questions.
> > > > > > > > > >
> > > > > > > > > > Please join me in congratulating Yang Wang for becoming a
> > Flink
> > > > > > > > > committer!
> > > > > > > > > >
> > > > > > > > > > Thank you~
> > > > > > > > > >
> > > > > > > > > > Xintong Song
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best,
> > > > > Benchao Li
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards!
> > > > Rui Li
> >
>

Re: [ANNOUNCE] New PMC member: Guowei Ma

[Lijie Wang]

Congratulations, Guowei Ma!

Till Rohrmann  于2021年7月7日周三 下午5:30写道：

> Congratulations, Guowei!
>
> Cheers,
> Till
>
> On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan  wrote:
>
> > Congratulations!
> >
> > Regards,
> > Roman
> >
> > On Wed, Jul 7, 2021 at 8:24 AM Rui Li  wrote:
> > >
> > > Congratulations Guowei!
> > >
> > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> wrote:
> > >
> > > > Congratulations!
> > > >
> > > > Dian Fu  于2021年7月7日周三 下午12:46写道：
> > > >
> > > > > Congratulations, Guowei!
> > > > >
> > > > > Regards,
> > > > > Dian
> > > > >
> > > > > > 2021年7月7日 上午10:37，Yun Gao  写道：
> > > > > >
> > > > > > Congratulations Guowei!
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > > Yun
> > > > > >
> > > > > >
> > > > > >
> --
> > > > > > Sender:JING ZHANG
> > > > > > Date:2021/07/07 10:33:51
> > > > > > Recipient:dev
> > > > > > Theme:Re: [ANNOUNCE] New PMC member: Guowei Ma
> > > > > >
> > > > > > Congratulations,  Guowei Ma!
> > > > > >
> > > > > > Best regards,
> > > > > > JING ZHANG
> > > > > >
> > > > > > Zakelly Lan  于2021年7月7日周三 上午10:30写道：
> > > > > >
> > > > > >> Congratulations, Guowei!
> > > > > >>
> > > > > >> Best,
> > > > > >> Zakelly
> > > > > >>
> > > > > >> On Wed, Jul 7, 2021 at 10:24 AM tison 
> > wrote:
> > > > > >>
> > > > > >>> Congrats! NB.
> > > > > >>>
> > > > > >>> Best,
> > > > > >>> tison.
> > > > > >>>
> > > > > >>>
> > > > > >>> Jark Wu  于2021年7月7日周三 上午10:20写道：
> > > > > >>>
> > > > >  Congratulations Guowei!
> > > > > 
> > > > >  Best,
> > > > >  Jark
> > > > > 
> > > > >  On Wed, 7 Jul 2021 at 09:54, XING JIN <
> jinxing.co...@gmail.com>
> > > > > wrote:
> > > > > 
> > > > > > Congratulations, Guowei~ !
> > > > > >
> > > > > > Best,
> > > > > > Jin
> > > > > >
> > > > > > Xintong Song  于2021年7月7日周三 上午9:37写道：
> > > > > >
> > > > > >> Congratulations, Guowei~!
> > > > > >>
> > > > > >> Thank you~
> > > > > >>
> > > > > >> Xintong Song
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Jul 7, 2021 at 9:31 AM Qingsheng Ren <
> > renqs...@gmail.com>
> > > > >  wrote:
> > > > > >>
> > > > > >>> Congratulations Guowei!
> > > > > >>>
> > > > > >>> --
> > > > > >>> Best Regards,
> > > > > >>>
> > > > > >>> Qingsheng Ren
> > > > > >>> Email: renqs...@gmail.com
> > > > > >>> 2021年7月7日 +0800 09:30 Leonard Xu ，写道：
> > > > >  Congratulations! Guowei Ma
> > > > > 
> > > > >  Best,
> > > > >  Leonard
> > > > > 
> > > > > > ÔÚ 2021Äê7ÔÂ6ÈÕ£¬21:56£¬Kurt Young 
> > дµÀ£º
> > > > > >
> > > > > > Hi all!
> > > > > >
> > > > > > I'm very happy to announce that Guowei Ma has joined the
> > > > > >> Flink
> > > > >  PMC!
> > > > > >
> > > > > > Congratulations and welcome Guowei!
> > > > > >
> > > > > > Best,
> > > > > > Kurt
> > > > > 
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > 
> > > > > >>>
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Best,
> > > > Benchao Li
> > > >
> > >
> > >
> > > --
> > > Best regards!
> > > Rui Li
> >
>

[jira] [Created] (FLINK-23300) Job fails very slow because of no notifyAllocationFailure for DeclarativeSlotManager

[Liu (Jira)]

Liu created FLINK-23300:
---

 Summary: Job fails very slow because of no notifyAllocationFailure 
for DeclarativeSlotManager
 Key: FLINK-23300
 URL: https://issues.apache.org/jira/browse/FLINK-23300
 Project: Flink
  Issue Type: Improvement
  Components: Runtime / Task
Affects Versions: 1.13.1
Reporter: Liu


When container is killed, flink on yarn can detect the problem very quickly. 
But when using default DeclarativeSlotManager, notifyAllocationFailure is not 
called and the task is not failed until heartbeat is timeout. So the failover 
will be very slow. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New PMC member: Guowei Ma

[Till Rohrmann]

Congratulations, Guowei!

Cheers,
Till

On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan  wrote:

> Congratulations!
>
> Regards,
> Roman
>
> On Wed, Jul 7, 2021 at 8:24 AM Rui Li  wrote:
> >
> > Congratulations Guowei!
> >
> > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li  wrote:
> >
> > > Congratulations!
> > >
> > > Dian Fu  于2021年7月7日周三 下午12:46写道：
> > >
> > > > Congratulations, Guowei!
> > > >
> > > > Regards,
> > > > Dian
> > > >
> > > > > 2021年7月7日 上午10:37，Yun Gao  写道：
> > > > >
> > > > > Congratulations Guowei!
> > > > >
> > > > >
> > > > > Best,
> > > > > Yun
> > > > >
> > > > >
> > > > > --
> > > > > Sender:JING ZHANG
> > > > > Date:2021/07/07 10:33:51
> > > > > Recipient:dev
> > > > > Theme:Re: [ANNOUNCE] New PMC member: Guowei Ma
> > > > >
> > > > > Congratulations,  Guowei Ma!
> > > > >
> > > > > Best regards,
> > > > > JING ZHANG
> > > > >
> > > > > Zakelly Lan  于2021年7月7日周三 上午10:30写道：
> > > > >
> > > > >> Congratulations, Guowei!
> > > > >>
> > > > >> Best,
> > > > >> Zakelly
> > > > >>
> > > > >> On Wed, Jul 7, 2021 at 10:24 AM tison 
> wrote:
> > > > >>
> > > > >>> Congrats! NB.
> > > > >>>
> > > > >>> Best,
> > > > >>> tison.
> > > > >>>
> > > > >>>
> > > > >>> Jark Wu  于2021年7月7日周三 上午10:20写道：
> > > > >>>
> > > >  Congratulations Guowei!
> > > > 
> > > >  Best,
> > > >  Jark
> > > > 
> > > >  On Wed, 7 Jul 2021 at 09:54, XING JIN 
> > > > wrote:
> > > > 
> > > > > Congratulations, Guowei~ !
> > > > >
> > > > > Best,
> > > > > Jin
> > > > >
> > > > > Xintong Song  于2021年7月7日周三 上午9:37写道：
> > > > >
> > > > >> Congratulations, Guowei~!
> > > > >>
> > > > >> Thank you~
> > > > >>
> > > > >> Xintong Song
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Jul 7, 2021 at 9:31 AM Qingsheng Ren <
> renqs...@gmail.com>
> > > >  wrote:
> > > > >>
> > > > >>> Congratulations Guowei!
> > > > >>>
> > > > >>> --
> > > > >>> Best Regards,
> > > > >>>
> > > > >>> Qingsheng Ren
> > > > >>> Email: renqs...@gmail.com
> > > > >>> 2021年7月7日 +0800 09:30 Leonard Xu ，写道：
> > > >  Congratulations! Guowei Ma
> > > > 
> > > >  Best,
> > > >  Leonard
> > > > 
> > > > > ÔÚ 2021Äê7ÔÂ6ÈÕ£¬21:56£¬Kurt Young 
> дµÀ£º
> > > > >
> > > > > Hi all!
> > > > >
> > > > > I'm very happy to announce that Guowei Ma has joined the
> > > > >> Flink
> > > >  PMC!
> > > > >
> > > > > Congratulations and welcome Guowei!
> > > > >
> > > > > Best,
> > > > > Kurt
> > > > 
> > > > >>>
> > > > >>
> > > > >
> > > > 
> > > > >>>
> > > > >>
> > > > >
> > > >
> > > >
> > >
> > > --
> > >
> > > Best,
> > > Benchao Li
> > >
> >
> >
> > --
> > Best regards!
> > Rui Li
>

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Till Rohrmann]

Congratulations, Yang!

Cheers,
Till

On Wed, Jul 7, 2021 at 9:41 AM Roman Khachatryan  wrote:

> Congrats!
>
> Regards,
> Roman
>
>
> On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren  wrote:
> >
> > Congratulations Yang!
> >
> > --
> > Best Regards,
> >
> > Qingsheng Ren
> > Email: renqs...@gmail.com
> > On Jul 7, 2021, 2:26 PM +0800, Rui Li , wrote:
> > > Congratulations Yang ~
> > >
> > > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li 
> wrote:
> > >
> > > > Congratulations!
> > > >
> > > > Peter Huang  于2021年7月7日周三 下午12:54写道：
> > > >
> > > > > Congratulations, Yang.
> > > > >
> > > > > Best Regards
> > > > > Peter Huang
> > > > >
> > > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu 
> wrote:
> > > > >
> > > > > > Congratulations, Yang,
> > > > > >
> > > > > > Regards,
> > > > > > Dian
> > > > > >
> > > > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > > > >
> > > > > > > Congratulations, Yang Wang.
> > > > > > >
> > > > > > > Best
> > > > > > > Jary
> > > > > > >
> > > > > > > Yun Gao  于2021年7月7日周三 上午10:38写道：
> > > > > > >
> > > > > > > > Congratulations Yang!
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Yun
> > > > > > > >
> > > > > > > >
> > > > > > > >
> --
> > > > > > > > Sender:Jark Wu
> > > > > > > > Date:2021/07/07 10:20:27
> > > > > > > > Recipient:dev
> > > > > > > > Cc:Yang Wang; <
> wangyang0...@apache.org>
> > > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang
> > > > > > > >
> > > > > > > > Congratulations Yang Wang!
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Jark
> > > > > > > >
> > > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song <
> tonysong...@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi everyone,
> > > > > > > > >
> > > > > > > > > On behalf of the PMC, I'm very happy to announce Yang Wang
> as a new
> > > > > > Flink
> > > > > > > > > committer.
> > > > > > > > >
> > > > > > > > > Yang has been a very active contributor for more than two
> years,
> > > > > mainly
> > > > > > > > > focusing on Flink's deployment components. He's a main
> contributor
> > > > > and
> > > > > > > > > maintainer of Flink's native Kubernetes deployment and
> native
> > > > > > Kubernetes
> > > > > > > > > HA. He's also very active on the mailing lists,
> participating in
> > > > > > > > > discussions and helping with user questions.
> > > > > > > > >
> > > > > > > > > Please join me in congratulating Yang Wang for becoming a
> Flink
> > > > > > > > committer!
> > > > > > > > >
> > > > > > > > > Thank you~
> > > > > > > > >
> > > > > > > > > Xintong Song
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best,
> > > > Benchao Li
> > > >
> > >
> > >
> > > --
> > > Best regards!
> > > Rui Li
>

Re: [VOTE] Release 1.12.5, release candidate #1

[Till Rohrmann]

Hi folks, are we sure that FLINK-23233 [1] does not affect the 1.12 release
branch. I think this problem was introduced with FLINK-21996 [2] which is
part of release-1.12. Hence, we might either fix this problem and cancel
this RC or we need to create a fast 1.12.6 afterwards.

[1] https://issues.apache.org/jira/browse/FLINK-23233
[2] https://issues.apache.org/jira/browse/FLINK-21996

Cheers,
Till

On Wed, Jul 7, 2021 at 7:43 AM JING ZHANG  wrote:

> +1 (non-binding)
>
> 1. built from source code flink-1.12.5-src.tgz
> <
> https://dist.apache.org/repos/dist/dev/flink/flink-1.12.5-rc1/flink-1.12.5-src.tgz
> >
> succeeded
> 2. Started a local Flink cluster, ran the WordCount example, WebUI looks
> good,  no suspicious output/log
> 3. started cluster and run some e2e sql queries using SQL Client, query
> result is expected.
> 4. Repeat Step 2 and 3 with flink-1.12.5-bin-scala_2.11.tgz
> <
> https://dist.apache.org/repos/dist/dev/flink/flink-1.12.5-rc1/flink-1.12.5-bin-scala_2.11.tgz
> >
>
> Best,
> JING ZHANG
>
> Yang Wang  于2021年7月7日周三 上午11:36写道：
>
> > +1 (non-binding)
> >
> > - verified checksums & signatures
> > - start a session cluster and verify HA data cleanup once job reached to
> > globally terminal state, FLINK-20695
> > - start a local standalone cluster, check the webUI good and JM/TM
> > without suspicious logs
> >
> >
> > Best,
> > Yang
> >
> > Leonard Xu  于2021年7月7日周三 上午10:52写道：
> >
> > > +1 (non-binding)
> > >
> > > - verified signatures and hashsums
> > > - built from source code with scala 2.11 succeeded
> > > - checked all denpendency artifacts are 1.12.5
> > > - started a cluster, ran a wordcount job, the result is expected
> > > - started SQL Client, ran a simple query, the result is expected
> > > - reviewed the web PR, left one minor name comment
> > >
> > > Best,
> > > Leonard
> > >
> > > > 在 2021年7月6日，10:02，Xintong Song  写道：
> > > >
> > > > +1 (binding)
> > > >
> > > > - verified checksums & signatures
> > > > - built from sources
> > > > - run example jobs with standalone and native k8s deployments
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > >
> > > > On Mon, Jul 5, 2021 at 11:18 AM Jingsong Li 
> > > wrote:
> > > >
> > > >> Hi everyone,
> > > >>
> > > >> Please review and vote on the release candidate #1 for the version
> > > 1.12.5,
> > > >> as follows:
> > > >> [ ] +1, Approve the release
> > > >> [ ] -1, Do not approve the release (please provide specific
> comments)
> > > >>
> > > >> The complete staging area is available for your review, which
> > includes:
> > > >> * JIRA release notes [1],
> > > >> * the official Apache source release and binary convenience releases
> > to
> > > be
> > > >> deployed to dist.apache.org [2], which are signed with the key with
> > > >> fingerprint FBB83C0A4FFB9CA8 [3],
> > > >> * all artifacts to be deployed to the Maven Central Repository [4],
> > > >> * source code tag "release-1.12.5-rc1" [5],
> > > >> * website pull request listing the new release and adding
> announcement
> > > blog
> > > >> post [6].
> > > >>
> > > >> The vote will be open for at least 72 hours. It is adopted by
> majority
> > > >> approval, with at least 3 PMC affirmative votes.
> > > >>
> > > >> Best,
> > > >> Jingsong Lee
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12350166
> > > >> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.12.5-rc1/
> > > >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > >> [4]
> > > https://repository.apache.org/content/repositories/orgapacheflink-1430
> > > >> [5] https://github.com/apache/flink/releases/tag/release-1.12.5-rc1
> > > >> [6] https://github.com/apache/flink-web/pull/455
> > > >>
> > >
> > >
> >
>

[jira] [Created] (FLINK-23299) Bump Zookeeper 3.5 to 3.5.9

[Chesnay Schepler (Jira)]

Chesnay Schepler created FLINK-23299:


 Summary: Bump Zookeeper 3.5 to 3.5.9
 Key: FLINK-23299
 URL: https://issues.apache.org/jira/browse/FLINK-23299
 Project: Flink
  Issue Type: Improvement
  Components: BuildSystem / Shaded
Reporter: Chesnay Schepler
Assignee: Chesnay Schepler
 Fix For: 1.14.0, shaded-1.14






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release 1.13.2, release candidate #1

[Till Rohrmann]

I think FLINK-23233 sounds like a very serious problem to me. We either
continue releasing 1.13.2 and do an immediate 1.13.3 afterwards or we
cancel this RC and fix the problem and then resume the 1.13.2 release.

Cheers,
Till

On Wed, Jul 7, 2021 at 5:49 AM JING ZHANG  wrote:

> Thanks Xintong for bringing the alarm, +1 on Xintong's proposal to hold the
> vote until deciding what to do with FLINK-23233.
>
> BTW, I made a typo in the previous voting. I mean +1 (non-binding) instead
> of +1 (binding). I'm very sorry for the confusion.
>
> +1 (non-binding).
>
> 1. Reviewed website pull request
> 2. Built from source code flink-1.13.2-src.tgz
> <
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-src.tgz
> >
>  succeeded
> 3. Started a local Flink cluster, ran the WordCount example, WebUI looks
> good,  no suspicious output/log
> 4. Started cluster and run some e2e sql queries using SQL Client, query
> result is expected.
> 5. Repeat Step 3 and 4 with flink-1.13.2-bin-scala_2.11.tgz
> <
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-bin-scala_2.11.tgz
> >
>
> Best regards,
> JING ZHANG
>
> Xintong Song  于2021年7月7日周三 上午11:09写道：
>
> > Hi everyone,
> >
> > We find a bug that may cause data loss in a rare condition, which IMHO
> is a
> > release blocker. Please see FLINK-23233 [1] for details.
> >
> > Given that this bug is not newly introduced but already exists in 1.13.0
> &
> > 1.13.1 releases, I would not replace my +1 with a -1 immediately.
> Instead,
> > I would like to ask the release manager for holding this vote until
> > deciding what to do with FLINK-23233.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-23233
> >
> > On Wed, Jul 7, 2021 at 12:09 AM JING ZHANG  wrote:
> >
> > > +1 (binding)
> > >
> > > 1. Reviewed website pull request
> > > 2. Built from source code flink-1.13.2-src.tgz
> > > <
> > >
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-src.tgz
> > > >
> > > succeeded
> > > 3. Started a local Flink cluster, ran the WordCount example, WebUI
> looks
> > > good,  no suspicious output/log
> > > 4. Started cluster and run some e2e sql queries using SQL Client, query
> > > result is expected.
> > > 5. Repeat Step 3 and 4 with flink-1.13.2-bin-scala_2.11.tgz
> > > <
> > >
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-1.13.2-rc1/flink-1.13.2-bin-scala_2.11.tgz
> > > >
> > >
> > > Best regards,
> > > JING ZHANG
> > >
> > >
> > > Zakelly Lan  于2021年7月5日周一 下午6:06写道：
> > >
> > > > +1 (non-binding)
> > > >
> > > > - built from sources
> > > > - run streaming job of wordcount
> > > > - web-ui looks good
> > > > - checkpoint and restore looks good
> > > >
> > > > Best,
> > > > Zakelly
> > > >
> > > > On Mon, Jul 5, 2021 at 2:40 PM Jingsong Li 
> > > wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > - Verified checksums and signatures
> > > > > - Built from sources
> > > > > - run table example jobs
> > > > > - web-ui looks good
> > > > > - sql-client looks good
> > > > >
> > > > > I think we should update the unresolved JIRAs in [1] to 1.13.3.
> > > > >
> > > > > And we should check resolved JIRAs in [2], commits of some are not
> in
> > > the
> > > > > 1.13.2. We should exclude them. For example FLINK-23196 FLINK-23166
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.13.2%20AND%20status%20not%20in%20(Closed%2C%20Resolved)%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12350218==12315522
> > > > >
> > > > > Best,
> > > > > Jingsong
> > > > >
> > > > > On Mon, Jul 5, 2021 at 2:27 PM Xingbo Huang 
> > > wrote:
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > - Verified checksums and signatures
> > > > > > - Built from sources
> > > > > > - Verified Python wheel package contents
> > > > > > - Pip install Python wheel package in Mac
> > > > > > - Run Python UDF job in Python shell
> > > > > >
> > > > > > Best,
> > > > > > Xingbo
> > > > > >
> > > > > > Yangze Guo  于2021年7月5日周一 上午11:17写道：
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > - built from sources
> > > > > > > - run example jobs with standalone and yarn.
> > > > > > > - check TaskManager's rest API from the JM master and its
> > standby,
> > > > > > > everything looks good
> > > > > > >
> > > > > > > Best,
> > > > > > > Yangze Guo
> > > > > > >
> > > > > > > On Mon, Jul 5, 2021 at 10:10 AM Xintong Song <
> > > tonysong...@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > - verified checksums & signatures
> > > > > > > > - built from sources
> > > > > > > > - run example jobs with standalone and native k8s (with
> custom
> > > > image)
> > > > > > > > deployments
> > > > 

[jira] [Created] (FLINK-23298) [datagen] Normalize parameter names in RandomGeneratorVisitor and SequenceGeneratorVisitor

[fhan (Jira)]

fhan created FLINK-23298:


 Summary: [datagen] Normalize parameter names in 
RandomGeneratorVisitor and SequenceGeneratorVisitor
 Key: FLINK-23298
 URL: https://issues.apache.org/jira/browse/FLINK-23298
 Project: Flink
  Issue Type: Improvement
  Components: Table SQL / API
Reporter: fhan


This PR normalized parameter names in RandomGeneratorVisitor and 
SequenceGeneratorVisitor.
related methods:
[!https://user-images.githubusercontent.com/5745228/118935994-b47c6080-b97e-11eb-86ef-43c191f602fd.jpg|width=416,height=287!|https://user-images.githubusercontent.com/5745228/118935994-b47c6080-b97e-11eb-86ef-43c191f602fd.jpg]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23297) Upgrade Protobuf to 3.17.3

[Till Rohrmann (Jira)]

Till Rohrmann created FLINK-23297:
-

 Summary: Upgrade Protobuf to 3.17.3
 Key: FLINK-23297
 URL: https://issues.apache.org/jira/browse/FLINK-23297
 Project: Flink
  Issue Type: Sub-task
  Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Affects Versions: 1.12.4, 1.13.1, 1.14.0
Reporter: Till Rohrmann


In order to support compilation with ARM (e.g. Apple M1 chip), we need to bump 
our Protobuf dependency to version 3.17.3.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[DISCUSS] Cleaning up outdated test instability JIRA tickets

[Xintong Song]

Hi devs,

I'd like to bring up a discussion on cleaning up JIRA tickets of outdated
test instabilities. The topic was brought up in the recent 1.14 release
bi-weekly, where we have not come to a conclusion and decided to collect
more opinions from the wider community.

We currently have 184 open test instability issues [1], many of which never
happen again since reported or have not happened for a long time. Cleaning
such outdated issues would be helpful for understanding the project's build
stability with less distractions.

Two questions need to be answered.

1. How long does an instability not observed for do we consider it as
*outdated*? Maybe 3 or 6 months?

FYI, I did some statistics around the last reported date of the
instabilities.
- before 2021: 18
- Jan: 6
- Feb: 10
- Mar: 24
- Apr: 28
- May: 29
- Jun: 46
- Jul: 23

2. What should we do with outdated tickets? The easiest way might be simply
to close such issues. The closed issues can be reopened if reappeared
later. Alternatively, we can leave the issue open, potentially with minor
priority, until someone looks into it and makes a technical judgement that
it can be closed.

Please let me know what do you think.

Thank you~

Xintong Song


[1] https://issues.apache.org/jira/issues/?filter=12343317

[jira] [Created] (FLINK-23296) Add RequestReplyClientFactory as a pluggable extension

[Tzu-Li (Gordon) Tai (Jira)]

Tzu-Li (Gordon) Tai created FLINK-23296:
---

 Summary: Add RequestReplyClientFactory as a pluggable extension
 Key: FLINK-23296
 URL: https://issues.apache.org/jira/browse/FLINK-23296
 Project: Flink
  Issue Type: Sub-task
  Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
 Fix For: statefun-3.1.0


Currently, we ship and always use by default a {{RequestReplyClient}} 
implementation based on OkHttp. We'd like to allow users to use their own 
implementations of the {{RequestReplyClient}} for remote invocations. This is 
strictly for extending the means of transport, and should not leave room to 
touch the invocation protocol.

The way this would look like in the module YAML files for remote modules would 
be:

{code}
module:
spec:
endpoints:
- endpoint:
   meta:
   kind: http
   spec:
   functions: com.foo.bar/*
   urlPathTemplate: 
http://bar.foo.com:8080/functions/{function.name}
   transport:
   extension: com.foo.bar/some.custom.client
   prop1: foobar
   prop2:
   - k: v
   - k2: v2
   
{code}

The important part is the transport section. If not specified, then the default 
OkHttp implementation will be used. Otherwise, if specified, an extension with 
the specified typename must be bound and exist in the application, and that 
extension needs to be a {{RequestReplyClientFactory}}:

{code}
interface RequestReplyClientFactory {
RequestReplyClient create(JSONNode properties, URI endpointUrl);
}
{code}

The provided JSON node properties will be as is provided in the {{transport}} 
block of the module YAML endpoint specification.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23294) Default NOTICE files should use artifact id instead of module name as their title.

[Caizhi Weng (Jira)]

Caizhi Weng created FLINK-23294:
---

 Summary: Default NOTICE files should use artifact id instead of 
module name as their title.
 Key: FLINK-23294
 URL: https://issues.apache.org/jira/browse/FLINK-23294
 Project: Flink
  Issue Type: Bug
Affects Versions: 1.14.0
Reporter: Caizhi Weng


Currently all NOTICE files use the artifact id instead of module name as their 
title. However for the “integrated” NOTICE file generated in modules like 
{{flink-table-uber}} , modules without hand-written NOTICE files are referenced 
with their module name, not with artifact id.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23295) Introduce extension module SPI to StateFun

[Tzu-Li (Gordon) Tai (Jira)]

Tzu-Li (Gordon) Tai created FLINK-23295:
---

 Summary: Introduce extension module SPI to StateFun
 Key: FLINK-23295
 URL: https://issues.apache.org/jira/browse/FLINK-23295
 Project: Flink
  Issue Type: Sub-task
  Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
 Fix For: statefun-3.1.0


To support pluggable extensions, e.g. custom HTTP clients for remote 
invocation, we'd like to add a generic way for users to provide their own 
implementations of various pluggable components.

This will take the form of {{ExtensionModule}}s, where a user that is providing 
an extension implements such a module to be included into the StateFun app. 
Each {{ExtensionModule}} may bind multiple extensions identified by unique 
typenames. Other components of the application, such as functions and IO 
modules, may access these extensions through their typenames.

The SPI would likely look like this:
{code}
public interface ExtensionModule {
void configure(Map globalConfiguration, Binder binder);

interface Binder {
 void bindExtension(TypeName typeName, T extension);
}
}
{code} 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23293) Support pluggable / extendable HTTP transport for StateFun remote invocations

[Tzu-Li (Gordon) Tai (Jira)]

Tzu-Li (Gordon) Tai created FLINK-23293:
---

 Summary: Support pluggable / extendable HTTP transport for 
StateFun remote invocations
 Key: FLINK-23293
 URL: https://issues.apache.org/jira/browse/FLINK-23293
 Project: Flink
  Issue Type: Improvement
  Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
 Fix For: statefun-3.1.0


For some of our advanced users, it is required for them to use their own HTTP 
client implementations for remote function invocations.

Towards that end, we'd like to support a generic way to plugin custom 
implementations, with the HTTP client being one of the initially supported 
extensions.

This includes a few separate sub-tasks that will be added under this ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New PMC member: Guowei Ma

[Roman Khachatryan]

Congratulations!

Regards,
Roman

On Wed, Jul 7, 2021 at 8:24 AM Rui Li  wrote:
>
> Congratulations Guowei!
>
> On Wed, Jul 7, 2021 at 1:01 PM Benchao Li  wrote:
>
> > Congratulations!
> >
> > Dian Fu  于2021年7月7日周三 下午12:46写道：
> >
> > > Congratulations, Guowei!
> > >
> > > Regards,
> > > Dian
> > >
> > > > 2021年7月7日 上午10:37，Yun Gao  写道：
> > > >
> > > > Congratulations Guowei!
> > > >
> > > >
> > > > Best,
> > > > Yun
> > > >
> > > >
> > > > --
> > > > Sender:JING ZHANG
> > > > Date:2021/07/07 10:33:51
> > > > Recipient:dev
> > > > Theme:Re: [ANNOUNCE] New PMC member: Guowei Ma
> > > >
> > > > Congratulations,  Guowei Ma!
> > > >
> > > > Best regards,
> > > > JING ZHANG
> > > >
> > > > Zakelly Lan  于2021年7月7日周三 上午10:30写道：
> > > >
> > > >> Congratulations, Guowei!
> > > >>
> > > >> Best,
> > > >> Zakelly
> > > >>
> > > >> On Wed, Jul 7, 2021 at 10:24 AM tison  wrote:
> > > >>
> > > >>> Congrats! NB.
> > > >>>
> > > >>> Best,
> > > >>> tison.
> > > >>>
> > > >>>
> > > >>> Jark Wu  于2021年7月7日周三 上午10:20写道：
> > > >>>
> > >  Congratulations Guowei!
> > > 
> > >  Best,
> > >  Jark
> > > 
> > >  On Wed, 7 Jul 2021 at 09:54, XING JIN 
> > > wrote:
> > > 
> > > > Congratulations, Guowei~ !
> > > >
> > > > Best,
> > > > Jin
> > > >
> > > > Xintong Song  于2021年7月7日周三 上午9:37写道：
> > > >
> > > >> Congratulations, Guowei~!
> > > >>
> > > >> Thank you~
> > > >>
> > > >> Xintong Song
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Jul 7, 2021 at 9:31 AM Qingsheng Ren 
> > >  wrote:
> > > >>
> > > >>> Congratulations Guowei!
> > > >>>
> > > >>> --
> > > >>> Best Regards,
> > > >>>
> > > >>> Qingsheng Ren
> > > >>> Email: renqs...@gmail.com
> > > >>> 2021年7月7日 +0800 09:30 Leonard Xu ，写道：
> > >  Congratulations! Guowei Ma
> > > 
> > >  Best,
> > >  Leonard
> > > 
> > > > ÔÚ 2021Äê7ÔÂ6ÈÕ£¬21:56£¬Kurt Young  дµÀ£º
> > > >
> > > > Hi all!
> > > >
> > > > I'm very happy to announce that Guowei Ma has joined the
> > > >> Flink
> > >  PMC!
> > > >
> > > > Congratulations and welcome Guowei!
> > > >
> > > > Best,
> > > > Kurt
> > > 
> > > >>>
> > > >>
> > > >
> > > 
> > > >>>
> > > >>
> > > >
> > >
> > >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>
>
> --
> Best regards!
> Rui Li

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Roman Khachatryan]

Congrats!

Regards,
Roman


On Wed, Jul 7, 2021 at 8:28 AM Qingsheng Ren  wrote:
>
> Congratulations Yang!
>
> --
> Best Regards,
>
> Qingsheng Ren
> Email: renqs...@gmail.com
> On Jul 7, 2021, 2:26 PM +0800, Rui Li , wrote:
> > Congratulations Yang ~
> >
> > On Wed, Jul 7, 2021 at 1:01 PM Benchao Li  wrote:
> >
> > > Congratulations!
> > >
> > > Peter Huang  于2021年7月7日周三 下午12:54写道：
> > >
> > > > Congratulations, Yang.
> > > >
> > > > Best Regards
> > > > Peter Huang
> > > >
> > > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu  wrote:
> > > >
> > > > > Congratulations, Yang,
> > > > >
> > > > > Regards,
> > > > > Dian
> > > > >
> > > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > > >
> > > > > > Congratulations, Yang Wang.
> > > > > >
> > > > > > Best
> > > > > > Jary
> > > > > >
> > > > > > Yun Gao  于2021年7月7日周三 上午10:38写道：
> > > > > >
> > > > > > > Congratulations Yang!
> > > > > > >
> > > > > > > Best,
> > > > > > > Yun
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Sender:Jark Wu
> > > > > > > Date:2021/07/07 10:20:27
> > > > > > > Recipient:dev
> > > > > > > Cc:Yang Wang; 
> > > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang
> > > > > > >
> > > > > > > Congratulations Yang Wang!
> > > > > > >
> > > > > > > Best,
> > > > > > > Jark
> > > > > > >
> > > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song 
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > On behalf of the PMC, I'm very happy to announce Yang Wang as a 
> > > > > > > > new
> > > > > Flink
> > > > > > > > committer.
> > > > > > > >
> > > > > > > > Yang has been a very active contributor for more than two years,
> > > > mainly
> > > > > > > > focusing on Flink's deployment components. He's a main 
> > > > > > > > contributor
> > > > and
> > > > > > > > maintainer of Flink's native Kubernetes deployment and native
> > > > > Kubernetes
> > > > > > > > HA. He's also very active on the mailing lists, participating in
> > > > > > > > discussions and helping with user questions.
> > > > > > > >
> > > > > > > > Please join me in congratulating Yang Wang for becoming a Flink
> > > > > > > committer!
> > > > > > > >
> > > > > > > > Thank you~
> > > > > > > >
> > > > > > > > Xintong Song
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Best,
> > > Benchao Li
> > >
> >
> >
> > --
> > Best regards!
> > Rui Li

[jira] [Created] (FLINK-23292) [DOCS]Some links on page docs/content/docs/dev/table/concepts/dynamic_tables is failed and 404 is returned

[wuguihu (Jira)]

wuguihu created FLINK-23292:
---

 Summary: [DOCS]Some links on page 
docs/content/docs/dev/table/concepts/dynamic_tables is failed and 404 is 
returned
 Key: FLINK-23292
 URL: https://issues.apache.org/jira/browse/FLINK-23292
 Project: Flink
  Issue Type: Bug
  Components: Documentation
Reporter: wuguihu
 Attachments: image-20210707153836711.png

 

The original is as follows：
{code:java}
 [time attributes](time_attributes.html) 
{code}
When i click the link [time attributes| 
https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/table/concepts/dynamic_tables/time_attributes.html],
 404 is returned.

 

The English page 
is：[https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/table/concepts/dynamic_tables]
{noformat}

//The markdown file: docs/content/docs/dev/table/concepts/dynamic_tables.md
//1、 line 121 
[time attributes](time_attributes.html) 

This link should be modified to [time attributes]({{< ref 
"docs/dev/table/concepts/time_attributes" >}}) 

//2、 line 158  
[Query Configuration](query_configuration.html)  

This link should be modified to  [Query Configuration]({{< ref 
"docs/dev/table/config" >}})

//3、line 180  
[TableSources and TableSinks](../sourceSinks.html#define-a-tablesink)

This link should be modified to [TableSources and TableSinks]({{< ref 
"docs/dev/table/sourcesSinks">}}#dynamic-table-sink){noformat}
The Chinese page 
is：[https://ci.apache.org/projects/flink/flink-docs-master/zh/docs/dev/table/concepts/dynamic_tables/]
{noformat}

//The markdown file: docs/content.zh/docs/dev/table/concepts/dynamic_tables.md

//1、 line 120 
[时间属性](time_attributes.html)

This link should be modified to [time attributes]({{< ref 
"docs/dev/table/concepts/time_attributes" >}}) 

//2、 line 159  
[查询配置](query_configuration.html)  

This link should be modified to  [查询配置]({{< ref "docs/dev/table/config" >}})

//3、line 182  
[TableSources and TableSinks](../sourceSinks.html#define-a-tablesink)

This link should be modified to [TableSources and TableSinks]({{< ref 
"docs/dev/table/sourcesSinks">}}#dynamic-table-sink){noformat}
 

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23291) Bump flink-shaded-jackson to 2.12.3

[Chesnay Schepler (Jira)]

Chesnay Schepler created FLINK-23291:


 Summary: Bump flink-shaded-jackson to 2.12.3
 Key: FLINK-23291
 URL: https://issues.apache.org/jira/browse/FLINK-23291
 Project: Flink
  Issue Type: Improvement
  Components: BuildSystem / Shaded
Reporter: Chesnay Schepler
Assignee: Chesnay Schepler
 Fix For: shaded-1.14, 1.14.0


2.12.1 -> 2.12.3



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23290) cast '(LZ *3' as boolean, get a null

[xiaojin.wy (Jira)]

xiaojin.wy created FLINK-23290:
--

 Summary: cast '(LZ *3' as boolean, get a null
 Key: FLINK-23290
 URL: https://issues.apache.org/jira/browse/FLINK-23290
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Runtime
Affects Versions: 1.14.0
Reporter: xiaojin.wy


CREATE TABLE database5_t0(
c0 VARCHAR , c1 BIGINT 
) WITH (
 'connector' = 'filesystem',
 'path' = 'hdfs:///tmp/database5_t0.csv',
 'format' = 'csv'
);
INSERT OVERWRITE database5_t0(c0, c1) VALUES('(LZ *3', 2135917226)
SELECT database5_t0.c0 AS ref0 FROM database5_t0 WHERE CAST (database5_t0.c0 AS 
BOOLEAN)
*After excuting that, you will get the error:
*
Caused by: java.lang.NullPointerException
at BatchExecCalc$20.processElement(Unknown Source)
at 
org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:101)
at 
org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:82)
at 
org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)
at 
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)
at 
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)
at 
org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:319)
at 
org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:414)
at 
org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction.run(InputFormatSourceFunction.java:92)
at 
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:104)
at 
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:62)
at 
org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269)




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Job Recovery Time on TM Lost

[刘建刚]

It is really helpful to find the lost container quickly. In our inner flink
version, we optimize it by task's report and jobmaster's probe. When a task
fails because of the connection, it reports to the jobmaster. The jobmaster
will try to confirm the liveness of the unconnected taskmanager for certain
times by config. If the jobmaster find the taskmanager unconnected or dead,
it releases the taskmanger. This will work for most cases. For an unstable
environment, config needs adjustment.

Gen Luo  于2021年7月6日周二 下午8:41写道：

> Yes, I have noticed the PR and commented there with some consideration
> about the new option. We can discuss further there.
>
> On Tue, Jul 6, 2021 at 6:04 PM Till Rohrmann  wrote:
>
> > This is actually a very good point Gen. There might not be a lot to gain
> > for us by implementing a fancy algorithm for figuring out whether a TM is
> > dead or not based on failed heartbeat RPCs from the JM if the TM <> TM
> > communication does not tolerate failures and directly fails the affected
> > tasks. This assumes that the JM and TM run in the same environment.
> >
> > One simple approach could be to make the number of failed heartbeat RPCs
> > until a target is marked as unreachable configurable because what
> > represents a good enough criterion in one user's environment might
> produce
> > too many false-positives in somebody else's environment. Or even simpler,
> > one could say that one can disable reacting to a failed heartbeat RPC as
> it
> > is currently the case.
> >
> > We currently have a discussion about this on this PR [1]. Maybe you wanna
> > join the discussion there and share your insights.
> >
> > [1] https://github.com/apache/flink/pull/16357
> >
> > Cheers,
> > Till
> >
> > On Tue, Jul 6, 2021 at 4:37 AM Gen Luo  wrote:
> >
> >> I know that there are retry strategies for akka rpc frameworks. I was
> >> just considering that, since the environment is shared by JM and TMs,
> and
> >> the connections among TMs (using netty) are flaky in unstable
> environments,
> >> which will also cause the job failure, is it necessary to build a
> >> strongly guaranteed connection between JM and TMs, or it could be as
> flaky
> >> as the connections among TMs?
> >>
> >> As far as I know, connections among TMs will just fail on their first
> >> connection loss, so behaving like this in JM just means "as flaky as
> >> connections among TMs". In a stable environment it's good enough, but
> in an
> >> unstable environment, it indeed increases the instability. IMO, though a
> >> single connection loss is not reliable, a double check should be good
> >> enough. But since I'm not experienced with an unstable environment, I
> can't
> >> tell whether that's also enough for it.
> >>
> >> On Mon, Jul 5, 2021 at 5:59 PM Till Rohrmann 
> >> wrote:
> >>
> >>> I think for RPC communication there are retry strategies used by the
> >>> underlying Akka ActorSystem. So a RpcEndpoint can reconnect to a remote
> >>> ActorSystem and resume communication. Moreover, there are also
> >>> reconciliation protocols in place which reconcile the states between
> the
> >>> components because of potentially lost RPC messages. So the main
> question
> >>> would be whether a single connection loss is good enough for
> triggering the
> >>> timeout or whether we want a more elaborate mechanism to reason about
> the
> >>> availability of the remote system (e.g. a couple of lost heartbeat
> >>> messages).
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Mon, Jul 5, 2021 at 10:00 AM Gen Luo  wrote:
> >>>
>  As far as I know, a TM will report connection failure once its
>  connected TM is lost. I suppose JM can believe the report and fail the
>  tasks in the lost TM if it also encounters a connection failure.
> 
>  Of course, it won't work if the lost TM is standalone. But I suppose
> we
>  can use the same strategy as the connected scenario. That is,
> consider it
>  possibly lost on the first connection loss, and fail it if double
> check
>  also fails. The major difference is the senders of the probes are the
> same
>  one rather than two different roles, so the results may tend to be
> the same.
> 
>  On the other hand, the fact also means that the jobs can be fragile in
>  an unstable environment, no matter whether the failover is triggered
> by TM
>  or JM. So maybe it's not that worthy to introduce extra
> configurations for
>  fault tolerance of heartbeat, unless we also introduce some retry
>  strategies for netty connections.
> 
> 
>  On Fri, Jul 2, 2021 at 9:34 PM Till Rohrmann 
>  wrote:
> 
> > Could you share the full logs with us for the second experiment, Lu?
> I
> > cannot tell from the top of my head why it should take 30s unless
> you have
> > configured a restart delay of 30s.
> >
> > Let's discuss FLINK-23216 on the JIRA ticket, Gen.
> >
> > I've now implemented FLINK-23209 [1] but it somehow has the 

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Qingsheng Ren]

Congratulations Yang!

--
Best Regards,

Qingsheng Ren
Email: renqs...@gmail.com
On Jul 7, 2021, 2:26 PM +0800, Rui Li , wrote:
> Congratulations Yang ~
>
> On Wed, Jul 7, 2021 at 1:01 PM Benchao Li  wrote:
>
> > Congratulations!
> >
> > Peter Huang  于2021年7月7日周三 下午12:54写道：
> >
> > > Congratulations, Yang.
> > >
> > > Best Regards
> > > Peter Huang
> > >
> > > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu  wrote:
> > >
> > > > Congratulations, Yang,
> > > >
> > > > Regards,
> > > > Dian
> > > >
> > > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > > >
> > > > > Congratulations, Yang Wang.
> > > > >
> > > > > Best
> > > > > Jary
> > > > >
> > > > > Yun Gao  于2021年7月7日周三 上午10:38写道：
> > > > >
> > > > > > Congratulations Yang!
> > > > > >
> > > > > > Best,
> > > > > > Yun
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Sender:Jark Wu
> > > > > > Date:2021/07/07 10:20:27
> > > > > > Recipient:dev
> > > > > > Cc:Yang Wang; 
> > > > > > Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang
> > > > > >
> > > > > > Congratulations Yang Wang!
> > > > > >
> > > > > > Best,
> > > > > > Jark
> > > > > >
> > > > > > On Wed, 7 Jul 2021 at 10:09, Xintong Song 
> > > > wrote:
> > > > > >
> > > > > > > Hi everyone,
> > > > > > >
> > > > > > > On behalf of the PMC, I'm very happy to announce Yang Wang as a 
> > > > > > > new
> > > > Flink
> > > > > > > committer.
> > > > > > >
> > > > > > > Yang has been a very active contributor for more than two years,
> > > mainly
> > > > > > > focusing on Flink's deployment components. He's a main contributor
> > > and
> > > > > > > maintainer of Flink's native Kubernetes deployment and native
> > > > Kubernetes
> > > > > > > HA. He's also very active on the mailing lists, participating in
> > > > > > > discussions and helping with user questions.
> > > > > > >
> > > > > > > Please join me in congratulating Yang Wang for becoming a Flink
> > > > > > committer!
> > > > > > >
> > > > > > > Thank you~
> > > > > > >
> > > > > > > Xintong Song
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > >
> >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>
>
> --
> Best regards!
> Rui Li

Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang

[Rui Li]

Congratulations Yang ~

On Wed, Jul 7, 2021 at 1:01 PM Benchao Li  wrote:

> Congratulations!
>
> Peter Huang  于2021年7月7日周三 下午12:54写道：
>
> > Congratulations, Yang.
> >
> > Best Regards
> > Peter Huang
> >
> > On Tue, Jul 6, 2021 at 9:48 PM Dian Fu  wrote:
> >
> > > Congratulations, Yang,
> > >
> > > Regards,
> > > Dian
> > >
> > > > 2021年7月7日 上午10:46，Jary Zhen  写道：
> > > >
> > > > Congratulations, Yang Wang.
> > > >
> > > > Best
> > > > Jary
> > > >
> > > > Yun Gao  于2021年7月7日周三 上午10:38写道：
> > > >
> > > >> Congratulations Yang!
> > > >>
> > > >> Best,
> > > >> Yun
> > > >>
> > > >>
> > > >> --
> > > >> Sender:Jark Wu
> > > >> Date:2021/07/07 10:20:27
> > > >> Recipient:dev
> > > >> Cc:Yang Wang; 
> > > >> Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yang Wang
> > > >>
> > > >> Congratulations Yang Wang!
> > > >>
> > > >> Best,
> > > >> Jark
> > > >>
> > > >> On Wed, 7 Jul 2021 at 10:09, Xintong Song 
> > > wrote:
> > > >>
> > > >>> Hi everyone,
> > > >>>
> > > >>> On behalf of the PMC, I'm very happy to announce Yang Wang as a new
> > > Flink
> > > >>> committer.
> > > >>>
> > > >>> Yang has been a very active contributor for more than two years,
> > mainly
> > > >>> focusing on Flink's deployment components. He's a main contributor
> > and
> > > >>> maintainer of Flink's native Kubernetes deployment and native
> > > Kubernetes
> > > >>> HA. He's also very active on the mailing lists, participating in
> > > >>> discussions and helping with user questions.
> > > >>>
> > > >>> Please join me in congratulating Yang Wang for becoming a Flink
> > > >> committer!
> > > >>>
> > > >>> Thank you~
> > > >>>
> > > >>> Xintong Song
> > > >>>
> > > >>
> > > >>
> > >
> > >
> >
>
>
> --
>
> Best,
> Benchao Li
>


-- 
Best regards!
Rui Li

Re: [ANNOUNCE] New PMC member: Guowei Ma

[Rui Li]

Congratulations Guowei!

On Wed, Jul 7, 2021 at 1:01 PM Benchao Li  wrote:

> Congratulations!
>
> Dian Fu  于2021年7月7日周三 下午12:46写道：
>
> > Congratulations, Guowei!
> >
> > Regards,
> > Dian
> >
> > > 2021年7月7日 上午10:37，Yun Gao  写道：
> > >
> > > Congratulations Guowei!
> > >
> > >
> > > Best,
> > > Yun
> > >
> > >
> > > --
> > > Sender:JING ZHANG
> > > Date:2021/07/07 10:33:51
> > > Recipient:dev
> > > Theme:Re: [ANNOUNCE] New PMC member: Guowei Ma
> > >
> > > Congratulations,  Guowei Ma!
> > >
> > > Best regards,
> > > JING ZHANG
> > >
> > > Zakelly Lan  于2021年7月7日周三 上午10:30写道：
> > >
> > >> Congratulations, Guowei!
> > >>
> > >> Best,
> > >> Zakelly
> > >>
> > >> On Wed, Jul 7, 2021 at 10:24 AM tison  wrote:
> > >>
> > >>> Congrats! NB.
> > >>>
> > >>> Best,
> > >>> tison.
> > >>>
> > >>>
> > >>> Jark Wu  于2021年7月7日周三 上午10:20写道：
> > >>>
> >  Congratulations Guowei!
> > 
> >  Best,
> >  Jark
> > 
> >  On Wed, 7 Jul 2021 at 09:54, XING JIN 
> > wrote:
> > 
> > > Congratulations, Guowei~ !
> > >
> > > Best,
> > > Jin
> > >
> > > Xintong Song  于2021年7月7日周三 上午9:37写道：
> > >
> > >> Congratulations, Guowei~!
> > >>
> > >> Thank you~
> > >>
> > >> Xintong Song
> > >>
> > >>
> > >>
> > >> On Wed, Jul 7, 2021 at 9:31 AM Qingsheng Ren 
> >  wrote:
> > >>
> > >>> Congratulations Guowei!
> > >>>
> > >>> --
> > >>> Best Regards,
> > >>>
> > >>> Qingsheng Ren
> > >>> Email: renqs...@gmail.com
> > >>> 2021年7月7日 +0800 09:30 Leonard Xu ，写道：
> >  Congratulations! Guowei Ma
> > 
> >  Best,
> >  Leonard
> > 
> > > ÔÚ 2021Äê7ÔÂ6ÈÕ£¬21:56£¬Kurt Young  дµÀ£º
> > >
> > > Hi all!
> > >
> > > I'm very happy to announce that Guowei Ma has joined the
> > >> Flink
> >  PMC!
> > >
> > > Congratulations and welcome Guowei!
> > >
> > > Best,
> > > Kurt
> > 
> > >>>
> > >>
> > >
> > 
> > >>>
> > >>
> > >
> >
> >
>
> --
>
> Best,
> Benchao Li
>


-- 
Best regards!
Rui Li