Re: [VOTE] FLIP-53: Fine Grained Operator Resource Management

+1 for FLIP-53. I would like to raise one minor concern regarding to implementing request absolute amount of memory case. Currently, it will be translated to a memory fraction during compile, and translate back to absolute value during execution. There is a risk that the user might get less than h

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Klou! Best, Kurt On Sat, Sep 7, 2019 at 2:37 PM ying wrote: > Congratulations Kostas! > > On Fri, Sep 6, 2019 at 11:21 PM Gary Yao wrote: > > > Congratulations Klou! > > > > On Sat, Sep 7, 2019 at 6:21 AM Thomas Weise wrote: > > > > > Congratulations! > > > > > > > > > On Fri

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Kostas! On Fri, Sep 6, 2019 at 11:21 PM Gary Yao wrote: > Congratulations Klou! > > On Sat, Sep 7, 2019 at 6:21 AM Thomas Weise wrote: > > > Congratulations! > > > > > > On Fri, Sep 6, 2019 at 9:22 AM zhijiang > .invalid> > > wrote: > > > > > Congratulations Klou! > > > > > > B

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Klou! On Sat, Sep 7, 2019 at 6:21 AM Thomas Weise wrote: > Congratulations! > > > On Fri, Sep 6, 2019 at 9:22 AM zhijiang .invalid> > wrote: > > > Congratulations Klou! > > > > Best, > > Zhijiang > > -- > > From:Zhu

[jira] [Created] (FLINK-13999) Correct the documentation of MATCH_RECOGNIZE

Dian Fu created FLINK-13999: --- Summary: Correct the documentation of MATCH_RECOGNIZE Key: FLINK-13999 URL: https://issues.apache.org/jira/browse/FLINK-13999 Project: Flink Issue Type: Bug

Re: [DISCUSS] Features for Apache Flink 1.10

Thanks Gary for kicking off this discussion. Really appreciate that you and Yu offer to help to manage 1.10 release. +1 for Gary and Yu as release managers. Thanks, Zhu Zhu Dian Fu 于2019年9月7日周六 下午12:26写道： > Hi Gary, > > Thanks for kicking off the release schedule of 1.10. +1 for you and Yu Li

Re: [DISCUSS] Features for Apache Flink 1.10

Hi Gary, Thanks for kicking off the release schedule of 1.10. +1 for you and Yu Li as the release manager. The feature freeze/release time sounds reasonable. Thanks, Dian > 在 2019年9月7日，上午11:30，Jark Wu 写道： > > Thanks Gary for kicking off the discussion for 1.10 release. > > +1 for Gary and Y

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations! On Fri, Sep 6, 2019 at 9:22 AM zhijiang wrote: > Congratulations Klou! > > Best, > Zhijiang > -- > From:Zhu Zhu > Send Time:2019年9月6日(星期五) 17:19 > To:dev > Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink PM

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Kostas! Well deserved. Best, Hequn On Sat, Sep 7, 2019 at 10:48 AM Jark Wu wrote: > Congratulations Klou! > > > > 在 2019年9月7日，00:21，zhijiang 写道： > > > > Congratulations Klou! > > > > Best, > > Zhijiang > > -- > > F

Re: [DISCUSS] Features for Apache Flink 1.10

Thanks Gary for kicking off the discussion for 1.10 release. +1 for Gary and Yu as release managers. Thank you for you effort. Best, Jark > 在 2019年9月7日，00:52，zhijiang 写道： > > Hi Gary, > > Thanks for kicking off the features for next release 1.10. I am very > supportive of you and Yu Li to

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Klou! > 在 2019年9月7日，00:21，zhijiang 写道： > > Congratulations Klou! > > Best, > Zhijiang > -- > From:Zhu Zhu > Send Time:2019年9月6日(星期五) 17:19 > To:dev > Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC > >

[jira] [Created] (FLINK-13998) Fix ORC test failure with Hive 2.0.x

Xuefu Zhang created FLINK-13998: --- Summary: Fix ORC test failure with Hive 2.0.x Key: FLINK-13998 URL: https://issues.apache.org/jira/browse/FLINK-13998 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-13997) Remove legacy LeaderAddressAndId

TisonKun created FLINK-13997: Summary: Remove legacy LeaderAddressAndId Key: FLINK-13997 URL: https://issues.apache.org/jira/browse/FLINK-13997 Project: Flink Issue Type: Sub-task Compo

Re: Checkpointing clarification

Hi Dominik, A record is already processed once it enters the window. Thus the checkpoint barrier does not get blocked before the window containing the leading records is triggered. A window is actually part of the states of the WindowOperator and the data records processing is to build up this sta

Re: [DISCUSS] Features for Apache Flink 1.10

Hi Gary, Thanks for kicking off the features for next release 1.10. I am very supportive of you and Yu Li to be the relaese managers. Just mention another two improvements which want to be covered in FLINK-1.10 and I already confirmed with Piotr to reach an agreement before. 1. Data serialize

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Klou! Best, Zhijiang -- From:Zhu Zhu Send Time:2019年9月6日(星期五) 17:19 To:dev Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC Congratulations Kostas! Thanks, Zhu Zhu Yu Li 于2019年9月6日周五 下午10:49写道： > Congra

Re: [DISCUSS] Features for Apache Flink 1.10

Hi Gary, Thanks for kicking off the feature discussion. +1 for Gary and Yu as release managers. Cheers, Kostas On Fri, Sep 6, 2019 at 5:06 PM Gary Yao wrote: > > Hi community, > > Since Apache Flink 1.9.0 has been released more than 2 weeks ago, I want to > start kicking off the discussion abo

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Kostas! Thanks, Zhu Zhu Yu Li 于2019年9月6日周五 下午10:49写道： > Congratulations Klou! > > Best Regards, > Yu > > > On Fri, 6 Sep 2019 at 22:43, Forward Xu wrote: > > > Congratulations Kloudas! > > > > > > Best, > > > > Forward > > > > Dawid Wysakowicz 于2019年9月6日周五 下午10:36写道： > > > > >

[DISCUSS] Features for Apache Flink 1.10

Hi community, Since Apache Flink 1.9.0 has been released more than 2 weeks ago, I want to start kicking off the discussion about what we want to achieve for the 1.10 release. Based on discussions with various people as well as observations from mailing list threads, Yu Li and I have compiled a li

Re: [ANNOUNCE] Java 11 cron builds activated on master

Great to know! Thanks for the efforts Chesnay and will keep an eye on it. Best Regards, Yu On Thu, 5 Sep 2019 at 21:58, Chesnay Schepler wrote: > Hello everyone, > > I just wanted to inform everyone that we now run Java 11 builds on > Travis as part of the cron jobs, subsuming the existing Jav

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Klou! Best Regards, Yu On Fri, 6 Sep 2019 at 22:43, Forward Xu wrote: > Congratulations Kloudas! > > > Best, > > Forward > > Dawid Wysakowicz 于2019年9月6日周五 下午10:36写道： > > > Congratulations Klou! > > > > Best, > > > > Dawid > > > > On 06/09/2019 14:55, Fabian Hueske wrote: > > >

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Kloudas! Best, Forward Dawid Wysakowicz 于2019年9月6日周五 下午10:36写道： > Congratulations Klou! > > Best, > > Dawid > > On 06/09/2019 14:55, Fabian Hueske wrote: > > Hi everyone, > > > > I'm very happy to announce that Kostas Kloudas is joining the Flink PMC. > > Kostas is contributin

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Klou! Best, Dawid On 06/09/2019 14:55, Fabian Hueske wrote: > Hi everyone, > > I'm very happy to announce that Kostas Kloudas is joining the Flink PMC. > Kostas is contributing to Flink for many years and puts lots of effort in > helping our users and growing the Flink community.

Re: [VOTE] FLIP-53: Fine Grained Operator Resource Management

Thanks for starting the vote @Xintong +1 for the FLIP-53 Best, Andrey On Fri, Sep 6, 2019 at 3:53 PM Till Rohrmann wrote: > Hi Xintong, > > thanks for starting this vote. The proposal looks good and, hence, +1 for > it. > > One comment I have is concerning the first implementation step. I woul

[jira] [Created] (FLINK-13996) Maven instructions for 3.3+ do not cover all shading special cases

Chesnay Schepler created FLINK-13996: Summary: Maven instructions for 3.3+ do not cover all shading special cases Key: FLINK-13996 URL: https://issues.apache.org/jira/browse/FLINK-13996 Project: F

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congrats Klou! Zili Chen 于2019年9月6日周五 下午9:51写道： > Congrats Klou! > > Best, > tison. > > > Till Rohrmann 于2019年9月6日周五 下午9:23写道： > >> Congrats Klou! >> >> Cheers, >> Till >> >> On Fri, Sep 6, 2019 at 3:00 PM Dian Fu wrote: >> >>> Congratulations Kostas! >>> >>> Regards, >>> Dian >>> >>> > 在 2019

Re: [VOTE] FLIP-53: Fine Grained Operator Resource Management

Hi Xintong, thanks for starting this vote. The proposal looks good and, hence, +1 for it. One comment I have is concerning the first implementation step. I would suggest to not add the flag allSourcesInSamePipelinedRegion to the ExecutionConfig because the ExecutionConfig is public API. Ideally w

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congrats Klou! Best, tison. Till Rohrmann 于2019年9月6日周五 下午9:23写道： > Congrats Klou! > > Cheers, > Till > > On Fri, Sep 6, 2019 at 3:00 PM Dian Fu wrote: > >> Congratulations Kostas! >> >> Regards, >> Dian >> >> > 在 2019年9月6日，下午8:58，Wesley Peng 写道： >> > >> > On 2019/9/6 8:55 下午, Fabian Hueske w

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congrats Klou! Cheers, Till On Fri, Sep 6, 2019 at 3:00 PM Dian Fu wrote: > Congratulations Kostas! > > Regards, > Dian > > > 在 2019年9月6日，下午8:58，Wesley Peng 写道： > > > > On 2019/9/6 8:55 下午, Fabian Hueske wrote: > >> I'm very happy to announce that Kostas Kloudas is joining the Flink PMC. > >>

[DISCUSS] FLIP-67: Global partitions lifecycle

Hello, FLIP-36 (interactive programming) proposes a new programming paradigm where jobs are built incrementally by the user. To support this in an efficient manner I propose to extend part

[jira] [Created] (FLINK-13995) Fix shading of the licence information of netty

Arvid Heise created FLINK-13995: --- Summary: Fix shading of the licence information of netty Key: FLINK-13995 URL: https://issues.apache.org/jira/browse/FLINK-13995 Project: Flink Issue Type: Bug

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congratulations Kostas! Regards, Dian > 在 2019年9月6日，下午8:58，Wesley Peng 写道： > > On 2019/9/6 8:55 下午, Fabian Hueske wrote: >> I'm very happy to announce that Kostas Kloudas is joining the Flink PMC. >> Kostas is contributing to Flink for many years and puts lots of effort in >> helping our users

[ANNOUNCE] Kostas Kloudas joins the Flink PMC

Hi everyone, I'm very happy to announce that Kostas Kloudas is joining the Flink PMC. Kostas is contributing to Flink for many years and puts lots of effort in helping our users and growing the Flink community. Please join me in congratulating Kostas! Cheers, Fabian

Re: [DISCUSS] Releasing Flink 1.8.2

Hi all, Thanks all of you for fixing issues for 1.8.2 release! The VOTE mail thread of the first RC of 1.8.2 already brought up. I would appreciate if you can help to check the release and VOTE the RC1. Thanks, Jark On Wed, 4 Sep 2019 at 16:57, Aljoscha Krettek wrote: > Hi, > > I’m just runnin

[VOTE] Release 1.8.2, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 1.8.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1],

[jira] [Created] (FLINK-13994) Translate "Getting Started" overview to Chinese

Fabian Hueske created FLINK-13994: - Summary: Translate "Getting Started" overview to Chinese Key: FLINK-13994 URL: https://issues.apache.org/jira/browse/FLINK-13994 Project: Flink Issue Type:

Re: Checkpointing clarification

When a WindowOperator receives all the barrier from the upstream, it will forward the barrier to downstream operator and perform the checkpoint asynchronously. It doesn't have to wait the window to trigger before sending out the barrier. Regards, Dian > 在 2019年9月6日，下午8:02，Dominik Wosiński 写道：

[jira] [Created] (FLINK-13993) Using FlinkUserCodeClassLoaders to load the user class in the perjob mode

Guowei Ma created FLINK-13993: - Summary: Using FlinkUserCodeClassLoaders to load the user class in the perjob mode Key: FLINK-13993 URL: https://issues.apache.org/jira/browse/FLINK-13993 Project: Flink

Checkpointing clarification

Hello, I have a slight doubt on checkpointing in Flink and wanted to clarify my understanding. Flink uses barriers internally to keep track of the records that were processed. The documentation[1] describes it as the checkpoint was only happening when the barriers are transferred to the sink. So l

Re: [VOTE] FLIP-53: Fine Grained Operator Resource Management

Thanks Xintong for proposing this better resource management. This helps a lot to users who want to better manage the job resources. And would be even more useful if in the future we can have auto-tuning mechanism for jobs. +1 (non-binding) Thanks, Zhu Zhu Xintong Song 于2019年9月6日周五 上午11:17写道：

Re: [DISCUSS] Flink Python User-Defined Function for Table API

Hi, Thanks for the quick response! I think this looks good now and it should be something that everyone can agree on as a first step. Best, Aljoscha > On 6. Sep 2019, at 12:22, Dian Fu wrote: > > Hi all, > > I have updated the FLIP and removed content relate to UDAF and also changed > the t

Re: [DISCUSS] Flink Python User-Defined Function for Table API

Hi all, I have updated the FLIP and removed content relate to UDAF and also changed the title of the FLIP to "Flink Python User-Defined Stateless Function for Table". Does it make sense to you? Regards, Dian > 在 2019年9月6日，下午6:09，Dian Fu 写道： > > Hi all, > > Thanks a lot for the discussion h

[jira] [Created] (FLINK-13992) Refactor Optional parameter in InputGateWithMetrics#updateMetrics

TisonKun created FLINK-13992: Summary: Refactor Optional parameter in InputGateWithMetrics#updateMetrics Key: FLINK-13992 URL: https://issues.apache.org/jira/browse/FLINK-13992 Project: Flink Is

Re: [DISCUSS] Flink Python User-Defined Function for Table API

Hi all, Thanks a lot for the discussion here. It makes sense to limit the scope of this FLIP to only ScalarFunction. I'll update the FLIP and remove the content relating to UDAF. Thanks, Dian > 在 2019年9月6日，下午6:02，jincheng sun 写道： > > Hi， > > Sure, for ensure the 1.10 relesae of flink, let's

[jira] [Created] (FLINK-13991) Add git exclusion for 1.9+ features to 1.8

Chesnay Schepler created FLINK-13991: Summary: Add git exclusion for 1.9+ features to 1.8 Key: FLINK-13991 URL: https://issues.apache.org/jira/browse/FLINK-13991 Project: Flink Issue Type

Re: Is Flink documentation deployment script broken ?

Thanks Chesnay for reporting this. > 在 2019年9月6日，17:47，Chesnay Schepler 写道： > > The scripts are fine, but the buildbot slave is currently down. > > I've already opened a ticket with INFRA: > https://issues.apache.org/jira/browse/INFRA-18986 > > On 06/09/2019 11:44, Jark Wu wrote: >> Hi all,

[jira] [Created] (FLINK-13990) Remove JobModificationException

TisonKun created FLINK-13990: Summary: Remove JobModificationException Key: FLINK-13990 URL: https://issues.apache.org/jira/browse/FLINK-13990 Project: Flink Issue Type: Sub-task Compon

Re: [DISCUSS] Flink Python User-Defined Function for Table API

Hi， Sure, for ensure the 1.10 relesae of flink, let's split the FLIPs, and FLIP-58 only do the stateless part. Cheers, Jincheng Aljoscha Krettek 于2019年9月6日周五 下午5:53写道： > Hi, > > Regarding stateful functions and MapView/DataView/ListView: I think it’s > best to keep that for a later FLIP and fo

[jira] [Created] (FLINK-13989) Remove legacy ClassloadingProps

TisonKun created FLINK-13989: Summary: Remove legacy ClassloadingProps Key: FLINK-13989 URL: https://issues.apache.org/jira/browse/FLINK-13989 Project: Flink Issue Type: Sub-task Compon

Re: [DISCUSS] Flink Python User-Defined Function for Table API

Hi, Regarding stateful functions and MapView/DataView/ListView: I think it’s best to keep that for a later FLIP and focus on a more basic version. Supporting stateful functions, especially with MapView can potentially be very slow so we have to see what we can do there. For the method names, I

[jira] [Created] (FLINK-13988) Remove legacy JobManagerMode

TisonKun created FLINK-13988: Summary: Remove legacy JobManagerMode Key: FLINK-13988 URL: https://issues.apache.org/jira/browse/FLINK-13988 Project: Flink Issue Type: Sub-task Component

Re: Is Flink documentation deployment script broken ?

The scripts are fine, but the buildbot slave is currently down. I've already opened a ticket with INFRA: https://issues.apache.org/jira/browse/INFRA-18986 On 06/09/2019 11:44, Jark Wu wrote: Hi all, I merged several documentation pull requests[1][2][3] days ago. AFAIK, the documentation depl

Is Flink documentation deployment script broken ?

Hi all, I merged several documentation pull requests[1][2][3] days ago. AFAIK, the documentation deployment is scheduled every day. However, I didn't see the changes are available in the Flink doc website[4] until now. The same to Till's PR[5] merged 3 days ago. Best, Jark [1]: https://github.c

Re: [VOTE] FLIP-62: Set default restart delay for FixedDelay- and FailureRateRestartStrategy to 1s

Thanks a lot for voting. I'm closing the vote now. The vote received: * +1 votes (4 binding, 5 non-binding): - Zhu Zhu - Jingsong - Chesnay (binding) - Till (binding) - David - Jark (binding) - Zhijiang (binding) - Yu - Vino Yang * 0/-1 votes: none Thereby, the community accepted FLIP-62. I'll u

Re: [VOTE] FLIP-61 Simplify Flink's cluster level RestartStrategy configuration

Thanks a lot for voting. I'm closing the vote now. The vote received: * +1 votes (3 binding; 3 non-binding): - Chesnay (binding) - Till (binding) - Zhijiang (binding) - Zhu Zhu - Zili Chen - Vino Yang * 0/-1 votes: none Thereby, the community has accepted FLIP-61. I will update the FLIP wiki pag

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

I agree the consequences of the decision are substantial. Let's see what others think. -- Catalog functions are defined by users, and we suppose they can drop/alter it in any way they want. Thus, overwriting a catalog function doesn't seem to be a strong use case that we should be concerned about.

Re: [jira] [Created] (FLINK-13987) add new logs api, see more log files and can see logs by pages

Hi folks, I have updated the description, this is the new : As the job running, the log files are becoming large. Current log api often blocks or don't work. It's unfriendly for user. As application runs on jvm, sometime user need see log of gc. Above all, i list new api: - list taskmanage

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

Hi Dawid, Thank you for your summary. While the only difference in the two proposals is one- or three-part in naming, the consequence would be substantial. To me, there are two major use cases of temporary functions compared to persistent ones: 1. Temporary in nature and auto managed by the sessi

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Hi all, @Jingsong Could you elaborate a bit more what do you mean by /"//some Connectors//are difficult to convert all states to properties"/ All the Flink provided connectors will definitely be expressible with properties (In the end you should be able to use them from DDL). I think if a TableS

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Thanks to Dawid for starting the discussion and writeup. It looks pretty good to me except that I'm a little concerned about the object reference and string parsing in the code, which seems to an anti-pattern to OOP. Have we considered using ObjectIdenitifier with optional catalog and db parts, esp

[jira] [Created] (FLINK-13987) add new logs api, see more log files and can see logs by pages

lining created FLINK-13987: -- Summary: add new logs api, see more log files and can see logs by pages Key: FLINK-13987 URL: https://issues.apache.org/jira/browse/FLINK-13987 Project: Flink Issue Ty

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

Hi Xuefu, Thank you for your answers. Let me summarize my understanding. In principle we differ only in regards to the fact if a temporary function can be only 1-part or only 3-part identified. I can reconfirm that if the community decides it prefers the 1-part approach I will commit to that, wit

Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

Hi Fu, That'll be nice. Thanks. Best, Shimin Dian Fu 于2019年9月6日周五 下午3:17写道： > Hi Shimin, > > It can be guaranteed to be an atomic operation. This is ensured by the RPC > framework. You could take a look at RpcEndpoint for more details. > > Regards, > Dian > > > 在 2019年9月6日，下午2:35，shimin yang

Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

Hi Shimin, It can be guaranteed to be an atomic operation. This is ensured by the RPC framework. You could take a look at RpcEndpoint for more details. Regards, Dian > 在 2019年9月6日，下午2:35，shimin yang 写道： > > Hi Fu, > > Thank you for the remind. I think it would work in my case as long as it's