Re: [DISCUSS] Release flink-shaded 9.0

2019-11-19 Thread Dian Fu
Hi Chesnay, Thanks a lot for kicking off this release. +1 to release flink-shaded 9.0. I'm willing to help on the release. Please feel free to let me know if there is anything I could help. Regards, Dian On Mon, Nov 18, 2019 at 8:43 PM Ufuk Celebi wrote: > @Chesnay: I know you said that you

[DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

2019-11-13 Thread Dian Fu
Hi all, I'm reaching out to see if there is an existing security specific mailing list in Flink. If there is, we should expose it in the offcial web site of Flink [1] to guide people to report security issues to this mailing list. If it still doesn't exist, I'm here to propose to setup a

[jira] [Created] (FLINK-14760) Documentation links check failed on travis

2019-11-13 Thread Dian Fu (Jira)
Dian Fu created FLINK-14760: --- Summary: Documentation links check failed on travis Key: FLINK-14760 URL: https://issues.apache.org/jira/browse/FLINK-14760 Project: Flink Issue Type: Bug

Re: [DISCUSS] PyFlink User-Defined Function Resource Management

2019-11-11 Thread Dian Fu
k UDF. So, It's great > if anyone can add more comments or inputs in the design doc or feedback in > ML. :) > > Best, > Jincheng > > Dian Fu 于2019年11月5日周二 上午11:32写道: > >> Hi everyone, >> >> In FLIP-58[1] it will add the support of Python user-def

Re: [DISCUSS] Releasing Flink 1.8.3

2019-11-08 Thread Dian Fu
Hi Jincheng, Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3. Regards, Dian On Sat, Nov 9, 2019 at 12:11 PM jincheng sun wrote: > Hi Flink devs, > > It has been more than 2 months since the 1.8.2 released. So, What do you > think about releasing Flink 1.8.3 soon? > > We

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

2019-11-08 Thread Dian Fu
Hi Jark, Congrats. Well deserved! Regards, Dian > 在 2019年11月8日,下午5:51,jincheng sun 写道: > > Hi all, > > On behalf of the Flink PMC, I'm happy to announce that Jark Wu is now > part of the Apache Flink Project Management Committee (PMC). > > Jark has been a committer since February 2017. He

Re: How long is the flink sql task state default ttl?

2019-11-07 Thread Dian Fu
It's disabled by default. BTW: You only need to send it to user ML and it's not necessary to send it to the dev ML. > 在 2019年11月7日,下午3:36,LakeShen 写道: > > Hi community, as I know I can use idle state retention time to clear the > flink sql task state,I have a question is that how long the

[DISCUSS] PyFlink User-Defined Function Resource Management

2019-11-04 Thread Dian Fu
Hi everyone, In FLIP-58[1] it will add the support of Python user-defined stateless function for Python Table API. It will launch a separate Python process for Python user-defined function execution. The resources used by the Python process should be managed properly by Flink’s resource

Re: [DISCUSS] How to prepare the Python environment of PyFlink release in the current Flink release process

2019-11-03 Thread Dian Fu
lease. Shouldn't that be done when user start to use pyflink ? >>>> Or do you mean to set up python environment for pyflink's CI build ? >>>> >>>> Regarding this problem "It needs a proper Python environment(i.e. >> Python >>>> 3.5+, setuptoo

[jira] [Created] (FLINK-14581) Support to run Python UDF jobs in a YARN cluster

2019-10-31 Thread Dian Fu (Jira)
Dian Fu created FLINK-14581: --- Summary: Support to run Python UDF jobs in a YARN cluster Key: FLINK-14581 URL: https://issues.apache.org/jira/browse/FLINK-14581 Project: Flink Issue Type: Sub-task

[DISCUSS] How to prepare the Python environment of PyFlink release in the current Flink release process

2019-10-28 Thread Dian Fu
Hi all, We have reached a consensus that the PyFlink package should be published to PyPI in [1]. Thanks to Jincheng's effort, the PyPI account has already been created and available to use now [2]. It means that we could publish PyFlink to PyPI in the coming releases and it also means that

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

2019-10-28 Thread Dian Fu
Congrats, Becket. > 在 2019年10月28日,下午6:07,Fabian Hueske 写道: > > Hi everyone, > > I'm happy to announce that Becket Qin has joined the Flink PMC. > Let's congratulate and welcome Becket as a new member of the Flink PMC! > > Cheers, > Fabian

[jira] [Created] (FLINK-14509) Improve the README.md in pyflink to prepare for PyPI release

2019-10-23 Thread Dian Fu (Jira)
Dian Fu created FLINK-14509: --- Summary: Improve the README.md in pyflink to prepare for PyPI release Key: FLINK-14509 URL: https://issues.apache.org/jira/browse/FLINK-14509 Project: Flink Issue

[jira] [Created] (FLINK-14506) Improve the release script for Python API release package

2019-10-23 Thread Dian Fu (Jira)
Dian Fu created FLINK-14506: --- Summary: Improve the release script for Python API release package Key: FLINK-14506 URL: https://issues.apache.org/jira/browse/FLINK-14506 Project: Flink Issue Type

[jira] [Created] (FLINK-14500) Support Flink Python User-Defined Stateless Function for Table(Phase 2)

2019-10-23 Thread Dian Fu (Jira)
Dian Fu created FLINK-14500: --- Summary: Support Flink Python User-Defined Stateless Function for Table(Phase 2) Key: FLINK-14500 URL: https://issues.apache.org/jira/browse/FLINK-14500 Project: Flink

Re: [VOTE] Accept Stateful Functions into Apache Flink

2019-10-21 Thread Dian Fu
+1 (non-binding) Regards, Dian > 在 2019年10月22日,上午9:10,Kurt Young 写道: > > +1 (binding) > > Best, > Kurt > > > On Tue, Oct 22, 2019 at 12:56 AM Fabian Hueske wrote: > >> +1 (binding) >> >> Am Mo., 21. Okt. 2019 um 16:18 Uhr schrieb Thomas Weise : >> >>> +1 (binding) >>> >>> >>> On Mon,

[jira] [Created] (FLINK-14469) Drop Python 2 support

2019-10-20 Thread Dian Fu (Jira)
Dian Fu created FLINK-14469: --- Summary: Drop Python 2 support Key: FLINK-14469 URL: https://issues.apache.org/jira/browse/FLINK-14469 Project: Flink Issue Type: Task Components: API

Re: [VOTE] Drop Python 2 support for 1.10

2019-10-18 Thread Dian Fu
日,下午3:07,vino yang 写道: > > +1 > > Bowen Li 于2019年10月16日周三 上午5:12写道: > >> +1 >> >> On Sun, Oct 13, 2019 at 10:54 PM Hequn Cheng wrote: >> >>> +1 >>> >>> Thanks a lot for driving this, Dian! >>> >>> On Mon, Oct

Re: [PROPOSAL] Contribute Stateful Functions to Apache Flink

2019-10-14 Thread Dian Fu
Hi Stephan, Big +1 for adding stateful functions to Apache Flink! The use cases unlocked with this feature are very interesting and promising. Regarding to whether to place it into Flink core repository, personally I perfer to put it in the main repository. This feature introduces a new set of

[jira] [Created] (FLINK-14388) Support all the data types in Python user-defined functions

2019-10-14 Thread Dian Fu (Jira)
Dian Fu created FLINK-14388: --- Summary: Support all the data types in Python user-defined functions Key: FLINK-14388 URL: https://issues.apache.org/jira/browse/FLINK-14388 Project: Flink Issue

Re: [VOTE] FLIP-78: Flink Python UDF Environment and Dependency Management

2019-10-14 Thread Dian Fu
Hi Wei, +1 (non-binding). Thanks for driving this. Thanks, Dian > 在 2019年10月14日,下午1:40,jincheng sun 写道: > > +1 > > Wei Zhong 于2019年10月12日周六 下午8:41写道: > >> Hi all, >> >> I would like to start the vote for FLIP-78[1] which is discussed and >> reached consensus in the discussion thread[2].

Re: [DISCUSS] Drop Python 2 support for 1.10

2019-10-13 Thread Dian Fu
9年10月10日周四 下午8:07写道: > >> I also heard from other companies that upgrading to Python 3 is in >> progress for data teams. >> >> +1 for simplifying the code base with option 1). >> >> Thanks, >> Timo >> >> On 08.10.19 16:34, Dian Fu wrote:

[VOTE] Drop Python 2 support for 1.10

2019-10-13 Thread Dian Fu
Hi all, I would like to start the vote for "Drop Python 2 support for 1.10", which is discussed and reached a consensus in the discussion thread[1]. The vote will be open for at least 72 hours. Unless there is an objection, I will try to close it by Oct 17, 2019 18:00 UTC if we have received

Re: [DISCUSS] Flink Python UDF Environment and Dependency Management

2019-10-11 Thread Dian Fu
nment_variable" > to set the python executable file path. > - Making the argument "requirements_cached_dir" of API > "set_python_requirements" optional to support only upload a requirement.txt > file. > > I'm also glad to hear any other opinions! > >

Re: [DISCUSS] Drop Python 2 support for 1.10

2019-10-09 Thread Dian Fu
ode in Flink job and more requirements >>> for >>>> the features of the Python language.So I think It's better to follow >> the >>>> rhythm of Python official. >>>> >>>> Option 2 is the most conservative and correct approach, but for the

[DISCUSS] Drop Python 2 support for 1.10

2019-10-08 Thread Dian Fu
Hi everyone, I would like to propose to drop Python 2 support(Currently Python 2.7, 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are planning to drop Python 2 support. The benefits of dropping Python 2

[jira] [Created] (FLINK-14342) Remove method FunctionDefinition#getLanguage

2019-10-07 Thread Dian Fu (Jira)
Dian Fu created FLINK-14342: --- Summary: Remove method FunctionDefinition#getLanguage Key: FLINK-14342 URL: https://issues.apache.org/jira/browse/FLINK-14342 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Releasing Flink 1.9.1

2019-09-29 Thread Dian Fu
; > >>>> next > > > >>>> > >> week. > > > >>>> > >> > > > > >>>> > >> > Best, > > > >>>> > >> > Jark > > > >>>> > >> &

[jira] [Created] (FLINK-14272) Support Blink planner for Python UDF

2019-09-29 Thread Dian Fu (Jira)
Dian Fu created FLINK-14272: --- Summary: Support Blink planner for Python UDF Key: FLINK-14272 URL: https://issues.apache.org/jira/browse/FLINK-14272 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14269) The close method does not work for UserDefinedFunction

2019-09-29 Thread Dian Fu (Jira)
Dian Fu created FLINK-14269: --- Summary: The close method does not work for UserDefinedFunction Key: FLINK-14269 URL: https://issues.apache.org/jira/browse/FLINK-14269 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-14240) The configuration added via TableConfig#configuration should be added to GlobalJobParameters for the legacy planner

2019-09-26 Thread Dian Fu (Jira)
Dian Fu created FLINK-14240: --- Summary: The configuration added via TableConfig#configuration should be added to GlobalJobParameters for the legacy planner Key: FLINK-14240 URL: https://issues.apache.org/jira/browse

Re: [DISCUSS] Flink Python UDF Environment and Dependency Management

2019-09-26 Thread Dian Fu
Hi Wei, Thanks a lot for bringing up this discussion. Python dependency management is very important for Python users. I have left a few comments on the design doc. Thanks, Dian > 在 2019年9月26日,下午12:23,jincheng sun 写道: > > Thanks for bring up the discussion, Wei. > Overall the design doc

[jira] [Created] (FLINK-14222) Optimize for Python UDFs with all parameters are constant values

2019-09-25 Thread Dian Fu (Jira)
Dian Fu created FLINK-14222: --- Summary: Optimize for Python UDFs with all parameters are constant values Key: FLINK-14222 URL: https://issues.apache.org/jira/browse/FLINK-14222 Project: Flink

[jira] [Created] (FLINK-14212) Support Python UDFs without arguments

2019-09-25 Thread Dian Fu (Jira)
Dian Fu created FLINK-14212: --- Summary: Support Python UDFs without arguments Key: FLINK-14212 URL: https://issues.apache.org/jira/browse/FLINK-14212 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14208) Support Python UDF with parameters of constant values

2019-09-25 Thread Dian Fu (Jira)
Dian Fu created FLINK-14208: --- Summary: Support Python UDF with parameters of constant values Key: FLINK-14208 URL: https://issues.apache.org/jira/browse/FLINK-14208 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-14202) Optimize the execution plan for Python Calc when there is a condition

2019-09-25 Thread Dian Fu (Jira)
Dian Fu created FLINK-14202: --- Summary: Optimize the execution plan for Python Calc when there is a condition Key: FLINK-14202 URL: https://issues.apache.org/jira/browse/FLINK-14202 Project: Flink

Re: [DISCUSS] Releasing Flink 1.9.1

2019-09-23 Thread Dian Fu
+1 for 1.9.1 release and Jark being the RM. Thanks Jark for kicking off this release and the volunteering. Regards, Dian > 在 2019年9月24日,上午10:45,Kurt Young 写道: > > +1 for the 1.9.1 release and for Jark being the RM. > Thanks Jark for the volunteering. > > Best, > Kurt > > > On Mon, Sep 23,

Re: use of org.apache.flink.yarn.cli.FlinkYarnSessionCli in Flink Sql client

2019-09-22 Thread Dian Fu
Hi Dipanjan, I think you are right that it's already been supported to submit a job to cluster via SQL client. This is supported in [1]. Besides, I think that it is not configured in the YAML. It's specified in the CLI options when you start up the SQL client CLI. [1]

Re: How to implement grouping set in stream

2019-09-19 Thread Dian Fu
AFAIK, grouping sets has already been supported for streaming in blink planner. You could check FLINK-12192 for details. Regards, Dian > 在 2019年9月10日,下午6:51,刘建刚 写道: > > I want to implement grouping set in stream. I am new to flink sql. I > want to find a example to teach me how to self

Re: [ANNOUNCE] Apache Flink 1.8.2 released

2019-09-14 Thread Dian Fu
Thanks Jark for being the release manager and the great work! Also thanks to everyone who has contributed to this release. Regards, Dian > 在 2019年9月14日,上午3:24,Till Rohrmann 写道: > > Thanks Jark for being our release manager and thanks to everyone who has > contributed. > > Cheers, > Till >

Re: Flink stream python

2019-09-13 Thread Dian Fu
Hi Milan, Hoping Till has answered your questions. WRT whether it supports to use libraries from Java code in Python, the answer is yes. Regarding to use Java connectors in Python, there are two ways: 1) If there is a Java TableFactory[1] defined for the connector, you could try to use the

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

2019-09-11 Thread Dian Fu
Congratulations! > 在 2019年9月11日,下午5:26,Jeff Zhang 写道: > > Congratulations Zili Chen! > > Wesley Peng mailto:wes...@thepeng.eu>> 于2019年9月11日周三 > 下午5:25写道: > Hi > > on 2019/9/11 17:22, Till Rohrmann wrote: > > I'm very happy to announce that Zili Chen (some of you might also know > > him as

Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

2019-09-09 Thread Dian Fu
accumulators are not fault-tolerant? >> >> Best, >> Jingsong Lee >> >> >> -- >> From:shimin yang >> Send Time:2019年9月6日(星期五) 15:21 >> To:dev >> Subject:Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete >&g

[jira] [Created] (FLINK-14028) Support logging aggregation in Python user-defined functions

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14028: --- Summary: Support logging aggregation in Python user-defined functions Key: FLINK-14028 URL: https://issues.apache.org/jira/browse/FLINK-14028 Project: Flink Issue

[jira] [Created] (FLINK-14027) Add documentation for Python user-defined functions

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14027: --- Summary: Add documentation for Python user-defined functions Key: FLINK-14027 URL: https://issues.apache.org/jira/browse/FLINK-14027 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-14026) Manage the resource of Python worker properly

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14026: --- Summary: Manage the resource of Python worker properly Key: FLINK-14026 URL: https://issues.apache.org/jira/browse/FLINK-14026 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14025) Support to run the Python worker in docker mode

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14025: --- Summary: Support to run the Python worker in docker mode Key: FLINK-14025 URL: https://issues.apache.org/jira/browse/FLINK-14025 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14024) Support use-defined metrics in Python user-defined functions

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14024: --- Summary: Support use-defined metrics in Python user-defined functions Key: FLINK-14024 URL: https://issues.apache.org/jira/browse/FLINK-14024 Project: Flink Issue

[jira] [Created] (FLINK-14023) Support accessing job parameters in Python user-defined functions

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14023: --- Summary: Support accessing job parameters in Python user-defined functions Key: FLINK-14023 URL: https://issues.apache.org/jira/browse/FLINK-14023 Project: Flink

[jira] [Created] (FLINK-14022) Add validation check for places where Python ScalarFunction cannot be used

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14022: --- Summary: Add validation check for places where Python ScalarFunction cannot be used Key: FLINK-14022 URL: https://issues.apache.org/jira/browse/FLINK-14022 Project: Flink

[jira] [Created] (FLINK-14021) Add rules to push down the Python ScalarFunctions contained in the join condition of Correlate node

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14021: --- Summary: Add rules to push down the Python ScalarFunctions contained in the join condition of Correlate node Key: FLINK-14021 URL: https://issues.apache.org/jira/browse/FLINK-14021

[jira] [Created] (FLINK-14020) User Apache Arrow as the serializer for data transmission between Java operator and Python harness

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14020: --- Summary: User Apache Arrow as the serializer for data transmission between Java operator and Python harness Key: FLINK-14020 URL: https://issues.apache.org/jira/browse/FLINK-14020

[jira] [Created] (FLINK-14019) Python environment and dependency management

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14019: --- Summary: Python environment and dependency management Key: FLINK-14019 URL: https://issues.apache.org/jira/browse/FLINK-14019 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14018) Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14018: --- Summary: Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work Key: FLINK-14018 URL: https://issues.apache.org/jira/browse/FLINK-14018

[jira] [Created] (FLINK-14017) Support to start up Python worker in process mode

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14017: --- Summary: Support to start up Python worker in process mode Key: FLINK-14017 URL: https://issues.apache.org/jira/browse/FLINK-14017 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14016) Introduce RelNodes FlinkLogicalPythonScalarFunctionExec and DataStreamPythonScalarFunctionExec which are containers for Python PythonScalarFunctions

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14016: --- Summary: Introduce RelNodes FlinkLogicalPythonScalarFunctionExec and DataStreamPythonScalarFunctionExec which are containers for Python PythonScalarFunctions Key: FLINK-14016 URL: https

[jira] [Created] (FLINK-14015) Introduce PythonScalarFunctionOperator as a standalone StreamOperator for Python ScalarFunction execution

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14015: --- Summary: Introduce PythonScalarFunctionOperator as a standalone StreamOperator for Python ScalarFunction execution Key: FLINK-14015 URL: https://issues.apache.org/jira/browse/FLINK-14015

[jira] [Created] (FLINK-14014) Introduce PythonScalarFunctionRunner to handle the communication with Python worker for Python ScalarFunction execution

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14014: --- Summary: Introduce PythonScalarFunctionRunner to handle the communication with Python worker for Python ScalarFunction execution Key: FLINK-14014 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-14013) Support Flink Python User-Defined Stateless Function for Table

2019-09-09 Thread Dian Fu (Jira)
Dian Fu created FLINK-14013: --- Summary: Support Flink Python User-Defined Stateless Function for Table Key: FLINK-14013 URL: https://issues.apache.org/jira/browse/FLINK-14013 Project: Flink Issue

Re: [VOTE] FLIP-58: Flink Python User-Defined Function for Table API

2019-09-09 Thread Dian Fu
i Dian, >>>>>>>> >>>>>>>> +1, >>>>>>>> Thanks a lot for driving this. >>>>>>>> >>>>>>>> Best, >>>>>>>> Xingbo >>>>>>>>> 在 2019年8

Re: [VOTE] Release 1.8.2, release candidate #1

2019-09-09 Thread Dian Fu
+1 (non-binding) - built from source successfully (mvn clean install -DskipTests) - checked gpg signature and hashes of the source release and binary release packages - All artifacts have been deployed to the maven central repository - no new dependencies were added since 1.8.1 - run a couple of

[jira] [Created] (FLINK-13999) Correct the documentation of MATCH_RECOGNIZE

2019-09-06 Thread Dian Fu (Jira)
Dian Fu created FLINK-13999: --- Summary: Correct the documentation of MATCH_RECOGNIZE Key: FLINK-13999 URL: https://issues.apache.org/jira/browse/FLINK-13999 Project: Flink Issue Type: Bug

Re: [DISCUSS] Features for Apache Flink 1.10

2019-09-06 Thread Dian Fu
Hi Gary, Thanks for kicking off the release schedule of 1.10. +1 for you and Yu Li as the release manager. The feature freeze/release time sounds reasonable. Thanks, Dian > 在 2019年9月7日,上午11:30,Jark Wu 写道: > > Thanks Gary for kicking off the discussion for 1.10 release. > > +1 for Gary and

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

2019-09-06 Thread Dian Fu
Congratulations Kostas! Regards, Dian > 在 2019年9月6日,下午8:58,Wesley Peng 写道: > > On 2019/9/6 8:55 下午, Fabian Hueske wrote: >> I'm very happy to announce that Kostas Kloudas is joining the Flink PMC. >> Kostas is contributing to Flink for many years and puts lots of effort in >> helping our

Re: Checkpointing clarification

2019-09-06 Thread Dian Fu
When a WindowOperator receives all the barrier from the upstream, it will forward the barrier to downstream operator and perform the checkpoint asynchronously. It doesn't have to wait the window to trigger before sending out the barrier. Regards, Dian > 在 2019年9月6日,下午8:02,Dominik Wosiński

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-09-06 Thread Dian Fu
Hi all, I have updated the FLIP and removed content relate to UDAF and also changed the title of the FLIP to "Flink Python User-Defined Stateless Function for Table". Does it make sense to you? Regards, Dian > 在 2019年9月6日,下午6:09,Dian Fu 写道: > > Hi all, > > Thanks

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-09-06 Thread Dian Fu
>>>>>>>>>> 1. DataViews: With the current non-annotation design for >> DataViews, >>>>>> we >>>>>>>>>>> cannot perform eager state declaration, right? At which point >>>> during >>>>>>>

Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

2019-09-06 Thread Dian Fu
my case as long as it's > an atomic operation. > > Dian Fu 于2019年9月6日周五 下午2:22写道: > >> Hi Jingsong, >> >> Thanks for bring up this discussion. You can try to look at the >> GlobalAggregateManager to see if it can meet your requirements. It can be >>

Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

2019-09-06 Thread Dian Fu
Hi Jingsong, Thanks for bring up this discussion. You can try to look at the GlobalAggregateManager to see if it can meet your requirements. It can be got via StreamingRuntimeContext#getGlobalAggregateManager(). Regards, Dian > 在 2019年9月6日,下午1:39,shimin yang 写道: > > Hi Jingsong, > > Big

Re: [DISCUSS] FLIP-66: Support time attribute in SQL DDL

2019-09-05 Thread Dian Fu
Hi Jark, Thanks for bringing up this discussion and the detailed design doc. This is definitely a critical feature for streaming SQL jobs. I have left a few comments in the design doc. Thanks, Dian > 在 2019年9月6日,上午11:48,Forward Xu 写道: > > Thanks Jark for this topic, This will be very

Re: State of FLIPs

2019-08-30 Thread Dian Fu
Hi Chesnay, Thanks a lot for the remind. FLIP-38 has been released in 1.9 and I have updated the status in the wiki page. Regards, Dian On Fri, Aug 30, 2019 at 9:38 PM Becket Qin wrote: > Hi Chesnay, > > You are right. FLIP-36 actually has not passed the vote yet. In fact some > of the key

Re: [DISCUSS] Releasing Flink 1.8.2

2019-08-30 Thread Dian Fu
Hi Jincheng, Thanks a lot for bring up this discussion. +1 for this release. Regards, Dian > 在 2019年8月30日,下午6:31,Maximilian Michels 写道: > > Hi Jincheng, > > +1 I would be for a 1.8.2 release such that we can fix the problems with the > nested closure cleaner which currently block 1.8.1

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-27 Thread Dian Fu
> > > On Mon, Aug 26, 2019 at 8:05 PM Dian Fu wrote: > >> Hi Jincheng, >> >> Thanks! It works. >> >> Thanks, >> Dian >> >>> 在 2019年8月27日,上午10:55,jincheng sun 写道: >>> >>> Hi Dian, can you check if you

[VOTE] FLIP-58: Flink Python User-Defined Function for Table API

2019-08-27 Thread Dian Fu
Hi all, I'd like to start a voting thread for FLIP-58 [1] since that we have reached an agreement on the design in the discussion thread [2], This vote will be open for at least 72 hours. Unless there is an objection, I will try to close it by Sept 2, 2019 00:00 UTC if we have received

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-26 Thread Dian Fu
Hi Jincheng, Thanks! It works. Thanks, Dian > 在 2019年8月27日,上午10:55,jincheng sun 写道: > > Hi Dian, can you check if you have edit access? :) > > > Dian Fu 于2019年8月26日周一 上午10:52写道: > >> Hi Jincheng, >> >> Appreciated for the kind tips and offering of

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-25 Thread Dian Fu
Hi Jincheng, Appreciated for the kind tips and offering of help. Definitely need it! Could you grant me write permission for confluence? My Id: Dian Fu Thanks, Dian > 在 2019年8月26日,上午9:53,jincheng sun 写道: > > Thanks for your feedback Hequn & Dian. > > Dian, I am glad t

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-22 Thread Dian Fu
Great news! Thanks Gordon and Kurt for pushing this forward and everybody who contributed to this release. Regards, Dian > 在 2019年8月23日,上午9:41,Guowei Ma 写道: > > Congratulations!! > Best, > Guowei > > > Congxian Qiu mailto:qcx978132...@gmail.com>> > 于2019年8月23日周五 上午9:32写道: >

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-22 Thread Dian Fu
comments if >> there are anything not describe clearly. >> >> Best, >> Jincheng >> >> >> Dian Fu 于2019年8月14日周三 上午10:08写道: >> >>> Hi Thomas, >>> >>> Thanks a lot the suggestions. >>> >>> Regarding to bundle processi

Re: CiBot Update

2019-08-22 Thread Dian Fu
Thanks Chesnay for your great work! A very useful feature! Just one minor suggestion: It will be better if we could add this command to the section "Bot commands" in the flinkbot template. Regards, Dian > 在 2019年8月23日,上午2:06,Ethan Li 写道: > > My question is specifically about implementation

[jira] [Created] (FLINK-13743) Port PythonTableUtils to flink-python module

2019-08-16 Thread Dian Fu (JIRA)
Dian Fu created FLINK-13743: --- Summary: Port PythonTableUtils to flink-python module Key: FLINK-13743 URL: https://issues.apache.org/jira/browse/FLINK-13743 Project: Flink Issue Type: Task

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-13 Thread Dian Fu
Python users, we'd like to start the >> discussion about the Python UDF support in the Python Table API. >> Aljoscha Krettek, Dian Fu and I have discussed offline and have drafted a >> design doc[1]. It includes the following items: >> >> - The user-defined function

Re: [ANNOUNCE] Hequn becomes a Flink committer

2019-08-07 Thread Dian Fu
Congratulations, Hequn! Well deserved! > 在 2019年8月7日,下午5:13,Jeff Zhang 写道: > > Congrats Hequn! > > Paul Lam mailto:paullin3...@gmail.com>> 于2019年8月7日周三 > 下午5:08写道: > Congrats Hequn! Well deserved! > > Best, > Paul Lam > >> 在 2019年8月7日,16:28,jincheng sun > >

Re: [VOTE] Publish the PyFlink into PyPI

2019-08-01 Thread Dian Fu
Hi Jincheng, Thanks a lot for driving this. +1 (non-binding). Regards, Dian > 在 2019年8月1日,下午3:24,jincheng sun 写道: > > Hi all, > > Publish the PyFlink into PyPI is very important for our user, Please vote > on the following proposal: > > 1. Create PyPI Project for Apache Flink Python API,

Re: [DISCUSS] Publish the PyFlink into PyPI

2019-07-23 Thread Dian Fu
ck and suggestions! >>> >>> Best, >>> Jincheng >>> >>> >>> >>> >>> Chesnay Schepler 于2019年7月3日周三 下午8:46写道: >>> >>>> So this would not be a source release then, but a full-blown binary >>>>

Re: [ANNOUNCE] Kete Young is now part of the Flink PMC

2019-07-23 Thread Dian Fu
Congrats, Kurt! > 在 2019年7月23日,下午5:33,Zili Chen 写道: > > Congratulations Kurt! > > Best, > tison. > > > JingsongLee 于2019年7月23日周二 下午5:29写道: > >> Congratulations Kurt! >> >> Best, Jingsong Lee >> >> >> -- >> From:Robert

Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project

2019-07-22 Thread Dian Fu
Congrats Zhijiang! > 在 2019年7月23日,上午9:14,Kurt Young 写道: > > Congratulations Zhijiang! > > Best, > Kurt > > > On Tue, Jul 23, 2019 at 8:59 AM Biao Liu wrote: > >> Congrats Zhijiang. Well deserved! >> >> SHI Xiaogang 于2019年7月23日 周二08:35写道: >> >>> Congratulations Zhijiang! >>> >>> Regards,

Re: [DISCUSS] Setup a bui...@flink.apache.org mailing list for travis builds

2019-07-20 Thread Dian Fu
Thanks Jark for the proposal, sounds reasonable for me. +1. This ML could be used for all the build notifications including master and CRON jobs. > 在 2019年7月20日,下午2:55,Xu Forward 写道: > > +1 ,Thanks jark for the proposal. > Best > Forward > > Jark Wu 于2019年7月20日周六 下午12:14写道: > >> Hi all, >>

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

2019-07-18 Thread Dian Fu
Congrats Becket! > 在 2019年7月18日,下午6:42,Danny Chan 写道: > >> Congratulations! > > Best, > Danny Chan > 在 2019年7月18日 +0800 PM6:29,Haibo Sun ,写道: >> Congratulations Becket!Best, >> Haibo >> 在 2019-07-18 17:51:06,"Hequn Cheng" 写道: >>> Congratulations Becket! >>> >>> Best, Hequn >>> >>> On Thu,

Re: flink-python failed on Travis

2019-07-17 Thread Dian Fu
; > Hi Dian, > > Is there any update on this? It seems have been failing for a day. > > > > On Tue, Jul 16, 2019 at 9:35 PM Dian Fu wrote: > >> Thanks for reporting this issue. I will take a look at it. >> >>> 在 2019年7月17日,上午11:50,Danny Chan 写道: &

Re: flink-python failed on Travis

2019-07-16 Thread Dian Fu
Thanks for reporting this issue. I will take a look at it. > 在 2019年7月17日,上午11:50,Danny Chan 写道: > > I have the same issue ~~ > > Best, > Danny Chan > 在 2019年7月17日 +0800 AM11:21,Haibo Sun ,写道: >> Hi, folks >> >> >> I noticed that all of the Travis tests reported the following failure. Is >>

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

2019-07-11 Thread Dian Fu
Congrats Rong! > 在 2019年7月12日,上午8:47,Chen YuZhao 写道: > > congratulations! > > 获取 Outlook for iOS > > 发件人: Rong Rong > 发送时间: 星期五, 七月 12, 2019 8:09 上午 > 收件人: Hao Sun > 抄送: Xuefu Z; dev; Flink ML > 主题: Re: [ANNOUNCE] Rong Rong becomes a Flink committer > > Thank you

Re: [DISCUSS] ARM support for Flink

2019-07-07 Thread Dian Fu
Hi Xiyuan, Thanks for bring the discussion. WRT the exception, it's because the native bundled in the rocksdb jar file isn't compiled with cross platform support. You can refer [1] for how to build rocksdb which has ARM platform. WRT ARM support, the rocksdb currently used in Flink is hosted

Re: [VOTE] Migrate to sponsored Travis account

2019-07-04 Thread Dian Fu
+1. Thanks Chesnay and Bowen for pushing this forward. Regards, Dian > 在 2019年7月4日,下午6:28,zhijiang 写道: > > +1 and thanks for Chesnay' work on this. > > Best, > Zhijiang > > -- > From:Haibo Sun > Send Time:2019年7月4日(星期四) 18:21 >

Re: [DISCUSS] Publish the PyFlink into PyPI

2019-07-03 Thread Dian Fu
Hi Chesnay, Thanks a lot for the suggestions. Regarding “distributing java/scala code to PyPI”: The Python Table API is just a wrapper of the Java Table API and without the java/scala code, two steps will be needed to set up an environment to execute a Python Table API program: 1) Install

Re: [ANNOUNCE] Apache Flink 1.8.1 released

2019-07-02 Thread Dian Fu
Awesome! Thanks a lot for being the release manager. Great job! @Jincheng Regards, Dian > 在 2019年7月3日,上午10:08,jincheng sun 写道: > > I've also tweeted about it from my twitter: > https://twitter.com/sunjincheng121/status/1146236834344648704 >

[jira] [Created] (FLINK-12991) Correct the implementation of Catalog.get_table_factory

2019-06-26 Thread Dian Fu (JIRA)
Dian Fu created FLINK-12991: --- Summary: Correct the implementation of Catalog.get_table_factory Key: FLINK-12991 URL: https://issues.apache.org/jira/browse/FLINK-12991 Project: Flink Issue Type

[jira] [Created] (FLINK-12990) Date type doesn't consider the local TimeZone

2019-06-25 Thread Dian Fu (JIRA)
Dian Fu created FLINK-12990: --- Summary: Date type doesn't consider the local TimeZone Key: FLINK-12990 URL: https://issues.apache.org/jira/browse/FLINK-12990 Project: Flink Issue Type: Bug

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

2019-06-24 Thread Dian Fu
Congratulations Jincheng! On Mon, Jun 24, 2019 at 11:09 PM Robert Metzger wrote: > Hi all, > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is now > part of the Apache Flink Project Management Committee (PMC). > > Jincheng has been a committer since July 2017. He has been

[jira] [Created] (FLINK-12962) Allows pyflink to be pip installed

2019-06-24 Thread Dian Fu (JIRA)
Dian Fu created FLINK-12962: --- Summary: Allows pyflink to be pip installed Key: FLINK-12962 URL: https://issues.apache.org/jira/browse/FLINK-12962 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-12920) Drop support for register_table_sink with field_names and field_types parameters

2019-06-20 Thread Dian Fu (JIRA)
Dian Fu created FLINK-12920: --- Summary: Drop support for register_table_sink with field_names and field_types parameters Key: FLINK-12920 URL: https://issues.apache.org/jira/browse/FLINK-12920 Project

<    2   3   4   5   6   7   8   >