Re: [VOTE] FLIP-58: Flink Python User-Defined Function for Table API

+1 (binding) This will be a great feature for Flink users, especially for the data science and AI engineers. Regards, Shaoxuan On Fri, Aug 30, 2019 at 1:35 PM Jeff Zhang wrote: > +1， very looking forward this feature in flink 1.10 > > > Yu Li 于2019年8月30日周五 上午11:08写道： > > > +1 (non-binding)

Re: [RESULT] [VOTE] Apache Flink 1.9.0, release candidate #3

Congratulations and thanks all for the great efforts on release1.9. I have verified the RC#3 with the following items: - Verified signatures and hashes. (OK) - Built from source archive. (OK) - Repository contains all artifacts. (OK) - Test WordCount on local cluster. (OK) a. Both streaming

Re: [VOTE] Flink Project Bylaws

+1 (binding) On Fri, Aug 16, 2019 at 7:48 PM Chesnay Schepler wrote: > +1 (binding) > > Although I think it would be a good idea to always cc > priv...@flink.apache.org when modifying bylaws, if anything to speed up > the voting process. > > On 16/08/2019 11:26, Ufuk Celebi wrote: > > +1

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

Congrats Becket! It is really great to have you on board. Regards, Shaoxuan On Fri, Jul 19, 2019 at 7:24 AM Becket Qin wrote: > Thanks everyone! > > It's really a pleasure working with all of you :) > > On Fri, Jul 19, 2019 at 3:48 AM Bowen Li wrote: > > > Congrats, Jiangjie! > > > > On Thu,

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

Congratulations Rong! On Thu, Jul 11, 2019 at 10:40 PM Jark Wu wrote: > Congratulations Rong Rong! > Welcome on board! > > On Thu, 11 Jul 2019 at 22:25, Fabian Hueske wrote: > >> Hi everyone, >> >> I'm very happy to announce that Rong Rong accepted the offer of the Flink >> PMC to become a

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations, Jincheng! On Tue, Jun 25, 2019 at 5:54 PM Fabian Hueske wrote: > Congrats Jincheng! > > Am Di., 25. Juni 2019 um 11:48 Uhr schrieb Aljoscha Krettek < > aljos...@apache.org>: > > > Congratulations! :-) > > > > > On 25. Jun 2019, at 11:34, Wei Zhong wrote: > > > > > >

Re: [DISCUSS] FLIP-39: Flink ML pipeline and ML libs

nd > the old one was also inspired by it. > > Best, > Stavros > On Wed, May 22, 2019 at 1:24 PM Shaoxuan Wang wrote: > > > Another consensus (from the offline discussion) is that we will > > delete/deprecate flink-libraries/flink-ml. I have started a survey and &g

[jira] [Created] (FLINK-12597) Remove the legacy flink-libraries/flink-ml

Shaoxuan Wang created FLINK-12597: - Summary: Remove the legacy flink-libraries/flink-ml Key: FLINK-12597 URL: https://issues.apache.org/jira/browse/FLINK-12597 Project: Flink Issue Type: Sub

Re: [SURVEY] Usage of flink-ml and [DISCUSS] Delete flink-ml

> working against @Stable APIs. > > On 22/05/2019 12:08, Shaoxuan Wang wrote: > > Hi Flink community, > > > > We plan to delete/deprecate the legacy flink-libraries/flink-ml package > in > > Flink1.9, and replace it with the new flink-ml interface proposed in > FL

Re: [DISCUSS] FLIP-39: Flink ML pipeline and ML libs

Another consensus (from the offline discussion) is that we will delete/deprecate flink-libraries/flink-ml. I have started a survey and discussion [1] in dev/user-ml to collect the feedback. Depending on the replies, we will decide if we shall delete it in Flink1.9 or deprecate in the next release

[SURVEY] Usage of flink-ml and [DISCUSS] Delete flink-ml

Hi Flink community, We plan to delete/deprecate the legacy flink-libraries/flink-ml package in Flink1.9, and replace it with the new flink-ml interface proposed in FLIP39 (FLINK-12470). Before we remove this package, I want to reach out to you and ask if there is any active project still uses

Re: [DISCUSS] FLIP-39: Flink ML pipeline and ML libs

ad and the PR on FLIP-23 with you and Boris :-) > > -- > Rong > > On Mon, May 6, 2019 at 3:30 AM Shaoxuan Wang wrote: > >> >> Thanks for the feedback, Rong and Flavio. >> >> @Rong Rong >> > There's another thread regarding a close to merge FLIP-23 imp

[jira] [Created] (FLINK-12473) Add ML pipeline and ML lib Core API

Shaoxuan Wang created FLINK-12473: - Summary: Add ML pipeline and ML lib Core API Key: FLINK-12473 URL: https://issues.apache.org/jira/browse/FLINK-12473 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-12470) FLIP39: Flink ML pipeline and ML libs

Shaoxuan Wang created FLINK-12470: - Summary: FLIP39: Flink ML pipeline and ML libs Key: FLINK-12470 URL: https://issues.apache.org/jira/browse/FLINK-12470 Project: Flink Issue Type: New

Re: [DISCUSS] FLIP-39: Flink ML pipeline and ML libs

tion in FLIP-23 in the future, (some > > adjustment might be needed) that would be super helpful. > > > > Best, > > Rong > > > > > > [1] > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-23-Model-Serving-td20260.

Re: [DISCUSS] FLIP-39: Flink ML pipeline and ML libs

led > > implementation plan to FLIP and google doc. > > > > What do you think? > > > > Best, > > Jincheng > > > > Shaoxuan Wang 于2019年4月29日周一 上午10:34写道： > > > >> Hi everyone, > >> > >> Weihua has proposed to rebuild Flink

[DISCUSS] FLIP-39: Flink ML pipeline and ML libs

Hi everyone, Weihua has proposed to rebuild Flink ML pipeline on top of TableAPI several months ago in this mail thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Embracing-Table-API-in-Flink-ML-td25368.html Luogen, Becket, Xu, Weihua and I have been working on this

Re: [ANNOUNCE] Apache Flink 1.8.0 released

Thanks Aljoscha and all others who made contributions to FLINK 1.8.0. Looking forward to FLINK 1.9.0. Regards, Shaoxuan On Wed, Apr 10, 2019 at 4:31 PM Aljoscha Krettek wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.8.0, which is the next major

Re: [DISCUSS] FLIP-38 Support python language in flink TableAPI

David, This proposal does not contradict with what we have discussed. Please check my reply in https://lists.apache.org/thread.html/f6f8116b4b38b0b2d70ed45b990d6bb1bcb33611fde6fdf32ec0e840@%3Cdev.flink.apache.org%3E on 2019/02/21. "Beam Python API and Flink Python TableAPI describe the

Re: [VOTE] Release 1.8.0, release candidate #4

+1 (non-binding) I tested RC4 with the following items: - Maven Central Repository contains all artifacts - Built the source with Maven (ensured all source files have Apache headers), and executed built-in tests via "mvn clean verify" - Manually executed the tests in IntelliJ IDE - Verify that

Re: [DISCUSS] Reorganizing Table-related Jira components some more

+1 to restructure the Jira components. How about: Table / API Table / Client Table / Classic Planner (or Legacy Planner) Table / New Planner Regards, Shaoxuan On Thu, Mar 21, 2019 at 12:39 AM Aljoscha Krettek wrote: > Hi, > > First of all, I hope I cc’ed all the relevant people. Sorry if I

Re: [PROGRESS UPDATE] [DISCUSS] Flink-Hive Integration and Catalogs

Hi Bowen, Thanks for driving this. I am CCing this email/survey to user-zh@ flink.apache.org as well. I heard there are lots of interests on Flink-Hive from the field. One of the biggest requests the hive users are raised is "the support of out-of-date hive version". A large amount of users are

Re: [VOTE] Release 1.8.0, release candidate #2

I tested RC2 with the following items: - Maven Central Repository contains all artifacts - Built the source with Maven (ensured all source files have Apache headers) - Checked checksums and GPG files (for instance, flink-core-1.8.0.jar) that match the corresponding release files - Verified that

Re: [DISCUSS] Create a Flink ecosystem website

Hi Becket and Robert, I like this idea! Let us roll this out with Flink connectors at the first beginning. We can start with a static page, and upgrade it when we find a better solution for dynamic one with rich functions. Regards, Shaoxuan On Wed, Mar 6, 2019 at 1:36 AM Robert Metzger

Re: [DISCUSS] Python (and Non-JVM) Language Support in Flink

> > > > - SDK (API, its Protobuf serialization, and interaction with the SDK > > > Harness) > > > - Runner (Translation from Protobuf pipeline to Flink job) > > > - SDK Harness (UDF execution, Interaction with the SDK and the > execution > > > engine) >

Re: [DISCUSS] Adding a mid-term roadmap to the Flink website

Hi Stephan, Thanks for summarizing the work into a roadmap. It really helps users to understand where Flink will forward to. The entire outline looks good to me. If appropriate, I would recommend to add another two attracting categories in the roadmap. *Flink ML Enhancement* - Refactor ML

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congratulations, Thomas! On Tue, Feb 12, 2019 at 5:59 PM Fabian Hueske wrote: > Hi everyone, > > On behalf of the Flink PMC I am happy to announce Thomas Weise as a new > member of the Apache Flink PMC. > > Thomas is a long time contributor and member of our community. > He is starting and

Re: [DISCUSS] Standard / Convention for common connector metrics

Becket, Thanks for the proposal. I am much agree that we should only design the standard/common Flink-related connector metrics, while keep the original names for the ones provided by the connector itself. If everyone agrees to make these changes, we should do it asap, the earlier to make the

Re: [DISCUSS] Contributing Chinese website and docs to Apache Flink

Thanks for spending efforts to build a Flink web in Chinese, everyone. Good job, Jark! This looks look to me. I have only one concern about the synchronization of the English and Chinese website. We can not expect the author of the English doc to produce Chinese doc at the same time. If no one

Re: Flink release schedule

I agree with Ufuk that we should publish the plan of the following release on the website, including the release manager, the included features, as well as feature freezing date, such that the Flink developers can better plan their time for PR and reviews. I am also interested in the “release

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

+1. Looking forward to it. On Thu, Jan 24, 2019 at 7:41 PM Robert Metzger wrote: > Thanks for your response. > > You are right, I'm proposing "user...@flink.apache.org" as the mailing > list's name! > > On Thu, Jan 24, 2019 at 12:37 PM Tzu-Li (Gordon) Tai > wrote: > > > Hi Robert, > > > >

Re: [ANNOUNCE] Contributing Alibaba's Blink

>>>> > >> >>>>>> On Tue, 22 Jan 2019 at 18:29, Dominik Wosiński > > >> >>> wrote: > >> >>>>>>> Hey! > >> >>>>>>> I also think that creating the separate branch for Blink in &g

Re: [ANNOUNCE] Contributing Alibaba's Blink

big +1 to contribute Blink codebase directly into the Apache Flink project. Looking forward to the new journey. Regards, Shaoxuan On Tue, Jan 22, 2019 at 3:52 AM Xiaowei Jiang wrote: > Thanks Stephan! We are hoping to make the process as non-disruptive as > possible to the Flink community.

Re: [DISCUSS] Python (and Non-JVM) Language Support in Flink

RE: Stephen's options ( http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/SURVEY-Usage-of-flink-python-and-flink-streaming-python-td25793.html ) * Option (1): Language portability via Apache Beam * Option (2): Implement own Python API * Option (3): Implement own portability layer Hi

Re: [DISCUSS] Flink SQL DDL Design

> (1) period definition on table > (2) Type DDL > (3) Index DDL > (4) Library DDL > (5) Drop statement > > [1] Flink DDL draft by Lin and Jark: > > https://docs.google.com/document/d/1o16jC-AxnZoxMfHQptkKQkSC6ZDDBRhKg6gm8VGnY-k/edit# > [2] Flink SQL DDL design by Shuyi

Re: [DISCUSS] Support Interactive Programming in Flink Table API

mp. > >> > >> In any of the above two cases, during a single run of the processing > logic, > >> the data cannot change. Otherwise the behavior of the processing logic > may > >> be undefined. In the above two examples, when writing the processing > l

Re: [DISCUSS] Support Interactive Programming in Flink Table API

> Table t1 = ??? > >> Table t2 = methodThatAppliesOperators(t1); > >> Table t3 = methodThatAppliesOtherOperators(t1); > >> > >> In this case, the behavior/performance of the plan that results from the > >> second method call depends on whether t1 was

Re: [DISCUSS] Support Interactive Programming in Flink Table API

fresh(). > This sounds quite useful in an interactive session mode. > > 3) Regarding the name, I can see both arguments. IMO, materialize() seems > to be more future proof. > > Best, Fabian > > Am Do., 29. Nov. 2018 um 12:56 Uhr schrieb Shaoxuan Wang < > wshaox...@gmail.c

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Piotr, Thanks for sharing your ideas on the method naming. We will think about your suggestions. But I don't understand why we need to change the return type of cache(). Cache() is a physical operation, it does not change the logic of the `Table`. On the tableAPI layer, we should not

Re: [DISCUSS] Flink SQL DDL Design

ummarizing it into a google doc is a good idea. We will prepare > it > > in the next few days. > > > > Thanks, > > Jark > > > > Shaoxuan Wang 于2018年11月28日周三 下午9:17写道： > > > > > Hi Lin and Jark, > > > Thanks for sharing those details

Re: [DISCUSS] Flink SQL DDL Design

Hi Lin and Jark, Thanks for sharing those details. Can you please consider summarizing your DDL design into a google doc. We can still continue the discussions on Shuyi's proposal. But having a separate google doc will be easy for the DEV to understand/comment/discuss on your proposed DDL

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Xingcan, I think you probably misunderstood our proposal. The proposed “cache()” API basically infers the data is only available for its session, but not forever available for other sessions to access. It will be cleaned when the session exits. “cache” does not imply the underlying

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Xingcan, Thanks for the comments. Yes, "cache/persistent the intermediate data" is useful. It can bring benefit to many scenarios. But different scenarios may have different ways to solve it. For instance, as I replied to

Re: [DISCUSS] Table API Enhancement Outline

>>> necessary, we can extend to table.map(Expression*) in the future >> > >>>> ? >> > Of >> > >>>> course, I also hope that we can do more perfection in this >> > >>>> proposal >> > >>> t

Re: [DISCUSS] Embracing Table API in Flink ML

Hi Weihua, Thanks for the proposal. I have quickly read through it. It looks great. A quick question. Do you consider changing the ML Lib (implementation of Estimator/Predictor/Transformer) also on top of the tableAPI? I will be very happy if this is also included in the scope. It is not easy and

Re: [DISCUSS] Table API Enhancement Outline

Hi xiaowei, Yes, I agree with you that the semantics of TableAggregateFunction emit is much more complex than AggregateFunction. The fundamental difference is that TableAggregateFunction emits a "table" while AggregateFunction outputs (a column of) a "row". In the case of AggregateFunction it

Re: [DISCUSS] Enhancing the functionality and productivity of Table API

; >> SQL > >>>> API. > >>>>>> But > >>>>>>>> as from my experience Table API was always giving me the > >>> impression > >>>>> to > >>>>>>> be a > >>>>>>>&

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

gt; > > *Fabian WollertZalando SE* > > E-Mail: fab...@zalando.de > > > Am Di., 6. Nov. 2018 um 12:10 Uhr schrieb Shaoxuan Wang < > wshaox...@gmail.com > >: > > > Till, > > Yes, it is a good idea to have a feature flag to switch the web UI before > &

Re: [DISCUSS] Flink Cluster Overview Dashboard Improvement Proposal

Lining, Thanks for the proposal. There is another ongoing ML ( http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Change-underlying-Frontend-Architecture-for-Flink-Web-Dashboard-td24902.html), where YaDong has shared some sample code (

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

t; > > > > > > > Cheers > > > > > > > > -- > > > > > > > > > > > > *Fabian WollertZalando SE* > > > > > > > > E-Mail: fab...@zalando.de > > > > > > > > > > >

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

+1 to keep the web UI stack to Angular. This will make the stack upgrading much easier and won't break the on-going efforts already made towards the improvements of Flink Web UI. Till, Robert, and Fabian, Please take a look at the upgrading plan from Angular 1.x to 7.0 proposed by Yadong. Any

Re: [DISCUSS] Enhancing the functionality and productivity of Table API

Hi Aljoscha， Glad that you like the proposal. We have completed the prototype of most new proposed functionalities. Once collect the feedback from community, we will come up with a concrete FLIP/design doc. Regards, Shaoxuan On Thu, Nov 1, 2018 at 8:12 PM Aljoscha Krettek wrote: > Hi

Re: [ANNOUNCE] New committer: Sihua Zhou

Welcome and Congratulations~~ Sihua! On Fri, Jun 22, 2018 at 10:28 PM, Fabian Hueske wrote: > Congratulations! :-) > > Cheers, Fabian > > 2018-06-22 16:07 GMT+02:00 Stefan Richter : > > > Congratulations Sihua! That is great news. > > > > Best, > > Stefan > > > > > Am 22.06.2018 um 15:17

Re: [ANNOUNCE] New committer: Haohui Mai

Congratulations! On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann wrote: > Congrats and welcome on board :-) > > On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske wrote: > > > Hi everybody, > > > > On behalf of the PMC I am delighted to announce Haohui Mai as

Re: [DISCUSS] api changes to support dml operations like ‘insert into’ clause in TableAPI

+1 to support this change, as it makes the sql API more accurate and elegant. I hope this will not introduce too much troubles in the release upgrading for the existing flink SQL users. On Mon, Jul 31, 2017 at 3:42 PM, Fabian Hueske wrote: > Hi Lincoln, > > thank you for

Re: [DISCUSS] Table API / SQL internal timestamp handling

; > > > timestamp indicator of when the processing happened (for log > > > purposes, > > > > > > provenance, traceability ...). Secondly - I do not think it is > > > > > > contradicting with the semantics in batch SQL as in SQL you have > &g

Re: [ANNOUNCE] New Flink PMC member: Chesnay Schepler

Congratulations Chesnay! On Fri, Jul 28, 2017 at 10:05 PM, Greg Hogan wrote: > Developers, > > On behalf of the Flink PMC I am delighted to announce Chesnay Schepler as > a member of the Flink PMC. > > Chesnay is a longtime contributor, reviewer, and committer whose breadth

Re: [DISCUSS] Table API / SQL internal timestamp handling

Hi Everyone, I like this proposal. The problem we used to have is that we have treated eventtime column as a special timestamp column. An eventtime column is nothing special than all other regular columns, but with a certain flag (eventtime-indicator) inferring that this column can be used as an

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

Hi Aljoscha, Glad to see that we have a more thorough testing procedure. Could you please share us what (the critical issues you mentioned) have been broken in 1.3.0 & 1.3.1, and how the new proposed "functional testing section and a combination of systems/configurations" can cover this. This will

Re: [ANNOUNCE] New Flink PMC member: Tzu-Li (Gordon) Tai

Congrats +1 On Mon, Jul 10, 2017 at 10:44 PM, Robert Metzger wrote: > Hi Everybody, > > On behalf of the Flink PMC, I'm very excited to announce Gordon as the > latest addition to the Flink PMC. > > Gordon is a very active community member, helping with a lot of the release

Re: [ANNOUNCE] New Flink committer Jincheng Sun

Congratulations, Jincheng! On Tue, Jul 11, 2017 at 2:01 PM, Zhuoluo Yang wrote: > Congrats, Jincheng! > > Thanks, > > Zhuoluo  > > > > > > 在 2017年7月11日，上午9:51，Jark 写道： > > Welcome on board and congratulations Jincheng! > > > 在

[jira] [Created] (FLINK-6963) User Defined Operator

Shaoxuan Wang created FLINK-6963: Summary: User Defined Operator Key: FLINK-6963 URL: https://issues.apache.org/jira/browse/FLINK-6963 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-6962) SQL DDL for input and output tables

Shaoxuan Wang created FLINK-6962: Summary: SQL DDL for input and output tables Key: FLINK-6962 URL: https://issues.apache.org/jira/browse/FLINK-6962 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-6961) Enable configurable early-firing rate

Shaoxuan Wang created FLINK-6961: Summary: Enable configurable early-firing rate Key: FLINK-6961 URL: https://issues.apache.org/jira/browse/FLINK-6961 Project: Flink Issue Type: New Feature

Re: FlinkML on slack

Hi Stavros, Can I get an invitation for the slack channel. Thanks, Shaoxuan On Thu, Jun 8, 2017 at 3:56 AM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Hi all, > > We took the initiative to create the organization for FlinkML on slack > (thnx Eron). > There is now a channel for

Re: [DISCUSS] Table API / SQL features for Flink 1.4.0

Nice timing, Fabian! Your checklist aligns our plans very well. Here are the things we are working on & planning to contribute to release 1.4: 1. DDL (with property waterMark config for source-table, and emit config on result-table) 2. unbounded stream-stream joins (with retraction supported) 3.

Re: [VOTE] Release Apache Flink 1.3.0 (RC3)

Hi Robert. Will doc update be a blocker for release? Release 1.3 has many updates on tableAPI and SQL, and the docs are kind of lagging. We are trying the efforts to update the doc as much as possible before the release is officially published, but I am hoping this is not a blocker for the

[jira] [Created] (FLINK-6725) make requiresOver as a contracted method in udagg

Shaoxuan Wang created FLINK-6725: Summary: make requiresOver as a contracted method in udagg Key: FLINK-6725 URL: https://issues.apache.org/jira/browse/FLINK-6725 Project: Flink Issue Type

[jira] [Created] (FLINK-6637) Move registerFunction to TableEnvironment

Shaoxuan Wang created FLINK-6637: Summary: Move registerFunction to TableEnvironment Key: FLINK-6637 URL: https://issues.apache.org/jira/browse/FLINK-6637 Project: Flink Issue Type

Re: [DISCUSS] Reorganize Table API / SQL documentation

Hello Fabian, Thanks for drafting the proposal. I like the entire organization in general and left a few comments. I think this will be a very good kick off to reorganize the tableAPI doc. -shaoxuan On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske wrote: > Hi everybody, > > I

Re: [DISCUSS] Reorganize Table API / SQL documentation

Hi Robert, This sounds great to me. While I am in the middle of writing-up UDAGG doc (FLINK-5905), I also feel it's not good to have entire tableAPI introduction in one page. We can move tableAPI under "application development", and split it into small sub-topics, such as

Re: Proposal about inner join in Flink

Hello Xingcan, Thanks for the proposal. It seems (I may miss something) the proposed semantics for unbounded inner join is similar as the one proposed in FLINK-5878. I did not create the PR for FLINK-5878, as the implementation for inner join is closely associated with "Retract" (proposed in

Re: [DISCUSS] Expose State Backend Interface for UDAGG

would allow you to manage the intermediate data directly in the state > instead of the memory > > > -Original Message- > From: Shaoxuan Wang [mailto:wshaox...@gmail.com] > Sent: Friday, May 12, 2017 1:20 PM > To: Dev > Cc: Stephan Ewen > Subject: Re: [DISCUSS] Expose State

Re: Towards a spec for robust streaming SQL, Part 1

(which should be many :-). > > Shaoxuan, can you please refresh my memory what a dynamic table means in > Flink? It's basically just a state table, right? The "dynamic" part of the > name is to simply to imply it can evolve and change over time? > > -Tyler > >

Re: [DISCUSS] Expose State Backend Interface for UDAGG

in the Table > API, but this will he a huge effort as well (maybe we have to do it anyway > at some point though). > > @Stephan knows more about the implications of allowing state in > AggregateFunctions. > > Best, Fabian > > 2017-05-12 11:53 GMT+02:00 Shaoxuan Wang &l

[DISCUSS] Expose State Backend Interface for UDAGG

Hi everyone, We made some progress in the implementation of UDAGG (FLINK-5564). However, we realized that there are cases where users may want to use state backend to store the data. For instance, the built-in MaxWithRetractAggFunction currently create a hashMap to store the historical data. It

Re: Towards a spec for robust streaming SQL, Part 1

Thanks to Tyler and Fabian for sharing your thoughts. Regarding to the early/late update control of FLINK. IMO, each dynamic table can have an EMIT config. For FLINK table-API, this can be easily implemented in different manners, case by case. For instance, in window aggregate, we could define

Re: [DISCUSS] Feature Freeze

Hi Ufuk, Thanks for the heads-up. In terms of table API and SQL, I am hoping we can get the following features (which have not completed merged yet) included in 1.3: 1. UDAGG (FLINK-5564), I am working on the last PR (FLINK-5906) to close this feature. 2. Retract (FLINK-6047), we are planning

[jira] [Created] (FLINK-6242) codeGen DataSet Goupingwindow Aggregates

Shaoxuan Wang created FLINK-6242: Summary: codeGen DataSet Goupingwindow Aggregates Key: FLINK-6242 URL: https://issues.apache.org/jira/browse/FLINK-6242 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-6240) codeGen dataStream aggregates that use AggregateAggFunction

Shaoxuan Wang created FLINK-6240: Summary: codeGen dataStream aggregates that use AggregateAggFunction Key: FLINK-6240 URL: https://issues.apache.org/jira/browse/FLINK-6240 Project: Flink

[jira] [Created] (FLINK-6216) DataStream unbounded groupby aggregate with early firing

Shaoxuan Wang created FLINK-6216: Summary: DataStream unbounded groupby aggregate with early firing Key: FLINK-6216 URL: https://issues.apache.org/jira/browse/FLINK-6216 Project: Flink Issue

[jira] [Created] (FLINK-6094) Implement and turn on retraction for stream-stream inner join

Shaoxuan Wang created FLINK-6094: Summary: Implement and turn on retraction for stream-stream inner join Key: FLINK-6094 URL: https://issues.apache.org/jira/browse/FLINK-6094 Project: Flink

[jira] [Created] (FLINK-6093) Implement and turn on retraction for table sink

Shaoxuan Wang created FLINK-6093: Summary: Implement and turn on retraction for table sink Key: FLINK-6093 URL: https://issues.apache.org/jira/browse/FLINK-6093 Project: Flink Issue Type

[jira] [Created] (FLINK-6092) Implement and turn on retraction for table source

Shaoxuan Wang created FLINK-6092: Summary: Implement and turn on retraction for table source Key: FLINK-6092 URL: https://issues.apache.org/jira/browse/FLINK-6092 Project: Flink Issue Type

[jira] [Created] (FLINK-6091) Implement and turn on the retraction for grouping window aggregate

Shaoxuan Wang created FLINK-6091: Summary: Implement and turn on the retraction for grouping window aggregate Key: FLINK-6091 URL: https://issues.apache.org/jira/browse/FLINK-6091 Project: Flink

[jira] [Created] (FLINK-6090) Implement optimizer for retraction and turn on retraction for over window aggregate

Shaoxuan Wang created FLINK-6090: Summary: Implement optimizer for retraction and turn on retraction for over window aggregate Key: FLINK-6090 URL: https://issues.apache.org/jira/browse/FLINK-6090

[jira] [Created] (FLINK-6089) Implement decoration phase for rewriting predicated logical plan after volcano optimization phase

Shaoxuan Wang created FLINK-6089: Summary: Implement decoration phase for rewriting predicated logical plan after volcano optimization phase Key: FLINK-6089 URL: https://issues.apache.org/jira/browse/FLINK-6089

Re: [Discuss] Retraction for Flink Streaming

n or > none. Otherwise, the behavior of the Table API / SQL will not be > predictable. > > I also think that we should define which operators we want to support in > Flink 1.3 in order to coordinate the development of retraction support. > > What do others think? > > Cheers

[Discuss] Retraction for Flink Streaming

Hello everyone, Flink is widely used in Alibaba Group, especially in our Search and Recommendation Infra. Retraction is one of the most important features that we needed. We have spent lots of efforts to try to solve this problem, and gladly at the end we develop an approach which can address

[jira] [Created] (FLINK-6047) Master Jira for "Retraction for Flink Streaming"

Shaoxuan Wang created FLINK-6047: Summary: Master Jira for "Retraction for Flink Streaming" Key: FLINK-6047 URL: https://issues.apache.org/jira/browse/FLINK-6047 Project: Flink

Re: Incremental checkpoint branch

Vinshnu, You can find the latest design discussion for incremental checkpoint in http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Incremental-Checkpointing-in-Flink-td15931.html @Stefan Richter

[jira] [Created] (FLINK-5956) Add retract method into the aggregateFunction

Shaoxuan Wang created FLINK-5956: Summary: Add retract method into the aggregateFunction Key: FLINK-5956 URL: https://issues.apache.org/jira/browse/FLINK-5956 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-5955) Merging a list of buffered records will have problem when ObjectReuse is turned on

Shaoxuan Wang created FLINK-5955: Summary: Merging a list of buffered records will have problem when ObjectReuse is turned on Key: FLINK-5955 URL: https://issues.apache.org/jira/browse/FLINK-5955

[jira] [Created] (FLINK-5927) Remove old Aggregate interface and built-in functions

Shaoxuan Wang created FLINK-5927: Summary: Remove old Aggregate interface and built-in functions Key: FLINK-5927 URL: https://issues.apache.org/jira/browse/FLINK-5927 Project: Flink Issue

[jira] [Created] (FLINK-5915) Add support for the aggregate on multi fields

Shaoxuan Wang created FLINK-5915: Summary: Add support for the aggregate on multi fields Key: FLINK-5915 URL: https://issues.apache.org/jira/browse/FLINK-5915 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-5914) remove aggregateResultType from streaming.api.datastream.aggregate

Shaoxuan Wang created FLINK-5914: Summary: remove aggregateResultType from streaming.api.datastream.aggregate Key: FLINK-5914 URL: https://issues.apache.org/jira/browse/FLINK-5914 Project: Flink

[jira] [Created] (FLINK-5900) Add non-partial merge Aggregates and unit tests

Shaoxuan Wang created FLINK-5900: Summary: Add non-partial merge Aggregates and unit tests Key: FLINK-5900 URL: https://issues.apache.org/jira/browse/FLINK-5900 Project: Flink Issue Type

[jira] [Created] (FLINK-5899) Fix the bug in initializing the DataSetTumbleTimeWindowAggReduceGroupFunction

Shaoxuan Wang created FLINK-5899: Summary: Fix the bug in initializing the DataSetTumbleTimeWindowAggReduceGroupFunction Key: FLINK-5899 URL: https://issues.apache.org/jira/browse/FLINK-5899 Project

[jira] [Created] (FLINK-5878) Add stream-stream inner join on TableAPI

Shaoxuan Wang created FLINK-5878: Summary: Add stream-stream inner join on TableAPI Key: FLINK-5878 URL: https://issues.apache.org/jira/browse/FLINK-5878 Project: Flink Issue Type: New

[jira] [Created] (FLINK-5813) code generation for user-defined aggregate functions

Shaoxuan Wang created FLINK-5813: Summary: code generation for user-defined aggregate functions Key: FLINK-5813 URL: https://issues.apache.org/jira/browse/FLINK-5813 Project: Flink Issue

  1   2   >