Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-07-09 Thread Seth Wiesman
Happy to help clarify, and certainly, I don't think anyone would reject the contribution if you wanted to add that functionality. The important point we want to communicate is that the table API has matured to a level where we do not see it as supplementary to DataStream but as an equal

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-07-09 Thread Etienne Chauchot
Hi Seth, Thanks for your comment. That is perfect then ! If the community agrees to give precedence to the table API connectors over the DataStream connectors as discussed here, all the features are already available then. I will not need to port my latest additions to table connectors

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-07-08 Thread Seth Wiesman
Hi Etienne, The `toDataStream` method supports converting to concrete Java types, not just Row, which can include your Avro specific-records. See example 2: https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/table/data_stream_api/#examples-for-todatastream On Thu, Jul 8, 2021 at

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-07-08 Thread Etienne Chauchot
Hi Timo, Thanks for your answers, no problem with the delay, I was in vacation too last week :) My comments are inline Best, Etienne On 07/07/2021 16:48, Timo Walther wrote: Hi Etienne, sorry for the late reply due to my vacation last week. Regarding: "support of aggregations in batch

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-07-07 Thread Timo Walther
Hi Etienne, sorry for the late reply due to my vacation last week. Regarding: "support of aggregations in batch mode for DataStream API [...] is there a plan to solve it before the actual drop of DataSet API" Just to clarify it again: we will not drop the DataSet API any time soon. So users

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-07-06 Thread Etienne Chauchot
Hi all, Any comments ? cheers, Etienne On 25/06/2021 15:09, Etienne Chauchot wrote: Hi everyone, @Timo, my comments are inline for steps 2, 4 and 5, please tell me what you think. Best Etienne On 23/06/2021 15:27, Chesnay Schepler wrote: If we want to publicize this plan more

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-06-25 Thread Etienne Chauchot
Hi everyone, @Timo, my comments are inline for steps 2, 4 and 5, please tell me what you think. Best Etienne On 23/06/2021 15:27, Chesnay Schepler wrote: If we want to publicize this plan more shouldn't we have a rough timeline for when 2.0 is on the table? On 6/23/2021 2:44 PM, Stephan

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Chesnay Schepler
If we want to publicize this plan more shouldn't we have a rough timeline for when 2.0 is on the table? On 6/23/2021 2:44 PM, Stephan Ewen wrote: Thanks for writing this up, this also reflects my understanding. I think a blog post would be nice, ideally with an explicit call for feedback so

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Stephan Ewen
Thanks for writing this up, this also reflects my understanding. I think a blog post would be nice, ideally with an explicit call for feedback so we learn about user concerns. A blog post has a lot more reach than an ML thread. Best, Stephan On Wed, Jun 23, 2021 at 12:23 PM Timo Walther

[DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Timo Walther
Hi everyone, I'm sending this email to make sure everyone is on the same page about slowly deprecating the DataSet API. There have been a few thoughts mentioned in presentations, offline discussions, and JIRA issues. However, I have observed that there are still some concerns or different