Re: [VOTE] FLIP-164: Improve Schema Handling in Catalogs

2021-02-18 Thread Timo Walther
21 at 8:25 AM Jark Wu wrote: +1 (binding) Best, Jark 2021年2月12日 20:37,Dawid Wysakowicz 写道: +1 (binding) Best, Dawid On 12/02/2021 13:33, Timo Walther wrote: Hi everyone, I'd like to start a vote on FLIP-164 [1] which was discussed in [2]. The vote will be open for at least 72 hou

[RESULT][VOTE] FLIP-164: Improve Schema Handling in Catalogs

2021-02-18 Thread Timo Walther
Sorry, I forgot to add the [RESULT] label in the mail's subject. I'm sending this mail to satisfy the process. Regards, Timo On 18.02.21 12:02, Timo Walther wrote: Hi everyone, The voting time for FLIP-164: Improve Schema Handling in Catalogs [1] has passed. I'm closi

Re: [VOTE] FLIP-163: SQL Client Improvements

2021-02-22 Thread Timo Walther
+1 (binding) Thanks, Timo On 22.02.21 04:44, Jark Wu wrote: +1 (binding) Best, Jark On Mon, 22 Feb 2021 at 11:06, Shengkai Fang wrote: Hi devs It seems we have reached consensus on FLIP-163[1] in the discussion[2]. So I'd like to start the vote for this FLIP. Please vote +1 to approve th

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-23 Thread Timo Walther
tax is SQL-client-specific, but if it's general Flink SQL syntax we should consider this (one way or another). Regards Ingo On Fri, Feb 12, 2021 at 3:53 PM Timo Walther wrote: Hi Shengkai, thanks for updating the FLIP. I have one last comment for the option `table.execution.mode`. Should

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-25 Thread Timo Walther
we might cause if we introduce such a method of our own, I think it's better to wait for some more feedback. Best, Kurt On Tue, Feb 23, 2021 at 9:45 PM Timo Walther wrote: Hi Kurt, we can also shorten it to `table.dml-sync` if that would help. Then it would confuse users that do a regular

[DISCUSS] Deprecation and removal of the legacy SQL planner

2021-02-25 Thread Timo Walther
Hi everyone, since Flink 1.9 we have supported two SQL planners. Most of the original plan of FLIP-32 [1] has been implemented. The Blink code merge has been completed and many additional features have been added exclusively to the new planner. The new planner is now in a much better shape tha

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-28 Thread Timo Walther
on. I don't think the drawback is really critical because many systems have commands play the same role with the different names. Best, Shengkai Timo Walther 于2021年2月25日周四 下午4:23写道: The `table.` prefix is meant to be a general option in the table ecosystem. Not necessarily attached to

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-02-28 Thread Timo Walther
ther systems and databases will evaluate these functions during query start. And for streaming users, I have already seen some users are expecting these functions to be calculated per record. Thus I think we can make the behavior determined together with execution mode. One exception would be PROCTIME(),

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-02-28 Thread Timo Walther
and btw it is interesting to notice that AWS seems to do the approach that I suggested first. All functions are SQL standard compliant, and only dedicated functions with a prefix such as CURRENT_ROW_TIMESTAMP divert from the standard. Regards, Timo On 01.03.21 08:45, Timo Walther wrote: How

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Timo Walther
we will suffer from such config option, because users can always make Flink SQL have strange behavior by setting this config to an undesired way. I prefer to not introduce such config until we have to. Leonard's proposal already makes almost all users happy thus I think we can still wait. Best

Re: [DISCUSS] Splitting User support mailing list

2021-03-01 Thread Timo Walther
I would vote -0 here. I fear that we are creating potential silos where a team doesn't know what is going on in the other teams. Regards, Timo On 01.03.21 10:47, Jark Wu wrote: I also have some concerns about splitting python and sql. Because I have seen some SQL questions users reported bu

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-03-01 Thread Timo Walther
Best, Kurt On Mon, Mar 1, 2021 at 3:58 PM Timo Walther wrote: and btw it is interesting to notice that AWS seems to do the approach that I suggested first. All functions are SQL standard compliant, and only dedicated functions with a prefix such as CURRENT_ROW_TIMESTAMP divert from the sta

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-03-02 Thread Timo Walther
Kurt Young 于2021年3月1日周一 下午5:11写道: I'm +1 for either: 1. introduce a sql client specific option, or 2. Introduce a table config option and make it apply to both table module & sql client. It would be the FLIP owner's call to decide. Best, Kurt On Mon, Mar 1, 2021 at 3:25 PM

Re: [VOTE] FLIP-162: Consistent Flink SQL time function behavior

2021-03-02 Thread Timo Walther
+1 (binding) Regards, Timo On 03.03.21 04:14, Jark Wu wrote: +1 (binding) Best, Jark On Tue, 2 Mar 2021 at 10:42, Leonard Xu wrote: Hi all, I would like to start the vote for FLIP-162 [1], which has been discussed and reached a consensus in the discussion thread [2]. Please vote +1 to ap

Re: [DISCUSS] Deprecation and removal of the legacy SQL planner

2021-03-04 Thread Timo Walther
's better for us to be more focused on a single planner. Your proposed roadmap looks good to me, +1 from my side and thanks again for all your efforts! Best, Kurt On Thu, Feb 25, 2021 at 5:01 PM Timo Walther wrote: Hi everyone, since Flink 1.9 we have supported two SQL planners. Most of

Re: [DISCUSS] FLIP-162 follow-up discussion

2021-03-09 Thread Timo Walther
Hi Leonard, I'm fine with dropping the old buggy behavior immediatly. Users can still implement a UDF with the old bavhior if needed. I hope the new functions will be well-tested so that a fallback to the old functions is not necessary as a workaround. It will definitely avoid confusion for u

Re: [DISCUSS] Deprecation and removal of the legacy SQL planner

2021-03-10 Thread Timo Walther
20:59, Leonard Xu wrote: +1 for the roadmap. Thanks Timo for driving this. Best, Leonard 在 2021年3月4日,20:40,Timo Walther 写道: Last call for feedback on this topic. It seems everyone agrees to finally complete FLIP-32. Since FLIP-32 has been accepted for a very long time, I think we don&#

Re: Apply external 3 days for FLINK-20387 after feature freeze

2021-03-31 Thread Timo Walther
Hi everyone, I support Leonard's request. It was foreseeable that the changes of FLIP-162 will be massive and will take some time. By looking at PRs such as https://github.com/apache/flink/pull/15280 I would also vote for giving a bit more time for proper reviews and finalizing this story fo

[DISCUSS] Merge StreamTableEnvironment.to/fromChangelogStream to release-1.13

2021-04-21 Thread Timo Walther
Hi everyone, sorry for being so late with this request, but fixing a couple of down stream bugs had higher priority than this issue and were also blocking it. Nevertheless, I would like to ask for permission to merge the FLINK-19980[1] to the 1.13 branch as an experimental feature to add the

Re: [DISCUSS] Merge StreamTableEnvironment.to/fromChangelogStream to release-1.13

2021-04-21 Thread Timo Walther
re is a new RC or 1.13.1 otherwise. Of course if there are no objections from others. Best, Dawid On 21/04/2021 10:52, Timo Walther wrote: > Hi everyone, > > sorry for being so late with this request, but fixing a couple of down > stream bugs

Re: [DISCUSS] Releasing Flink 1.13.1

2021-05-17 Thread Timo Walther
Hi Konstantin, thanks for starting the discussion. From the Table API side, we also fixed a couple of critical issues already that justify releasing a 1.13.1 asap. Personally, I would like to include https://issues.apache.org/jira/browse/FLINK-22666 that fixes some last issues with the Scal

Re: [DISCUSS] Feedback Collection Jira Bot

2021-05-21 Thread Timo Walther
Hi Konstantin, thanks for starting this discussion. I was also about to provide some feedback because I have the feeling that the bot is too aggressive at the moment. Even a 14 days interval is a short period of time for bigger efforts that might include several subtasks. Currently, if we sp

Re: [VOTE] FLIP-129 (Update): Registering sources/sinks on Table API without SQL

2021-06-10 Thread Timo Walther
Hi Ingo, thanks for giving FLIP-129 an update before finally implementing it. I wouldn't start with a voting thread right away but collect more feedback from the community in a [DISCUSS] thread before. Also, voting threads should be performed on the updated wiki page and include the voting d

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-08-10 Thread Timo Walther
Hi Xuannan, sorry for joining the discussion so late. I agree that this is a very nice and useful feature. However, the impact it has to many components in the stack requires more discussion in my opinion. 1) Separation of concerns: The current design seems to mix different layers. We should

Re: [VOTE] Release 1.10.2, release candidate #1

2020-08-12 Thread Timo Walther
+1 (binding) I went through the commit diff and changed files of this release. Could not spot anything suspicious. Regards, Timo On 11.08.20 14:47, Zhu Zhu wrote: Hi everyone, Please review and vote on the release candidate #1 for the version 1.10.2, as follows: [ ] +1, Approve the release

[DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-08-19 Thread Timo Walther
Hi everyone, I would like to propose a FLIP that aims to resolve the remaining shortcomings in the Table API: https://cwiki.apache.org/confluence/display/FLINK/FLIP-136%3A++Improve+interoperability+between+DataStream+and+Table+API The Table API has received many new features over the last yea

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-08-19 Thread Timo Walther
SQL or Table API) together. The statements in a statement set are jointly optimized and executed as a single Flink job. Maybe if you add this to the FLIP it will help other readers as well. Best, David On Wed, Aug 19, 2020 at 10:22 AM Timo Walther wrote: Hi everyone, I would like to

Re: [VOTE] Remove deprecated DataStream#fold and DataStream#split in 1.12

2020-08-31 Thread Timo Walther
+1 Thanks for removing legacy. Regards, Timo On 28.08.20 11:55, David Anderson wrote: +1 David On Fri, Aug 28, 2020 at 9:41 AM Dawid Wysakowicz wrote: Hi all, I would like to start a vote for removing deprecated, but Public(Evolving) methods in the upcoming 1.12 release: - XxxDataSt

Re: [DISCUSS] FLIP-139: General Python User-Defined Aggregate Function on Table API

2020-08-31 Thread Timo Walther
Hi Wei, is `reset_accumulator` still necessary? We dropped it recently in the Java API because it was not used anymore by the planner. Regards, Timo On 31.08.20 15:00, Wei Zhong wrote: Hi Jincheng & Xingbo, Thanks for your suggestions. I agree that we should keep the user interface uniform

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-01 Thread Timo Walther
ullable Map fieldNames)" - Maybe `List fieldNames` or `String[] fieldNames` parameter is enough and more handy than Map ? - Currently, the fieldNames member variable is mutable, is it on purpose? Can we make it immutable? For example, only accept from the constructor. - Why do we accept a null

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-01 Thread Timo Walther
eamExecutionEnvironment#execute. 2. @Timo What is the interaction between Row setters from the different modes? What happens if the user calls both in different order. E.g. row.setField(0, "ABC"); row.setField("f0", "ABC"); // is this a valid call ? or row.setFi

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-02 Thread Timo Walther
an we must do that in the DataStream API explicitly. In the SQL world, no projection function outputs type of time-attribute, we better still put the time-attributes in the scope of the table metadata. Best, Danny Chan 在 2020年8月19日 +0800 PM4:22,Timo Walther ,写道: Hi everyone, I would like to propos

Re: [DISCUSS] Introduce partitioning strategies to Table/SQL

2020-09-02 Thread Timo Walther
Hi Jingsong, I haven't looked at your proposal but I think it make sense to have a separate FLIP for the parititioning topic. I'm currently working on an update to FLIP-107 and would suggest to remove the paritioning topic there. FLIP-107 will only focus on accessing metadata and expressing k

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-03 Thread Timo Walther
m).select()`. Best, Danny Chan 在 2020年9月2日 +0800 PM4:19,Timo Walther ,写道: Hi everyone thanks for your feedback. It's a lot of content that needs to be digested. I will update the FLIP shortly to incorporate some of the feedback already. But let me respond to some topics first: "not

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-03 Thread Timo Walther
Hi Danny, "if ChangelogMode.INSERT is the default, existing pipelines should be compatible" It is not about changelog mode compatibility, it is about the type compatibility. The renaming to `toInsertStream` is only to have a mean of dealing with data type inconsistencies that could break exi

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-04 Thread Timo Walther
Hi everyone, I completely reworked FLIP-107. It now covers the full story how to read and write metadata from/to connectors and formats. It considers all of the latest FLIPs, namely FLIP-95, FLIP-132 and FLIP-122. It introduces the concept of PERSISTED computed columns and leaves out partition

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-04 Thread Timo Walther
think it should be covered in the public interface section, i.e., Table and TableEnvironment. Other than those, all the changes should only affect the internal class. Please let me know if you have any further comments. Best, Xuannan On Aug 10, 2020, 9:32 PM +0800, Timo Walther , wrote:

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

2020-09-04 Thread Timo Walther
+1 for getting rid of the TypeComparator interface and rely on the serialized representation for grouping. Adding a new type to DataStream API is quite difficult at the moment due to too many components that are required: TypeInformation (tries to deal with logical fields for TypeComparators),

Re: Apply for contributor permissions

2020-09-04 Thread Timo Walther
Hi Jun, you don't need contributor permissions to contribute to Apache Flink. Just open/pick an issue and ping a committer to assign you to this ticket. Regards, Timo On 04.09.20 14:28, jun su wrote: Hi Guys, I want to contribute to Apache Flink. Would you please give me the permission as a

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-07 Thread Timo Walther
Hi Leonard, thanks for your feedback. (1) Actually, I discuss this already in the FLIP. But let me summarize our options again if it was not clear enough in the FLIP: a) CREATE TABLE t (a AS CAST(SYSTEM_METADATA("offset") AS INT)) pro: readable, complex arithmetic possible, more SQL compliant

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-07 Thread Timo Walther
Why does the `DecodingFormat#applyReadableMetadata(List metadataKeys)` don't need the `DataType outputDataType` parameter? 3. I think it would be great if we can list the metadata keys (and readable/writable) we want to expose in the first version. I think they are also important public APIs, li

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-08 Thread Timo Walther
[VIRTUAL] Which is more straight-forward. 2. “SYSTEM_METADATA("offset")` returns the NULL type by default” The default type should not be NULL because only NULL literal does that. Usually we use ANY as the type if we do not know the specific type in the SQL context. ANY means the physi

Re: Merge SupportsComputedColumnPushDown and SupportsWatermarkPushDown

2020-09-08 Thread Timo Walther
Hi Shengkai, first of I would not consider the section Problems of SupportsWatermarkPushDown" as a "problem". It was planned to update the WatermarkProvider once the interfaces are ready. See the comment in WatermarkProvider: // marker interface that will be filled after FLIP-126: // Waterma

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-08 Thread Timo Walther
ctors/kafka/FlinkKafkaConsumerBase.java#L1066 On Tue, 8 Sep 2020 at 16:35, Timo Walther wrote: Hi everyone, I updated the FLIP again and hope that I could address the mentioned concerns. @Leonard: Thanks for the explanation. I wasn't aware that ts_ms and source.ts_ms have different sema

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-08 Thread Timo Walther
is means potentially duplicate definition of fields and their data types etc” I agree, because table already has an underlying schema there. Best, Danny Chan 在 2020年9月3日 +0800 PM8:12,Timo Walther ,写道: Hi Danny, "if ChangelogMode.INSERT is the default, existing pipelines should be compatib

Re: Merge SupportsComputedColumnPushDown and SupportsWatermarkPushDown

2020-09-08 Thread Timo Walther
a strong use case that needs this interface so far. Best, Jark On Tue, 8 Sep 2020 at 17:13, Timo Walther wrote: Hi Shengkai, first of I would not consider the section Problems of SupportsWatermarkPushDown" as a "problem". It was planned to update the WatermarkProvider once

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-08 Thread Timo Walther
columns.html <https://www.postgresql.org/docs/12/ddl-generated-columns.html> 在 2020年9月8日,17:31,Timo Walther 写道: Hi Jark, according to Flink's and Calcite's casting definition in [1][2] TIMESTAMP WITH LOCAL TIME ZONE should be castable from BIGINT. If not, we will make it possi

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-08 Thread Timo Walther
han 在 2020年9月8日 +0800 PM8:22,Timo Walther ,写道: Hi Danny, Your proposed signatures sound good to me. fromDataStream(dataStream, Schema) toDataStream(table, AbstractDataType) They address all my concerns. The API would not be symmetric anymore, but this is fine with me. Others raised concerns

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-09 Thread Timo Walther
TEM_METADATA("timestamp"), // get the timestamp field from metadata ts AS to_timestamp(timestamp) // normal computed column, parse the string to TIMESTAMP type by using the metadata field ) WITH ( ... ) Best, Kurt On Tue, Sep 8, 2020 at 11:57 PM Timo Walther wrote: Hi Leonard, t

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-09 Thread Timo Walther
I had this in the inital design, but Jark had concerns at least for the `toChangelogStream(ChangelogMode)` (see earlier discussion). `fromDataStream(dataStream, schema, changelogMode)` would be possible. But in this case I would vote for a symmetric API. If we keep toChangelogStream we should

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-09 Thread Timo Walther
kafka_table SELECT id, name, col1, col2, rowtime FROM another_table; I think this can solve all the problems without introducing any new syntax. The only minor disadvantage is that we separate the definition way/syntax of read-only metadata and read-write fields. However, I don't think th

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-09 Thread Timo Walther
read-only and read-write). In this FLIP, we declare the Kafka key fields with table options but SYSTEM_METADATA for other metadata, that is a hacky thing or something in-consistent. Kurt Young 于2020年9月9日周三 下午4:48写道: I would vote for `offset INT SYSTEM_METADATA("offset")`. I don't

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-09 Thread Timo Walther
there are some confusions: • Should DataStream of Row type always use #fromChangelogStream ? • Does fromChangelogStream works for only INSERT ChangelogMode ? Best, Danny Chan 在 2020年9月9日 +0800 PM4:21,Timo Walther ,写道: I had this in the inital design, but Jark had concerns at least fo

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-09 Thread Timo Walther
name'" is only needed when the name conflicts with the declared table column name, when there are no conflicts, we can simplify it to: timestamp INT METADATA By default, the field is non-virtual and can be read and written, users need to mark the column as virtual when it is only readable

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-09 Thread Timo Walther
`SYSTEM_METADATA ` a lot, First of all, the word `TIME` has broad meanings but the word `METADATA ` not, `METADATA ` has specific meaning, Secondly, `FOR SYSTEM_TIME AS OF` exists in SQL standard but `SYSTEM_METADATA ` not. Personally, I like more simplify way,sometimes less is more. Best, Leon

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-10 Thread Timo Walther
:52, Timo Walther wrote: "If virtual by default, when a user types "timestamp int" ==> persisted column, then adds a "metadata" after that ==> virtual column, then adds a "persisted" after that ==> persisted column." Thanks for this nice mental mo

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-10 Thread Timo Walther
voting. What do you think? Regards, Timo On 09.09.20 14:31, Danny Chan wrote: Thanks, i'm fine with that. Timo Walther 于2020年9月9日周三 下午7:02写道: I agree with Jark. It reduces confusion. The DataStream API doesn't know changelog processing at all. A DataStream of Row can be used

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-10 Thread Timo Walther
st, Xuannan On Sep 4, 2020, 7:29 PM +0800, Timo Walther , wrote: Hi Xuannan, thanks for the update. Here is some final feedback from my side. Maybe others have some final feedback as well before we continue to a voting? Some mistakes that we should fix in the FLIP: - The FLIP declares `Table c

[VOTE] FLIP-107: Handling of metadata in SQL connectors

2020-09-10 Thread Timo Walther
Hi all, after the discussion in [1], I would like to open a voting thread for FLIP-107 [2] which discusses how to handle data next to the main payload (i.e. key and value formats as well as metadata in general) in SQL connectors and DDL. The vote will be open until September 15th (72h + week

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-10 Thread Timo Walther
implicit cast may not work... I don't have other objections. But maybe we should wait for the opinion from @Kurt for the new syntax. Best, Jark On Thu, 10 Sep 2020 at 16:21, Danny Chan wrote: Thanks for driving this Timo, +1 for voting ~ Best, Danny Chan 在 2020年9月10日 +0800 PM3:47,Timo Wa

Re: [DISCUSS] Drop Scala 2.11

2020-09-11 Thread Timo Walther
Big +1 to drop Scala 2.11 This would mean that we can finally use Java 8 language features that are integrated with Scala. Regards, Timo On 11.09.20 13:15, Igal Shilman wrote: @Galen FYI: the upcoming StateFun release would use Scala2.12 On Thu, Sep 10, 2020 at 5:14 PM Seth Wiesman wrote:

[VOTE] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-11 Thread Timo Walther
Hi all, after the discussion in [1], I would like to open a voting thread for FLIP-136 [2] which covers different topic to improve the back-and-forth communication between DataStream API and Table API. The vote will be open until September 16th (72h + weekend), unless there is an objection o

Re: random fetch

2020-09-17 Thread Timo Walther
Hi Chenyang, we will relax this constraint in 1.12. The issue has alsready been implemented: https://issues.apache.org/jira/browse/FLINK-18569 Until then, you can use SQL. It supports FETCH and LIMIT already without ordering. Btw please use the user@ mailing list for questions of this kind.

[RESULT][VOTE] FLIP-107: Handling of metadata in SQL connectors

2020-09-17 Thread Timo Walther
nding ~ Konstantin Knauf 于2020年9月11日 周五上午2:04写道: +1 (binding) On Thu, Sep 10, 2020 at 4:29 PM Dawid Wysakowicz < dwysakow...@apache.org> wrote: +1 (binding) On 10/09/2020 14:03, Aljoscha Krettek wrote: +1 (binding) Aljoscha On 10.09.20 13:57, Timo Walther wrote

Re: [VOTE] FLIP-36 - Support Interactive Programming in Flink Table API

2020-09-17 Thread Timo Walther
+1 (binding) Looking forward to review the pull requests for this valuable feature. Regards, Timo On 16.09.20 07:35, Aljoscha Krettek wrote: +1 (binding) Nice work! :-) Aljoscha On 16.09.20 06:00, Xuannan Su wrote: Hi all, I'd like to start the vote for FLIP-36[1], which has been discuss

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-17 Thread Timo Walther
. Regards, Timo On 10.09.20 10:21, Danny Chan wrote: Thanks for driving this Timo, +1 for voting ~ Best, Danny Chan 在 2020年9月10日 +0800 PM3:54,Timo Walther ,写道: Thanks everyone for this healthy discussion. I updated the FLIP with the outcome. I think the result is one of the last core API refactoring

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-18 Thread Timo Walther
n also come up with a new class for the field names which will construct the Map and be shared among all Row instances. What do you think? Best, Jark On Thu, 17 Sep 2020 at 16:48, Timo Walther wrote: Hi everyone, thanks for all the feedback. I updated the FLIP again on Thursday to integrate

Re: Can you unify the language ?

2020-09-20 Thread Timo Walther
Hi, you are right. Having two languages in the code base doesn't make our lives easier. But Flink is a big project with a long history, multiple design shifts, and many contributors. It is naturally that the bigger a code base gets, the messier it looks like. It must be a continuous effort to

[CANCEL][VOTE] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-23 Thread Timo Walther
There is still a discussion ongoing. I will cancel the vote for now. Regards, Timo On 14.09.20 09:48, Dawid Wysakowicz wrote: +1 On 11/09/2020 16:19, Timo Walther wrote: Hi all, after the discussion in [1], I would like to open a voting thread for FLIP-136 [2] which covers different topic

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-23 Thread Timo Walther
ns. Best, Jark On Fri, 18 Sep 2020 at 22:14, Timo Walther wrote: Hi Jark, the fieldNames map is not intended for users. I would also be fine to make it a default scope constructor and access it with some internal utility class next to the Row class. The fieldNames map must only be used by

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-23 Thread Timo Walther
fields are NULL. They would expect that all the fields they didn't set should be NULL. Row.withNames(String[] filedNames) or Row.withNames(List fieldNames) seems to be a safer choice. I agree that simplicity is important but making API safer to use is also important. Best, Kurt On Wed, Sep 23

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-23 Thread Timo Walther
sequence of setting method calls: Row row = Row.withNames(); row.setField("b", 2); row.setField("a", 1); I don't think anyone would expect these two rows are actually different. Instead, if we at least define the field names first, which will fix the order, we would not have su

Re: [DISCUSS] Move Hive document to "Table & SQL Connectors" from "Table API & SQL"

2020-09-23 Thread Timo Walther
+1 On 24.09.20 06:54, Jark Wu wrote: +1 to move it there. On Thu, 24 Sep 2020 at 12:16, Jingsong Li wrote: Hi devs and users: After the 1.11 release, I heard some voices recently: How can't Hive's documents be found in the "Table & SQL Connectors". Actually, Hive's documents are in the "Ta

[VOTE] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-24 Thread Timo Walther
Hi all, after the discussion in [1], I would like to open a second voting thread for FLIP-136 [2] which covers different topic to improve the back-and-forth communication between DataStream API and Table API. The vote will be open until September 29th (72h + weekend), unless there is an obje

Re: [DISCUSS][Code-Style] The approach to implement singleton pattern

2020-09-25 Thread Timo Walther
Hi, honstely, I find using enums is more of a hack. `enum` stands for enumeration and is meant for listing flags or options. Using it for singleton patterns is just abusing a concept due to certain implementation details and less code. I feel this topic is like using Lombok for generating ha

Re: [VOTE] FLIP-145: Support SQL windowing table-valued function

2020-10-13 Thread Timo Walther
Hi Jark, as far as I can see it is too early to close the voting process. The official end time is in roughly 1 hour. Also, usually we exclude weekends from the voting period as you can see here [1] and here [2] to give people more time to respond discussions. But this is more of an informal

Re: [VOTE] FLIP-145: Support SQL windowing table-valued function

2020-10-13 Thread Timo Walther
e for voting and didn't count weekends in. Let's open for the voting until 15th Oct (UTC). Thank you for the valuable feedback. What about continuing the discussion in the DISUCSS thread? We can start a new vote if the FLIP needs some modifications. Best, Jark On Tue, 13 Oct 2020 at 16:5

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-13 Thread Timo Walther
Hi Jark, 1a) Deprecation: I totally agree that dropping old syntax is not very easy in SQL. That's why I am also extremely cautious about adding new syntax in this FLIP. However, I would already deprecate the old syntax and only use the new one in all examples, docs, slides etc. This hopefull

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-15 Thread Timo Walther
3780?focusedCommentId=17123472&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17123472 On Tue, 13 Oct 2020 at 22:24, Timo Walther wrote: Hi Jark, 1a) Deprecation: I totally agree that dropping old syntax is not very easy in SQL. That's why I am also ext

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-22 Thread Timo Walther
Hi Shengkai, Hi Jark, thanks for this great proposal. It is time to finally connect the changelog processor with a compacted Kafka topic. "The operator will produce INSERT rows, or additionally generate UPDATE_BEFORE rows for the previous image, or produce DELETE rows with all columns filled

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-22 Thread Timo Walther
"kafka-compacted" can even remind users to enable log compaction. I don't agree to introduce "model = table/stream" option, or "connector=kafka-table", because this means we are introducing Table vs Stream concept from KSQL. However, we don't have such

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-22 Thread Timo Walther
s the same physical kafka sink under the hood and it will lead to other confusion like does it offer the same persistence guarantees? I think we are capable of adding good valdiation messaging that solves Jark and Kurts concerns. On Thu, Oct 22, 2020 at 8:51 AM Timo Walther wrote: Hi Jark,

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-23 Thread Timo Walther
ct is just an optimization? - ktable let me think about KSQL. - kafka-compacted it is not just compacted, more than that, it still has the ability of CDC - upsert-kafka , upsert is back, and I don't really want to see it again since we have CDC Best, Jingsong On Fri, Oct 23, 2020 at 2:21

Re: [VOTE]FLIP-149: Introduce the upsert-kafka connector

2020-10-23 Thread Timo Walther
+1 Thanks, Timo On 23.10.20 10:21, Jingsong Li wrote: +1 On Fri, Oct 23, 2020 at 3:52 PM Konstantin Knauf wrote: +1 On Fri, Oct 23, 2020 at 9:36 AM Jark Wu wrote: +1 On Fri, 23 Oct 2020 at 15:25, Shengkai Fang wrote: Hi, all, I would like to start the vote for FLIP-149[1], which is

[RESULT][VOTE] FLIP-136: Improve interoperability between DataStream and Table API

2020-11-04 Thread Timo Walther
(binding) Best, Jingsong On Thu, Sep 24, 2020 at 4:18 PM Kurt Young wrote: +1 (binding) Best, Kurt On Thu, Sep 24, 2020 at 4:01 PM Timo Walther wrote: Hi all, after the discussion in [1], I would like to open a second voting thread for FLIP-136 [2] which covers different topic to

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-11-09 Thread Timo Walther
Hi Jark, thanks for the deep investigation and communication with Calcite and Beam folks. Given the new findings, +1 to vote. Regards, Timo On 09.11.20 05:22, Jark Wu wrote: Hi all, After some offline discussion and investigation with Timo and Danny, I have updated the FLIP-145. FLIP-145:

Re: [VOTE] FLIP-145: Support SQL windowing table-valued function (2nd)

2020-11-11 Thread Timo Walther
+1 (binding) Thanks, Timo On 11.11.20 07:14, Pengcheng Liu wrote: +1 (binding) Jark Wu 于2020年11月11日周三 上午10:13写道: +1 (binding) On Tue, 10 Nov 2020 at 14:59, Jark Wu wrote: Hi all, There is new feedback on the FLIP-145. So I would like to start a new vote for FLIP-145 [1], which has be

Re: [DISCUSS] Make SQL docs Blink only

2020-12-08 Thread Timo Walther
Hi Seth, this is a very good idea. We might not be able to remove the legacy planner immediately but at least we can make the docs easier for current and future users of the Blink planner. Making the SQL docs Blink-only with a dedicated legacy planner page sounds good to me. Regards, Timo

Re: [VOTE] Release 1.11.3, release candidate #2

2020-12-16 Thread Timo Walther
+1 (binding) I manually scanned the commit diff and checked for incompatible changes: - checked for changed @Public(Evolving) signatures - checked for SQL plan changes - checked for newly introduced features What I found a bit concerning is the large number of commits around the new source API

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Timo Walther
Hi Dian, thanks for the proposed FLIP. I haven't taken a deep look at the proposal yet but will do so shortly. In general, we should aim to make the Table API as concise and self-explaining as possible. E.g. `dropna` does not sound obvious to me. Regarding `myTable.coalesce($("a"), 1).as("a"

Re: [VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread Timo Walther
+1 (binding) Thanks for driving this. Regards, Timo On 21.06.21 13:24, Ingo Bürk wrote: Hi everyone, thanks for all the feedback so far. Based on the discussion[1] we seem to have consensus, so I would like to start a vote on FLIP-129 for which the FLIP has now also been updated[2]. The vote

[DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Timo Walther
Hi everyone, I'm sending this email to make sure everyone is on the same page about slowly deprecating the DataSet API. There have been a few thoughts mentioned in presentations, offline discussions, and JIRA issues. However, I have observed that there are still some concerns or different op

Re: 回复:[DISCUSS] [FLINK-23122] Provide the Dynamic register converter

2021-06-24 Thread Timo Walther
Hi Jack, thanks for sharing your proposal with us. I totally understand the issues that you are trying to solve. Having a more flexible type support in the connectors is definitely a problem that we would like to address in the mid term. It is already considered in on our internal roadmap pla

[ANNOUNCE] The term "blink" has been removed from the code base

2021-07-06 Thread Timo Walther
Hi everyone, as discussed previously [1] and tracked in FLINK-14437, we executed the last step of FLIP-32 and removed all occurences of the term "blink" in the code base. This includes renaming the following Maven modules: flink-table-planner-blink -> flink-table-planner flink-table-runtime-

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-07-07 Thread Timo Walther
Thanks for writing this up, this also reflects my understanding. I think a blog post would be nice, ideally with an explicit call for feedback so we learn about user concerns. A blog post has a lot more reach than an ML thread. Best, Stephan On Wed, Jun 23, 2021 at 12:23 PM Timo Walther wro

Re: [DISCUSS] Feedback Collection Jira Bot

2021-07-08 Thread Timo Walther
.@apache.org wrote: Hi Timo, Thanks for joining the discussion. All rules except the unassigned rule do not apply to Sub-Tasks actually (like deprioritization, closing). Additionally, activity on a Sub-Taks counts as activity for the parent. So, the parent ticket would not be t

Re: [VOTE] Release flink-shaded 14.0, release candidate 1

2021-07-19 Thread Timo Walther
+1 (binding) I went through all commits one more time and could not spot anything that would block a release. Thanks Chesnay! Timo On 15.07.21 09:02, Chesnay Schepler wrote: Hi everyone, Please review and vote on the release candidate #1 for the version 14.0, as follows: [ ] +1, Approve t

Re: Introduction email

2021-07-19 Thread Timo Walther
Hi Srini, welcome aboard! Great to see more adoption in the SQL space. Looking forward to collaboration. Regards, Timo On 19.07.21 10:58, Till Rohrmann wrote: Hi Srini, Welcome to the Flink community :-) Great to hear what you are planning to do with Flink at LinkedIn. I think sharing this

Re: Incompatible RAW types in Table API

2021-08-09 Thread Timo Walther
Hi Dominik, `toAppendStream` is soft deprecated in Flink 1.13 and will be deprecated in Flink 1.13. It uses the old type system and might not match perfectly with the other reworked type system in new functions and sources. For SQL, a lot of Avro classes need to be treated as RAW types. But w

Re: Incompatible RAW types in Table API

2021-08-09 Thread Timo Walther
Sorry, I meant "will be deprecated in Flink 1.14" On 09.08.21 19:32, Timo Walther wrote: Hi Dominik, `toAppendStream` is soft deprecated in Flink 1.13 and will be deprecated in Flink 1.13. It uses the old type system and might not match perfectly with the other reworked type sys

<    1   2   3   4   5   6   7   8   9   10   >