Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Hi, Thanks for the PR! I'll have a look at it later today. The problem of the retraction stream conversion is probably that the return type is a Tuple2[Boolean, Row]. The boolean flag indicates whether the row is added or retracted. Best, Fabian 2018-07-04 15:38 GMT+02:00 Jungtaek Lim : >

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Thanks Fabian, filed FLINK-9742 [1]. I'll submit a PR for FLINK-8094 via providing my TimestampExtractor. The implementation is also described as FLINK-9742. I'll start with current implementation which just leverages automatic cast from STRING to SQL_TIMESTAMP, but we could improve it from PR.

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Hi, Glad you could get it to work! That's great :-) Regarding you comments: 1) Yes, I think we should make resultType() public. Please open a Jira issue and describe your use case. Btw. would you like to contribute your TimestampExtractor to Flink (or even a more generic one that allows to

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Thanks again Fabian for providing nice suggestion! Finally I got it working with applying your suggestion. Couple of tricks was needed: 1. I had to apply a hack (create new TimestampExtractor class to package org.apache.flink.blabla...) since Expression.resultType is defined as "package private"

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Hi Jungtaek, If it is "only" about the missing support to parse a string as timestamp, you could also implement a custom TimestampExtractor that works similar to the ExistingField extractor [1]. You would need to adjust a few things and use the expression "Cast(Cast('tsString,

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Thanks Chesnay! Great news to hear. I'll try out with latest master branch. Thanks Fabian for providing the docs! I guess I already tried out with KafkaJsonTableSource and failed back to custom TableSource since the type of rowtime field is string unfortunately, and I needed to parse and map to

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Hi Jungtaek, Flink 1.5.0 features a TableSource for Kafka and JSON [1], incl. timestamp & watemark generation [2]. It would be great if you could let us know, if that addresses your use case and if not what's missing or not working. So far Table API / SQL does not have support for late-data side

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

The watermark display in the UI is bugged in 1.5.0. It is fixed on master and the release-1.5 branch, and will be included in 1.5.1 that is slated to be released next week. On 04.07.2018 10:22, Jungtaek Lim wrote: Sorry I forgot to mention the version: Flink 1.5.0, and I ran the app in

Re: [Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Sorry I forgot to mention the version: Flink 1.5.0, and I ran the app in IntelliJ, not tried from cluster. 2018년 7월 4일 (수) 오후 5:15, Jungtaek Lim 님이 작성: > Hi Flink users, > > I'm new to Flink and trying to evaluate couple of streaming frameworks via > implementing same apps. > > While

[Table API/SQL] Finding missing spots to resolve why 'no watermark' is presented in Flink UI

Hi Flink users, I'm new to Flink and trying to evaluate couple of streaming frameworks via implementing same apps. While implementing apps with both Table API and SQL, I found there's 'no watermark' presented in Flink UI, whereas I had been struggling to apply row time attribute. For example,