Re: flink 1.10 support LONG as watermark?

2020-03-31 Thread jingjing bai
Thanks a lot! Jark Wu 于2020年4月1日周三 上午1:13写道: > Hi Jing, > > I created https://issues.apache.org/jira/browse/FLINK-16889 to support > converting from BIGINT to TIMESTAMP. > > Best, > Jark > > On Mon, 30 Mar 2020 at 20:30, jingjing bai > wrote: > >> Hi

Re: flink 1.10 support LONG as watermark?

2020-03-30 Thread jingjing bai
; ); > > Here my_func is a UDF which converts BIGINT into TIMESTAMP(3). > > Best, > Jark > > On Mon, 30 Mar 2020 at 18:16, jingjing bai > wrote: > >> >> Hi: >> flinkers! >> >> I try to upgrade our production to 1.10V from 1.9 which is our

flink 1.10 support LONG as watermark?

2020-03-30 Thread jingjing bai
Hi: flinkers! I try to upgrade our production to 1.10V from 1.9 which is our current product version. in our case,the event_time is Long ,and we had implement this function which support long type as a watermark in our inner version, it is a different from the official version on 1.10. on 1.10

Re: The assigned slot bae00218c818157649eb9e3c533b86af_11 was removed

2019-12-25 Thread jingjing bai
tm挂掉了,可以看下是否存在checkpoint连续失败导致OOM, 或者是大数据集大窗口运算,如果数据量大也会导致这个问题。 Xintong Song 于2019年12月25日周三 上午10:28写道: > 这个应该不是root cause,slot was removed通常是tm挂掉了导致的,需要找下对应的tm日志看下挂掉的原因。 > > Thank you~ > > Xintong Song > > > > On Tue, Dec 24, 2019 at 10:06 PM hiliuxg <736742...@qq.com> wrote: > > >

Re: java.lang.NoClassDefFoundError: org/apache/kafka/common/serialization/ByteArrayDeserializer

2019-12-24 Thread jingjing bai
Typically , *NoClassDefFoundError* is caused by jar conflicts , it means, there are two same class and not same version jar in your classpath, I suggest you can check which jar is this class from ,and then to confirm is there more too than one jar in u classpath. if is and remove unuse jar .

Re: CEP匹配乱序数据的问题

2019-12-24 Thread jingjing bai
CEP的 sql中order by , 不会因为乱序导致不匹配。 在api中没用过,可以看看是否有对应的api qishang zhong 于2019年12月23日周一 下午9:37写道: > HI,大家好。 > > 咨询一个问题,flink-training-exercises练习的工程里面 > com.ververica.flinktraining.solutions.datastream_java.cep.LongRidesSolution > > Pattern completedRides = > Pattern.begin("start") > .where(new

Re: Flink task node shut it self off.

2019-12-20 Thread jingjing bai
hi john in our experience , the checkpoint interval we set interval 1-10 minute and timeout usurally 5*interval . mostly we set 2 or 5 minute and 10 or 20timeout. it depend on u data bulk per second and which window used. John Smith 于2019年12月21日周六 上午5:26写道: > Hi, using Flink 1.8.0 > > 1st

Re: flink如何动态修改窗口大小和类型?

2019-12-18 Thread jingjing bai
目前一个任务中,仅支持一种窗口。 动态修改本身应该是一个伪需求 如果你仅仅是为了方便快速开发,建议你用sql方式去开发,比如自己扩展sql client。 LakeShen 于2019年12月18日周三 下午2:12写道: > 使用自定义 Trigger 试试,在 Trigger 中自定义业务触发逻辑。 > > 陈帅 于2019年12月14日周六 下午6:44写道: > > > flink目前支不支持动态修改窗口大小和类型呢?例如先是统计5分钟窗口,而后修改成统计10分钟窗口. > > >

Re: FlinkSQL中关于TIMESTAMPDIFF函数官网E.g的疑问

2019-12-16 Thread jingjing bai
不知道你的版本是什么, 在1.9中, 申明为TIMESTAMP类型的属性,需要是 格式化为-MM-DD'T'HH:mm:ss.SSS'Z' 不过你可以从外部传入13位时间戳,也可以转换成TIMESTAMP,比如DDL中定义 CREATE TABLE `t` ( ctm TIMESTAMP, ) WITH ( 'format.schema' = 'ROW' )" 。如果数据源也要定义为TIMESTAMP类型,则通过下面方式去定义外部数据源格式: DateTimeFormatter t = new DateTimeFormatterBuilder()