Re: Incorrect checkpoint id used when job is recovering

2022-05-13 Thread tao xiao
Hi team, Does anyone have any ideas? On Thu, May 12, 2022 at 9:20 PM tao xiao wrote: > Forgot to mention the Flink version is 1.13.2 and we use kubernetes native > mode > > On Thu, May 12, 2022 at 9:18 PM tao xiao wrote: > >> Hi team, >> >> I met a weird issue when a job tries to recover from

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-13 Thread Austin Cawley-Edwards
Hi all, Would just like to share an interesting article from the dbt community[1], which in part describes some of their challenges in managing Slack in a large community. The biggest point it seems to make is that their Slack has become a marketing tool for dbt/data vendors instead of a

Re:回复:DataStream 转换 Table 后事件时间不对

2022-05-13 Thread 赢峰
没有任何变化 在 2022-05-13 21:52:51,"renzhaojin" 写道: >hello,你加上设置时区试试呢 >Configuration configuration = tableEnvironment.getConfig().getConfiguration(); > >configuration.setString("table.local-time-zone", "Asia/Shanghai"); > > >在 2022年5月13日 21:40,赢峰 写道: > > >DataStream 转换 Table 后事件时间怎么不对?是时区的问题吗?怎么解决? ```

回复:DataStream 转换 Table 后事件时间不对

2022-05-13 Thread renzhaojin
hello,你加上设置时区试试呢 Configuration configuration = tableEnvironment.getConfig().getConfiguration(); configuration.setString("table.local-time-zone", "Asia/Shanghai"); 在 2022年5月13日 21:40,赢峰 写道: DataStream 转换 Table 后事件时间怎么不对?是时区的问题吗?怎么解决? ``` DataStream sourceStream = env.fromElements( new

DataStream 转换 Table 后事件时间不对

2022-05-13 Thread 赢峰
DataStream 转换 Table 后事件时间怎么不对?是时区的问题吗?怎么解决? ``` DataStream sourceStream = env.fromElements( new UserBehavior(1001L, 3827899L, 2920476L, "pv", 1511713473000L, "2017-11-27 00:24:33"), new UserBehavior(1001L, 3745169L, 2891509L, "pv", 1511725471000L, "2017-11-27 03:44:31"),

Re: Converting from table to stream, following Avro schema

2022-05-13 Thread Dhavan Vaidya
Hi Dian, This is great news, really! Hope to see this soon in stable :D Thanks a lot! On Fri, 13 May 2022 at 15:25, Dian Fu wrote: > Hi Dhavan, > > This should be a known issue and has already been fixed in > https://issues.apache.org/jira/browse/FLINK-27282. However, this ticket > was fixed

Re: Converting from table to stream, following Avro schema

2022-05-13 Thread Dian Fu
Hi Dhavan, This should be a known issue and has already been fixed in https://issues.apache.org/jira/browse/FLINK-27282. However, this ticket was fixed recently and so it's still not released. Regards, Dian On Thu, May 12, 2022 at 7:26 PM Dhavan Vaidya wrote: > Hey Dian, > > Though my HTTP

Re: Join two data stream using DataStream API using PyFlink

2022-05-13 Thread Dhavan Vaidya
Hi Ian, You need to create two streams, and use https://nightlies.apache.org/flink/flink-docs-master/api/python/pyflink.datastream.html#pyflink.datastream.DataStream.connect to "join" them. On Fri, 13 May 2022 at 14:06, lan tran wrote: > Hi team, I have the use case is that I want to join two

Join two data stream using DataStream API using PyFlink

2022-05-13 Thread lan tran
Hi team, I have the use case is that I want to join two datastream that have the same id. If we convert into sql we will have something like thisSELECT suppliers.supplier_id, suppliers.supplier_name, orders.order_dateFROM suppliers INNER JOIN ordersON suppliers.supplier_id =

Re: Split string flatmap -> Arraylist/List

2022-05-13 Thread Alexander Preuß
Hi Zain, There are different options to achieve this. You can take a look at the WordCount example [1] to get some inspiration. [1] https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java Best

Split string flatmap -> Arraylist/List

2022-05-13 Thread Zain Haider Nemati
Hi, I have a comma separated string of the format x,y,z \n a,b,c ... I want to split the string on the basis of '\n' and insert each string into the data stream separately using *flatmap*. Any example on how to do that? If I add the chunks into an Arralist or List and return that from the flatmap

Re: Re: flink sql无法读取Hive映射的HBase表

2022-05-13 Thread Jingsong Li
Hi, 推荐 https://www.deepl.com/translator 非常好用 我记得对Hive Custom Storage Handler(hbase)是有问题的 Best, Jingsong On Fri, May 13, 2022 at 2:12 PM 18579099...@163.com <18579099...@163.com> wrote: > 我英文能力不允许啊 > > > > 18579099...@163.com > > 发件人: yuxia > 发送时间: 2022-05-11 15:11 > 收件人: user-zh > 主题: Re:

Re: Re: flink sql无法读取Hive映射的HBase表

2022-05-13 Thread 18579099...@163.com
我英文能力不允许啊 18579099...@163.com 发件人: yuxia 发送时间: 2022-05-11 15:11 收件人: user-zh 主题: Re: flink sql无法读取Hive映射的HBase表 不好意思,我尝试复现你的问题,但是我没有 hbase 环境,不过看起来是只有当 STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 有问题? 我之后空了再debug 看看。 不过我看了一下 flink 这块的代码,从 flink 这块的代码来看,应该是 get 这个