flink-connector和flink-sql-connector的区别

2022-04-21 Thread weishishuo...@163.com
cdc项目中每种connector都分成flink-connector-xxx和flink-sql-connector-xxx,比如flink-connector-mysql-cdc和flink-sql-connector-mysql-cdc,这两个的区别是什么呢?在什么场景下用前者,什么场景下用后者? weishishuo...@163.com

Re: flink-connector和flink-sql-connector的区别

2022-04-21 Thread Shengkai Fang
hi sql jar 往往是 shade 了相关的依赖,而 普通的 jar 则不带有相关的依赖。正如名字所说,在 table api/sql 的情况下建议使用 sql jar,datastream 建议使用 普通的jar。 Best, Shengkai weishishuo...@163.com 于2022年4月21日周四 16:52写道: > > cdc项目中每种connector都分成flink-connector-xxx和flink-sql-connector-xxx,比如flink-connector-mysql-cdc和flink-sql-connector-mysql-

flink添加表的comment信息无效

2022-04-21 Thread 草莓
Java代码如下: @Test public void test(){ StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env); DataStream dataStream = env.fromElements("Alice", "Bob", "John"); Schema.Builder builder = Schema.new

web ui中能查看到job失败的原因吗?

2022-04-21 Thread weishishuo...@163.com
我提交一个postgresql cdc 同步数据到 mysql jdbc sink的job,过了一会儿就失败了,点击job的链接,web ui界面的状态是FAILED,但是异常信息不明确 ``` 2022-04-21 17:30:50 org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandl

flink cdc 时间格式和时区问题

2022-04-21 Thread casel.chen
我在使用flink cdc 2.2.0获取mysql数据变更, mysqlSource设置了 .serverTimeZone("Asia/Shanghai") 发现mysql timestamp 类型的数据在mysql workbench里显示的是 "2021-06-24 16:26:47",通过JsonDebeziumDeserializationSchema解析后得到的json string串是 "2021-06-24T08:26:47Z"。继而在通过org.apache.flink.formats.json.JsonToRowDataConverters转成RowData时解析t

JobManager doesn't bring up new TaskManager during failure recovery

2022-04-21 Thread Zheng, Chenyu
Hi developers! I got a strange bug during failure recovery of Flink. It seems the JobManager doesn't bring up new TaskManager during failure recovery. Some logs and information of the Flink job are pasted below. Can you take a look and give me some guidance? Thank you so much! Flink version: 1

JobManager doesn't bring up new TaskManager during failure recovery

2022-04-21 Thread Zheng, Chenyu
Hi developers! I got a strange bug during failure recovery of Flink. It seems the JobManager doesn't bring up new TaskManager during failure recovery. Some logs and information of the Flink job are pasted below. Can you take a look and give me some guidance? Thank you so much! Flink version: 1