Re: Flink write ADLS show error: No FileSystem for scheme "file"

2024-06-26 Thread Xiao Xu
Hi, Gabar, Thanks to reply, I make sure that not conflict in maven, all the hadoop dependency is in provided scope, and checked my result jar it not contains (src/main/resources/META-INF/services). This is my pom: http://maven.apache.org/POM/4.0.0;

Flink k8s operator starts wrong job config from FlinkSessionJob

2024-06-26 Thread Peter Klauke
Hi all, we're running a session cluster and I submit around 20 jobs to it at the same time by creating FlinkSessionJob Kubernetes resources. After sufficient time there are 20 jobs created and running healthy. However, it appears that some jobs are started with the wrong config. As a result some

Re: Flink write ADLS show error: No FileSystem for scheme "file"

2024-06-26 Thread Gabor Somogyi
Hi Xiao, I'm not quite convinced that the azure plugin ruined your workload, I would take a look at the dependency graph you've in the pom. Adding multiple deps can conflict in terms of class loader services (src/main/resources/META-INF/services). As an example you've 2 such dependencies where

Re: Re:cdc读取oracle数据如何解析

2024-06-26 Thread Yanquan Lv
可以的,通过设置 debezium 的 decimal.handling.mode [1] 参数可以实现你的需求,转成 double 或者 string 来处理。 [1] https://debezium.io/documentation/reference/1.9/connectors/oracle.html#oracle-numeric-types

??????????

2024-06-26 Thread wjw_bigd...@163.com
| ?? | <402987...@qq.com.INVALID> | | | 2024??06??26?? 16:38 | | ?? | user-zh | | ?? | | | | ?? | ---- ??:

回复:退订

2024-06-26 Thread 费文杰
在 2024-06-26 15:07:45,"15868861416" <15868861...@163.com> 写道: >你好,可以把ID和PRICE的类型改为NUMBER试一下,我这边flink-sql试过number类型对应到iceberg的decimal数据是正常的 > > >| | >博星 >| >| >15868861...@163.com >| > > > 回复的原邮件 >| 发件人 | Yanquan Lv | >| 发送日期 | 2024年06月26日 14:46 | >| 收件人 | | >| 主题 | Re:

回复: cdc读取oracle数据如何解析

2024-06-26 Thread 15868861416
你好,可以把ID和PRICE的类型改为NUMBER试一下,我这边flink-sql试过number类型对应到iceberg的decimal数据是正常的 | | 博星 | | 15868861...@163.com | 回复的原邮件 | 发件人 | Yanquan Lv | | 发送日期 | 2024年06月26日 14:46 | | 收件人 | | | 主题 | Re: 回复:cdc读取oracle数据如何解析 | 你好,你的 ID 和 PRINCE 字段类型是 decimal 吗,decimal 默认的展示方式是使用 BASE64 处理

Flink write ADLS show error: No FileSystem for scheme "file"

2024-06-26 Thread Xiao Xu
Hi, all I try to use Flink to write Azure Blob Storage which called ADLS, I put the flink-azure-fs-hadoop jar in plugins directory and when I start my write job it shows: Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "file" at

Re: 回复:cdc读取oracle数据如何解析

2024-06-26 Thread Yanquan Lv
你好,你的 ID 和 PRINCE 字段类型是 decimal 吗,decimal 默认的展示方式是使用 BASE64 处理 可以通过添加下面代码来让展示信息更直观。 Map customConverterConfigs = new HashMap<>(); customConverterConfigs.put(JsonConverterConfig.DECIMAL_FORMAT_CONFIG, "numeric"); JsonDebeziumDeserializationSchema schema = new