Hello
This is my first setup with Apache Beam and trying to use the flink runner
for it. Following the example of
https://github.com/apache/flink-kubernetes-operator/tree/main/examples/flink-beam-example
I got my app running inside my local minikube. The app is pretty stateless
- minimal
Hi,
connector里面配置的主键控制是写入存储的,有些存储在写入数据的时候,能根据主键自动更新去重
所以我感觉,你这里的应该是想在计算的时候shuffle(写入之前),你应该需要先执行一个 group by 主键,然后再执行insert into
Thanks
在 2023-02-17 15:56:51,"casel.chen" 写道:
>作业场景是kafka cdc数据源关联几张redis维表再和其他流表进行双流regular inner
>join,最后将打宽表写入mongodb。使用的是flink 1.13.2 sql模式。开了debug日志。
Hi Zhipeng
IterativeStreams does have keyBy() methods, but they all throw
UnsupportedOperationException [1]
For some context: the whole thing is to do message enrichment with asyncIO,
caching the enrichment info in state (with TTL).
I am using an iteration as RichAsyncFunction does not support
Hi Lorenzo,
Could you provide some code example to reproduce your question? As I
understand, IterativeStream#keyBy is supported since it is a subclass
of DataStream.
Moreover, we have implemented an unified iteration module for Flink
[1] in Flink ML [2], which relies on Flink 1.15.2. Probably
Hi all,
I am trying to implement an iterative streaming job that processes the loop
with a KeyedProcessFunction.
I need a KeyedProcessFunction to use keyed state and to emit a side-output
(that after further transformations becomes the feedback)
Problem is IterativeStream.process() only accepts