Re: 使用hbase连接器插入数据，一个列族下有多列时如何只更新其中一列

flink在写入时需要所有DDL中定义的字段都必须被同时写入，不支持sql中只使用部分字段。 如果你确定只需写入部分数据，在DDL中只定义你用到的部分 zboyu0104 于2024年6月14日周五 15:43写道： > 怎么退订 > from 阿里邮箱 > iPhone-- > 发件人：谢县东 > 日 期：2024年06月06日 16:07:05 > 收件人： > 主 题：使用hbase连接器插入数据，一个列族下有多列时如何只更新其中一列 > >

Re: Flink如何做到动态关联join多张维度表中的n张表？

lookup join可以关联多张维表，但是维表的更新不会触发历史数据刷新。 多维表关联的时候，需要考虑多次关联导致的延迟，以及查询tps对维表数据库的压力。 斗鱼 <1227581...@qq.com.invalid> 于2024年6月19日周三 23:12写道： > 好的，感谢大佬的回复，之前有了解到Flink的Lookup join好像可以实现类似逻辑，只是不知道Lookup join会不会支持多张动态维度表呢？ > > > 斗鱼 > 1227581...@qq.com > > > > > > > > >

Re:Re: Checkpoints and windows size

Hi banu: > Not all old sst files are present. Few are removed (i think it is because of > compaction). You are right, rocksdb implement delete a key by insert a entry with null value, the space will be release after compaction. > Now how can I maintain check points size under control??.

Re:Checkpoints and windows size

Hi banu, First of all, it should be noted that the checkpoint interval does not affect the state data live time of the window operator. The life cycle of state data is the same as the life cycle of the tumbling window itself. A checkpoint is a consistent snapshot of the job ( include state

Re: Flink如何做到动态关联join多张维度表中的n张表？

维表更新后要刷新历史的事实表吗？这个用flink来做的话，几乎不太可能实现，尤其是涉及到多个维表，相当于每次维表又更新了，就要从整个历史数据里面找到关联的数据，重新写入。不管是状态存储，还是更新数据量，需要的资源都太高，无法处理。 在我们目前的实时宽表应用里面，实时表部分一般都是流水类的，取到的维表信息，就应该是业务事实发生时的数据。 维表更新后刷新事实的，一般都是夜间批量再更新。如果有强实时更新需求的，只能在查询时再关联维表取最新值 王旭 于2024年6月16日周日 21:20写道： > 互相交流哈，我们也在做类似的改造 >

Re: Checkpoints and windows size

Hi Wang, Thanks a lot for your reply. Currently I have 2s window and check point interval as 10s. Minimum pass between check point is 5s. What happens is my check points size is growing gradually. I checked the content inside my rocks db local dir and also the shared checkpoints directory.

Re: A way to meter number of deserialization errors

Hi Ilya, I have not got any experience of doing this, but wonder if we could use the Flink Metrics . I wonder: - There could be hook point at that part of the code to discover some custom code that implements the metrics.

Runtime issue while using statefun-datastream v3.3.0

Hi all, We are trying to leverage statefun datastream features. After adding *org.apache.flink:statefun-flink-datastream:3.3.0* in our gradle file, we are experiencing a runtime error like this, *Caused by: java.lang.NoSuchMethodError: ‘com.google.protobuf.Descriptors$FileDescriptor

Re: A way to meter number of deserialization errors

Does anybody experience the problem of metering deserialization errors? пн, 17 июн. 2024 г. в 14:39, Ilya Karpov : > Hi all, > we are planning to use flink as a connector between kafka and > external systems. We use protobuf as a message format in kafka. If > non-backward compatible changes