Got it. Thanks for clarifying.
On Fri, Apr 23, 2021 at 6:36 AM Ayush Chauhan
wrote:
> Hi Matthias,
>
> I am using RocksDB as a state backend. I think the iceberg sink is not
> able to propagate back pressure to the source which is resulting in OOM for
> my CDC pipeline.
> Please refer to this -
Hi Matthias,
I am using RocksDB as a state backend. I think the iceberg sink is not able
to propagate back pressure to the source which is resulting in OOM for my
CDC pipeline.
Please refer to this - https://github.com/apache/iceberg/issues/2504
On Thu, Apr 22, 2021 at 8:44 PM Matthias Pohl
wr
Hi Ayush,
Which state backend have you configured [1]? Have you considered trying out
RocksDB [2]? RocksDB might help with persisting at least keyed state.
Best,
Matthias
[1]
https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/state_backends.html#choose-the-right-state-backend
[2]
ht
Hi,
I am using flink cdc to stream CDC changes in an iceberg table. When I
first run the flink job for a topic which has all the data for a table, it
get out of heap memory as flink try to load all the data during my 15mins
checkpointing interval. Right now, only solution I have is to pass *-ytm
81
