Hi all,

Currently, for Fluss PrimaryKey Table, the number of kv snapshots retained
per bucket is controlled by the server
option `kv.snapshot.num-retained`  (default value: 1). If this value is set
too small, Kv snapshots that are being actively consumed may be deleted
while a consumer is still consuming them.

This case will cause a Flink job which read PrimaryKey table fail and
cannot be restarted from its previous state.

To avoid this case, the fluss server needs to be aware of which consumers
are actively consuming the corresponding kv snapshots, and can not delete
these kv snapshots that are currently being consumed.

So, I'd like to propose FIP-22: Support Kv Snapshot Consumer[1].

Any feedback are suggestions on this proposal are welcome!

[1]:
https://cwiki.apache.org/confluence/display/FLUSS/FIP-22+Support+Kv+Snapshot+Consumer

Regards,
Yunhong

Reply via email to