Re:Re: [DISCUSS] FLIP-415: Introduce a new join operator to support minibatch

2024-01-14 Thread Xuyang
Hi, shuai. Thanks for this explaination. This scenario sounds reasonable to me. I agree that we need to split the behavior in minibatch into two types of options: 1. Whether to open minibatch to save batch data; 2. Whether to compress the changelog data while saving the batch, and merge the

Re:Re: [DISCUSS] FLIP-415: Introduce a new join operator to support minibatch

2024-01-11 Thread Xuyang
Hi, Xu Shuai. Thanks for driving this flip. The CDC message amplification of cascade join has always been a problem for users. Judging from the nexmark results, this optimization is very meaningful. I just have the same doubts as Benchao, why can't we use minibatch join as the default