Hi, OpenInx:
Yes, I've read that pr. But if we want to maintain the out of memory map, I
lean towards using flink's operator state, which has already provided high
level api for us, including the type serialization, configuration, etc. It
would be easier to maintain by reusing what flink has provi
Hi, Ryan:
Yes, but as I have stated in the doc, this may make simple ingestion jobs
more difficult to maintain. What's your opinion about the second approach?
On Mon, Dec 11, 2023 at 11:16 AM OpenInx wrote:
> https://github.com/apache/iceberg/pull/2680/files
>
> On Mon, Dec 11, 2023 at 11:15 AM
https://github.com/apache/iceberg/pull/2680/files
On Mon, Dec 11, 2023 at 11:15 AM OpenInx wrote:
> Just provided a little context: there was a stale PR which was trying to
> maintain the insertedRowMap into RocksDB..
>
> On Sat, Dec 9, 2023 at 1:52 AM Ryan Blue wrote:
>
>> Thanks, Renjie!
>>
>
Just provided a little context: there was a stale PR which was trying to
maintain the insertedRowMap into RocksDB..
On Sat, Dec 9, 2023 at 1:52 AM Ryan Blue wrote:
> Thanks, Renjie!
>
> The option to use Flink's state tracking system seems like a good idea to
> me.
>
> On Thu, Dec 7, 2023 at 8:1
Thanks, Renjie!
The option to use Flink's state tracking system seems like a good idea to
me.
On Thu, Dec 7, 2023 at 8:19 PM Renjie Liu wrote:
> Hi:
> I want to raise a discussion about controlling flink's upsert sink's
> memory usage:
>
> https://toys-flash-4hl.craft.me/3VHrdWbV30QMk6
>
> Welc
Hi:
I want to raise a discussion about controlling flink's upsert sink's memory
usage:
https://toys-flash-4hl.craft.me/3VHrdWbV30QMk6
Welcome to comment and share your thoughts.
