Re: RFC: Control flink upsert sink’s memory usage of insertedRowMap

2023-12-10 Thread OpenInx
https://github.com/apache/iceberg/pull/2680/files On Mon, Dec 11, 2023 at 11:15 AM OpenInx wrote: > Just provided a little context: there was a stale PR which was trying to > maintain the insertedRowMap into RocksDB.. > > On Sat, Dec 9, 2023 at 1:52 AM Ryan Blue wrote: > >> Thanks, Renjie! >> >

Re: RFC: Control flink upsert sink’s memory usage of insertedRowMap

2023-12-10 Thread OpenInx
Just provided a little context: there was a stale PR which was trying to maintain the insertedRowMap into RocksDB.. On Sat, Dec 9, 2023 at 1:52 AM Ryan Blue wrote: > Thanks, Renjie! > > The option to use Flink's state tracking system seems like a good idea to > me. > > On Thu, Dec 7, 2023 at 8:1

Re: Proposal for REST APIs for Iceberg table scans

2023-12-10 Thread Ryan Blue
Rahil, thanks for working on this. It has some really good ideas that we hadn't considered before like a way for the service to plan how to break up the work of scan planning. I really like that idea because it makes it much easier for the service to keep memory consumption low across requests. My