twuebi opened a new pull request, #948: URL: https://github.com/apache/iceberg-go/pull/948
Keeps at most one partition writer open at a time, mirroring iceberg-java's ClusteredWriter. Revisiting a closed partition returns an error. Wired into RewriteDataFiles since each compaction group is single-partition, drops the fanout writer's per-worker in-flight batches and redundant compute.Take copies. Also exposes WithMaxWriteWorkers to tune fanout concurrency for the unclustered path. Sorts partitionMapNode.collectPartitions by first-row index so within-batch processing follows input row order. Required by the clustered writer; the fanout writer is unaffected. Fixes a pre-existing leak in partitionedFanoutWriter.processRecord where compute.Take sub-records were retained by Add but never released locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
