twuebi commented on code in PR #948:
URL: https://github.com/apache/iceberg-go/pull/948#discussion_r3161186427
##########
table/write_records.go:
##########
@@ -52,6 +54,37 @@ func WithWriteUUID(id uuid.UUID) WriteRecordOption {
}
}
+// WithMaxWriteWorkers overrides the default number of fanout workers
+// used for partitioned writes. Each worker processes record batches,
+// partitions them, and writes to the appropriate partition files.
+// Fewer workers means fewer concurrent parquet writers compressing
+// pages simultaneously, which reduces peak memory. A value of 0
+// (the default) uses [config.EnvConfig.MaxWorkers].
+//
+// This option is ignored when [WithClusteredWrite] is set.
+func WithMaxWriteWorkers(n int) WriteRecordOption {
Review Comment:
added an error return & updated docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
