zeroshade commented on code in PR #1221:
URL: https://github.com/apache/arrow-go/pull/1221#discussion_r3883692122
##########
arrow/ipc/file_writer.go:
##########
@@ -256,6 +256,7 @@ type FileWriter struct {
codec flatbuf.CompressionType
compressNP int
compressors []compressor
+ encoder *recordEncoder
Review Comment:
Please likewise clear `f.encoder` when the file writer closes. A
4,096-column probe retained capacities of 4,096 fields and 8,192 metadata
entries after close; a 100,000-column probe retained approximately 6.15 MiB
until the writer became unreachable.
##########
arrow/ipc/writer.go:
##########
@@ -96,6 +96,7 @@ type Writer struct {
codec flatbuf.CompressionType
compressNP int
compressors []compressor
+ encoder *recordEncoder
Review Comment:
This cached encoder retains its peak-capacity metadata scratch after
`Close()` and close-after-failure. Please clear `w.encoder` on every terminal
close path so a reachable closed stream writer does not retain potentially
large scratch arrays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
