fallintoplace opened a new pull request, #1780: URL: https://github.com/apache/iceberg-go/pull/1780
## Summary - snapshot the current schema, table properties, and name mapping once per Arrow scan - build projected read field IDs once before file workers start - reuse the scan-owned values across file reads and record batch projection - keep the defensive metadata getter behavior unchanged ## Benchmark `BenchmarkArrowScanManyFilesAndBatches` scans 32 logical files with 32 one-row batches per file, 16 nested struct fields, and 4 workers. These are medians from five local runs on an Apple M1 Pro. | Extra properties | Before | After | Bytes/op | Allocs/op | | --- | ---: | ---: | ---: | ---: | | 0 | 110.5 ms | 94.1 ms | 259.0 MB -> 257.9 MB | 2,077,360 -> 2,071,319 | | 16 | 103.1 ms | 89.2 ms | 259.9 MB -> 257.9 MB | 2,079,328 -> 2,071,319 | | 64 | 92.6 ms | 86.7 ms | 263.8 MB -> 257.9 MB | 2,079,395 -> 2,071,143 | The timing is noisier than the allocation measurements. The 64-property case removes about 5.9 MB and 8.2k allocations per scan operation. ## Tests - `go test ./... -count=1` - `go test -race ./table -run TestArrowScanSnapshotsInvariantsOnce\|TestWriteRecordsRecoversExactProjJSONCRSOnRead -count=1` - `go vet ./table/...` - `golangci-lint run --timeout=10m ./table/...` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
