ShivamSharma43 opened a new pull request, #1612:
URL: https://github.com/apache/arrow-go/pull/1612

   ### Rationale for this change
   
   The IPC writer fails on a sliced sparse or dense union array when its type 
ids buffer has no spare capacity after the data:
   
   ```go
   typeIDs := memory.NewBufferBytes([]byte{0, 1, 0, 1, 0, 1, 0, 1, 0, 1})
   u := array.NewSparseUnion(arrow.SparseUnionOf(fields, codes), 10, children, 
typeIDs, 0)
   slice := array.NewSlice(u, 3, 7)
   // writing a record batch that holds slice:
   // arrow/ipc: unknown error while writing: runtime error: slice bounds out 
of range [:13] with capacity 10
   ```
   
   Arrays imported through the C Data interface have buffers like this, since 
the imported memory is exactly `(offset + length) * byteWidth` bytes long.
   
   `getTruncatedBuffer` slices the buffer starting at `offset * byteWidth`, but 
caps the length at the full `buf.Len()` instead of the bytes left after the 
start. With a non-zero offset the slice ends past the end of the buffer. When 
the buffer has spare capacity there is no panic, but the bytes past the end of 
the buffer are written along with the type ids.
   
   ### What changes are included in this PR?
   
   `getTruncatedBuffer` now caps the length at `buf.Len() - offset*byteWidth`. 
This is the same bound the fixed-width and binary view cases in `visit` already 
use. `getTruncatedBuffer` is only used for the union type ids and for the dense 
union value offsets when the offset is zero, so this one change covers both 
union types. I checked the other places in `writer.go` that slice buffers by 
offset and they already account for it.
   
   ### Are these changes tested?
   
   Yes. `TestGetTruncatedBuffer` checks the function directly on a buffer with 
no spare capacity, with byte widths 1 and 4. `TestWriteSlicedUnion` writes 
slices of a sparse and a dense union built on such buffers, reads them back, 
and checks that the result passes `ValidateFull` and equals the input, using a 
checked allocator. On `main`, `TestGetTruncatedBuffer` panics and every slice 
with a non-zero offset in `TestWriteSlicedUnion` fails with the error above. 
Both pass with this change.
   
   `go test ./arrow/ipc/...` passes, as do runs with `-tags assert,test -short` 
and with `-tags assert,test,noasm -short`. `go vet`, gofmt and golangci-lint 
report nothing for the changed package.
   
   ### Are there any user-facing changes?
   
   Yes, a bug fix. Sliced union arrays whose buffers have no spare capacity can 
now be written with the IPC writer, and the written type ids buffer no longer 
includes bytes from outside the array's buffer.
   
   AI assistance: I used Claude Code (Anthropic) to find this bug and write the 
fix and the test. I reviewed the code and ran the tests locally.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to