ShivamSharma43 opened a new pull request, #1590:
URL: https://github.com/apache/arrow-go/pull/1590

   ### Rationale for this change
   
   The IPC writer fails on any list view or large list view array that has a 
non-zero offset:
   
   ```go
   arr, _, _ := array.FromJSON(mem, 
arrow.ListViewOf(arrow.PrimitiveTypes.Int32),
        strings.NewReader(`[[1, 2], [3], null, [4, 5, 6]]`))
   slice := array.NewSlice(arr, 1, 4)
   // writing a record batch that holds slice:
   // arrow/ipc: unknown error while writing: arrow/array: index out of range
   ```
   
   `RecordBatch.NewSlice` produces arrays like this, so a sliced record with a 
list view column can't be written to IPC or Flight.
   
   `getZeroBasedListViewOffsets` searches for the smallest offset in the slice. 
It starts that search from `(^OffsetT(0)) << (bits - 1)`, which is the 
*minimum* int32/int64 value, although the comment says maximum. The minimum 
therefore never changes, and the values child is sliced starting at 
`math.MinInt32` (or `math.MinInt64`), which panics.
   
   ### What changes are included in this PR?
   
   The bounds now come from the raw offsets and sizes of every view in the 
slice, null and empty views included. Arrow C++ does the same in 
[`GetZeroBasedListViewOffsets`](https://github.com/apache/arrow/blob/main/cpp/src/arrow/ipc/writer.cc).
 The search starts from the first view instead of a sentinel.
   
   The previous loop used `ValueOffsets(i)`, which returns `(0, 0)` for empty 
views. That loop would not be enough even with the right starting value: an 
empty view whose offset lies past the other views would be rebased to a 
position outside the values that are written. For example, slicing only the 
empty view out of `[[1, 2], [3], null, [], ...]` would write offset 3 with no 
values. Using every raw offset keeps all written offsets within the values.
   
   ### Are these changes tested?
   
   Yes. `TestWriteSlicedListView` writes every slice of a `list_view` and a 
`large_list_view` array that contains null and empty views. It reads each slice 
back and checks that the result passes `ValidateFull`, equals the input, and 
has every offset within the written values. A third case builds a list view 
whose offsets are out of order. The test uses a checked allocator. On `main`, 
every slice with a non-zero offset fails with the error above. With only the 
starting value fixed, the offset bound check fails. Everything passes with this 
change.
   
   `go test ./arrow/ipc/...` passes, as do runs with `-tags assert,test -short` 
and with `-tags assert,test,noasm -short`. `go vet`, gofmt and golangci-lint 
(staticcheck) report nothing for the changed package.
   
   ### Are there any user-facing changes?
   
   Yes, a bug fix. Sliced list view and large list view arrays can now be 
written with the IPC writer.
   
   AI assistance: I used Claude Code (Anthropic) to find this bug and write the 
fix and the test. I reviewed the code and ran the tests locally.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to