adriangb opened a new issue, #12182: URL: https://github.com/apache/arrow-rs/issues/12182
**Describe the bug** The IPC writer writes the full child values of a sliced `ListViewArray` or `LargeListViewArray`. A slice of 10 rows from a large list view costs about as much as the whole parent. The writer already narrows other sliced types. `List`, `LargeList` and `Map` rebase their offsets and slice the child. Sparse `Union` slices each child to the parent slice. `RunEndEncoded` unslices the run array. For list views, the writer slices the offsets and sizes buffers but passes the child through unchanged. **To Reproduce** Write a `ListViewArray` with `StreamWriter`, then write a 10-row slice of it. | Array | Whole stream (bytes) | 10-row slice (bytes) | | --- | --- | --- | | ListView<UInt32>, 100,000 rows of 3 values | 2,012,792 | 1,200,368 | | ListView<Utf8>, 10,000 rows | 237,128 | 155,960 | | ListView<ListView<UInt32>>, 10,000 rows | 681,168 | 599,992 | **Expected behavior** The writer narrows the child to the values that the slice references and rebases the offsets. Views can overlap or be out of order, so the range runs from the smallest offset to the largest `offset + size` of the non-empty views in the slice. The data reads back logically equal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
