hcrosse opened a new pull request, #1585:
URL: https://github.com/apache/arrow-go/pull/1585

   ### Rationale for this change
   
   Typed `Skip` methods, such as `Int32ColumnChunkReader.Skip`, return the 
requested count even when an error or the end of the column stops them early. 
On a column with 5 rows, `Skip(20)` returns 20 and a nil error, and a further 
`Skip(1)` returns 1.
   
   I reproduced the end of column case on v18.8.0 and main, with Go 1.27.4 on 
macOS arm64.
   
   ### What changes are included in this PR?
   
   `skipRows` now returns the number of rows that it skipped, and all typed 
`Skip` methods return that count. After an error on a repeated column, the 
count can include a row that continues onto the page that failed. `SeekToRow` 
ignores the count.
   
   For columns without repetition levels, the shorter count at the end of a 
column matches C++:
   
   
https://github.com/apache/arrow/blob/976779b2aac587f51b87c3bd2e6ae4c36f75ab6c/cpp/src/parquet/column_reader.cc#L1458-L1481
   
   ### Are these changes tested?
   
   This change adds count checks to `TestSkipStopsAtCorruptPage` and 
`TestSkipEmptyRepeatedRows`, and adds `TestSkipPastEndOfColumn` and 
`TestSkipStopsAtDecodeError`. All four pass with this change, and all but 
`TestSkipEmptyRepeatedRows` fail on main. `TestSkipEmptyRepeatedRows` passes on 
main because it requests exactly as many rows as it skips.
   
   Full `go test -race -tags assert,test ./parquet/...` and `golangci-lint` 
2.13.2 passed on macOS arm64 with pinned test data.
   
   ### Are there any user-facing changes?
   
   Yes. `Skip` returns a shorter count when it stops early. There is no API 
change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to