Copilot commented on code in PR #1585:
URL: https://github.com/apache/arrow-go/pull/1585#discussion_r4591215014
##########
parquet/file/column_reader.go:
##########
@@ -670,13 +671,13 @@ func (c *columnChunkReader) skipRows(nrows int64) error {
ndefs, nvals, err := c.determineNumToRead(toSkip, nil,
nil)
if err != nil {
c.err = err
- return err
+ return nrows - toSkip, err
}
skipped, err := c.curDecoder.Discard(int(nvals))
if err != nil {
c.err = err
- return err
+ return nrows - toSkip, err
Review Comment:
`Discard` can skip a valid prefix before returning an error, but this return
drops that progress: on a required BYTE_ARRAY page with one valid value
followed by a truncated value, `Skip(2)` returns `(0, err)` instead of `(1,
err)`. Include the successfully discarded prefix, mapping physical values
through definition levels for optional columns. Add an exact-count regression
test with a valid prefix; the current decode-error test fails on the first
value.
##########
parquet/file/column_reader.go:
##########
@@ -729,14 +730,14 @@ func (c *columnChunkReader) skipRows(nrows int64) error {
_, err = c.curDecoder.Discard(int(valuesToSkip))
if err != nil {
c.err = err
- return err
+ return nrows - toSkip, err
Review Comment:
The repeated-column branch also drops progress within the failing page: with
repetition levels `[0, 0, 0]`, all values defined, and one valid BYTE_ARRAY
value followed by a truncated value, `Skip(2)` discards the first row but
returns `(0, err)`. Retain the count returned by `Discard` and map the consumed
prefix through definition and repetition levels to count successfully skipped
rows. Add a repeated-column regression test with a valid row before the corrupt
value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]