Copilot commented on code in PR #1601:
URL: https://github.com/apache/arrow-go/pull/1601#discussion_r4607745687
##########
parquet/file/column_reader.go:
##########
@@ -707,19 +708,23 @@ func (c *columnChunkReader) skipRows(nrows int64) (int64,
error) {
} else {
// with repetition levels, we have to check them to
determine
// how many rows to skip. we can't just skip the number
of values
- // because there could be multiple values per row. So
we read in
- // the repetition levels for the entire page at once
and then go
- // through them to find the right row.
- repLvls := c.getRepLvlBuffer(c.numBuffered)
- nreps, _, err := c.repetitionDecoder.Decode(repLvls)
- if err != nil {
- c.err = err
- return nrows - toSkip, err
+ // because there could be multiple values per row. So
we decode the
+ // rest of the page's repetition levels once, at their
page
+ // positions, and then go through them to find the
right row.
+ if len(c.repLvlBuffer) == 0 {
+ buf := c.getRepLvlBuffer(c.numBuffered)
+ nreps, _, err :=
c.repetitionDecoder.Decode(buf[c.numDecoded:])
Review Comment:
This assumes the repetition decoder has advanced by `numDecoded`, but
`determineNumToRead` does not advance it when `ReadBatch` receives nil
repetition output (which the public API permits). For a page containing rows
`[10]`, `[20]`, `[30,31]`, calling `ReadBatch(1, ..., nil)`, then `Skip(1)`,
then reading the last two values with repetition output now returns `[0,0]`
instead of `[0,1]`: the decoder's page prefix is written into the cache's
suffix. Keep the repetition decoder aligned even when repetition output is
omitted, or track its position separately before populating this cache, and add
this sequence to the regression tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]