zeroshade commented on code in PR #1606:
URL: https://github.com/apache/arrow-go/pull/1606#discussion_r4617921270
##########
parquet/file/column_reader.go:
##########
@@ -644,8 +644,12 @@ func (c *columnChunkReader) determineNumToRead(batchLen
int64, defLvls, repLvls
// finds the page that contains the desired row index, and the Column Chunk
// reader will discard values until it reaches the desired row index according
// to the definition and repetition levels.
+//
+// If the seek fails, the reader stays stopped and returns the error until a
+// later SeekToRow succeeds.
func (c *columnChunkReader) SeekToRow(rowIdx int64) error {
if err := c.pager().SeekToPageWithRow(rowIdx); err != nil {
+ c.err = err
Review Comment:
This also stops the reader when `SeekToPageWithRow` rejects the row index
(`rowIdx < 0 || rowIdx >= p.nrows`). That check runs before the page reader
releases its page or moves the stream, so the reader is still in a valid state,
and stopping it penalises a caller who only passed a bad argument.
On main, `ReadBatch(5)`, then `SeekToRow(1000)`, then `ReadBatch(3)` returns
`[5 6 7]` from the last read. With this change the last read returns the seek
error.
pqarrow isn't affected, because `leafReader.SeekToRow` checks the row with
`FindChunkForRow` first. Direct users of `ColumnChunkReader` are.
Could you keep the error only once the page reader has moved? For example,
have the range check in `SeekToPageWithRow` return a sentinel error that
`SeekToRow` returns without storing. The `index < 0` "seek out of range" error
on the offset-index path comes after the current page is released, so that one
should still stop the reader. The new doc comment and the description's
user-facing note would need the same adjustment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]