alamb commented on code in PR #2027:
URL: https://github.com/apache/arrow-rs/pull/2027#discussion_r917001366
##########
parquet/src/arrow/record_reader/mod.rs:
##########
@@ -731,4 +739,48 @@ mod tests {
assert_eq!(5000, record_reader.num_values());
}
}
+
+ #[test]
+ fn test_row_group_boundary() {
Review Comment:
This test passes on master, without the changes in this PR 🤔
##########
parquet/src/arrow/arrow_reader.rs:
##########
@@ -1528,4 +1528,57 @@ mod tests {
assert_eq!(total_rows, expected_rows);
}
+
+ #[test]
+ fn test_row_group_exact_multiple() {
Review Comment:
When I remove the code in this PR this test fails like the following (I was
expecting the reader to miss the last record)
```
---- arrow::arrow_reader::tests::test_row_group_exact_multiple stdout ----
thread 'arrow::arrow_reader::tests::test_row_group_exact_multiple' panicked
at 'called `Result::unwrap()` on an `Err` value: ParquetError("Parquet error:
Not all children array length are the same!")',
parquet/src/arrow/arrow_reader.rs:1581:53
stack backtrace:
0: rust_begin_unwind
at
/rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/panicking.rs:584:5
1: core::panicking::panic_fmt
at
/rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/panicking.rs:142:14
2: core::result::unwrap_failed
at
/rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/result.rs:1785:5
3: core::result::Result<T,E>::unwrap
at
/rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/result.rs:1078:23
4: parquet::arrow::arrow_reader::tests::test_row_group_exact_multiple
at ./src/arrow/arrow_reader.rs:1581:23
5:
parquet::arrow::arrow_reader::tests::test_row_group_exact_multiple::{{closure}}
at ./src/arrow/arrow_reader.rs:1533:5
6: core::ops::function::FnOnce::call_once
at
/rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/ops/function.rs:248:5
7: core::ops::function::FnOnce::call_once
at
/rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose
backtrace.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
