klion26 commented on PR #6965:
URL: https://github.com/apache/arrow-rs/pull/6965#issuecomment-2706039110
Thanks for the contribution.
I have some question after apply this patch when using the following code
```
#[test]
fn test_contet() {
let file = "/path/to/56f216f3-7e24-40b0-a76a-87a63a5bc254-m0.avro";
let rb: RecordBatch = read_file(file, 1024, false);
println!("{:?}", rb);
}
```
encounter the exception, not sure is it the check too strict?
```
called `Result::unwrap()` on an `Err` value:
InvalidArgumentError("Inconsistent struct child length for field #3. Expected
1, got 0")
thread 'reader::test::test_contet' panicked at
arrow-avro/src/reader/mod.rs:125:25:
called `Result::unwrap()` on an `Err` value:
InvalidArgumentError("Inconsistent struct child length for field #3. Expected
1, got 0")
stack backtrace:
0: rust_begin_unwind
at
/rustc/ee612c45f00391aff71ec0c52b7fc08fae18c711/library/std/src/panicking.rs:665:5
1: core::panicking::panic_fmt
at
/rustc/ee612c45f00391aff71ec0c52b7fc08fae18c711/library/core/src/panicking.rs:76:14
2: core::result::unwrap_failed
at
/rustc/ee612c45f00391aff71ec0c52b7fc08fae18c711/library/core/src/result.rs:1699:5
3: core::result::Result<T,E>::unwrap
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:1104:23
4: arrow_avro::reader::test::read_file
at ./src/reader/mod.rs:125:9
5: arrow_avro::reader::test::test_contet
at ./src/reader/mod.rs:131:31
6: arrow_avro::reader::test::test_contet::{{closure}}
```
after comment the check logic in `records.rs Decoder#flush` for there will
throw some other exception
```
called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Incorrect
array length for StructArray field \"partition\", expected 1 got 0")
thread 'reader::test::test_contet' panicked at
arrow-array/src/array/struct_array.rs:91:46:
called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Incorrect
array length for StructArray field \"partition\", expected 1 got 0")
stack backtrace:
0: rust_begin_unwind
at
/rustc/ee612c45f00391aff71ec0c52b7fc08fae18c711/library/std/src/panicking.rs:665:5
1: core::panicking::panic_fmt
at
/rustc/ee612c45f00391aff71ec0c52b7fc08fae18c711/library/core/src/panicking.rs:76:14
2: core::result::unwrap_failed
at
/rustc/ee612c45f00391aff71ec0c52b7fc08fae18c711/library/core/src/result.rs:1699:5
3: core::result::Result<T,E>::unwrap
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:1104:23
4: arrow_array::array::struct_array::StructArray::new
at
/Users/qiucongxian/arrow-rs/arrow-array/src/array/struct_array.rs:91:9
5: arrow_avro::reader::record::Decoder::flush
at ./src/reader/record.rs:450:28
6: arrow_avro::reader::record::RecordDecoder::flush::{{closure}}
at ./src/reader/record.rs:83:22
7: core::iter::adapters::map::map_try_fold::{{closure}}
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/adapters/map.rs:95:28
8: core::iter::traits::iterator::Iterator::try_fold
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:2370:21
9: <core::iter::adapters::map::Map<I,F> as
core::iter::traits::iterator::Iterator>::try_fold
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/adapters/map.rs:121:9
10: <core::iter::adapters::GenericShunt<I,R> as
core::iter::traits::iterator::Iterator>::try_fold
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/adapters/mod.rs:191:9
11: core::iter::traits::iterator::Iterator::try_for_each
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:2431:9
12: <core::iter::adapters::GenericShunt<I,R> as
core::iter::traits::iterator::Iterator>::next
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/adapters/mod.rs:174:14
13: alloc::vec::Vec<T,A>::extend_desugared
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs:3518:35
14: <alloc::vec::Vec<T,A> as
alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/spec_extend.rs:19:9
15: <alloc::vec::Vec<T> as
alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/spec_from_iter_nested.rs:42:9
16: <alloc::vec::Vec<T> as
alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/spec_from_iter.rs:34:9
17: <alloc::vec::Vec<T> as
core::iter::traits::collect::FromIterator<T>>::from_iter
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs:3410:9
18: core::iter::traits::iterator::Iterator::collect
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:1971:9
19: <core::result::Result<V,E> as
core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:1980:51
20: core::iter::adapters::try_process
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/adapters/mod.rs:160:17
21: <core::result::Result<V,E> as
core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:1980:9
22: core::iter::traits::iterator::Iterator::collect
at
/Users/qiucongxian/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:1971:9
23: arrow_avro::reader::record::RecordDecoder::flush
at ./src/reader/record.rs:80:22
24: arrow_avro::reader::test::read_file
```
the file used to test is as following, please remove the ".txt" suffix
[56f216f3-7e24-40b0-a76a-87a63a5bc254-m0.avro.txt](https://github.com/user-attachments/files/19124467/56f216f3-7e24-40b0-a76a-87a63a5bc254-m0.avro.txt)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
