codeAnqiang-ma opened a new issue, #10676:
URL: https://github.com/apache/arrow-rs/issues/10676
### Describe the bug
Same as #10382 (csv) and #10613 (ipc file reader), but for the ipc stream
reader.
When a projection is set, `StreamReader::schema()` and its
`RecordBatchReader::schema()` impl return the full stream schema, while the
batches the reader yields have the projected schema. #10627 fixed `FileReader`,
`StreamReader` has the same problem.
`StreamReader::try_new` already computes the projected schema and stores it
in the private `projection` field, but that value is never read back.
### To Reproduce
```rust
// arrow-ipc/src/reader.rs (mirrors
test_file_reader_projected_schema_matches_batch_schema)
#[test]
fn test_stream_reader_projected_schema_matches_batch_schema() {
let schema = create_test_projection_schema();
let batch = create_test_projection_batch_data(&schema);
let mut buf = Vec::new();
{
let mut writer = crate::writer::StreamWriter::try_new(&mut buf,
&schema).unwrap();
writer.write(&batch).unwrap();
writer.finish().unwrap();
}
let projection = vec![3, 2, 1];
let mut reader = StreamReader::try_new(Cursor::new(buf),
Some(projection)).unwrap();
let reader_schema = RecordBatchReader::schema(&reader);
let read_batch = reader.next().unwrap().unwrap();
assert_eq!(reader_schema, read_batch.schema());
}
```
On `eaf21ed` this fails with a 14 column schema on the left (the full stream
schema) and a 3 column schema on the right (the projected batch schema):
```
assertion `left == right` failed
left: Schema { fields: [f0 UInt32, f1 Utf8, f2 Boolean, f3 Union(..), f4
Null, f5 Float64,
f6 List, f7 FixedSizeBinary(3), f8 FixedSizeList, f9 Struct, f10
RunEndEncoded,
f11 Boolean, f12 Dictionary(Int8, Utf8), f13 Utf8], metadata: {} }
right: Schema { fields: [f3 Union(..), f2 Boolean, f1 Utf8], metadata: {} }
```
The same projection against `FileReader` passes, so the two IPC readers
currently disagree.
### Expected behavior
`RecordBatchReader::schema` should match the schema of the output batches
https://github.com/apache/arrow-rs/blob/eaf21ed4d4740c019148f436e89cd928963777a0/arrow-array/src/record_batch.rs#L27-L36
### Additional context
Found by auditing the remaining `RecordBatchReader` impls after #10613 was
fixed. This report was prepared with AI assistance; I ran the reproduction
locally and reviewed the result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
