techgopal opened a new issue, #11689:
URL: https://github.com/apache/arrow-rs/issues/11689
### Describe the bug
Reading a `FIXED_LEN_BYTE_ARRAY` column whose data page contains a
dictionary index that is out of range for the dictionary page panics instead of
returning an error.
`ValueDecoder::read` in
`parquet/src/arrow/array_reader/fixed_len_byte_array.rs` slices the dictionary
buffer with `key * byte_length` without checking the key against the number of
dictionary entries:
```rust
let offset = *key as usize * self.byte_length;
let val = &dict.as_ref()[offset..offset + self.byte_length];
```
`RleDecoder::get_batch_with_dict` returns an error for out-of-range indices
(#9434), but this path indexes the dictionary directly.
### To Reproduce
With `parquet = "60.0.0"`, `arrow-array = "60.0.0"`, `arrow-schema =
"60.0.0"`, `bytes = "1"`:
```rust
use std::sync::Arc;
use arrow_array::{FixedSizeBinaryArray, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
use bytes::Bytes;
use parquet::arrow::arrow_reader::ParquetRecordBatchReaderBuilder;
use parquet::arrow::ArrowWriter;
use parquet::file::properties::{WriterProperties, WriterVersion};
fn main() {
// A FIXED_LEN_BYTE_ARRAY(4) column: 50 x "aaaa" then 50 x "bbbb", so the
// data page holds dictionary indices as two RLE runs (0 then 1).
let values: Vec<&[u8]> = (0..100).map(|i| if i < 50 { b"aaaa" } else {
b"bbbb" } as &[u8]).collect();
let array =
FixedSizeBinaryArray::try_from_iter(values.into_iter()).unwrap();
let schema = Arc::new(Schema::new(vec![Field::new("c",
DataType::FixedSizeBinary(4), false)]));
let batch = RecordBatch::try_new(schema.clone(),
vec![Arc::new(array)]).unwrap();
let mut buf = Vec::new();
let props = WriterProperties::builder()
.set_writer_version(WriterVersion::PARQUET_2_0) // dictionary for
FLBA
.set_dictionary_enabled(true)
.build();
let mut writer = ArrowWriter::try_new(&mut buf, schema,
Some(props)).unwrap();
writer.write(&batch).unwrap();
writer.close().unwrap();
// Data page values: bit width 1, RLE run (50 x 0), RLE run (50 x 1).
// Change the bit width to 8 and the second run's index to 255; the
// dictionary only has 2 entries.
let pattern = [0x01, 0x64, 0x00, 0x64, 0x01];
let pos = buf.windows(pattern.len()).position(|w| w ==
pattern).expect("pattern");
buf[pos] = 8;
buf[pos + 4] = 0xFF;
let reader = ParquetRecordBatchReaderBuilder::try_new(Bytes::from(buf))
.unwrap()
.build()
.unwrap();
for batch in reader {
println!("{:?}", batch.map(|b| b.num_rows()));
}
}
```
Output (debug and release):
```
thread 'main' panicked at
parquet-60.0.0/src/arrow/array_reader/fixed_len_byte_array.rs:497:49:
range start index 1020 out of range for slice of length 8
```
Backtrace (abridged):
```
<parquet::arrow::array_reader::fixed_len_byte_array::ValueDecoder as
ColumnValueDecoder>::read::{{closure}}
parquet::arrow::decoder::dictionary_index::DictIndexDecoder::read
<parquet::arrow::array_reader::fixed_len_byte_array::ValueDecoder as
ColumnValueDecoder>::read
parquet::column::reader::GenericColumnReader<R,D,V>::read_records_with_reservation
...
```
### Expected behavior
The reader returns a `ParquetError` (e.g. "dictionary index out of bounds")
instead of panicking.
### Additional context
- Reproduces with `parquet` 60.0.0 and on `main` at 33d338ddf (panic at
`fixed_len_byte_array.rs:432`).
- Found while fuzzing a Parquet reader built on arrow-rs. Related: #9434,
#9742.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
