Fabio Batista da Silva created ARROW-5317: ---------------------------------------------
Summary: [Rust] [Parquet] impl IntoIterator for SerializedFileReader Key: ARROW-5317 URL: https://issues.apache.org/jira/browse/ARROW-5317 Project: Apache Arrow Issue Type: Improvement Components: Rust Reporter: Fabio Batista da Silva This is a follow up to [https://github.com/apache/arrow/issues/4301]. The current implementation of a row iterator *RowIter* borrows the *FileReader* which the user has to keep the file reader alive for as long as the iterator is alive.. And make is hard to iterate over multiple *FileReader* / *RowIter*.. {code:java} fn main() { let path1 = Path::new("path-to/1.snappy.parquet"); let path2 = Path::new("path-to/2.snappy.parquet"); let vec = vec![path1, path2]; let it = vec.iter() .map(|p| { File::open(p).unwrap() }) .map(|f| { SerializedFileReader::new(f).unwrap() }) .flat_map(|reader| -> RowIter { RowIter::from_file(None, &reader).unwrap() //| | | //| | `reader` is borrowed here //| returns a value referencing data owned by the current function }) ; for r in it { println!("{}", r); } } {code} One solution could be to implement a row iterator that takes owners of the reader. Perhaps implementing *std::iter::IntoIterator* for the *SerializedFileReader* {code:java} .... .map(|p| { File::open(p).unwrap() }) .map(|f| { SerializedFileReader::new(f).unwrap() }) .flat_map(|r| -> r) .... {code} Happy to put a PR out with this.. Please let me know if this makes sense and you guys already have some way of doing this.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)