alamb opened a new issue, #6968: URL: https://github.com/apache/arrow-rs/issues/6968
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** - Part of https://github.com/apache/arrow-rs/issues/3287 We are contemplating making the arrow IPC reader/writer faster by allowing the user to opt out of validation but currently have no way to test the validation **Describe the solution you'd like** To make sure this actually improves performance we should have benchmarks for the ipc reader/writer **Describe alternatives you've considered** For benchmarks, what I would recommend is add two new benches: - `arrow-rs/arrow-ipc/benches/ipc_reader.rs` - `arrow-rs/arrow-ipc/benches/ipc_writer.rs` We can following the existing example from parquet like this: - https://github.com/apache/arrow-rs/blob/main/parquet/benches/arrow_writer.rs - https://github.com/apache/arrow-rs/blob/main/parquet/benches/arrow_reader.rs So someone would run them like ```shell cargo bench --bench ipc_reader ``` The actual benchmarks I would recommend starting with two sets of data: A record batch with primitive arrays (Int32Array, UInt64 and Float64Array) for example Then adding tests for 1. [`StreamWriter`](https://docs.rs/arrow-ipc/54.0.0/arrow_ipc/writer/struct.StreamWriter.html) (how fast can the data be serialized to a stream) 2. [`FileWriter`](https://docs.rs/arrow-ipc/54.0.0/arrow_ipc/writer/struct.FileWriter.html) 3. [`StreamReader`](https://docs.rs/arrow-ipc/54.0.0/arrow_ipc/reader/struct.StreamReader.html) (how fast can serialized data be read back) 4. [`FileReader`](https://docs.rs/arrow-ipc/54.0.0/arrow_ipc/reader/struct.FileReader.html) With the basic foundation, we can then **Additional context** Inspired by @totoroyyb - https://github.com/apache/arrow-rs/pull/6938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
