tustvold commented on issue #7284: URL: https://github.com/apache/arrow-rs/issues/7284#issuecomment-2721628180
As arrow-rs is designed to be embedded in various different environments, it makes no assumptions about runtime environment, instead providing the raw primitives for people to use as appropriate. For reading parquet, row groups can be decoded in parallel. See https://docs.rs/parquet/latest/parquet/arrow/arrow_reader/struct.ArrowReaderBuilder.html#method.new_with_metadata For writing parquet see https://docs.rs/parquet/latest/parquet/arrow/arrow_writer/struct.ArrowColumnWriter.html These can be used with a threadpool like rayon or similar. If you'd prefer a more batteries included experience I would recommend looking at a fully fledged query engine, such as DataFusion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
