andygrove commented on code in PR #7435:
URL: https://github.com/apache/arrow-datafusion/pull/7435#discussion_r1312210705
##########
datafusion/core/src/dataframe.rs:
##########
@@ -2292,4 +2365,41 @@ mod tests {
Ok(())
}
+
+ #[tokio::test]
+ async fn write_parquet_with_compression() -> Result<()> {
+ let test_df = test_table().await?;
+ let tmp_dir = TempDir::new()?;
+ let local = Arc::new(LocalFileSystem::new_with_prefix(&tmp_dir)?);
+ let local_url = Url::parse("file://local").unwrap();
+ let ctx = &test_df.session_state;
+ ctx.runtime_env().register_object_store(&local_url, local);
+
+ let output_path = "file://local/test.parquet";
+ test_df
+ .write_parquet(
+ output_path,
+ DataFrameWriteOptions::new().with_single_file_output(true),
+ Some(
+ WriterProperties::builder()
+ .set_compression(parquet::basic::Compression::SNAPPY)
Review Comment:
The test looks good. I wonder if it is worth looping over a list of
supported compressions rather than just testing this with one codec? If the
default changed to `SNAPPY` in the future then this test would not really be
testing that the WriterProperties value is respected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
