hknlof commented on issue #13323:
URL: https://github.com/apache/datafusion/issues/13323#issuecomment-3131126145
This still is happening with DataFusion 49. Using `.parquet` suffix in
output str, aligns to expected behavior of this issue.
```rust
use datafusion::{
dataframe::DataFrameWriteOptions,
prelude::{ParquetReadOptions, SessionContext},
};
#[tokio::main]
async fn main() -> datafusion::error::Result<()> {
let ctx = SessionContext::new();
let options = ParquetReadOptions::new();
let df = ctx
.read_parquet("./data/yellow_tripdata_2025-01.parquet", options)
.await?;
let clone_df = df.clone();
df.write_parquet(
"./data/new_yellow_caps.parquet", // pre-serve the shape of the issue
DataFrameWriteOptions::new().with_single_file_output(true),
None,
)
.await?;
clone_df
.write_parquet(
"./data/new_yellow_caps", // pre-serve the shape of the issue
DataFrameWriteOptions::new().with_single_file_output(true),
None,
)
.await?;
Ok(())
}
```
```console
hknlof@localhost data_samples % find data -type f | sort
data/new_yellow_caps.parquet
data/new_yellow_caps/kWLvaFapgf6OoXk5_0.parquet
data/new_yellow_caps/kWLvaFapgf6OoXk5_1.parquet
data/new_yellow_caps/kWLvaFapgf6OoXk5_2.parquet
data/new_yellow_caps/kWLvaFapgf6OoXk5_3.parquet
data/yellow_tripdata_2025-01.parquet
```
```toml
# Cargo.toml
[package]
name = "data_samples"
version = "0.1.0"
edition = "2024"
[dependencies]
datafusion = "49.0.0"
tokio = { version = "1.47.0", features = ["rt-multi-thread", "tokio-macros"]
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
