rdettai commented on a change in pull request #8910:
URL: https://github.com/apache/arrow/pull/8910#discussion_r543331995
##########
File path: rust/datafusion/src/logical_plan/builder.rs
##########
@@ -90,77 +75,51 @@ impl LogicalPlanBuilder {
options: CsvReadOptions,
projection: Option<Vec<usize>>,
) -> Result<Self> {
- let schema: Schema = match options.schema {
- Some(s) => s.to_owned(),
- None => CsvFile::try_new(path, options)?
- .schema()
- .as_ref()
- .to_owned(),
- };
-
- let projected_schema = projection
- .map(|p| Schema::new(p.iter().map(|i|
schema.field(*i).clone()).collect()))
- .map_or(SchemaRef::new(schema), SchemaRef::new)
- .to_dfschema_ref()?;
-
let provider = Arc::new(CsvFile::try_new(path, options)?);
- let schema = provider.schema();
-
- let table_scan = LogicalPlan::TableScan {
- schema_name: "".to_string(),
- source: TableSource::FromProvider(provider),
- table_schema: schema,
- projected_schema,
- projection: None,
- };
-
- Ok(Self::from(&table_scan))
+ Self::scan("", provider, projection)
}
/// Scan a Parquet data source
pub fn scan_parquet(path: &str, projection: Option<Vec<usize>>) ->
Result<Self> {
let provider = Arc::new(ParquetTable::try_new(path)?);
+ Self::scan("", provider, projection)
+ }
+
+ /// Scan an empty data source, mainly used in tests
+ pub fn scan_empty(
+ name: &str,
Review comment:
This is what I did originally, to match the other similar methods in the
block (`scan_csv`, `scan_memory`...). But this one is used by many tests in the
codebase, and often they are using names, so we would have needed to edit
asserts all over the place. I find that it does not do much harm to leave the
it here, but if there is a consensus on the fact that its better removed, I
don't mind!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
