rich7420 commented on code in PR #6246:
URL: https://github.com/apache/datafusion-comet/pull/6246#discussion_r4536148819
##########
native/core/src/parquet/parquet_exec.rs:
##########
@@ -198,10 +376,34 @@ pub(crate) fn init_datasource_exec(
.flat_map(|s| s.fields().iter())
.map(|f| Arc::new(Field::new(f.name(), f.data_type().clone(),
f.is_nullable())))
.collect();
- let table_schema = TableSchema::builder(base_schema)
- .with_table_partition_cols(partition_fields)
+ let output_table_schema = TableSchema::builder(Arc::clone(&base_schema))
+ .with_table_partition_cols(partition_fields.clone())
+ .build();
+ // A scan that projects no string column has nothing to read as Binary, so
it keeps the
+ // projection-indices path below.
+ let binary_as_string = binary_as_string
+ && required_schema
+ .fields()
+ .iter()
+ .any(|field| is_binary_as_string_read(field.data_type()));
+ let read_base_schema = if binary_as_string {
+ binary_as_string_read_schema(&base_schema)
Review Comment:
Sorry, I got the scope wrong. An unannotated BINARY column with a Utf8 or
Dictionary<Utf8> ARROW:schema hint panics in debug builds, and main does too
with binaryAsString off, so it isn't from this PR (same root cause as
apache/arrow-rs#11140). I'm withdrawing this request. Repro on main:
https://github.com/rich7420/datafusion-comet/commit/424ab31dd96b4cb2f3663fa6c164d1b072cfe27e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
