alamb commented on issue #27089: URL: https://github.com/apache/datafusion/issues/27089#issuecomment-7009558156
> @alamb on the SQL repro, I took 2 files, shredded differently: So that wrong result is tracked by this ticket I think - https://github.com/datafusion-contrib/datafusion-variant/issues/177 selecting from the directory gets the wrong results ```sql > SELECT event:id, event:status FROM 'mixed_events/' ORDER BY event:id::BIGINT; +------------------------+----------------------------+ | mixed_events/.event:id | mixed_events/.event:status | +------------------------+----------------------------+ | 1 | 200 | | 2 | 500 | | 3 | 504 | | 4 | null | | 5 | null | | 6 | null | +------------------------+----------------------------+ ``` But first creating an external table, gets the right results ``` > CREATE EXTERNAL TABLE mixed_events STORED AS PARQUET LOCATION 'mixed_events/'; 0 row(s) fetched. Elapsed 0.008 seconds. > SELECT event:id, event:status FROM mixed_events ORDER BY event:id::BIGINT; +-----------------------+---------------------------+ | mixed_events.event:id | mixed_events.event:status | +-----------------------+---------------------------+ | 1 | 200 | | 2 | 500 | | 3 | 504 | | 4 | 200 | | 5 | 503 | | 6 | 200 | +-----------------------+---------------------------+ 6 row(s) fetched. Elapsed 0.007 seconds. ``` I think the issue is that the dynamic table provider that supports querying from a directory doesn't use the schema adapter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
