LuciferYang opened a new pull request, #10663: URL: https://github.com/apache/paimon/pull/10663
### Purpose close #10581 With `location-in-properties`, the metastore location of a Paimon table is Hive's warehouse default and the data lives under `paimon_location`. `HiveSplitGenerator` skipped every input location that does not start with the Paimon location, so Hive read no rows from these tables. When no input location belongs to the table and the Hive table has no partition columns, that location is now planned once as the whole table, with the pushed-down filter. `PaimonStorageHandler` tells the input format whether the Hive table has partition columns, taken from `partition_columns` of the Hive table rather than from Paimon options, which a session can override. Tables with Hive partitions keep the current behavior: their partitions all share the table location, so planning it would read every partition once per Hive partition. The skip was added in #5891 for Hive dummy files. Hive does not create those for storage-handler tables, so for a table without Hive partitions a foreign input location is the table's own metastore location. ### Tests - `HiveCatalogITCaseBase#testHiveReadLocationInProperties` (Hive 2.3 and 3.1): with `location-in-properties`, Hive reads an unpartitioned table and a table partitioned only in Paimon, with filters and through `count(*)`. On a table with Hive partitions it returns no row under the wrong partition. On master the first query returns no rows. - `HiveSplitGeneratorTest#testForeignLocation`: a foreign location is planned as the whole table only when the Hive table has no partition columns. ### API and Format no ### Documentation no Generated-by: Claude Opus -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
