deepakpanda93 commented on PR #19551: URL: https://github.com/apache/hudi/pull/19551#issuecomment-5217008039
Correcting something in my own earlier change here, found while testing the sibling troubleshooting page against Hive 3.1.3. The note I added in 5aabd164fb13 said the `metaconf:` form "works from Hive or Beeline only". The Beeline half is right; **the Hive CLI half is wrong**, and I've fixed it in 0e432984016c. Measured against a standalone HMS 3.1.3, using `ALTER TABLE ... REPLACE COLUMNS (id int, age int, name string, ts bigint)` on a table that starts as `(id, name, ts)`: | Client | plain `set …=false` | `set metaconf:…=false` | |---|---|---| | Beeline (HiveServer2) | :x: fails | :white_check_mark: **succeeds** — `describe` returns `id, age, name, ts` | | Hive CLI | :x: fails | :x: fails | The Hive CLI case is the interesting one, because it does not fail silently at the protocol level. Reading the value back mid-session shows the metastore accepted it: ``` hive> set metaconf:hive.metastore.disallow.incompatible.col.type.changes; metaconf:hive.metastore.disallow.incompatible.col.type.changes=true hive> set metaconf:hive.metastore.disallow.incompatible.col.type.changes=false; hive> set metaconf:hive.metastore.disallow.incompatible.col.type.changes; metaconf:hive.metastore.disallow.incompatible.col.type.changes=false ``` …and the very next `ALTER` in that same session is still rejected with `The following columns have types incompatible with the existing columns in their respective positions : age`. `setMetaConf` binds to the metastore connection that received it (`HMSHandler` keeps it in a thread-local conf), and the CLI's DDL path does not run on that connection. A raw `HiveMetaStoreClient` doing `setMetaConf` then `alter_table` on one client object does work, which is consistent. So the override is real but connection-scoped, and only HiveServer2 reliably gives you one session on one connection. The note now says so and folds the Hive CLI in with Spark SQL: use the `hive-site.xml` option instead. Nothing else in the PR changes — the embedded-vs-remote split and the `spark.hadoop.` and `hive-site.xml` remedies are unaffected, and the build is still clean with no new warnings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
