rangareddy commented on issue #5372: URL: https://github.com/apache/hudi/issues/5372#issuecomment-5101016748
This issue was reviewed as part of the JIRA-migrated backlog triage. Findings: This is fixed -- HBase has been removed from Hudi entirely, so the version check that produced this error can no longer be triggered by a Hudi bundle. Root cause: Hudi shipped HBase as a dependency because the metadata table's HFile reader/writer was built on it, and that pulled in an `hbase-default.xml` pinned to 2.4.9. `HBaseConfiguration.checkDefaultsVersion` then threw `hbase-default.xml file seems to be for an older version of HBase (2.4.9), this version is 2.1.0-cdh6.3.x` on any cluster running an older HBase. The same dependency is what made building Hudi against CDH HBase 2.1.0 fail on the missing `HFileInfo` class, as reported on 2023-04-12. The fix was to replace HBase's HFile implementation with a native one and drop the dependency: - #10241 -- [HUDI-7170] Implement HFile reader independent of HBase (merged 2024-01-18) - #13160 -- [MINOR] Use native file reader by default (merged 2025-04-18) - #12866 -- [HUDI-9012] Implement and utilize native HFile writer (merged 2025-05-25) Verified against the release tags by counting HBase references in the build files -- still present in 1.0.0, fully gone from 1.1.0 onward: | file | `release-0.14.0` | `release-1.0.0` | `release-1.1.1` | `master` | | --- | --- | --- | --- | --- | | `hudi-common/pom.xml` | 8 | 8 | 0 | 0 | | `packaging/hudi-spark-bundle/pom.xml` | - | 4 | 0 | 0 | | `packaging/hudi-hadoop-mr-bundle/pom.xml` | - | 1 | 0 | 0 | | `hudi-client/hudi-spark-client/pom.xml` | - | 4 | 0 | 0 | The reader is now `hudi-common/src/main/java/org/apache/hudi/core/io/storage/HoodieNativeAvroHFileReader.java`. `SparkHoodieHBaseIndex` is also gone and `HBASE` has been removed from `HoodieIndex.IndexType` (`HoodieIndex.java:161`), so no code path in current Hudi loads HBase configuration at all -- including the `hudi-hadoop-mr-bundle` in Hive's `auxlib` case reported on 2022-05-09. If you are on a version before 1.1.0 and cannot upgrade, the documented workaround remains `hbase.defaults.for.version.skip=true` -- see the [FAQ entry](https://hudi.apache.org/docs/faq#how-can-i-resolve-the-runtimeexception-saying-hbase-defaultxml-file-seems-to-be-for-an-older-version-of-hbase). On 1.1.0+ no workaround is needed. Closing as fixed. If you still hit an HBase version conflict on 1.1.0 or later, please reopen with the stack trace -- it would mean HBase is arriving from somewhere other than Hudi. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
