deepakpanda93 commented on issue #10249: URL: https://github.com/apache/hudi/issues/10249#issuecomment-4845858453
It's a Hadoop version mismatch in the CLI setup, not a Hudi bug. In your stack, hadoop-common-2.10.1 is on the classpath (the CLI was built with the default Hadoop 2.10.x), but you added hadoop-aws-3.2.2.jar. The missing class org.apache.hadoop.fs.store.EtagChecksum is a Hadoop 3.x class — it's referenced by hadoop-aws 3.2.2 but doesn't exist in hadoop-common 2.10.1, hence the NoClassDefFoundError. Two ways to fix it: - Use the Hudi CLI bundle (available since 0.13.0, and in all 1.x): launch via packaging/hudi-cli-bundle/hudi-cli-with-bundle.sh, which packages a consistent dependency set so you don't have to hand-add Hadoop/AWS jars. - If you assemble the classpath yourself, make hadoop-common and hadoop-aws the same Hadoop major version — e.g., use hadoop-aws 2.10.x to match the default hadoop-common 2.10.x, or build the CLI with the Hadoop 3.x profile and use hadoop-aws 3.x. Recommend moving to a recent 1.x release and using the CLI bundle. If you still hit a class-loading error on 1.x with the bundled CLI, please share your exact classpath and we'll help. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
