deepakpanda93 commented on issue #5451: URL: https://github.com/apache/hudi/issues/5451#issuecomment-4842432832
Closing this as resolved in current Hudi. Original root cause: the AWS classes used by DynamoDBBasedLockProvider (the dynamodb-lock-client + AWS SDK) were split out of the Spark bundle in 0.10.1, so the provider couldn't load them → NoClassDefFoundError: com/amazonaws/services/dynamodbv2/model/LockNotGrantedException. Fixes since then: hudi-aws-bundle (HUDI-4011 / [#5674](https://github.com/apache/hudi/pull/5674)) packages the lock client + AWS SDK, so those jars no longer need to be passed manually. Available since 0.12.0 and in all 1.x releases. On EMR/Glue, enabling --datalake-formats hudi pulls the right jars. The lock provider was migrated to AWS SDK v2 ([HUDI-6638 / #9347](https://github.com/apache/hudi/pull/9347), since 0.14.0 and all 1.x), so the SDK-v1 class in the original stack trace is no longer on the code path. The config confusion (the hoodie.write.lock.dynamodb.partition_key is the value; the table's key attribute name is fixed to key, and Hudi can create the table for you) was documented in [HUDI-4010 / #6752](https://github.com/apache/hudi/pull/6752) — that was the cause of the follow-up "The provided key element does not match the schema" error. The original report was confirmed resolved back in 2022, and DynamoDB-based locking was later verified working on 0.12.2/master and Glue 3.0/4.0. If anyone still hits this on a recent 1.x release using hudi-aws-bundle (or --datalake-formats hudi on Glue/EMR), please reopen with the version, the jars on the classpath, and a minimal repro. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
