linliu-code commented on code in PR #13656:
URL: https://github.com/apache/hudi/pull/13656#discussion_r2277143405
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestCOWDataSource.scala:
##########
@@ -2065,6 +2065,66 @@ class TestCOWDataSource extends
HoodieSparkClientTestBase with ScalaAssertionSup
assertEquals(count, 0)
}
+ @ParameterizedTest
+ @MethodSource(Array("provideParamsForKeyGenTest"))
+ def testUserProvidedKeyGeneratorClass(keyGenClass: String,
+ keyGenType: String): Unit = {
Review Comment:
This is not the expected one for some special cases. For example, if the
keyGenClass is empty, Its default value is used.
Second thoughts, I think I could change the name.
I leave it for now; after your review, I will fix it before the merge.
##########
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/util/SparkKeyGenUtils.scala:
##########
@@ -52,7 +52,8 @@ object SparkKeyGenUtils {
* @return partition column names only, concatenated by ","
*/
def getPartitionColumns(KeyGenClassNameOption: Option[String], props:
TypedProperties): String = {
- val keyGenerator = if (KeyGenClassNameOption.isEmpty) {
+ val keyGenerator = if (KeyGenClassNameOption.isEmpty
+ ||
KeyGeneratorType.USER_PROVIDED.getClassName.equals(KeyGenClassNameOption.get)) {
Review Comment:
Removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
