wangxianghu commented on a change in pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#discussion_r642188609
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -72,8 +71,11 @@
public static final String PRECOMBINE_FIELD_PROP =
"hoodie.datasource.write.precombine.field";
public static final String WRITE_PAYLOAD_CLASS =
"hoodie.datasource.write.payload.class";
public static final String DEFAULT_WRITE_PAYLOAD_CLASS =
OverwriteWithLatestAvroPayload.class.getName();
+
public static final String KEYGENERATOR_CLASS_PROP =
"hoodie.datasource.write.keygenerator.class";
- public static final String DEFAULT_KEYGENERATOR_CLASS =
SimpleAvroKeyGenerator.class.getName();
+ public static final String KEYGENERATOR_TYPE_PROP =
"hoodie.datasource.write.keygenerator.type";
+ public static final String DEFAULT_KEYGENERATOR_TYPE = "simple";
Review comment:
> oops, missed something. you can't deprecate the old config. user
defined key gen will be derived from the same config.
> So, what we could do is that.
> check if KEYGENERATOR_TYPE_PROP is set. // there is no default for
KEYGENERATOR_TYPE_PROP
> else use value from KEYGENERATOR_CLASS_PROP.
> KEYGENERATOR_CLASS_PROP can have a default of SIMPLE key gen.
> Let me know if this would work.
Yes, I just wanted to report this issue to you
1. we should keep `KEYGENERATOR_CLASS_PROP` for user-defined Kergenerator
2. Convert class prop to type is not very convenient/elegant, since
`HoodieWriteConfig` is in a common module, The actual KeyGenerator is
Implemented in sub-module, we'd better not use the sub-class name in
`HoodieWriteConfig`.
Meanwhile, in the sub-module, there is no need to convert the
`KEYGENERATOR_CLASS_PROP` to `KEYGENERATOR_TYPE_PROP`, we can initialize the
Keygenerator directly using reflection.
In the current implementation, `KEYGENERATOR_CLASS_PROP` has higher priority
than `KEYGENERATOR_TYPE_PROP`, because we recommend user use
`KEYGENERATOR_TYPE_PROP` as it is simpler.
I will check the `KEYGENERATOR_CLASS_PROP` first (no default value), if it
has value, it must de setted by the user, use it to create key generator, if
not, use `KEYGENERATOR_TYPE_PROP`(default SIMPLE).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
