yihua commented on code in PR #9746:
URL: https://github.com/apache/hudi/pull/9746#discussion_r1330974362
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java:
##########
@@ -146,21 +146,12 @@ public class HoodieCompactionConfig extends HoodieConfig {
+ "compaction during each compaction run. By default. Hudi picks the
log file "
+ "with most accumulated unmerged data");
- public static final ConfigProperty<String> COMPACTION_LAZY_BLOCK_READ_ENABLE
= ConfigProperty
- .key("hoodie.compaction.lazy.block.read")
- .defaultValue("true")
- .markAdvanced()
- .withDocumentation("When merging the delta log files, this config helps
to choose whether the log blocks "
- + "should be read lazily or not. Choose true to use lazy block
reading (low memory usage, but incurs seeks to each block"
- + " header) or false for immediate block read (higher memory
usage)");
-
- public static final ConfigProperty<String>
COMPACTION_REVERSE_LOG_READ_ENABLE = ConfigProperty
- .key("hoodie.compaction.reverse.log.read")
- .defaultValue("false")
- .markAdvanced()
- .withDocumentation("HoodieLogFormatReader reads a logfile in the forward
direction starting from pos=0 to pos=file_length. "
- + "If this config is set to true, the reader reads the logfile in
reverse direction, from pos=file_length to pos=0");
+ // These config properties are kept to make sure defaults are properly
propagated
+ public static final ConfigProperty<String> COMPACTION_LAZY_BLOCK_READ_ENABLE
=
Review Comment:
The propagation of default values are done using Java reflections. See
`HoodieCompactionConfig$Builder#build`
```
public HoodieCompactionConfig build() {
compactionConfig.setDefaults(HoodieCompactionConfig.class.getName());
return compactionConfig;
}
```
which checks the declared fields as below:
```
protected void setDefaults(String configClassName) {
Class<?> configClass = ReflectionUtils.getClass(configClassName);
Arrays.stream(configClass.getDeclaredFields())
.filter(f -> Modifier.isStatic(f.getModifiers()))
.filter(f -> f.getType().isAssignableFrom(ConfigProperty.class))
.forEach(f -> {
try {
ConfigProperty<?> cfgProp = (ConfigProperty<?>) f.get("null");
if (cfgProp.hasDefaultValue() || cfgProp.hasInferFunction()) {
setDefaultValue(cfgProp);
}
} catch (IllegalAccessException e) {
e.printStackTrace();
}
});
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
