codope commented on code in PR #5630:
URL: https://github.com/apache/hudi/pull/5630#discussion_r917262903
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieMultiTableDeltaStreamer.java:
##########
@@ -457,4 +457,4 @@ public Set<String> getFailedTables() {
public List<TableExecutionContext> getTableExecutionContexts() {
return this.tableExecutionContexts;
}
-}
+}
Review Comment:
nit: a new line
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -542,9 +542,12 @@ public static final Config getConfig(String[] args) {
public static void main(String[] args) throws Exception {
final Config cfg = getConfig(args);
Map<String, String> additionalSparkConfigs =
SchedulerConfGenerator.getSparkSchedulingConfigs(cfg);
- JavaSparkContext jssc =
- UtilHelpers.buildSparkContext("delta-streamer-" + cfg.targetTableName,
cfg.sparkMaster, additionalSparkConfigs);
-
+ JavaSparkContext jssc = null;
+ if (cfg.sparkMaster.isEmpty()) {
Review Comment:
Use `StringUtils.isNullOrEmpty`
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -844,4 +847,4 @@ public TypedProperties getProps() {
public DeltaSyncService getDeltaSyncService() {
return deltaSyncService.get();
}
-}
+}
Review Comment:
nit: new line
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -326,8 +326,8 @@ public static class Config implements Serializable {
description = "the min sync interval of each sync in continuous mode")
public Integer minSyncIntervalSeconds = 0;
- @Parameter(names = {"--spark-master"}, description = "spark master to
use.")
- public String sparkMaster = "local[2]";
+ @Parameter(names = {"--spark-master"}, description = "spark master to use,
if not defined inherits (e.g. not using spark-summit command).")
Review Comment:
nit: `spark-submit`
Also, it's not very clear from the message, what does it inherit from? Would
be good to mention the default port and compression coded at least.
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java:
##########
@@ -554,4 +572,4 @@ public static String
getSchemaFromLatestInstant(HoodieTableMetaClient metaClient
Schema schema = schemaResolver.getTableAvroSchema(false);
return schema.toString();
}
-}
+}
Review Comment:
nit: newline at the end of file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
