[
https://issues.apache.org/jira/browse/HUDI-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17380995#comment-17380995
]
ASF GitHub Bot commented on HUDI-2164:
--------------------------------------
zhangyue19921010 commented on a change in pull request #3259:
URL: https://github.com/apache/hudi/pull/3259#discussion_r670109986
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java
##########
@@ -121,17 +141,26 @@ public static void main(String[] args) {
public int cluster(int retry) {
this.fs = FSUtils.getFs(cfg.basePath, jsc.hadoopConfiguration());
int ret = UtilHelpers.retry(retry, () -> {
- if (cfg.runSchedule) {
- LOG.info("Do schedule");
- Option<String> instantTime = doSchedule(jsc);
- int result = instantTime.isPresent() ? 0 : -1;
- if (result == 0) {
- LOG.info("The schedule instant time is " + instantTime.get());
+ String runningMode = cfg.runningMode == null ? "" :
cfg.runningMode.toLowerCase();
+ switch (runningMode) {
+ case SCHEDULE: {
+ LOG.info("Running Mode: [" + SCHEDULE + "]; Do schedule");
+ Option<String> instantTime = doSchedule(jsc);
+ int result = instantTime.isPresent() ? 0 : -1;
+ if (result == 0) {
+ LOG.info("The schedule instant time is " + instantTime.get());
+ }
+ return result;
+ }
+ case SCHEDULE_AND_EXECUTE: {
+ LOG.info("Running Mode: [" + SCHEDULE_AND_EXECUTE + "]");
+ return doScheduleAndCluster(jsc);
+ }
+ case EXECUTE:
+ default: {
+ LOG.info("Running Mode: [" + EXECUTE + "]; Do cluster");
Review comment:
Nice catching. I changed the default behavior as `LOG.info("Unsupported
running mode [" + runningMode + "], quit the job directly");` in case users set
a wrong value of --mode like `--mode abcd`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Build cluster plan and execute this plan at once for HoodieClusteringJob
> ------------------------------------------------------------------------
>
> Key: HUDI-2164
> URL: https://issues.apache.org/jira/browse/HUDI-2164
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Yue Zhang
> Priority: Major
> Labels: pull-request-available
>
> For now, Hudi can let users submit a HoodieClusteringJob to build a
> clustering plan or execute a clustering plan through --schedule or
> --instant-time config.
> If users want to trigger a clustering job, he has to
> # Submit a HoodieClusteringJob to build a clustering job through --schedule
> config
> # Copy the created clustering Instant time form Log info.
> # Submit the HoodieClusteringJob again to execute this created clustering
> plan through --instant-time config.
> The pain point is that there are too many steps when trigger a clustering and
> need to copy and paste the instant time from log file manually so that we
> can't make it automatically.
>
> I just raise a PR to offer a new config named --mode or -m in short
> ||--mode||remarks||
> |execute|Execute a cluster plan at given instant which means --instant-time
> is needed here. default value. |
> |schedule|Make a clustering plan.|
> |*scheduleAndExecute*|Make a cluster plan first and execute that plan
> immediately|
> Now users can use --mode scheduleAndExecute to Build cluster plan and execute
> this plan at once using HoodieClusteringJob.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
