github-actions[bot] commented on code in PR #62238:
URL: https://github.com/apache/doris/pull/62238#discussion_r3055231189


##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingJobSchedulerTask.java:
##########
@@ -83,28 +86,41 @@ private void handleRunningState() throws JobException {
         streamingInsertJob.fetchMeta();
     }
 
-    private void autoResumeHandler() throws JobException {
-        final FailureReason failureReason = 
streamingInsertJob.getFailureReason();
+    private void handleRetryingState() throws JobException {
         final long latestAutoResumeTimestamp = 
streamingInsertJob.getLatestAutoResumeTimestamp();
         final long autoResumeCount = streamingInsertJob.getAutoResumeCount();
         final long current = System.currentTimeMillis();
 
-        if (failureReason != null
-                && failureReason.getCode() != 
InternalErrorCode.MANUAL_PAUSE_ERR
-                && failureReason.getCode() != 
InternalErrorCode.TOO_MANY_FAILURE_ROWS_ERR
-                && failureReason.getCode() != 
InternalErrorCode.CANNOT_RESUME_ERR) {
-            long autoResumeIntervalTimeSec = autoResumeCount < 5
-                        ? Math.min((long) Math.pow(2, autoResumeCount) * 
BACK_OFF_BASIC_TIME_SEC,
-                                MAX_BACK_OFF_TIME_SEC) : MAX_BACK_OFF_TIME_SEC;
-            if (current - latestAutoResumeTimestamp > 
autoResumeIntervalTimeSec * 1000L) {
-                streamingInsertJob.setLatestAutoResumeTimestamp(current);
-                if (autoResumeCount < Long.MAX_VALUE) {
-                    streamingInsertJob.setAutoResumeCount(autoResumeCount + 1);
-                }
-                streamingInsertJob.updateJobStatus(JobStatus.PENDING);
+        long autoResumeIntervalTimeSec = autoResumeCount < 5
+                    ? Math.min((long) Math.pow(2, autoResumeCount) * 
BACK_OFF_BASIC_TIME_SEC,
+                            MAX_BACK_OFF_TIME_SEC) : MAX_BACK_OFF_TIME_SEC;
+        if (current - latestAutoResumeTimestamp <= autoResumeIntervalTimeSec * 
1000L) {
+            // backoff not reached, wait
+            return;
+        }
+
+        streamingInsertJob.setLatestAutoResumeTimestamp(current);
+        if (autoResumeCount < Long.MAX_VALUE) {
+            streamingInsertJob.setAutoResumeCount(autoResumeCount + 1);
+        }
+
+        if (Config.isCloudMode()) {
+            try {
+                streamingInsertJob.replayOnCloudMode();
+            } catch (JobException e) {
+                streamingInsertJob.setFailureReason(
+                    new FailureReason(InternalErrorCode.INTERNAL_ERR, 
e.getMessage()));
+                // stay in RETRYING, will retry next round
                 return;
             }
         }
+        if (streamingInsertJob.hasReachedEnd()) {
+            streamingInsertJob.updateJobStatus(JobStatus.FINISHED);
+            return;
+        }
+        streamingInsertJob.createStreamingTask();
+        streamingInsertJob.setSampleStartTime(System.currentTimeMillis());

Review Comment:
   `handleRetryingState()` now creates a new streaming task but keeps the job 
in `RETRYING`. Because `needScheduleTask()` returns `true` for `RETRYING` and 
there is no guard here for an already-active `runningStreamTask`, the next 
timer-driven `StreamingJobSchedulerTask` can come back through this branch 
again before the first retry attempt finishes and call `createStreamingTask()` 
a second time. That opens the door to two concurrent streaming tasks for the 
same job. The old flow moved back to `PENDING`, which avoided this 
duplicate-task window.



##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -646,6 +646,11 @@ public void onStreamTaskSuccess(AbstractStreamingTask 
task) throws JobException
             }
 
             
Env.getCurrentEnv().getJobManager().getStreamingTaskManager().removeRunningTask(task);
+            // If the job was retrying due to previous failures, transition 
back to RUNNING
+            if (JobStatus.RETRYING.equals(getJobStatus())) {
+                setAutoResumeCount(0);
+                updateJobStatus(JobStatus.RUNNING);
+            }
             if (offsetProvider.hasReachedEnd()) {
                 // offset provider has reached a natural end, mark job as 
finished

Review Comment:
   `fetchMeta()` explicitly preserves `MANUAL_PAUSE_ERR` before switching to 
`RETRYING`, but this task-failure path now unconditionally flips the job to 
`RETRYING`. If the user runs `PAUSE JOB` while a streaming task is failing, 
`onStreamTaskFail()` overwrites the manual pause reason/status and the job 
starts auto-retrying again. That violates the new contract that `PAUSED` is 
reserved for user-initiated pause and unrecoverable errors. This path needs the 
same manual-pause guard as `fetchMeta()`; the scheduling-failure path has the 
same race.



##########
regression-test/suites/job_p0/streaming_job/cdc/tvf/test_streaming_job_cdc_stream_postgres_latest_alter_cred.groovy:
##########
@@ -173,7 +173,7 @@ 
suite("test_streaming_job_cdc_stream_postgres_latest_alter_cred",
             Awaitility.await().atMost(120, SECONDS).pollInterval(2, 
SECONDS).until({
                 def s = sql """select status from jobs("type"="insert") where 
Name='${jobName}'"""
                 log.info("status after wrong cred resume: " + s)
-                s.size() == 1 && s.get(0).get(0) == "PAUSED"
+                s.size() == 1 && s.get(0).get(0) == "RETRYING"

Review Comment:
   Once this assertion is updated to wait for `RETRYING`, the later `ALTER JOB` 
in scenario C no longer works: `AlterJobCommand.validate()` still rejects 
anything except `PAUSED`. Before this patch the auto-error state was `PAUSED`, 
so the alter step was reachable; now this test needs an explicit `PAUSE JOB` 
before the alter, or the product code needs to allow altering a `RETRYING` job.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to