github-actions[bot] commented on code in PR #64301:
URL: https://github.com/apache/doris/pull/64301#discussion_r3387744992
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -1409,14 +1410,17 @@ public void gsonPostProcess() throws IOException {
* Only applies to StreamingMultiTask.
*/
public void processTimeoutTasks() throws JobException {
- if (!(runningStreamTask instanceof StreamingMultiTblTask)) {
+ AbstractStreamingTask task = this.runningStreamTask;
+ if (!(task instanceof StreamingMultiTblTask)) {
return;
}
+ StreamingMultiTblTask runningMultiTask = (StreamingMultiTblTask) task;
+ StreamingTaskProgress progress = runningMultiTask.fetchProgress();
writeLock();
try {
- StreamingMultiTblTask runningMultiTask = (StreamingMultiTblTask)
this.runningStreamTask;
- if (TaskStatus.RUNNING.equals(runningMultiTask.getStatus())
- && runningMultiTask.isTimeout()) {
+ if (this.runningStreamTask == runningMultiTask
+ && TaskStatus.RUNNING.equals(runningMultiTask.getStatus())
Review Comment:
`writeRecords` is asynchronous: `/api/writeRecords` returns accepted, and if
the CDC worker later fails, `PipelineCoordinator.writeRecordsAsync` only stores
the reason in `taskErrorMaps` and removes progress. This path still calls
`getTimeoutReason()` only after `isTimeout(progress)` is true. With the new
default 300s `streaming_task_min_timeout_sec`, a job with a small `maxInterval`
that hits an immediate stream-load/data-quality error can stay RUNNING for
about five minutes before the stored failure reason is consumed; after recent
progress, the delay is another full timeout window. Please propagate the fail
reason independently of the no-progress timeout, e.g. fetch it outside the
write lock or include it in the progress response, so hard failures do not wait
for the min timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
