[
https://issues.apache.org/jira/browse/KYLIN-6011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17929007#comment-17929007
]
ASF subversion and git services commented on KYLIN-6011:
--------------------------------------------------------
Commit aeef1338b004e213ae92fa10fea1f181694afe19 in kylin's branch
refs/heads/kylin5 from Zhong.Zhu
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=aeef1338b0 ]
KYLIN-6011 unify job_lock fetch sequence when consuming job and producing job
> After the build task is completed, it is not cleared from the lock table,
> causing the remaining build tasks to fail to start
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-6011
> URL: https://issues.apache.org/jira/browse/KYLIN-6011
> Project: Kylin
> Issue Type: Bug
> Affects Versions: 5.0.0
> Reporter: Guoliang Sun
> Assignee: Guoliang Sun
> Priority: Major
>
> h3. Root cause
> META and ROUTINE type tasks are set as the highest priority, see:
> `org.apache.kylin.job.execution.ExecutableManager#checkAndSubmitCronJob`.
> The job lock table prioritizes checking and cleaning the earliest created
> records when clearing expired locks, and by default retrieves only 30 entries
> at a time (`kylin.job.master-pull-batch-size`).
> If a large number of regular build tasks (> 30) are submitted first, followed
> by a large number of META or ROUTINE type tasks, the job lock table will fail
> to clean up completed META or ROUTINE tasks.
> The job scheduler will then continuously read completed META or ROUTINE tasks
> from the job lock table, preventing new tasks from being scheduled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
