[
https://issues.apache.org/jira/browse/KYLIN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351720#comment-16351720
]
Shaofeng SHI commented on KYLIN-3140:
-------------------------------------
Okay I see; you want to control the concurrent new jobs, but don't want to it
be impacted by the merge jobs. The auto-merge job can not be discarded easily
as once the old is discarded, a new will be sumbmitted immediately.
> Auto merge jobs should not block user build jobs
> ------------------------------------------------
>
> Key: KYLIN-3140
> URL: https://issues.apache.org/jira/browse/KYLIN-3140
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Reporter: Wang, Gang
> Assignee: Shaofeng SHI
> Priority: Major
>
> Although in the latest version, Kylin support concurrent jobs. If the
> concurrency is set to 1, there is some possibility that cube build job will
> have dead lock. Say, when there is some issue which causes merge job failed,
> even when you discard the job, another job will be launched and failed again
> due to auto merge policy. And this failed merge job blocks user to build
> incremental segment.
> Even if the concurrency is set to larger than 1, the auto merge jobs occupy
> some concurrency quota.
> While, from user perspective, they don't care much about the auto merge jobs,
> and the auto merge jobs should not block the building/refresh jobs they
> submit manually.
> A better way may be separating the auto merge jobs from the job queue,
> parameter max-building-segments only limit jobs submitted by users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
