[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3421: Fix Version/s: v2.4.1 > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v2.4.1, v2.5.0 > > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Fix Version/s: v2.5.0 > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v2.5.0 > > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Attachment: APACHE-KYLIN-3421.patch > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Attachment: (was: APACHE-KYLIN-3421.patch) > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Attachment: (was: APACHE-KYLIN-3421.patch) > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Attachment: APACHE-KYLIN-3421.patch > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Attachment: APACHE-KYLIN-3421.patch > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Attachment: (was: APACHE-KYLIN-3421.patch) > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Attachment: APACHE-KYLIN-3421.patch > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Attachments: APACHE-KYLIN-3421.patch > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3421: Fix Version/s: (was: v2.4.0) As 2.4.0 has been kicked off (almost done), so I would suggest move it to next version. > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
[ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3421: -- Description: Currently, there are several improvements we can do for job scheduler: * For {{DistributedScheduler}}, it does not check {{isReady()}} for checkpoint jobs * Since the state lots of executables fetched from HBase are not READY, it's better to fetch their output first, which can reduce lots of get requests to hbase * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, which needs to change the lockPath from *{color:#f79232}segmentId{color}* to *{color:#f79232}jobId{color}* > Improve job scheduler fetch performance > --- > > Key: KYLIN-3421 > URL: https://issues.apache.org/jira/browse/KYLIN-3421 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v2.4.0 > > > Currently, there are several improvements we can do for job scheduler: > * For {{DistributedScheduler}}, it does not check {{isReady()}} for > checkpoint jobs > * Since the state lots of executables fetched from HBase are not READY, it's > better to fetch their output first, which can reduce lots of get requests to > hbase > * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, > which needs to change the lockPath from *{color:#f79232}segmentId{color}* to > *{color:#f79232}jobId{color}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)