[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-07-01 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3421:

Fix Version/s: v2.4.1

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.1, v2.5.0
>
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-25 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Fix Version/s: v2.5.0

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-22 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Attachment: APACHE-KYLIN-3421.patch

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-22 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Attachment: (was: APACHE-KYLIN-3421.patch)

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-21 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Attachment: (was: APACHE-KYLIN-3421.patch)

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-21 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Attachment: APACHE-KYLIN-3421.patch

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-21 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Attachment: APACHE-KYLIN-3421.patch

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-21 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Attachment: (was: APACHE-KYLIN-3421.patch)

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-20 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Attachment: APACHE-KYLIN-3421.patch

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Attachments: APACHE-KYLIN-3421.patch
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-20 Thread Shaofeng SHI (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-3421:

Fix Version/s: (was: v2.4.0)

As 2.4.0 has been kicked off (almost done), so I would suggest move it to next 
version.

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance

2018-06-20 Thread Zhong Yanghong (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhong Yanghong updated KYLIN-3421:
--
Description: 
Currently, there are several improvements we can do for job scheduler:
* For {{DistributedScheduler}}, it does not check {{isReady()}} for checkpoint 
jobs
* Since the state lots of executables fetched from HBase are not READY, it's 
better to fetch their output first, which can reduce lots of get requests to 
hbase
* {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
*{color:#f79232}jobId{color}*

> Improve job scheduler fetch performance
> ---
>
> Key: KYLIN-3421
> URL: https://issues.apache.org/jira/browse/KYLIN-3421
> Project: Kylin
>  Issue Type: Improvement
>  Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.0
>
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for 
> checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's 
> better to fetch their output first, which can reduce lots of get requests to 
> hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, 
> which needs to change the lockPath from *{color:#f79232}segmentId{color}* to 
> *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)