[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Summary: SparkMergingDictionary parallelize not work (was: SparkMergingDictionary parallelize not work 。) > SparkMergingDictionary parallelize not work > > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Major > Attachments: Completed Stages (2) copy.png, Pasted Graphic 1.png, > Pasted Graphic 2.png, Stage Id.png, Storage Environment.png, > image-2020-11-25-12-46-05-750.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Priority: Major (was: Critical) > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Major > Attachments: Completed Stages (2) copy.png, Pasted Graphic 1.png, > Pasted Graphic 2.png, Stage Id.png, Storage Environment.png, > image-2020-11-25-12-46-05-750.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238505#comment-17238505 ] JiangYang edited comment on KYLIN-4827 at 11/25/20, 4:52 AM: - When run SparkMergingDictionary. Only one task run in job0. !Storage Environment.png! cube has cols need to merge !Pasted Graphic 2.png! was (Author: jiangyang): When run SparkMergingDictionary. Only one task run in job0. > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Completed Stages (2) copy.png, Pasted Graphic 1.png, > Pasted Graphic 2.png, Stage Id.png, Storage Environment.png, > image-2020-11-25-12-46-05-750.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238508#comment-17238508 ] JiangYang commented on KYLIN-4827: -- Do experiment in Spark-shell sc.parallelize(1 to 1000, 1000).coalesce(1, true); sc.parallelize(1 to 1000, 1000).coalesce(1,false); 1. !Stage Id.png! 2. !Completed Stages (2) copy.png! > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Completed Stages (2) copy.png, Pasted Graphic 1.png, > Pasted Graphic 2.png, Stage Id.png, Storage Environment.png, > image-2020-11-25-12-46-05-750.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Attachment: Completed Stages (2) copy.png > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Completed Stages (2) copy.png, Pasted Graphic 1.png, > Pasted Graphic 2.png, Stage Id.png, Storage Environment.png, > image-2020-11-25-12-46-05-750.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Attachment: Stage Id.png > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Completed Stages (2) copy.png, Pasted Graphic 1.png, > Pasted Graphic 2.png, Stage Id.png, Storage Environment.png, > image-2020-11-25-12-46-05-750.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Attachment: image-2020-11-25-12-46-05-750.png > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Pasted Graphic 1.png, Pasted Graphic 2.png, Storage > Environment.png, image-2020-11-25-12-46-05-750.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[
https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238506#comment-17238506
]
JiangYang commented on KYLIN-4827:
--
My opinion is: caused by the coalesce:
{code:java}
colToDictPathRDD.coalesce(1, false).saveAsNewAPIHadoopFile(dictOutputPath,
Text.class, Text.class,
SequenceFileOutputFormat.class);
{code}
In Spark Api:
{code:java}
# However, if you're doing a drastic coalesce, e.g. to numPartitions = 1,
* this may result in your computation taking place on fewer nodes than
* you like (e.g. one node in the case of numPartitions = 1). To avoid this,
* you can pass shuffle = true. This will add a shuffle step, but means the
* current upstream partitions will be executed in parallel (per whatever
* the current partitioning is).
{code}
should set coalesce(1, true).
> SparkMergingDictionary parallelize not work 。
> -
>
> Key: KYLIN-4827
> URL: https://issues.apache.org/jira/browse/KYLIN-4827
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Reporter: JiangYang
>Priority: Critical
> Attachments: Pasted Graphic 1.png, Pasted Graphic 2.png, Storage
> Environment.png
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Attachment: Pasted Graphic 2.png > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Pasted Graphic 1.png, Pasted Graphic 2.png, Storage > Environment.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Attachment: Pasted Graphic 1.png > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Pasted Graphic 1.png, Storage Environment.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4827: - Attachment: Storage Environment.png > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Storage Environment.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
[ https://issues.apache.org/jira/browse/KYLIN-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238505#comment-17238505 ] JiangYang commented on KYLIN-4827: -- When run SparkMergingDictionary. Only one task run in job0. > SparkMergingDictionary parallelize not work 。 > - > > Key: KYLIN-4827 > URL: https://issues.apache.org/jira/browse/KYLIN-4827 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Priority: Critical > Attachments: Storage Environment.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4827) SparkMergingDictionary parallelize not work 。
JiangYang created KYLIN-4827: Summary: SparkMergingDictionary parallelize not work 。 Key: KYLIN-4827 URL: https://issues.apache.org/jira/browse/KYLIN-4827 Project: Kylin Issue Type: Bug Components: Spark Engine Reporter: JiangYang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
[
https://issues.apache.org/jira/browse/KYLIN-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238495#comment-17238495
]
ASF GitHub Bot commented on KYLIN-4826:
---
wolfboys opened a new pull request #1487:
URL: https://github.com/apache/kylin/pull/1487
## Proposed changes
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting input path error.
After kylin is started for the first time, when it is started again,
${dir}/cached-hive-dependency.sh will be used, now "
kylin.source.hive.warehouse-dir" is not in cached-hive- dependency.sh, the
parameter is null, causing a bug
## Types of changes
What types of changes does your code introduce to Kylin?
_Put an `x` in the boxes that apply_
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation Update (if none of the other choices apply)
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [x] I have create an issue on [Kylin's
jira](https://issues.apache.org/jira/browse/KYLIN-4826), and have described the
bug/feature there in detail
- [ ] Commit messages in my PR start with the related jira ID, like
"KYLIN- Make Kylin project open-source"
- [ ] Compiling and unit tests pass locally with my changes
- [ ] I have added tests that prove my fix is effective or that my feature
works
- [ ] If this change need a document change, I will prepare another pr
against the `document` branch
- [ ] Any dependent changes have been merged
## Further comments
If this is a relatively large or complex change, kick off the discussion at
user@kylin or dev@kylin by explaining why you chose the solution you did and
what alternatives you considered, etc...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> The value of config kylin.source.hive.warehouse-dir can not be found
>
>
> Key: KYLIN-4826
> URL: https://issues.apache.org/jira/browse/KYLIN-4826
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.1
> Environment: kylin 3.1.1
>Reporter: huajiewang
>Priority: Major
> Fix For: Future
>
> Attachments: WechatIMG841.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
> when using kylin 3.1.1's global dictionary, building the global dictionary
> failed because it could not be found "kylin.source.hive.warehouse-dir",
> resulting input path error.
> After kylin is started for the first time, when it is started again,
> ${dir}/cached-hive-dependency.sh will be used, now "
> kylin.source.hive.warehouse-dir" is not in cached-hive- dependency.sh, the
> parameter is null, causing a bug
> In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
> necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] wolfboys opened a new pull request #1487: KYLIN-4826 kylin.source.hive.warehouse-dir can't found bug fix
wolfboys opened a new pull request #1487:
URL: https://github.com/apache/kylin/pull/1487
## Proposed changes
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting input path error.
After kylin is started for the first time, when it is started again,
${dir}/cached-hive-dependency.sh will be used, now "
kylin.source.hive.warehouse-dir" is not in cached-hive- dependency.sh, the
parameter is null, causing a bug
## Types of changes
What types of changes does your code introduce to Kylin?
_Put an `x` in the boxes that apply_
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation Update (if none of the other choices apply)
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [x] I have create an issue on [Kylin's
jira](https://issues.apache.org/jira/browse/KYLIN-4826), and have described the
bug/feature there in detail
- [ ] Commit messages in my PR start with the related jira ID, like
"KYLIN- Make Kylin project open-source"
- [ ] Compiling and unit tests pass locally with my changes
- [ ] I have added tests that prove my fix is effective or that my feature
works
- [ ] If this change need a document change, I will prepare another pr
against the `document` branch
- [ ] Any dependent changes have been merged
## Further comments
If this is a relatively large or complex change, kick off the discussion at
user@kylin or dev@kylin by explaining why you chose the solution you did and
what alternatives you considered, etc...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (KYLIN-4813) Refine spark logger for Kylin 4 build engine
[ https://issues.apache.org/jira/browse/KYLIN-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238463#comment-17238463 ] ASF GitHub Bot commented on KYLIN-4813: --- lgtm-com[bot] commented on pull request #1481: URL: https://github.com/apache/kylin/pull/1481#issuecomment-733421209 This pull request **fixes 1 alert** when merging d87f319d9257268f4f0bbe10fb141f4c3c02d72e into 8da4b26c2cb3e9d46100ca64fe5bb4fcdfaf56ac - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-e78acbefcbb532dbc08827a89430a41798508fc8) **fixed alerts:** * 1 for Spurious Javadoc @param tags This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refine spark logger for Kylin 4 build engine > > > Key: KYLIN-4813 > URL: https://issues.apache.org/jira/browse/KYLIN-4813 > Project: Kylin > Issue Type: Improvement >Affects Versions: v4.0.0-alpha >Reporter: Xiaoxiang Yu >Assignee: Yaqian Zhang >Priority: Major > Fix For: v4.0.0-beta > > > - Separate spark log from kylin log > - Store driver/executor log into HDFS. > - Provided a API to view driver log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] lgtm-com[bot] commented on pull request #1481: KYLIN-4813 Refactor spark build log
lgtm-com[bot] commented on pull request #1481: URL: https://github.com/apache/kylin/pull/1481#issuecomment-733421209 This pull request **fixes 1 alert** when merging d87f319d9257268f4f0bbe10fb141f4c3c02d72e into 8da4b26c2cb3e9d46100ca64fe5bb4fcdfaf56ac - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-e78acbefcbb532dbc08827a89430a41798508fc8) **fixed alerts:** * 1 for Spurious Javadoc @param tags This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
[
https://issues.apache.org/jira/browse/KYLIN-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huajiewang updated KYLIN-4826:
--
Description:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting input path error.
After kylin is started for the first time, when it is started again,
${dir}/cached-hive-dependency.sh will be used, now "
kylin.source.hive.warehouse-dir" is not in cached-hive- dependency.sh, the
parameter is null, causing a bug
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
was:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting input path error.
After kylin is started for the first time, when it is started again,
${dir}/cached-hive-dependency.sh will be used, now "
kylin.source.hive.warehouse dir " is not in cached-hive- dependency.sh, the
parameter is null, causing a bug
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
> The value of config kylin.source.hive.warehouse-dir can not be found
>
>
> Key: KYLIN-4826
> URL: https://issues.apache.org/jira/browse/KYLIN-4826
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.1
> Environment: kylin 3.1.1
>Reporter: huajiewang
>Priority: Major
> Fix For: Future
>
> Attachments: WechatIMG841.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
> when using kylin 3.1.1's global dictionary, building the global dictionary
> failed because it could not be found "kylin.source.hive.warehouse-dir",
> resulting input path error.
> After kylin is started for the first time, when it is started again,
> ${dir}/cached-hive-dependency.sh will be used, now "
> kylin.source.hive.warehouse-dir" is not in cached-hive- dependency.sh, the
> parameter is null, causing a bug
> In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
> necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
[
https://issues.apache.org/jira/browse/KYLIN-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huajiewang updated KYLIN-4826:
--
Attachment: (was: 微信图片_20201124222001.png)
> The value of config kylin.source.hive.warehouse-dir can not be found
>
>
> Key: KYLIN-4826
> URL: https://issues.apache.org/jira/browse/KYLIN-4826
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.1
> Environment: kylin 3.1.1
>Reporter: huajiewang
>Priority: Major
> Fix For: Future
>
> Attachments: WechatIMG841.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
> when using kylin 3.1.1's global dictionary, building the global dictionary
> failed because it could not be found "kylin.source.hive.warehouse-dir",
> resulting input path error.
> After kylin is started for the first time, when it is started again,
> ${dir}/cached-hive-dependency.sh will be used, now "
> kylin.source.hive.warehouse dir " is not in cached-hive- dependency.sh, the
> parameter is null, causing a bug
> In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
> necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
[
https://issues.apache.org/jira/browse/KYLIN-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huajiewang updated KYLIN-4826:
--
Attachment: WechatIMG841.png
> The value of config kylin.source.hive.warehouse-dir can not be found
>
>
> Key: KYLIN-4826
> URL: https://issues.apache.org/jira/browse/KYLIN-4826
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.1
> Environment: kylin 3.1.1
>Reporter: huajiewang
>Priority: Major
> Fix For: Future
>
> Attachments: WechatIMG841.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
> when using kylin 3.1.1's global dictionary, building the global dictionary
> failed because it could not be found "kylin.source.hive.warehouse-dir",
> resulting input path error.
> After kylin is started for the first time, when it is started again,
> ${dir}/cached-hive-dependency.sh will be used, now "
> kylin.source.hive.warehouse dir " is not in cached-hive- dependency.sh, the
> parameter is null, causing a bug
> In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
> necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
[
https://issues.apache.org/jira/browse/KYLIN-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huajiewang updated KYLIN-4826:
--
Description:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting input path error.
After kylin is started for the first time, when it is started again,
${dir}/cached-hive-dependency.sh will be used, now "
kylin.source.hive.warehouse dir " is not in cached-hive- dependency.sh, the
parameter is null, causing a bug
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
was:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting input path error.
After kylin is started for the first time, when it is started again, source
${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
warehouse dir " is not in cached-hive- dependency.sh, the parameter is null,
causing a bug
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
> The value of config kylin.source.hive.warehouse-dir can not be found
>
>
> Key: KYLIN-4826
> URL: https://issues.apache.org/jira/browse/KYLIN-4826
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.1
> Environment: kylin 3.1.1
>Reporter: huajiewang
>Priority: Major
> Fix For: Future
>
> Attachments: 微信图片_20201124222001.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
> when using kylin 3.1.1's global dictionary, building the global dictionary
> failed because it could not be found "kylin.source.hive.warehouse-dir",
> resulting input path error.
> After kylin is started for the first time, when it is started again,
> ${dir}/cached-hive-dependency.sh will be used, now "
> kylin.source.hive.warehouse dir " is not in cached-hive- dependency.sh, the
> parameter is null, causing a bug
> In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
> necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
[
https://issues.apache.org/jira/browse/KYLIN-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huajiewang updated KYLIN-4826:
--
Description:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting input path error.
After kylin is started for the first time, when it is started again, source
${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
warehouse dir " is not in cached-hive- dependency.sh, the parameter is null,
causing a bug
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
was:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting in the dictionary input path error.
After kylin is started for the first time, when it is started again, source
${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
warehouse dir " is not incached-hive- dependency.sh, the parameter is null,
causing a bug
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
> The value of config kylin.source.hive.warehouse-dir can not be found
>
>
> Key: KYLIN-4826
> URL: https://issues.apache.org/jira/browse/KYLIN-4826
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.1
> Environment: kylin 3.1.1
>Reporter: huajiewang
>Priority: Major
> Fix For: Future
>
> Attachments: 微信图片_20201124222001.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
> when using kylin 3.1.1's global dictionary, building the global dictionary
> failed because it could not be found "kylin.source.hive.warehouse-dir",
> resulting input path error.
> After kylin is started for the first time, when it is started again, source
> ${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
> warehouse dir " is not in cached-hive- dependency.sh, the parameter is null,
> causing a bug
> In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
> necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
[
https://issues.apache.org/jira/browse/KYLIN-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huajiewang updated KYLIN-4826:
--
Description:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive.warehouse-dir",
resulting in the dictionary input path error.
After kylin is started for the first time, when it is started again, source
${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
warehouse dir " is not incached-hive- dependency.sh, the parameter is null,
causing a bug
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
was:
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive . warehouse-dir",
resulting in the dictionary input path error.
After kylin is started for the first time, when it is started again, source
${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
warehouse dir " is not incached-hive- dependency.sh, the parameter is null,
causing a bug，
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
> The value of config kylin.source.hive.warehouse-dir can not be found
>
>
> Key: KYLIN-4826
> URL: https://issues.apache.org/jira/browse/KYLIN-4826
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.1
> Environment: kylin 3.1.1
>Reporter: huajiewang
>Priority: Major
> Fix For: Future
>
> Attachments: 微信图片_20201124222001.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
> when using kylin 3.1.1's global dictionary, building the global dictionary
> failed because it could not be found "kylin.source.hive.warehouse-dir",
> resulting in the dictionary input path error.
> After kylin is started for the first time, when it is started again, source
> ${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
> warehouse dir " is not incached-hive- dependency.sh, the parameter is null,
> causing a bug
> In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
> necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4826) The value of config kylin.source.hive.warehouse-dir can not be found
huajiewang created KYLIN-4826:
-
Summary: The value of config kylin.source.hive.warehouse-dir can
not be found
Key: KYLIN-4826
URL: https://issues.apache.org/jira/browse/KYLIN-4826
Project: Kylin
Issue Type: Bug
Components: Environment
Affects Versions: v3.1.1
Environment: kylin 3.1.1
Reporter: huajiewang
Fix For: Future
Attachments: 微信图片_20201124222001.png
when using kylin 3.1.1's global dictionary, building the global dictionary
failed because it could not be found "kylin.source.hive . warehouse-dir",
resulting in the dictionary input path error.
After kylin is started for the first time, when it is started again, source
${dir}/cached-hive-dependency.sh will be used, now " kylin.source.hive .
warehouse dir " is not incached-hive- dependency.sh, the parameter is null,
causing a bug，
In addition https://issues.apache.org/jira/browse/KYLIN-4028 Whether this is
necessary is worth discussing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4813) Refine spark logger for Kylin 4 build engine
[ https://issues.apache.org/jira/browse/KYLIN-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238090#comment-17238090 ] ASF GitHub Bot commented on KYLIN-4813: --- lgtm-com[bot] commented on pull request #1481: URL: https://github.com/apache/kylin/pull/1481#issuecomment-732947289 This pull request **fixes 1 alert** when merging 71b91b4bb99cc170d9728e40d5eab24b23950eaa into 8da4b26c2cb3e9d46100ca64fe5bb4fcdfaf56ac - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-62e5293ad9653a757559bc4d47d7de8b030e91c0) **fixed alerts:** * 1 for Spurious Javadoc @param tags This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refine spark logger for Kylin 4 build engine > > > Key: KYLIN-4813 > URL: https://issues.apache.org/jira/browse/KYLIN-4813 > Project: Kylin > Issue Type: Improvement >Affects Versions: v4.0.0-alpha >Reporter: Xiaoxiang Yu >Assignee: Yaqian Zhang >Priority: Major > Fix For: v4.0.0-beta > > > - Separate spark log from kylin log > - Store driver/executor log into HDFS. > - Provided a API to view driver log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] lgtm-com[bot] commented on pull request #1481: KYLIN-4813 Refactor spark build log
lgtm-com[bot] commented on pull request #1481: URL: https://github.com/apache/kylin/pull/1481#issuecomment-732947289 This pull request **fixes 1 alert** when merging 71b91b4bb99cc170d9728e40d5eab24b23950eaa into 8da4b26c2cb3e9d46100ca64fe5bb4fcdfaf56ac - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-62e5293ad9653a757559bc4d47d7de8b030e91c0) **fixed alerts:** * 1 for Spurious Javadoc @param tags This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
