[GitHub] [kylin] codecov-io commented on issue #537: KYLIN-3896 Implement IFlinkOutput based on HBase
codecov-io commented on issue #537: KYLIN-3896 Implement IFlinkOutput based on HBase URL: https://github.com/apache/kylin/pull/537#issuecomment-474698645 # [Codecov](https://codecov.io/gh/apache/kylin/pull/537?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`engine-flink@e9b520b`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/537?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## engine-flink#537 +/- ## == Coverage? 24.2% Complexity ?5029 == Files ?1171 Lines ? 71146 Branches? 10096 == Hits? 17218 Misses ? 52189 Partials?1739 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/537?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...a/org/apache/kylin/storage/hbase/HBaseStorage.java](https://codecov.io/gh/apache/kylin/pull/537/diff?src=pr&el=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9IQmFzZVN0b3JhZ2UuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...torage/hbase/steps/HBaseFlinkOutputTransition.java](https://codecov.io/gh/apache/kylin/pull/537/diff?src=pr&el=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9IQmFzZUZsaW5rT3V0cHV0VHJhbnNpdGlvbi5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/537?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/537?src=pr&el=footer). Last update [e9b520b...e880db6](https://codecov.io/gh/apache/kylin/pull/537?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3896) Implement IFlinkOutput based on HBase
[ https://issues.apache.org/jira/browse/KYLIN-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796821#comment-16796821 ] ASF GitHub Bot commented on KYLIN-3896: --- yanghua commented on pull request #537: KYLIN-3896 Implement IFlinkOutput based on HBase URL: https://github.com/apache/kylin/pull/537 JIRA ref : https://issues.apache.org/jira/browse/KYLIN-3896 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Implement IFlinkOutput based on HBase > - > > Key: KYLIN-3896 > URL: https://issues.apache.org/jira/browse/KYLIN-3896 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: vinoyang >Assignee: vinoyang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3897) Implement IFlinkInput based on Hive
vinoyang created KYLIN-3897: --- Summary: Implement IFlinkInput based on Hive Key: KYLIN-3897 URL: https://issues.apache.org/jira/browse/KYLIN-3897 Project: Kylin Issue Type: Sub-task Components: Flink Engine Reporter: vinoyang Assignee: vinoyang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] asfgit commented on issue #537: KYLIN-3896 Implement IFlinkOutput based on HBase
asfgit commented on issue #537: KYLIN-3896 Implement IFlinkOutput based on HBase URL: https://github.com/apache/kylin/pull/537#issuecomment-474693921 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #537: KYLIN-3896 Implement IFlinkOutput based on HBase
asfgit commented on issue #537: KYLIN-3896 Implement IFlinkOutput based on HBase URL: https://github.com/apache/kylin/pull/537#issuecomment-474693922 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] yanghua opened a new pull request #537: KYLIN-3896 Implement IFlinkOutput based on HBase
yanghua opened a new pull request #537: KYLIN-3896 Implement IFlinkOutput based on HBase URL: https://github.com/apache/kylin/pull/537 JIRA ref : https://issues.apache.org/jira/browse/KYLIN-3896 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-3896) Implement IFlinkOutput based on HBase
vinoyang created KYLIN-3896: --- Summary: Implement IFlinkOutput based on HBase Key: KYLIN-3896 URL: https://issues.apache.org/jira/browse/KYLIN-3896 Project: Kylin Issue Type: Sub-task Components: Flink Engine Reporter: vinoyang Assignee: vinoyang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] codecov-io commented on issue #536: KYLIN-3474 Tableau 10.5 get malformed token
codecov-io commented on issue #536: KYLIN-3474 Tableau 10.5 get malformed token URL: https://github.com/apache/kylin/pull/536#issuecomment-474670454 # [Codecov](https://codecov.io/gh/apache/kylin/pull/536?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@4abe712`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/536?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #536 +/- ## = Coverage ? 24.41% Complexity? 5025 = Files ? 1161 Lines ?70460 Branches ?10020 = Hits ?17206 Misses?51518 Partials ? 1736 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/536?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...va/org/apache/kylin/rest/security/ManagedUser.java](https://codecov.io/gh/apache/kylin/pull/536/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VjdXJpdHkvTWFuYWdlZFVzZXIuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/536?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/536?src=pr&el=footer). Last update [4abe712...ebedd89](https://codecov.io/gh/apache/kylin/pull/536?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796732#comment-16796732
]
Temple Zhou edited comment on KYLIN-3474 at 3/20/19 3:05 AM:
-
{code:java}
public class KylinUserManager {
private static final Logger logger =
LoggerFactory.getLogger(KylinUserManager.class);
public static KylinUserManager getInstance(KylinConfig config) {
return config.getManager(KylinUserManager.class);
}
// called by reflection
static KylinUserManager newInstance(KylinConfig config) throws IOException {
return new KylinUserManager(config);
}
//
private KylinConfig config;
// user ==> ManagedUser
private CaseInsensitiveStringCache userMap;
private CachedCrudAssist crud;
private AutoReadWriteLock lock = new AutoReadWriteLock();
{code}
The userMap is case-insensitive, so I can authenticate successfully with
uppercase or lowercase username such as ADMIN or admin，but
org.apache.kylin.rest.security.ManagedUser#equals compare the username in a
case sensitive method.
org.apache.kylin.rest.security.KylinAuthenticationProvider#authenticate
{code:java}
String username = user.getUsername();
logger.debug("User {} authorities : {}", username,
user.getAuthorities());
if (!userService.userExists(username)) {
userService.createUser(user);
} else if
(!userService.loadUserByUsername(username).equals(user)) {
// in case ldap users changing.
userService.updateUser(user);
}
{code}
Finally, the userService.loadUserByUsername(username).equals(user) will always
be false, and the
org.springframework.security.provisioning.UserDetailsManager#updateUser will be
called many times per second.
was (Author: temple.zhou):
{code:java}
public class KylinUserManager {
private static final Logger logger =
LoggerFactory.getLogger(KylinUserManager.class);
public static KylinUserManager getInstance(KylinConfig config) {
return config.getManager(KylinUserManager.class);
}
// called by reflection
static KylinUserManager newInstance(KylinConfig config) throws IOException {
return new KylinUserManager(config);
}
//
private KylinConfig config;
// user ==> ManagedUser
private CaseInsensitiveStringCache userMap;
private CachedCrudAssist crud;
private AutoReadWriteLock lock = new AutoReadWriteLock();
{code}
The userMap is case-insensitive, so I can authenticate successfully with
uppercase or lowercase username，but
org.apache.kylin.rest.security.ManagedUser#equals compare the username in a
case sensitive method.
org.apache.kylin.rest.security.KylinAuthenticationProvider#authenticate
{code:java}
String username = user.getUsername();
logger.debug("User {} authorities : {}", username,
user.getAuthorities());
if (!userService.userExists(username)) {
userService.createUser(user);
} else if
(!userService.loadUserByUsername(username).equals(user)) {
// in case ldap users changing.
userService.updateUser(user);
}
{code}
Finally, the userService.loadUserByUsername(username).equals(user) will always
be false, and the
org.springframework.security.provisioning.UserDetailsManager#updateUser will be
called many times per second.
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.1
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.4.1, v2.4.2
>
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] codecov-io commented on issue #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true
codecov-io commented on issue #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true URL: https://github.com/apache/kylin/pull/535#issuecomment-474666849 # [Codecov](https://codecov.io/gh/apache/kylin/pull/535?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@4abe712`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/535?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #535 +/- ## = Coverage ? 24.42% Complexity? 5025 = Files ? 1161 Lines ?70461 Branches ?10020 = Hits ?17208 Misses?51518 Partials ? 1735 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/535?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../apache/kylin/rest/metrics/QueryMetricsFacade.java](https://codecov.io/gh/apache/kylin/pull/535/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvbWV0cmljcy9RdWVyeU1ldHJpY3NGYWNhZGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/535?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/535?src=pr&el=footer). Last update [4abe712...70e2f49](https://codecov.io/gh/apache/kylin/pull/535?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796732#comment-16796732
]
Temple Zhou commented on KYLIN-3474:
{code:java}
public class KylinUserManager {
private static final Logger logger =
LoggerFactory.getLogger(KylinUserManager.class);
public static KylinUserManager getInstance(KylinConfig config) {
return config.getManager(KylinUserManager.class);
}
// called by reflection
static KylinUserManager newInstance(KylinConfig config) throws IOException {
return new KylinUserManager(config);
}
//
private KylinConfig config;
// user ==> ManagedUser
private CaseInsensitiveStringCache userMap;
private CachedCrudAssist crud;
private AutoReadWriteLock lock = new AutoReadWriteLock();
{code}
The userMap is case-insensitive, so I can authenticate successfully with
uppercase or lowercase username，but
org.apache.kylin.rest.security.ManagedUser#equals compare the username in a
case sensitive method.
org.apache.kylin.rest.security.KylinAuthenticationProvider#authenticate
{code:java}
String username = user.getUsername();
logger.debug("User {} authorities : {}", username,
user.getAuthorities());
if (!userService.userExists(username)) {
userService.createUser(user);
} else if
(!userService.loadUserByUsername(username).equals(user)) {
// in case ldap users changing.
userService.updateUser(user);
}
{code}
Finally, the userService.loadUserByUsername(username).equals(user) will always
be false, and the
org.springframework.security.provisioning.UserDetailsManager#updateUser will be
called many times per second.
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.1
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.4.1, v2.4.2
>
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3895) Failed to register new MBean when "kylin.server.query-metrics-enabled" set true
[
https://issues.apache.org/jira/browse/KYLIN-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796726#comment-16796726
]
ASF GitHub Bot commented on KYLIN-3895:
---
guangxuCheng commented on pull request #535: KYLIN-3895 Failed to register new
MBean when "kylin.server.query-metrics-enabled" set true
URL: https://github.com/apache/kylin/pull/535
see: https://issues.apache.org/jira/browse/KYLIN-3895
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Failed to register new MBean when "kylin.server.query-metrics-enabled" set
> true
>
>
> Key: KYLIN-3895
> URL: https://issues.apache.org/jira/browse/KYLIN-3895
> Project: Kylin
> Issue Type: Bug
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
>
> {code}
> 2019-03-20 10:17:25,753 WARN [Query 46cd99cc-8eb2-8370-d24c-6c10f18da9e0-54]
> util.MBeans:94 : Error creating MBean object name: Ha
> doop:service=Kylin,name=KYLIN_SYSTEM,sub=CUBE[name=KYLIN_HIVE_METRICS_JOB_QA]
> org.apache.hadoop.metrics2.MetricsException:
> javax.management.MalformedObjectNameException: Invalid character '=' in value
> part of property
> at
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:122)
> at
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
> at
> org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:92)
> at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:55)
> {code}
> The subname can't contain '='
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796729#comment-16796729 ] ASF GitHub Bot commented on KYLIN-3474: --- TempleZhou commented on pull request #536: KYLIN-3474 Tableau 10.5 get malformed token URL: https://github.com/apache/kylin/pull/536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.1 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Fix For: v2.4.1, v2.4.2 > > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] TempleZhou opened a new pull request #536: KYLIN-3474 Tableau 10.5 get malformed token
TempleZhou opened a new pull request #536: KYLIN-3474 Tableau 10.5 get malformed token URL: https://github.com/apache/kylin/pull/536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #536: KYLIN-3474 Tableau 10.5 get malformed token
asfgit commented on issue #536: KYLIN-3474 Tableau 10.5 get malformed token URL: https://github.com/apache/kylin/pull/536#issuecomment-474662782 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #536: KYLIN-3474 Tableau 10.5 get malformed token
asfgit commented on issue #536: KYLIN-3474 Tableau 10.5 get malformed token URL: https://github.com/apache/kylin/pull/536#issuecomment-474662781 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true
asfgit commented on issue #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true URL: https://github.com/apache/kylin/pull/535#issuecomment-474661101 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true
asfgit commented on issue #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true URL: https://github.com/apache/kylin/pull/535#issuecomment-474661100 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] guangxuCheng opened a new pull request #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true
guangxuCheng opened a new pull request #535: KYLIN-3895 Failed to register new MBean when "kylin.server.query-metrics-enabled" set true URL: https://github.com/apache/kylin/pull/535 see: https://issues.apache.org/jira/browse/KYLIN-3895 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-3895) Failed to register new MBean when "kylin.server.query-metrics-enabled" set true
Guangxu Cheng created KYLIN-3895:
Summary: Failed to register new MBean when
"kylin.server.query-metrics-enabled" set true
Key: KYLIN-3895
URL: https://issues.apache.org/jira/browse/KYLIN-3895
Project: Kylin
Issue Type: Bug
Reporter: Guangxu Cheng
Assignee: Guangxu Cheng
{code}
2019-03-20 10:17:25,753 WARN [Query 46cd99cc-8eb2-8370-d24c-6c10f18da9e0-54]
util.MBeans:94 : Error creating MBean object name: Ha
doop:service=Kylin,name=KYLIN_SYSTEM,sub=CUBE[name=KYLIN_HIVE_METRICS_JOB_QA]
org.apache.hadoop.metrics2.MetricsException:
javax.management.MalformedObjectNameException: Invalid character '=' in value
part of property
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:122)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:92)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:55)
{code}
The subname can't contain '='
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3788:
---
Assignee: zhao jintao
Hi Jintao, welcome to send a PR to Kylin, thank you!
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796672#comment-16796672
]
Shaofeng SHI commented on KYLIN-3893:
-
Shaohui, thank you! I will let Guosheng take a look from the front-end.
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: v2.6.2
>
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3862) Check the binary packages
[ https://issues.apache.org/jira/browse/KYLIN-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-3862. - Resolution: Fixed > Check the binary packages > - > > Key: KYLIN-3862 > URL: https://issues.apache.org/jira/browse/KYLIN-3862 > Project: Kylin > Issue Type: Task >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > > As to the approval of binary packages: > > It's not possible in general to check the exact contents of a binary, however > there are some checks that should be made: > - sigs and hashes must be OK > - the package must contain the correct NOTICE and LICENSE files for the > included content > - the package must not contain any content not derived from the source. > - in the case of bundled binaries, reviewers must check that all contents are > represented in the LICENSE (and NOTICE file if required). > The bundle must not contain any files that are prohibited from distribution > (category X). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3886) Missing argument for options for yarn command
[
https://issues.apache.org/jira/browse/KYLIN-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-3886.
-
Resolution: Fixed
> Missing argument for options for yarn command
> --
>
> Key: KYLIN-3886
> URL: https://issues.apache.org/jira/browse/KYLIN-3886
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> The app id is empty which cause the yarn command failed for missing argument
>
> {code:java}
> 2019-03-13 11:48:08,604 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : Missing
> argument for options
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : usage:
> application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appStates Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application states. The valid application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : state
> can be one of the following:
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> NING,FINISHED,FAILED,KILLED
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appTypes Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application types.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -help
> Displays help for all commands.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -kill
> Kills the application.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -list
> List applications. Supports optional use
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : of
> -appTypes to filter applications based{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3893:
Fix Version/s: v2.6.2
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: v2.6.2
>
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
[
https://issues.apache.org/jira/browse/KYLIN-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-3884.
-
Resolution: Fixed
> loading hfile to HBase failed for temporary dir in output path
> ---
>
> Key: KYLIN-3884
> URL: https://issues.apache.org/jira/browse/KYLIN-3884
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> {code:java}
> 2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
> LoadIncrementalHFiles
> 2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197
> : Skipping non-directory
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
> 2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352
> : -
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> 2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
> error execute
> HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08, name=Load
> HFile to HBase Table, state=RUNNING}
> java.io.FileNotFoundException: Path is not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
> not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
[
https://issues.apache.org/jira/browse/KYLIN-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1679#comment-1679
]
ASF subversion and git services commented on KYLIN-3884:
Commit 204111999caf34686e848bca7cb552039a87babc in kylin's branch
refs/heads/2.6.x from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=2041119 ]
KYLIN-3884: loading hfile to HBase failed for temporary dir in output path
> loading hfile to HBase failed for temporary dir in output path
> ---
>
> Key: KYLIN-3884
> URL: https://issues.apache.org/jira/browse/KYLIN-3884
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> {code:java}
> 2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
> LoadIncrementalHFiles
> 2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197
> : Skipping non-directory
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
> 2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352
> : -
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> 2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
> error execute
> HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08, name=Load
> HFile to HBase Table, state=RUNNING}
> java.io.FileNotFoundException: Path is not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
> not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui reassigned KYLIN-3893:
--
Assignee: Liu Shaohui
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3886) Missing argument for options for yarn command
[
https://issues.apache.org/jira/browse/KYLIN-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796667#comment-16796667
]
ASF subversion and git services commented on KYLIN-3886:
Commit 27c77f7555f1d3e093e47d06434bf3012dc61e1e in kylin's branch
refs/heads/2.6.x from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=27c77f7 ]
KYLIN-3886: Missing argument for options for yarn command
> Missing argument for options for yarn command
> --
>
> Key: KYLIN-3886
> URL: https://issues.apache.org/jira/browse/KYLIN-3886
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> The app id is empty which cause the yarn command failed for missing argument
>
> {code:java}
> 2019-03-13 11:48:08,604 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : Missing
> argument for options
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : usage:
> application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appStates Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application states. The valid application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : state
> can be one of the following:
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> NING,FINISHED,FAILED,KILLED
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appTypes Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application types.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -help
> Displays help for all commands.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -kill
> Kills the application.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -list
> List applications. Supports optional use
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : of
> -appTypes to filter applications based{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3892) Set cubing job priority
[ https://issues.apache.org/jira/browse/KYLIN-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796649#comment-16796649 ] Yuzhang QIU commented on KYLIN-3892: +1 nice feature! > Set cubing job priority > --- > > Key: KYLIN-3892 > URL: https://issues.apache.org/jira/browse/KYLIN-3892 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > > The cubing job with high priority will be delayed when there are too many > tasks running. > So I want to set the job priority for the important cubing jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] coveralls commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description
coveralls commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description URL: https://github.com/apache/kylin/pull/534#issuecomment-474412547 ## Pull Request Test Coverage Report for [Build 4265](https://coveralls.io/builds/22270769) * **2** of **4** **(50.0%)** changed or added relevant lines in **1** file are covered. * **8** unchanged lines in **3** files lost coverage. * Overall coverage decreased (**-0.005%**) to **26.889%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-cube/src/main/java/org/apache/kylin/cube/kv/CubeDimEncMap.java](https://coveralls.io/builds/22270769/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fkv%2FCubeDimEncMap.java#L76) | 2 | 4 | 50.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-metadata/src/main/java/org/apache/kylin/source/datagen/ColumnGenerator.java](https://coveralls.io/builds/22270769/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fdatagen%2FColumnGenerator.java#L319) | 1 | 81.08% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/22270769/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/22270769/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/22270769) | | :-- | --: | | Change from base [Build 4263](https://coveralls.io/builds/22259325): | -0.005% | | Covered Lines: | 18944 | | Relevant Lines: | 70453 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3886) Missing argument for options for yarn command
[
https://issues.apache.org/jira/browse/KYLIN-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796167#comment-16796167
]
ASF GitHub Bot commented on KYLIN-3886:
---
shaofengshi commented on pull request #530: KYLIN-3886: Missing argument for
options for yarn command
URL: https://github.com/apache/kylin/pull/530
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Missing argument for options for yarn command
> --
>
> Key: KYLIN-3886
> URL: https://issues.apache.org/jira/browse/KYLIN-3886
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Priority: Major
>
> The app id is empty which cause the yarn command failed for missing argument
>
> {code:java}
> 2019-03-13 11:48:08,604 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : Missing
> argument for options
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : usage:
> application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appStates Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application states. The valid application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : state
> can be one of the following:
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> NING,FINISHED,FAILED,KILLED
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appTypes Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application types.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -help
> Displays help for all commands.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -kill
> Kills the application.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -list
> List applications. Supports optional use
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : of
> -appTypes to filter applications based{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
[
https://issues.apache.org/jira/browse/KYLIN-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3884:
---
Assignee: Liu Shaohui
Fix Version/s: v2.6.2
Component/s: Storage - HBase
> loading hfile to HBase failed for temporary dir in output path
> ---
>
> Key: KYLIN-3884
> URL: https://issues.apache.org/jira/browse/KYLIN-3884
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> {code:java}
> 2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
> LoadIncrementalHFiles
> 2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197
> : Skipping non-directory
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
> 2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352
> : -
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> 2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
> error execute
> HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08, name=Load
> HFile to HBase Table, state=RUNNING}
> java.io.FileNotFoundException: Path is not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
> not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] codecov-io commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description
codecov-io commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description URL: https://github.com/apache/kylin/pull/534#issuecomment-474411847 # [Codecov](https://codecov.io/gh/apache/kylin/pull/534?src=pr&el=h1) Report > Merging [#534](https://codecov.io/gh/apache/kylin/pull/534?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/62d5b0336b65013522e258499f640416f7e5ca82?src=pr&el=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `50%`. [](https://codecov.io/gh/apache/kylin/pull/534?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #534 +/- ## - Coverage 24.42% 24.41% -0.01% - Complexity 5025 5026 +1 Files 1161 1161 Lines 7045070453 +3 Branches 1001810018 - Hits 1720817204 -4 - Misses5150751513 +6 - Partials 1735 1736 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/534?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...n/java/org/apache/kylin/cube/kv/CubeDimEncMap.java](https://codecov.io/gh/apache/kylin/pull/534/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2t2L0N1YmVEaW1FbmNNYXAuamF2YQ==) | `61.29% <50%> (-3%)` | `5 <0> (ø)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/534/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0%> (-2.33%)` | `12% <0%> (ø)` | | | [...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/534/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=) | `70.94% <0%> (-1.36%)` | `8% <0%> (ø)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/534/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/534/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `7% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/534?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/534?src=pr&el=footer). Last update [62d5b03...49045a5](https://codecov.io/gh/apache/kylin/pull/534?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (KYLIN-3886) Missing argument for options for yarn command
[
https://issues.apache.org/jira/browse/KYLIN-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3886:
---
Assignee: Liu Shaohui
Fix Version/s: v2.6.2
Component/s: Spark Engine
> Missing argument for options for yarn command
> --
>
> Key: KYLIN-3886
> URL: https://issues.apache.org/jira/browse/KYLIN-3886
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> The app id is empty which cause the yarn command failed for missing argument
>
> {code:java}
> 2019-03-13 11:48:08,604 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : Missing
> argument for options
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : usage:
> application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appStates Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application states. The valid application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : state
> can be one of the following:
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> NING,FINISHED,FAILED,KILLED
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appTypes Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application types.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -help
> Displays help for all commands.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -kill
> Kills the application.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -list
> List applications. Supports optional use
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : of
> -appTypes to filter applications based{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3886) Missing argument for options for yarn command
[
https://issues.apache.org/jira/browse/KYLIN-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796168#comment-16796168
]
ASF subversion and git services commented on KYLIN-3886:
Commit 24d08d2527dad4d8089e3c248cd3fc08b713a658 in kylin's branch
refs/heads/master from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=24d08d2 ]
KYLIN-3886: Missing argument for options for yarn command
> Missing argument for options for yarn command
> --
>
> Key: KYLIN-3886
> URL: https://issues.apache.org/jira/browse/KYLIN-3886
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Priority: Major
>
> The app id is empty which cause the yarn command failed for missing argument
>
> {code:java}
> 2019-03-13 11:48:08,604 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : Missing
> argument for options
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : usage:
> application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appStates Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application states. The valid application
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : state
> can be one of the following:
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> NING,FINISHED,FAILED,KILLED
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> -appTypes Works with -list to filter applications
> 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on
> input comma-separated list of
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 :
> application types.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -help
> Displays help for all commands.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -kill
> Kills the application.
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -list
> List applications. Supports optional use
> 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job
> f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : of
> -appTypes to filter applications based{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] shaofengshi merged pull request #530: KYLIN-3886: Missing argument for options for yarn command
shaofengshi merged pull request #530: KYLIN-3886: Missing argument for options for yarn command URL: https://github.com/apache/kylin/pull/530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
[
https://issues.apache.org/jira/browse/KYLIN-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796164#comment-16796164
]
ASF subversion and git services commented on KYLIN-3884:
Commit 980b9674d984ee27c30b30bb63710554d7b54dcc in kylin's branch
refs/heads/master from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=980b967 ]
KYLIN-3884: loading hfile to HBase failed for temporary dir in output path
> loading hfile to HBase failed for temporary dir in output path
> ---
>
> Key: KYLIN-3884
> URL: https://issues.apache.org/jira/browse/KYLIN-3884
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> {code:java}
> 2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
> LoadIncrementalHFiles
> 2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197
> : Skipping non-directory
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
> 2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352
> : -
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> 2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
> error execute
> HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08, name=Load
> HFile to HBase Table, state=RUNNING}
> java.io.FileNotFoundException: Path is not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
> not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
[
https://issues.apache.org/jira/browse/KYLIN-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796163#comment-16796163
]
ASF GitHub Bot commented on KYLIN-3884:
---
shaofengshi commented on pull request #529: KYLIN-3884: loading hfile to HBase
failed for temporary dir in output path
URL: https://github.com/apache/kylin/pull/529
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> loading hfile to HBase failed for temporary dir in output path
> ---
>
> Key: KYLIN-3884
> URL: https://issues.apache.org/jira/browse/KYLIN-3884
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> {code:java}
> 2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
> LoadIncrementalHFiles
> 2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197
> : Skipping non-directory
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
> 2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352
> : -
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> 2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
> error execute
> HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08, name=Load
> HFile to HBase Table, state=RUNNING}
> java.io.FileNotFoundException: Path is not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
> not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] shaofengshi merged pull request #529: KYLIN-3884: loading hfile to HBase failed for temporary dir in output path
shaofengshi merged pull request #529: KYLIN-3884: loading hfile to HBase failed for temporary dir in output path URL: https://github.com/apache/kylin/pull/529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
[
https://issues.apache.org/jira/browse/KYLIN-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796161#comment-16796161
]
Shaofeng SHI commented on KYLIN-3884:
-
I see, it is clear now. Thank you shaohui!
> loading hfile to HBase failed for temporary dir in output path
> ---
>
> Key: KYLIN-3884
> URL: https://issues.apache.org/jira/browse/KYLIN-3884
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Priority: Major
>
> {code:java}
> 2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
> LoadIncrementalHFiles
> 2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197
> : Skipping non-directory
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
> 2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352
> : -
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
>
> hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> 2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
> e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
> error execute
> HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08, name=Load
> HFile to HBase Table, state=RUNNING}
> java.io.FileNotFoundException: Path is not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
> not a file:
> /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3894) Build buildSupportsSnappy Error When Doing Integration Testing
[ https://issues.apache.org/jira/browse/KYLIN-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanwen Lin updated KYLIN-3894: -- Priority: Blocker (was: Major) > Build buildSupportsSnappy Error When Doing Integration Testing > -- > > Key: KYLIN-3894 > URL: https://issues.apache.org/jira/browse/KYLIN-3894 > Project: Kylin > Issue Type: Test > Components: Tools, Build and Test >Affects Versions: v2.6.0 > Environment: Hortonworks HDP 3.0.1.0-187 Docker container. >Reporter: Yanwen Lin >Priority: Blocker > Labels: test > > Hi all, > I am currently running integration test. However, I met the following error. > Could you please share some suggestions on this? > I've passed maven install(skip test) and maven test. > > *1. Command*: > mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox > > *2. Error message from Yarn Container Attempt:* > 2019-03-18 16:43:25,583 INFO [main] org.apache.kylin.engine.mr.KylinMapper: > Accepting Mapper Key with ordinal: 12019-03-18 16:43:25,583 INFO [main] > org.apache.kylin.engine.mr.KylinMapper: Do map, available memory: > 322m2019-03-18 16:43:25,596 INFO [main] org.apache.kylin.common.KylinConfig: > Creating new manager instance of class > org.apache.kylin.cube.cuboid.CuboidManager2019-03-18 16:43:25,599 INFO [main] > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output > Committer Algorithm version is 12019-03-18 16:43:25,599 INFO [main] > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: > FileOutputCommitter skip cleanup _temporary folders under output > directory:false, ignore cleanup failures: false2019-03-18 16:43:25,795 ERROR > [main] org.apache.kylin.engine.mr.KylinMapper: > java.lang.UnsatisfiedLinkError: > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at > org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63) > at > org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136) > at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150) > at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168) > at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1304) at > org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:1192) at > org.apache.hadoop.io.SequenceFile$BlockCompressWriter.(SequenceFile.java:1552) > at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:289) at > org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:542) at > org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getSequenceWriter(SequenceFileOutputFormat.java:64) > at > org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:75) > at > org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat$LazyRecordWriter.write(LazyOutputFormat.java:113) > at > org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:468) > at > org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:85) > at > org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:44) > at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)2019-03-18 > 16:43:25,797 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Do cleanup, > available memory: 318m2019-03-18 16:43:25,813 INFO [main] > org.apache.kylin.engine.mr.KylinMapper: Total rows: 12019-03-18 16:43:25,813 > ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : > java.lang.UnsatisfiedLinkError: > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at > org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63) > at > org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136) > at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150) > at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168) > at org.apache.hadoop.io.S
[jira] [Updated] (KYLIN-3894) Build buildSupportsSnappy Error When Doing Integration Testing
[ https://issues.apache.org/jira/browse/KYLIN-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanwen Lin updated KYLIN-3894: -- Description: Hi all, I am currently running integration test. However, I met the following error. Could you please share some suggestions on this? I've passed maven install(skip test) and maven test. *1. Command*: mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox *2. Error message from Yarn Container Attempt:* 2019-03-18 16:43:25,583 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Accepting Mapper Key with ordinal: 12019-03-18 16:43:25,583 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Do map, available memory: 322m2019-03-18 16:43:25,596 INFO [main] org.apache.kylin.common.KylinConfig: Creating new manager instance of class org.apache.kylin.cube.cuboid.CuboidManager2019-03-18 16:43:25,599 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 12019-03-18 16:43:25,599 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false2019-03-18 16:43:25,795 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63) at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168) at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1304) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:1192) at org.apache.hadoop.io.SequenceFile$BlockCompressWriter.(SequenceFile.java:1552) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:289) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:542) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getSequenceWriter(SequenceFileOutputFormat.java:64) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:75) at org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat$LazyRecordWriter.write(LazyOutputFormat.java:113) at org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:468) at org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:85) at org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:44) at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)2019-03-18 16:43:25,797 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Do cleanup, available memory: 318m2019-03-18 16:43:25,813 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Total rows: 12019-03-18 16:43:25,813 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63) at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168) at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1304) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:1192) at org.apache.hadoop.io.SequenceFile$BlockCompressWriter.(SequenceFile.java:1552) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:289) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:542) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getSequenceWriter(SequenceFileOutputFormat.java:64) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:75) at org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat$LazyRecordWriter.write(LazyOutputForm
[jira] [Created] (KYLIN-3894) Build buildSupportsSnappy Error When Doing Integration Testing
Yanwen Lin created KYLIN-3894:
-
Summary: Build buildSupportsSnappy Error When Doing Integration
Testing
Key: KYLIN-3894
URL: https://issues.apache.org/jira/browse/KYLIN-3894
Project: Kylin
Issue Type: Test
Components: Tools, Build and Test
Affects Versions: v2.6.0
Environment: Hortonworks HDP 3.0.1.0-187 Docker container.
Reporter: Yanwen Lin
Hi all,
I am currently running integration test. However, I met the following error.
Could you please share some suggestions on this?
I've passed maven install(skip test) and maven test.
*1. Command*:
mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
*2. Error message from Yarn Container Attempt:*
{noformat}
2019-03-18 16:43:25,583 INFO [main] org.apache.kylin.engine.mr.KylinMapper:
Accepting Mapper Key with ordinal: 12019-03-18 16:43:25,583 INFO [main]
org.apache.kylin.engine.mr.KylinMapper: Do map, available memory:
322m2019-03-18 16:43:25,596 INFO [main] org.apache.kylin.common.KylinConfig:
Creating new manager instance of class
org.apache.kylin.cube.cuboid.CuboidManager2019-03-18 16:43:25,599 INFO [main]
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
Committer Algorithm version is 12019-03-18 16:43:25,599 INFO [main]
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter
skip cleanup _temporary folders under output directory:false, ignore cleanup
failures: false2019-03-18 16:43:25,795 ERROR [main]
org.apache.kylin.engine.mr.KylinMapper:
java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168) at
org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1304) at
org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:1192) at
org.apache.hadoop.io.SequenceFile$BlockCompressWriter.(SequenceFile.java:1552)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:289) at
org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:542) at
org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getSequenceWriter(SequenceFileOutputFormat.java:64)
at
org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:75)
at
org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat$LazyRecordWriter.write(LazyOutputFormat.java:113)
at
org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:468)
at
org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:85)
at
org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:44)
at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77) at
org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)2019-03-18
16:43:25,797 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Do cleanup,
available memory: 318m2019-03-18 16:43:25,813 INFO [main]
org.apache.kylin.engine.mr.KylinMapper: Total rows: 12019-03-18 16:43:25,813
ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child :
java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168) at
org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1304) at
org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:1192) at
org.apache.hadoop.io.SequenceFile$BlockCompressWriter.(SequenceFile.java:1552)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:289) at
org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:542) at
org.apache.hadoop.mapreduce.lib.output.SequenceFileO
[jira] [Commented] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796130#comment-16796130
]
ASF GitHub Bot commented on KYLIN-3893:
---
lshmouse commented on pull request #534: KYLIN-3893: Cube build failed for
wrong row key column description
URL: https://github.com/apache/kylin/pull/534
See: https://issues.apache.org/jira/browse/KYLIN-3893
We can catch the exception and make the exception the issue clear.
It's better to check this in the front end, but i can not find how to fix
it. Anyone can give some advice? Thanks~
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Priority: Minor
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] asfgit commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description
asfgit commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description URL: https://github.com/apache/kylin/pull/534#issuecomment-474389016 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description
asfgit commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description URL: https://github.com/apache/kylin/pull/534#issuecomment-474389012 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] lshmouse opened a new pull request #534: KYLIN-3893: Cube build failed for wrong row key column description
lshmouse opened a new pull request #534: KYLIN-3893: Cube build failed for wrong row key column description URL: https://github.com/apache/kylin/pull/534 See: https://issues.apache.org/jira/browse/KYLIN-3893 We can catch the exception and make the exception the issue clear. It's better to check this in the front end, but i can not find how to fix it. Anyone can give some advice? Thanks~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796116#comment-16796116
]
yuzhang commented on KYLIN-3893:
aha! I meet this error too tonight
yuzhang
Email:shifengdefan...@163.com
Signature is customized by Netease Mail Master
On 03/19/2019 21:46, Liu Shaohui (JIRA) wrote:
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated KYLIN-3893:
---
Priority: Minor (was: Major)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Priority: Minor
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3893) Cube build failed for wrong row key column description
Liu Shaohui created KYLIN-3893:
--
Summary: Cube build failed for wrong row key column description
Key: KYLIN-3893
URL: https://issues.apache.org/jira/browse/KYLIN-3893
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
User created wrong RowKeyColDesc, eg,
RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
encoding=integer:undefined}
which cause the cube build forever.
{code:java}
org.apache.kylin.engine.mr.exception.HadoopShellException:
java.lang.NumberFormatException: For input string: "undefined" at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580) at
java.lang.Integer.parseInt(Integer.java:615) at
org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
at
org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74) at
org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
at
org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
at
org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)result code:2 at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated KYLIN-3893:
---
Priority: Minor (was: Major)
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Priority: Minor
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-3474: --- Affects Version/s: v2.6.1 v2.5.0 > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.1 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Fix For: v2.4.1, v2.4.2 > > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou reopened KYLIN-3474: The problem still exists. :( > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Fix For: v2.4.1, v2.4.2 > > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3881) Calcite isolating expression with its condition may throw 'Division Undefined' exception
[
https://issues.apache.org/jira/browse/KYLIN-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795906#comment-16795906
]
Zhong Yanghong commented on KYLIN-3881:
---
The patch is based on calcite tag *calcite-1.16.0*.
> Calcite isolating expression with its condition may throw 'Division
> Undefined' exception
>
>
> Key: KYLIN-3881
> URL: https://issues.apache.org/jira/browse/KYLIN-3881
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
> Attachments: APACHE-KYLIN-3881.patch
>
>
> When invoke the following sql:
> {code:java}
> SELECT LSTG_FORMAT_NAME, (case when sum(PRICE) <> 0 then
> sum(PRICE)/min(price) else 0 end)
> FROM TEST_KYLIN_FACT
> GROUP BY LSTG_FORMAT_NAME
> HAVING (case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end >= 0.5
> and case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end
> <= 0.75)
> {code}
> The generated code of moveMext part is as follow:
> {code:java}
> /* 157 */ public boolean moveNext() {
> /* 158 */ while (inputEnumerator.moveNext()) {
> /* 159 */ final Object[] current = (Object[]) inputEnumerator.current();
> /* 160 */ final java.math.BigDecimal inp1_ = current[1] == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[1]);
> /* 161 */ final java.math.BigDecimal inp2_ = current[2] == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[2]);
> /* 162 */ final java.math.BigDecimal v0 =
> org.apache.calcite.runtime.SqlFunctions.divide(inp1_, inp2_);
> /* 163 */ if (inp1_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
> $L4J$C$new_java_math_BigDecimal_0_) && (inp1_ != null && inp2_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ge(v0,
> $L4J$C$new_java_math_BigDecimal_0_5_)) && (!(inp1_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
> $L4J$C$new_java_math_BigDecimal_0_)) || inp1_ != null && inp2_ != null &&
> org.apache.calcite.runtime.SqlFunctions.le(v0,
> $L4J$C$new_java_math_BigDecimal_0_75_))) {
> /* 164 */ return true;
> /* 165 */ }
> /* 166 */ }
> /* 167 */ return false;
> /* 168 */ }
> {code}
> line 162, the divide logic should be combined with its condition. Otherwise,
> the 'Division Undefined' exception will happen.
> The whole generated code is as follows:
> {code:java}
> /* 1 */ public static class Record4_0 implements java.io.Serializable {
> /* 2 */ public java.math.BigDecimal f0;
> /* 3 */ public boolean f1;
> /* 4 */ public java.math.BigDecimal f2;
> /* 5 */ public boolean f3;
> /* 6 */ public Record4_0() {}
> /* 7 */ public boolean equals(Object o) {
> /* 8 */ if (this == o) {
> /* 9 */ return true;
> /* 10 */ }
> /* 11 */ if (!(o instanceof Record4_0)) {
> /* 12 */ return false;
> /* 13 */ }
> /* 14 */ return java.util.Objects.equals(this.f0, ((Record4_0) o).f0) &&
> this.f1 == ((Record4_0) o).f1 && java.util.Objects.equals(this.f2,
> ((Record4_0) o).f2) && this.f3 == ((Record4_0) o).f3;
> /* 15 */ }
> /* 16 */
> /* 17 */ public int hashCode() {
> /* 18 */ int h = 0;
> /* 19 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f0);
> /* 20 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f1);
> /* 21 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f2);
> /* 22 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f3);
> /* 23 */ return h;
> /* 24 */ }
> /* 25 */
> /* 26 */ public int compareTo(Record4_0 that) {
> /* 27 */ int c;
> /* 28 */ c =
> org.apache.calcite.runtime.Utilities.compareNullsLast(this.f0, that.f0);
> /* 29 */ if (c != 0) {
> /* 30 */ return c;
> /* 31 */ }
> /* 32 */ c = org.apache.calcite.runtime.Utilities.compare(this.f1,
> that.f1);
> /* 33 */ if (c != 0) {
> /* 34 */ return c;
> /* 35 */ }
> /* 36 */ c =
> org.apache.calcite.runtime.Utilities.compareNullsLast(this.f2, that.f2);
> /* 37 */ if (c != 0) {
> /* 38 */ return c;
> /* 39 */ }
> /* 40 */ c = org.apache.calcite.runtime.Utilities.compare(this.f3,
> that.f3);
> /* 41 */ if (c != 0) {
> /* 42 */ return c;
> /* 43 */ }
> /* 44 */ return 0;
> /* 45 */ }
> /* 46 */
> /* 47 */ public String toString() {
> /* 48 */ return "{f0=" + this.f0 + ", f1=" + this.f1 + ", f2=" + this.f2
> + ", f3=" + this.f3 + "}";
> /* 49 */ }
> /* 50 */
> /* 51 */ }
> /* 52 */
> /* 53 */ org.apache.calcite.DataContext root;
> /* 54 */
> /* 55 */ public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.c
[jira] [Updated] (KYLIN-3881) Calcite isolating expression with its condition may throw 'Division Undefined' exception
[
https://issues.apache.org/jira/browse/KYLIN-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-3881:
--
Attachment: APACHE-KYLIN-3881.patch
> Calcite isolating expression with its condition may throw 'Division
> Undefined' exception
>
>
> Key: KYLIN-3881
> URL: https://issues.apache.org/jira/browse/KYLIN-3881
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
> Attachments: APACHE-KYLIN-3881.patch
>
>
> When invoke the following sql:
> {code:java}
> SELECT LSTG_FORMAT_NAME, (case when sum(PRICE) <> 0 then
> sum(PRICE)/min(price) else 0 end)
> FROM TEST_KYLIN_FACT
> GROUP BY LSTG_FORMAT_NAME
> HAVING (case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end >= 0.5
> and case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end
> <= 0.75)
> {code}
> The generated code of moveMext part is as follow:
> {code:java}
> /* 157 */ public boolean moveNext() {
> /* 158 */ while (inputEnumerator.moveNext()) {
> /* 159 */ final Object[] current = (Object[]) inputEnumerator.current();
> /* 160 */ final java.math.BigDecimal inp1_ = current[1] == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[1]);
> /* 161 */ final java.math.BigDecimal inp2_ = current[2] == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[2]);
> /* 162 */ final java.math.BigDecimal v0 =
> org.apache.calcite.runtime.SqlFunctions.divide(inp1_, inp2_);
> /* 163 */ if (inp1_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
> $L4J$C$new_java_math_BigDecimal_0_) && (inp1_ != null && inp2_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ge(v0,
> $L4J$C$new_java_math_BigDecimal_0_5_)) && (!(inp1_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
> $L4J$C$new_java_math_BigDecimal_0_)) || inp1_ != null && inp2_ != null &&
> org.apache.calcite.runtime.SqlFunctions.le(v0,
> $L4J$C$new_java_math_BigDecimal_0_75_))) {
> /* 164 */ return true;
> /* 165 */ }
> /* 166 */ }
> /* 167 */ return false;
> /* 168 */ }
> {code}
> line 162, the divide logic should be combined with its condition. Otherwise,
> the 'Division Undefined' exception will happen.
> The whole generated code is as follows:
> {code:java}
> /* 1 */ public static class Record4_0 implements java.io.Serializable {
> /* 2 */ public java.math.BigDecimal f0;
> /* 3 */ public boolean f1;
> /* 4 */ public java.math.BigDecimal f2;
> /* 5 */ public boolean f3;
> /* 6 */ public Record4_0() {}
> /* 7 */ public boolean equals(Object o) {
> /* 8 */ if (this == o) {
> /* 9 */ return true;
> /* 10 */ }
> /* 11 */ if (!(o instanceof Record4_0)) {
> /* 12 */ return false;
> /* 13 */ }
> /* 14 */ return java.util.Objects.equals(this.f0, ((Record4_0) o).f0) &&
> this.f1 == ((Record4_0) o).f1 && java.util.Objects.equals(this.f2,
> ((Record4_0) o).f2) && this.f3 == ((Record4_0) o).f3;
> /* 15 */ }
> /* 16 */
> /* 17 */ public int hashCode() {
> /* 18 */ int h = 0;
> /* 19 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f0);
> /* 20 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f1);
> /* 21 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f2);
> /* 22 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f3);
> /* 23 */ return h;
> /* 24 */ }
> /* 25 */
> /* 26 */ public int compareTo(Record4_0 that) {
> /* 27 */ int c;
> /* 28 */ c =
> org.apache.calcite.runtime.Utilities.compareNullsLast(this.f0, that.f0);
> /* 29 */ if (c != 0) {
> /* 30 */ return c;
> /* 31 */ }
> /* 32 */ c = org.apache.calcite.runtime.Utilities.compare(this.f1,
> that.f1);
> /* 33 */ if (c != 0) {
> /* 34 */ return c;
> /* 35 */ }
> /* 36 */ c =
> org.apache.calcite.runtime.Utilities.compareNullsLast(this.f2, that.f2);
> /* 37 */ if (c != 0) {
> /* 38 */ return c;
> /* 39 */ }
> /* 40 */ c = org.apache.calcite.runtime.Utilities.compare(this.f3,
> that.f3);
> /* 41 */ if (c != 0) {
> /* 42 */ return c;
> /* 43 */ }
> /* 44 */ return 0;
> /* 45 */ }
> /* 46 */
> /* 47 */ public String toString() {
> /* 48 */ return "{f0=" + this.f0 + ", f1=" + this.f1 + ", f2=" + this.f2
> + ", f3=" + this.f3 + "}";
> /* 49 */ }
> /* 50 */
> /* 51 */ }
> /* 52 */
> /* 53 */ org.apache.calcite.DataContext root;
> /* 54 */
> /* 55 */ public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root0) {
> /* 56 */ root = root0;
> /* 57 */
[jira] [Updated] (KYLIN-3881) Calcite isolating expression with its condition may throw 'Division Undefined' exception
[
https://issues.apache.org/jira/browse/KYLIN-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-3881:
--
Description:
When invoke the following sql:
{code:java}
SELECT LSTG_FORMAT_NAME, (case when sum(PRICE) <> 0 then sum(PRICE)/min(price)
else 0 end)
FROM TEST_KYLIN_FACT
GROUP BY LSTG_FORMAT_NAME
HAVING (case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end >= 0.5
and case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end <=
0.75)
{code}
The generated code of moveMext part is as follow:
{code:java}
/* 157 */ public boolean moveNext() {
/* 158 */ while (inputEnumerator.moveNext()) {
/* 159 */ final Object[] current = (Object[]) inputEnumerator.current();
/* 160 */ final java.math.BigDecimal inp1_ = current[1] == null ?
(java.math.BigDecimal) null :
org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[1]);
/* 161 */ final java.math.BigDecimal inp2_ = current[2] == null ?
(java.math.BigDecimal) null :
org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[2]);
/* 162 */ final java.math.BigDecimal v0 =
org.apache.calcite.runtime.SqlFunctions.divide(inp1_, inp2_);
/* 163 */ if (inp1_ != null &&
org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
$L4J$C$new_java_math_BigDecimal_0_) && (inp1_ != null && inp2_ != null &&
org.apache.calcite.runtime.SqlFunctions.ge(v0,
$L4J$C$new_java_math_BigDecimal_0_5_)) && (!(inp1_ != null &&
org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
$L4J$C$new_java_math_BigDecimal_0_)) || inp1_ != null && inp2_ != null &&
org.apache.calcite.runtime.SqlFunctions.le(v0,
$L4J$C$new_java_math_BigDecimal_0_75_))) {
/* 164 */ return true;
/* 165 */ }
/* 166 */ }
/* 167 */ return false;
/* 168 */ }
{code}
line 162, the divide logic should be combined with its condition. Otherwise,
the 'Division Undefined' exception will happen.
The whole generated code is as follows:
{code:java}
/* 1 */ public static class Record4_0 implements java.io.Serializable {
/* 2 */ public java.math.BigDecimal f0;
/* 3 */ public boolean f1;
/* 4 */ public java.math.BigDecimal f2;
/* 5 */ public boolean f3;
/* 6 */ public Record4_0() {}
/* 7 */ public boolean equals(Object o) {
/* 8 */ if (this == o) {
/* 9 */ return true;
/* 10 */ }
/* 11 */ if (!(o instanceof Record4_0)) {
/* 12 */ return false;
/* 13 */ }
/* 14 */ return java.util.Objects.equals(this.f0, ((Record4_0) o).f0) &&
this.f1 == ((Record4_0) o).f1 && java.util.Objects.equals(this.f2, ((Record4_0)
o).f2) && this.f3 == ((Record4_0) o).f3;
/* 15 */ }
/* 16 */
/* 17 */ public int hashCode() {
/* 18 */ int h = 0;
/* 19 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f0);
/* 20 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f1);
/* 21 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f2);
/* 22 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f3);
/* 23 */ return h;
/* 24 */ }
/* 25 */
/* 26 */ public int compareTo(Record4_0 that) {
/* 27 */ int c;
/* 28 */ c =
org.apache.calcite.runtime.Utilities.compareNullsLast(this.f0, that.f0);
/* 29 */ if (c != 0) {
/* 30 */ return c;
/* 31 */ }
/* 32 */ c = org.apache.calcite.runtime.Utilities.compare(this.f1,
that.f1);
/* 33 */ if (c != 0) {
/* 34 */ return c;
/* 35 */ }
/* 36 */ c =
org.apache.calcite.runtime.Utilities.compareNullsLast(this.f2, that.f2);
/* 37 */ if (c != 0) {
/* 38 */ return c;
/* 39 */ }
/* 40 */ c = org.apache.calcite.runtime.Utilities.compare(this.f3,
that.f3);
/* 41 */ if (c != 0) {
/* 42 */ return c;
/* 43 */ }
/* 44 */ return 0;
/* 45 */ }
/* 46 */
/* 47 */ public String toString() {
/* 48 */ return "{f0=" + this.f0 + ", f1=" + this.f1 + ", f2=" + this.f2 +
", f3=" + this.f3 + "}";
/* 49 */ }
/* 50 */
/* 51 */ }
/* 52 */
/* 53 */ org.apache.calcite.DataContext root;
/* 54 */
/* 55 */ public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root0) {
/* 56 */ root = root0;
/* 57 */ final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
((org.apache.kylin.query.schema.OLAPTable)
root.getRootSchema().getSubSchema("DEFAULT").getTable("TEST_KYLIN_FACT")).executeOLAPQuery(root,
0);
/* 58 */ final org.apache.calcite.linq4j.AbstractEnumerable child = new
org.apache.calcite.linq4j.AbstractEnumerable(){
/* 59 */ public org.apache.calcite.linq4j.Enumerator enumerator() {
/* 60 */ return new org.apache.calcite.linq4j.Enumerator(){
/* 61 */ public final org.apache.calcite.linq4j.Enumerator
inputEnumerator = _inputEnumerable.enumerator();
/* 62 */ public void reset() {
/* 63 */ inputEnumerator.reset();
/* 64 */ }
/* 65 */
/* 66 */ public boolean moveNext() {
/* 67 */
[GitHub] [kylin] hit-lacus commented on a change in pull request #507: KYLIN-3857 Disable quote when needed
hit-lacus commented on a change in pull request #507: KYLIN-3857 Disable quote
when needed
URL: https://github.com/apache/kylin/pull/507#discussion_r266774070
##
File path:
core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
##
@@ -982,6 +982,14 @@ public int getDefaultDecimalScale() {
}
}
+public boolean enableHiveDdlQuote(){
+return
Boolean.parseBoolean(getOptional("kylin.source.hive.quote-enabled", TRUE));
+}
+
+public static String getQuoteCharacter(){
Review comment:
Make it configurable do not make harm, I will change it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hit-lacus commented on a change in pull request #507: KYLIN-3857 Disable quote when needed
hit-lacus commented on a change in pull request #507: KYLIN-3857 Disable quote
when needed
URL: https://github.com/apache/kylin/pull/507#discussion_r266772588
##
File path:
core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
##
@@ -982,6 +982,14 @@ public int getDefaultDecimalScale() {
}
}
+public boolean enableHiveDdlQuote(){
+return
Boolean.parseBoolean(getOptional("kylin.source.hive.quote-enabled", TRUE));
+}
+
+public static String getQuoteCharacter(){
Review comment:
The quote character used by Hive is **`**, I think it wouldn't be changed
easily.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (KYLIN-3882) kylin master build failed for pom issues
[
https://issues.apache.org/jira/browse/KYLIN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-3882.
-
Resolution: Fixed
> kylin master build failed for pom issues
>
>
> Key: KYLIN-3882
> URL: https://issues.apache.org/jira/browse/KYLIN-3882
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> As title.
> 1, Kyligence repo id : nexus conflicts with local maven settings.xml
> {code:java}
> [ERROR] Failed to execute goal on project kylin-core-metadata: Could not
> resolve dependencies for project
> org.apache.kylin:kylin-core-metadata:jar:3.0.0-SNAPSHOT: Failure to find
> org.apache.calcite:calcite-core:jar:1.16.0-kylin-r2 in
> http://nexus.x./nexus/content/groups/public was cached in the local
> repository, resolution will not be reattempted until the update interval of
> nexus has elapsed or updates are forced -> [Help 1]
> {code}
>
> 2, maven.compiler.source/target is not set
> {code:java}
> [INFO] Compiling 2 Scala sources and 18 Java sources to
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/target/classes ...
> [WARNING] [Warn] : bootstrap class path not set in conjunction with -source
> 1.6
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkBatchCubingJobBuilder2.java:148:
> diamond operator is not supported in -source 1.6
> (use -source 7 or higher to enable diamond operator)
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:239:
> try-with-resources is not supported in -source 1.6
> (use -source 7 or higher to enable try-with-resources)
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:251:
> diamond operator is not supported in -source 1.6
> (use -source 7 or higher to enable diamond operator){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3882) kylin master build failed for pom issues
[
https://issues.apache.org/jira/browse/KYLIN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3882:
Fix Version/s: v2.6.2
> kylin master build failed for pom issues
>
>
> Key: KYLIN-3882
> URL: https://issues.apache.org/jira/browse/KYLIN-3882
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> As title.
> 1, Kyligence repo id : nexus conflicts with local maven settings.xml
> {code:java}
> [ERROR] Failed to execute goal on project kylin-core-metadata: Could not
> resolve dependencies for project
> org.apache.kylin:kylin-core-metadata:jar:3.0.0-SNAPSHOT: Failure to find
> org.apache.calcite:calcite-core:jar:1.16.0-kylin-r2 in
> http://nexus.x./nexus/content/groups/public was cached in the local
> repository, resolution will not be reattempted until the update interval of
> nexus has elapsed or updates are forced -> [Help 1]
> {code}
>
> 2, maven.compiler.source/target is not set
> {code:java}
> [INFO] Compiling 2 Scala sources and 18 Java sources to
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/target/classes ...
> [WARNING] [Warn] : bootstrap class path not set in conjunction with -source
> 1.6
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkBatchCubingJobBuilder2.java:148:
> diamond operator is not supported in -source 1.6
> (use -source 7 or higher to enable diamond operator)
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:239:
> try-with-resources is not supported in -source 1.6
> (use -source 7 or higher to enable try-with-resources)
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:251:
> diamond operator is not supported in -source 1.6
> (use -source 7 or higher to enable diamond operator){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] shaofengshi commented on a change in pull request #507: KYLIN-3857 Disable quote when needed
shaofengshi commented on a change in pull request #507: KYLIN-3857 Disable
quote when needed
URL: https://github.com/apache/kylin/pull/507#discussion_r266769769
##
File path:
core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
##
@@ -982,6 +982,14 @@ public int getDefaultDecimalScale() {
}
}
+public boolean enableHiveDdlQuote(){
+return
Boolean.parseBoolean(getOptional("kylin.source.hive.quote-enabled", TRUE));
+}
+
+public static String getQuoteCharacter(){
Review comment:
Why not make this configurable?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] xbirbird commented on issue #528: KYLIN-3883 Kylin supports column count aggregation
xbirbird commented on issue #528: KYLIN-3883 Kylin supports column count aggregation URL: https://github.com/apache/kylin/pull/528#issuecomment-474239909 Ok, I'll add some test cases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-3883) Kylin supports column count aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3883: Fix Version/s: v3.0.0 > Kylin supports column count aggregation > --- > > Key: KYLIN-3883 > URL: https://issues.apache.org/jira/browse/KYLIN-3883 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: all >Reporter: xiaodongzhang >Assignee: xiaodongzhang >Priority: Major > Fix For: v3.0.0 > > > Kylin目前只支持对常量1进行count聚合预计算，即count(1)，count(col_1)的查询内部全部重写到对count(1)的查询，这样会导致一个问题：当col_1列中存在null值时，count(col_1)的查询结果并不准确。这样就导致Kylin的查询结果和Hive、Spark等不一致。该Patch中，提供了对count(col_1)的支持。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3892) Set cubing job priority
[ https://issues.apache.org/jira/browse/KYLIN-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795810#comment-16795810 ] Shaofeng SHI commented on KYLIN-3892: - +1 welcome to make Kylin better! > Set cubing job priority > --- > > Key: KYLIN-3892 > URL: https://issues.apache.org/jira/browse/KYLIN-3892 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > > The cubing job with high priority will be delayed when there are too many > tasks running. > So I want to set the job priority for the important cubing jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
