[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909130#comment-16909130 ] Ajay commented on KYLIN-3989: - Any further update on this issue > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896706#comment-16896706 ] Yaqian Zhang commented on KYLIN-3989: - Hi Frederic, Sorry for late. You answered yourself question very well. Now do you have any other questions? > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846510#comment-16846510 ] Frederic Souchu commented on KYLIN-3989: I tracked down the 'bug' to this line: {{\core-common\src\main\java\org\apache\kylin\common\KylinConfigBase.java}} {code:java} public String getHdfsWorkingDirectory() { ... // append metadata-url prefix root = new Path(path, StringUtils.replaceChars(getMetadataUrlPrefix(), ':', '-')).toString(); ... }{code} Whatever 'path' value is, it always comes back as 'getMetadataUrlPrefix()'. This is somehow expected as 1st and 2nd parameters are from 2 completely different roots! > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836148#comment-16836148 ] Frederic Souchu commented on KYLIN-3989: No effect - tried: {code:java} maprfs:///apps/kylin/ /apps/kylin/{code} The setting seems to be completely ignored (except for this HBase log line - seems incorrect as well since HBase is actually working in /apps/kylin_metadata!!). > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835090#comment-16835090 ] Iñigo Martinez commented on KYLIN-3989: --- Try adding an extra slash at the end: {code:java} kylin.env.hdfs-working-dir=maprfs:///apps/kylin/{code} > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834566#comment-16834566 ] Frederic Souchu commented on KYLIN-3989: Tried with 2.6.1 with same result - configuration setting is ignored by HIVE statement :/ > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834549#comment-16834549 ] Frederic Souchu commented on KYLIN-3989: I tried with qualified and unqualified path but HIVE location keeps using HBase path. I can see in 'System' page: kylin.env.hdfs-working-dir=maprfs:///apps/kylin >From logs: ... 2019-05-07 08:54:00,198 DEBUG [http-bio-7070-exec-2] hbase.HBaseConnection:180 : Using the working dir FS for HBase: maprfs:///apps/kylin ... But HIVE still issues command to /apps/kylin_metadata CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_txn_cube_a945be5b_f275_3435_9122_5288330589b1 ( PAYMENTS_GLOBALMERCHANTUID int ) STORED AS SEQUENCEFILE LOCATION 'maprfs:///apps/kylin_metadata/kylin-132800cf-527e-c0f3-91e5-ecede8896c15/kylin_intermediate_txn_cube_a945be5b_f275_3435_9122_5288330589b1'; Any clue? > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834469#comment-16834469 ] Iñigo Martinez commented on KYLIN-3989: --- Try with kylin.env.hdfs-working-dir=maprfs:///path/ > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833761#comment-16833761 ] Frederic Souchu commented on KYLIN-3989: Understood - one question still (might help me solve the issue myself): * Kylin creates a HBase table for metadata: /apps/kylin_metadata * Kylin tries to create a sequence file within that HBase table: maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e I don't get how this was supposed to work in the first place. Secondly, is there a configuration key I can use to move temporary files under a different root? > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[ https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16827630#comment-16827630 ] Shaofeng SHI commented on KYLIN-3989: - Hello Frederic, Apache Kylin doesn't support MapR out-of-box. You can contact MapR or Kyligence to get support as they have the joint solution. > Invalid temporary table path for kylin_metadata > --- > > Key: KYLIN-3989 > URL: https://issues.apache.org/jira/browse/KYLIN-3989 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 > Environment: MapR 6.1 > Hive 2.3 >Reporter: Frederic Souchu >Priority: Major > > How to reproduce: > * create a mode > * define a cube > * build the cube > The cube building will fail with the following Hive logs: > {code:java} > USE default; > No rows affected (0.067 seconds) > 0: jdbc:hive2://x.com> > 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e; > No rows affected (0.023 seconds) > 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09 > e > . . . . . . . . . . . . . . . . . . . . . . .> ( > . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int > . . . . . . . . . . . . . . . . . . . . . . .> ) > . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE > . . . . . . . . . . . . . . . . . . . . . . .> LOCATION > 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia > te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e'; > Error: org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > java.io.IOException Error: Not a directory(20), file: > kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name: > mapr, ID: 5000) > {code} > using metastore.sh to list content gives: > {code} > 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading > kylin-defaults.properties from > file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties > 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF > property was not set, will seek KYLIN_HOME env variable > 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a > new KylinConfig from getInstanceFromEnv : 453523494 > 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using > metadata url /apps/kylin_metadata@hbase for resource store > 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the > working dir FS for HBase: maprfs:/// > 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection > is null or closed, creating a new one > 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 : > ConnectionFactory receives mapr.hbase.default.db(maprdb), set > clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false) > 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable > '/apps/kylin_metadata' already exists > null > 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 : > Closing HBase connections... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)