[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[
https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16846510#comment-16846510
]
Frederic Souchu commented on KYLIN-3989:
I tracked down the 'bug' to this line:
{{\core-common\src\main\java\org\apache\kylin\common\KylinConfigBase.java}}
{code:java}
public String getHdfsWorkingDirectory() {
...
// append metadata-url prefix
root = new Path(path, StringUtils.replaceChars(getMetadataUrlPrefix(), ':',
'-')).toString();
...
}{code}
Whatever 'path' value is, it always comes back as 'getMetadataUrlPrefix()'.
This is somehow expected as 1st and 2nd parameters are from 2 completely
different roots!
> Invalid temporary table path for kylin_metadata
> ---
>
> Key: KYLIN-3989
> URL: https://issues.apache.org/jira/browse/KYLIN-3989
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.2
> Environment: MapR 6.1
> Hive 2.3
>Reporter: Frederic Souchu
>Priority: Major
>
> How to reproduce:
> * create a mode
> * define a cube
> * build the cube
> The cube building will fail with the following Hive logs:
> {code:java}
> USE default;
> No rows affected (0.067 seconds)
> 0: jdbc:hive2://x.com>
> 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
> e
> . . . . . . . . . . . . . . . . . . . . . . .> (
> . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
> . . . . . . . . . . . . . . . . . . . . . . .> )
> . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
> . . . . . . . . . . . . . . . . . . . . . . .> LOCATION
> 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
> te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
> java.io.IOException Error: Not a directory(20), file:
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
> mapr, ID: 5000)
> {code}
> using metastore.sh to list content gives:
> {code}
> 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
> kylin-defaults.properties from
> file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
> 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
> property was not set, will seek KYLIN_HOME env variable
> 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a
> new KylinConfig from getInstanceFromEnv : 453523494
> 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
> metadata url /apps/kylin_metadata@hbase for resource store
> 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
> working dir FS for HBase: maprfs:///
> 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection
> is null or closed, creating a new one
> 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
> ConnectionFactory receives mapr.hbase.default.db(maprdb), set
> clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
> 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
> '/apps/kylin_metadata' already exists
> null
> 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
> Closing HBase connections...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[
https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836148#comment-16836148
]
Frederic Souchu commented on KYLIN-3989:
No effect - tried:
{code:java}
maprfs:///apps/kylin/
/apps/kylin/{code}
The setting seems to be completely ignored (except for this HBase log line -
seems incorrect as well since HBase is actually working in
/apps/kylin_metadata!!).
> Invalid temporary table path for kylin_metadata
> ---
>
> Key: KYLIN-3989
> URL: https://issues.apache.org/jira/browse/KYLIN-3989
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.2
> Environment: MapR 6.1
> Hive 2.3
>Reporter: Frederic Souchu
>Priority: Major
>
> How to reproduce:
> * create a mode
> * define a cube
> * build the cube
> The cube building will fail with the following Hive logs:
> {code:java}
> USE default;
> No rows affected (0.067 seconds)
> 0: jdbc:hive2://x.com>
> 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
> e
> . . . . . . . . . . . . . . . . . . . . . . .> (
> . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
> . . . . . . . . . . . . . . . . . . . . . . .> )
> . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
> . . . . . . . . . . . . . . . . . . . . . . .> LOCATION
> 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
> te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
> java.io.IOException Error: Not a directory(20), file:
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
> mapr, ID: 5000)
> {code}
> using metastore.sh to list content gives:
> {code}
> 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
> kylin-defaults.properties from
> file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
> 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
> property was not set, will seek KYLIN_HOME env variable
> 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a
> new KylinConfig from getInstanceFromEnv : 453523494
> 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
> metadata url /apps/kylin_metadata@hbase for resource store
> 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
> working dir FS for HBase: maprfs:///
> 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection
> is null or closed, creating a new one
> 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
> ConnectionFactory receives mapr.hbase.default.db(maprdb), set
> clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
> 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
> '/apps/kylin_metadata' already exists
> null
> 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
> Closing HBase connections...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[
https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834566#comment-16834566
]
Frederic Souchu commented on KYLIN-3989:
Tried with 2.6.1 with same result - configuration setting is ignored by HIVE
statement :/
> Invalid temporary table path for kylin_metadata
> ---
>
> Key: KYLIN-3989
> URL: https://issues.apache.org/jira/browse/KYLIN-3989
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.2
> Environment: MapR 6.1
> Hive 2.3
>Reporter: Frederic Souchu
>Priority: Major
>
> How to reproduce:
> * create a mode
> * define a cube
> * build the cube
> The cube building will fail with the following Hive logs:
> {code:java}
> USE default;
> No rows affected (0.067 seconds)
> 0: jdbc:hive2://x.com>
> 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
> e
> . . . . . . . . . . . . . . . . . . . . . . .> (
> . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
> . . . . . . . . . . . . . . . . . . . . . . .> )
> . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
> . . . . . . . . . . . . . . . . . . . . . . .> LOCATION
> 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
> te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
> java.io.IOException Error: Not a directory(20), file:
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
> mapr, ID: 5000)
> {code}
> using metastore.sh to list content gives:
> {code}
> 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
> kylin-defaults.properties from
> file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
> 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
> property was not set, will seek KYLIN_HOME env variable
> 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a
> new KylinConfig from getInstanceFromEnv : 453523494
> 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
> metadata url /apps/kylin_metadata@hbase for resource store
> 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
> working dir FS for HBase: maprfs:///
> 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection
> is null or closed, creating a new one
> 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
> ConnectionFactory receives mapr.hbase.default.db(maprdb), set
> clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
> 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
> '/apps/kylin_metadata' already exists
> null
> 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
> Closing HBase connections...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[
https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834549#comment-16834549
]
Frederic Souchu commented on KYLIN-3989:
I tried with qualified and unqualified path but HIVE location keeps using HBase
path.
I can see in 'System' page:
kylin.env.hdfs-working-dir=maprfs:///apps/kylin
>From logs:
...
2019-05-07 08:54:00,198 DEBUG [http-bio-7070-exec-2] hbase.HBaseConnection:180
: Using the working dir FS for HBase: maprfs:///apps/kylin
...
But HIVE still issues command to /apps/kylin_metadata
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_txn_cube_a945be5b_f275_3435_9122_5288330589b1
(
PAYMENTS_GLOBALMERCHANTUID int
)
STORED AS SEQUENCEFILE
LOCATION
'maprfs:///apps/kylin_metadata/kylin-132800cf-527e-c0f3-91e5-ecede8896c15/kylin_intermediate_txn_cube_a945be5b_f275_3435_9122_5288330589b1';
Any clue?
> Invalid temporary table path for kylin_metadata
> ---
>
> Key: KYLIN-3989
> URL: https://issues.apache.org/jira/browse/KYLIN-3989
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.2
> Environment: MapR 6.1
> Hive 2.3
>Reporter: Frederic Souchu
>Priority: Major
>
> How to reproduce:
> * create a mode
> * define a cube
> * build the cube
> The cube building will fail with the following Hive logs:
> {code:java}
> USE default;
> No rows affected (0.067 seconds)
> 0: jdbc:hive2://x.com>
> 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
> e
> . . . . . . . . . . . . . . . . . . . . . . .> (
> . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
> . . . . . . . . . . . . . . . . . . . . . . .> )
> . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
> . . . . . . . . . . . . . . . . . . . . . . .> LOCATION
> 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
> te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
> java.io.IOException Error: Not a directory(20), file:
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
> mapr, ID: 5000)
> {code}
> using metastore.sh to list content gives:
> {code}
> 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
> kylin-defaults.properties from
> file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
> 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
> property was not set, will seek KYLIN_HOME env variable
> 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a
> new KylinConfig from getInstanceFromEnv : 453523494
> 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
> metadata url /apps/kylin_metadata@hbase for resource store
> 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
> working dir FS for HBase: maprfs:///
> 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection
> is null or closed, creating a new one
> 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
> ConnectionFactory receives mapr.hbase.default.db(maprdb), set
> clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
> 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
> '/apps/kylin_metadata' already exists
> null
> 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
> Closing HBase connections...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[
https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833761#comment-16833761
]
Frederic Souchu edited comment on KYLIN-3989 at 5/6/19 1:23 PM:
Understood - [~Shaofengshi] one question still (might help me solve the issue
myself):
* Kylin creates a HBase table for metadata: /apps/kylin_metadata
* Kylin tries to create a sequence file within that HBase table:
maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e
I don't get how this was supposed to work in the first place (e.g. creating an
external HIVE table pointing to a HBase path?!?!)
Is there a configuration key I can use to move temporary flat file location
under a different path?
was (Author: freds):
Understood - one question still (might help me solve the issue myself):
* Kylin creates a HBase table for metadata: /apps/kylin_metadata
* Kylin tries to create a sequence file within that HBase table:
maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e
I don't get how this was supposed to work in the first place.
Secondly, is there a configuration key I can use to move temporary files under
a different root?
> Invalid temporary table path for kylin_metadata
> ---
>
> Key: KYLIN-3989
> URL: https://issues.apache.org/jira/browse/KYLIN-3989
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.2
> Environment: MapR 6.1
> Hive 2.3
>Reporter: Frederic Souchu
>Priority: Major
>
> How to reproduce:
> * create a mode
> * define a cube
> * build the cube
> The cube building will fail with the following Hive logs:
> {code:java}
> USE default;
> No rows affected (0.067 seconds)
> 0: jdbc:hive2://x.com>
> 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
> e
> . . . . . . . . . . . . . . . . . . . . . . .> (
> . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
> . . . . . . . . . . . . . . . . . . . . . . .> )
> . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
> . . . . . . . . . . . . . . . . . . . . . . .> LOCATION
> 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
> te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
> java.io.IOException Error: Not a directory(20), file:
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
> mapr, ID: 5000)
> {code}
> using metastore.sh to list content gives:
> {code}
> 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
> kylin-defaults.properties from
> file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
> 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
> property was not set, will seek KYLIN_HOME env variable
> 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a
> new KylinConfig from getInstanceFromEnv : 453523494
> 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
> metadata url /apps/kylin_metadata@hbase for resource store
> 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
> working dir FS for HBase: maprfs:///
> 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection
> is null or closed, creating a new one
> 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
> ConnectionFactory receives mapr.hbase.default.db(maprdb), set
> clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
> 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
> '/apps/kylin_metadata' already exists
> null
> 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
> Closing HBase connections...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[
https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833761#comment-16833761
]
Frederic Souchu commented on KYLIN-3989:
Understood - one question still (might help me solve the issue myself):
* Kylin creates a HBase table for metadata: /apps/kylin_metadata
* Kylin tries to create a sequence file within that HBase table:
maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e
I don't get how this was supposed to work in the first place.
Secondly, is there a configuration key I can use to move temporary files under
a different root?
> Invalid temporary table path for kylin_metadata
> ---
>
> Key: KYLIN-3989
> URL: https://issues.apache.org/jira/browse/KYLIN-3989
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.2
> Environment: MapR 6.1
> Hive 2.3
>Reporter: Frederic Souchu
>Priority: Major
>
> How to reproduce:
> * create a mode
> * define a cube
> * build the cube
> The cube building will fail with the following Hive logs:
> {code:java}
> USE default;
> No rows affected (0.067 seconds)
> 0: jdbc:hive2://x.com>
> 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
> e
> . . . . . . . . . . . . . . . . . . . . . . .> (
> . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
> . . . . . . . . . . . . . . . . . . . . . . .> )
> . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
> . . . . . . . . . . . . . . . . . . . . . . .> LOCATION
> 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
> te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
> java.io.IOException Error: Not a directory(20), file:
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
> mapr, ID: 5000)
> {code}
> using metastore.sh to list content gives:
> {code}
> 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
> kylin-defaults.properties from
> file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
> 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
> property was not set, will seek KYLIN_HOME env variable
> 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a
> new KylinConfig from getInstanceFromEnv : 453523494
> 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
> metadata url /apps/kylin_metadata@hbase for resource store
> 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
> working dir FS for HBase: maprfs:///
> 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection
> is null or closed, creating a new one
> 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
> ConnectionFactory receives mapr.hbase.default.db(maprdb), set
> clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
> 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
> '/apps/kylin_metadata' already exists
> null
> 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
> Closing HBase connections...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3989) Invalid temporary table path for kylin_metadata
Frederic Souchu created KYLIN-3989:
--
Summary: Invalid temporary table path for kylin_metadata
Key: KYLIN-3989
URL: https://issues.apache.org/jira/browse/KYLIN-3989
Project: Kylin
Issue Type: Bug
Affects Versions: v2.5.2
Environment: MapR 6.1
Hive 2.3
Reporter: Frederic Souchu
How to reproduce:
* create a mode
* define a cube
* build the cube
The cube building will fail with the following Hive logs:
{code:java}
USE default;
No rows affected (0.067 seconds)
0: jdbc:hive2://x.com>
0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
No rows affected (0.023 seconds)
0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
e
. . . . . . . . . . . . . . . . . . . . . . .> (
. . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
. . . . . . . . . . . . . . . . . . . . . . .> )
. . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
. . . . . . . . . . . . . . . . . . . . . . .> LOCATION
'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
java.io.IOException Error: Not a directory(20), file:
kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
mapr, ID: 5000)
{code}
using metastore.sh to list content gives:
{code}
2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
kylin-defaults.properties from
file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
property was not set, will seek KYLIN_HOME env variable
2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a new
KylinConfig from getInstanceFromEnv : 453523494
2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
metadata url /apps/kylin_metadata@hbase for resource store
2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
working dir FS for HBase: maprfs:///
2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection is
null or closed, creating a new one
2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
ConnectionFactory receives mapr.hbase.default.db(maprdb), set
clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
'/apps/kylin_metadata' already exists
null
2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
Closing HBase connections...
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
