[
https://issues.apache.org/jira/browse/KYLIN-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Iñigo Martinez updated KYLIN-3802:
----------------------------------
Description:
When building several cubes in parallel that use a hive view as source,
sometimes they fail at step two (Materialize Hive View in Lookup Tables).
This is due to since we are using a view as source instead of a table, Kylin
intermediate table is created without UUID and so second build drops previous
built intermediate table.
For example:
{code:java}
0: jdbc:hive2://bi-horton-hive.internalserver> DROP TABLE IF EXISTS
kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW; No rows affected (0.586 seconds)
0: jdbc:hive2://bi-horton-hive.internalserver> CREATE EXTERNAL TABLE IF NOT
EXISTS kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW LIKE
DW.DI_OPERADORES_SMS_VIEW LOCATION 'hdfs://XXXXXX/kylin/kylin_me
tadata/kylin-5df95c88-a123-44e0-9b1a-c35ecf1599fb/kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW';
No rows affected (0.308 seconds)
0: jdbc:hive2://bi-horton-hive.internalserver> ALTER TABLE
kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW SET
TBLPROPERTIES('auto.purge'='true'); Error: Error while processing statement:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. Table not found
kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW (state=08S01,code=1)
{code}
If no views are used, Kylin appends an uuid to avoid conflicts.
{code:java}
DROP TABLE IF EXISTS
kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4;
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4{code}
was:
When building several cubes in parallel that use a hive view as source,
sometimes they fail at step two (Materialize Hive View in Lookup Tables).
This is due to since we are using a view as source instead of a table, Kylin
intermediate table is created without UUID and so second build drops previous
built intermediate table.
For example:
{code:java}
0: jdbc:hive2://bi-horton-hive.internalserver> DROP TABLE IF EXISTS
kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW; No rows affected (0.586 seconds)
0: jdbc:hive2://bi-horton-hive.internalserver> CREATE EXTERNAL TABLE IF NOT
EXISTS kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW LIKE
DW.DI_OPERADORES_SMS_VIEW LOCATION 'hdfs://XXXXXX/kylin/kylin_me
tadata/kylin-5df95c88-a123-44e0-9b1a-c35ecf1599fb/kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW';
No rows affected (0.308 seconds)
0: jdbc:hive2://bi-horton-hive.internalserver> ALTER TABLE
kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW SET
TBLPROPERTIES('auto.purge'='true'); Error: Error while processing statement:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. Table not found
kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW (state=08S01,code=1)
{code}
If no view are used, Kylin appends an uuid to avoid conflicts.
{code:java}
DROP TABLE IF EXISTS
kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4;
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4{code}
> Kylin build process fails at step "Materialize Hive View in Lookup Tables"
> --------------------------------------------------------------------------
>
> Key: KYLIN-3802
> URL: https://issues.apache.org/jira/browse/KYLIN-3802
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v2.4.1
> Reporter: Iñigo Martinez
> Priority: Major
>
> When building several cubes in parallel that use a hive view as source,
> sometimes they fail at step two (Materialize Hive View in Lookup Tables).
> This is due to since we are using a view as source instead of a table, Kylin
> intermediate table is created without UUID and so second build drops previous
> built intermediate table.
> For example:
> {code:java}
> 0: jdbc:hive2://bi-horton-hive.internalserver> DROP TABLE IF EXISTS
> kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW; No rows affected (0.586 seconds)
> 0: jdbc:hive2://bi-horton-hive.internalserver> CREATE EXTERNAL TABLE IF NOT
> EXISTS kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW LIKE
> DW.DI_OPERADORES_SMS_VIEW LOCATION 'hdfs://XXXXXX/kylin/kylin_me
> tadata/kylin-5df95c88-a123-44e0-9b1a-c35ecf1599fb/kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW';
> No rows affected (0.308 seconds)
> 0: jdbc:hive2://bi-horton-hive.internalserver> ALTER TABLE
> kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW SET
> TBLPROPERTIES('auto.purge'='true'); Error: Error while processing statement:
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. Table not found
> kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW (state=08S01,code=1)
> {code}
>
> If no views are used, Kylin appends an uuid to avoid conflicts.
> {code:java}
> DROP TABLE IF EXISTS
> kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4;
> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
