[jira] [Commented] (HIVE-13280) Error when more than 1 mapper for HBase storage handler
[
https://issues.apache.org/jira/browse/HIVE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16188388#comment-16188388
]
sandeep mehta commented on HIVE-13280:
--
I have tried by setting up {code} hbase.mapred.output.outputtable {code} but
it's throwing the same error but when I only try to insert one record into the
table then it works properly and no error.
Even after the load job fails some of the data gets to the table anyways.
> Error when more than 1 mapper for HBase storage handler
> ---
>
> Key: HIVE-13280
> URL: https://issues.apache.org/jira/browse/HIVE-13280
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Damien Carol
>Assignee: Damien Carol
>
> With a simple query (select from orc table and insert into HBase external
> table):
> {code:sql}
> insert into table register.register select * from aa_temp
> {code}
> The aa_temp table have 45 orc files. It generate 45 mappers.
> Some mappers fail with this error:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at
> org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
> at
> org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
> ... 25 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:35, Vertex vertex_1457964631631_0015_3_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0 (state=08S01,code=2)
> {noformat}
> If I do an ALTER CONCATENATE for aa_temp. And redo the query. Everything is
> fine because there are only one mapper.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-13280) Error when more than 1 mapper for HBase storage handler
[
https://issues.apache.org/jira/browse/HIVE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568866#comment-15568866
]
Vsevolod Ostapenko commented on HIVE-13280:
---
Hive-Hbase integration documentation
(https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration) claims that
hbase.mapred.output.outputtable property is optional, and provides no good
explanation under what circumstances one would want or need to define it. In
all the provided samples values of hbase.mapred.output.outputtable and
hbase.table.name are the same, so samples are hot helpful and not
self-explanatory.
If TEZ does require hbase.mapred.output.outputtable property to be explicitly
set, documentation needs to be updated to indicate that fact.
Also, it would be helpful to provide some background why this property exists
in the first place.
> Error when more than 1 mapper for HBase storage handler
> ---
>
> Key: HIVE-13280
> URL: https://issues.apache.org/jira/browse/HIVE-13280
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Damien Carol
>Assignee: Damien Carol
>
> With a simple query (select from orc table and insert into HBase external
> table):
> {code:sql}
> insert into table register.register select * from aa_temp
> {code}
> The aa_temp table have 45 orc files. It generate 45 mappers.
> Some mappers fail with this error:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at
> org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
> at
> org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
> ... 25 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:35, Vertex vertex_1457964631631_0015_3_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0 (state=08S01,code=2)
> {noformat}
> If I do an ALTER CONCATENATE for aa_temp. And redo the query. Everything is
> fine because there are only one mapper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13280) Error when more than 1 mapper for HBase storage handler
[
https://issues.apache.org/jira/browse/HIVE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568493#comment-15568493
]
Damien Carol commented on HIVE-13280:
-
Yes this fix the pb.
> Error when more than 1 mapper for HBase storage handler
> ---
>
> Key: HIVE-13280
> URL: https://issues.apache.org/jira/browse/HIVE-13280
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Damien Carol
>
> With a simple query (select from orc table and insert into HBase external
> table):
> {code:sql}
> insert into table register.register select * from aa_temp
> {code}
> The aa_temp table have 45 orc files. It generate 45 mappers.
> Some mappers fail with this error:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at
> org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
> at
> org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
> ... 25 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:35, Vertex vertex_1457964631631_0015_3_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0 (state=08S01,code=2)
> {noformat}
> If I do an ALTER CONCATENATE for aa_temp. And redo the query. Everything is
> fine because there are only one mapper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13280) Error when more than 1 mapper for HBase storage handler
[
https://issues.apache.org/jira/browse/HIVE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568492#comment-15568492
]
Damien Carol commented on HIVE-13280:
-
Yes this fix the pb.
> Error when more than 1 mapper for HBase storage handler
> ---
>
> Key: HIVE-13280
> URL: https://issues.apache.org/jira/browse/HIVE-13280
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Damien Carol
>
> With a simple query (select from orc table and insert into HBase external
> table):
> {code:sql}
> insert into table register.register select * from aa_temp
> {code}
> The aa_temp table have 45 orc files. It generate 45 mappers.
> Some mappers fail with this error:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at
> org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
> at
> org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
> ... 25 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:35, Vertex vertex_1457964631631_0015_3_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0 (state=08S01,code=2)
> {noformat}
> If I do an ALTER CONCATENATE for aa_temp. And redo the query. Everything is
> fine because there are only one mapper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13280) Error when more than 1 mapper for HBase storage handler
[
https://issues.apache.org/jira/browse/HIVE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566536#comment-15566536
]
Vaibhav Gumashta commented on HIVE-13280:
-
[~damien.carol] Per the docs here:
https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration, the
hbase.mapred.output.outputtable table property will need to be set for an
insert query. Can you check the output of show create table and describe
formatted to see if that property is set?
> Error when more than 1 mapper for HBase storage handler
> ---
>
> Key: HIVE-13280
> URL: https://issues.apache.org/jira/browse/HIVE-13280
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Damien Carol
>
> With a simple query (select from orc table and insert into HBase external
> table):
> {code:sql}
> insert into table register.register select * from aa_temp
> {code}
> The aa_temp table have 45 orc files. It generate 45 mappers.
> Some mappers fail with this error:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at
> org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
> at
> org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
> ... 25 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:35, Vertex vertex_1457964631631_0015_3_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0 (state=08S01,code=2)
> {noformat}
> If I do an ALTER CONCATENATE for aa_temp. And redo the query. Everything is
> fine because there are only one mapper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13280) Error when more than 1 mapper for HBase storage handler
[
https://issues.apache.org/jira/browse/HIVE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312867#comment-15312867
]
Vsevolod Ostapenko commented on HIVE-13280:
---
We are observing a similar behavior in HDP 2.4.0 when using TEZ execution
engine.
Insert into HBased-backed tables that do not require multiple mappers succeed.
Ones that require multiple mappers fail with exactly the same errors.
If hive execution engine is set to MR, statements succeed and data are inserted
as expected.
> Error when more than 1 mapper for HBase storage handler
> ---
>
> Key: HIVE-13280
> URL: https://issues.apache.org/jira/browse/HIVE-13280
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Damien Carol
>
> With a simple query (select from orc table and insert into HBase external
> table):
> {code:sql}
> insert into table register.register select * from aa_temp
> {code}
> The aa_temp table have 45 orc files. It generate 45 mappers.
> Some mappers fail with this error:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at
> org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
> at
> org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
> ... 25 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:35, Vertex vertex_1457964631631_0015_3_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0 (state=08S01,code=2)
> {noformat}
> If I do an ALTER CONCATENATE for aa_temp. And redo the query. Everything is
> fine because there are only one mapper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13280) Error when more than 1 mapper for HBase storage handler
[
https://issues.apache.org/jira/browse/HIVE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294851#comment-15294851
]
Rafik Naccache - Turbopape commented on HIVE-13280:
---
I have the same problem,
I am on HDP 2.4
I created a big external hive table with a location
I've set some hive views on that table
And I launch a hive to insert the contents of these views inside a hbase
storage backed table.
I get the exact same error you're getting.
> Error when more than 1 mapper for HBase storage handler
> ---
>
> Key: HIVE-13280
> URL: https://issues.apache.org/jira/browse/HIVE-13280
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Damien Carol
>
> With a simple query (select from orc table and insert into HBase external
> table):
> {code:sql}
> insert into table register.register select * from aa_temp
> {code}
> The aa_temp table have 45 orc files. It generate 45 mappers.
> Some mappers fail with this error:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at
> org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
> at
> org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
> ... 25 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:35, Vertex vertex_1457964631631_0015_3_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0 (state=08S01,code=2)
> {noformat}
> If I do an ALTER CONCATENATE for aa_temp. And redo the query. Everything is
> fine because there are only one mapper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
