[
https://issues.apache.org/jira/browse/HIVE-15128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Linte updated HIVE-15128:
-----------------------------------
Description:
The property "hive.exec.stagingdir" is producing an AccessControlException
error, only when working with a temporary table and with an INSERT INTO TABLE
statement.
The configuration used in the hive-site.xml is:
{noformat}
<property>
<name>hive.exec.stagingdir</name>
<value>/user/${user.name}/.hiveStaging/.staging</value>
<description>In Hive >= 0.14, set to
${hive.exec.scratchdir}/${user.name}/.staging</description>
</property>
{noformat}
Trying to insert a value into a temporary table will produce the following
output:
{noformat}
hive (shfs3453)> INSERT INTO TABLE test_table_tmp VALUES (11101, 'John',
'Oakland');
Query ID = shfs3453_20161103180522_5fceae9b-6fb5-4cdd-a0d9-37cc93b27a9f
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478096256452_0018)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 1 1 0 0
0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 16.09 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
Failed with exception org.apache.hadoop.security.AccessControlException: User
does not belong to hadoop
at
org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask.
org.apache.hadoop.security.AccessControlException: User does not belong to
hadoop
at
org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
{noformat}
But if you do an INSERT OVERWRITE TABLE, there will be a warning only. No
AccessControlException error here and erverything is properly inserted.
{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John',
'Oakland');
Query ID = shfs3453_20161103180603_d8d5f689-cc55-44d0-b160-bba0df8d186a
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478096256452_0018)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0
0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 6.74 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
chgrp: changing ownership of
'hdfs://sandbox/tmp/hive/shfs3453/edab56b7-b1e1-4f87-bdfd-eeece41f6fc9/_tmp_space.db/874bfdbd-7bab-4e0c-85ac-ec4552ea560b':
User does not belong to hadoop
OK
Time taken: 8.224 seconds
{noformat}
If the previous query is done one more time, the warning will not be displayed.
{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John',
'Oakland');
Query ID = shfs3453_20161103180831_40350dc3-1f66-45e2-8304-90b00933aead
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478096256452_0018)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 1 1 0 0
0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 13.00 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
OK
Time taken: 14.161 seconds
{noformat}
Note: if the warning is displayed one time, an INSERT INTO TABLE will work.
> "hive.exec.stagingdir" produces an "AccessControlException" when working with
> temp tables
> -----------------------------------------------------------------------------------------
>
> Key: HIVE-15128
> URL: https://issues.apache.org/jira/browse/HIVE-15128
> Project: Hive
> Issue Type: Bug
> Components: Beeline, CLI, Hive
> Affects Versions: 2.1.0
> Reporter: Alexandre Linte
>
> The property "hive.exec.stagingdir" is producing an AccessControlException
> error, only when working with a temporary table and with an INSERT INTO TABLE
> statement.
> The configuration used in the hive-site.xml is:
> {noformat}
> <property>
> <name>hive.exec.stagingdir</name>
> <value>/user/${user.name}/.hiveStaging/.staging</value>
> <description>In Hive >= 0.14, set to
> ${hive.exec.scratchdir}/${user.name}/.staging</description>
> </property>
> {noformat}
> Trying to insert a value into a temporary table will produce the following
> output:
> {noformat}
> hive (shfs3453)> INSERT INTO TABLE test_table_tmp VALUES (11101, 'John',
> 'Oakland');
> Query ID = shfs3453_20161103180522_5fceae9b-6fb5-4cdd-a0d9-37cc93b27a9f
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id
> application_1478096256452_0018)
> ----------------------------------------------------------------------------------------------
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> ----------------------------------------------------------------------------------------------
> Map 1 .......... container SUCCEEDED 1 1 0 0
> 0 0
> ----------------------------------------------------------------------------------------------
> VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 16.09 s
> ----------------------------------------------------------------------------------------------
> Loading data to table shfs3453.test_table_tmp
> Failed with exception org.apache.hadoop.security.AccessControlException: User
> does not belong to hadoop
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask.
> org.apache.hadoop.security.AccessControlException: User does not belong to
> hadoop
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
> {noformat}
> But if you do an INSERT OVERWRITE TABLE, there will be a warning only. No
> AccessControlException error here and erverything is properly inserted.
> {noformat}
> hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John',
> 'Oakland');
> Query ID = shfs3453_20161103180603_d8d5f689-cc55-44d0-b160-bba0df8d186a
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id
> application_1478096256452_0018)
> ----------------------------------------------------------------------------------------------
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> ----------------------------------------------------------------------------------------------
> Map 1 .......... container SUCCEEDED 2 2 0 0
> 0 0
> ----------------------------------------------------------------------------------------------
> VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 6.74 s
> ----------------------------------------------------------------------------------------------
> Loading data to table shfs3453.test_table_tmp
> chgrp: changing ownership of
> 'hdfs://sandbox/tmp/hive/shfs3453/edab56b7-b1e1-4f87-bdfd-eeece41f6fc9/_tmp_space.db/874bfdbd-7bab-4e0c-85ac-ec4552ea560b':
> User does not belong to hadoop
> OK
> Time taken: 8.224 seconds
> {noformat}
> If the previous query is done one more time, the warning will not be
> displayed.
> {noformat}
> hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John',
> 'Oakland');
> Query ID = shfs3453_20161103180831_40350dc3-1f66-45e2-8304-90b00933aead
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id
> application_1478096256452_0018)
> ----------------------------------------------------------------------------------------------
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> ----------------------------------------------------------------------------------------------
> Map 1 .......... container SUCCEEDED 1 1 0 0
> 0 0
> ----------------------------------------------------------------------------------------------
> VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 13.00 s
> ----------------------------------------------------------------------------------------------
> Loading data to table shfs3453.test_table_tmp
> OK
> Time taken: 14.161 seconds
> {noformat}
> Note: if the warning is displayed one time, an INSERT INTO TABLE will work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
