[jira] [Updated] (HIVE-15128) "hive.exec.stagingdir" produces an "AccessControlException" when working with temp tables
[
https://issues.apache.org/jira/browse/HIVE-15128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Linte updated HIVE-15128:
---
Description:
The property "hive.exec.stagingdir" is producing an AccessControlException
error, only when working with a temporary table and with an INSERT INTO TABLE
statement.
The configuration used in the hive-site.xml is:
{noformat}
hive.exec.stagingdir
/user/${user.name}/.hiveStaging/.staging
In Hive >= 0.14, set to
${hive.exec.scratchdir}/${user.name}/.staging
{noformat}
Trying to insert a value into a temporary table will produce the following
output:
{noformat}
hive (shfs3453)> INSERT INTO TABLE test_table_tmp VALUES (11101, 'John',
'Oakland');
Query ID = shfs3453_20161103180522_5fceae9b-6fb5-4cdd-a0d9-37cc93b27a9f
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478096256452_0018)
--
VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
--
Map 1 .. container SUCCEEDED 1 100
0 0
--
VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 16.09 s
--
Loading data to table shfs3453.test_table_tmp
Failed with exception org.apache.hadoop.security.AccessControlException: User
does not belong to hadoop
at
org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask.
org.apache.hadoop.security.AccessControlException: User does not belong to
hadoop
at
org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
{noformat}
But if you do an INSERT OVERWRITE TABLE, there will be a warning only. No
AccessControlException error here and erverything is properly inserted.
{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John',
'Oakland');
Query ID = shfs3453_20161103180603_d8d5f689-cc55-44d0-b160-bba0df8d186a
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478096256452_0018)
--
VERTICES
[jira] [Updated] (HIVE-15128) "hive.exec.stagingdir" produces an "AccessControlException" when working with temp tables
[
https://issues.apache.org/jira/browse/HIVE-15128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Linte updated HIVE-15128:
---
Environment: Hive 2.1.0, Hadoop 2.7.2
> "hive.exec.stagingdir" produces an "AccessControlException" when working with
> temp tables
> -
>
> Key: HIVE-15128
> URL: https://issues.apache.org/jira/browse/HIVE-15128
> Project: Hive
> Issue Type: Bug
> Components: Beeline, CLI, Hive
>Affects Versions: 2.1.0
> Environment: Hive 2.1.0, Hadoop 2.7.2
>Reporter: Alexandre Linte
>
> The property "hive.exec.stagingdir" is producing an AccessControlException
> error, only when working with a temporary table and with an INSERT INTO TABLE
> statement.
> The configuration used in the hive-site.xml is:
> {noformat}
>
> hive.exec.stagingdir
> /user/${user.name}/.hiveStaging/.staging
> In Hive >= 0.14, set to
> ${hive.exec.scratchdir}/${user.name}/.staging
>
> {noformat}
> Trying to insert a value into a temporary table will produce the following
> output:
> {noformat}
> hive (shfs3453)> INSERT INTO TABLE test_table_tmp VALUES (11101, 'John',
> 'Oakland');
> Query ID = shfs3453_20161103180522_5fceae9b-6fb5-4cdd-a0d9-37cc93b27a9f
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id
> application_1478096256452_0018)
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 1 100
> 0 0
> --
> VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 16.09 s
> --
> Loading data to table shfs3453.test_table_tmp
> Failed with exception org.apache.hadoop.security.AccessControlException: User
> does not belong to hadoop
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask.
> org.apache.hadoop.security.AccessControlException: User does not belong to
> hadoop
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>
[jira] [Updated] (HIVE-15128) "hive.exec.stagingdir" produces an "AccessControlException" when working with temp tables
[
https://issues.apache.org/jira/browse/HIVE-15128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Linte updated HIVE-15128:
---
Environment: (was: The property "hive.exec.stagingdir" is producing an
AccessControlException error, only when working with a temporary table and with
an INSERT INTO TABLE statement.
The configuration used in the hive-site.xml is:
{noformat}
hive.exec.stagingdir
/user/${user.name}/.hiveStaging/.staging
In Hive >= 0.14, set to
${hive.exec.scratchdir}/${user.name}/.staging
{noformat}
Trying to insert a value into a temporary table will produce the following
output:
{noformat}
hive (shfs3453)> INSERT INTO TABLE test_table_tmp VALUES (11101, 'John',
'Oakland');
Query ID = shfs3453_20161103180522_5fceae9b-6fb5-4cdd-a0d9-37cc93b27a9f
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478096256452_0018)
--
VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
--
Map 1 .. container SUCCEEDED 1 100
0 0
--
VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 16.09 s
--
Loading data to table shfs3453.test_table_tmp
Failed with exception org.apache.hadoop.security.AccessControlException: User
does not belong to hadoop
at
org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask.
org.apache.hadoop.security.AccessControlException: User does not belong to
hadoop
at
org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
{noformat}
But if you do an INSERT OVERWRITE TABLE, there will be a warning only. No
AccessControlException error here and erverything is properly inserted.
{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John',
'Oakland');
Query ID = shfs3453_20161103180603_d8d5f689-cc55-44d0-b160-bba0df8d186a
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478096256452_0018)
--
