[
https://issues.apache.org/jira/browse/IMPALA-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar resolved IMPALA-2211.
-----------------------------------------
Resolution: Won't Fix
Closing this as won't fix since the error is coming from HMS which tries to
create a directory. Also, my understanding is that in case of external table
HMS will only try to create the directory if it doesn't exist already. So this
may be some setup issue as well.
> Impala reports missing WRITE access on dir in creating an external table
> ------------------------------------------------------------------------
>
> Key: IMPALA-2211
> URL: https://issues.apache.org/jira/browse/IMPALA-2211
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 2.2
> Reporter: Vamsee K. Yarlagadda
> Priority: Minor
> Labels: ramp-up
>
> It looks like Impala complains about missing write permissions on a HDFS
> directory when trying to create an external table through Impyla.
> {code}
> File
> "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py",
> line 152, in execute
> configuration=configuration)
> File
> "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py",
> line 166, in execute_async
> self._execute_async(op)
> File
> "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py",
> line 172, in _execute_async
> operation_fn()
> File
> "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/dbapi/hiveserver2.py",
> line 164, in op
> configuration)
> File
> "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/_rpc/hiveserver2.py",
> line 134, in wrapper
> return func(*args, **kwargs)
> File
> "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/_rpc/hiveserver2.py",
> line 240, in execute_statement
> err_if_rpc_not_ok(resp)
> File
> "/Users/vamsee/Cloudera/OtherProjects/cmf/systest/target/env/src/impyla/impala/_rpc/hiveserver2.py",
> line 81, in err_if_rpc_not_ok
> raise HiveServer2Error(resp.status.errorMessage)
> HiveServer2Error:
> ImpalaRuntimeException: Error making 'createTable' RPC to Hive Metastore:
> CAUSED BY: MetaException: Got exception:
> org.apache.hadoop.security.AccessControlException Permission denied:
> user=impala, access=WRITE,
> inode="/user/systest/tpcds_10_text":systest:supergroup:drwxr-xr-x
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:257)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:238)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:216)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:145)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:138)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6599)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6581)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6533)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4307)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4280)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:853)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.mkdirs(AuthorizationProviderProxyClientProtocol.java:321)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:601)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038)
> {code}
> My main worry is if i am creating an external table (pointing to an HDFS
> location), why does impala need WRITE access during table creations step.
> Ideally it should just have READ access right?
> DDL statement being invoked
> {code}
> create external table call_center(
> cc_call_center_sk int
> , cc_call_center_id string
> , cc_rec_start_date string
> , cc_rec_end_date string
> , cc_closed_date_sk int
> , cc_open_date_sk int
> , cc_name string
> , cc_class string
> , cc_employees int
> , cc_sq_ft int
> , cc_hours string
> , cc_manager string
> , cc_mkt_id int
> , cc_mkt_class string
> , cc_mkt_desc string
> , cc_market_manager string
> , cc_division int
> , cc_division_name string
> , cc_company int
> , cc_company_name string
> , cc_street_number string
> , cc_street_name string
> , cc_street_type string
> , cc_suite_number string
> , cc_city string
> , cc_county string
> , cc_state string
> , cc_zip string
> , cc_country string
> , cc_gmt_offset decimal(5,2)
> , cc_tax_percentage decimal(5,2)
> )
> row format delimited fields terminated by '|'
> location '/user/systest/tpcds_10_text/call_center';
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
