niklaus xiao created HIVE-12154: ----------------------------------- Summary: Load data inpath 'PATTERN' into table should only check files match the PATTERN Key: HIVE-12154 URL: https://issues.apache.org/jira/browse/HIVE-12154 Project: Hive Issue Type: Bug Components: SQLStandardAuthorization Affects Versions: 1.2.1, 1.1.0, 1.2.0, 1.0.0, 0.13.1 Reporter: niklaus xiao Priority: Minor
We are using flume to sink data to HDFS directory '/tmp/test/', temporal files that flume actively writes into has a suffix .tmp, after writes finish, the file will be renamed to SAMPLE.data. Hive periodic task execute script like {quote} load data inpath '/tmp/test/*.data' into table t1; {quote} This exception happens sometimes {quote} 2015-10-12 19:38:00,133 | ERROR | HiveServer2-Handler-Pool: Thread-57 | FAILED: HiveAuthzPluginException Error getting permissions for hdfs://hacluster/tmp/test/*.data: null org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: Error getting permissions for hdfs://hacluster/tmp/test/*.data: null ... Caused by: java.io.FileNotFoundException: Path not found at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAccess(FSNamesystem.java:8175) {quote} I digged into the code, and find that SQLStdHiveAuthorizationValidator checks all the files in /tmp/test/ directory, but when checks the permission of .tmp file, the file is renamed to .data, hdfs cannot find this file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)