[
https://issues.apache.org/jira/browse/HIVE-16357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barna Zsombor Klara updated HIVE-16357:
---------------------------------------
Resolution: Cannot Reproduce
Status: Resolved (was: Patch Available)
[~spena] is right in that the transactional listener should not be called after
a failed table creation.
I still have two issues with the current code:
- transactional and non transactional listeners are not typed. There is nothing
preventing me from configuring the DbNotificationListener as a non
transactional one. This is exactly the error I did when testing my patch.
- error reporting is still brittle as the non transactional listener is
throwing checked exception which are not handled in the finally block so may
end up overwriting the actual exception.
However since I cannot reproduce the issue (I honestly don't remember how I
encountered it) and since it seems so rare that no one else ran into it, I will
close the issue as cannot reproduce. I don't think the issue is severe enough
to warrant any more attention.
> Failed folder creation when creating a new table is reported incorrectly
> ------------------------------------------------------------------------
>
> Key: HIVE-16357
> URL: https://issues.apache.org/jira/browse/HIVE-16357
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 2.3.0, 3.0.0
> Reporter: Barna Zsombor Klara
> Assignee: Barna Zsombor Klara
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HIVE-16357.01.patch, HIVE-16357.02.patch,
> HIVE-16357.03.patch, HIVE-16357.04.patch
>
>
> If the directory for a Hive table could not be created, them the HMS will
> throw a metaexception:
> {code}
> if (tblPath != null) {
> if (!wh.isDir(tblPath)) {
> if (!wh.mkdirs(tblPath, true)) {
> throw new MetaException(tblPath
> + " is not a directory or unable to create one");
> }
> madeDir = true;
> }
> }
> {code}
> However in the finally block we always try to call the
> DbNotificationListener, which in turn will also throw an exception because
> the directory is missing, overwriting the initial exception with a
> FileNotFoundException.
> Actual stacktrace seen by the caller:
> {code}
> 2017-04-03T05:58:00,128 ERROR [pool-7-thread-2] metastore.RetryingHMSHandler:
> MetaException(message:java.lang.RuntimeException:
> java.io.FileNotFoundException: File file:/.../0 does not exist)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:6074)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1496)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy28.create_table_with_environment_context(Unknown
> Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:11125)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:11109)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: File
> file:/.../0 does not exist
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener$FileIterator.<init>(DbNotificationListener.java:203)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener.onCreateTable(DbNotificationListener.java:137)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1463)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1482)
> ... 20 more
> Caused by: java.io.FileNotFoundException: File file:/.../0 does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:429)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1515)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1555)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:574)
> at
> org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:243)
> at
> org.apache.hadoop.fs.ProxyFileSystem.listStatus(ProxyFileSystem.java:195)
> at
> org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:243)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1515)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1555)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener$FileIterator.<init>(DbNotificationListener.java:200)
> ... 23 more
> {code}
> The original exception should be thrown back to the caller while any
> exception from the finally block should be caught/handled/logged out locally.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
