Jason Dere created HIVE-20515:
---------------------------------
Summary: Empty query results when using results cache and query
temp dir, results cache dir in different filesystems
Key: HIVE-20515
URL: https://issues.apache.org/jira/browse/HIVE-20515
Project: Hive
Issue Type: Bug
Reporter: Jason Dere
Assignee: Jason Dere
If the scratchdir for temporary query results and the results cache dir are in
different filesystems, moving the query from the temp directory to results
cache will fail.
Looking at the moveResultsToCacheDirectory() logic in QueryResultsCache.java, I
see the following issues:
- FileSystem.rename() is used, which only works if the files are on the same
filesystem. Need to use something like Hive.mvFile or something similar which
can work between different filesystems.
- The return code from rename() was not checked which might possibly have
caught the error here. This may not be applicable if a different method from
FS.rename() is used in the proper fix.
With some filesystems (noticed this with WASB), if FileSystem.rename() returns
false on failure rather than throwing an exception, then this results in empty
results showing up for the query because the return code was not checked
properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
