[
https://issues.apache.org/jira/browse/HADOOP-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17142033#comment-17142033
]
Syed Shameerur Rahman edited comment on HADOOP-16963 at 6/22/20, 1:23 PM:
--------------------------------------------------------------------------
[~ste...@apache.org] Yes, hive uses *ProxyFileSystem* for running qtests. As
you said we need to override FilterFS.mkdir(path) in hive to avoid qtests
failing. Verified locally by overriding mkdir(path) in ProxyFileSystem. I will
raise a corresponding jira in hive.
Sample failures:
{code:java}
Caused by: java.lang.IllegalArgumentException: Wrong FS:
pfile:/media/ebs1/workspace/hive-3.1-qtest/group/5/label/HiveQTest/hive-1.2.0/itests/qtest/target/warehouse/dest1,
expected: file:///
{code}
cc: [~kgyrtkirk]
was (Author: srahman):
[~ste...@apache.org] Yes, hive uses *ProxyFileSystem* for running qtests. As
you said we need to override FilterFS.mkdir(path) in hive to avoid qtests
failing. Verified locally by overriding mkdir(path) in ProxyFileSystem. I will
raise a corresponding jira in hive
cc: [~kgyrtkirk]
> HADOOP-16582 changed mkdirs() behavior
> --------------------------------------
>
> Key: HADOOP-16963
> URL: https://issues.apache.org/jira/browse/HADOOP-16963
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.10.0, 3.3.0, 2.8.6, 2.9.3, 3.1.3, 3.2.2
> Reporter: Wei-Chiu Chuang
> Priority: Critical
>
> HADOOP-16582 changed behavior of {{mkdirs()}}
> Some Hive tests depend on the old behavior and they fail miserably.
> {quote}
> earlier:
> all plain mkdirs(somePath) were fast-tracked to FileSystem.mkdirs which have
> rerouted them to mkdirs(somePath, somePerm) method with some defaults (which
> were static)
> an implementation of FileSystem have only needed implement "mkdirs(somePath,
> somePerm)" - because the other was not neccessarily called if it was always
> in a FilterFileSystem or something like that
> now:
> especially FilterFileSystem forwards the call of mkdirs(p) to the actual fs
> implementation...which may skip overriden mkdirs(somPath,somePerm) methods
> ...and could cause issues for existing FileSystem implementations
> {quote}
> File this jira to address this problem.
> [~kgyrtkirk] [~ste...@apache.org] [~kihwal]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
