[
https://issues.apache.org/jira/browse/HIVE-26887?focusedWorklogId=837714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-837714
]
ASF GitHub Bot logged work on HIVE-26887:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Jan/23 01:34
Start Date: 08/Jan/23 01:34
Worklog Time Spent: 10m
Work Description: skysiders commented on PR #3894:
URL: https://github.com/apache/hive/pull/3894#issuecomment-1374681627
Hi @cnauroth , thanks for your review. I'm glad to see your reply. In the
MAPREDUCE-7375 you mentioned, I saw your approval of this fix, as well as the
extremely special situation that you mentioned earlier that the umask would not
be set to 777, and I very much agree with these. The repairs I submitted in
other projects are also based on the way of thinking I mentioned above, that
is, after we create a file, should we check and judge the permissions of this
file and grant permissions to it. I think this matter is necessary, because as
an upper-level application, it is difficult to determine the umask setting of
the underlying file system (for example, the umask under Linux is set by the
system administrator root, and the umask of the distributed filesystem is set
by the underlying hdfs The user decides when starting dfs, while hive or other
systems are run by program administrators, and these two users may not be the
same user) This is my most fundamental point of view, and it is also the reason
why I insist on this repair.
I would like to discuss this issue with you further. Do you think that this
repair method should be applied to those permissions of 770 instead of
permissions like 700 (because 770 may be defaulted by the file system umask=022
affect)?
Finally, I am very happy to have such an in-depth discussion with you on the
issue of file permissions. Thank you for your views on my point of view.
Hi @abstractdog @ayushtkn , could you please have a look at this?
Issue Time Tracking
-------------------
Worklog Id: (was: 837714)
Time Spent: 1h (was: 50m)
> Make sure dirPath has the correct permissions
> ---------------------------------------------
>
> Key: HIVE-26887
> URL: https://issues.apache.org/jira/browse/HIVE-26887
> Project: Hive
> Issue Type: Improvement
> Reporter: Zhang Dongsheng
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In the QueryResultsCache function of class QueryResultsCache, there is the
> following code segment
> {code:java}
> private QueryResultsCache(HiveConf configuration) throws IOException {
> ......
> FileSystem fs = cacheDirPath.getFileSystem(conf);
> FsPermission fsPermission = new FsPermission("700");
> fs.mkdirs(cacheDirPath, fsPermission);
> ......
> }
> {code}
> It can be seen that the function will use the mkdirs to create cacheDirPath,
> and the parameters passed in include the path variable cacheDirPath and a
> permission 700. But we haven't confirmed whether the permission is correctly
> assigned to the file.
> The above question is raised because there are two mkdir functions of hadoop,
> {code:java}
> mkdirs(Path f, FsPermission permission)
> {code}
> and
> {code:java}
> mkdirs(FileSystem fs, Path dir, FsPermission permission)
> {code}
> and the first one is used here. The permissions of this function will be
> affected by the underlying umask. Although 700 here will hardly be affected
> by umask, but I think from a rigorous point of view, we should have one more
> permission check and permission grant here.
> And I find same issue in other three methods here.
> In class Context
> {code:java}
> private Path getScratchDir(String scheme, String authority,
> boolean mkdir, String scratchDir) {
> ......
> FileSystem fs = dirPath.getFileSystem(conf);
> dirPath = new Path(fs.makeQualified(dirPath).toString());
> FsPermission fsPermission = new FsPermission(scratchDirPermission);
> if (!fs.mkdirs(dirPath, fsPermission)) {
> throw new RuntimeException("Cannot make directory: "
> + dirPath.toString());
> ......
> }
> {code}
> In class SessionState
> {code:java}
> static void createPath(HiveConf conf, Path path, String permission, boolean
> isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> ......
> if (!fs.mkdirs(path, fsPermission)) {
> throw new IOException("Failed to create directory " + path + " on fs "
> + fs.getUri());
> }
> ......
> }
> {code}
> and in class TezSessionState
> {code:java}
> private Path createTezDir(String sessionId, String suffix) throws IOException
> {
> ......
> Path tezDir = new Path(hdfsScratchDir, TEZ_DIR);
> FileSystem fs = tezDir.getFileSystem(conf);
> FsPermission fsPermission = new FsPermission(HiveConf.getVar(conf,
> HiveConf.ConfVars.SCRATCHDIRPERMISSION));
> fs.mkdirs(tezDir, fsPermission);
> ......
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
