[
https://issues.apache.org/jira/browse/HIVE-25474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
guangbao zhao updated HIVE-25474:
---------------------------------
Fix Version/s: (was: 3.1.2)
Target Version/s: (was: 3.1.2)
Description: In the Linux environment, adding multiple jars
concurrently through HiveCli or JDBC will increase the system cpu and even
affect the service. Finally, we found that when the add jar is executed, the
FileUtil chmod method is used to grant permissions to the downloaded jar file.
The performance of this method is very low. So we use the
setPosixFilePermissions method of the Files class to test. The performance is
seventy to eighty times that of FileUtil (the same file is given permissions in
multiple cycles, when it is cycled 1000 times). But the file requires jdk7+,
which is not friendly to windows. Therefore, if you use the
setPosixFilePermissions method of the Files class to grant permissions to files
in an operating system that conforms to the posix specification(tested on Mac
and Linux), the performance will be improved. (was: In the Linux environment,
when there are multiple concurrent add jars through HiveCli or JDBC, the system
cpu will increase. The currently used FileUtil.chmod(dest, "ugo+rx", true);
method is used for file authorization, However, in jdk7+, can use
Files.setPosixFilePermissions(path, perms); for file authorization. The
performance is seventy to eighty times that of the above. Why not apply this
method?)
Summary: Improvement concurrency add jars cause hiveserver2 sys
cpu to high (was: concurrency add jars cause hiveserver2 sys cpu to high)
> Improvement concurrency add jars cause hiveserver2 sys cpu to high
> ------------------------------------------------------------------
>
> Key: HIVE-25474
> URL: https://issues.apache.org/jira/browse/HIVE-25474
> Project: Hive
> Issue Type: Improvement
> Components: Hive, HiveServer2
> Affects Versions: 3.1.2
> Reporter: guangbao zhao
> Assignee: guangbao zhao
> Priority: Major
> Attachments: HIVE-25474.patch
>
>
> In the Linux environment, adding multiple jars concurrently through HiveCli
> or JDBC will increase the system cpu and even affect the service. Finally, we
> found that when the add jar is executed, the FileUtil chmod method is used to
> grant permissions to the downloaded jar file. The performance of this method
> is very low. So we use the setPosixFilePermissions method of the Files class
> to test. The performance is seventy to eighty times that of FileUtil (the
> same file is given permissions in multiple cycles, when it is cycled 1000
> times). But the file requires jdk7+, which is not friendly to windows.
> Therefore, if you use the setPosixFilePermissions method of the Files class
> to grant permissions to files in an operating system that conforms to the
> posix specification(tested on Mac and Linux), the performance will be
> improved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
