[
https://issues.apache.org/jira/browse/HIVE-25474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
guangbao zhao updated HIVE-25474:
---------------------------------
Description: In the Linux environment, adding multiple jars concurrently
through HiveCli or JDBC will increase the system cpu and even affect the
service. Finally, we found that when the add jar is executed, the FileUtil
chmod method is used to grant permissions to the downloaded jar file. The
performance of this method is very low. So we use the setPosixFilePermissions
method of the Files class to test. The performance is seventy to eighty times
that of FileUtil (the same file is given permissions in multiple cycles, when
it is cycled 1000 times), and as the number of cycles increases, the gap
becomes larger and larger. But the file requires jdk7+, which is not friendly
to windows. Therefore, if you use the setPosixFilePermissions method of the
Files class to grant permissions to files in an operating system that conforms
to the posix specification(tested on Mac and Linux), the performance will be
improved. (was: In the Linux environment, adding multiple jars concurrently
through HiveCli or JDBC will increase the system cpu and even affect the
service. Finally, we found that when the add jar is executed, the FileUtil
chmod method is used to grant permissions to the downloaded jar file. The
performance of this method is very low. So we use the setPosixFilePermissions
method of the Files class to test. The performance is seventy to eighty times
that of FileUtil (the same file is given permissions in multiple cycles, when
it is cycled 1000 times). But the file requires jdk7+, which is not friendly to
windows. Therefore, if you use the setPosixFilePermissions method of the Files
class to grant permissions to files in an operating system that conforms to the
posix specification(tested on Mac and Linux), the performance will be improved.)
> Concurrency add jars cause hiveserver2 sys cpu to high
> ------------------------------------------------------
>
> Key: HIVE-25474
> URL: https://issues.apache.org/jira/browse/HIVE-25474
> Project: Hive
> Issue Type: Improvement
> Components: Hive, HiveServer2
> Affects Versions: 3.1.2
> Reporter: guangbao zhao
> Assignee: guangbao zhao
> Priority: Major
> Attachments: HIVE-25474.jpg, HIVE-25474.patch, PermissionTest.java
>
>
> In the Linux environment, adding multiple jars concurrently through HiveCli
> or JDBC will increase the system cpu and even affect the service. Finally, we
> found that when the add jar is executed, the FileUtil chmod method is used to
> grant permissions to the downloaded jar file. The performance of this method
> is very low. So we use the setPosixFilePermissions method of the Files class
> to test. The performance is seventy to eighty times that of FileUtil (the
> same file is given permissions in multiple cycles, when it is cycled 1000
> times), and as the number of cycles increases, the gap becomes larger and
> larger. But the file requires jdk7+, which is not friendly to windows.
> Therefore, if you use the setPosixFilePermissions method of the Files class
> to grant permissions to files in an operating system that conforms to the
> posix specification(tested on Mac and Linux), the performance will be
> improved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
