[
https://issues.apache.org/jira/browse/OOZIE-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junfan Zhang updated OOZIE-3396:
--------------------------------
Description:
Phenomenon:
Recently we upgraded the {{AuthOozieClient}} and added the OOZIE-2447. When we
were running online, the server memory usage continued to increase. Through the
jmap and jhat command analysis, it was found that the {{deleteOnExist}} method
in the writeAuthToken method under the {{AuthOozieClient}} class occupies a
large amount of memory, which is caused by OOZIE-2447. Also, because I have
applied the OOZIE-3379 in the production environment. I still find that the
.oozie-auth-token-xxxx file update time has changed, but the file content has
not changed.
After reviewing the code, I found that {{currentToken.equals(readToken)}} in
the {{AuthOozieClient}} class is always false because the
{{AuthenticatedURL.token}} does not have an equals method. The reason is
because {{currentToken.equals(readToken)}} is always false, causing the tmp
file to be created continuously, but there is a memory leak due to the
{{deleteOnExist}} method
([https://stackoverflow.com/questions/40119188/memory-leak-on-deleteonexithook])
Solution:
1. In {{AuthOozieClient}}, change to
{{currentToken.toString().equals(readToken.toString())}}
2. In addition, the deleteOnExist method should be changed to the delete() in
AuthOozieClient class.
Doubt:
Why delete the equals method in the {{AuthenticatedURL.Token}} class in Hadoop?
See: https://issues.apache.org/jira/browse/HADOOP-10771
was:
Phenomenon:
Recently we upgraded the {{AuthOozieClient}} and added the OOZIE-2447 (Illegal
character 0x0 oozie client) patch. When we were running online, the server
memory usage continued to increase. Through the jmap and jhat command analysis,
it was found that the {{deleteOnExist}} method in the writeAuthToken method
under the {{AuthOozieClient}} class occupies a large amount of memory, which is
caused by OOZIE-2447. Also, because I have applied the OOZIE-3379 in the
production environment. I still find that the .oozie-auth-token-xxxx file
update time has changed, but the file content has not changed.
After reviewing the code, I found that {{currentToken.equals(readToken)}} in
the {{AuthOozieClient}} class is always false because the
{{AuthenticatedURL.token}} does not have an equals method. The reason is
because {{currentToken.equals(readToken)}} is always false, causing the tmp
file to be created continuously, but there is a memory leak due to the
{{deleteOnExist}} method
([https://stackoverflow.com/questions/40119188/memory-leak-on-deleteonexithook])
Solution:
1. In {{AuthOozieClient}}, change to
{{currentToken.toString().equals(readToken.toString())}}
2. In addition, the deleteOnExist method should be changed to the delete() in
AuthOozieClient class.
Doubt:
Why delete the equals method in the {{AuthenticatedURL.Token}} class in Hadoop?
See: https://issues.apache.org/jira/browse/HADOOP-10771
> AuthOozieClient leaks memory
> ----------------------------
>
> Key: OOZIE-3396
> URL: https://issues.apache.org/jira/browse/OOZIE-3396
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 5.0.0
> Reporter: Junfan Zhang
> Assignee: Junfan Zhang
> Priority: Major
>
> Phenomenon:
> Recently we upgraded the {{AuthOozieClient}} and added the OOZIE-2447. When
> we were running online, the server memory usage continued to increase.
> Through the jmap and jhat command analysis, it was found that the
> {{deleteOnExist}} method in the writeAuthToken method under the
> {{AuthOozieClient}} class occupies a large amount of memory, which is caused
> by OOZIE-2447. Also, because I have applied the OOZIE-3379 in the production
> environment. I still find that the .oozie-auth-token-xxxx file update time
> has changed, but the file content has not changed.
> After reviewing the code, I found that {{currentToken.equals(readToken)}} in
> the {{AuthOozieClient}} class is always false because the
> {{AuthenticatedURL.token}} does not have an equals method. The reason is
> because {{currentToken.equals(readToken)}} is always false, causing the tmp
> file to be created continuously, but there is a memory leak due to the
> {{deleteOnExist}} method
> ([https://stackoverflow.com/questions/40119188/memory-leak-on-deleteonexithook])
> Solution:
> 1. In {{AuthOozieClient}}, change to
> {{currentToken.toString().equals(readToken.toString())}}
> 2. In addition, the deleteOnExist method should be changed to the delete() in
> AuthOozieClient class.
> Doubt:
> Why delete the equals method in the {{AuthenticatedURL.Token}} class in
> Hadoop? See: https://issues.apache.org/jira/browse/HADOOP-10771
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
