[
https://issues.apache.org/jira/browse/HIVE-25416?focusedWorklogId=756448&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-756448
]
ASF GitHub Bot logged work on HIVE-25416:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Apr/22 14:54
Start Date: 13/Apr/22 14:54
Worklog Time Spent: 10m
Work Description: nrg4878 commented on code in PR #2555:
URL: https://github.com/apache/hive/pull/2555#discussion_r849582154
##########
pom.xml:
##########
@@ -108,10 +108,10 @@
<avro.version>1.8.2</avro.version>
<bcprov-jdk15on.version>1.64</bcprov-jdk15on.version>
<calcite.version>1.25.0</calcite.version>
- <datanucleus-api-jdo.version>5.2.4</datanucleus-api-jdo.version>
- <datanucleus-core.version>5.2.4</datanucleus-core.version>
+ <datanucleus-api-jdo.version>5.2.6</datanucleus-api-jdo.version>
Review Comment:
According to the release notes, this memory leak issue is fixed in 5.2.7 and
NOT in 5.2.6.
https://www.datanucleus.org/documentation/news/access_platform_5_2.html#_datanucleus_accessplatform_5_2_7
So this PR will not resolve the issue.
Also there are some important bug fixes like support for sequences in MS SQL
etc in later minor releases. We should atleast try to move up to 5.2.10 (5.2.11
is also available, but not many fixes that directly affect hive).
Issue Time Tracking
-------------------
Worklog Id: (was: 756448)
Time Spent: 1.5h (was: 1h 20m)
> Hive metastore memory leak because datanucleus-api-jdo bug
> ----------------------------------------------------------
>
> Key: HIVE-25416
> URL: https://issues.apache.org/jira/browse/HIVE-25416
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
> Affects Versions: 3.1.2
> Reporter: shezm
> Assignee: shezm
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.3
>
> Attachments: leak.jpg
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> I encountered a memory leak case. The MAT info :
> !leak.jpg!
> Full error message is :
> {code:java}
> Cannot get Long result for param = 8 for column "`FUNCS`.`FUNC_ID`" :
> Operation not allowed after ResultSet closed{code}
> This is because there is a bug in the JDOPersistenceManager.retrieveAll code.
> {code:java}
> // code placeholder
> JDOPersistenceManager{
> public void retrieveAll(Collection pcs, boolean useFetchPlan) {
> this.assertIsOpen();
> ArrayList failures = new ArrayList();
> Iterator i = pcs.iterator();
> while(i.hasNext()) {
> try {
> this.jdoRetrieve(i.next(), useFetchPlan);
> } catch (RuntimeException var6) {
> failures.add(var6);
> }
> }
> if (!failures.isEmpty()) {
> throw new JDOUserException(Localiser.msg("010038"),
> (Exception[])((Exception[])failures.toArray(new Exception[failures.size()])));
> }
> }
> }
> {code}
> In some extreme cases the function of next() does not work . This will
> result in a very large failures ArrayList like as shown above.
>
> The bug detail can see this :
> [https://github.com/datanucleus/datanucleus-api-jdo/issues/106]
> This problem is fixed in datanucleus-api-jdo version 5.2.6. So we should
> upgrade it .
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
