[
https://issues.apache.org/jira/browse/HIVE-24316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17403893#comment-17403893
]
Sungwoo commented on HIVE-24316:
--------------------------------
Hello,
It seems that with ORC-361, the use of MemoryManagerImpl in
LlapAwareMemoryManager is inconsistent.
Before merging ORC-361, LlapAwareMemoryManager sets its own totalMemoryPool and
MemoryManagerImpl accesses totalMemoryPool via getTotalMemoryPool(), so
everything is fine.
With ORC-361 merged, we have the following:
1. LlapAwareMemoryManager sets its own totalMemoryPool as a private field.
2. MemoryManagerImpl sets its own totalMemoryPool as a private field.
3. LlapAwareMemoryManager overrides getTotalMemoryPool() using its own
totalMemoryPool.
Now it is unclear whether or not getTotalMemoryPool() should be overridden.
Here are my thoughts on ORC-361:
1. Is MemoryManagerImpl intended to coordinate all threads writing to ORC files
inside a process (like LLAP Daemon)? Then is it necessary to create
LlapAwareMemoryManager as a ThreadLocal object? Why not just call
OrcFile.getStaticMemoryManager() to obtain the shared MemoryManagerImpl?
3. LlapAwareMemoryManager sets its own totalMemoryPool:
{code:java}
long memPerExecutor = LlapDaemonInfo.INSTANCE.getMemoryPerExecutor();
totalMemoryPool = (long) (memPerExecutor * maxLoad);
{code}
>From my understanding, this has no effect because MemoryManagerImpl sets its
>own totalMemoryPool.
Any comment would be appreciated.
> Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1
> ---------------------------------------------
>
> Key: HIVE-24316
> URL: https://issues.apache.org/jira/browse/HIVE-24316
> Project: Hive
> Issue Type: Bug
> Components: ORC
> Affects Versions: 3.1.3
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.3
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> This will bring eleven bug fixes.
> * ORC 1.5.7: [https://issues.apache.org/jira/projects/ORC/versions/12345702]
> * ORC 1.5.8: [https://issues.apache.org/jira/projects/ORC/versions/12346462]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
