William Lo created GOBBLIN-1905:
-----------------------------------
Summary: Fix bug where ORC self tune would under estimate native
ORC writer memory
Key: GOBBLIN-1905
URL: https://issues.apache.org/jira/browse/GOBBLIN-1905
Project: Apache Gobblin
Issue Type: Bug
Components: gobblin-core
Reporter: William Lo
Assignee: Abhishek Tiwari
During close, the native ORC writer would flush itself - emptying its buffers
and leading to the memory estimate to be extremely minimal, then record its
memory estimate in the saved state if it has not recorded an estimate before.
So for datasets that write a small number of records to file it would greatly
underestimate the memory within the native ORC writer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
