Wail Y. Alkowaileet created ASTERIXDB-3314:
----------------------------------------------
Summary: Reduce buffer cache pressure when operating against
columnar datasets
Key: ASTERIXDB-3314
URL: https://issues.apache.org/jira/browse/ASTERIXDB-3314
Project: Apache AsterixDB
Issue Type: Bug
Components: STO - Storage
Affects Versions: 0.9.9
Reporter: Wail Y. Alkowaileet
Assignee: Wail Y. Alkowaileet
Fix For: 0.9.9
When ingesting columnar datasets, the bulkloader and its columnar writers rely
on the buffer cache to provide the necessary buffers for writing. Specifically,
the buffer cache provides temporary buffers to the columnar writers. However,
not all columns require a full 128KB buffers (e.g., sparse columns). Instead of
using precious buffer cache pages for such columns, we should allow column
writers to allocate smaller buffers to be used initially. In case they needed
more space, they can ask the buffer cache for more. This approach relieve the
pressure against the buffer cache.
Another issue/bug (related to the buffer cache), columnar filters are not
unpinning their pages when they're done – holding those pages indefinitely
until the next restart.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
