[
https://issues.apache.org/jira/browse/ARROW-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philipp Moritz updated ARROW-2953:
----------------------------------
Description:
While doing some memory profiling on the store, it became clear that at the
moment the metadata of the objects takes up much more space than it should. In
particular, for each object:
* The object id (20 bytes) is stored three times
* The object checksum (8 bytes) is stored twice
We can therefore significantly reduce the metadata overhead with some
refactoring.
was:
While doing some memory profiling on the store, it became clear that at the
moment the metadata of the objects takes up much more space than it should. In
particular, for each object:
* The object id (20 bytes) is stored three times
* The object checksum (8 bytes) is stored twice
* data_size and metadata_size (each 8 bytes) are stored twice
We can therefore significantly reduce the metadata overhead with some
refactoring.
> [Plasma] Store memory usage
> ---------------------------
>
> Key: ARROW-2953
> URL: https://issues.apache.org/jira/browse/ARROW-2953
> Project: Apache Arrow
> Issue Type: Improvement
> Reporter: Philipp Moritz
> Priority: Major
>
> While doing some memory profiling on the store, it became clear that at the
> moment the metadata of the objects takes up much more space than it should.
> In particular, for each object:
> * The object id (20 bytes) is stored three times
> * The object checksum (8 bytes) is stored twice
> We can therefore significantly reduce the metadata overhead with some
> refactoring.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
