[
https://issues.apache.org/jira/browse/SAMZA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13894982#comment-13894982
]
Chris Riccomini commented on SAMZA-47:
--------------------------------------
I misunderstood what you were saying. I agree with you.
We should update the patch to have something like
stores.my-store.container.cache.size.bytes. This would define per-container
LevelDB cache sizes. Thus, each partition in a container gets a byte cache
computed by:
stores.my-store.container.cache.size.bytes / # of partitions in container.
The # of partitions in container is already defined in SamzaContainer, and
should be exposed to StorageEngineFactory. It seems a bit hacky to just add a
specific parameter for this use case to the StorageEngineFactory. Maybe a
better solution is to create a SamzaContainerContext object that has the
container name, the partitions it's responsible for, etc. We could then give
the whole container context to the StorageEngineFactory.
Martin, do you want to have a look at this?
> LevelDB and L1 cache use the same configuration value in KeyValueStorageEngine
> ------------------------------------------------------------------------------
>
> Key: SAMZA-47
> URL: https://issues.apache.org/jira/browse/SAMZA-47
> Project: Samza
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Jay Kreps
> Assignee: Jay Kreps
>
> Both seem to key off of
> cache.size
> This is not right. The L1 cache is caching a number of objects and leveldb is
> allocating a number of bytes. In general the leveldb cache should be big
> (tens of MBs) and the L1 cache small (a few thousand).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
