[
https://issues.apache.org/jira/browse/SAMZA-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14300294#comment-14300294
]
Chinmay Soman commented on SAMZA-256:
-------------------------------------
[~jkreps] You mean extending from BaseKeyValueStorageEngineFactory (which in
turn wraps the inner store with serialized and cached stores) ? It was done at
the time to maintain consistency (otherwise the code becomes a bit messy).
Although - you're right. I'll open up a ticket to change the store wrapping
logic in case of an in-memory store:
* Ignore SerializedKeyValueStore
* Use generic types for the in-memory store
This will break the nice layering we have today. getKVStore() returns a
KeyValueStore[Array[Byte], Array[Byte]]. We'll now have to do something
different for in-memory store. But at a high level - I agree - the benefits are
worth the effort.
> Provide in-memory data store implementation
> -------------------------------------------
>
> Key: SAMZA-256
> URL: https://issues.apache.org/jira/browse/SAMZA-256
> Project: Samza
> Issue Type: Improvement
> Components: kv
> Affects Versions: 0.6.0
> Reporter: Jakob Homan
> Assignee: Chinmay Soman
> Fix For: 0.8.0
>
> Attachments: samza_256.patch, samza_256_1.patch, samza_256_2.patch
>
>
> The sole current kv store, LevelDbKeyValueStore, works well when the amount
> of data to be stored is prohibitively large to keep it all in memory.
> However, in cases where the state is small enough to comfortably fit in
> whatever memory is available, it would be better to provide an in-memory
> implementation. This can be backed by either a native Java class, or perhaps
> a Guava class, if that is found to scale better (or, of course, the backing
> implementation could be configurable).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
