[
https://issues.apache.org/jira/browse/SAMZA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304266#comment-14304266
]
Chris Riccomini commented on SAMZA-545:
---------------------------------------
That's how it works today. The problem is that there's layering in the KV store:
KVStore
CachedStore
SerdeStore
LoggingStore
RocksDBStore/InMemory
The Serde has to happen before logging, and the logging has to happen before
RocksDB. This is for writes. For reads, if the underlying store is in-memory,
the reads could be served without serde. So maybe some tweaking to the layering
could fix this.
> Make in-memory key-value store skip serde
> -----------------------------------------
>
> Key: SAMZA-545
> URL: https://issues.apache.org/jira/browse/SAMZA-545
> Project: Samza
> Issue Type: Bug
> Components: kv
> Affects Versions: 0.9.0
> Reporter: Chris Riccomini
>
> SAMZA-256 added an in-memory implementation of the samza-kv store. Due to the
> layering in Samza's KV-store APIs, the in-memory store still holds raw bytes,
> and the Serde is used to transform objects back into POJOs. On the read-side,
> it is unnecessary to deserialize the bytes back into an object. The in-memory
> KV store should just hold the raw object.
> Semantically, this does change the behavior of the KV-store a bit, when using
> in-memory stores. If an object is mutated after it's been written to an
> in-memory store, and then store.get is called, the mutated object will be
> returned. This is not the case with regular (LevelDB/RocksDB) KV-stores.
> Writes will still require serializing the object if a changelog is attached.
> If a changelog is not attached, then I'd argue that the in-memory KV store
> should not be used at all, and a simple HashMap should be used instead.
> When updating the code, we should be mindful to keep the API as clean as
> possible, while shifting the layers around.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
