[
https://issues.apache.org/jira/browse/SAMZA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304616#comment-14304616
]
Jay Kreps commented on SAMZA-545:
---------------------------------
Another thing worth mentioning is that the existing key-value layer is smart
about logging. If you do 1000 updates in a row and then commit, the logging
only happens when the commit happens, so it only logs one entry per key not one
per update. So e.g. if all 1000 updates were to the same row, you only need to
log one row. It may or may not be possible to retain this but it is nice
because it allows keeping a small number of counters without going crazy on
logging.
> Make in-memory key-value store skip serde
> -----------------------------------------
>
> Key: SAMZA-545
> URL: https://issues.apache.org/jira/browse/SAMZA-545
> Project: Samza
> Issue Type: Bug
> Components: kv
> Affects Versions: 0.9.0
> Reporter: Chris Riccomini
>
> SAMZA-256 added an in-memory implementation of the samza-kv store. Due to the
> layering in Samza's KV-store APIs, the in-memory store still holds raw bytes,
> and the Serde is used to transform objects back into POJOs. On the read-side,
> it is unnecessary to deserialize the bytes back into an object. The in-memory
> KV store should just hold the raw object.
> Semantically, this does change the behavior of the KV-store a bit, when using
> in-memory stores. If an object is mutated after it's been written to an
> in-memory store, and then store.get is called, the mutated object will be
> returned. This is not the case with regular (LevelDB/RocksDB) KV-stores.
> Writes will still require serializing the object if a changelog is attached.
> If a changelog is not attached, then I'd argue that the in-memory KV store
> should not be used at all, and a simple HashMap should be used instead.
> When updating the code, we should be mindful to keep the API as clean as
> possible, while shifting the layers around.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
