[
https://issues.apache.org/jira/browse/SAMZA-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14149946#comment-14149946
]
Chris Riccomini commented on SAMZA-236:
---------------------------------------
[~nsomasun], I've applied the patch locally, alongside [my
patch|https://github.com/facebook/rocksdb/issues/308]. It doesn't seem to
compile.
{noformat}
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala:19:
error: value setCacheSize is not a member of org.rocksdb.Options
options.setCacheSize(cacheSize / containerContext.taskNames.size)
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala:21:
error: value setBlockSize is not a member of org.rocksdb.Options
options.setBlockSize(storeConfig.getInt("rocksdb.block.size.bytes", 4096))
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala:30:
error: value setFilter is not a member of org.rocksdb.Options
options.setFilter(new BloomFilter);
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala:32:
error: value setCacheSize is not a member of org.rocksdb.Options
options.setCacheSize(128*1024*1024)
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala:49:
error: not found: type Logging
val metrics: KeyValueStoreMetrics = new KeyValueStoreMetrics) extends
KeyValueStore[Array[Byte], Array[Byte]] with Logging {
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala:126:
error: not found: value trace
trace("Flush in RocksDbKeyValueStore is not supported, ignoring")
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala:130:
error: not found: value trace
trace("Closing.")
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/TestRocksDbApi.scala:15:
error: value setDisableSeekCompaction is not a member of org.rocksdb.Options
options.setCreateIfMissing(true).createStatistics().setWriteBufferSize(8 *
SizeUnit.KB).setMaxWriteBufferNumber(3).setDisableSeekCompaction(
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/TestRocksDbApi.scala:22:
error: value disableSeekCompaction is not a member of org.rocksdb.Options
assert(options.disableSeekCompaction() == true);
^
/Users/criccomi/Code/incubator-samza/samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/TestRocksDbApi.scala:23:
error: value blockSize is not a member of org.rocksdb.Options
assert(options.blockSize() == 64 * SizeUnit.KB);
^
10 errors found
:samza-kv-rocksdb_2.10:compileScala FAILED
FAILURE: Build failed with an exception.
{noformat}
Sure enough, these methods don't seem to be in RocksDB's master branch. Do I
need to apply some patch to master to get this to work? Thought your patches
were merged in?
> Switch samza-kv to use RocksDB
> ------------------------------
>
> Key: SAMZA-236
> URL: https://issues.apache.org/jira/browse/SAMZA-236
> Project: Samza
> Issue Type: Bug
> Components: kv
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Assignee: Naveen
> Fix For: 0.8.0
>
>
> The samza-kv implementation is currently using LevelDB. RocksDB seems to have
> a number of interesting options that would be useful to Samza:
> # Ability to swap out skip list for a faster implementation when restoring
> data from a changelog stream.
> # Ability to disable compaction during changelog stream restoration.
> # Ability to have a single level (or single SST) rather than multiple levels
> in order to reduce read amplification.
> # A merge operation for doing lazy read-modify-write during compaction.
> And a lot more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
