[
https://issues.apache.org/jira/browse/NIFI-12442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792419#comment-17792419
]
David Handermann commented on NIFI-12442:
-----------------------------------------
Thanks for the reply and additional background.
With the comments around external storage of a RocksDB file, it sounds like the
general idea is using it as a cache that could be relocated? If that is
correct, are there advantages to RocksDB over a simpler format? The challenge
in the longer term could be upgrades to the RocksDB library and potential
compatibility with stored database files.
Another consideration is the compatibility and maintainability of RocksDB
itself. More recent versions appear to work across most major operating system
platforms, but at the expense of very large JAR files containing all possible
platform-specific libraries. Version 8.8.1 of the RocksDB JNI JAR is over 60
MB, which is very large for an optional extension component.
NiFi 2.0.0-M1 and 1.24.0 transitioned from H2 Database Engine to JetBrains
Xodus for local storage of Flow Configuration History, and it is worth noting
that both of these libraries are pure Java and do not have platform-specific
elements. This is a framework-level feature, but it is worth mentioning in this
context as they provide file-backed persistent storage.
If the persistent caching concept is the primary use case, it seems like there
are better solutions, unless there is something unique that RocksDB provides.
> Adding support for RocksDB
> --------------------------
>
> Key: NIFI-12442
> URL: https://issues.apache.org/jira/browse/NIFI-12442
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.23.2
> Reporter: Giovanni
> Priority: Minor
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> I would like to suggest the creation of 3 new components:
> The first one is a service, which opens an existing RocksDb or, eventually,
> using RocksOptions, create it from scratch. It will manage all the open
> options (classic, read/write, only read, secondary).
> The second one, is a RocksDbReader, that uses the service to communicate with
> the RocksDb in order to retrieve informations through a lookup. It can save
> the searched content inside an attribute or inside the flowFile content. It
> will be capable of using both APIs, such as simple "db.get" and via
> RocksIterator.
> The last one, is a RocksDbWriter, that uses the same service as the reader,
> but can write values inside the RocksDb, both from flowFile attribute or
> flowFile content, using properties to determine the key to use.
>
> Feel free to express your opinions, if you think this will be useful or
> useless.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
