[
https://issues.apache.org/jira/browse/NIFI-12442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794728#comment-17794728
]
David Handermann commented on NIFI-12442:
-----------------------------------------
Thanks for the link to the repository [~kommpn], it is helpful to see the
implementation.
On initial evaluation, it seems like the Reader and Writer Processors would not
be necessary if you used RocksDB to implement the
[DistributedMapCacheClient|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-standard-services/nifi-distributed-cache-client-service-api/src/main/java/org/apache/nifi/distributed/cache/client/DistributedMapCacheClient.java]
interface. That interface can be used with existing Processors to add and
remove elements, which appears to be the primary purpose of the custom
Processors, if I am following the implementation.
It is worth noting that the DistributedMapCacheClient, by its name, aims to
support services that are in fact distributed, which RocksDB is not. However,
for a proof-of-concept, the interface seems like a good fit, and it would
enable integration with existing components.
In terms of Apache NiFi maintenance, there is still the concern about the
RocksDB JNI JAR containing 60 MB of native libraries, but as a generally
available component through Git, you could also publish it to Maven Central for
others to evaluate.
> Adding support for RocksDB
> --------------------------
>
> Key: NIFI-12442
> URL: https://issues.apache.org/jira/browse/NIFI-12442
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.23.2
> Reporter: Giovanni
> Priority: Minor
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> I would like to suggest the creation of 3 new components:
> The first one is a service, which opens an existing RocksDb or, eventually,
> using RocksOptions, create it from scratch. It will manage all the open
> options (classic, read/write, only read, secondary).
> The second one, is a RocksDbReader, that uses the service to communicate with
> the RocksDb in order to retrieve informations through a lookup. It can save
> the searched content inside an attribute or inside the flowFile content. It
> will be capable of using both APIs, such as simple "db.get" and via
> RocksIterator.
> The last one, is a RocksDbWriter, that uses the same service as the reader,
> but can write values inside the RocksDb, both from flowFile attribute or
> flowFile content, using properties to determine the key to use.
>
> Feel free to express your opinions, if you think this will be useful or
> useless.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
