I understand your desire for a bundled SDK for data routing, but NiFi/MiNiFi
isn’t designed for that right now. As the code is open source, you are welcome
to extract pieces you find useful to build your own. Kafka Connect or some
other libraries are probably more relevant to what you are trying
Thanks for that detailed response Andy.
I was not sure if it is advisable to use MiNiFi without connecting it back
to NiFi, so good to get clarity on that.
To take a stab at providing a reasoning for the 'MiNiFi as a library' use
case:
- I am looking to provide a library that exposes an SDK to c
I'm not sure I understand this proposal. MiNiFi can be installed completely
separately from NiFi (and usually they are not co-located on the same machine).
If your requirements are to connect an arbitrary Java program with Kafka to
produce data that is published to a Kafka topic, MiNiFi is a pot
As an alternative approach, is it conceivable to write a Custom Processor
and run that in the MiNiFi 'pipeline' without connecting back to NiFi
(Custom Processor -> Some Connector with socket / buffer input ->
PublishToKafka) ? This wouldn't hide the MiNiFi installation / runtime from
the user, but
Thanks for the pointer Joe. Let me look at Kafka Connect and see if that
can be used to address this use case.
On Wed, Sep 27, 2017 at 6:35 AM, Joe Witt wrote:
> Praveen
>
> I think the direction of MiNiFi and where it can head will support
> this case nicely. Today though I dont think we offer
Praveen
I think the direction of MiNiFi and where it can head will support
this case nicely. Today though I dont think we offer the simple
library model you're looking for. Fortunately, for Apache Kafka
specifically their community has developed Kafka Connect [1] which
sounds like it could be ju
Hi Aldrin,
I am looking at a couple of different flows, but to take the most simple
scenario, I would like to leverage the PublishToKafka_0_10 processor of
MiNiFi alone. The application will obtain some data, send it to a Processor
that has a simple upstream connection interface like a socket or i
Hello,
Neither NiFi nor MiNiFi (Java) is currently packaged as such. Would you be
able to expand upon your use case of publishing to Kafka and/or what facets
of MiNiFi you are looking to utilize? There may be other avenues that get
you toward your solution.
Thanks,
Aldrin
On Tue, Sep 26, 2017
Hi,
I am starting to look at MiNiFi for a use case that involves publishing to
Kafka. Is it feasible to use MiNiFi as a library (or set of libraries) from
within another Java program ? The primary reason for looking at such an
approach is the desire to hide the MiNiFi installation / process manage
