HI All,
I understand that Apache Nifi has integration with many systems but what If
I have an application that talks a custom protocol ? How do I integrate
Apache Nifi with the custom protocol?
Thanks,
kant
Kant,
Look into a custom processor. You have a choice of either implementing e.g.
a parser for your data, or, if the protocol is more involved, implement a
receiver in the processor as well which would spit out meaningful data
messages next.
Andrew
On Tue, Dec 6, 2016, 7:47 PM kant kodali wrote
Kant
Typically this is as simple as creating your processor to utilize that protocol.
Take a look at the developer guide [1] and of course you can always
look at the many example processors and components that come in NiFi
out of the box [2].
[1] https://nifi.apache.org/docs.html
[2] https://git
Hi Kant
What you’re trying to accomplish is definitely possible, however more
information may be needed from you.
For example, the way I understand your statement about “integration with many
systems” is something like JMS, Kafka, TCP, FTP etc…. If that is the case such
integration is definitel
Thanks a ton guys! Didn't expect Nifi community to be so good! (Another
convincing reason!)
Coming back to the problem, We use NSQ a lot (although not my favorite) and
want to be able integrate Nifi with NSQ to other systems such as Kafka,
Spark, Cassandra, ElasticSearch, some micro services that
Hi Kant,
Although I'm not aware of existing processor for HTTP2 or NSQ, NiFi
has a set of processors for WebSocket since 1.1.0.
It enables NiFi to act as a WebSocket client to communicate with a
remote WebSocket server, or makes NiFi a WebSocket server so that
remote clients access to it via WebSo
Hi Koji,
That is an awesome explanation! I expected processors for HTTP2 at very
least since it is widely used ( the entire GRPC stack runs on that). I am
not sure how easy or hard it is to build one?
Thanks!
On Wed, Dec 7, 2016 at 1:08 AM, Koji Kawamura
wrote:
> Hi Kant,
>
> Although I'm not
I am also confused a little bit since I am new to Nifi. I wonder why Nifi
would act as a server? isn't Nifi a routing layer between systems? because
this brings in another question about Nifi in general.
When I write my applications/microservices do I need to worry about how my
service would talk
Kant
There are couple of questions here so, let me try one at the time.
1. “why Nifi would act as a server?”. Well, NiFi is a runtime environment where
things are happening. To be more exact; NiFi is a runtime environment where
things are triggered. The big distinction here is trigger vs happeni
Sorry I should have been more clear. My question is even more simpler and
naive. Say I am writing a HTTP based microservice (Now I assume Nifi has
integration with HTTP since it is a well known protocol ). Now, how would I
integrate Nifi with my HTTP based server?
On Wed, Dec 7, 2016 at 4:55 AM, O
Kant
So, yes NiFi already provides basic integration with HTTP by virtue of PostHTTP
processor (post a content of the FlowFile to HTTP endpoint) and the same in
reverse with GetHTTP. So for you case your integration will be as simple as
posting or getting from your micro service
And what makes
so do I need to import PostHTTP processor, GetHTTP processor Libraries in
my application code? or can I just build a HTTP Server and tell Nifi JVM
that "hey here is my Application (it runs on specific IP and port) and it
uses HTTP".
On Wed, Dec 7, 2016 at 12:00 PM, Oleg Zhurakousky <
ozhurakou...@
Ohh no, not at all. Just tell it the host/port combination and off you go,
basically the second part of what you said
Oleg
On Dec 7, 2016, at 3:08 PM, kant kodali
mailto:kanth...@gmail.com>> wrote:
so do I need to import PostHTTP processor, GetHTTP processor Libraries in my
application code?
Thats what I was looking for. That sounds awesome because it is decoupled
from the application itself!
Finally, is there any example or some pointer you can point me to that
shows how Nifi takes cares in case of different protocols (for now you can
assume all standard protocol) so say a simplest c
Hi Kant,
Glad to know that you liked the explanation, thanks :)
By reading discussion in this thread, I assume what you'd like to do
with NiFi is, get messages from a system amd do some routing/filtering
on NiFi, then send it to another system, using a custom protocol.
If so, you can write two cu
Hi Koji,
Thanks again for your response.
"Should an application/micro service worry about how it talk with NiFi?
My answer would be no. The service should focus and use the best
protocol or technology for its purpose."
This sounds great. This is what we want! I also understand the other
capabil
Hi Kant,
Before we get too much farther, I think the Overview [1] and Getting Started
[2] guide and Joe Witt’s OSCON talk [3] might be valuable because they really
underline the problem(s) NiFi solves and the behaviors it uses to do that.
In the scenario you described, you only need one Nifi no
Hi Andy,
I am a bit confused again. In my example my deployment is fixed right
Node1 has my HTTP service and my Node 2 has Cassandra.
The communication between Node1 and Node2 is through CQL3.
A client will initiate the request to the my service on Node 1 via HTTP and
you can assume a client is a
If the only communication protocol allowed between the two nodes is CQL3, then
yes, you can place NiFi on Node 1 and have the HTTP micro service communicate
to the same host. Then NiFi will communicate to Node 2 via CQL. Regardless of
which node it is residing on, for this scenario you only need
19 matches
Mail list logo
