nifi questions

2017-07-04 Thread Clay Teahouse
Hello All, I am new to nifi. I'd appreciate your help with some questions. 1) Is there a processor like TCP Listen that would work with protobuf messages? In particular, I am interested in processing protobuf messages prefixed with the length of the message. I 2) Is there a processor like Consu

Re: NiFi questions

2018-05-03 Thread Clay Teahouse
Thanks Andy for the feedback On Wed, May 2, 2018 at 10:00 PM, Andy LoPresto wrote: > Hi Clay, > > A common use case for NiFi and Kafka in conjunction is when you want the > capabilities of a message broker system like Kafka with very low latency > and multiple publishers/consumers, but you also

Re: NiFi questions

2018-05-03 Thread Clay Teahouse
On Wed, May 2, 2018 at 10:00 PM, Andy LoPresto wrote: > Hi Clay, > > A common use case for NiFi and Kafka in conjunction is when you want the > capabilities of a message broker system like Kafka with very low latency > and multiple publishers/consumers, but you also have the need for some of > th

Re: nifi questions

2017-07-04 Thread Joe Witt
Clay Here some answers to each. Happy to discuss further. #1) No processors exist in the apache nifi codebase to receive or send data using google protobuf that I know of right now. This could work very well with our record oriented format and schema aware readers/writers though so perhaps it w

Re: nifi questions

2017-07-05 Thread Michael Hogue
Clay, Regarding number one, Joe is correct. There current isn't a processor that can process arbitrary protobuf messages, but InvokeGRPC and ListenGRPC were recently added (targeting 1.4.0) that can accept and send gRPC messages (which wrap protobuf) defined by an IDL [1]. There's a how-to artic

Re: nifi questions

2017-07-05 Thread Joe Witt
ah! Good call Mike and thanks for adding that. On Wed, Jul 5, 2017 at 2:21 PM, Michael Hogue wrote: > Clay, > > Regarding number one, Joe is correct. There current isn't a processor that > can process arbitrary protobuf messages, but InvokeGRPC and ListenGRPC were > recently added (targeting 1.

Re: nifi questions

2017-07-05 Thread Clay Teahouse
Thank you for very helpful feedback. On Wed, Jul 5, 2017 at 1:46 PM, Joe Witt wrote: > ah! Good call Mike and thanks for adding that. > > On Wed, Jul 5, 2017 at 2:21 PM, Michael Hogue > wrote: > > Clay, > > > > Regarding number one, Joe is correct. There current isn't a processor > that > > c