Awesome product. Thank you Ali
On Wed, Apr 6, 2016 at 4:27 PM, Joe Witt <joe.w...@gmail.com> wrote: > Ali - I can confirm that all formats are supported for Put/Get Kafka > processors. The interaction with Kafka as far as the messages are > concerned is just 'bytes'. The reason it isn't working well is due to > the bug I'm mentioning and that is a NiFi bug (not Kafka). It is > fixed and we'll have a release for you asap but if you want to build > that branch then you're in great shape right away. > > Thanks > > On Wed, Apr 6, 2016 at 10:24 AM, Ali Karray <c...@karray.net> wrote: >> I see the patch in Jira. >> You confirm that avro format is supported for putKafka and getKafka >> processor ? >> >> Thanks >> Ali >> >> On Wed, Apr 6, 2016 at 4:14 PM, Joe Witt <joe.w...@gmail.com> wrote: >>> Ali >>> >>> It is very possible you're being bitten by a bug we have with >>> PutKafka. It is fixed in this branch which we're in the release >>> motions for right now. >>> >>> https://github.com/apache/nifi/tree/support/nifi-0.6.x >>> >>> Thanks >>> Joe >>> >>> On Wed, Apr 6, 2016 at 10:07 AM, Ali Karray <c...@karray.net> wrote: >>>> I'm using the last version 0.6 of Nifi and the 0.9 kafka version >>>> When I use plain csv files, all is working. >>>> >>>> Thanks, >>>> Ali >>>> >>>> >>>> On Wed, Apr 6, 2016 at 3:23 PM, Bryan Bende <bbe...@gmail.com> wrote: >>>>> >>>>> Hello, >>>>> >>>>> What version of Apache NiFi are you using? >>>>> >>>>> Also, lets take Avro and kafka-avro-console-consumer out of the mix for a >>>>> minute, what happens if you used PutKafka to send some plain text to >>>>> Kafka, and then tried to consumer from that topic with GetKafka? >>>>> You mentioned this didn't work, but what happened? was there an error >>>>> message? did it just receive 0 messages from Kafka? >>>>> >>>>> Thanks, >>>>> >>>>> Bryan >>>>> >>>>> On Wed, Apr 6, 2016 at 6:10 AM, Ali Karray <c...@karray.net> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm exploring integration possibility of nifi processors with kafka. >>>>>> I use PutKafka to puch avro generated flow from ExecuteSQL. >>>>>> I verify that the topic created in Kafka is alimented with data using >>>>>> kafka-console-consumer bash command. >>>>>> Then I try to use getKafka processor to pull all the Data stored in >>>>>> the kafka topic and store all that data in a local File using the >>>>>> PutFile processor. I realize that the getKafka processor don't pull >>>>>> the topic content. >>>>>> >>>>>> I verify if all is ok with kafka-avro-console-consumer (confluent >>>>>> command), I get this exception : >>>>>> org.apache.kafka.common.errors.SerializationException: Error >>>>>> deserializing Avro message for id -1 >>>>>> Caused by: org.apache.kafka.common.errors.SerializationException: >>>>>> Unknown magic byte! >>>>>> >>>>>> I tested also getKafka from a basic topic format and it doesn't work >>>>>> for me. >>>>>> >>>>>> So I have some questions : >>>>>> How can I deal with avro format using getKafka anf putKafka processor ? >>>>>> Any idea to resolve getKafka processor problem to pull content from a >>>>>> kafka Topic ? >>>>>> >>>>>> >>>>>> Thank you in advance >>>>>> >>>>>> A. Karray >>>>> >>>>>
