Re: Approaches to Array in Json with Nifi?

Sorry, Aldrin. I forgot to answer your question. My arrays do not share fixed number of values. Thanks. Hong *Hong Li* *Centric Consulting* *In Balance* (888) 781-7567 office (614) 296-7644 mobile www.centricconsulting.com | @Centric On Wed, Apr 6, 2016 at 4:3

Re: Approaches to Array in Json with Nifi?

I have two goals here for the information embedded in the array. The first goal is to be able to do some kind of aggregation on the array. For example, given {"company":"xyz", "rate":[0.02, 0.03, 0.04]}, if we could have a Json variable "rate", I would like to have something such as ${rate[*]:coun

Re: Few doubts about NiFi cluster

Hi Sudeep, Answers to your questions: 1) In a clustered environment, each processor will have the same configuration the flows are identical across each node in the flow with the NCM establishing mutations across each instance via a two-phase commit. Through the mentioned scheduling strategy, onl

Re: nifi.content.repository.archive.max.retention.period

Andre, Sorry this got lost in the mail pile. Your assumptions are correct and that is the anticipated behavior. Has this been a reproducible and/or persistent issue (the lack of archiving causes disk consumption to grow unbounded)? If so, any level of context you can provide to the flow such as

Re: custom processor development : problems with scops in parent POM

Hello! Are you trying to incorporate this into the repository check out or a standalone bundle? In the case of the latter, could you confirm you have established a project dependency structure such as that outlined in the Wiki [1]? Not only is a provided scope needed to be able to build/compile,

Re: Approaches to Array in Json with Nifi?

​Does it store as an attribute if you change the return type from auto-detect to JSON ? *Return Type* auto-detect - auto-detect - json - scalar Indicates the desired return type of the JSON Path expressions. Selecting 'auto-detect' will set the return type to 'json' for a Destination o

Re: Approaches to Array in Json with Nifi?

Hi Hong! What are you seeking to do with the rate data after extracting? Is the number of rates fixed? If not, not sure there is an easy path forward with just EvaluateJsonPath to make this happen. I am wondering if SplitJson might be appropriate to use as a follow-on, but could use some more c

Re: Approaches to Array in Json with Nifi?

The issue I run into there is when I specify under EvaluationJsonPath, firstRateValue = $.rate[0] $.rate[0] is a scalar value that is assigned to the processor property firstRateValue, which is also a scalar. But $.rate[*] is an array or vector. Nifi will send error there and the process will f

Re: problem with putElasticsearch processor

Hey Philippe Couple of things we can do to see what might be going on. 1. Check the port being used by put elastic search. Port 9200 is usually the default http port and 9300 is the default transport port. In put elastic search we need to use the transport port. 2. If the port is correct try to

Re: Approaches to Array in Json with Nifi?

$.rate[*] sorry, forget to tell you your answer. Thad +ThadGuidry

Re: Approaches to Array in Json with Nifi?

Internally it uses Jayway JsonPath (which is a port of Stefan Goessner's JsonPath) There are many JsonPath online tests for you to use. My preferred that does realtime updates as you type is http://www.jsonquerytool.com/ Thad +ThadGuidry On Wed, Apr 6, 2016

problem with putElasticsearch processor

Hello My context nifi 0.6.0 on Ubuntu 14.0 my small use case is : 'sending JSON notifications arriving on http to an ElasticSearch instance ' So I started to develop a DataFlow with 1 handleHttprequest sending in parallel to 2 processors -to a Putfile ( perfect I see the JSON notifs on

Approaches to Array in Json with Nifi?

Let's say we have object and array in Json as {"company":"xyz", "rate":[0.02, 0.03, 0.04]} With Nifi processor EvaluateJsonPath, we may get the individual values of the object and array such as companyValue = $.company firstRateValue = $.rate[0] What would you recommend if I need to capture a

Re: Question about content repository archiving

All, Just to help close the loop on this one the root cause of the issue appears to have been identified and in a really specific case we can end up in an infinite and non-productive loop in the archive cleanup thread. Have the patch staged for the 0.6.1 release we're working and will make sure i

Re: putKafka processor with Avro Format

Awesome product. Thank you Ali On Wed, Apr 6, 2016 at 4:27 PM, Joe Witt wrote: > Ali - I can confirm that all formats are supported for Put/Get Kafka > processors. The interaction with Kafka as far as the messages are > concerned is just 'bytes'. The reason it isn't working well is due to > th

Re: putKafka processor with Avro Format

I see the patch in Jira. You confirm that avro format is supported for putKafka and getKafka processor ? Thanks Ali On Wed, Apr 6, 2016 at 4:14 PM, Joe Witt wrote: > Ali > > It is very possible you're being bitten by a bug we have with > PutKafka. It is fixed in this branch which we're in the r

Re: putKafka processor with Avro Format

Ali - I can confirm that all formats are supported for Put/Get Kafka processors. The interaction with Kafka as far as the messages are concerned is just 'bytes'. The reason it isn't working well is due to the bug I'm mentioning and that is a NiFi bug (not Kafka). It is fixed and we'll have a rel

Re: putKafka processor with Avro Format

Ali It is very possible you're being bitten by a bug we have with PutKafka. It is fixed in this branch which we're in the release motions for right now. https://github.com/apache/nifi/tree/support/nifi-0.6.x Thanks Joe On Wed, Apr 6, 2016 at 10:07 AM, Ali Karray wrote: > I'm using the last ve

Re: putKafka processor with Avro Format

I'm using the last version 0.6 of Nifi and the 0.9 kafka version When I use plain csv files, all is working. Thanks, Ali On Wed, Apr 6, 2016 at 3:23 PM, Bryan Bende wrote: > > Hello, > > What version of Apache NiFi are you using? > > Also, lets take Avro and kafka-avro-console-consumer out of

Re: putKafka processor with Avro Format

Hello, What version of Apache NiFi are you using? Also, lets take Avro and kafka-avro-console-consumer out of the mix for a minute, what happens if you used PutKafka to send some plain text to Kafka, and then tried to consumer from that topic with GetKafka? You mentioned this didn't work, but wha

putKafka processor with Avro Format

Hi, I'm exploring integration possibility of nifi processors with kafka. I use PutKafka to puch avro generated flow from ExecuteSQL. I verify that the topic created in Kafka is alimented with data using kafka-console-consumer bash command. Then I try to use getKafka processor to pull all the Data

UNSUBSCRIBE

Manoj Seshan - Senior Architect Platform Content Technology, Bangalore [cid:image001.gif@01C95541.6801BF70] Voice: +91-9686578756 +91-80-67492572