HandleHttpRequest

Hello all, Is it considered a best practice to leave the thread count on HanddleHttpRequest to 1? Assuming the JETTY server is handling requests? are we making things worst by increasing that? We are moving towards using HanddleHttpRequest versus ListenHttp but we are struggling with performance

Re: NiFi Processor Lifecycle Diagram

Thanks for links. I've whiteboarded some stuff, but nothing serious. I'll be sure to share if I get something going. Ryan On Mon, Oct 30, 2017 at 7:18 PM, Andy LoPresto wrote: > Hi Ryan, > > I am not aware of a diagram, but the best resource for the component > lifecycle is the Developer G

Re: NiFi Processor Lifecycle Diagram

Hi, There is actually a JIRA for that [1]. Feel free to add attachments to this JIRA or submit a PR with your work. I'm sure someone will review and get it merged into our documentation! [1] https://issues.apache.org/jira/browse/NIFI-1345 Thanks 2017-11-02 15:51 GMT+01:00 Ryan H : > Thanks for

What would be a good way to build NoSQL entity from SQL data?

Hi fellows, I'm planning a flow to feed data from a relational database into elasticsearch and would like to ask for some advice. The database has an entity Person with 1:N relation to Email and Phone (email and phone are objects, not just scalar values). The flow needs to feed ES with JSON obje

Re: NIFI:how to transfer certain amount of flowfile

Hi Sally, Glad you have found NiFi useful. From the way you describe it, it sounds almost like you are trying to create a stateful flow. This can be tricky. I recommend giving MattyB's script tester a shot to see if you can develop your script in isolation before deploying it to a NiFi instanc

SplitAvro fails to split data

I have a NiFi flow that queries a relational Table that has a latestUpdate date timestamp column  and forwards data in that table that was  inserted/modified in the last 10 min sends it to a JMS queue. The flow is something like thisExecuteSQL --> splitAvro --> convertAvroToJson --> some more pr

Issue with Executescript

Hi, I'm using the executescript process to generate some fake data using "Faker" package and replacing it in the original data.I have attached the script for your reference. import java.io from org.apache.commons.io import IOUtils from java.nio.charset import StandardCharsets from org.apache.nifi

Reading flowfile in a stream callback

In python, I can use the requests library to post content something like htis: import requests url="https://abc.test.org"; files={'file':open('/somedir/myfile.txt','rb')} r = requests.post(url,files=files) If I am in a python stream callback, how can I read the flowfile payload in the same way th

Re: Reading flowfile in a stream callback

James, The Python API should be the same as the Java FlowFile.java interface [1]. Matt Burgess’ blog has a good post about using Jython to do flowfile content manipulation. Something like: flowFile = session.get() if (flowFile != None): flowFile = session.write(flowFile,PyStreamCallback())

Nifi:can't import getFileSystem in customProcessor?

I want to use getFileSytem() for pulling in any file that is newer than the timestamp that we have but I can't import it (I mean getFileSytem()) I have this code: final FileSystem hdfs = getFileSystem(); final boolean recursive = context.getProperty(RECURSE_SUBDIRS).asBoolean(); final Set status

NiFi Slowness after thousand processors

Hello All, We are currently running NiFi 1.3.0 on Linux(RHEL) box (standalone instance), we are facing some strange issues with this instance. Whenever the total processor count exceeds 1500 the whole instance is slowing down (I think there is no limit on the number of processors an instance ca