Questions about the ordering of the FlowFile.

2015-12-10 Thread Paresh Shah
Here’s my use case. We have a application protocol between the start and end processors in a data flow, that expect the flow files to arrive in the order they are generated. For e.g Start Record Flowfile End Record Flowfile. The first processor does the following. 1. Generates and transfer

Re: Regarding Nifi packaging and deployment

2015-12-10 Thread Joe Witt
Shweta, The primary mechanism is flow templates [1]. They do have some important limitations today though that you'll want to understand. First, some properties are sensitive, like passwords, and thus are not included in the templates so you'll have to reenter them when you apply the template in

Regarding Nifi packaging and deployment

2015-12-10 Thread shweta
Hi all, I'm new to Nifi. I have created some sample flows. I want to know how can I package and deploy the same from development environment to testing environment or do I need to recreate the entire data flow again in different environment. Thanks, Shweta -- View this message in context: ht

Re: Asynchronous JMS Consumer for IBM MQ

2015-12-10 Thread Joe Witt
Ian, With run duration the idea is that the processor will be allowed to keep executing for that period of time and the framework will keep giving it the same process session. For the developer this means they get to keep their logic very simple and discrete to a single operation but that the fra

Re: Asynchronous JMS Consumer for IBM MQ

2015-12-10 Thread ianwork
Bryan/Aldrin, Adding yielding into my processor prevent the number of tasks was rapidly increasing. Thanks! Aldrin, I would like to dig a little more into the details. My application is basically set do process logs like logstash. The application is reading and parsing a high volume of logs. The

Re: GetHTTP processor not working

2015-12-10 Thread Joe Percivall
Hello Shweta, I think there is a combination of things going on. The error you're probably seeing first is "Illegal character in fragment at index 239". This is due to the "{" and "}" in your URL. They both need to be URL encoded to %7B and %7D respectively. The URL you should be using is below

Re: Facing Issue while connecting with HDFS

2015-12-10 Thread Bryan Bende
Site-to-Site is a direct connection between NiFi instances/clusters over a socket, so TCP based. There will always have to be at least one local machine involved. When NiFi pulls/receives data from somewhere, it takes that data under control and stores it in the NiFi content repository on disk (co

GetHTTP processor not working

2015-12-10 Thread shweta
I have a url as following http://unify.impetus.co.in/BigData/_layouts/15/start.aspx#/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2FBigData%2FShared%20Documents%2F3%20CU%2FSkillset%20Analyzer%2FResumes&FolderCTID=0x012000D7E70BB8AE01E840A767ECB4D05AC5ED&View={107FFCED-34CD-4354-B0D3-422058A2

Re: Facing Issue while connecting with HDFS

2015-12-10 Thread digvijayp
Hi Bryan, So in edge node approach how data sent in site-to-site ?I mean to say is it using any protocol to transfer it like FTP,SFTP. As you are saying If both clusters can fully talk to each other than you don't need this edge node approach, you could just have a NiFi instance, or cluster, that p