Re: NiFi processor for Redis

Suddep Also, when you say push/retrieve do you mean publish/subscribe or put/get since Redis is both storage and messaging system. As Joe mentioned we don’t have any at the moment but would be nice to start prioritizing. Cheers Oleg > On Jan 13, 2016, at 12:41 AM, Joe Witt

Re: PutSQL question

Hello Ralph, A common way to replace the contents of a FlowFile with specific text is to use the ReplaceText processor. The default search query will match the entire content and replace it with whatever your replacement value is. So if you set the replacement value (complete with expression

PutSQL question

Hi I want to use the PutSQL processor to execute an insert statement. The use case is straightforward. I need to pull an id attribute from a JSON message and write it to a database. I am using EvaluateJSONPath to grab the id but I’m hung up on how to then pass the SQL statement as the

Re: PutSQL question

Ralph, Hello. An easy way to do this is to use ReplaceText to set the content of the FlowFile. You can set the Replacement Strategy to "Always Replace" and set the Replacement Value to something like: INSERT INTO MY_TABLE (ID) VALUES ('${id}') You could also use SQL parameters (though it's

Re: Data Ingestion forLarge Source Files and Masking

Quick observation for now off latest data: - GC looks pretty good though it is surprising there were any full GCs during that short test - cpu has low utilization - disk has low utilization Can you share your sample input data, processor code, flow as a template? Attaching to a JIRA for example

Re: NIFI connecting to Activemq

This page provides some good examples to consider for your configuration http://activemq.apache.org/uri-protocols.html On Thu, Jan 14, 2016 at 1:17 AM, Christopher Hamm wrote: > Thanks for the update. Do you have suggestion what I should set my processor > to? > > On Jan 9,

Re: NIFI connecting to Activemq

Thanks for the update. Do you have suggestion what I should set my processor to? On Jan 9, 2016 8:25 PM, "Joe Witt" wrote: > Chris, > > Thanks for sending the screenshot. The NullPointerException is > trivially reproduced using your settings. I didn't even need a JMS >

Re: Data Ingestion forLarge Source Files and Masking

Joe, Last time it was below: java.arg.2=-Xms512m java.arg.3=-Xmx512m Now I made as below: java.arg.2=-Xms5120m java.arg.3=-Xmx10240m latest jstate & iostate output are attached. To me it is still slow, no significant improvements. -Obaid On Thu, Jan 14, 2016 at 12:41 PM, Joe Witt

Re: PutSQL question

Mark and Joe – Thanks for the quick replay – that did the trick. Worked great. From: Mark Payne > Reply-To: "users@nifi.apache.org" > Date: Wednesday, January 13,

Re: Data Ingestion forLarge Source Files and Masking

Hi Joe & Others, Thanks for all of your suggestions. Now I am using below code: 1. Buffered reader (I tried to use NLKBufferedReader, but it requires too many libs & Nifi failed to start. I was lost.) 2. Buffered writer 3. Using appending line end instead to concat new line Still no performance

Re: Data Ingestion forLarge Source Files and Masking

Hi Joe, Please find attached jstat & iostat output. So far it seems to me that it is CPU bound. However, your eyes are better tan mine :). -Obaid On Thu, Jan 14, 2016 at 11:51 AM, Joe Witt wrote: > Hello > > Let's narrow in on potential issues. So while this process is

Re: PutDistributedMapCache

Is it possible to build the code for only a particular processor? Just curious if we can build and deploy a particular processor in an existing NiFi environment. On Wed, Jan 13, 2016 at 9:33 PM, sudeep mishra wrote: > Thanks Joe. I will try out the patch. > > On Wed,

Re: Data Ingestion forLarge Source Files and Masking

Obaid, Great so this is helpful info. Iostat output shows both CPU and disk are generally bored and ready for more work. Looking at the gc output though suggests trouble. We see there are 32 samples at 1 second spread each and in that time spent more than 6 seconds of it doing garbage

Re: PutDistributedMapCache

Thank you very much Joe. Can you please let me know how I can use the .patch file? I am using the NiFi via the binaries... Do I need to setup the source code and build the same along with the patch? Thanks & Regards, Sudeep On Wed, Jan 13, 2016 at 9:02 PM, Joe Percivall

Re: PutDistributedMapCache

Thanks Joe. I will try out the patch. On Wed, Jan 13, 2016 at 9:31 PM, Joe Percivall wrote: > You would need to clone the nifi source from github and then apply the > patch using git. > > Here is how to clone a repo: >

Re: PutDistributedMapCache

Hello Sudeep, I put up a patch on the GetDistributedMapCache ticket[1]. Let me know what you think. The PutDistributedMapCache processor and GetDistributedMapCache work with the data as a byte[] so it should be format agnostic. That being said it will be up to you to know what is in there in