Re: How to increase the processing speed of the ExtractText and ReplaceText Processor?

2016-10-24 Thread Andy LoPresto
Excellent information here. Thanks Lee and Peter. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 24, 2016, at 6:57 AM, Peter Wicks (pwicks) wrote: > > Prabhu, > > Lee mentioned making sure you have good

RE: Can we configure NiFi to run execute process with specific Kerberos Principal?

2016-10-24 Thread Nathamuni, Ramanujam
I did this stuff with SAS and OBIEE little outside of the application configuration. 1.)On NIFI server and clusters - modify /etc/krb5.conf to match with your Kerberos enabled Hadoop cluster ( you can change this location using the Kerberos configuration variables) 2.)Get the Keytab

Re: Can we configure NiFi to run execute process with specific Kerberos Principal?

2016-10-24 Thread Bryan Bende
Hi Ravi, I'm not very familiar with Sqoop, but from quickly reading their documentation and some other forums/blogs, it seems like the script that NiFi is calling should be doing something like the follow: kinit /bin/sqoop.sh client I would think however you execute the script successfully outs

Can we configure NiFi to run execute process with specific Kerberos Principal?

2016-10-24 Thread Ravi Papisetti (rpapiset)
Hi, We are planning to use "ExecuteProcess" to run a sqoop script wrapped by shell. As part of this we want NiFi to use its service principal in secure mode while submiting executing the script. Otherwise sqoop script is failed to execute saying "Caused by GSSException: No valid credentials pro

RE: How to increase the processing speed of the ExtractText and ReplaceText Processor?

2016-10-24 Thread Peter Wicks (pwicks)
Prabhu, Lee mentioned making sure you have good indexes, but I would caution you on this point. If you have a unique constraint then SQL Server will build an index on this automatically, but I would suggest dropping all other indexes that aren’t related to data integrity. Each time SQL Server

Re: How to increase the processing speed of the ExtractText and ReplaceText Processor?

2016-10-24 Thread Lee Laim
Hello Prabhu, 50 minutes is a good start! Now we have to *determine where the next bottleneck is *-check to see where the flow files are queueing. You can also check the "average task duration" statistic for each processor. I suspect the bottleneck is at PutSQL and will carry this assumption fo

Re: start it ONCE

2016-10-24 Thread Joe Witt
Alessio, We don't presently support cron scheduling for the primary node only processors. This does come up fairly often and a nice PR/contrib came in a while ago from the community https://issues.apache.org/jira/browse/NIFI-401. It is now getting some good traction so hopefully it will be in an

Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups

2016-10-24 Thread Conrad Crampton
No problem – the efforts I have gone through pale into insignificance compared to the sterling work being done to produce such a fine product as NiFi I am sure! ;-) On 24/10/2016, 13:49, "Joe Witt" wrote: Thanks for following up Conrad and sorry for the challenges you went through get

Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups

2016-10-24 Thread Joe Witt
Thanks for following up Conrad and sorry for the challenges you went through getting there. We have to make this important feature easier to setup. Joe On Mon, Oct 24, 2016 at 8:46 AM, Conrad Crampton wrote: > Hi, > > Yes, I think this is the issue. > > I have changed the nifi.remote.input.sock

start it ONCE

2016-10-24 Thread Alessio Palma
Hello All, here is a question which is driving me crazy... I'm using the PutHiveQL processor to create some tables and view into HIVE. This processor need the SQL to execute into a flowfile. I'm using a GenerateFlowFile to build an empty flowfile which is later filled using the ReplaceText process