PutHDFS with mapr

2018-03-22 Thread Ravi Papisetti (rpapiset)
Hi, I have re-compiled nifi with mapr dependencies as per instructions at http://hariology.com/integrating-mapr-fs-and-apache-nifi/ Created process flow with ListFile > FetchFile > PutHDFS. As soon as I start this process group nifi-bootstrap.log is getting filled with 2018-03-21 22:56:26,806

Re: PutHDFS with mapr

2018-03-23 Thread Andre
Ravi, There are two ways of solving this. One of them (suggested to me MapR representatives) is to deploy MapR's FUSE client to your NiFi nodes, use the PutFile processor instead of PutHDFS and let the MapR client pump coordinate the API engagement with MapR-FS. This is a very clean and robust ap

Re: PutHDFS with mapr

2018-03-24 Thread Mark Payne
Andre, I knew this was possible but had no idea how. Thanks for the great explanation and associates caveats! -Mark On Mar 24, 2018, at 1:04 AM, Andre mailto:andre-li...@fucs.org>> wrote: Ravi, There are two ways of solving this. One of them (suggested to me MapR representatives) is to dep

Re: PutHDFS with mapr

2018-03-25 Thread Joey Frazee
I'm kinda going on memory here because I lost some notes I had about doing this, but I think the compile against the mapr libs presumes you have also have the C-based mapr client libs on your machine at compile time and run time. I skimmed that blog post, albeit very quickly, and didn't see that

Re: PutHDFS with mapr

2018-03-25 Thread Andre
Joey, Yes. The client must be installed and setup (this is a requirement for the compiled NiFi as well). Without the client installed and configured the MapR libraries (java and native) would be lost in to what ZK connect in order to get information about the CLDB (their alternative to namenode).

Re: PutHDFS with mapr

2018-03-25 Thread Ravi Papisetti (rpapiset)
ay, 25 March 2018 at 3:56 PM To: "users@nifi.apache.org" Subject: Re: PutHDFS with mapr Joey, Yes. The client must be installed and setup (this is a requirement for the compiled NiFi as well). Without the client installed and configured the MapR libraries (java and native) would be

Re: PutHDFS with mapr

2018-03-25 Thread Andre
ot; , " > andre-li...@fucs.org" > *Date: *Sunday, 25 March 2018 at 3:56 PM > *To: *"users@nifi.apache.org" > *Subject: *Re: PutHDFS with mapr > > > > Joey, > > > > Yes. The client must be installed and setup (this is a requirement for the &g

Re: PutHDFS with mapr

2018-03-25 Thread Ravi Papisetti (rpapiset)
o Employee Cc: "users@nifi.apache.org" , "andre-li...@fucs.org" Subject: Re: PutHDFS with mapr Ravi, Have you tried without recompiling MapR but following the second approach I listed before? Kind regards On Mon, Mar 26, 2018 at 1:17 PM, Ravi Papisetti (rpapiset) mai

Re: PutHDFS with mapr

2018-03-26 Thread Ravi Papisetti (rpapiset)
rch 2018 at 3:56 PM To: "users@nifi.apache.org" Subject: Re: PutHDFS with mapr Joey, Yes. The client must be installed and setup (this is a requirement for the compiled NiFi as well). Without the client installed and configured the MapR libraries (java and native) would be lo

Re: PutHDFS with mapr

2018-03-27 Thread Andre
pProcessor. > resetHDFSResources(AbstractHadoopProcessor.java:281) > > at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor. > abstractOnScheduled(AbstractHadoopProcessor.java:205) > > > > Thanks, > > Ravi Papisetti > > > > > > *From: *Andre

Re: PutHDFS with mapr

2018-03-28 Thread Ravi Papisetti (rpapiset)
path. Thanks, Ravi Papisetti From: Andre Reply-To: "andre-li...@fucs.org" Date: Tuesday, 27 March 2018 at 8:27 PM To: Cisco Employee Cc: "users@nifi.apache.org" , "andre-li...@fucs.org" Subject: Re: PutHDFS with mapr Ravi, I assume the MapR client package is w

Re: PutHDFS with mapr

2018-03-28 Thread Bryan Bende
; Thanks, > > Ravi Papisetti > > > > From: Andre > Reply-To: "andre-li...@fucs.org" > Date: Tuesday, 27 March 2018 at 8:27 PM > To: Cisco Employee > Cc: "users@nifi.apache.org" , "andre-li...@fucs.org" > > > > Subject: Re: PutHDFS w