Ravi, Have you tried without recompiling MapR but following the second approach I listed before?
Kind regards On Mon, Mar 26, 2018 at 1:17 PM, Ravi Papisetti (rpapiset) < rpapi...@cisco.com> wrote: > Thanks Andre. In the absence of FUSE license, I have recompiled NiFi with > mpar dependencies. > > > > I started getting below errors when I run PurHDFS processor (in a flow) > nifi on mac with mapr client libraries setup. However same package (nifi > recompiled with mapr libraries) worked fine on linux version. > > 2018-03-21 22:56:26,806 ERROR [NiFi logging handler] > org.apache.nifi.StdErr 2018-03-21 22:56:26,8003 select failed(-1) error > Invalid argument > > 2018-03-21 22:56:26,806 ERROR [NiFi logging handler] > org.apache.nifi.StdErr 2018-03-21 22:56:26,8003 select failed(-1) error > Invalid argument > > For now I am moving forward with testing from linux machine. Not sure what > is the specific issue with mac. > > I am still trying to work through connecting to maprDb from nifi. > > Thanks, > > Ravi Papisetti > > > > *From: *Andre <andre-li...@fucs.org> > *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>, " > andre-li...@fucs.org" <andre-li...@fucs.org> > *Date: *Sunday, 25 March 2018 at 3:56 PM > *To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Subject: *Re: PutHDFS with mapr > > > > Joey, > > > > Yes. The client must be installed and setup (this is a requirement for the > compiled NiFi as well). > > > > Without the client installed and configured the MapR libraries (java and > native) would be lost in to what ZK connect in order to get information > about the CLDB (their alternative to namenode). > > > > Cheers > > > > On Mon, Mar 26, 2018 at 1:20 AM, Joey Frazee <joey.fra...@icloud.com> > wrote: > > I'm kinda going on memory here because I lost some notes I had about doing > this, but I think the compile against the mapr libs presumes you have also > have the C-based mapr client libs on your machine at compile time and run > time. I skimmed that blog post, albeit very quickly, and didn't see that > explicitly mentioned in there. > > Using the additional jars in PutHDFS would presumably require them too. > > Andre, that's correct isn't it? > > > On Mar 24, 2018, 8:26 AM -0500, Mark Payne <marka...@hotmail.com>, wrote: > > Andre, > > > > I knew this was possible but had no idea how. Thanks for the great > explanation and associates caveats! > > > > -Mark > > > On Mar 24, 2018, at 1:04 AM, Andre <andre-li...@fucs.org> wrote: > > Ravi, > > > > There are two ways of solving this. > > > > One of them (suggested to me MapR representatives) is to deploy MapR's > FUSE client to your NiFi nodes, use the PutFile processor instead of > PutHDFS and let the MapR client pump coordinate the API engagement with > MapR-FS. This is a very clean and robust approach, however it may have > licensing implications as the FUSE client is licensed. (per node if I > recall correctly). > > > > The other one is to use the out of box PutHDFS processor with a bit of > configurations (it works on both Secure and Insecure clusters). > > > > Try this out > > > > Instead of recompiling PutHDFS simply point it to the mapr-client jars and > use a core-site.xml with the following content: > > > > <configuration> > > <property> > > <name>fs.defaultFS</name> > > <value>maprfs:///</value> > > </property> > > </configuration> > > > > Please note maprclients don't play ball with kerberos nicely and you will > be required to use a MapR ticket to access the system. This can be easily > done by: > > > > sudo -u <whatever_user_nifi_uses> "kinit -kt /path/to/your/keytab && > maprlogin kerberos" > > > > Cheers > > > > [1] https://lists.apache.org/thread.html/af9244266e89990618152bb59b5bf9 > 5c9a49dc2428ea3fa0e6aaa682@%3Cusers.nifi.apache.org%3E > > [2] https://cwiki.apache.org/confluence/x/zI5zAw > > > > > > > > On Fri, Mar 23, 2018 at 5:05 AM, Ravi Papisetti (rpapiset) < > rpapi...@cisco.com> wrote: > > Hi, > > > > I have re-compiled nifi with mapr dependencies as per instructions at > http://hariology.com/integrating-mapr-fs-and-apache-nifi/ > > > > Created process flow with ListFile > FetchFile > PutHDFS. As soon as I > start this process group nifi-bootstrap.log is getting filled with > > 2018-03-21 22:56:26,806 ERROR [NiFi logging handler] > org.apache.nifi.StdErr 2018-03-21 22:56:26,8003 select failed(-1) error > Invalid argument > > 2018-03-21 22:56:26,806 ERROR [NiFi logging handler] > org.apache.nifi.StdErr 2018-03-21 22:56:26,8003 select failed(-1) error > Invalid argument > > > > This log grows into GBs in minutes. I had to stop nifi to stop the > flooding. > > > > I found similar issue in petaho forum: https://jira.pentaho.com/ > browse/PDI-16270 > > > > Any one has any thoughts why this error might be coming? > > > > Appreciate any help. > > > > Thanks, > > Ravi Papisetti > > > > >
