I'm not sure if this would make a difference, but typically the configuration resources would be the full paths to core-site.xml and hdfs-site.xml. Wondering if using those instead of yarn-site.xml changes anything.
On Monday, June 13, 2016, Ravi Papisetti (rpapiset) <rpapi...@cisco.com> wrote: > Yes, Aldrin. Tried listHDFS, gets the similar error complaining directory > doesn't exist. > > NiFi – 0.6.1 > MapR – 5.1 > > NiFi is local standalone instance. Target cluster is enabled with token > based authentication. I am able to execute "hadoop fs –ls <path>" from cli > on the node with NiFi installed. > > > Thanks, > > Ravi Papisetti > > Technical Leader > > Services Technology Incubation Center > <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/> > > rpapi...@cisco.com <javascript:_e(%7B%7D,'cvml','rpapi...@cisco.com');> > > Phone: +1 512 340 3377 > > > [image: stic-logo-email-blue] > > From: Aldrin Piri <aldrinp...@gmail.com > <javascript:_e(%7B%7D,'cvml','aldrinp...@gmail.com');>> > Reply-To: "users@nifi.apache.org > <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>" < > users@nifi.apache.org > <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>> > Date: Monday, June 13, 2016 at 6:24 PM > To: "users@nifi.apache.org > <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>" < > users@nifi.apache.org > <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>> > Subject: Re: Writing files to MapR File system using putHDFS > > Hi Ravi, > > Could you provide some additional details in terms of both your NiFi > environment and the MapR destination? > > Is your NiFi a single instance or clustered? In the case of the latter, > is security established for your ZooKeeper ensemble? > > Is your target cluster Kerberized? What version are you running? Have > you attempted to use the List/GetHDFS processors? Do they also have errors > in reading? > > Thanks! > --aldrin > > On Mon, Jun 13, 2016 at 5:19 PM, Ravi Papisetti (rpapiset) < > rpapi...@cisco.com <javascript:_e(%7B%7D,'cvml','rpapi...@cisco.com');>> > wrote: > >> Thanks Conrad for your reply. >> >> Yes, I have configured putHDFS with "Remove Owner" and "Renive Group" >> with same values as on HDFS. Also, nifi service is started under the same >> user. >> >> >> >> Thanks, >> >> Ravi Papisetti >> >> Technical Leader >> >> Services Technology Incubation Center >> <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/> >> >> rpapi...@cisco.com <javascript:_e(%7B%7D,'cvml','rpapi...@cisco.com');> >> >> Phone: +1 512 340 3377 >> >> >> [image: stic-logo-email-blue] >> >> From: Conrad Crampton <conrad.cramp...@secdata.com >> <javascript:_e(%7B%7D,'cvml','conrad.cramp...@secdata.com');>> >> Reply-To: "users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>" < >> users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>> >> Date: Monday, June 13, 2016 at 4:01 PM >> To: "users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>" < >> users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>> >> Subject: Re: Writing files to MapR File system using putHDFS >> >> Hi, >> >> Sounds like a permissions problem. Have you set the Remote Owner and >> Remote Groups settings in the processor appropriate for HDFS permissions? >> >> Conrad >> >> >> >> *From: *"Ravi Papisetti (rpapiset)" <rpapi...@cisco.com >> <javascript:_e(%7B%7D,'cvml','rpapi...@cisco.com');>> >> *Reply-To: *"users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>" < >> users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>> >> *Date: *Monday, 13 June 2016 at 21:25 >> *To: *"users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>" < >> users@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','users@nifi.apache.org');>>, " >> d...@nifi.apache.org <javascript:_e(%7B%7D,'cvml','d...@nifi.apache.org');>" >> <d...@nifi.apache.org >> <javascript:_e(%7B%7D,'cvml','d...@nifi.apache.org');>> >> *Subject: *Writing files to MapR File system using putHDFS >> >> >> >> Hi, >> >> >> >> We just started exploring apache nifi for data onboarding into MapR >> distribution. Have configured putHDFS with yarn-site.xml from on local mapr >> client where cluster information is provided, configured the "Directory" >> with mapr fs directory to write the files, configured nifi to run as user >> has permission to write to mapr fs, inspie of that we are getting below >> error while writing the file into given file system path. I am doubting, >> nifi is not talking to the cluster or talking with wrong user, appreciate >> if you some can guide me to troubleshoot this issue or any solutions if we >> are doing something wrong: >> >> Nifi workflow is very simple: GetFile is configure to read from locla >> file system, connected to PutHDFS with yarn-site.xml and directory >> information configured. >> >> 2016-06-13 15:14:36,305 INFO [Timer-Driven Process Thread-2] >> o.apache.nifi.processors.hadoop.PutHDFS >> PutHDFS[id=07abcfaa-fa8d-496b-81f0-b1b770672719] Kerberos relogin >> successful or ticket still valid >> 2016-06-13 15:14:36,324 ERROR [Timer-Driven Process Thread-2] >> o.apache.nifi.processors.hadoop.PutHDFS >> PutHDFS[id=07abcfaa-fa8d-496b-81f0-b1b770672719] Failed to write to HDFS >> due to java.io.IOException: /app/DataAnalyticsFramework/catalog/nifi could >> not be created: java.io.IOException: >> /app/DataAnalyticsFramework/catalog/nifi could not be created >> 2016-06-13 15:14:36,330 ERROR [Timer-Driven Process Thread-2] >> o.apache.nifi.processors.hadoop.PutHDFS >> java.io.IOException: /app/DataAnalyticsFramework/catalog/nifi could not >> be created >> at >> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:238) >> ~[nifi-hdfs-processors-0.6.1.jar:0.6.1] >> at >> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) >> [nifi-api-0.6.1.jar:0.6.1] >> at >> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059) >> [nifi-framework-core-0.6.1.jar:0.6.1] >> at >> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) >> [nifi-framework-core-0.6.1.jar:0.6.1] >> at >> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) >> [nifi-framework-core-0.6.1.jar:0.6.1] >> at >> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123) >> [nifi-framework-core-0.6.1.jar:0.6.1] >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> [na:1.7.0_101] >> at >> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) >> [na:1.7.0_101] >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) >> [na:1.7.0_101] >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >> [na:1.7.0_101] >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> [na:1.7.0_101] >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> [na:1.7.0_101] >> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] >> >> Appreciate any help. >> >> >> >> Thanks, >> >> Ravi Papisetti >> >> Technical Leader >> >> Services Technology Incubation Center >> <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/> >> >> rpapi...@cisco.com <javascript:_e(%7B%7D,'cvml','rpapi...@cisco.com');> >> >> Phone: +1 512 340 3377 >> >> >> >> [image: tic-logo-email-blue] >> >> >> >> ***This email originated outside SecureData*** >> >> Click here <https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==> to >> report this email as spam. >> >> >> SecureData, combating cyber threats >> >> ------------------------------ >> >> The information contained in this message or any of its attachments may >> be privileged and confidential and intended for the exclusive use of the >> intended recipient. If you are not the intended recipient any disclosure, >> reproduction, distribution or other dissemination or use of this >> communications is strictly prohibited. The views expressed in this email >> are those of the individual and not necessarily of SecureData Europe Ltd. >> Any prices quoted are only valid if followed up by a formal written quote. >> >> SecureData Europe Limited. Registered in England & Wales 04365896. >> Registered Address: SecureData House, Hermitage Court, Hermitage Lane, >> Maidstone, Kent, ME16 9NT >> > > -- Sent from Gmail Mobile
