Ravi,
You likely need to build the Hbase connectors with the MapR libraries as
well.
Can you please change your ~/.m2/settings.xml so that you have:
<profile>
<id>mapr</id>
<repositories>
<repository>
<id>mapr-repo</id>
<name>MapR Repository</name>
<url>http://repository.mapr.com/maven/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
</profile>
and then compile nifi (without Sumo's patch) using:
mvn <whatever you usually use> -Pmapr
-Dhadoop.version=2.7.0-mapr-1602 (assuming you are running 5.1 - otherwise
check the docs for the appropriate version)
This should cause hbase connector to be compiled with the MapR libraries as
dependencies.
Cheers
On Thu, Jun 16, 2016 at 9:32 AM, Ravi Papisetti (rpapiset) <
rpapi...@cisco.com> wrote:
> Thanks Sumo. We are able to use 0.6.1.nar file to connect with maps 5.1.
>
> Can we use HBase_1_1_2_ClientService to connect with Hbase 0.98.12? Also,
> again here we are using mapr cluster (mapr db). Connection fails with below
> error. Any thoughts?
>
> 2016-06-15 18:22:33,131 ERROR [StandardProcessScheduler Thread-8]
> o.a.n.c.s.StandardControllerServiceNode Failed to invoke @OnEnabled method
> of HBase_1_1_2_ClientService[id=076c05f1-a2b9-4e6a-803b-c5eb76da1c6d] due
> to org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
>
> Wed Jun 15 18:22:33 CDT 2016,
> RpcRetryingCaller{globalStartTime=1466032935495, pause=100, retries=1},
> org.apache.hadoop.hbase.MasterNotRunningException:
> org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
> ZooKeeper: KeeperErrorCode = ConnectionLoss for /hbase
>
>
> Thanks,
>
> Ravi Papisetti
>
> Technical Leader
>
> Services Technology Incubation Center
> <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/>
>
> rpapi...@cisco.com
>
> Phone: +1 512 340 3377
>
>
> [image: stic-logo-email-blue]
>
> From: Sumanth Chinthagunta <xmlk...@gmail.com>
> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
> Date: Wednesday, June 15, 2016 at 5:16 PM
> To: "users@nifi.apache.org" <users@nifi.apache.org>
> Subject: Re: Writing files to MapR File system using putHDFS
>
> Just want to conform that:
> After fresh MapR Client (Secure) installation on NiFi host, I was able to
> use putHDFS from NiFi *without* any core-site.xml changes I described in
> mapr-client.md
> we are using custom MapR nifi-hadoop-libraries-nar-0.6.1.nar
> <https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/releases/download/0.6.2/nifi-hadoop-libraries-nar-0.6.1.nar>
> bundle
> and /opt/mapr/conf/mapr-clusters.conf now.
> If Matt’s PR is released, ideally we don’t have to do any
> custom configuration to make NiFi work with MapR
>
> PS: my MapR env is 4.0.2
> -Sumo
>
> On Jun 14, 2016, at 3:55 PM, Andre <andre-li...@fucs.org> wrote:
>
> Ravi,
>
> "1.
> https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/blob/master/mapr-client.md
> –
> has a step to configure fs.defaultFS, should I configure the cluster
> resource manage here? Or should I get maprfs url from our IT who is
> supporting the cluster?"
>
>
> Depends on your configuration, but generally the fs.defaultFS can be
> configured just by using your own settings (and pointing the NiFi processor
> to your settings).
>
> Ideally you should be connecting to a cluster defined under
> /opt/mapr/conf/mapr-clusters.conf
> (assuming you haven't played with client configs)
>
> Also have in mind that mapr-client.md file seems to be focused on
> insecure MapR clusters. Are you using insecure (terrible choice), MapR
> Hybrid security (not ideal IMHO) or Kerberos (recommended IMHO)?
>
> Cheers
>
> On Wed, Jun 15, 2016 at 3:36 AM, Ravi Papisetti (rpapiset) <
> rpapi...@cisco.com> wrote:
>
>> Hi,
>>
>> I have configure the way it is mentioned in below e-mail, still no
>> luck:-(.
>>
>> I have two questions:
>> 1.
>> https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/blob/master/mapr-client.md
>> –
>> has a step to configure fs.defaultFS, should I configure the cluster
>> resource manage here? Or should I get maprfs url from our IT who is
>> supporting the cluster?
>> 2. Matt: What is MaprPR? Can you please elaborate
>>
>> Appreciate all your responses.
>>
>> Thanks,
>> Ravi Papisetti
>> Technical Leader
>> Services Technology Incubation Center
>> <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/>
>> rpapi...@cisco.com
>> Phone: +1 512 340 3377
>>
>> <0B65E9AA-485A-49EE-9FB4-8600F4D55880[28].png>
>>
>> From: Matt Burgess <mattyb...@gmail.com>
>> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
>> Date: Monday, June 13, 2016 at 8:26 PM
>> To: "users@nifi.apache.org" <users@nifi.apache.org>
>> Subject: Re: Writing files to MapR File system using putHDFS
>>
>> Sumo,
>>
>> I'll try the MapR PR with your additional settings below. If they work,
>> they'll need to be added to the doc (or ideally, the profile if possible).
>> That's what I suspected had been missing but haven't had a chance to try
>> yet, will do that shortly :)
>>
>> Thanks,
>> Matt
>>
>> On Jun 13, 2016, at 9:17 PM, Sumanth Chinthagunta <xmlk...@gmail.com>
>> wrote:
>>
>>
>> I had been using custom build nifi-hadoop-libraries-nar-0.6.1.nar that
>> worked with MapR 4.02
>> make sure you add java.security.auth.login.config and follow mapR client
>> setup on the NiFi server (
>> https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/blob/master/mapr-client.md)
>>
>>
>> $NIFI_HOME/conf/bootstrap.conf
>>
>>
>> java.arg.15=-Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
>> I just build *Nar* with MapR 2.7.0-mapr-1602 libs. I haven’t tested with
>> MapR 5.1 but you can try and let us know.
>>
>> Hadoop bundle for NiFi v0.6.1 and MapR 2.7.0-mapr-1602
>> https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/releases
>>
>> -Sumo
>>
>>
>> On Jun 13, 2016, at 5:57 PM, Bryan Bende <bbe...@gmail.com> wrote:
>>
>> I'm not sure if this would make a difference, but typically the
>> configuration resources would be the full paths to core-site.xml and
>> hdfs-site.xml. Wondering if using those instead of yarn-site.xml changes
>> anything.
>>
>> On Monday, June 13, 2016, Ravi Papisetti (rpapiset) <rpapi...@cisco.com>
>> wrote:
>>
>>> Yes, Aldrin. Tried listHDFS, gets the similar error complaining
>>> directory doesn't exist.
>>>
>>> NiFi – 0.6.1
>>> MapR – 5.1
>>>
>>> NiFi is local standalone instance. Target cluster is enabled with token
>>> based authentication. I am able to execute "hadoop fs –ls <path>" from cli
>>> on the node with NiFi installed.
>>>
>>> Thanks,
>>> Ravi Papisetti
>>> Technical Leader
>>> Services Technology Incubation Center
>>> <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/>
>>> rpapi...@cisco.com
>>> Phone: +1 512 340 3377
>>>
>>> <0B65E9AA-485A-49EE-9FB4-8600F4D55880[22].png>
>>>
>>> From: Aldrin Piri <aldrinp...@gmail.com>
>>> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
>>> Date: Monday, June 13, 2016 at 6:24 PM
>>> To: "users@nifi.apache.org" <users@nifi.apache.org>
>>> Subject: Re: Writing files to MapR File system using putHDFS
>>>
>>> Hi Ravi,
>>>
>>> Could you provide some additional details in terms of both your NiFi
>>> environment and the MapR destination?
>>>
>>> Is your NiFi a single instance or clustered? In the case of the latter,
>>> is security established for your ZooKeeper ensemble?
>>>
>>> Is your target cluster Kerberized? What version are you running? Have
>>> you attempted to use the List/GetHDFS processors? Do they also have errors
>>> in reading?
>>>
>>> Thanks!
>>> --aldrin
>>>
>>> On Mon, Jun 13, 2016 at 5:19 PM, Ravi Papisetti (rpapiset) <
>>> rpapi...@cisco.com> wrote:
>>>
>>>> Thanks Conrad for your reply.
>>>>
>>>> Yes, I have configured putHDFS with "Remove Owner" and "Renive Group"
>>>> with same values as on HDFS. Also, nifi service is started under the same
>>>> user.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>> Ravi Papisetti
>>>> Technical Leader
>>>> Services Technology Incubation Center
>>>> <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/>
>>>> rpapi...@cisco.com
>>>> Phone: +1 512 340 3377
>>>>
>>>> <0B65E9AA-485A-49EE-9FB4-8600F4D55880[21].png>
>>>>
>>>> From: Conrad Crampton <conrad.cramp...@secdata.com>
>>>> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
>>>> Date: Monday, June 13, 2016 at 4:01 PM
>>>> To: "users@nifi.apache.org" <users@nifi.apache.org>
>>>> Subject: Re: Writing files to MapR File system using putHDFS
>>>>
>>>> Hi,
>>>>
>>>> Sounds like a permissions problem. Have you set the Remote Owner and
>>>> Remote Groups settings in the processor appropriate for HDFS permissions?
>>>>
>>>> Conrad
>>>>
>>>>
>>>>
>>>> *From: *"Ravi Papisetti (rpapiset)" <rpapi...@cisco.com>
>>>> *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>
>>>> *Date: *Monday, 13 June 2016 at 21:25
>>>> *To: *"users@nifi.apache.org" <users@nifi.apache.org>, "
>>>> d...@nifi.apache.org" <d...@nifi.apache.org>
>>>> *Subject: *Writing files to MapR File system using putHDFS
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>>
>>>> We just started exploring apache nifi for data onboarding into MapR
>>>> distribution. Have configured putHDFS with yarn-site.xml from on local mapr
>>>> client where cluster information is provided, configured the "Directory"
>>>> with mapr fs directory to write the files, configured nifi to run as user
>>>> has permission to write to mapr fs, inspie of that we are getting below
>>>> error while writing the file into given file system path. I am doubting,
>>>> nifi is not talking to the cluster or talking with wrong user, appreciate
>>>> if you some can guide me to troubleshoot this issue or any solutions if we
>>>> are doing something wrong:
>>>>
>>>> Nifi workflow is very simple: GetFile is configure to read from locla
>>>> file system, connected to PutHDFS with yarn-site.xml and directory
>>>> information configured.
>>>>
>>>> 2016-06-13 15:14:36,305 INFO [Timer-Driven Process Thread-2]
>>>> o.apache.nifi.processors.hadoop.PutHDFS
>>>> PutHDFS[id=07abcfaa-fa8d-496b-81f0-b1b770672719] Kerberos relogin
>>>> successful or ticket still valid
>>>> 2016-06-13 15:14:36,324 ERROR [Timer-Driven Process Thread-2]
>>>> o.apache.nifi.processors.hadoop.PutHDFS
>>>> PutHDFS[id=07abcfaa-fa8d-496b-81f0-b1b770672719] Failed to write to HDFS
>>>> due to java.io.IOException: /app/DataAnalyticsFramework/catalog/nifi could
>>>> not be created: java.io.IOException:
>>>> /app/DataAnalyticsFramework/catalog/nifi could not be created
>>>> 2016-06-13 15:14:36,330 ERROR [Timer-Driven Process Thread-2]
>>>> o.apache.nifi.processors.hadoop.PutHDFS
>>>> java.io.IOException: /app/DataAnalyticsFramework/catalog/nifi could not
>>>> be created
>>>> at
>>>> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:238)
>>>> ~[nifi-hdfs-processors-0.6.1.jar:0.6.1]
>>>> at
>>>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>>>> [nifi-api-0.6.1.jar:0.6.1]
>>>> at
>>>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059)
>>>> [nifi-framework-core-0.6.1.jar:0.6.1]
>>>> at
>>>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>>>> [nifi-framework-core-0.6.1.jar:0.6.1]
>>>> at
>>>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>>>> [nifi-framework-core-0.6.1.jar:0.6.1]
>>>> at
>>>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123)
>>>> [nifi-framework-core-0.6.1.jar:0.6.1]
>>>> at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>> [na:1.7.0_101]
>>>> at
>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>>> [na:1.7.0_101]
>>>> at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>>> [na:1.7.0_101]
>>>> at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>> [na:1.7.0_101]
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> [na:1.7.0_101]
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> [na:1.7.0_101]
>>>> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101]
>>>>
>>>> Appreciate any help.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Ravi Papisetti
>>>>
>>>> Technical Leader
>>>>
>>>> Services Technology Incubation Center
>>>> <http://wwwin.cisco.com/CustAdv/ts/cstg/stic/>
>>>>
>>>> rpapi...@cisco.com
>>>>
>>>> Phone: +1 512 340 3377
>>>>
>>>>
>>>>
>>>> <image001.png>
>>>>
>>>>
>>>>
>>>> ***This email originated outside SecureData***
>>>>
>>>> Click here <https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==>
>>>> to report this email as spam.
>>>>
>>>>
>>>> SecureData, combating cyber threats
>>>>
>>>> ------------------------------
>>>>
>>>> The information contained in this message or any of its attachments may
>>>> be privileged and confidential and intended for the exclusive use of the
>>>> intended recipient. If you are not the intended recipient any disclosure,
>>>> reproduction, distribution or other dissemination or use of this
>>>> communications is strictly prohibited. The views expressed in this email
>>>> are those of the individual and not necessarily of SecureData Europe Ltd.
>>>> Any prices quoted are only valid if followed up by a formal written quote.
>>>>
>>>> SecureData Europe Limited. Registered in England & Wales 04365896.
>>>> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
>>>> Maidstone, Kent, ME16 9NT
>>>>
>>>
>>>
>>
>> --
>> Sent from Gmail Mobile
>>
>>
>>
>
>
