Great, it worked for you.
The reason slider client did not pick up the change in
/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml is probably because
HADOOP_CONF_DIR in slider-env.sh is not set to that path. It is ok to set
that property in slider-client.xml which made the client submit the app,
but remember even the Slider AM would need that value when it comes up. So
you might have to add it to slider-server.xml as well.
I usually set HADOOP_CONF_DIR in slider-env.sh and that takes care of both.
Copying the 3 jars to slider lib dir is also the ok thing to do.
-Gour
On 2/17/17, 1:08 PM, "Nicolás Velásquez O." <gnico...@gmail.com> wrote:
>By the way, I didn't modify the classpath in slider-env.sh. Instead I
>copied three jars from the MapR dist: maprfs-5.2.0-mapr.jar,
>hadoop-common-2.7.0.jar and json-20080701.jar
>
>
>
>On Fri, Feb 17, 2017 at 10:07 PM, Nicolás Velásquez O.
><gnico...@gmail.com>
>wrote:
>
>> Hi Gour!
>> Thanks for pointing me in the right direction!! First I tried to
>>configure
>> "fs.maprfs.impl" on
>>/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml,
>> but that didn't work. I hoped Slider would read that core-site file,
>>but it
>> was not the case. However, since it was slider's client failing and the
>> cluster is MapR, I simply added that property to slider-client.xml and
>>it
>> worked!
>> Ps: I'm still getting some errors, but something unrelated to maprfs.
>> Cheers!
>> Nicolas
>>
>> On Fri, Feb 17, 2017 at 8:39 PM, Gour Saha <gs...@hortonworks.com>
>>wrote:
>>
>>> Can you set the config property "fs.maprfs.impl" to the MapR
>>> implementation class of org.apache.hadoop.fs.FileSystem and try again?
>>>
>>> I think you can set this property in any of the cluster site xmls,
>>> preferably core-site.xml.
>>>
>>> -Gour
>>>
>>> On 2/17/17, 11:14 AM, "jonma...@gmail.com" <jonma...@gmail.com> wrote:
>>>
>>> >My guess is there's actually a need to have the MapR hadoop common jar
>>> >(the
>>> >one I would assume has an updated file system implementation) be
>>> >pre-pended
>>> >to the class path. I don't have an installation in front of me, but
>>>I'd
>>> >look into modifying slider-env or the like to prepend at least that
>>>one
>>> >jar
>>> >to the class path.
>>> >
>>> >On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O.
>>> ><gnico...@gmail.com>
>>> >wrote:
>>> >
>>> >> Hey,
>>> >> It's great to initiate MapR to slider!
>>> >> Unfortunately, following the suggestions did not work (remove the
>>> >>classpath
>>> >> modification on slider-env.sh and copied mapr's jars to slider's lib
>>> >> folder).
>>> >>
>>> >> I'm guessing support for parsing the fs.defaultFS string
>>>("maprfs:///")
>>> >>has
>>> >> to be added to Slider's code. Something similar to
>>> >> https://issues.apache.org/jira/browse/FLINK-1006.
>>> >>
>>> >> Should I just open an incident on Jira?
>>> >>
>>> >> Thanks!
>>> >> Nicolas
>>> >>
>>> >>
>>> >>
>>> >> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi
>>> >><billie.rina...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > I don't think anyone has tried running Slider on MapR yet, so you
>>>are
>>> >> > breaking new ground. I would not try to change CLASSPATH in
>>> >> slider-env.sh.
>>> >> > Slider has a standalone lib directory, meaning that all of its
>>> >>required
>>> >> > jars are in the lib dir (including hadoop jars). It might be the
>>>case
>>> >> that
>>> >> > more jars are needed from the mapr lib directory. I would try
>>>copying
>>> >>all
>>> >> > the jars from the mapr lib directory to the slider lib directory
>>>and
>>> >>see
>>> >> if
>>> >> > that makes a difference. If it works, then you might be able to
>>> remove
>>> >> some
>>> >> > of the jars.
>>> >> >
>>> >> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
>>> >> gnico...@gmail.com>
>>> >> > wrote:
>>> >> >
>>> >> > > Hi,
>>> >> > >
>>> >> > > I'm having problems to run Solr using Apache Slider on MapR.
>>> >> > Specifically,
>>> >> > > when I try to install the slider package it fails because slider
>>> >> doesn't
>>> >> > > have support for maprfs:
>>> >> > > $ slider install-package --replacepkg --name solr --package
>>> >> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
>>> >> > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService -
>>> >>Service
>>> >> > > Slider Client failed in state INITED; cause:
>>>java.io.IOException:
>>> No
>>> >> > > FileSystem for scheme: maprfs
>>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>>> >> FileSystem.java:2644)
>>> >> > > at
>>> >> > >
>>>
>>>>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>>> >> > FileSystem.java:92)
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>>> >> FileSystem.java:2687)
>>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>>> >> > FileSystem.java:2669)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:371)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:170)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>>> >> > > CoreFileSystem.java:82)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>>> >> > > SliderFileSystem.java:38)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>>> >> > > SliderClient.java:498)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>>> >> SliderClient.java:318)
>>> >> > > at
>>> >> > > org.apache.hadoop.service.AbstractService.init(
>>> >> AbstractService.java:163)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>>> >> > > ServiceLauncher.java:182)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>>> >> > > ServiceLauncher.java:475)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>>> >> > > ServiceLauncher.java:403)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>>> >> > > ServiceLauncher.java:630)
>>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>>> >> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher -
>>> >>Exception:
>>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > org.apache.hadoop.service.ServiceStateException:
>>> >>java.io.IOException:
>>> >> No
>>> >> > > FileSystem for scheme: maprfs
>>> >> > > at
>>> >> > > org.apache.hadoop.service.ServiceStateException.convert(
>>> >> > > ServiceStateException.java:59)
>>> >> > > at
>>> >> > > org.apache.hadoop.service.AbstractService.init(
>>> >> AbstractService.java:172)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>>> >> > > ServiceLauncher.java:182)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>>> >> > > ServiceLauncher.java:475)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>>> >> > > ServiceLauncher.java:403)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>>> >> > > ServiceLauncher.java:630)
>>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>>> >> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>>> >> FileSystem.java:2644)
>>> >> > > at
>>> >> > >
>>>
>>>>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>>> >> > FileSystem.java:92)
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>>> >> FileSystem.java:2687)
>>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>>> >> > FileSystem.java:2669)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:371)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:170)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>>> >> > > CoreFileSystem.java:82)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>>> >> > > SliderFileSystem.java:38)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>>> >> > > SliderClient.java:498)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>>> >> SliderClient.java:318)
>>> >> > > at
>>> >> > > org.apache.hadoop.service.AbstractService.init(
>>> >> AbstractService.java:163)
>>> >> > > ... 5 more
>>> >> > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting
>>>with
>>> >> status
>>> >> > 56
>>> >> > >
>>> >> > > I have tried (with no success) to copy the maprfs jar to the
>>>slider
>>> >>lib
>>> >> > > path, and to set the CLASSPATH on slider-env.sh as:
>>> >> > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
>>> >> > > and
>>> >> > > export CLASSPATH=`hadoop classpath`
>>> >> > >
>>> >> > > Has anybody got apache slider to work on top of mapr?
>>> >> > >
>>> >> > > Thanks!
>>> >> > > Nicolas
>>> >> > >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Nicolás Velásquez O.
>>> >> Mobile +41.767189348 <076%20718%2093%2048>
>>> >>
>>> >
>>> >
>>> >
>>> >--
>>> >Sent from some random computer....
>>>
>>>
>>
>>
>> --
>> Nicolás Velásquez O.
>> Mobile +41.767189348 <076%20718%2093%2048>
>>
>
>
>
>--
>Nicolás Velásquez O.
>Mobile +41.767189348
