By the way, I didn't modify the classpath in slider-env.sh. Instead I
copied three jars from the MapR dist: maprfs-5.2.0-mapr.jar,
hadoop-common-2.7.0.jar and json-20080701.jar
On Fri, Feb 17, 2017 at 10:07 PM, Nicolás Velásquez O. <gnico...@gmail.com>
wrote:
> Hi Gour!
> Thanks for pointing me in the right direction!! First I tried to configure
> "fs.maprfs.impl" on /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml,
> but that didn't work. I hoped Slider would read that core-site file, but it
> was not the case. However, since it was slider's client failing and the
> cluster is MapR, I simply added that property to slider-client.xml and it
> worked!
> Ps: I'm still getting some errors, but something unrelated to maprfs.
> Cheers!
> Nicolas
>
> On Fri, Feb 17, 2017 at 8:39 PM, Gour Saha <gs...@hortonworks.com> wrote:
>
>> Can you set the config property "fs.maprfs.impl" to the MapR
>> implementation class of org.apache.hadoop.fs.FileSystem and try again?
>>
>> I think you can set this property in any of the cluster site xmls,
>> preferably core-site.xml.
>>
>> -Gour
>>
>> On 2/17/17, 11:14 AM, "jonma...@gmail.com" <jonma...@gmail.com> wrote:
>>
>> >My guess is there's actually a need to have the MapR hadoop common jar
>> >(the
>> >one I would assume has an updated file system implementation) be
>> >pre-pended
>> >to the class path. I don't have an installation in front of me, but I'd
>> >look into modifying slider-env or the like to prepend at least that one
>> >jar
>> >to the class path.
>> >
>> >On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O.
>> ><gnico...@gmail.com>
>> >wrote:
>> >
>> >> Hey,
>> >> It's great to initiate MapR to slider!
>> >> Unfortunately, following the suggestions did not work (remove the
>> >>classpath
>> >> modification on slider-env.sh and copied mapr's jars to slider's lib
>> >> folder).
>> >>
>> >> I'm guessing support for parsing the fs.defaultFS string ("maprfs:///")
>> >>has
>> >> to be added to Slider's code. Something similar to
>> >> https://issues.apache.org/jira/browse/FLINK-1006.
>> >>
>> >> Should I just open an incident on Jira?
>> >>
>> >> Thanks!
>> >> Nicolas
>> >>
>> >>
>> >>
>> >> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi
>> >><billie.rina...@gmail.com>
>> >> wrote:
>> >>
>> >> > I don't think anyone has tried running Slider on MapR yet, so you are
>> >> > breaking new ground. I would not try to change CLASSPATH in
>> >> slider-env.sh.
>> >> > Slider has a standalone lib directory, meaning that all of its
>> >>required
>> >> > jars are in the lib dir (including hadoop jars). It might be the case
>> >> that
>> >> > more jars are needed from the mapr lib directory. I would try copying
>> >>all
>> >> > the jars from the mapr lib directory to the slider lib directory and
>> >>see
>> >> if
>> >> > that makes a difference. If it works, then you might be able to
>> remove
>> >> some
>> >> > of the jars.
>> >> >
>> >> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
>> >> gnico...@gmail.com>
>> >> > wrote:
>> >> >
>> >> > > Hi,
>> >> > >
>> >> > > I'm having problems to run Solr using Apache Slider on MapR.
>> >> > Specifically,
>> >> > > when I try to install the slider package it fails because slider
>> >> doesn't
>> >> > > have support for maprfs:
>> >> > > $ slider install-package --replacepkg --name solr --package
>> >> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
>> >> > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService -
>> >>Service
>> >> > > Slider Client failed in state INITED; cause: java.io.IOException:
>> No
>> >> > > FileSystem for scheme: maprfs
>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>> >> FileSystem.java:2644)
>> >> > > at
>> >> > >
>> >>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>> >> > FileSystem.java:92)
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>> >> FileSystem.java:2687)
>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>> >> > FileSystem.java:2669)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:371)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:170)
>> >> > > at
>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>> >> > > CoreFileSystem.java:82)
>> >> > > at
>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>> >> > > SliderFileSystem.java:38)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>> >> > > SliderClient.java:498)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>> >> SliderClient.java:318)
>> >> > > at
>> >> > > org.apache.hadoop.service.AbstractService.init(
>> >> AbstractService.java:163)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>> >> > > ServiceLauncher.java:182)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>> >> > > ServiceLauncher.java:475)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>> >> > > ServiceLauncher.java:403)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>> >> > > ServiceLauncher.java:630)
>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>> >> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher -
>> >>Exception:
>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > org.apache.hadoop.service.ServiceStateException:
>> >>java.io.IOException:
>> >> No
>> >> > > FileSystem for scheme: maprfs
>> >> > > at
>> >> > > org.apache.hadoop.service.ServiceStateException.convert(
>> >> > > ServiceStateException.java:59)
>> >> > > at
>> >> > > org.apache.hadoop.service.AbstractService.init(
>> >> AbstractService.java:172)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>> >> > > ServiceLauncher.java:182)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>> >> > > ServiceLauncher.java:475)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>> >> > > ServiceLauncher.java:403)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>> >> > > ServiceLauncher.java:630)
>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>> >> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>> >> FileSystem.java:2644)
>> >> > > at
>> >> > >
>> >>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>> >> > FileSystem.java:92)
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>> >> FileSystem.java:2687)
>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>> >> > FileSystem.java:2669)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:371)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:170)
>> >> > > at
>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>> >> > > CoreFileSystem.java:82)
>> >> > > at
>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>> >> > > SliderFileSystem.java:38)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>> >> > > SliderClient.java:498)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>> >> SliderClient.java:318)
>> >> > > at
>> >> > > org.apache.hadoop.service.AbstractService.init(
>> >> AbstractService.java:163)
>> >> > > ... 5 more
>> >> > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with
>> >> status
>> >> > 56
>> >> > >
>> >> > > I have tried (with no success) to copy the maprfs jar to the slider
>> >>lib
>> >> > > path, and to set the CLASSPATH on slider-env.sh as:
>> >> > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
>> >> > > and
>> >> > > export CLASSPATH=`hadoop classpath`
>> >> > >
>> >> > > Has anybody got apache slider to work on top of mapr?
>> >> > >
>> >> > > Thanks!
>> >> > > Nicolas
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Nicolás Velásquez O.
>> >> Mobile +41.767189348 <076%20718%2093%2048>
>> >>
>> >
>> >
>> >
>> >--
>> >Sent from some random computer....
>>
>>
>
>
> --
> Nicolás Velásquez O.
> Mobile +41.767189348 <076%20718%2093%2048>
>
--
Nicolás Velásquez O.
Mobile +41.767189348
