As Sungwook said, the classpath pointing to the mapr jar is the key for
that error.
MapR has a Spark install that hopefully makes it easier. I don't have the
instructions handy but you can asking their support about it.
-Suren
On Wed, Oct 1, 2014 at 7:18 PM, Matei Zaharia <matei.zaha...@gmail.com>
wrote:
> It should just work in PySpark, the same way it does in Java / Scala apps.
>
> Matei
>
> On Oct 1, 2014, at 4:12 PM, Sungwook Yoon <sy...@maprtech.com> wrote:
>
>
> Yes.. you should use maprfs://
>
> I personally haven't used pyspark, I just used scala shell or standalone
> with MapR.
>
> I think you need to set classpath right, adding jar like
> /opt/mapr/hadoop/hadoop-0.20.2/lib/hadoop-0.20.2-dev-core.jar to the
> classpath
> in the classpath.
>
> Sungwook
>
> On Wed, Oct 1, 2014 at 4:09 PM, Addanki, Santosh Kumar <
> santosh.kumar.adda...@sap.com> wrote:
>
>> Hi
>>
>>
>>
>> We would like to do this in PySpark Environment
>>
>>
>>
>> i.e something like
>>
>>
>>
>> test = sc.textFile("maprfs:///user/root/test") or
>>
>> test = sc.textFile("hdfs:///user/root/test") or
>>
>>
>>
>>
>>
>> Currently when we try
>>
>> test = sc.textFile("maprfs:///user/root/test")
>>
>>
>>
>> It throws the error
>>
>> No File-System for scheme: maprfs
>>
>>
>>
>>
>>
>> Best Regards
>>
>> Santosh
>>
>>
>>
>>
>>
>>
>>
>> *From:* Vladimir Rodionov [mailto:vrodio...@splicemachine.com]
>> *Sent:* Wednesday, October 01, 2014 3:59 PM
>> *To:* Addanki, Santosh Kumar
>> *Cc:* user@spark.apache.org
>> *Subject:* Re: Spark And Mapr
>>
>>
>>
>> There is doc on MapR:
>>
>>
>>
>> http://doc.mapr.com/display/MapR/Accessing+MapR-FS+in+Java+Applications
>>
>>
>>
>> -Vladimir Rodionov
>>
>>
>>
>> On Wed, Oct 1, 2014 at 3:00 PM, Addanki, Santosh Kumar <
>> santosh.kumar.adda...@sap.com> wrote:
>>
>> Hi
>>
>>
>>
>> We were using Horton 2.4.1 as our Hadoop distribution and now switched to
>> MapR
>>
>>
>>
>> Previously to read a text file we would use :
>>
>>
>>
>> test = sc.textFile(\"hdfs://10.48.101.111:8020/user/hdfs/test\
>> <http://10.48.101.111:8020/user/hdfs/test%5C>")"
>>
>>
>>
>>
>>
>> What would be the equivalent of the same for Mapr.
>>
>>
>>
>> Best Regards
>>
>> Santosh
>>
>>
>>
>
>
>
--
SUREN HIRAMAN, VP TECHNOLOGY
Velos
Accelerating Machine Learning
440 NINTH AVENUE, 11TH FLOOR
NEW YORK, NY 10001
O: (917) 525-2466 ext. 105
F: 646.349.4063
E: suren.hiraman@v <suren.hira...@sociocast.com>elos.io
W: www.velos.io
