Re: writing to maprfs?

[Akanksha Sharma B]

Hi,


Answering my own question 😊


writing to maprfs worked after I added following property to core-site.xml:-


    <property>
        <name>fs.maprfs.impl</name>
        <value>com.mapr.fs.MapRFileSystem</value>
    </property>



Regards,

Akanksha


________________________________
From: Akanksha Sharma B
Sent: Thursday, May 24, 2018 9:24:25 AM
To: user@beam.apache.org
Subject: writing to maprfs?


Hi All,


I have a pipeline; it writes to hdfs, using "org.apache.beam.sdk.io.hdfs" 
package.

I was hoping that it will work with maprfs as well. However I have been 
debugging for some days, with no success.


I do not provide hdfsConfiguration from command line, and instead use the 
configuration file /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml 
(please find it attached with the mail)

However I get the stack trace ("No FileSystem for scheme: maprfs"). Please find 
the detailed trace attached. I made sure that I have 
/opt/mapr/lib/maprfs-5.2.2-mapr.jar on classpath. I use spark submit to run the 
pipeline.


If you have already tried writing to maprfs from beam, ccould you please help?


Regards,

Akanksha