Re: Spark - Hadoop custom filesystem service loading

2019-03-23 Thread Felix Cheung
Hmm thanks. Do you have a proposed solution? From: Jhon Anderson Cardenas Diaz Sent: Monday, March 18, 2019 1:24 PM To: user Subject: Spark - Hadoop custom filesystem service loading Hi everyone, On spark 2.2.0, if you wanted to create a custom file system

Spark - Hadoop custom filesystem service loading

2019-03-18 Thread Jhon Anderson Cardenas Diaz
Hi everyone, On spark 2.2.0, if you wanted to create a custom file system implementation, you just created an extension of org.apache.hadoop.fs.FileSystem and put the canonical name of the custom class on the file src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem. Once you