[Spark submit] getting error when use properties file parameter in spark submit

Hi, I am getting below error if I try to use properties file paramater in spark-submit Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.fs.s3a.S3AFileSystem could not be instantiated at java.util.ServiceLoader.fail(ServiceL

Re: [Spark submit] getting error when use properties file parameter in spark submit

Looks like a classpath issue - Caused by: java.lang.ClassNotFoundException: com.amazonaws.services.s3.AmazonS3 Are you using S3 somewhere? Are the required jars in place? Best Regards, Sonal Founder, Nube Technologies Reifier at Strata Hadoop World

Re: [Spark submit] getting error when use properties file parameter in spark submit

Yes I am reading from s3 bucket .. Strangely the error goes off when I remove the properties girl parameter . On Sep 6, 2016 8:35 PM, "Sonal Goyal" wrote: > Looks like a classpath issue - Caused by: java.lang.ClassNotFoundException: > com.amazonaws.services.s3.AmazonS3 > > Are you using S3 some