Re: ClassNotFoundException for Workers

I've included that in my build file for the fat jar already. libraryDependencies += "com.amazonaws" % "aws-java-sdk" % "1.11.155" libraryDependencies += "com.amazonaws" % "aws-java-sdk-s3" % "1.11.155" libraryDependencies += "com.amazonaws" % "aws-java-sdk-core" % "1.11.155" Not sure if I need

Re: ClassNotFoundException for Workers

Ensure com.amazonaws.services.s3.AmazonS3ClientBuilder in your classpath which include your application jar and attached executor jars. 2017-07-20 6:12 GMT+08:00 Noppanit Charassinvichai : > I have this spark job which is using S3 client in mapPartition. And I get > this

ClassNotFoundException for Workers

I have this spark job which is using S3 client in mapPartition. And I get this error Job aborted due to stage failure: Task 0 in stage 3.0 failed 4 times, most recent failure: Lost task 0.3 in stage 3.0 (TID 74, ip-10-90-78-177.ec2.internal, executor 11): java.lang.NoClassDefFoundError: Could not