Hmmm, I seem to be able to get around this by setting hadoopConf1.setBoolean("fs.s3n.impl.disable.cache", true) in my code. Is there anybody more familiar with Hadoop who can confirm that the filesystem cache would cause this issue?
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/newAPIHadoopFile-uses-AWS-credentials-from-other-threads-tp26081p26082.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org