I see, but this is really a. big issue. anyway for me to work around? I try to set the fs.default.name = s3n, but looks like it doesn't work.
I must upgrade to 1.3.0 because I face the package incompatible issue in 1.2.1, and if I must patch something, I rather go with latest version. Regards, Shuai From: Kelly, Jonathan [mailto:jonat...@amazon.com] Sent: Monday, March 16, 2015 2:54 PM To: Shuai Zheng; user@spark.apache.org Subject: Re: sqlContext.parquetFile doesn't work with s3n in version 1.3.0 See https://issues.apache.org/jira/browse/SPARK-6351 ~ Jonathan From: Shuai Zheng <szheng.c...@gmail.com> Date: Monday, March 16, 2015 at 11:46 AM To: "user@spark.apache.org" <user@spark.apache.org> Subject: sqlContext.parquetFile doesn't work with s3n in version 1.3.0 Hi All, I just upgrade the system to use version 1.3.0, but then the sqlContext.parquetFile doesn't work with s3n. I have test the same code with 1.2.1 and it works. A simple test running in spark-shell: val parquetFile = sqlContext.parquetFile("""s3n://****/test/2.parq """) java.lang.IllegalArgumentException: Wrong FS: s3n://****/test/2.parq, expected: file:/// <file:///\\> And same test work with spark-shell under 1.2.1 Regards, Shuai
