RE: sc.textFile() on windows cannot access UNC path

2015-03-12 Thread Wang, Ningjun (LNG-NPV)
: Wednesday, March 11, 2015 2:40 AM To: Wang, Ningjun (LNG-NPV) Cc: java8964; user@spark.apache.org Subject: Re: sc.textFile() on windows cannot access UNC path ​​ I don't have a complete example for your usecase, but you can see a lot of codes showing how to use new APIHadoopFile from herehttps

Re: sc.textFile() on windows cannot access UNC path

2015-03-12 Thread Akhil Das
[mailto:ak...@sigmoidanalytics.com] *Sent:* Wednesday, March 11, 2015 2:40 AM *To:* Wang, Ningjun (LNG-NPV) *Cc:* java8964; user@spark.apache.org *Subject:* Re: sc.textFile() on windows cannot access UNC path ​​ I don't have a complete example for your usecase, but you can see a lot of codes

Re: sc.textFile() on windows cannot access UNC path

2015-03-11 Thread Akhil Das
...@sigmoidanalytics.com] *Sent:* Tuesday, March 10, 2015 9:14 AM *To:* java8964 *Cc:* Wang, Ningjun (LNG-NPV); user@spark.apache.org *Subject:* Re: sc.textFile() on windows cannot access UNC path You can create your own Input Reader (using java.nio.*) and pass it to the sc.newAPIHadoopFile while reading

RE: sc.textFile() on windows cannot access UNC path

2015-03-10 Thread java8964
I think the work around is clear. Using JDK 7, and implement your own saveAsRemoteWinText() using java.nio.path. Yong From: ningjun.w...@lexisnexis.com To: java8...@hotmail.com; user@spark.apache.org Subject: RE: sc.textFile() on windows cannot access UNC path Date: Tue, 10 Mar 2015 03:02:37

RE: sc.textFile() on windows cannot access UNC path

2015-03-09 Thread java8964
This is a Java problem, not really Spark. From this page: http://stackoverflow.com/questions/18520972/converting-java-file-url-to-file-path-platform-independent-including-u You can see that using Java.nio.* on JDK 7, it will fix this issue. But Path class in Hadoop will use java.io.*, instead of

RE: sc.textFile() on windows cannot access UNC path

2015-03-09 Thread Wang, Ningjun (LNG-NPV)
(...)? Ningjun From: java8964 [mailto:java8...@hotmail.com] Sent: Monday, March 09, 2015 5:33 PM To: Wang, Ningjun (LNG-NPV); user@spark.apache.org Subject: RE: sc.textFile() on windows cannot access UNC path This is a Java problem, not really Spark. From this page: http://stackoverflow.com/questions