So, just before running the job, if you run the HDFS command at a shell prompt: "hdfs dfs -ls hdfs://172.31.42.10:54310/...../weblogReadResult". Does it say the path doesn't exist?
Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com On Mon, Aug 10, 2015 at 7:58 AM, Yasemin Kaya <godo...@gmail.com> wrote: > Thanx Dean, i am giving unique output path and in every time i also delete > the directory before i run the job. > > 2015-08-10 15:30 GMT+03:00 Dean Wampler <deanwamp...@gmail.com>: > >> Following Hadoop conventions, Spark won't overwrite an existing >> directory. You need to provide a unique output path every time you run the >> program, or delete or rename the target directory before you run the job. >> >> dean >> >> Dean Wampler, Ph.D. >> Author: Programming Scala, 2nd Edition >> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) >> Typesafe <http://typesafe.com> >> @deanwampler <http://twitter.com/deanwampler> >> http://polyglotprogramming.com >> >> On Mon, Aug 10, 2015 at 7:08 AM, Yasemin Kaya <godo...@gmail.com> wrote: >> >>> Hi, >>> >>> I have EC2 cluster, and am using spark 1.3, yarn and HDFS . When i >>> submit at local there is no problem , but i run at cluster, saveAsTextFile >>> doesn't work."*It says me User class threw exception: Output directory >>> hdfs://172.31.42.10:54310/...../weblogReadResult >>> <http://172.31.42.10:54310/...../weblogReadResult> already exists*" >>> >>> Is there anyone can help me about this issue ? >>> >>> Best, >>> yasemin >>> >>> >>> >>> -- >>> hiç ender hiç >>> >> >> > > > -- > hiç ender hiç >
