Re: Writing Spark SQL output in Local and HDFS path
It’s since spark version 2.0.0, if you are using under the version, you can
try the below code.
result.write.format("csv").save(path)
--
Hi,
I tried the below code, as
result.write.csv(home/Prasad/)
It is not working,
It says
Error: value csv is not member of org.apache.spark.sql.DataFrameWriter.
Regards
Prasad
On Thu, Jan 19, 2017 at 4:35 PM, smartzjp wrote:
Beacause the reduce number will be not one, so it will out put a fold on the
HDFS, You can use “result.write.csv(foldPath)”.
--
Hi,
Can anyone please let us know how to write the output of the Spark SQL in
Local and HDFS path using Scala code.
Code :-
scala> val result = sqlContext.sql("select empno , name from emp");
scala > result.show();
If I give the command result.show() then It will print the output in the
console.
I need to redirect the output in local file as well as HDFS file.
with the delimiter as "|".
We tried with the below code
result.saveAsTextFile ("home/Prasad/result.txt")
It is not working as expected.
--
--
Prasad. T
--
--
Regards,
RAVI PRASAD. T
Re: Writing Spark SQL output in Local and HDFS path
Hi,
I tried the below code, as
result.write.csv(home/Prasad/)
It is not working,
It says
Error: value csv is not member of org.apache.spark.sql.DataFrameWriter.
Regards
Prasad
On Thu, Jan 19, 2017 at 4:35 PM, smartzjp wrote:
> Beacause the reduce number will be not one, so it will out put a fold on
> the HDFS, You can use “result.write.csv(foldPath)”.
>
>
>
> --
>
> Hi,
> Can anyone please let us know how to write the output of the Spark SQL
> in
> Local and HDFS path using Scala code.
>
> *Code :-*
>
> scala> val result = sqlContext.sql("select empno , name from emp");
> scala > result.show();
>
> If I give the command result.show() then It will print the output in the
> console.
> I need to redirect the output in local file as well as HDFS file.
> with the delimiter as "|".
>
> We tried with the below code
> result.saveAsTextFile ("home/Prasad/result.txt")
> It is not working as expected.
>
>
> --
> --
> Prasad. T
>
--
--
Regards,
RAVI PRASAD. T
Re: Writing Spark SQL output in Local and HDFS path
Beacause the reduce number will be not one, so it will out put a fold on the
HDFS, You can use “result.write.csv(foldPath)”.
--
Hi,
Can anyone please let us know how to write the output of the Spark SQL in
Local and HDFS path using Scala code.
Code :-
scala> val result = sqlContext.sql("select empno , name from emp");
scala > result.show();
If I give the command result.show() then It will print the output in the
console.
I need to redirect the output in local file as well as HDFS file.
with the delimiter as "|".
We tried with the below code
result.saveAsTextFile ("home/Prasad/result.txt")
It is not working as expected.
--
--
Prasad. T
