Foreach runs on the executors and so is not able to modify an array list
that is only present on the driver. You should just call collectAsList on
the DataFrame.
On Mon, Apr 24, 2017 at 10:36 AM, Devender Yadav <
devender.ya...@impetus.co.in> wrote:
> Hi All,
>
>
> I am using Spark 1.6.2 and Jav
Hi Franke,
I want to convert DataFrame to JSON String.
Regards,
Devender
From: Jörn Franke
Sent: Monday, April 24, 2017 11:15:08 PM
To: Devender Yadav
Cc: user@spark.apache.org
Subject: Re: Arraylist is empty after JavaRDD.foreach
I am not sure what you try
I am not sure what you try to achieve here. You should never use the arraylist
as you use it here as a global variable (an anti-pattern). Why don't you use
the count function of the dataframe?
> On 24. Apr 2017, at 19:36, Devender Yadav
> wrote:
>
> Hi All,
>
>
> I am using Spark 1.6.2 and
Hi All,
I am using Spark 1.6.2 and Java 7.
Sample json (total 100 records):
{"name":"dev","salary":1,"occupation":"engg","address":"noida"}
{"name":"karthik","salary":2,"occupation":"engg","address":"noida"}
Useful code:
final List> jsonData = new ArrayList<>();
DataFrame df