df.rdd.foreach
Thanks
Deepak
On Oct 26, 2017 18:07, "Noorul Islam Kamal Malmiyoda"
wrote:
> Hi all,
>
> I have a Dataframe with 1000 records. I want to split them into 100
> each and post to rest API.
>
> If it was RDD, I could use something like this
>
> myRDD.foreachRDD {
> rdd =>
>
Just hints: Repartition in 10? Get the RDD from the dataframe?
What about a forEach row and send every 100? (I just did that actually)
jg
> On Oct 26, 2017, at 13:37, Noorul Islam Kamal Malmiyoda
> wrote:
>
> Hi all,
>
> I have a Dataframe with 1000 records. I want to split them into 100
>
Hi all,
I have a Dataframe with 1000 records. I want to split them into 100
each and post to rest API.
If it was RDD, I could use something like this
myRDD.foreachRDD {
rdd =>
rdd.foreachPartition {
partition => {
This will ensure that code is executed on executors a
