Re: What is the equivalent of forearchRDD in DataFrames?

df.rdd.foreach Thanks Deepak On Oct 26, 2017 18:07, "Noorul Islam Kamal Malmiyoda" wrote: > Hi all, > > I have a Dataframe with 1000 records. I want to split them into 100 > each and post to rest API. > > If it was RDD, I could use something like this > >

Re: What is the equivalent of forearchRDD in DataFrames?

Just hints: Repartition in 10? Get the RDD from the dataframe? What about a forEach row and send every 100? (I just did that actually) jg > On Oct 26, 2017, at 13:37, Noorul Islam Kamal Malmiyoda > wrote: > > Hi all, > > I have a Dataframe with 1000 records. I want to

What is the equivalent of forearchRDD in DataFrames?

Hi all, I have a Dataframe with 1000 records. I want to split them into 100 each and post to rest API. If it was RDD, I could use something like this myRDD.foreachRDD { rdd => rdd.foreachPartition { partition => { This will ensure that code is executed on executors