You need to insert per partition per batch. Normally database drivers meant for spark have bulk update feature built in. They take a RDD and do a bulk insert per partition. In case db driver you are using doesn't provide this feature, you can aggregate records per partition and then send out to db by writing your own code.
-- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
