Yes, though it's a little more complex than that: http://mail-archives.apache.org/mod_mbox/spark-user/201407.mbox/%3CCAPH-c_O9kQO6yJ4khXUVdO=+D4vj=JfG2tP9eqn5RPko=dr...@mail.gmail.com%3E
On Fri, Oct 3, 2014 at 9:58 AM, Mayur Rustagi <mayur.rust...@gmail.com> wrote: > Current approach is to use mappartition, initialize the connection in the > beginning, iterate through the data & close off the connector. > > > Mayur Rustagi > Ph: +1 (760) 203 3257 > http://www.sigmoidanalytics.com > @mayur_rustagi > > > On Fri, Oct 3, 2014 at 10:16 AM, Stephen Boesch <java...@gmail.com> wrote: >> >> >> Consider there is some connection / external resource allocation required >> to be accessed/mutated by each of the rows from within a single worker >> thread. That connection should only be opened/closed before the first row >> is accessed / after the last row is completed. >> >> It is my understanding that there is work presently underway (Reynold Xin >> and others) on defining an external resources API to address this. What is >> the recommended approach in the meanwhile? > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
