Something like?
val a = myRDD.mapPartitions(p => {
//Do the init
//Perform some operations
//Shut it down?
})
Thanks
Best Regards
On Sun, Dec 28, 2014 at 1:53 AM, Kevin Burton <[email protected]> wrote:
> I have a job where I want to map over all data in a cassandra database.
>
> I’m then selectively sending things to my own external system (ActiveMQ)
> if the item matches criteria.
>
> The problem is that I need to do some init and shutdown. Basically on
> init I need to create ActiveMQ connections and on shutdown I need to close
> them or daemon threads will be left running.
>
> What’s the best way to accomplish this. I could find it after I RTFMd…(but
> perhaps I missed it)
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
