I see. Agree that lazy eval is not suitable for proper setup and teardown. We also abandoned it due to inherent incompatibility between implicit and lazy. It was fun to come up this trick though.
Jianshi On Tue, Nov 18, 2014 at 10:28 AM, Tobias Pfeiffer <t...@preferred.jp> wrote: > Hi, > > On Fri, Nov 14, 2014 at 2:49 PM, Jianshi Huang <jianshi.hu...@gmail.com> > wrote: > >> Ok, then we need another trick. >> >> let's have an *implicit lazy var connection/context* around our code. >> And setup() will trigger the eval and initialization. >> > > Due to lazy evaluation, I think having setup/teardown is a bit tricky. In > particular teardown, because it is not easy to execute code after all > computation is done. You can check > http://apache-spark-user-list.1001560.n3.nabble.com/Keep-state-inside-map-function-tp10968p11009.html > for an example of what worked for me. > > Tobias > > -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/
