Hi Guys,
Are scala lazy values instantiated once per executor, or once per partition?
For example, if I have:
object Something =
val lazy context = create()
def foo(item) = context.doSomething(item)
and I do
someRdd.foreach(Something.foo)
then will context get instantiated once per
A val in an object should be instantiated once per JVM (really,
ClassLoader, but probably won't make a difference here). Therefore I
expect it is going to live effectively as long as the executor, across
partitions but also across jobs.
On Fri, Dec 19, 2014 at 11:21 AM, Ashic Mahtab
It will be instantiated once per VM, which translates to once per executor.
-kr, Gerard.
On Fri, Dec 19, 2014 at 12:21 PM, Ashic Mahtab as...@live.com wrote:
Hi Guys,
Are scala lazy values instantiated once per executor, or once per
partition? For example, if I have:
object Something =
...@gmail.com
Date: Fri, 19 Dec 2014 12:52:23 +0100
Subject: Re: Scala Lazy values and partitions
To: as...@live.com
CC: user@spark.apache.org
It will be instantiated once per VM, which translates to once per executor.
-kr, Gerard.
On Fri, Dec 19, 2014 at 12:21 PM, Ashic Mahtab as...@live.com wrote:
Hi
