Re: Brodcast Variable updated from one transformation and used from another

2015-02-25 Thread Yiannis Gkoufas
: *Yiannis Gkoufas [johngou...@gmail.com] *Sent: *Tuesday, February 24, 2015 12:12 PM Eastern Standard Time *To: *user@spark.apache.org *Subject: *Brodcast Variable updated from one transformation and used from another Hi all, I am trying to do the following. val myObject = new MyObject

Re: Brodcast Variable updated from one transformation and used from another

2015-02-25 Thread Imran Rashid
. Sent with Good (www.good.com) -Original Message- *From: *Yiannis Gkoufas [johngou...@gmail.com] *Sent: *Tuesday, February 24, 2015 12:12 PM Eastern Standard Time *To: *user@spark.apache.org *Subject: *Brodcast Variable updated from one transformation and used from another Hi all

Re: Brodcast Variable updated from one transformation and used from another

2015-02-24 Thread Yiannis Gkoufas
: *Tuesday, February 24, 2015 12:12 PM Eastern Standard Time *To: *user@spark.apache.org *Subject: *Brodcast Variable updated from one transformation and used from another Hi all, I am trying to do the following. val myObject = new MyObject(); val myObjectBroadcasted = sc.broadcast(myObject

RE: Brodcast Variable updated from one transformation and used from another

2015-02-24 Thread Ganelin, Ilya
with Good (www.good.com) -Original Message- From: Yiannis Gkoufas [johngou...@gmail.commailto:johngou...@gmail.com] Sent: Tuesday, February 24, 2015 12:12 PM Eastern Standard Time To: user@spark.apache.org Subject: Brodcast Variable updated from one transformation and used from another Hi all

Brodcast Variable updated from one transformation and used from another

2015-02-24 Thread Yiannis Gkoufas
Hi all, I am trying to do the following. val myObject = new MyObject(); val myObjectBroadcasted = sc.broadcast(myObject); val rdd1 = sc.textFile(/file1).map(e = { myObject.insert(e._1); (e._1,1) }); rdd.cache.count(); //to make sure it is transformed. val rdd2 = sc.textFile(/file2).map(e = {