Re: Worker can not find custom KryoRegistrator

2014-07-02 Thread Baoxu Shi(Dash)
Don’t know why the setting does not appear in the last mail: .set(spark.serializer, org.apache.spark.serializer.KryoSerializer) .set(spark.kryo.registrator, new HDTMKryoRegistrator().getClass.getName) On Jul 2, 2014, at 1:03 PM, dash b...@nd.edu wrote: Hi, I'm using Spark 1.1.0

Re: Question about VD and ED

2014-07-01 Thread Baoxu Shi(Dash)
Hi Bin, VD and ED are ClassTags, you could treat them as placeholder, or template T in C (not 100% clear). You do not need convert graph[String, Double] to Graph[VD,ED]. Check ClassTag’s definition in Scala could help. Best, On Jul 1, 2014, at 4:49 AM, Bin WU bw...@connect.ust.hk wrote: Hi

Re: Alternative to checkpointing and materialization for truncating lineage in high iteration jobs

2014-06-28 Thread Baoxu Shi(Dash)
I’m facing the same situation. It would be great if someone could provide a code snippet as example. On Jun 28, 2014, at 12:36 PM, Nilesh Chakraborty nil...@nileshc.com wrote: Hello, In a thread about java.lang.StackOverflowError when calling count() [1] I saw Tathagata Das share an