[Spark Dataset]: How to conduct co-partition join in the new Dataset API in Spark 2.0

2016-12-01 Thread Dale Wang
but failed. Will the Catalyst Optimizer handle the co-partition in its query plan optimization process? Thanks a lot if anyone can provide any clue on the problem :-) Zhaokang(Dale) Wang

Re: GraphX Pregel not update vertex state properly, cause messages loss

2016-11-23 Thread Dale Wang
The problem comes from the inconsistency between graph’s triplet view and vertex view. The message may not be lost but the message is just not sent in sendMsgfunction because sendMsg function gets wrong value of srcAttr! It is not a new bug. I met a similar bug that appeared in version 1.2.1