Thanks Patrick.
I've been testing some 1.2 features, looks good so far.
I have some example code that I think will be helpful for certain MR-style
use cases (secondary sort).
Can I still add that to the 1.2 documentation, or is that frozen at this
point?
-
--
Madhu
I was actually about to post this myself- I have a complex join that could
benefit from something like a GroupComparator vs having to do multiple
grouyBy operations. This is probably the wrong thread for a full discussion
on this but I didn't see a JIRA ticket for this or anything similar- any
BTW, this PR https://github.com/apache/spark/pull/2524 is related to a blocker
level bug,
and this is actually close to be merged (have been reviewed for several rounds)
I would appreciated if anyone can continue the process,
@mateiz
--
Nan Zhu
http://codingcat.me
On Thursday, November
I'm getting a lot of task lost with this build in a large mesos cluster.
Happens with both hash and sort shuffles.
14/11/20 18:08:38 WARN TaskSetManager: Lost task 9.1 in stage 1.0 (TID 897,
i-d4d6553a.inst.aws.airbnb.com): FetchFailed(null, shuffleId=1, mapId=-1,
reduceId=9, message=
You can still send patches for docs until the release goes out -- please do if
you see stuff.
Matei
On Nov 20, 2014, at 6:39 AM, Madhu ma...@madhu.com wrote:
Thanks Patrick.
I've been testing some 1.2 features, looks good so far.
I have some example code that I think will be helpful for
Seeing issues with sort-based shuffle (OOM errors and memory leak):
https://issues.apache.org/jira/browse/SPARK-4515.
Good performance gains for TeraSort as compared to hash (as expected).
Thanks,
Nishkam
On Thu, Nov 20, 2014 at 11:20 AM, Matei Zaharia matei.zaha...@gmail.com
wrote:
You can
