what's the dump info by jstack?
Yours, Xuefeng Wu 吴雪峰 敬上
On 2015年2月6日, at 上午10:20, Michael Albert m_albert...@yahoo.com.INVALID
wrote:
My apologies for following up my own post, but I thought this might be of
interest.
I terminated the java process corresponding to executor which had
could you find the shuffle files? or the files were deleted by other processes?
Yours, Xuefeng Wu 吴雪峰 敬上
On 2015年2月5日, at 下午11:14, Yifan LI iamyifa...@gmail.com wrote:
Hi,
I am running a heavy memory/cpu overhead graphx application, I think the
memory is sufficient and set RDDs
: user-h...@spark.apache.org
--
~Yours, Xuefeng Wu/吴雪峰 敬上
how about save as object?
Yours, Xuefeng Wu 吴雪峰 敬上
On 2014年12月30日, at 下午9:27, Jason Hong begger3...@gmail.com wrote:
Dear all:)
We're trying to make a graph using large input data and get a subgraph
applied some filter.
Now, we wanna save this graph to HDFS so that we can load later
looks good.
I concern about the foldLeftByKey which looks break the consistence from
foldLeft in RDD and aggregateByKey in PairRDD
Yours, Xuefeng Wu 吴雪峰 敬上
On 2014年12月4日, at 上午7:47, Koert Kuipers ko...@tresata.com wrote:
foldLeftByKey
= for {
(_, ageScores) - takeTop(scores, _.age)
(_, numScores) - takeTop(ageScores, _.num)
} yield {
numScores
}
topScores.size
--
~Yours, Xuefeng Wu/吴雪峰 敬上
hi Debasish，
I found test code in map translate,
would it collect all products too？
+ val sortedProducts = products.toArray.sorted(ord.reverse)
Yours, Xuefeng Wu 吴雪峰 敬上
On 2014年12月2日, at 上午1:33, Debasish Das debasish.da...@gmail.com wrote:
rdd.top collects it on master...
If you want
, 2014 at 5:39 PM, Xuefeng Wu ben...@gmail.com wrote:
scala import scala.collection.GenSeq
scala val seq = GenSeq(This, is, an, example)
scala seq.aggregate(0)(_ + _, _ + _)
res0: String = 0Thisisanexample
scala seq.par.aggregate(0)(_ + _, _ + _)
res1: String = 0This0is0an0example
there is docker script for spark 0.9 in spark git
Yours, Xuefeng Wu 吴雪峰 敬上
On 2014年8月10日, at 下午8:27, 诺铁 noty...@gmail.com wrote:
hi, all,
I am playing with docker, trying to create a spark cluster with docker
containers.
since spark master, worker, driver all need to visit each
