ast 51 from broadcast
>> > at
>> > DAGScheduler.scala:1006
>> >
>> > 16/07/26 00:03:08 INFO DAGScheduler: Submitting 4 missing tasks from
>> > ResultStage 34 (MapPartitionsRDD[78] at map at DecisionTree.scala:642)
>> >
>> > 16/07/26 00:03:08 INFO TaskSchedulerImpl: Adding task set 34.0 with 4
>> > tasks
>> >
>> > 16/07/26 00:03:08 INFO TaskSetManager: Starting task 0.0 in stage 34.0
>> > (TID
>> > 193, x.x.x.x, partition 0,NODE_LOCAL, 1956 bytes)
>> >
>> > 16/07/26 00:03:08 INFO TaskSetManager: Starting task 1.0 in stage 34.0
>> > (TID
>> > 194, x.x.x.x, partition 1,NODE_LOCAL, 1956 bytes)
>> >
>> > 16/07/26 00:03:08 INFO TaskSetManager: Starting task 2.0 in stage 34.0
>> > (TID
>> > 195, x.x.x.x, partition 2,NODE_LOCAL, 1956 bytes)
>> >
>> > 16/07/26 00:03:08 INFO TaskSetManager: Starting task 3.0 in stage 34.0
>> > (TID
>> > 196, x.x.x.x, partition 3,NODE_LOCAL, 1956 bytes)
>> >
>> > 16/07/26 00:03:08 INFO BlockManagerInfo: Added broadcast_51_piece0 in
>> > memory
>> > on x.x.x.x:58784 (size: 444.7 KB, free: 5.0 GB)
>> >
>> > 16/07/26 00:03:08 INFO MapOutputTrackerMasterEndpoint: Asked to send map
>> > output locations for shuffle 13 to x.x.x.x:44434
>> >
>> > 16/07/26 00:03:08 INFO MapOutputTrackerMaster: Size of output statuses
>> > for
>> > shuffle 13 is 180 bytes
>> >
>> > 16/07/26 00:03:08 INFO BlockManagerInfo: Added broadcast_51_piece0 in
>> > memory
>> > on x.x.x.x:46186 (size: 444.7 KB, free: 2.2 GB)
>> >
>> > 16/07/26 00:03:08 INFO MapOutputTrackerMasterEndpoint: Asked to send map
>> > output locations for shuffle 13 to x.x.x.x:47272
>> >
>> > 16/07/26 00:03:08 INFO BlockManagerInfo: Added broadcast_51_piece0 in
>> > memory
>> > on x.x.x.x:50132 (size: 444.7 KB, free: 5.0 GB)
>> >
>> > 16/07/26 00:03:08 INFO MapOutputTrackerMasterEndpoint: Asked to send map
>> > output locations for shuffle 13 to x.x.x.x:46802
>> >
>> > 16/07/26 00:03:10 INFO TaskSetManager: Finished task 1.0 in stage 34.0
>> > (TID
>> > 194) in 2240 ms on x.x.x.x (1/4)
>> >
>> > 16/07/26 00:03:10 INFO TaskSetManager: Finished task 0.0 in stage 34.0
>> > (TID
>> > 193) in 2749 ms on x.x.x.x (2/4)
>> >
>> > 16/07/26 00:03:11 INFO TaskSetManager: Finished task 2.0 in stage 34.0
>> > (TID
>> > 195) in 3818 ms on x.x.x.x (3/4)
>> >
>> > 16/07/26 00:03:11 INFO TaskSetManager: Finished task 3.0 in stage 34.0
>> > (TID
>> > 196) in 3901 ms on x.x.x.x (4/4)
>> >
>> > 16/07/26 00:03:11 INFO DAGScheduler: ResultStage 34 (collectAsMap at
>> > DecisionTree.scala:651) finished in 3.902 s
>> >
>> > 16/07/26 00:03:11 INFO TaskSchedulerImpl: Removed TaskSet 34.0, whose
>> > tasks
>> > have all completed, from pool
>> >
>> > 16/07/26 00:03:11 INFO DAGScheduler: Job 20 finished: collectAsMap at
>> > DecisionTree.scala:651, took 19.556700 s
>> >
>> > Killed
>> >
>> >
>> >
>> >
>
>
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
code works well if with 20
> > trees (lower accuracy, about 68%). When trying the training phase with
> more
> > tree, I set to 200 trees, it returned:
> >
> > "DAGScheduler: Job 20 finished: collectAsMap at DecisionTree.scala:651,
> took
> > 19.556700
rying the training phase with more
> tree, I set to 200 trees, it returned:
>
> "DAGScheduler: Job 20 finished: collectAsMap at DecisionTree.scala:651, took
> 19.556700 s Killed" . There is no WARN or ERROR from console, the task is
> just stopped in the end.
>
> Any idea
with
> more tree, I set to 200 trees, it returned:
>
> "DAGScheduler: Job 20 finished: collectAsMap at DecisionTree.scala:651,
> took 19.556700 s Killed" . There is no WARN or ERROR from console, the
> task is just stopped in the end.
>
> Any idea how to resolve it? Sh
Hi,
spark: 1.6.1
java: java 1.8_u40
I tried random forest training phase, the same code works well if with 20
trees (lower accuracy, about 68%). When trying the training phase with
more tree, I set to 200 trees, it returned:
"DAGScheduler: Job 20 finished: collectAsMap at DecisionTree.scal
