ed as bytes in memory (estimated size 2.1 KB,
> free 21.9 KB)
> [2016-06-15 09:26:01.383] [INFO ] [dispatcher-event-loop-1]
> [BlockManagerInfo] Added broadcast_1_piece0 in memory on node2:44871 (size:
> 2.1 KB, free: 2.4 GB)
> [2016-06-15 09:26:01.384] [INFO ] [dag-scheduler-event-loop] [SparkContext]
> Created broadcast 1 from broadcast at DAGScheduler.scala:1006
> [2016-06-15 09:26:01.385] [INFO ] [dag-scheduler-event-loop] [DAGScheduler]
> Submitting 5 missing tasks from ResultStage 1 (ShuffledRDD[3] at reduceByKey
> at EquityTCAAnalytics.java:87)
> [2016-06-15 09:26:01.386] [INFO ] [dag-scheduler-event-loop]
> [TaskSchedulerImpl] Adding task set 1.0 with 5 tasks
> [2016-06-15 09:26:01.390] [INFO ] [dispatcher-event-loop-4] [TaskSetManager]
> Starting task 0.0 in stage 1.0 (TID 5, node1, partition 0,NODE_LOCAL, 2786
> bytes)
> [2016-06-15 09:26:01.390] [INFO ] [dispatcher-event-loop-4] [TaskSetManager]
> Starting task 1.0 in stage 1.0 (TID 6, node1, partition 1,NODE_LOCAL, 2786
> bytes)
> [2016-06-15 09:26:01.397] [INFO ] [dispatcher-event-loop-4] [TaskSetManager]
> Starting task 2.0 in stage 1.0 (TID 7, node1, partition 2,NODE_LOCAL, 2786
> bytes)
> [2016-06-15 09:26:01.398] [INFO ] [dispatcher-event-loop-4] [TaskSetManager]
> Starting task 3.0 in stage 1.0 (TID 8, node1, partition 3,NODE_LOCAL, 2786
> bytes)
> [2016-06-15 09:26:01.406] [INFO ] [dispatcher-event-loop-4] [TaskSetManager]
> Starting task 4.0 in stage 1.0 (TID 9, node1, partition 4,NODE_LOCAL, 2786
> bytes)
> [2016-06-15 09:26:01.429] [INFO ] [dispatcher-event-loop-4]
> [BlockManagerInfo] Added broadcast_1_piece0 in memory on node1:36512 (size:
> 2.1 KB, free: 511.1 MB)
> [2016-06-15 09:26:01.452] [INFO ] [dispatcher-event-loop-6]
> [MapOutputTrackerMasterEndpoint] Asked to send map output locations for
> shuffle 0 to node1:41122
> [2016-06-15 09:26:01.456] [INFO ] [dispatcher-event-loop-6]
> [MapOutputTrackerMaster] Size of output statuses for shuffle 0 is 161 bytes
> [2016-06-15 09:26:01.526] [INFO ] [task-result-getter-1] [TaskSetManager]
> Finished task 4.0 in stage 1.0 (TID 9) in 128 ms on node1 (1/5)
> [2016-06-15 09:26:01.575] [INFO ] [task-result-getter-3] [TaskSetManager]
> Finished task 2.0 in stage 1.0 (TID 7) in 184 ms on node1 (2/5)
> [2016-06-15 09:26:01.580] [INFO ] [task-result-getter-2] [TaskSetManager]
> Finished task 0.0 in stage 1.0 (TID 5) in 193 ms on node1 (3/5)
> [2016-06-15 09:26:01.589] [INFO ] [task-result-getter-3] [TaskSetManager]
> Finished task 1.0 in stage 1.0 (TID 6) in 199 ms on node1 (4/5)
> [2016-06-15 09:26:01.599] [INFO ] [task-result-getter-2] [TaskSetManager]
> Finished task 3.0 in stage 1.0 (TID 8) in 200 ms on node1 (5/5)
> [2016-06-15 09:26:01.599] [INFO ] [task-result-getter-2] [TaskSchedulerImpl]
> Removed TaskSet 1.0, whose tasks have all completed, from pool
> [2016-06-15 09:26:01.599] [INFO ] [dag-scheduler-event-loop] [DAGScheduler]
> ResultStage 1 (collect at EquityTCAAnalytics.java:88) finished in 0.202 s
> [2016-06-15 09:26:01.612] [INFO ] [main] [DAGScheduler] Job 0 finished:
> collect at EquityTCAAnalytics.java:88, took 32.496470 s
> [2016-06-15 09:26:01.634] [INFO ] [main] [EquityTCAAnalytics] [((2016-06-10
> 13:45:00.0,DA),6944), ((2016-06-10 14:25:00.0,B),5241), ..., ((2016-06-10
> 10:55:00.0,QD),109080), ((2016-06-10 14:55:00.0,A),1300)]
> [2016-06-15 09:26:01.641] [INFO ] [main] [EquityTCAAnalytics] finish
>
> 32.5 s is normal?
> --
> View this message in context: Is that normal spark performance?
> <http://apache-spark-user-list.1001560.n3.nabble.com/Is-that-normal-spark-performance-tp27174.html>
> Sent from the Apache Spark User List mailing list archive
> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com
> <http://nabble.com>.
>
>
