Hi all,
I am doing some tests using JavaNetworkWordcount and I have some
questions about the performance machine, my tests' time are
approximately 2 min.
Why does the RAM Memory decrease meaningly? I have done tests with 2, 3
machines and I had gotten the same behavior.
What should I do to get a better performance in this case?
##### Star Test
computer1
total used free shared buffers cached
Mem: 3945 711 3233 0 3 430
-/+ buffers/cache: 276 3668
Swap: 0 0 0
14:42:50 up 73 days, 3:32, 2 users, load average: 0.00, 0.06, 0.21
14/04/04 14:24:56 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1396614314400 in memory on computer1.ant-net:60820 (size: 826.1
KB, free: 542.9 MB)
14/04/04 14:24:56 INFO MemoryStore: ensureFreeSpace(845956) called with
curMem=47278100, maxMem=825439027
14/04/04 14:24:56 INFO MemoryStore: Block input-0-1396614314400 stored
as bytes to memory (size 826.1 KB, free 741.3 MB)
14/04/04 14:24:56 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1396614314400 in memory on computer8.ant-net:49743 (size: 826.1
KB, free: 741.3 MB)
14/04/04 14:24:56 INFO BlockManagerMaster: Updated info of block
input-0-1396614314400
14/04/04 14:24:56 INFO TaskSetManager: Finished TID 272 in 84 ms on
computer1.ant-net (progress: 0/1)
14/04/04 14:24:56 INFO TaskSchedulerImpl: Remove TaskSet 43.0 from pool
14/04/04 14:24:56 INFO DAGScheduler: Completed ResultTask(43, 0)
14/04/04 14:24:56 INFO DAGScheduler: Stage 43 (take at
DStream.scala:594) finished in 0.088 s
14/04/04 14:24:56 INFO SparkContext: Job finished: take at
DStream.scala:594, took 1.872875734 s
-------------------------------------------
Time: 1396614289000 ms
-------------------------------------------
(Santiago,1)
(liveliness,1)
(Sun,1)
(reapers,1)
(offer,3)
(BARBER,3)
(shrewdness,1)
(truism,1)
(hits,1)
(merchant,1)
##### End Test
computer1
total used free shared buffers cached
Mem: 3945 2209 1735 0 5 773
-/+ buffers/cache: 1430 2514
Swap: 0 0 0
14:46:05 up 73 days, 3:35, 2 users, load average: 2.69, 1.07, 0.55
14/04/04 14:26:57 INFO TaskSetManager: Starting task 183.0:0 as TID 696
on executor 0: computer1.ant-net (PROCESS_LOCAL)
14/04/04 14:26:57 INFO TaskSetManager: Serialized task 183.0:0 as 1981
bytes in 0 ms
14/04/04 14:26:57 INFO MapOutputTrackerMasterActor: Asked to send map
output locations for shuffle 81 to sp...@computer1.ant-net:44817
14/04/04 14:26:57 INFO MapOutputTrackerMaster: Size of output statuses
for shuffle 81 is 212 bytes
14/04/04 14:26:57 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1396614336600 on disk on computer1.ant-net:60820 (size: 1441.7 KB)
14/04/04 14:26:57 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1396614435200 in memory on computer1.ant-net:60820 (size: 1295.7
KB, free: 589.3 KB)
14/04/04 14:26:57 INFO TaskSetManager: Finished TID 696 in 56 ms on
computer1.ant-net (progress: 0/1)
14/04/04 14:26:57 INFO TaskSchedulerImpl: Remove TaskSet 183.0 from pool
14/04/04 14:26:57 INFO DAGScheduler: Completed ResultTask(183, 0)
14/04/04 14:26:57 INFO DAGScheduler: Stage 183 (take at
DStream.scala:594) finished in 0.057 s
14/04/04 14:26:57 INFO SparkContext: Job finished: take at
DStream.scala:594, took 1.575268894 s
-------------------------------------------
Time: 1396614359000 ms
-------------------------------------------
(hapless,9)
(reapers,8)
(amazed,113)
(feebleness,7)
(offer,148)
(rabble,27)
(exchanging,7)
(merchant,20)
(incentives,2)
(quarrel,48)
...
Thanks Guys
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
