Any suggestions on this please On Wed, Jun 15, 2016 at 10:42 PM, VG <vlin...@gmail.com> wrote:
> I have a very simple driver which loads a textFile and filters a >> sub-string from each line in the textfile. >> When the collect action is executed , I am getting an exception. (The >> file is only 90 MB - so I am confused what is going on..) I am running on a >> local standalone cluster >> >> 16/06/15 19:45:22 INFO BlockManagerInfo: Removed broadcast_2_piece0 on >> 192.168.56.1:56413 in memory (size: 2.5 KB, free: 2.4 GB) >> 16/06/15 19:45:22 INFO BlockManagerInfo: Removed broadcast_1_piece0 on >> 192.168.56.1:56413 in memory (size: 1900.0 B, free: 2.4 GB) >> 16/06/15 19:45:22 INFO BlockManagerInfo: Added rdd_2_1 on disk on >> 192.168.56.1:56413 (size: 2.7 MB) >> 16/06/15 19:45:22 INFO MemoryStore: Block taskresult_7 stored as bytes in >> memory (estimated size 2.7 MB, free 2.4 GB) >> 16/06/15 19:45:22 INFO BlockManagerInfo: Added taskresult_7 in memory on >> 192.168.56.1:56413 (size: 2.7 MB, free: 2.4 GB) >> 16/06/15 19:45:22 INFO Executor: Finished task 1.0 in stage 2.0 (TID 7). >> 2823777 bytes result sent via BlockManager) >> 16/06/15 19:45:22 INFO TaskSetManager: Starting task 2.0 in stage 2.0 >> (TID 8, localhost, partition 2, PROCESS_LOCAL, 5422 bytes) >> 16/06/15 19:45:22 INFO Executor: Running task 2.0 in stage 2.0 (TID 8) >> 16/06/15 19:45:22 INFO HadoopRDD: Input split: >> file:/C:/Users/i303551/Downloads/ariba-logs/ssws/access.2016.04.26/access.2016.04.26:67108864+25111592 >> 16/06/15 19:45:22 INFO BlockManagerInfo: Added rdd_2_2 on disk on >> 192.168.56.1:56413 (size: 2.0 MB) >> 16/06/15 19:45:22 INFO MemoryStore: Block taskresult_8 stored as bytes in >> memory (estimated size 2.0 MB, free 2.4 GB) >> 16/06/15 19:45:22 INFO BlockManagerInfo: Added taskresult_8 in memory on >> 192.168.56.1:56413 (size: 2.0 MB, free: 2.4 GB) >> 16/06/15 19:45:22 INFO Executor: Finished task 2.0 in stage 2.0 (TID 8). >> 2143771 bytes result sent via BlockManager) >> 16/06/15 19:45:43 ERROR RetryingBlockFetcher: Exception while beginning >> fetch of 1 outstanding blocks >> java.io.IOException: Failed to connect to /192.168.56.1:56413 >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:228) >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:179) >> at >> org.apache.spark.network.netty.NettyBlockTransferService$$anon$1.createAndStart(NettyBlockTransferService.scala:96) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.fetchAllOutstanding(RetryingBlockFetcher.java:140) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.start(RetryingBlockFetcher.java:120) >> at >> org.apache.spark.network.netty.NettyBlockTransferService.fetchBlocks(NettyBlockTransferService.scala:105) >> at >> org.apache.spark.network.BlockTransferService.fetchBlockSync(BlockTransferService.scala:92) >> at >> org.apache.spark.storage.BlockManager.getRemoteBytes(BlockManager.scala:546) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:76) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57) >> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.net.ConnectException: Connection timed out: no further >> information: /192.168.56.1:56413 >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) >> at >> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224) >> at >> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >> at >> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) >> ... 1 more >> 16/06/15 19:45:43 INFO RetryingBlockFetcher: Retrying fetch (1/3) for 1 >> outstanding blocks after 5000 ms >> 16/06/15 19:46:04 ERROR RetryingBlockFetcher: Exception while beginning >> fetch of 1 outstanding blocks >> java.io.IOException: Failed to connect to /192.168.56.1:56413 >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:228) >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:179) >> at >> org.apache.spark.network.netty.NettyBlockTransferService$$anon$1.createAndStart(NettyBlockTransferService.scala:96) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.fetchAllOutstanding(RetryingBlockFetcher.java:140) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.start(RetryingBlockFetcher.java:120) >> at >> org.apache.spark.network.netty.NettyBlockTransferService.fetchBlocks(NettyBlockTransferService.scala:105) >> at >> org.apache.spark.network.BlockTransferService.fetchBlockSync(BlockTransferService.scala:92) >> at >> org.apache.spark.storage.BlockManager.getRemoteBytes(BlockManager.scala:546) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:76) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57) >> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.net.ConnectException: Connection timed out: no further >> information: /192.168.56.1:56413 >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) >> at >> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224) >> at >> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >> at >> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) >> ... 1 more >> 16/06/15 19:46:04 INFO RetryingBlockFetcher: Retrying fetch (1/3) for 1 >> outstanding blocks after 5000 ms >> 16/06/15 19:46:25 ERROR RetryingBlockFetcher: Exception while beginning >> fetch of 1 outstanding blocks >> java.io.IOException: Failed to connect to /192.168.56.1:56413 >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:228) >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:179) >> at >> org.apache.spark.network.netty.NettyBlockTransferService$$anon$1.createAndStart(NettyBlockTransferService.scala:96) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.fetchAllOutstanding(RetryingBlockFetcher.java:140) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.start(RetryingBlockFetcher.java:120) >> at >> org.apache.spark.network.netty.NettyBlockTransferService.fetchBlocks(NettyBlockTransferService.scala:105) >> at >> org.apache.spark.network.BlockTransferService.fetchBlockSync(BlockTransferService.scala:92) >> at >> org.apache.spark.storage.BlockManager.getRemoteBytes(BlockManager.scala:546) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:76) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57) >> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793) >> at >> org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.net.ConnectException: Connection timed out: no further >> information: /192.168.56.1:56413 >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) >> at >> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224) >> at >> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >> at >> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) >> ... 1 more >> 16/06/15 19:46:25 INFO RetryingBlockFetcher: Retrying fetch (1/3) for 1 >> outstanding blocks after 5000 ms >> 16/06/15 19:46:46 ERROR RetryingBlockFetcher: Exception while beginning >> fetch of 1 outstanding blocks (after 1 retries) >> java.io.IOException: Failed to connect to /192.168.56.1:56413 >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:228) >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:179) >> at >> org.apache.spark.network.netty.NettyBlockTransferService$$anon$1.createAndStart(NettyBlockTransferService.scala:96) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.fetchAllOutstanding(RetryingBlockFetcher.java:140) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.access$200(RetryingBlockFetcher.java:43) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher$1.run(RetryingBlockFetcher.java:170) >> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >> at java.util.concurrent.FutureTask.run(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.net.ConnectException: Connection timed out: no further >> information: /192.168.56.1:56413 >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) >> at >> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224) >> at >> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >> at >> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) >> ... 1 more >> 16/06/15 19:46:46 INFO RetryingBlockFetcher: Retrying fetch (2/3) for 1 >> outstanding blocks after 5000 ms >> 16/06/15 19:47:07 ERROR RetryingBlockFetcher: Exception while beginning >> fetch of 1 outstanding blocks (after 1 retries) >> java.io.IOException: Failed to connect to /192.168.56.1:56413 >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:228) >> at >> org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:179) >> at >> org.apache.spark.network.netty.NettyBlockTransferService$$anon$1.createAndStart(NettyBlockTransferService.scala:96) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.fetchAllOutstanding(RetryingBlockFetcher.java:140) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher.access$200(RetryingBlockFetcher.java:43) >> at >> org.apache.spark.network.shuffle.RetryingBlockFetcher$1.run(RetryingBlockFetcher.java:170) >> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >> at java.util.concurrent.FutureTask.run(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.net.ConnectException: Connection timed out: no further >> information: /192.168.56.1:56413 >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) >> at >> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224) >> at >> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >> at >> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) >> ... 1 more >> 16/06/15 19:47:07 INFO RetryingBlockFetcher: Retrying fetch (2/3) for 1 >> outstanding blocks after 5000 ms >> >> >> Appreciate some help. >> >> Regards >> > > >