Hi Team,
I am geting the attached exception once in a while while starting the
ignite cluster.
Even though no one is running ignite i get some times address already in
use.
I have specified a ramnge of 10 for both the discovery and communication
ports.
Any help what i am missing here.
my code below
val igniteConfig = new IgniteConfiguration()
igniteConfig.setGridName(igniteGridName)
val tcpDiscoverySpi = new TcpDiscoverySpi()
val ipFinder = new TcpDiscoveryVmIpFinder()
ipFinder.setAddresses(ipAddresses.value.toList.asJava)
tcpDiscoverySpi.setIpFinder(ipFinder)
tcpDiscoverySpi.setLocalPort(Integer.valueOf(portNumber.value))
tcpDiscoverySpi.setLocalPortRange(10)
igniteConfig.setDiscoverySpi(tcpDiscoverySpi)
val commSpi=new TcpCommunicationSpi()
commSpi.setLocalPort(Integer.valueOf(portNumber.value)+1000)
commSpi.setLocalPortRange(10)
// Overriding communication SPI.
igniteConfig.setCommunicationSpi(commSpi)
Ignition.getOrStart(igniteConfig)
Thanks,
Ranjit
17/10/23 10:46:06 WARN scheduler.TaskSetManager: Lost task 18.0 in
stage 1.0 (TID 58, c157mgh.int.westgroup.com, executor 25): class
org.apache.ignite.IgniteException: Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:927)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:417)
at un.api.resolver.dao.CompanyDAO.getIgnite(CompanyDAO.scala:92)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:71)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:67)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
start manager: GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1589)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:857)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1794)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1598)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1041)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:568)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:529)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
... 13 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
get SPI attributes.
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:249)
at
org.apache.ignite.internal.managers.communication.GridIoManager.start(GridIoManager.java:231)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1584)
... 20 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
initialize TCP server: 0.0.0.0/0.0.0.0
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1481)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:232)
... 22 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
bind to any port within range [startPort=48750, portRange=10,
locHost=0.0.0.0/0.0.0.0]
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1762)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1478)
... 23 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
initialize NIO selector.
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:670)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:289)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:88)
at
org.apache.ignite.internal.util.nio.GridNioServer$Builder.build(GridNioServer.java:2439)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1711)
... 24 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:654)
... 28 more
17/10/23 10:46:06 INFO scheduler.TaskSetManager: Starting task 18.1 in
stage 1.0 (TID 80, xxxxx, executor 25, partition 18, PROCESS_LOCAL,
1965 bytes)
17/10/23 10:46:06 INFO scheduler.TaskSetManager: Lost task 39.0 in
stage 1.0 (TID 79) on xxxx, executor 25:
org.apache.ignite.IgniteException (Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager])
[duplicate 1]
17/10/23 10:46:06 INFO scheduler.TaskSetManager: Starting task 39.1 in
stage 1.0 (TID 81, xxxx, executor 25, partition 39, PROCESS_LOCAL,
1932 bytes)
17/10/23 10:46:06 INFO scheduler.TaskSetManager: Lost task 18.1 in
stage 1.0 (TID 80) on xxxxx, executor 25:
org.apache.ignite.IgniteException (Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager])
[duplicate 2]
17/10/23 10:46:06 ERROR scheduler.TaskSetManager: Task 18 in stage 1.0
failed 4 times; aborting job
17/10/23 10:46:06 INFO cluster.YarnClusterScheduler: Cancelling stage 1
17/10/23 10:46:06 INFO cluster.YarnClusterScheduler: Stage 1 was cancelled
17/10/23 10:46:06 INFO scheduler.DAGScheduler: ResultStage 1
(foreachPartition at IgniteDataLoader.scala:67) failed in 1.324 s due
to Job aborted due to stage failure: Task 18 in stage 1.0 failed 4
times, most recent failure: Lost task 18.3 in stage 1.0 (TID 84,
c157mgh.int.westgroup.com, executor 25): class
org.apache.ignite.IgniteException: Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:927)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:417)
at un.api.resolver.dao.CompanyDAO.getIgnite(CompanyDAO.scala:92)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:71)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:67)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
start manager: GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1589)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:857)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1794)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1598)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1041)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:568)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:529)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
... 13 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
get SPI attributes.
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:249)
at
org.apache.ignite.internal.managers.communication.GridIoManager.start(GridIoManager.java:231)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1584)
... 20 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
initialize TCP server: 0.0.0.0/0.0.0.0
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1481)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:232)
... 22 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
bind to any port within range [startPort=48750, portRange=10,
locHost=0.0.0.0/0.0.0.0]
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1762)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1478)
... 23 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
initialize NIO selector.
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:670)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:289)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:88)
at
org.apache.ignite.internal.util.nio.GridNioServer$Builder.build(GridNioServer.java:2439)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1711)
... 24 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:654)
... 28 more
Driver stacktrace:
17/10/23 10:46:06 INFO scheduler.DAGScheduler: Job 1 failed:
foreachPartition at IgniteDataLoader.scala:67, took 1.346020 s
17/10/23 10:46:06 ERROR yarn.ApplicationMaster: User class threw
exception: org.apache.spark.SparkException: Job aborted due to stage
failure: Task 18 in stage 1.0 failed 4 times, most recent failure:
Lost task 18.3 in stage 1.0 (TID 84, xxxxx, executor 25): class
org.apache.ignite.IgniteException: Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:927)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:417)
at un.api.resolver.dao.CompanyDAO.getIgnite(CompanyDAO.scala:92)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:71)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:67)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
start manager: GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1589)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:857)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1794)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1598)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1041)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:568)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:529)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
... 13 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
get SPI attributes.
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:249)
at
org.apache.ignite.internal.managers.communication.GridIoManager.start(GridIoManager.java:231)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1584)
... 20 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
initialize TCP server: 0.0.0.0/0.0.0.0
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1481)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:232)
... 22 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
bind to any port within range [startPort=48750, portRange=10,
locHost=0.0.0.0/0.0.0.0]
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1762)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1478)
... 23 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
initialize NIO selector.
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:670)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:289)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:88)
at
org.apache.ignite.internal.util.nio.GridNioServer$Builder.build(GridNioServer.java:2439)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1711)
... 24 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:654)
... 28 more
Driver stacktrace:
org.apache.spark.SparkException: Job aborted due to stage failure:
Task 18 in stage 1.0 failed 4 times, most recent failure: Lost task
18.3 in stage 1.0 (TID 84, c157mgh.int.westgroup.com, executor 25):
class org.apache.ignite.IgniteException: Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:927)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:417)
at un.api.resolver.dao.CompanyDAO.getIgnite(CompanyDAO.scala:92)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:71)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:67)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
start manager: GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1589)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:857)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1794)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1598)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1041)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:568)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:529)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
... 13 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
get SPI attributes.
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:249)
at
org.apache.ignite.internal.managers.communication.GridIoManager.start(GridIoManager.java:231)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1584)
... 20 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
initialize TCP server: 0.0.0.0/0.0.0.0
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1481)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:232)
... 22 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
bind to any port within range [startPort=48750, portRange=10,
locHost=0.0.0.0/0.0.0.0]
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1762)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1478)
... 23 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
initialize NIO selector.
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:670)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:289)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:88)
at
org.apache.ignite.internal.util.nio.GridNioServer$Builder.build(GridNioServer.java:2439)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1711)
... 24 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:654)
... 28 more
Driver stacktrace:
at
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1457)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1445)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1444)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at
org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1444)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at scala.Option.foreach(Option.scala:236)
at
org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1668)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1627)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1616)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at
org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1840)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1853)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1866)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1937)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:920)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:918)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:918)
at
un.api.dataloader.IgniteDataLoader$.initIgniteCluster(IgniteDataLoader.scala:67)
at
un.api.resolver.accessor.UnifiedAccessor$.getCompanyAccessor(UnifiedAccessor.scala:110)
at
un.api.resolver.CompanyResolverSparkTest$.main(CompanyResolverSparkTest.scala:33)
at
un.api.resolver.CompanyResolverSparkTest.main(CompanyResolverSparkTest.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:553)
Caused by: class org.apache.ignite.IgniteException: Failed to start
manager: GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:927)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:417)
at un.api.resolver.dao.CompanyDAO.getIgnite(CompanyDAO.scala:92)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:71)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:67)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
start manager: GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1589)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:857)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1794)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1598)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1041)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:568)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:529)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
... 13 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
get SPI attributes.
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:249)
at
org.apache.ignite.internal.managers.communication.GridIoManager.start(GridIoManager.java:231)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1584)
... 20 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
initialize TCP server: 0.0.0.0/0.0.0.0
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1481)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:232)
... 22 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
bind to any port within range [startPort=48750, portRange=10,
locHost=0.0.0.0/0.0.0.0]
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1762)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1478)
... 23 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
initialize NIO selector.
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:670)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:289)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:88)
at
org.apache.ignite.internal.util.nio.GridNioServer$Builder.build(GridNioServer.java:2439)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1711)
... 24 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:654)
... 28 more
17/10/23 10:46:06 INFO yarn.ApplicationMaster: Final app status:
FAILED, exitCode: 15, (reason: User class threw exception:
org.apache.spark.SparkException: Job aborted due to stage failure:
Task 18 in stage 1.0 failed 4 times, most recent failure: Lost task
18.3 in stage 1.0 (TID 84, c157mgh.int.westgroup.com, executor 25):
class org.apache.ignite.IgniteException: Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:927)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:417)
at un.api.resolver.dao.CompanyDAO.getIgnite(CompanyDAO.scala:92)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:71)
at
un.api.dataloader.IgniteDataLoader$$anonfun$initIgniteCluster$2.apply(IgniteDataLoader.scala:67)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1866)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
start manager: GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.communication.GridIoManager]
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1589)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:857)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1794)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1598)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1041)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:568)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:529)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
... 13 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
get SPI attributes.
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:249)
at
org.apache.ignite.internal.managers.communication.GridIoManager.start(GridIoManager.java:231)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1584)
... 20 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
initialize TCP server: 0.0.0.0/0.0.0.0
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1481)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:232)
... 22 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
bind to any port within range [startPort=48750, portRange=10,
locHost=0.0.0.0/0.0.0.0]
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1762)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getNodeAttributes(TcpCommunicationSpi.java:1478)
... 23 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
initialize NIO selector.
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:670)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:289)
at
org.apache.ignite.internal.util.nio.GridNioServer.<init>(GridNioServer.java:88)
at
org.apache.ignite.internal.util.nio.GridNioServer$Builder.build(GridNioServer.java:2439)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.resetNioServer(TcpCommunicationSpi.java:1711)
... 24 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at
org.apache.ignite.internal.util.nio.GridNioServer.createSelector(GridNioServer.java:654)
... 28 more
