Re: Ignite for Spark on YARN Deployment

[percent620]

$cat default-config.xml




http://www.springframework.org/schema/beans";
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
   xsi:schemaLocation="
   http://www.springframework.org/schema/beans
   http://www.springframework.org/schema/beans/spring-beans.xsd";>





$cat example-default.xml





http://www.springframework.org/schema/beans";
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
   xmlns:util="http://www.springframework.org/schema/util";
   xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd";>








































127.0.0.1:47500..47509












--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp6910p6963.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite for Spark on YARN Deployment

[Nikolai Tikhonov]

Hi,

Could you please provide logs from containers?
Also IGNITE_PATH property incorrect. The property should contains path to
apache ignite zip archive. For example:
/ignite/apache-ignite-fabric-1.7.0-bin.zip.
Also IGNITE_USERS_LIBS not needed here. The property need to use when you
want deploy to cluster your own libs.

On Thu, Aug 11, 2016 at 7:54 AM, percent620  wrote:

> Thanks vkulichenko's quick response.
>
> Here is my detailed steps about how to deploy and integration spark with
> ignite as below.
>
> 1、Followed this guidelines about how to deploy ignite-yarn application.
>  >
>
> it's successfully and the log is displayed ok
> ==
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/u01/hbase/hadoop-2.5.0-cdh5.3.0/share/hadoop/
> common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/u01/hbase/tmp/nm-local-dir/usercache/hbase/
> appcache/application_1455892346017_5077/filecache/
> 10/ignite-yarn.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 16/08/10 22:54:57 INFO impl.ContainerManagementProtocolProxy:
> yarn.client.max-cached-nodemanagers-proxies : 0
> Aug 10, 2016 10:54:58 PM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Application master registered.
> Aug 10, 2016 10:54:58 PM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Making request. Memory: 2,432, cpu 1.
> Aug 10, 2016 10:54:58 PM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Making request. Memory: 2,432, cpu 1.
> 16/08/10 22:54:59 INFO impl.AMRMClientImpl: Received new token for :
> vmsecdomain010194062066.cm10:61362
> 16/08/10 22:54:59 INFO impl.AMRMClientImpl: Received new token for :
> vmsecdomain010194062042.cm10:42077
> Aug 10, 2016 10:54:59 PM org.apache.ignite.yarn.ApplicationMaster
> onContainersAllocated
> INFO: Launching container: container_1455892346017_5077_02_02.
> 16/08/10 22:54:59 INFO impl.ContainerManagementProtocolProxy: Opening
> proxy
> : vmsecdomain010194062066.cm10:61362
> Aug 10, 2016 10:54:59 PM org.apache.ignite.yarn.ApplicationMaster
> onContainersAllocated
> INFO: Launching container: container_1455892346017_5077_02_03.
> 16/08/10 22:54:59 INFO impl.ContainerManagementProtocolProxy: Opening
> proxy
> : vmsecdomain010194062042.cm10:42077
> Aug 10, 2016 10:55:08 PM org.apache.ignite.yarn.ApplicationMaster
> onContainersCompleted
> INFO: Container completed. Container id:
> container_1455892346017_5077_02_02. State: COMPLETE.
> Aug 10, 2016 10:55:09 PM org.apache.ignite.yarn.ApplicationMaster
> onContainersCompleted
> INFO: Container completed. Container id:
> container_1455892346017_5077_02_03. State: COMPLETE.
>
>
> 2、downloaded this apache-ignite-fabric-1.6.0-bin.zip and unzip this file
> to
> the /u01/XXX/apache-ignite-fabric-1.6.0-bin directory.
>
> 3、cluster16.properis content is as below
> $cat cluster16.properties
> # The number of nodes in the cluster.
> IGNITE_NODE_COUNT=2
> # The number of CPU Cores for each Apache Ignite node.
> IGNITE_RUN_CPU_PER_NODE=1
> # The number of Megabytes of RAM for each Apache Ignite node.
> IGNITE_MEMORY_PER_NODE=2048
> # The version ignite which will be run on nodes.
> IGNITE_VERSION=1.6.0
> IGNITE_WORK_DIR=/u01/XXX/apache-ignite-fabric-1.6.0-bin/
> IGNITE_XML_CONFIG=/ignite/release16/apache-ignite-fabric-1.6.0-bin/config/
> default-config.xml
> IGNITE_RELEASES_DIR=/ignite/release16/
> IGNITE_USERS_LIBS=/u01/XXX/apache-ignite-fabric-1.6.0-bin/libs/
> IGNITE_PATH=/ignite/release16/
>
>
> hdfs directory is as below
> ===
> ./hdfs dfs -ls /ignite/
> drwxr-xr-x   - hbase hbase  0 2016-08-10 17:07 /ignite/release16
> drwxr-xr-x   - hbase hbase  0 2016-08-02 06:25 /ignite/releases
> drwxr-xr-x   - hbase hbase  0 2016-08-10 17:24 /ignite/workdir
> -rw-r--r--   3 hbase hbase   27710331 2016-08-10 17:04 /ignite/yarn
> =
> $./hdfs dfs -ls /ignite/release16
> drwxr-xr-x   - hbase hbase  0 2016-08-11 12:44
> /ignite/release16/apache-ignite-fabric-1.6.0-bin
>
> =
>
> 4、Running spark code on yarn and the code is as below
> val igniteContext = new IgniteContext[String, BaseLine](sc,() => new
> IgniteConfiguration())
>
>
> the code is hanging on this and i think that this client can't connect this
> server
>
> 5、from yarn console and found the following error message
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/u01/hbase/hadoop-2.5.0-cdh5.3.0/share/hadoop/
> common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/u01/hbase/tmp/nm-local-dir/usercache/hbase/
> appcache/application_1455892346017_5077/filec

Re: Ignite for Spark on YARN Deployment

[percent620]

Thanks vkulichenko's quick response.

Here is my detailed steps about how to deploy and integration spark with
ignite as below.

1、Followed this guidelines about how to deploy ignite-yarn application.
 

it's successfully and the log is displayed ok
==
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/u01/hbase/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/u01/hbase/tmp/nm-local-dir/usercache/hbase/appcache/application_1455892346017_5077/filecache/10/ignite-yarn.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/08/10 22:54:57 INFO impl.ContainerManagementProtocolProxy:
yarn.client.max-cached-nodemanagers-proxies : 0
Aug 10, 2016 10:54:58 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Application master registered.
Aug 10, 2016 10:54:58 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Aug 10, 2016 10:54:58 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
16/08/10 22:54:59 INFO impl.AMRMClientImpl: Received new token for :
vmsecdomain010194062066.cm10:61362
16/08/10 22:54:59 INFO impl.AMRMClientImpl: Received new token for :
vmsecdomain010194062042.cm10:42077
Aug 10, 2016 10:54:59 PM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_1455892346017_5077_02_02.
16/08/10 22:54:59 INFO impl.ContainerManagementProtocolProxy: Opening proxy
: vmsecdomain010194062066.cm10:61362
Aug 10, 2016 10:54:59 PM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_1455892346017_5077_02_03.
16/08/10 22:54:59 INFO impl.ContainerManagementProtocolProxy: Opening proxy
: vmsecdomain010194062042.cm10:42077
Aug 10, 2016 10:55:08 PM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_1455892346017_5077_02_02. State: COMPLETE.
Aug 10, 2016 10:55:09 PM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_1455892346017_5077_02_03. State: COMPLETE.


2、downloaded this apache-ignite-fabric-1.6.0-bin.zip and unzip this file to
the /u01/XXX/apache-ignite-fabric-1.6.0-bin directory.

3、cluster16.properis content is as below
$cat cluster16.properties 
# The number of nodes in the cluster.
IGNITE_NODE_COUNT=2
# The number of CPU Cores for each Apache Ignite node.
IGNITE_RUN_CPU_PER_NODE=1
# The number of Megabytes of RAM for each Apache Ignite node.
IGNITE_MEMORY_PER_NODE=2048
# The version ignite which will be run on nodes.
IGNITE_VERSION=1.6.0
IGNITE_WORK_DIR=/u01/XXX/apache-ignite-fabric-1.6.0-bin/
IGNITE_XML_CONFIG=/ignite/release16/apache-ignite-fabric-1.6.0-bin/config/default-config.xml
IGNITE_RELEASES_DIR=/ignite/release16/
IGNITE_USERS_LIBS=/u01/XXX/apache-ignite-fabric-1.6.0-bin/libs/
IGNITE_PATH=/ignite/release16/


hdfs directory is as below
===
./hdfs dfs -ls /ignite/
drwxr-xr-x   - hbase hbase  0 2016-08-10 17:07 /ignite/release16
drwxr-xr-x   - hbase hbase  0 2016-08-02 06:25 /ignite/releases
drwxr-xr-x   - hbase hbase  0 2016-08-10 17:24 /ignite/workdir
-rw-r--r--   3 hbase hbase   27710331 2016-08-10 17:04 /ignite/yarn
=
$./hdfs dfs -ls /ignite/release16
drwxr-xr-x   - hbase hbase  0 2016-08-11 12:44
/ignite/release16/apache-ignite-fabric-1.6.0-bin

=

4、Running spark code on yarn and the code is as below
val igniteContext = new IgniteContext[String, BaseLine](sc,() => new
IgniteConfiguration())


the code is hanging on this and i think that this client can't connect this
server

5、from yarn console and found the following error message
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/u01/hbase/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/u01/hbase/tmp/nm-local-dir/usercache/hbase/appcache/application_1455892346017_5077/filecache/10/ignite-yarn.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/08/10 17:24:16 INFO impl.ContainerManagementProtocolProxy:
yarn.client.max-cached-nodemanagers-proxies : 0
Aug 10, 2016 5:24:16 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Application master registered.
Aug 10, 2016 5:24:16 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Aug 10, 2016 5:24:16 PM org.apache.ignite.yar

Re: Ignite for Spark on YARN Deployment

[vkulichenko]

Hi,

Can you please properly subscribe to the mailing list so that the community
can receive email notifications? Here is the instruction:
http://apache-ignite-users.70518.x6.nabble.com/mailing_list/MailingListOptions.jtp?forum=1


percent620 wrote
> Ignite-Yarn application has been running successfully. but when running
> the spark-shell, get hanging when running this code as below
> val cfg = new IgniteConfiguration()
> val spi = new TcpDiscoverySpi()
> 
> val ipFinder = new TcpDiscoveryVmIpFinder()
> ipFinder.setAddresses(util.Arrays.asList(":47500..47510"))
> 
> spi.setIpFinder(ipFinder)
> cfg.setDiscoverySpi(spi)
> 
> val igniteContext = new IgniteContext[String, BaseLine](sc,() => cfg)
> 
> I'm sure that this client can't connect to yarn server,right? 
> 
> Can anyone help me on this issues? thanks!

Can you attach the log files from the server nodes? Are they discovering
each other?

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp6910p6935.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite for Spark on YARN Deployment

[Nikolai Tikhonov]

Hi Hongmei Zong,

Could you use the following configuration for Ignite [1]?  You need to use
the same configuration file for YARN integration and IgniteContext. The
configuration uses TcpDiscoveryMulticastIpFinder ip finder which will be
more acceptable for your case. You can get more details about it there (
https://apacheignite.readme.io/docs/cluster-config).

Also I see that you try use different ignite versions. Could you download
apache ignite fabric [2], copy it to hdfs (for example
/user/hongmei/ignite/apache-ignite-fabric-1.6.0-bin.zip) and add the
following property to file.

IGNITE_PATH=/user/hongmei/ignite/apache-ignite-fabric-1.6.0-bin.zip

1. https://gist.github.com/ntikhonov/b07bdb12a3c2e5d90bef2210eac74b83
2, http://ignite.apache.org/download.cgi#binaries

On Fri, Jun 10, 2016 at 5:52 PM, Hongmei Zong  wrote:

> Hi Nikolai,
>
> I ran the command to start the Spark-shell and Spark started successfully
> Then I import two classes and create a new IgniteContext, I got the error
> as posted below.  Any good suggestions??? Thank you very much!
>
> Hongmei
>
> *First run:*
>
> /usr/bin/spark-shell --jars
> /u/hongmei/apache-ignite/libs/ignite-core-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-spark/ignite-spark-1.6.0.jar,/u/hongmei/apache-ignite/libs/cache-api-1.0.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/ignite-log4j-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/log4j-1.2.17.jar
> --packages
> org.apache.ignite:ignite-spark:1.6.0,org.apache.ignite:ignite-spring:1.6.0
>
> *Then import and new a ic:*
>
> scala> import org.apache.ignite.spark._
> import org.apache.ignite.spark._
>
> scala> import org.apache.ignite.configuration._
> import org.apache.ignite.configuration._
>
> scala> val ic = new IgniteContext[Integer, Integer](sc,
> "config/ignite-default-config.xml")
> 16/06/10 10:32:17 INFO XmlBeanDefinitionReader: Loading XML bean
> definitions from URL
> [file:/u/hongmei/apache-ignite/config/ignite-default-config.xml]
> 16/06/10 10:32:18 INFO GenericApplicationContext: Refreshing
> org.springframework.context.support.GenericApplicationContext@3cfbcf6a:
> startup date [Fri Jun 10 10:32:18 EDT 2016]; root of context hierarchy
> 16/06/10 10:32:19 INFO IgniteKernal:
>
> >>>__  
> >>>   /  _/ ___/ |/ /  _/_  __/ __/
> >>>  _/ // (7 7// /  / / / _/
> >>> /___/\___/_/|_/___/ /_/ /___/
> >>>
> >>> ver. 1.6.0#20160518-sha1:0b22c45b
> >>> 2016 Copyright(C) Apache Software Foundation
> >>>
> >>> Ignite documentation: http://ignite.apache.org
>
> 16/06/10 10:32:19 INFO IgniteKernal: Config URL: n/a
> 16/06/10 10:32:19 INFO IgniteKernal: Daemon mode: off
> 16/06/10 10:32:19 INFO IgniteKernal: OS: Linux 2.6.32-573.26.1.el6.x86_64
> amd64
> 16/06/10 10:32:19 INFO IgniteKernal: OS user: hongmei
> 16/06/10 10:32:19 INFO IgniteKernal: Language runtime: Scala ver. 2.10.4
> 16/06/10 10:32:19 INFO IgniteKernal: VM information: Java(TM) SE Runtime
> Environment 1.7.0_45-b18 Oracle Corporation Java HotSpot(TM) 64-Bit Server
> VM 24.45-b08
> 16/06/10 10:32:19 INFO IgniteKernal: VM total memory: 4.2GB
> 16/06/10 10:32:19 INFO IgniteKernal: Remote Management [restart: off,
> REST: on, JMX (remote: off)]
> 16/06/10 10:32:19 INFO IgniteKernal: IGNITE_HOME=/u/hongmei/apache-ignite
> 16/06/10 10:32:19 INFO IgniteKernal: VM arguments:
> [-Dhdp.version=2.3.4.0-3485, -Dscala.usejavacp=true, -Xms4500m, -Xmx4500m,
> -XX:MaxPermSize=1024m, -XX:PermSize=256m]
> 16/06/10 10:32:19 INFO IgniteKernal: Configured caches
> ['ignite-marshaller-sys-cache', 'ignite-sys-cache',
> 'ignite-atomics-sys-cache']
>
> …….
>
> Caused by: class org.apache.ignite.IgniteCheckedException: *Failed to
> start manager: GridManagerAdapter [enabled=true,
> name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]*
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1536)
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:897)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1736)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1589)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1042)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:569)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:530)
> at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
> ... 53 more
> Caused by: class org.apache.ignite.IgniteCheckedException: *Failed to
> start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000,
> ackTimeout=5000, reconCnt=10, maxAckTimeout=60, forceSrvMode=false,
> clientReconnectDisabled=false]*
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:258)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:677)
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1531)
> ... 6

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Nikolai,

I ran the command to start the Spark-shell and Spark started successfully Then 
I import two classes and create a new IgniteContext, I got the error as posted 
below.  Any good suggestions??? Thank you very much!

Hongmei

First run:

/usr/bin/spark-shell --jars 
/u/hongmei/apache-ignite/libs/ignite-core-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-spark/ignite-spark-1.6.0.jar,/u/hongmei/apache-ignite/libs/cache-api-1.0.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/ignite-log4j-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/log4j-1.2.17.jar
 --packages 
org.apache.ignite:ignite-spark:1.6.0,org.apache.ignite:ignite-spring:1.6.0

Then import and new a ic:

scala> import org.apache.ignite.spark._
import org.apache.ignite.spark._

scala> import org.apache.ignite.configuration._
import org.apache.ignite.configuration._

scala> val ic = new IgniteContext[Integer, Integer](sc, 
"config/ignite-default-config.xml")
16/06/10 10:32:17 INFO XmlBeanDefinitionReader: Loading XML bean definitions 
from URL [file:/u/hongmei/apache-ignite/config/ignite-default-config.xml]
16/06/10 10:32:18 INFO GenericApplicationContext: Refreshing 
org.springframework.context.support.GenericApplicationContext@3cfbcf6a: startup 
date [Fri Jun 10 10:32:18 EDT 2016]; root of context hierarchy
16/06/10 10:32:19 INFO IgniteKernal: 

>>>__    
>>>   /  _/ ___/ |/ /  _/_  __/ __/  
>>>  _/ // (7 7// /  / / / _/
>>> /___/\___/_/|_/___/ /_/ /___/   
>>> 
>>> ver. 1.6.0#20160518-sha1:0b22c45b
>>> 2016 Copyright(C) Apache Software Foundation
>>> 
>>> Ignite documentation: http://ignite.apache.org

16/06/10 10:32:19 INFO IgniteKernal: Config URL: n/a
16/06/10 10:32:19 INFO IgniteKernal: Daemon mode: off
16/06/10 10:32:19 INFO IgniteKernal: OS: Linux 2.6.32-573.26.1.el6.x86_64 amd64
16/06/10 10:32:19 INFO IgniteKernal: OS user: hongmei
16/06/10 10:32:19 INFO IgniteKernal: Language runtime: Scala ver. 2.10.4
16/06/10 10:32:19 INFO IgniteKernal: VM information: Java(TM) SE Runtime 
Environment 1.7.0_45-b18 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 
24.45-b08
16/06/10 10:32:19 INFO IgniteKernal: VM total memory: 4.2GB
16/06/10 10:32:19 INFO IgniteKernal: Remote Management [restart: off, REST: on, 
JMX (remote: off)]
16/06/10 10:32:19 INFO IgniteKernal: IGNITE_HOME=/u/hongmei/apache-ignite
16/06/10 10:32:19 INFO IgniteKernal: VM arguments: [-Dhdp.version=2.3.4.0-3485, 
-Dscala.usejavacp=true, -Xms4500m, -Xmx4500m, -XX:MaxPermSize=1024m, 
-XX:PermSize=256m]
16/06/10 10:32:19 INFO IgniteKernal: Configured caches 
['ignite-marshaller-sys-cache', 'ignite-sys-cache', 'ignite-atomics-sys-cache']

…….

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start 
manager: GridManagerAdapter [enabled=true, 
name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
at 
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1536)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:897)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1736)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1589)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1042)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:569)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:530)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
... 53 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI: 
TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, 
reconCnt=10, maxAckTimeout=60, forceSrvMode=false, 
clientReconnectDisabled=false]
at 
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:258)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:677)
at 
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1531)
... 60 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Join process timed 
out, did not receive response for join request (consider increasing 
'joinTimeout' configuration property) [joinTimeout=6, 
sock=Socket[addr=c5hdp112.c5.runwaynine.com/10.138.10.47,port=47500,localport=59822]]
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1335)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)


scala>



> On Jun 10, 2016, at 8:20 AM, Nikolay Tikhonov  wrote:
> 
> Could you download a configuration file 
> (https://gist.github.com/ntikhonov/d2a2ede2faca7b533dc643e0da475959 
> ) and put 
> it to 
> /user/hongmei/ignite/config/ignite-default-config.xml? Also change 
> IGNITE_XML_CONFIG from /user/hongmei/ignite/co

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Nikolay,


Thank you very much for your solution. I think the ignite started successfully 
with your updated version of configuration file!!! :)
I think so far ignite running ok on YARN, right?  Next I will try to Test 
ignite from Spark-shell. Should I follow the link below:

https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell

In Starting up the cluster, step 1-4, Since our spark clusters are alway 
running on YARN, and I do not think I need to start spark, right?
In Starting up the cluster, step 5 , do I need to start Ignite node using the 
following command? Since I have started a ignite YARN application and currently 
running.
 
On each of the worker nodes cd to $IGNITE_HOME and start an Ignite node by 
running the following command:bin/ignite.sh

In Working with Spark shell section, I run the following command and Spark 
shell launched successfully.
/usr/bin/spark-shell --jars 
/u/hongmei/apache-ignite/libs/ignite-core-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-spark/ignite-spark-1.6.0.jar,/u/hongmei/apache-ignite/libs/cache-api-1.0.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/ignite-log4j-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/log4j-1.2.17.jar


Then I ran these three command: The Warning message as below: I think I need to 
put the ip addresses of ignite server node into the ignite-default-config.xml, 
am I right?
import org.apache.ignite.spark._
import org.apache.ignite.configuration._
val ic = new IgniteContext[Integer, Integer](sc, () => new 
IgniteConfiguration())

16/06/10 09:18:13 WARN TcpDiscoverySpi: IP finder returned empty addresses 
list. Please check IP finder configuration and make sure multicast works on 
your network. Will retry every 2 secs.

Thank you very much for all your help

Hongmei

Starting up the cluster

 

Here is the log from my console:
Logs for container_e24_1464374946035_32224_01_01



SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/usr/hdp/2.3.4.0-3485/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/disk/09/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_32224/filecache/10/ignite-yarn.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/06/10 08:43:48 INFO impl.ContainerManagementProtocolProxy: 
yarn.client.max-cached-nodemanagers-proxies : 0
16/06/10 08:43:48 INFO client.ConfiguredRMFailoverProxyProvider: Failing over 
to rm2
Jun 10, 2016 8:43:48 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Application master registered.
Jun 10, 2016 8:43:48 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 10, 2016 8:43:48 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
16/06/10 08:43:49 INFO impl.AMRMClientImpl: Received new token for : 
c5hdp105.c5.runwaynine.com:45454
16/06/10 08:43:49 INFO impl.AMRMClientImpl: Received new token for : 
c5hdp112.c5.runwaynine.com:45454
Jun 10, 2016 8:43:49 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 10, 2016 8:43:49 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 10, 2016 8:43:49 AM org.apache.ignite.yarn.ApplicationMaster 
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_32224_01_02.
16/06/10 08:43:49 INFO impl.ContainerManagementProtocolProxy: Opening proxy : 
c5hdp105.c5.runwaynine.com:45454
Jun 10, 2016 8:43:49 AM org.apache.ignite.yarn.ApplicationMaster 
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_32224_01_03.
16/06/10 08:43:49 INFO impl.ContainerManagementProtocolProxy: Opening proxy : 
c5hdp112.c5.runwaynine.com:45454
16/06/10 08:43:50 INFO impl.AMRMClientImpl: Received new token for : 
c5hdp114.c5.runwaynine.com:45454
16/06/10 08:43:50 INFO impl.AMRMClientImpl: Received new token for : 
c5hdp115.c5.runwaynine.com:45454
Jun 10, 2016 8:43:50 AM org.apache.ignite.yarn.ApplicationMaster 
onContainersCompleted
INFO: Container completed. Container id: 
container_e24_1464374946035_32224_01_06. State: COMPLETE.
Jun 10, 2016 8:43:50 AM org.apache.ignite.yarn.ApplicationMaster 
onContainersCompleted
INFO: Container completed. Container id: 
container_e24_1464374946035_32224_01_07. State: COMPLETE.
Jun 10, 2016 8:43:50 AM org.apache.ignite.yarn.ApplicationMaster 
onContainersCompleted
INFO: Container completed. Container id: 
container_e24_1464374946035_32224_01_04. State: COMPLETE.
Jun 10, 2016 8:43:50 AM org.apache.ignite.yarn.ApplicationMaster 
onContainersCompleted
INFO: Container completed. Container id: 
container_e24_1464374946035_32224_01_05. State: COMPLETE.

Logs for container_e24_1464374946035_32224_01_

Re: Ignite for Spark on YARN Deployment

[Nikolay Tikhonov]

Could you download a configuration file (
https://gist.github.com/ntikhonov/d2a2ede2faca7b533dc643e0da475959) and put
it to
/user/hongmei/ignite/config/ignite-default-config.xml? Also change
IGNITE_XML_CONFIG from /user/hongmei/ignite/config/ to
/user/hongmei/ignite/config/ignite-default-config.xml?

IGNITE_XML_CONFIG=/user/hongmei/ignite/config/ignite-default-config.xml

On Fri, Jun 10, 2016 at 2:58 PM, Hongmei Zong  wrote:

> Hi Nikolai,
>
> Here is the log of one of the container:
>
> Logs for container_e24_1464374946035_32114_01_14
>
>
> Showing 4096 bytes. Click here
> 
>  for
> full log
>
> SpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
>   at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
>   at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
>   at 
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556)
>   at org.apache.ignite.Ignition.start(Ignition.java:347)
>   ... 1 more
> Caused by: 
> org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 1 
> in XML document from URL 
> [file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_32114/container_e24_1464374946035_32114_01_14/./ignite-config.xml/]
>  is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 
> 1; columnNumber: 1; Content is not allowed in prolog.
>   at 
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398)
>   at 
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335)
>   at 
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)
>   at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379)
>   ... 9 more
> Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; 
> Content is not allowed in prolog.
>   at 
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
>   at 
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
>   at 
> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
>   at 
> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368)
>   at 
> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1436)
>   at 
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:999)
>   at 
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
>   at 
> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
>   at 
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
>   at 
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
>   at 
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
>   at 
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>   at 
> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
>   at 
> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
>   at 
> org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)
>   at 
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:428)
>   at 
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:390)
>   ... 12 more
> Failed to start grid: Failed to instantiate Spring XML application context 
> [springUrl=file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_32114/container_e24_1464374946035_32114_01_14/./ignite-config.xml/,
>  err=Line 1 in XML document from URL 
> [file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_32114/container_e24_1464374946035_32114_0

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Nikolai,

Here is the log of one of the container:

Logs for container_e24_1464374946035_32114_01_14


Showing 4096 bytes. Click here 

 for full log
SpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
at 
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
at 
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
at 
org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556)
at org.apache.ignite.Ignition.start(Ignition.java:347)
... 1 more
Caused by: 
org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 1 
in XML document from URL 
[file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_32114/container_e24_1464374946035_32114_01_14/./ignite-config.xml/]
 is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; 
columnNumber: 1; Content is not allowed in prolog.
at 
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398)
at 
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335)
at 
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)
at 
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379)
... 9 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; 
Content is not allowed in prolog.
at 
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
at 
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
at 
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
at 
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368)
at 
com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1436)
at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:999)
at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at 
com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at 
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at 
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
at 
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at 
org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)
at 
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:428)
at 
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:390)
... 12 more
Failed to start grid: Failed to instantiate Spring XML application context 
[springUrl=file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_32114/container_e24_1464374946035_32114_01_14/./ignite-config.xml/,
 err=Line 1 in XML document from URL 
[file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_32114/container_e24_1464374946035_32114_01_14/./ignite-config.xml/]
 is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; 
columnNumber: 1; Content is not allowed in prolog.]


Thank you very much!
Hongmei

> On Jun 10, 2016, at 5:56 AM, Nikolai Tikhonov  wrote:
> 
> Hi Hongmei Zong!
> 
> Could you show logs from other containers 
> (container_e24_1464374946035_29722_01_15) which was completed?
> 
> On Thu, Jun 9, 2016 at 6:29 PM, Hongmei Zong  > wrote:
> Hi Nikolay,
> 
> After I changed the value of

Re: Ignite for Spark on YARN Deployment

[Nikolai Tikhonov]

Hi Hongmei Zong!

Could you show logs from other containers (container_e24_1464374946035_
29722_01_15) which was completed?

On Thu, Jun 9, 2016 at 6:29 PM, Hongmei Zong  wrote:

> Hi Nikolay,
>
> After I changed the value of IGNITE_XML_CONFIG=/user/hongmei/ignite/config/
>  (a HDFS path). Ignite RARN is running now. I use the Hadoop UI console to
> check the log of the application, the attached is the *stderr *log information
> about containers:
>
> It looks like that the containers are allocated and then completed! The*
> stderr *log is very long and the container ID from X01_001 to
> XX01_013582.  Finally all these containers are completed.
>
> I have no idea, is there anything not right?
>
> There is no information in *stdout* log.
>
> Thank you!
>
> Hongmei
> Logs for container_e24_1464374946035_29722_01_01
>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/usr/hdp/2.3.4.0-3485/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_29722/filecache/10/ignite-yarn.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 16/06/09 10:48:16 INFO impl.ContainerManagementProtocolProxy: 
> yarn.client.max-cached-nodemanagers-proxies : 0
> 16/06/09 10:48:16 INFO client.ConfiguredRMFailoverProxyProvider: Failing over 
> to rm2
> Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Application master registered.
> Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Making request. Memory: 2,432, cpu 1.
> Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Making request. Memory: 2,432, cpu 1.
> 16/06/09 10:48:17 INFO impl.AMRMClientImpl: Received new token for : 
> c5hdp108.c5.runwaynine.com:45454
> 16/06/09 10:48:17 INFO impl.AMRMClientImpl: Received new token for : 
> c5hdp111.c5.runwaynine.com:45454
> Jun 09, 2016 10:48:17 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersAllocated
> INFO: Launching container: container_e24_1464374946035_29722_01_02.
> 16/06/09 10:48:17 INFO impl.ContainerManagementProtocolProxy: Opening proxy : 
> c5hdp108.c5.runwaynine.com:45454
> Jun 09, 2016 10:48:17 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersAllocated
> INFO: Launching container: container_e24_1464374946035_29722_01_03.
> 16/06/09 10:48:17 INFO impl.ContainerManagementProtocolProxy: Opening proxy : 
> c5hdp111.c5.runwaynine.com:45454
> Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersCompleted
> INFO: Container completed. Container id: 
> container_e24_1464374946035_29722_01_02. State: COMPLETE.
> Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersCompleted
> INFO: Container completed. Container id: 
> container_e24_1464374946035_29722_01_03. State: COMPLETE.
> Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Making request. Memory: 2,432, cpu 1.
> Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster run
> INFO: Making request. Memory: 2,432, cpu 1.
> 16/06/09 10:48:25 INFO impl.AMRMClientImpl: Received new token for : 
> c5hdp109.c5.runwaynine.com:45454
> 16/06/09 10:48:25 INFO impl.AMRMClientImpl: Received new token for : 
> c5hdp105.c5.runwaynine.com:45454
> Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersAllocated
> INFO: Launching container: container_e24_1464374946035_29722_01_04.
> 16/06/09 10:48:25 INFO impl.ContainerManagementProtocolProxy: Opening proxy : 
> c5hdp108.c5.runwaynine.com:45454
> Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersAllocated
> INFO: Launching container: container_e24_1464374946035_29722_01_05.
> 16/06/09 10:48:25 INFO impl.ContainerManagementProtocolProxy: Opening proxy : 
> c5hdp111.c5.runwaynine.com:45454
> Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersCompleted
> INFO: Container completed. Container id: 
> container_e24_1464374946035_29722_01_06. State: COMPLETE.
> Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersCompleted
> INFO: Container completed. Container id: 
> container_e24_1464374946035_29722_01_07. State: COMPLETE.
> Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersCompleted
> INFO: Container completed. Container id: 
> container_e24_1464374946035_29722_01_04. State: COMPLETE.
> Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster 
> onContainersCompleted
> INFO: Container completed. Container id: 
> container_e24_1464374946035_29722_01_05. State: COMPLETE.
> Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.Appli

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Nikolay,

After I changed the value of IGNITE_XML_CONFIG=/user/hongmei/ignite/config/
 (a HDFS path). Ignite RARN is running now. I use the Hadoop UI console to
check the log of the application, the attached is the *stderr *log information
about containers:

It looks like that the containers are allocated and then completed! The*
stderr *log is very long and the container ID from X01_001 to
XX01_013582.  Finally all these containers are completed.

I have no idea, is there anything not right?

There is no information in *stdout* log.

Thank you!

Hongmei
Logs for container_e24_1464374946035_29722_01_01

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/hdp/2.3.4.0-3485/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/disk2/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_29722/filecache/10/ignite-yarn.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/06/09 10:48:16 INFO impl.ContainerManagementProtocolProxy:
yarn.client.max-cached-nodemanagers-proxies : 0
16/06/09 10:48:16 INFO client.ConfiguredRMFailoverProxyProvider:
Failing over to rm2
Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Application master registered.
Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:16 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
16/06/09 10:48:17 INFO impl.AMRMClientImpl: Received new token for :
c5hdp108.c5.runwaynine.com:45454
16/06/09 10:48:17 INFO impl.AMRMClientImpl: Received new token for :
c5hdp111.c5.runwaynine.com:45454
Jun 09, 2016 10:48:17 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_02.
16/06/09 10:48:17 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp108.c5.runwaynine.com:45454
Jun 09, 2016 10:48:17 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_03.
16/06/09 10:48:17 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp111.c5.runwaynine.com:45454
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_02. State: COMPLETE.
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_03. State: COMPLETE.
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:24 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
16/06/09 10:48:25 INFO impl.AMRMClientImpl: Received new token for :
c5hdp109.c5.runwaynine.com:45454
16/06/09 10:48:25 INFO impl.AMRMClientImpl: Received new token for :
c5hdp105.c5.runwaynine.com:45454
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_04.
16/06/09 10:48:25 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp108.c5.runwaynine.com:45454
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_05.
16/06/09 10:48:25 INFO impl.ContainerManagementProtocolProxy: Opening
proxy : c5hdp111.c5.runwaynine.com:45454
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_06. State: COMPLETE.
Jun 09, 2016 10:48:25 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_07. State: COMPLETE.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_04. State: COMPLETE.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster
onContainersCompleted
INFO: Container completed. Container id:
container_e24_1464374946035_29722_01_05. State: COMPLETE.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:27 AM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Jun 09, 2016 10:48:28 AM org.apache.ignite.yarn.ApplicationMaster
onContainersAllocated
INFO: Launching container: container_e24_1464374946035_29722_01_08.
16/06/09 10:48:28 INFO impl.ContainerManagementProtocolProxy

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Nikolay,

I tried to comment the line IGNITE_XML_CONFIG= ,but the Ignite YARN failed.

[hongmei@c5hdpe001 config]$ hadoop jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/config/cluster.properties

WARNING: Use "yarn jar" to launch YARN applications.

16/06/09 10:31:43 INFO impl.TimelineClientImpl: Timeline service address:
http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

16/06/09 10:31:47 INFO client.ConfiguredRMFailoverProxyProvider: Failing
over to rm2

16/06/09 10:31:49 INFO impl.YarnClientImpl: Submitted application
application_1464374946035_29710

Jun 09, 2016 10:31:49 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Submitted application. Application id: application_1464374946035_29710

Jun 09, 2016 10:31:59 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Application application_1464374946035_29710 is FAILED.

On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov 
wrote:

> You set wrong value to IGNITE_XML_CONFIG property. The property should
> contains path to ignite configuration file. For example
>
> IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml
>
> I think you can comment this line in property file and ignite will start
> with default configuration.
>
> On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:
>
>> Hi nikolai,
>>
>> Thank you very much for prompt reply!
>>
>> I did not find the ignite-config.xml file under my ignite home directory(
>>  /u/hongmei/apache-ignite/  ).
>>
>> I find a "default-config.xml" at the path:
>> /u/hongmei/apache-ignite/config/default-config.xml
>>
>> 
>>
>>
>> 
>>
>>
>> http://www.springframework.org/schema/beans";
>>
>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>>
>>xsi:schemaLocation="
>>
>>http://www.springframework.org/schema/beans
>>
>>http://www.springframework.org/schema/beans/spring-beans.xsd";>
>>
>> 
>>
>> > "org.apache.ignite.configuration.IgniteConfiguration">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>
>> > "addresses">
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>> c5hdpe001.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe002.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe003.c5.runwaynine.com:47500..47509
>>
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>>
>>
>> The "cluster.properties" at the path:
>> /u/hongmei/apache-ignite/config/cluster.properties
>>
>> # The number of nodes in the cluster.
>>
>> IGNITE_NODE_COUNT=2
>>
>>
>> # The number of CPU Cores for each Apache Ignite node.
>>
>> IGNITE_RUN_CPU_PER_NODE=1
>>
>>
>> # The number of Megabytes of RAM for each Apache Ignite node.
>>
>> IGNITE_MEMORY_PER_NODE=2048
>>
>>
>> # The version ignite which will be run on nodes.
>>
>> IGNITE_VERSION=1.6.0
>>
>>
>> # The hdfs directory which will be used for saving Apache Ignite
>> disbributives.
>>
>> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>>
>>
>> #The directory which will be used for saving Apache Ignite
>> distributives(copy .jar file to it).
>>
>> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>>
>>
>> #The hdfs path to Apache Ignite config file.
>>
>> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>>
>> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>>
>>
>> #The hdfs path to libs which will be added to classpath.
>>
>> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>>
>>
>> #The constraint on slave hosts.
>>
>> #IGNITE_HOSTENAME_CONSTRAINT=
>>
>>
>> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>>
>>
>>
>> Thank you very much
>>
>> Hongmei
>>
>>
>>
>>
>>
>> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
>> wrote:
>>
>>> It seems that your ignite configuration invalid. Could you share 
>>> ignite-config.xml
>>> and /u/hongmei/apache-ignite/config/cluster.properties?
>>>
>>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>>>
 Thank you very much Nikolai !

 I found another issue regarding my Ignite YARN Integration:

 I  run Ignite YARN application on one client server machine

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Nikolay,

I tried to comment the line IGNITE_XML_CONFIG= ,but the Ignite YARN failed.

[hongmei@c5hdpe001 config]$ hadoop jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/config/cluster.properties

WARNING: Use "yarn jar" to launch YARN applications.

16/06/09 10:31:43 INFO impl.TimelineClientImpl: Timeline service address:
http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

16/06/09 10:31:47 INFO client.ConfiguredRMFailoverProxyProvider: Failing
over to rm2

16/06/09 10:31:49 INFO impl.YarnClientImpl: Submitted application
application_1464374946035_29710

Jun 09, 2016 10:31:49 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Submitted application. Application id: application_1464374946035_29710

Jun 09, 2016 10:31:59 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Application application_1464374946035_29710 is FAILED.

On Thu, Jun 9, 2016 at 10:23 AM, Hongmei Zong  wrote:

> Thank you very much Nikolay, I will give it a try! :)
>
> Hongmei
>
> On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov 
> wrote:
>
>> You set wrong value to IGNITE_XML_CONFIG property. The property should
>> contains path to ignite configuration file. For example
>>
>> IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml
>>
>> I think you can comment this line in property file and ignite will start
>> with default configuration.
>>
>> On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:
>>
>>> Hi nikolai,
>>>
>>> Thank you very much for prompt reply!
>>>
>>> I did not find the ignite-config.xml file under my ignite home directory(
>>>  /u/hongmei/apache-ignite/  ).
>>>
>>> I find a "default-config.xml" at the path:
>>> /u/hongmei/apache-ignite/config/default-config.xml
>>>
>>> 
>>>
>>>
>>> 
>>>
>>>
>>> http://www.springframework.org/schema/beans";
>>>
>>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>>>
>>>xsi:schemaLocation="
>>>
>>>http://www.springframework.org/schema/beans
>>>
>>>http://www.springframework.org/schema/beans/spring-beans.xsd";>
>>>
>>> 
>>>
>>> >> "org.apache.ignite.configuration.IgniteConfiguration">
>>>
>>> 
>>>
>>> >> "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>>>
>>> 
>>>
>>> >> "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"
>>> >
>>>
>>> >> "addresses">
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>> c5hdpe001.c5.runwaynine.com:47500..47509
>>>
>>> 
>>> c5hdpe002.c5.runwaynine.com:47500..47509
>>>
>>> 
>>> c5hdpe003.c5.runwaynine.com:47500..47509
>>>
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>> 
>>>
>>>
>>>
>>> The "cluster.properties" at the path:
>>> /u/hongmei/apache-ignite/config/cluster.properties
>>>
>>> # The number of nodes in the cluster.
>>>
>>> IGNITE_NODE_COUNT=2
>>>
>>>
>>> # The number of CPU Cores for each Apache Ignite node.
>>>
>>> IGNITE_RUN_CPU_PER_NODE=1
>>>
>>>
>>> # The number of Megabytes of RAM for each Apache Ignite node.
>>>
>>> IGNITE_MEMORY_PER_NODE=2048
>>>
>>>
>>> # The version ignite which will be run on nodes.
>>>
>>> IGNITE_VERSION=1.6.0
>>>
>>>
>>> # The hdfs directory which will be used for saving Apache Ignite
>>> disbributives.
>>>
>>> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>>>
>>>
>>> #The directory which will be used for saving Apache Ignite
>>> distributives(copy .jar file to it).
>>>
>>> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>>>
>>>
>>> #The hdfs path to Apache Ignite config file.
>>>
>>> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>>>
>>> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>>>
>>>
>>> #The hdfs path to libs which will be added to classpath.
>>>
>>> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>>>
>>>
>>> #The constraint on slave hosts.
>>>
>>> #IGNITE_HOSTENAME_CONSTRAINT=
>>>
>>>
>>> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>>>
>>>
>>>
>>> Thank you very much
>>>
>>> Hongmei
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
>>> wrote:
>>>
 It seems that your ignite configuration invalid. Could you share 
 ignite-config.xml
 

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Thank you very much Nikolay, I will give it a try! :)

Hongmei

On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov 
wrote:

> You set wrong value to IGNITE_XML_CONFIG property. The property should
> contains path to ignite configuration file. For example
>
> IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml
>
> I think you can comment this line in property file and ignite will start
> with default configuration.
>
> On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:
>
>> Hi nikolai,
>>
>> Thank you very much for prompt reply!
>>
>> I did not find the ignite-config.xml file under my ignite home directory(
>>  /u/hongmei/apache-ignite/  ).
>>
>> I find a "default-config.xml" at the path:
>> /u/hongmei/apache-ignite/config/default-config.xml
>>
>> 
>>
>>
>> 
>>
>>
>> http://www.springframework.org/schema/beans";
>>
>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>>
>>xsi:schemaLocation="
>>
>>http://www.springframework.org/schema/beans
>>
>>http://www.springframework.org/schema/beans/spring-beans.xsd";>
>>
>> 
>>
>> > "org.apache.ignite.configuration.IgniteConfiguration">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>>
>> 
>>
>> > "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>
>> > "addresses">
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>> c5hdpe001.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe002.c5.runwaynine.com:47500..47509
>>
>> 
>> c5hdpe003.c5.runwaynine.com:47500..47509
>>
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>>
>>
>> The "cluster.properties" at the path:
>> /u/hongmei/apache-ignite/config/cluster.properties
>>
>> # The number of nodes in the cluster.
>>
>> IGNITE_NODE_COUNT=2
>>
>>
>> # The number of CPU Cores for each Apache Ignite node.
>>
>> IGNITE_RUN_CPU_PER_NODE=1
>>
>>
>> # The number of Megabytes of RAM for each Apache Ignite node.
>>
>> IGNITE_MEMORY_PER_NODE=2048
>>
>>
>> # The version ignite which will be run on nodes.
>>
>> IGNITE_VERSION=1.6.0
>>
>>
>> # The hdfs directory which will be used for saving Apache Ignite
>> disbributives.
>>
>> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>>
>>
>> #The directory which will be used for saving Apache Ignite
>> distributives(copy .jar file to it).
>>
>> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>>
>>
>> #The hdfs path to Apache Ignite config file.
>>
>> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>>
>> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>>
>>
>> #The hdfs path to libs which will be added to classpath.
>>
>> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>>
>>
>> #The constraint on slave hosts.
>>
>> #IGNITE_HOSTENAME_CONSTRAINT=
>>
>>
>> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>>
>>
>>
>> Thank you very much
>>
>> Hongmei
>>
>>
>>
>>
>>
>> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
>> wrote:
>>
>>> It seems that your ignite configuration invalid. Could you share 
>>> ignite-config.xml
>>> and /u/hongmei/apache-ignite/config/cluster.properties?
>>>
>>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>>>
 Thank you very much Nikolai !

 I found another issue regarding my Ignite YARN Integration:

 I  run Ignite YARN application on one client server machine
 'c5hdpe001', screenshot as the following:
 Then I log onto one of the container node and the log is as the
 following:

 [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
 /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
 /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
 /u/hongmei/apache-ignite/config/cluster.properties

 WARNING: Use "yarn jar" to launch YARN applications.

 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service
 address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider:
 Failing over to rm2

 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
 application_1464374946035_29511

 Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main

 INFO

Re: Ignite for Spark on YARN Deployment

[Nikolay Tikhonov]

You set wrong value to IGNITE_XML_CONFIG property. The property should
contains path to ignite configuration file. For example

IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml

I think you can comment this line in property file and ignite will start
with default configuration.

On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong  wrote:

> Hi nikolai,
>
> Thank you very much for prompt reply!
>
> I did not find the ignite-config.xml file under my ignite home directory(
>  /u/hongmei/apache-ignite/  ).
>
> I find a "default-config.xml" at the path:
> /u/hongmei/apache-ignite/config/default-config.xml
>
> 
>
>
> 
>
>
> http://www.springframework.org/schema/beans";
>
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>
>xsi:schemaLocation="
>
>http://www.springframework.org/schema/beans
>
>http://www.springframework.org/schema/beans/spring-beans.xsd";>
>
> 
>
>  "org.apache.ignite.configuration.IgniteConfiguration">
>
> 
>
>  "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>
> 
>
>  "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>  >
>
> 
>
> 
>
> 
>
> 
>
> 
> c5hdpe001.c5.runwaynine.com:47500..47509
>
> 
> c5hdpe002.c5.runwaynine.com:47500..47509
>
> 
> c5hdpe003.c5.runwaynine.com:47500..47509
>
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
>
>
> The "cluster.properties" at the path:
> /u/hongmei/apache-ignite/config/cluster.properties
>
> # The number of nodes in the cluster.
>
> IGNITE_NODE_COUNT=2
>
>
> # The number of CPU Cores for each Apache Ignite node.
>
> IGNITE_RUN_CPU_PER_NODE=1
>
>
> # The number of Megabytes of RAM for each Apache Ignite node.
>
> IGNITE_MEMORY_PER_NODE=2048
>
>
> # The version ignite which will be run on nodes.
>
> IGNITE_VERSION=1.6.0
>
>
> # The hdfs directory which will be used for saving Apache Ignite
> disbributives.
>
> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>
>
> #The directory which will be used for saving Apache Ignite
> distributives(copy .jar file to it).
>
> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>
>
> #The hdfs path to Apache Ignite config file.
>
> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>
> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>
>
> #The hdfs path to libs which will be added to classpath.
>
> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>
>
> #The constraint on slave hosts.
>
> #IGNITE_HOSTENAME_CONSTRAINT=
>
>
> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>
>
>
> Thank you very much
>
> Hongmei
>
>
>
>
>
> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
> wrote:
>
>> It seems that your ignite configuration invalid. Could you share 
>> ignite-config.xml
>> and /u/hongmei/apache-ignite/config/cluster.properties?
>>
>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>>
>>> Thank you very much Nikolai !
>>>
>>> I found another issue regarding my Ignite YARN Integration:
>>>
>>> I  run Ignite YARN application on one client server machine 'c5hdpe001',
>>> screenshot as the following:
>>> Then I log onto one of the container node and the log is as the
>>> following:
>>>
>>> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>>> /u/hongmei/apache-ignite/config/cluster.properties
>>>
>>> WARNING: Use "yarn jar" to launch YARN applications.
>>>
>>> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service
>>> address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/
>>>
>>> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
>>> over to rm2
>>>
>>> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
>>> application_1464374946035_29511
>>>
>>> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main
>>>
>>> INFO: Submitted application. Application id:
>>> application_1464374946035_29511
>>>
>>> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
>>> *INFO: Application application_1464374946035_29511 is RUNNING.*
>>>
>>>
>>> Then I log onto one of the container node and the log is as the
>>> following: "It failed to star

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi nikolai,

Thank you very much for prompt reply!

I did not find the ignite-config.xml file under my ignite home directory(
 /u/hongmei/apache-ignite/  ).

I find a "default-config.xml" at the path:
/u/hongmei/apache-ignite/config/default-config.xml







http://www.springframework.org/schema/beans";

   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";

   xsi:schemaLocation="

   http://www.springframework.org/schema/beans

   http://www.springframework.org/schema/beans/spring-beans.xsd";>
























c5hdpe001.c5.runwaynine.com:47500..47509


c5hdpe002.c5.runwaynine.com:47500..47509


c5hdpe003.c5.runwaynine.com:47500..47509




















The "cluster.properties" at the path:
/u/hongmei/apache-ignite/config/cluster.properties

# The number of nodes in the cluster.

IGNITE_NODE_COUNT=2


# The number of CPU Cores for each Apache Ignite node.

IGNITE_RUN_CPU_PER_NODE=1


# The number of Megabytes of RAM for each Apache Ignite node.

IGNITE_MEMORY_PER_NODE=2048


# The version ignite which will be run on nodes.

IGNITE_VERSION=1.6.0


# The hdfs directory which will be used for saving Apache Ignite
disbributives.

IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn


#The directory which will be used for saving Apache Ignite
distributives(copy .jar file to it).

IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir


#The hdfs path to Apache Ignite config file.

#IGNITE_XML_CONFIG=/user/yarn/ignite/

IGNITE_XML_CONFIG=/user/hongmei/ignite/


#The hdfs path to libs which will be added to classpath.

IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/


#The constraint on slave hosts.

#IGNITE_HOSTENAME_CONSTRAINT=


IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite



Thank you very much

Hongmei





On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov 
wrote:

> It seems that your ignite configuration invalid. Could you share 
> ignite-config.xml
> and /u/hongmei/apache-ignite/config/cluster.properties?
>
> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:
>
>> Thank you very much Nikolai !
>>
>> I found another issue regarding my Ignite YARN Integration:
>>
>> I  run Ignite YARN application on one client server machine 'c5hdpe001',
>> screenshot as the following:
>> Then I log onto one of the container node and the log is as the following:
>>
>> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>> /u/hongmei/apache-ignite/config/cluster.properties
>>
>> WARNING: Use "yarn jar" to launch YARN applications.
>>
>> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service address:
>> http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/
>>
>> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
>> over to rm2
>>
>> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
>> application_1464374946035_29511
>>
>> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main
>>
>> INFO: Submitted application. Application id:
>> application_1464374946035_29511
>>
>> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
>> *INFO: Application application_1464374946035_29511 is RUNNING.*
>>
>>
>> Then I log onto one of the container node and the log is as the
>> following: "It failed to start grid".
>> Any good suggestion? Thank you very much!!!
>>
>>
>> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
>> application context
>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
>> err=Line 1 in XML document from URL
>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
>> 1; columnNumber: 1; Content is not allowed in prolog.]
>> at
>> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904)
>> at org.apache.

Re: Ignite for Spark on YARN Deployment

[Nikolai Tikhonov]

It seems that your ignite configuration invalid. Could you share
ignite-config.xml
and /u/hongmei/apache-ignite/config/cluster.properties?

On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong  wrote:

> Thank you very much Nikolai !
>
> I found another issue regarding my Ignite YARN Integration:
>
> I  run Ignite YARN application on one client server machine 'c5hdpe001',
> screenshot as the following:
> Then I log onto one of the container node and the log is as the following:
>
> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/config/cluster.properties
>
> WARNING: Use "yarn jar" to launch YARN applications.
>
> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service address:
> http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/
>
> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
> over to rm2
>
> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
> application_1464374946035_29511
>
> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main
>
> INFO: Submitted application. Application id:
> application_1464374946035_29511
>
> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
> *INFO: Application application_1464374946035_29511 is RUNNING.*
>
>
> Then I log onto one of the container node and the log is as the following:
> "It failed to start grid".
> Any good suggestion? Thank you very much!!!
>
>
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
> application context
> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
> err=Line 1 in XML document from URL
> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
> 1; columnNumber: 1; Content is not allowed in prolog.]
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904)
> at org.apache.ignite.Ignition.start(Ignition.java:350)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
> *Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> instantiate Spring XML application context
> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
> err=Line 1 in XML document from URL
> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
> 1; columnNumber: 1; Content is not allowed in prolog.]*
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
> at
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556)
> at org.apache.ignite.Ignition.start(Ignition.java:347)
> ... 1 more
> *Caused by:
> org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line
> 1 in XML document from URL
> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
> 1; columnNumber: 1; Content is not allowed in prolog.*
> at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398)
> at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335)
> at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)
> at
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379)
> ... 9 more
> *Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1;
> Content is not allowed in pr

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Thank you very much Nikolai !

I found another issue regarding my Ignite YARN Integration:

I  run Ignite YARN application on one client server machine 'c5hdpe001',
screenshot as the following:
Then I log onto one of the container node and the log is as the following:

[hongmei@c5hdpe001 apache-ignite]$ hadoop jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
/u/hongmei/apache-ignite/config/cluster.properties

WARNING: Use "yarn jar" to launch YARN applications.

16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service address:
http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/

16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
over to rm2

16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
application_1464374946035_29511

Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main

INFO: Submitted application. Application id: application_1464374946035_29511

Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
*INFO: Application application_1464374946035_29511 is RUNNING.*


Then I log onto one of the container node and the log is as the following:
"It failed to start grid".
Any good suggestion? Thank you very much!!!


class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
application context
[springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
err=Line 1 in XML document from URL
[file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
1; columnNumber: 1; Content is not allowed in prolog.]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904)
at org.apache.ignite.Ignition.start(Ignition.java:350)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
*Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
instantiate Spring XML application context
[springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
err=Line 1 in XML document from URL
[file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
1; columnNumber: 1; Content is not allowed in prolog.]*
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
at
org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556)
at org.apache.ignite.Ignition.start(Ignition.java:347)
... 1 more
*Caused by:
org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line
1 in XML document from URL
[file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
1; columnNumber: 1; Content is not allowed in prolog.*
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398)
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335)
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)
at
org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379)
... 9 more
*Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1;
Content is not allowed in prolog.*
at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
at
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
 

Re: Ignite for Spark on YARN Deployment

[Nikolai Tikhonov]

>
> *your_address1:47500..47510,your_address2:47500..47510
> and your_address3:47500..47510 are the YARN master_host address, right?*
>

No, this addresses hosts on which deploy YARN cluster. For example, you
have YARN cluster which contains two servers: 10.0.0.1 and 10.0.0.2. In
this case you will have the following configuration:

ipFinder.setAddresses(Arrays.asList("10.0.0.1:47500..47510", "10.0.0.2:47500
..47510"));

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Nikolai,

Thank you very much for your solution! I would like to double check with you 
about my address1…...

ipFinder.setAddresses(Arrays.asList("your_address1:47500..47510", 
"your_address2:47500..47510",
> 
> "your_address3:47500..47510"));
your_address1:47500..47510,your_address2:47500..47510 and 
your_address3:47500..47510 are the YARN master_host address, right?

Is that necessary to install Ignite on these three YARN master_host? I guess 
the answer is “NO”.
Is the the master_port should be 47500 or 47510?

Sorry for these questions, I am new to Ignite and YARN framework.
Thanks again!!!

Hongmei


> On Jun 8, 2016, at 12:30 PM, Nikolai Tikhonov  wrote:
> 
> Hi Hongmei Zong!
> 
> Client node which started from IgniteContext can't to find server nodes. By 
> default ignite integration with YARN uses TcpDiscoveryVmIpFinder (if you 
> don't use another ip finder in your configuration).
> In the case you should set the ip finder in IgniteContext. The following code 
> snippet shows how do it on Java:
> 
> IgniteConfiguration cfg = new IgniteConfiguration();
> 
> TcpDiscoverySpi spi = new TcpDiscoverySpi();
> 
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
> ipFinder.setAddresses(Arrays.asList("your_address1:47500..47510", 
> "your_address2:47500..47510",
> "your_address3:47500..47510"));
> 
> spi.setIpFinder(ipFinder);
> 
> cfg.setDiscoverySpi(spi);
> 
> On Wed, Jun 8, 2016 at 6:31 PM, Hongmei Zong  > wrote:
> Hi Denis,
> 
> I tried testing Ignite as the following steps:
> 
> Background information: 
> 1. Our Spark is running on YARN deployment; There are three Master hosts and 
> many Worker nodes and three client nodes in the Spark clusters.
> 2. I installed Ignite on one of the client node and can launch Ignite-shell 
> locally on this client node. Since for testing purpose, I did not install 
> Ignite on any master nodes or worker nodes.
> 3. I logged into the client node which Ignite was installed and launched 
> Ignite YARN application using the following command:
> 
> hadoop jar 
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar 
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar 
> /u/hongmei/apache-ignite/config/cluster.properties
> 
> I open the UI console for the Ignite YARN application, and it shows that 
> Ignite YARN is running, Containers, CPU Cores, Memory are all allocated by 
> YARN. screenshot is as the following 
> 
> application_1464374946035_26956 
> 
>   hongmei ignitionYARNdefault Wed Jun 8 10:21:04 -0400 2016   N/A 
> RUNNING UNDEFINED   12  12  34816   9.6 3.8 
> ApplicationMaster 
> 
> 
> 4. I open an other terminal log into the same client node and run the Ignite 
> for Spark Shell command as the following:
> The Spark shell started
> 
> /usr/bin/spark-shell --jars 
> /u/hongmei/apache-ignite/libs/ignite-core-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-spark/ignite-spark-1.6.0.jar,/u/hongmei/apache-ignite/libs/cache-api-1.0.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/ignite-log4j-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/log4j-1.2.17.jar
>  --packages 
> org.apache.ignite:ignite-spark:1.6.0,org.apache.ignite:ignite-spring:1.6.0
> 
> The Spark shell launched successfully and I use these two commands to import 
> the ignite spark files:
> import org.apache.ignite.spark._
> import org.apache.ignite.configuration._
> 
> Next I create an instance of Ignite context as the following syntax:
> val ic = new IgniteContext[Integer, Integer](sc, () => new 
> IgniteConfiguration())
> 
> I got the following message: So I stuck at this point.
> 
> 16/06/08 10:28:27 WARN TcpDiscoverySpi: IP finder returned empty addresses 
> list. Please check IP finder configuration and make sure multicast works on 
> your network. Will retry every 2 secs.
> 
> So I stuck at this point.
> 
> The other scenario when I run command listed as Step 4. The Spark shell can 
> not be launched successfully with the status of “Accept”, but never get 
> chance to run.
> 
> Any good suggestions?? Is there anything wrong with my test procedure??
> Thanks in advance! 
> 
> Mei
> 
> 
>> On Jun 7, 2016, at 10:26 AM, Denis Magda > > wrote:
>> 
>> Hi,
>> 
>> I’m not an expert in this area however have you tried to specify a Spark 
>> master like the following documentation says?
>> https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell#working-with-spark-shell
>>  
>> 
>> 
>> If you did try please share the full logs, someone from the community will 
>> respond.
>> 
>> —
>> Denis
>> 
>>> On Jun 6, 2016, at 5:58 PM, Hongmei Zong 

Re: Ignite for Spark on YARN Deployment

[Nikolai Tikhonov]

Hi Hongmei Zong!

Client node which started from IgniteContext can't to find server nodes. By
default ignite integration with YARN uses TcpDiscoveryVmIpFinder (if you
don't use another ip finder in your configuration).
In the case you should set the ip finder in IgniteContext. The following
code snippet shows how do it on Java:

IgniteConfiguration cfg = new IgniteConfiguration();

TcpDiscoverySpi spi = new TcpDiscoverySpi();

TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
ipFinder.setAddresses(Arrays.asList("your_address1:47500..47510",
"your_address2:47500..47510",
"your_address3:47500..47510"));

spi.setIpFinder(ipFinder);

cfg.setDiscoverySpi(spi);


On Wed, Jun 8, 2016 at 6:31 PM, Hongmei Zong  wrote:

> Hi Denis,
>
> I tried testing Ignite as the following steps:
>
> Background information:
> 1. Our Spark is running on YARN deployment; There are three Master hosts
> and many Worker nodes and three client nodes in the Spark clusters.
> 2. I installed Ignite on one of the client node and can launch
> Ignite-shell locally on this client node. Since for testing purpose, I did
> not install Ignite on any master nodes or worker nodes.
> 3. I logged into the client node which Ignite was installed and launched
> Ignite YARN application using the following command:
>
> hadoop jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/config/cluster.properties
>
> I open the UI console for the Ignite YARN application, and it shows that
> Ignite YARN is running, Containers, CPU Cores, Memory are all allocated by
> YARN. screenshot is as the following
>
> application_1464374946035_26956
> 
> hongmei ignition YARN default Wed Jun 8 10:21:04 -0400 2016 N/A RUNNING
> UNDEFINED 12 12 34816 9.6 3.8
> ApplicationMaster
> 
>
> 4. I open an other terminal log into the same client node and run the
> Ignite for Spark Shell command as the following:
> The Spark shell started
>
> /usr/bin/spark-shell --jars /u/hongmei/apache-ignite/libs/
> ignite-core-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-spark/
> ignite-spark-1.6.0.jar,/u/hongmei/apache-ignite/libs/cache-api-1.0.0.jar,
> /u/hongmei/apache-ignite/libs/optional/ignite-log4j/
> ignite-log4j-1.6.0.jar,
> /u/hongmei/apache-ignite/libs/optional/ignite-log4j/log4j-1.2.17.jar
> --packages
> org.apache.ignite:ignite-spark:1.6.0,org.apache.ignite:ignite-spring:1.6.0
>
> The Spark shell launched successfully and I use these two commands
> to import the ignite spark files:
>
> import org.apache.ignite.spark._import org.apache.ignite.configuration._
>
>
> Next I create an instance of Ignite context as the following syntax:
>
> val ic = new IgniteContext[Integer, Integer](sc, () => new 
> IgniteConfiguration())
>
>
> I got the following message: So I stuck at this point.
>
> 16/06/08 10:28:27 WARN TcpDiscoverySpi: IP finder returned empty addresses
> list. Please check IP finder configuration and make sure multicast works on
> your network. Will retry every 2 secs.
>
> So I stuck at this point.
>
> The other scenario when I run command listed as Step 4. The Spark shell
> can not be launched successfully with the status of “Accept”, but never get
> chance to run.
>
> Any good suggestions?? Is there anything wrong with my test procedure??
> Thanks in advance!
>
> Mei
>
>
> On Jun 7, 2016, at 10:26 AM, Denis Magda  wrote:
>
> Hi,
>
> I’m not an expert in this area however have you tried to specify a Spark
> master like the following documentation says?
>
> https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell#working-with-spark-shell
>
> If you did try please share the full logs, someone from the community will
> respond.
>
> —
> Denis
>
> On Jun 6, 2016, at 5:58 PM, Hongmei Zong  wrote:
>
> Hi there,
>
> I would like to use "Ignite for Spark" to save the states of Spark jobs in
> memory and those states can be used for later jobs. For Shared Deployment,
> the document only offer two ways to deploy Ignite cluster. First is the
> standalone deployment, second is MESOS deployment. But Our Spark clusters
> are running on YARN. My question is: is it possible to run Ignite for Spark
> on YARN deployment???
>
> I downloaded and installed Ignite on my machine. Next, I referenced the
> link
> below for YARN Deployment.
> http://apacheignite.gridgain.org/docs/yarn-deployment
>
> I created the cluster.properties file and ran the application using the
> command:hadoop jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/config/cluster.properties
>
> Form the YARN console, The YARN ignite application works ok. It shows
> running

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Denis,

I tried testing Ignite as the following steps:

Background information: 
1. Our Spark is running on YARN deployment; There are three Master hosts and 
many Worker nodes and three client nodes in the Spark clusters.
2. I installed Ignite on one of the client node and can launch Ignite-shell 
locally on this client node. Since for testing purpose, I did not install 
Ignite on any master nodes or worker nodes.
3. I logged into the client node which Ignite was installed and launched Ignite 
YARN application using the following command:

hadoop jar 
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar 
/u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar 
/u/hongmei/apache-ignite/config/cluster.properties

I open the UI console for the Ignite YARN application, and it shows that Ignite 
YARN is running, Containers, CPU Cores, Memory are all allocated by YARN. 
screenshot is as the following 

application_1464374946035_26956 

hongmei ignitionYARNdefault Wed Jun 8 10:21:04 -0400 2016   N/A 
RUNNING UNDEFINED   12  12  34816   9.6 3.8 
ApplicationMaster 


4. I open an other terminal log into the same client node and run the Ignite 
for Spark Shell command as the following:
The Spark shell started

/usr/bin/spark-shell --jars 
/u/hongmei/apache-ignite/libs/ignite-core-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-spark/ignite-spark-1.6.0.jar,/u/hongmei/apache-ignite/libs/cache-api-1.0.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/ignite-log4j-1.6.0.jar,/u/hongmei/apache-ignite/libs/optional/ignite-log4j/log4j-1.2.17.jar
 --packages 
org.apache.ignite:ignite-spark:1.6.0,org.apache.ignite:ignite-spring:1.6.0

The Spark shell launched successfully and I use these two commands to import 
the ignite spark files:
import org.apache.ignite.spark._
import org.apache.ignite.configuration._

Next I create an instance of Ignite context as the following syntax:
val ic = new IgniteContext[Integer, Integer](sc, () => new 
IgniteConfiguration())

I got the following message: So I stuck at this point.

16/06/08 10:28:27 WARN TcpDiscoverySpi: IP finder returned empty addresses 
list. Please check IP finder configuration and make sure multicast works on 
your network. Will retry every 2 secs.

So I stuck at this point.

The other scenario when I run command listed as Step 4. The Spark shell can not 
be launched successfully with the status of “Accept”, but never get chance to 
run.

Any good suggestions?? Is there anything wrong with my test procedure??
Thanks in advance! 

Mei


> On Jun 7, 2016, at 10:26 AM, Denis Magda  wrote:
> 
> Hi,
> 
> I’m not an expert in this area however have you tried to specify a Spark 
> master like the following documentation says?
> https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell#working-with-spark-shell
> 
> If you did try please share the full logs, someone from the community will 
> respond.
> 
> —
> Denis
> 
>> On Jun 6, 2016, at 5:58 PM, Hongmei Zong  wrote:
>> 
>> Hi there,
>> 
>> I would like to use "Ignite for Spark" to save the states of Spark jobs in
>> memory and those states can be used for later jobs. For Shared Deployment,
>> the document only offer two ways to deploy Ignite cluster. First is the
>> standalone deployment, second is MESOS deployment. But Our Spark clusters
>> are running on YARN. My question is: is it possible to run Ignite for Spark
>> on YARN deployment???
>> 
>> I downloaded and installed Ignite on my machine. Next, I referenced the link
>> below for YARN Deployment.
>> http://apacheignite.gridgain.org/docs/yarn-deployment
>> 
>> I created the cluster.properties file and ran the application using the
>> command:hadoop jar
>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>> /u/hongmei/apache-ignite/config/cluster.properties 
>> 
>> Form the YARN console, The YARN ignite application works ok. It shows
>> running, and 16 containers are allocated for the ignite application.
>> 
>> After this step, what should I do in order to run Spark with Ignite on YARN
>> deployment??
>> 
>> Many Thanks!!!
>> 
>> Mei
>> 
>> 
>> 
>> --
>> View this message in context: 
>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp5465.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> 

Re: Ignite for Spark on YARN Deployment

[Hongmei Zong]

Hi Denis,

Thank you very much for your good suggestions! If my Spark running on YARN
and there are three master_hosts running, should I pick up one randomly? I
will give a try to specify the master_host and master_port.

Thank you!
Mei

On Tue, Jun 7, 2016 at 10:26 AM, Denis Magda  wrote:

> Hi,
>
> I’m not an expert in this area however have you tried to specify a Spark
> master like the following documentation says?
>
> https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell#working-with-spark-shell
>
> If you did try please share the full logs, someone from the community will
> respond.
>
> —
> Denis
>
> > On Jun 6, 2016, at 5:58 PM, Hongmei Zong  wrote:
> >
> > Hi there,
> >
> > I would like to use "Ignite for Spark" to save the states of Spark jobs
> in
> > memory and those states can be used for later jobs. For Shared
> Deployment,
> > the document only offer two ways to deploy Ignite cluster. First is the
> > standalone deployment, second is MESOS deployment. But Our Spark clusters
> > are running on YARN. My question is: is it possible to run Ignite for
> Spark
> > on YARN deployment???
> >
> > I downloaded and installed Ignite on my machine. Next, I referenced the
> link
> > below for YARN Deployment.
> > http://apacheignite.gridgain.org/docs/yarn-deployment
> >
> > I created the cluster.properties file and ran the application using the
> > command:hadoop jar
> > /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> > /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> > /u/hongmei/apache-ignite/config/cluster.properties
> >
> > Form the YARN console, The YARN ignite application works ok. It shows
> > running, and 16 containers are allocated for the ignite application.
> >
> > After this step, what should I do in order to run Spark with Ignite on
> YARN
> > deployment??
> >
> > Many Thanks!!!
> >
> > Mei
> >
> >
> >
> > --
> > View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp5465.html
> > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
>

Re: Ignite for Spark on YARN Deployment

[Denis Magda]

Hi,

I’m not an expert in this area however have you tried to specify a Spark master 
like the following documentation says?
https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell#working-with-spark-shell

If you did try please share the full logs, someone from the community will 
respond.

—
Denis

> On Jun 6, 2016, at 5:58 PM, Hongmei Zong  wrote:
> 
> Hi there,
> 
> I would like to use "Ignite for Spark" to save the states of Spark jobs in
> memory and those states can be used for later jobs. For Shared Deployment,
> the document only offer two ways to deploy Ignite cluster. First is the
> standalone deployment, second is MESOS deployment. But Our Spark clusters
> are running on YARN. My question is: is it possible to run Ignite for Spark
> on YARN deployment???
> 
> I downloaded and installed Ignite on my machine. Next, I referenced the link
> below for YARN Deployment.
> http://apacheignite.gridgain.org/docs/yarn-deployment
> 
> I created the cluster.properties file and ran the application using the
> command:hadoop jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
> /u/hongmei/apache-ignite/config/cluster.properties 
> 
> Form the YARN console, The YARN ignite application works ok. It shows
> running, and 16 containers are allocated for the ignite application.
> 
> After this step, what should I do in order to run Spark with Ignite on YARN
> deployment??
> 
> Many Thanks!!!
> 
> Mei
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp5465.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.