1、Adjusted cluster.properites as below
$cat cluster16.properties
# The number of nodes in the cluster.
IGNITE_NODE_COUNT=4
# The number of CPU Cores for each Apache Ignite node.
IGNITE_RUN_CPU_PER_NODE=1
# The number of Megabytes of RAM for each Apache Ignite node.
IGNITE_MEMORY_PER_NODE=2048
# The version ignite which will be run on nodes.
IGNITE_VERSION=1.6.0
IGNITE_WORK_DIR=/u01/yueyi/apache-ignite-fabric-1.6.0-bin/
IGNITE_XML_CONFIG=/ignite/release16/apache-ignite-fabric-1.6.0-bin/config/default-config.xml
IGNITE_RELEASES_DIR=/ignite/release16/
#IGNITE_USERS_LIBS=/u01/yueyi/apache-ignite-fabric-1.6.0-bin/libs/
#IGNITE_HOSTNAME_CONSTRAINT=vmsecdomain010194070026.cm10
IGNITE_PATH=/ignite/release16/apache-ignite-fabric-1.6.0-bin.zip
2、hdfs directory
$./hdfs dfs -ls /ignite/release16
drwxr-xr-x - hbase hbase 0 2016-08-11 12:44
/ignite/release16/apache-ignite-fabric-1.6.0-bin
-rw-r--r-- 3 hbase hbase 175866626 2016-08-11 18:31
/ignite/release16/apache-ignite-fabric-1.6.0-bin.zip
3、yarn console
INFO: Application master registered.
Aug 11, 2016 6:32:57 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Aug 11, 2016 6:32:57 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Aug 11, 2016 6:32:57 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
Aug 11, 2016 6:32:57 PM org.apache.ignite.yarn.ApplicationMaster run
INFO: Making request. Memory: 2,432, cpu 1.
16/08/11 18:32:57 INFO impl.AMRMClientImpl: Received new token for :
xx1:29077
16/08/11 18:32:57 INFO impl.AMRMClientImpl: Received new token for :
xxx2:57492
16/08/11 18:32:57 INFO impl.AMRMClientImpl: Received new token for :
xxx3:59929
16/08/11 18:32:57 INFO impl.AMRMClientImpl: Received new token for :
xx4:23159
5、contain logs as below
1)
[18:33:13] Security status [authentication=off, tls/ssl=off]
[18:33:18] To start Console Management & Monitoring run
ignitevisorcmd.{sh|bat}
[18:33:18]
[18:33:18] Ignite node started OK (id=a060a3ee)
[18:33:18] Topology snapshot [ver=3, servers=3, clients=0, CPUs=72,
heap=6.0GB]
[18:33:19] Topology snapshot [ver=4, servers=4, clients=0, CPUs=96,
heap=8.0GB]
2)[18:33:13] Security status [authentication=off, tls/ssl=off]
[18:33:19] To start Console Management & Monitoring run
ignitevisorcmd.{sh|bat}
[18:33:19]
[18:33:19] Ignite node started OK (id=5c8dfd50)
[18:33:19] Topology snapshot [ver=4, servers=4, clients=0, CPUs=96,
heap=8.0GB]
3)[18:33:12] Ignite node started OK (id=4e75d238)
[18:33:12] Topology snapshot [ver=1, servers=1, clients=0, CPUs=24,
heap=2.0GB]
[18:33:14] Topology snapshot [ver=2, servers=2, clients=0, CPUs=48,
heap=4.0GB]
[18:33:17] Topology snapshot [ver=3, servers=3, clients=0, CPUs=72,
heap=6.0GB]
[18:33:19] Topology snapshot [ver=4, servers=4, clients=0, CPUs=96,
heap=8.0GB]
4)[18:33:14] Ignite node started OK (id=250fcb93)
[18:33:14] Topology snapshot [ver=2, servers=2, clients=0, CPUs=48,
heap=4.0GB]
[18:33:17] Topology snapshot [ver=3, servers=3, clients=0, CPUs=72,
heap=6.0GB]
[18:33:19] Topology snapshot [ver=4, servers=4, clients=0, CPUs=96,
heap=8.0GB]
faced issues=========
1、spark-shell test
./spark-shell --jars
/u01/xxx/apache-ignite-hadoop-1.6.0-bin/libs/ignite-core-1.6.0.jar,/u01/xxx/apache-ignite-hadoop-1.6.0-bin/libs/ignite-spark/ignite-spark-1.6.0.jar,/u01/xxx/apache-ignite-hadoop-1.6.0-bin/libs/cache-api-1.0.0.jar,/u01/xxx/apache-ignite-hadoop-1.6.0-bin/libs/ignite-log4j/ignite-log4j-1.6.0.jar,/u01/xxx/apache-ignite-hadoop-1.6.0-bin/libs/ignite-log4j/log4j-1.2.17.jar
--packages
org.apache.ignite:ignite-spark:1.6.0,org.apache.ignite:ignite-spring:1.6.0
2、SQL context available as sqlContext.
scala> import org.apache.ignite.spark._
import org.apache.ignite.spark._
scala> import org.apache.ignite.configuration._
import org.apache.ignite.configuration._
scala> val ic = new IgniteContext[Integer, Integer](sc, () => new
IgniteConfiguration())
the spark-shell hang here........
Thanks again!!!!
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp6910p6961.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
