Hi Ivan, 1. Need more time for experiments... 3. Yes, logs are full. I started every node with this command: $IGNITE_HOME/bin/ignite.sh -v -J"-Xms10g -Xmx10g -XX:MaxMetaspaceSize=4g" 2>&1 | tee /home/ignite/apache-ignite-hadoop-1.9.0-bin/work/log/ignite-node-dnX.log I thought ver=72 because I did 72 attempts)) I don't know how to reset this counter. I stop ignite nodes Ctrl-C or Ctrl-Z (then kill PID) or in Visor kill -k. Look, there is no Ignite process, but now ver=109: *[ignite@dev-dn1 ~]$ ps -ef | grep ignite* ignite 6796 21325 0 15:04 pts/5 00:00:00 ps -ef ignite 6797 21325 0 15:04 pts/5 00:00:00 grep ignite root 21324 21290 0 May02 pts/5 00:00:00 su - ignite ignite 21325 21324 0 May02 pts/5 00:00:00 -bash root 27287 17525 0 May02 pts/1 00:00:00 su - ignite ignite 27288 27287 0 May02 pts/1 00:00:00 -bash *[ignite@dev-dn1 ~]$ $IGNITE_HOME/bin/ignite.sh -v -J"-Xms10g -Xmx10g -XX:MaxMetaspaceSize=4g" 2>&1 | tee /home/ignite/apache-ignite-hadoop-1.9.0-bin/work/log/ignite-node.log* Ignite Command Line Startup, ver. 1.9.0#20170302-sha1:a8169d0a 2017 Copyright(C) Apache Software Foundation
[15:04:08,630][INFO ][main][IgniteKernal] >>> __________ ________________ >>> / _/ ___/ |/ / _/_ __/ __/ >>> _/ // (7 7 // / / / / _/ >>> /___/\___/_/|_/___/ /_/ /___/ >>> >>> ver. 1.9.0#20170302-sha1:a8169d0a >>> 2017 Copyright(C) Apache Software Foundation >>> >>> Ignite documentation: http://ignite.apache.org [15:04:08,630][INFO ][main][IgniteKernal] Config URL: file:/home/ignite/apache-ignite-hadoop-1.9.0-bin/config/default-config.xml [15:04:08,630][INFO ][main][IgniteKernal] Daemon mode: off [15:04:08,631][INFO ][main][IgniteKernal] OS: Linux 2.6.32-696.el6.x86_64 amd64 [15:04:08,631][INFO ][main][IgniteKernal] OS user: ignite [15:04:08,631][INFO ][main][IgniteKernal] PID: 6891 [15:04:08,631][INFO ][main][IgniteKernal] Language runtime: Java Platform API Specification ver. 1.8 [15:04:08,631][INFO ][main][IgniteKernal] VM information: Java(TM) SE Runtime Environment 1.8.0_101-b13 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.101-b13 [15:04:08,633][INFO ][main][IgniteKernal] VM total memory: 9.6GB [15:04:08,633][INFO ][main][IgniteKernal] Remote Management [restart: on, REST: on, JMX (remote: on, port: 49199, auth: off, ssl: off)] [15:04:08,633][INFO ][main][IgniteKernal] IGNITE_HOME=/home/ignite/apache-ignite-hadoop-1.9.0-bin [15:04:08,633][INFO ][main][IgniteKernal] VM arguments: [-Xms1g, -Xmx1g, -XX:+AggressiveOpts, -XX:MaxMetaspaceSize=256m, -Djava.library.path=/usr/hdp/current/hadoop-client/lib/native/, -DIGNITE_QUIET=false, -DIGNITE_SUCCESS_FILE=/home/ignite/apache-ignite-hadoop-1.9.0-bin/work/ignite_success_6a954010-244e-42bb-9cf7-b4fbbf39519a, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.port=49199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -DIGNITE_HOME=/home/ignite/apache-ignite-hadoop-1.9.0-bin, -DIGNITE_PROG_NAME=/home/ignite/apache-ignite-hadoop-1.9.0-bin/bin/ignite.sh, -Xms10g, -Xmx10g, -XX:MaxMetaspaceSize=4g] [15:04:08,634][INFO ][main][IgniteKernal] Configured caches ['ignite-marshaller-sys-cache', 'ignite-sys-cache', 'ignite-hadoop-mr-sys-cache', 'ignite-atomics-sys-cache', 'igfs-meta', 'igfs-data'] [15:04:08,638][INFO ][main][IgniteKernal] 3-rd party licenses can be found at: /home/ignite/apache-ignite-hadoop-1.9.0-bin/libs/licenses [15:04:08,725][INFO ][main][IgnitePluginProcessor] Configured plugins: [15:04:08,725][INFO ][main][IgnitePluginProcessor] ^-- None [15:04:08,725][INFO ][main][IgnitePluginProcessor] [15:04:08,786][INFO ][main][TcpCommunicationSpi] Successfully bound communication NIO server to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0, selectorsCnt=4, selectorSpins=0, pairedConn=false] [15:04:08,790][WARN ][main][TcpCommunicationSpi] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides. [15:04:08,810][WARN ][main][NoopCheckpointSpi] Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation) [15:04:08,842][WARN ][main][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival). [15:04:08,846][WARN ][main][NoopSwapSpaceSpi] Swap space is disabled. To enable use FileSwapSpaceSpi. [15:04:08,847][INFO ][main][IgniteKernal] Security status [authentication=off, tls/ssl=off] [15:04:09,292][INFO ][main][GridTcpRestProtocol] Command protocol successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211] [15:04:09,718][INFO ][main][IpcServerTcpEndpoint] IPC server loopback endpoint started [port=10500] [15:04:09,720][INFO ][main][IpcServerTcpEndpoint] IPC server loopback endpoint started [port=11400] [15:04:09,729][INFO ][main][HadoopProcessor] HADOOP_HOME is set to /usr/hdp/current/hadoop-client [15:04:09,730][INFO ][main][HadoopProcessor] Resolved Hadoop classpath locations: /usr/hdp/2.6.0.3-8/hadoop, /usr/hdp/current/hadoop-hdfs-client/, /usr/hdp/current/hadoop-mapreduce-client/ [15:04:09,756][INFO ][main][IgniteKernal] Non-loopback local IPs: 10.255.242.96, fe80:0:0:0:21c:c4ff:fe5b:663c%bond0 [15:04:09,756][INFO ][main][IgniteKernal] Enabled local MACs: 001CC45B663C [15:04:09,802][INFO ][main][TcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=0.0.0.0/0.0.0.0, locNodeId=65d7c352-5a5c-44c2-a4ac-852e98abb46a] [15:04:10,387][INFO ][main][GridCacheProcessor] Started cache [name=igfs-data, mode=PARTITIONED] [15:04:10,406][INFO ][main][GridCacheProcessor] Started cache [name=ignite-hadoop-mr-sys-cache, mode=REPLICATED] [15:04:10,408][INFO ][main][GridCacheProcessor] Started cache [name=ignite-sys-cache, mode=REPLICATED] [15:04:10,410][INFO ][main][GridCacheProcessor] Started cache [name=ignite-atomics-sys-cache, mode=PARTITIONED] [15:04:10,414][INFO ][main][GridCacheProcessor] Started cache [name=igfs-meta, mode=REPLICATED] [15:04:10,438][INFO ][main][GridCacheProcessor] Started cache [name=ignite-marshaller-sys-cache, mode=REPLICATED] [15:04:10,582][INFO ][exchange-worker-#33%null%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=109, minorTopVer=0], evt=NODE_JOINED, node=65d7c352-5a5c-44c2-a4ac-852e98abb46a] [15:04:10,667][INFO ][main][IgniteKernal] Performance suggestions for grid (fix if possible) [15:04:10,667][INFO ][main][IgniteKernal] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true [15:04:10,668][INFO ][main][IgniteKernal] ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options) [15:04:10,668][INFO ][main][IgniteKernal] ^-- Set max direct memory size if getting 'OOME: Direct buffer memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options) [15:04:10,668][INFO ][main][IgniteKernal] ^-- Disable processing of calls to System.gc() (add '-XX:+DisableExplicitGC' to JVM options) [15:04:10,668][INFO ][main][IgniteKernal] ^-- Speed up flushing of dirty pages by OS (alter vm.dirty_expire_centisecs parameter by setting to 500) [15:04:10,668][INFO ][main][IgniteKernal] ^-- Reduce pages swapping ratio (set vm.swappiness=10) [15:04:10,668][INFO ][main][IgniteKernal] ^-- Avoid direct reclaim and page allocation failures (set vm.extra_free_kbytes=1240000) [15:04:10,668][INFO ][main][IgniteKernal] Refer to this page for more performance suggestions: https://apacheignite.readme.io/docs/jvm-and-system-tuning [15:04:10,668][INFO ][main][IgniteKernal] [15:04:10,668][INFO ][main][IgniteKernal] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat} [15:04:10,669][INFO ][main][IgniteKernal] [15:04:10,669][INFO ][main][IgniteKernal] >>> +----------------------------------------------------------------------+ >>> Ignite ver. 1.9.0#20170302-sha1:a8169d0ac448b8f19b8cdc51ee6f339ee6864c29 >>> +----------------------------------------------------------------------+ >>> OS name: Linux 2.6.32-696.el6.x86_64 amd64 >>> CPU(s): 8 >>> Heap: 10.0GB >>> VM name: 6...@dev-dn1.co.vectis.local >>> Local node [ID=65D7C352-5A5C-44C2-A4AC-852E98ABB46A, order=109, >>> clientMode=false] >>> Local node addresses: [dev-dn1.co.vectis.local/0:0:0:0:0:0:0:1%lo, >>> /10.255.242.96, /127.0.0.1] >>> Local ports: TCP:10500 TCP:11211 TCP:11400 TCP:47100 TCP:47500 [15:04:10,671][INFO ][main][GridDiscoveryManager] Topology snapshot [*ver=109*, servers=1, clients=0, CPUs=8, heap=10.0GB] [15:04:20,047][INFO ][ignite-update-notifier-timer][GridUpdateNotifier] Your version is up to date. [15:05:10,669][INFO ][grid-timeout-worker-#15%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=65d7c352, name=null, uptime=00:01:00:009] ^-- H/N/C [hosts=1, nodes=1, CPUs=8] ^-- CPU [cur=0.1%, avg=0.64%, GC=0%] ^-- Heap [used=631MB, free=93.57%, comm=9813MB] ^-- Non heap [used=49MB, free=99.08%, comm=50MB] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=6, qSize=0] ^-- Outbound messages queue [size=0] 4. Evgeniy said (I found it in email): Alena, i think you can`t obtain any significant results here, look tez takes all data into memory and runs DAGs with no flush to disk, using IGFS here you simply append additional memory burden here. TEZ standalone is good enough. Ivan, does Ignite have some tests in case of TEZ usage ? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12419.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.