Hi Ivan, Thanks a lot for very useful guidelines! Using your explanations I could run Ignite Map-Reduce on my cluster but only twice (next attempts failed with various errors) and result was unexpected. I run simle query /select calday, count(*) from price.toprice where calday between '2017-03-20' and '2017-03-21' group by calday order by calday;/ and in works twice slowly than TEZ (40 sec vs 22 sec) and what is more query result is wrong. Expected result is 37mln records per day, but Ignite MR calculated only 12 mln records per day. *Why?*
I run one ignite node on every cluster node and I set in Ambari /mapreduce.jobtracker.address=127.0.0.1:11211;/ But query runs only when I set in beeline /set mapreduce.jobtracker.address=dev-dn1:11211;/ My next attempts to run the same query failde with different errors: • Caused by: java.lang.OutOfMemoryError: Metaspace • java.lang.OutOfMemoryError: GC overhead limit exceeded • [ERROR][pool-2-thread-1][HadoopJobTracker] Unhandled exception while processing event. java.lang.NullPointerException *How to tune Ignite MR that it works stable?* If you do not mind I ask a few more questions here: 1. Where can I find an example of default-config.xml with all possible parameters with default values? I just copypast some example from this forum, but it is not complete and with some mistakes. 2. Am I right that the IGFS can only be used if the cluster is not HA? Otherwise, HDFS does not start because fs.default.name=igfs://myhost:10500/ 3. When I run Hive query with Ignite MR this is not YARN application. Is it possible to run Hive queries on Ignite as YARN applications? 4. Does Ignite take advantage of the ORC files (built-in indexes, statistics)? 5. Suppose compressed ORC file is 500Mb, the same uncompressed csv-file is 9Gb. How much memory Ignite needs to work with this file? 6. What is right approach for Ignite MR: either run one ignite-node on every cluster node or run as many ignite-nodes on one cluster node as possible? 7. I stopped one ignite-node during the query execution then job (query execution) failed. Is it normal? Thanks in advance! -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12238.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
