Hi Ivan,

Thanks a lot for very useful guidelines! Using your explanations I could run
Ignite Map-Reduce on my cluster but only twice (next attempts failed with
various errors) and result was unexpected.
I run simle query 
/select calday, count(*) from price.toprice where calday between
'2017-03-20' and '2017-03-21' group by calday order by calday;/
and in works twice slowly than TEZ (40 sec vs 22 sec) and what is more query
result is wrong. Expected result is 37mln records per day, but Ignite MR
calculated only 12 mln records per day. *Why?*

I run one ignite node on every cluster node and I set in Ambari 
/mapreduce.jobtracker.address=127.0.0.1:11211;/
But query runs only when I set in beeline
/set mapreduce.jobtracker.address=dev-dn1:11211;/

My next attempts to run the same query failde with different errors:
•       Caused by: java.lang.OutOfMemoryError: Metaspace
•       java.lang.OutOfMemoryError: GC overhead limit exceeded
•       [ERROR][pool-2-thread-1][HadoopJobTracker] Unhandled exception while
processing event. java.lang.NullPointerException
*How to tune Ignite MR that it works stable?*

If you do not mind I ask a few more questions here:
1.      Where can I find an example of default-config.xml with all possible
parameters with default values? I just copypast some example from this
forum, but it is not complete and with some mistakes.
2.      Am I right that the IGFS can only be used if the cluster is not HA?
Otherwise, HDFS does not start because fs.default.name=igfs://myhost:10500/
3.      When I run Hive query with Ignite MR this is not YARN application. Is it
possible to run Hive queries on Ignite as YARN applications?
4.      Does Ignite take advantage of the ORC files (built-in indexes,
statistics)?
5.      Suppose compressed ORC file is 500Mb, the same uncompressed csv-file is
9Gb. How much memory Ignite needs to work with this file?
6.      What is right approach for Ignite MR: either run one ignite-node on 
every
cluster node or run as many ignite-nodes on one cluster node as possible?
7.      I stopped one ignite-node during the query execution then job (query
execution) failed. Is it normal?

Thanks in advance!




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12238.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Reply via email to