Hi Yaniv and Eyal, Sorry about the hiatus. Day job has been hectic the last couple of months.
I am really glad that we now have full blown YARN support. Thanks a lot !! Is there a place where I could find a rough document around how to submit jobs on YARN. If you could respond to this thread, I am more than happy to contribute to the docs. I would like to do a POC of sorts for one of my projects at work. A really dumbed-down version of the application is at : https://github.com/arunma/ama_datapopulator https://github.com/arunma/ama_reconciler The first Spark job populates the data in a bunch of Hive tables The second Spark job runs pre-configured queries against these tables and compares them against another data in another Hive table (reconciliation table). For now, we can safely assume that there's no data shared between these dataframes. Greatly appreciate your response on the YARN job submission. Cheers, Arun
