Hi Ilya, Thanks a lot for your help, I'll try and the test the patch.
Regarding how I run this, I wait for the cluster to enter in WAITING state and then I ssh to the master node as described in https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-connect-master-node-ssh.html, and then I just run the commands there. Before running the command I put the files in HDFS. We cannot use a bootstrap action here because they run before HDFS starts, but another option is using a EMR step for this. For the node manager logs, I follow https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-web-interfaces.html to have access to the resource manager UI, and then I see the nodes that are involved with the YARN application. Then I ssh to some of the slave nodes involved with the job from the EMR master, and check the node manager logs in /var/log/hadoop-yarn. Also we could configure yarn like suggested in https://slider.incubator.apache.org/docs/getting_started.html to have 1 hour of timeout before cleaning up containers, using https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html to configure that on EMR. The container files are then available in the EMR slaves at /mnt/yarn/ I'll tell you how it goes when I have tested the patch. Thanks a lot! Juan On Fri, Dec 1, 2017 at 6:43 AM, ilya.kasnacheev <[email protected]> wrote: > Hello once more! > > I have prepared a patch. I think it won't be hard for you to try it out > since you are accustomized to building Ignite. Please try it, tell if it > solves the current issue. ignite-7080.patch > <http://apache-ignite-users.70518.x6.nabble.com/file/ > t1312/ignite-7080.patch> > > I was trying to set Ignite up on AWS EMR today, without much success. I > wonder if you could provide me with some directions. > > > IGNITE_YARN_JAR=/mnt/ignite/apache-ignite-2.3.0-src/ > modules/yarn/target/ignite-yarn-2.3.0.jar > > yarn jar ${IGNITE_YARN_JAR} ${IGNITE_YARN_JAR} > > /mnt/ignite/ignite_yarn.properties > > Can you tell me what were you using to launch this command? I have tried > executing "Custom JAR" step on EMR cluster, after copying Ignite properties > and JAR to /mnt on master manually. So I was informed that an application > was started, and is now running, but no indication about execution results. > I suspect it hung with the same problem that you outlined (or earlier), but > I haven't found a way to view application logs, neither "node manager" logs > that you are referring to. Can you please point me where I can watch > intermediate execution results? > > Also there's two data access questions. First, Ignite YARN jar expects two > files on local FS, the jar itself, and properties file. Is there a way to > make sure that these files are on file system, at predictable locations, > when starting EMR cluster? To be able to add "Custom JAR" step to cluster > initialization? E.g. specify an s3 bucket to be mounted to master node? > > Another question is that you are specifying ignite zip and ignite config > are > on hdfs. How would you put these files to hdfs when cluster is created? Or, > for that matter, after it is created? I haven't found any access to HDFS in > EMR console, neither any instructions on how to access it. > > Maybe we should change this process in order to use Ignite YARN with EMR > without extensive boilerplate? > > Regards, > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
