Re: pio train on Amazon EMR

2018-02-05 Thread Pat Ferrel
@predictionio.apache.org Date: February 5, 2018 at 11:45:46 AM To: user@predictionio.apache.org Subject:  Re: pio train on Amazon EMR Hi Malik, This is a topic I've been investigating as well. Given how EMR manages its clusters & their runtime, I don't think hacking configs to make the Pre

Re: pio train on Amazon EMR

2018-02-05 Thread Mars Hall
Hi Malik, This is a topic I've been investigating as well. Given how EMR manages its clusters & their runtime, I don't think hacking configs to make the PredictionIO host act like a cluster member will be a simple or sustainable approach. PredictionIO already operates Spark by building `spark-su

pio train on Amazon EMR

2018-02-05 Thread Malik Twain
I'm trying to run pio train with Amazon EMR. I copied core-site.xml and yarn-site.xml from EMR to my training machine, and configured HADOOP_CONF_DIR in pio-env.sh accordingly. I'm running pio train as below: pio train -- --master yarn --deploy-mode cluster It's failing with the following errors