[GitHub] [submarine] yuanzac commented on a change in pull request #149: SUBMARINE-335 Add a guide of yarn runtime with docker containers

[GitBox]

yuanzac commented on a change in pull request #149: SUBMARINE-335 Add a guide 
of yarn runtime with docker containers
URL: https://github.com/apache/submarine/pull/149#discussion_r365637080
 
 

 ##########
 File path: dev-support/mini-submarine/submarine/run_submarine_mnist_tony.sh
 ##########
 @@ -33,27 +33,37 @@ else
   JAVA_CMD="java"
 fi
 
-while getopts 'd:' OPT; do
+while getopts 'd:c' OPT; do
   case $OPT in
     d)
       DATA_URL="$OPTARG";;
+    c)
+      USE_DOCKER=1;;
   esac
 done
 shift $(($OPTIND - 1))
 
 if [[ -n "$DATA_URL" ]]; then
-  WORKER_CMD="myvenv.zip/venv/bin/python mnist_distributed.py --steps 2 
--data_dir /tmp/data --working_dir /tmp/mode --mnist_data_url ${DATA_URL}"
+  WORKER_CMD="venv/bin/python mnist_distributed.py --steps 2 --data_dir 
/tmp/data --working_dir /tmp/mode --mnist_data_url ${DATA_URL}"
 else
-  WORKER_CMD="myvenv.zip/venv/bin/python mnist_distributed.py --steps 2 
--data_dir /tmp/data --working_dir /tmp/mode"
-fi 
+  WORKER_CMD="venv/bin/python mnist_distributed.py --steps 2 --data_dir 
/tmp/data --working_dir /tmp/mode"
+fi
+
+if [[ -n "$USE_DOCKER" ]]; then
+  WORKER_CMD="/opt/$WORKER_CMD"
+  # tony-mnist-tf-1.13.1:0.0.1 is built from the 
Dockerfile.tony.tf.mnist.tf_1.13.1 under docs/helper/docker/tensorflow/mnist
+  DOCKER_CONF="--conf tony.docker.containers.image=tony-mnist-tf-1.13.1:0.0.1 
--conf tony.docker.enabled=true"
+else
+  WORKER_CMD="myvenv.zip/$WORKER_CMD"
+fi
 
 SUBMARINE_VERSION=0.3.0-SNAPSHOT
-HADOOP_VERSION=2.9
-SUBMARINE_PATH=/opt/submarine-current
-HADOOP_CONF_PATH=/usr/local/hadoop/etc/hadoop
-MNIST_PATH=/home/yarn/submarine
+SUBMARINE_HADOOP_VERSION=2.9
 
 Review comment:
   HADOOP_VERSION needs to be changed to SUBMARINE_HADOOP_VERSION in the line 
407 of InstallationGuide.md as well.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org