[GitHub] [submarine] oliverhu commented on a change in pull request #149: SUBMARINE-335. Add a guide of yarn runtime with docker containers

[GitBox]

oliverhu commented on a change in pull request #149: SUBMARINE-335. Add a guide 
of yarn runtime with docker containers
URL: https://github.com/apache/submarine/pull/149#discussion_r365652590
 
 

 ##########
 File path: dev-support/mini-submarine/submarine/run_submarine_mnist_tony.sh
 ##########
 @@ -33,27 +33,37 @@ else
   JAVA_CMD="java"
 fi
 
-while getopts 'd:' OPT; do
+while getopts 'd:c' OPT; do
   case $OPT in
     d)
       DATA_URL="$OPTARG";;
+    c)
+      USE_DOCKER=1;;
   esac
 done
 shift $(($OPTIND - 1))
 
 if [[ -n "$DATA_URL" ]]; then
-  WORKER_CMD="myvenv.zip/venv/bin/python mnist_distributed.py --steps 2 
--data_dir /tmp/data --working_dir /tmp/mode --mnist_data_url ${DATA_URL}"
+  WORKER_CMD="venv/bin/python mnist_distributed.py --steps 2 --data_dir 
/tmp/data --working_dir /tmp/mode --mnist_data_url ${DATA_URL}"
 else
-  WORKER_CMD="myvenv.zip/venv/bin/python mnist_distributed.py --steps 2 
--data_dir /tmp/data --working_dir /tmp/mode"
-fi 
+  WORKER_CMD="venv/bin/python mnist_distributed.py --steps 2 --data_dir 
/tmp/data --working_dir /tmp/mode"
+fi
+
+if [[ -n "$USE_DOCKER" ]]; then
+  WORKER_CMD="/opt/$WORKER_CMD"
+  # tony-mnist-tf-1.13.1:0.0.1 is built from the 
Dockerfile.tony.tf.mnist.tf_1.13.1 under docs/helper/docker/tensorflow/mnist
+  DOCKER_CONF="--conf tony.docker.containers.image=tony-mnist-tf-1.13.1:0.0.1 
--conf tony.docker.enabled=true"
+else
+  WORKER_CMD="myvenv.zip/$WORKER_CMD"
+fi
 
 SUBMARINE_VERSION=0.3.0-SNAPSHOT
-HADOOP_VERSION=2.9
-SUBMARINE_PATH=/opt/submarine-current
-HADOOP_CONF_PATH=/usr/local/hadoop/etc/hadoop
-MNIST_PATH=/home/yarn/submarine
+SUBMARINE_HADOOP_VERSION=2.9
+SUBMARINE_PATH=/home/pi/apache/submarine/submarine-all/target
 
 Review comment:
   i see, these changes are not committed either. Reverted

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org