Hi Vinay, For question one, IIRC, we cannot set multiple "*--input_path" *flag at present. The "--input_path" is designed originally as a placeholder to store a path and then the path is used to replace "%input_path%" in worker command like "python worker.sh -input %input_path% ..". So from this perspective, you can directly append the other input paths to your worker command in your own way.
For question two, because YARN might set a wrong HADOOP_COMMON_HOME by default. So submarine provides the environment variable to be set in the worker's launch script if the worker wants to access HDFS. And there's no data plane relation between outside Hadoop and the container except YARN will localize resources for the container. Hope this can answer your questions. Best Regards, Zhankun On Fri, 22 Feb 2019 at 15:35, Vinay Kashyap <vinu.k...@gmail.com> wrote: > Hi all, > > I am using Hadoop 3.2.0. I am trying few examples using Submarine to run > TensorFlow jobs in a docker container. > I would like to understand few details regarding Read/Write HDFS data > during/after application launch/execution. Have highlighted the questions > line. > > When launching the application which reads input from HDFS, we configure > *--input_path* to a hdfs path, as mentioned in the standard example. > > yarn jar hadoop-yarn-applications-submarine-<version>.jar job run \ > --name tf-job-001 --docker_image <your docker image> \ > --input_path hdfs://default/dataset/cifar-10-data \ > --checkpoint_path hdfs://default/tmp/cifar-10-jobdir \ > --env DOCKER_JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre/ \ > --env DOCKER_HADOOP_HDFS_HOME=/hadoop-3.1.0 \ > --num_workers 2 \ > --worker_resources memory=8G,vcores=2,gpu=1 --worker_launch_cmd "cmd for > worker ..." \ > --num_ps 2 \ > --ps_resources memory=4G,vcores=2,gpu=0 --ps_launch_cmd "cmd for ps" \ > > *Question 1 : What if I have more than 1 dataset in a separate HDFS paths? > Can --input_path take multiple paths in any fashion or is it expected to > maintain all the datasets under one path.?* > > "DOCKER_JAVA_HOME points to JAVA_HOME inside Docker image" > and "DOCKER_HADOOP_HDFS_HOME points to HADOOP_HDFS_HOME inside Docker > image". > > *Question 2 : What is the exact expectation here.? In the sense, is there > any relation/connection with the Hadoop running outside the docker > container.? I guess read HDFS data into the docker container happens during > Container localization, but how does output data write back happens to HDFS > running outside the docker container.?* > > Assuming a scenario where Application 1 creates a model and Application 2 > performs scoring. Both the applications run in a separate docker > containers. I would like the understand how does the data read and write > across applications happen in this case. > Would be of great help if anyone can be guide me understanding this or > direct me to a blog or write up which explains the above. > > *Thanks and regards* > *Vinay Kashyap* >
