Hi Vinay,
IIRC, YARN will have the host's Hadoop environments set in container launch
script by default. And in the submarine case, the user's worker command is
used to generate a worker script which is invoked in the container launch
script. If submarine doesn't override the default Hadoop enviro
Hi everyone,
Spark supports in application, job concurrency execution by using pools and
Spark's Fair scheduler (different than Yarn's Fair scheduler).
link:
https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application
Is this feature supported when Yarn is used as a
Thanks Zhankun for the clarification.
Also, is my understanding correct on --checkpoint_path as I mentioned
earlier in the thread..?? Quoting the comment again in this thread.
[There is another argument called *--checkpoint_path* which acts as a path
where all the outputs (models or datasets) whic
