[
https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560436#comment-16560436
]
Yinan Li commented on SPARK-24724:
----------------------------------
Sorry haven't got a chance to look into this. What pieces of info and what kind
of access we need to provide? I saw some comments on thesimilar Jira for Yarn
and particularly the one quoted below:
"The main problem is how to provide necessary information for barrier tasks to
start MPI job in a password-less manner".
Is the main problem the same for Kubernetes?
> Discuss necessary info and access in barrier mode + Kubernetes
> --------------------------------------------------------------
>
> Key: SPARK-24724
> URL: https://issues.apache.org/jira/browse/SPARK-24724
> Project: Spark
> Issue Type: Story
> Components: Kubernetes, ML, Spark Core
> Affects Versions: 3.0.0
> Reporter: Xiangrui Meng
> Assignee: Yinan Li
> Priority: Major
>
> In barrier mode, to run hybrid distributed DL training jobs, we need to
> provide users sufficient info and access so they can set up a hybrid
> distributed training job, e.g., using MPI.
> This ticket limits the scope of discussion to Spark + Kubernetes. There were
> some past and on-going attempts from the Kubenetes community. So we should
> find someone with good knowledge to lead the discussion here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
