[
https://issues.apache.org/jira/browse/SUBMARINE-507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120592#comment-17120592
]
Wangda Tan commented on SUBMARINE-507:
--------------------------------------
[~maniraj...@gmail.com],
{quote} # How do we decide that we need to talk to Docker hub's
"apache/submarine" or end-user's existing Docker registry? Using docker image
name?{quote}
Yes, it should based on the image name and Handled by Docker daemon.
{quote}2. For both the categories, Can we safely assume images will be always
available for use at run time? Nothing to worry about this while creating
environment?
{quote}
Yes, if it has any issue (like image cannot be pulled), the experiment run will
fail (naturally).
{quote}3. For first category of base images ("apache/submarine"),When & How are
we going to create images and release to "apache/submarine"? Using docker file
manually? Admin would be doing this?
{quote}
This is part of the Submarine release process, I believe we already doing this,
cc:[~ztang].
{quote}4. For second category of base images, Can we assume end users would
have created the image in their docker registry?
{quote}
Yes, for whatever base image specified, user need to ensure it can be pulled.
{quote}5. It seems we are in favour of storing environment spec as is in a
string based column in "environments" table?
{quote}
It looks like a good solution to me, I'm not sure if there's any other
alternatives.
{quote}6. conda env activation happens at run time while running the notebook
or experiment. When are we going to create an env in conda?
{quote}
Yes, activation only happens at run time. To create an env, I think we can have
following Python APIs:
1) Create Environment using SDK (pseudo code)
{code:java}
env = create_new_environment("my_env")
# Set Docker image
env.set_docker_image("apache/submarine-123:123")
# Set conda kernel using spec
env.add_conda_kernel(conda_kernel_from_spec("""
name: team_default_python_3.7
channels:
- defaults
dependencies:
- _ipyw_jlab_nb_ext_conf=0.1.0=py37_0
- alabaster=0.7.12=py37_0
- anaconda=2020.02=py37_0
- anaconda-client=1.7.2=py37_0
- anaconda-navigator=1.9.12=py37_0
""")
# Alternatively, set conda kernel using APIs
conda_kernel = create_new_conda_kernel("my_kernel")
conda_kernel.set_channels(["..."])
conda_kernel.add_pip_dependency("...")
# Finally, save the env, this will save to metadata
submarine.register_env(env)
{code}
2) Similarily, we need List API, Get API, Delete API for environment.
Thoughts?
> Submarine Environment Management
> --------------------------------
>
> Key: SUBMARINE-507
> URL: https://issues.apache.org/jira/browse/SUBMARINE-507
> Project: Apache Submarine
> Issue Type: New Feature
> Reporter: Manikandan R
> Assignee: Manikandan R
> Priority: Major
> Labels: pull-request-available
>
> Scope of this JIRA is to support environment management. It includes the
> following:
> 1. Create Environment
> 2. Update Environment
> 3. Delete Environment
> 4. List Environments
> In addition, this JIRA should also ensures that environments has been
> persisted like experiments so that it can used for later use.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
