Confluence for Submarine online & roadmap wiki
Hi dev, Confluence space for Submarine got created, and I just created a page for roadmap: https://cwiki.apache.org/confluence/display/SUBMARINE/Roadmap Can we get help from the community to update 0.3.0 and 0.4.0 timeline, scope, etc. Thanks, Wangda
Re: [ANNOUNCE] Apache Submarine 0.3.0 release!
Awesome! Thanks everybody for helping this release! Best, Wangda On Thu, Feb 6, 2020 at 6:44 AM Xun Liu wrote: > Thanks to all contributors! > Let's continue to develop better features for submarine!! > > On Thu, Feb 6, 2020 at 7:25 PM Sunil Govindan wrote: > >> Awesome! Hearty Congratulations to all of you for the first release of >> Submarine as Apache TLP! >> Kudos to all who helped in this release! >> >> - Sunil >> >> On Thu, Feb 6, 2020 at 4:36 PM Zhankun Tang wrote: >> >> > Hi folks, >> > >> > It's a great honor for me to announce that the Apache Submarine >> community >> > has released Apache Submarine 0.3.0! >> > >> > Apache Submarine 0.3.0 is the first release after the spin-off from >> Apache >> > Hadoop. It includes 196 patches since the prior version. >> > The highlighted features are: >> > - Mini-submarine (YARN) >> > - Basic Tensorflow job submission to k8s through submarine-server >> RESTful >> > API >> > - Job submission on YARN through submarine-server RPC protocol >> > >> > Tons of thanks to our contributors and community! Let's keep fighting! >> > >> > *Apache Submarine 0.3.0 released*: >> > http://submarine.apache.org/releases/submarine-release-0.3.0.html >> > *Changelog*: https://s.apache.org/4ezw1 >> > >> > BR, >> > Zhankun >> > >> >
[DISCUSS] Rearchitecture & roadmap change proposal
Hello devs, Spent some time did some small-group discussion for the Submarine architecture, use cases, and roadmaps. (Thanks for all the inputs from Yanbo, Xun, Wei-Chiu, Zhankun and many others). Basically, we want to better align the goal of UI and backend development, target to create a unique project which allows users (like data scientists) to use Submarine and do complete daily jobs. Here's the proposal: https://docs.google.com/document/d/1J6SxfcICc2DrmxxQhO71Xk2zFb4SjDCV60bHsKhiDdI/edit Inside the proposal, I included UI mocks, requirement & architecture design, and task prioritization & breakdown. Please check these docs in detail, and I will go through these materials tonight in the weekly sync up. Thanks, Wangda
Re: [DISCUSS] Rearchitecture & roadmap change proposal
After discussed with the community members, I updated the design doc into 3 pieces: 1) Removed lots of duplicated contents and detailed implementations from the architecture <https://github.com/wangdatan/submarine/blob/SUBMARINE-469/docs/design/architecture-and-requirements.md> doc. 2) Added implementation notes for *environment creation* and *experiment*: https://github.com/wangdatan/submarine/blob/SUBMARINE-469/docs/design/implementation-notes.md Please review and let me know your thoughts. What I plan to do next: 1) Finish experiment implementation notes (I plan to add at least pre-defined template part), and a basic notes for storage. 2) Task breakdown and prioritization (assume there's no opposite opinion about design itself). Thanks, Wangda On Mon, Apr 13, 2020 at 3:46 PM Wangda Tan wrote: > Hello devs, > > Spent some time did some small-group discussion for the Submarine > architecture, use cases, and roadmaps. (Thanks for all the inputs from > Yanbo, Xun, Wei-Chiu, Zhankun and many others). > > Basically, we want to better align the goal of UI and backend development, > target to create a unique project which allows users (like data scientists) > to use Submarine and do complete daily jobs. > > Here's the proposal: > https://docs.google.com/document/d/1J6SxfcICc2DrmxxQhO71Xk2zFb4SjDCV60bHsKhiDdI/edit > > > Inside the proposal, I included UI mocks, requirement & architecture > design, and task prioritization & breakdown. > > Please check these docs in detail, and I will go through these materials > tonight in the weekly sync up. > > Thanks, > Wangda >
[DISCUSS] Proposal of Task Breakdown and Timeline of Submarine 0.4.0/0.5.0/1.0.0 release
Hi dev, Here's a WIP proposal of Submarine task breakdown: https://docs.google.com/spreadsheets/d/19g9RCxQEPE74CsOHK-nAf5UO32sFCCP4jCJB3RuFivE/edit#gid=0 *Please comment to add your thoughts. Some highlights: * - We will do release every two months (release cadence) - Target to release 0.4.0 by end of May, which experiments submission functionalities. - (Jul) 0.5.0 will include UI and internal implementation for notebook/experiment/environment. - (Sep) Will do 1.0.0 (GA) release, will include more end to end logics, comprehensive docs, must-to-have security-related features/guidances. - (Nov) 1.1.0 release, consider to include model management. Thoughts? Thanks, Wangda
Re: [DISCUSS] Proposal of Task Breakdown and Timeline of Submarine 0.4.0/0.5.0/1.0.0 release
Good question, unless we got strong demand from the community to include it and contributors have bandwidth to do it, making new experiment framework work with YARN is a lower priority comparing to running on K8s. Even items are dockerized, but implementation still different, and deploy Submarine server on YARN needs a lot of net-new works in addition to support K8s. We will not remove/deprecate existing YARN support, so anybody uses 0.3.0 release can continue. Thanks, Wangda On Thu, Apr 30, 2020 at 7:06 PM Zhankun Tang wrote: > Thanks Wangda for the proposal. It LGTM. > One doubt for running experiment on YARN. Will it be included? > > BR, > Zhankun > > Wei-Chiu Chuang 于2020年5月1日 周五上午2:18写道： > > > Looks good to me. Thanks! > > > > Yes we need security to call a 1.0 GA > > > > On Wed, Apr 29, 2020 at 9:58 PM Wangda Tan wrote: > > > > > Hi dev, > > > > > > Here's a WIP proposal of Submarine task breakdown: > > > > > > > > > > > > https://docs.google.com/spreadsheets/d/19g9RCxQEPE74CsOHK-nAf5UO32sFCCP4jCJB3RuFivE/edit#gid=0 > > > > > > *Please comment to add your thoughts. Some highlights: * > > > > > > - We will do release every two months (release cadence) > > > - Target to release 0.4.0 by end of May, which experiments submission > > > functionalities. > > > - (Jul) 0.5.0 will include UI and internal implementation for > > > notebook/experiment/environment. > > > - (Sep) Will do 1.0.0 (GA) release, will include more end to end > logics, > > > comprehensive docs, must-to-have security-related features/guidances. > > > - (Nov) 1.1.0 release, consider to include model management. > > > > > > Thoughts? > > > > > > Thanks, > > > Wangda > > > > > >
Re: [VOTE] Submarine-0.4.0-RC0 is ready for a vote!
+1 (binding), I have done the following: a. Download source code tarball and verify the signature. b. Follow Submarine-K8s documentation, install helm and run examples. Thanks Wanqiang for driving this release! Best, Wangda On Mon, Jun 29, 2020 at 2:57 AM Zhankun Tang wrote: > Thanks for Wanqiang's hard work! > > I've done below testing: > 1. Download the source code and verify the signature > 2. Build it successfully > 3. Install Helm charts > 4. Run an experiment with the SDK example. > > +1 from me. > > One thing to mention is that we need to publish the python SDK to " > pypi.org" > when releasing. > > BR, > Zhankun > > On Fri, 26 Jun 2020 at 22:19, Xun Liu wrote: > > > +1, Thanks Wanqiang! > > > > Best Regards > > Xun Liu > > > > On Fri, Jun 26, 2020 at 10:15 PM kevin su wrote: > > > > > +1, Thanks Wanqiang > > > > > > Best Regards, > > > Kevin Su > > > > > > Wanqiang Ji 於 2020年6月26日 週五，下午9:53寫道： > > > > > > > Hi folks, > > > > > > > > Thanks to everyone's help on this release. Special thanks to Wangda, > > > > Zhankun, Xun, Kevin and JohnTing! > > > > > > > > I've created a release candidate (RC0) for submarine 0.4.0. The > > > highlighted > > > > features are as follows: > > > > 1. Submarine Experiments: Refactor the Job to experiment and > redefined > > > the > > > > experiment spec. > > > > 2. Submarine Helm Charts: Provides one command to install the > submarine > > > > into the Kubernetes cluster. > > > > 3. PySubmarine: Submarine Python SDK > > > > > > > > The RC tag in git is here: > > > > https://github.com/apache/submarine/releases/tag/release-0.4.0-RC0 > > > > > > > > The RC release artifacts are available at: > > > > http://home.apache.org/~jiwq/submarine-0.4.0-RC0/ > > > > > > > > The mini-submarine image is here: > > > > > > > > > > > > > > https://hub.docker.com/layers/apache/submarine/mini-0.4.0-RC0/images/sha256-a8e7bd98f1f0325223d68e0ba64fd48bd56ee91736461d289945e70ad138e08f?context=explore > > > > > > > > The server image is here: > > > > > > > > > > > > > > https://hub.docker.com/layers/apache/submarine/server-0.4.0/images/sha256-9e9775e0ed2ac9d4b108dd747dfb4f4fc27cee0fdd19501e1f53db30d966a3db?context=repo > > > > > > > > The database image is here: > > > > > > > > > > > > > > https://hub.docker.com/layers/apache/submarine/database-0.4.0/images/sha256-5edf4aac2c14e14f78a9955bd84757879e1c05266c1edd5a7d5ef08b6359a400?context=repo > > > > > > > > The Maven staging repository is here: > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachesubmarine-1003 > > > > > > > > My public key is here: > > > > https://dist.apache.org/repos/dist/release/submarine/KEYS > > > > > > > > *This vote will run for 7 days, ending on Jul 2, 2020 at 11:59 pm > PST.* > > > > > > > > For the testing, I have verified the build from source, run the mnist > > > > example with mini-submarine(both local and remote mode) and verified > > > > the experiment operations to K8s by Submarine Server REST and > > > PySubmarine. > > > > > > > > Please follow the document to test these features. > > > > * > > > > > > > > > > https://github.com/apache/submarine/tree/master/dev-support/mini-submarine > > > > * > > > > https://github.com/apache/submarine/blob/master/docs/user-guide-home.md > > > > > > > > My +1 to start. Thanks! > > > > > > > > BR, > > > > Wanqiang Ji > > > > > > > > > >
Re: [ANNOUNCE] Apache Submarine 0.4.0 release!
Thanks Wanqiang for driving this release effort, and thanks to everybody from the community who contributed to this release! The announcement blogpost for 0.4.0 is also published: https://medium.com/@apache.submarine/apache-submarine-0-4-0-release-whats-new-and-coming-fa14b4ed8aec Best, Wangda Tan On Mon, Jul 6, 2020 at 4:21 AM Sunil Govindan wrote: > Great to see 0.4.0 release out! Kudos to all who helped! > > Thanks > Sunil > > On Mon, Jul 6, 2020 at 3:35 PM Zhankun Tang wrote: > >> Really happy to see 0.4.0 released! Thanks for leading these efforts! >> Wanqiang! >> >> BR, >> Zhankun >> >> On Sun, 5 Jul 2020 at 18:45, Wanqiang Ji wrote: >> >>> Hi forks, >>> >>> It's a great honor for me to announce that the Apache Submarine Community >>> has released Apache Submarine 0.4.0! >>> >>> Apache Submarine 0.4.0 is the second release after the spin-off from >>> Apache >>> Hadoop. It includes 175 patches since the prior version. >>> The highlighted features are: >>> - Submarine Experiments: Refactor the Job to experiment and redefined >>> the experiment spec >>> - Submarine Helm Charts: Provides one command to install the submarine >>> into the Kubernetes cluster >>> - PySubmarine: Submarine Python SDK >>> >>> Tons of thanks to our contributors and community! Let's keep fighting! >>> >>> *Apache Submarine 0.4.0 released*: >>> http://submarine.apache.org/releases/submarine-release-0.4.0.html >>> *Changelog*: https://s.apache.org/5barn >>> >>> BR, >>> Wanqiang Ji >>> >>>
UI Design for Submarine Experiment Submission flow
Hi folks, Based on our yesterday's conversation, I just created a UI interaction flow to submit an experiment. Please review it and let me know what you think: https://miro.com/app/board/o9J_kucuj6Q=/ See the (Experiment UI flow) block. Thanks, Wangda
Re: [VOTE] Submarine-0.5.0-RC1 is ready for a vote!
Hi Kevin,
Thank you so much for running this release.
Trying to follow the helm install stage, but notebook controller is failed
to start.
I downloaded RC1 source code, and follow the guidance:
https://github.com/apache/submarine/blob/master/docs/userdocs/k8s/helm.md
The pods in my system:
NAMESPACE NAME READY
STATUS RESTARTS AGE
default notebook-controller-deployment-58797bdd75-9fstx 0/1
CrashLoopBackOff 7 12m
default pytorch-operator-75fd845678-dpc52 1/1
Running0 12m
default submarine-database-54776644c6-x2zrm 1/1
Running0 12m
default submarine-server-5d846f7b4f-m2fw8 1/1
Running0 12m
default submarine-traefik-d55c689b5-cq6db 1/1
Running0 12m
default tf-job-operator-598686fd84-2fwt9 1/1
Running0 12m
And notebook-controller pod has the following information: (kubectl
describe pods notebook-controller-deployment-58797bdd75-9fstx)
Events:
Type Reason Age From Message
-- ---
Normal Scheduled 12m default-scheduler
Successfully assigned
default/notebook-controller-deployment-58797bdd75-9fstx to docker-desktop
Normal Pulling12m kubelet, docker-desktop
Pulling image "apache/submarine:notebook-controller-v1.1.0-g253890cb"
Normal Pulled 12m kubelet, docker-desktop
Successfully pulled image
"apache/submarine:notebook-controller-v1.1.0-g253890cb"
Normal Pulled 11m (x4 over 12m) kubelet, docker-desktop
Container image "apache/submarine:notebook-controller-v1.1.0-g253890cb"
already present on machine
Normal Created11m (x5 over 12m) kubelet, docker-desktop
Created container manager
Normal Started11m (x5 over 12m) kubelet, docker-desktop
Started container manager
Warning BackOff2m45s (x51 over 12m) kubelet, docker-desktop
Back-off restarting failed container
Logs: (kubectl log notebook-controller-deployment-58797bdd75-9fstx)
2020-11-02T17:11:53.159Z ERROR setup unable to create controller
{"controller": "Notebook", "error": "no matches for kind \"Notebook\" in
version \"kubeflow.org/v1beta1\""}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
main.main
/workspace/notebook-controller/main.go:76
runtime.main
/usr/local/go/src/runtime/proc.go:200
I guess it might be version of K8s, I'm using DockerDesktop:
kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.6",
GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean",
BuildDate:"2019-11-13T11:20:18Z", GoVersion:"go1.12.12", Compiler:"gc",
Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+",
GitVersion:"v1.16.6-beta.0",
GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean",
BuildDate:"2020-01-15T08:18:29Z", GoVersion:"go1.13.5", Compiler:"gc",
Platform:"linux/amd64"}
I'm not sure what I should do now? Last time when we release 0.4.0, we
verified 1.14, 1.15, 1.16 should all work.
https://github.com/apache/submarine/blob/master/docs/userdocs/k8s/README.md
What is our recommendation for now?
Thanks,
Wangda
On Fri, Oct 30, 2020 at 8:47 PM Wanqiang Ji wrote:
> +1 for this RC1. Thanks Kevin drive this release and everyone for the great
> works.
> I've done the following tests:
> 1. Install the helm charts against the minikube
> 2. Install the helm charts against the kind
> 3. Check the UI's features, create the notebook/experiment and run work
>
> BR,
> Wanqiang Ji
>
>
> On Wed, Oct 28, 2020 at 4:52 PM Zhankun Tang wrote:
>
> > Thanks for the great efforts! Kevin!
> > I've done the below testing
> > 1. Verify the signatures (I signed Kevin's key and updated the KEYS file)
> > 2. Build from source
> > 3. Install the helm charts against Docker desktop k8s (v1.14.8)
> > 4. Check basic UI's experiment and notebook features
> >
> > And I like the built-in examples in our notebook image. It's
> > straightforward for beginners to get familiar with our SDK.
> > One minor suggestion is that we can add a link in parent readme
> > <
> >
> https://github.com/apache/submarine/blob/master/docs/userdocs/k8s/README.md
> > >
> > to our notebook.md doc. But this is not a blocker, we can improve docs
> > later since it's in GitHub for now.
> >
> > I'll give my *+1(binding)* to this RC1.
> >
> > @Wei-Chiu Chuang BTW, I can download the
> > "apache/submarine:mini-0.5.0-RC1" image on my laptop.
> >
> > BR,
> > Zhankun
> >
> > On Mon, 26 Oct 2020 at 10:21, Wei-Chiu Chuang
> > >
> > wrote:
> >
> > > Curious -- I see mini-0.5.0-RC1 and RC0 docker images, however, the
> > > operator, server, database and j
Re: [VOTE] Submarine-0.5.0-RC1 is ready for a vote!
Nevermind, please ignore the previous message. I think it is caused by previous installation (0.4.0), I reset K8s cluster and now everything can be installed, trying to follow other steps now. 1) Why by default Submarine sever login has maria_dev as default user name? 2) There's no doc about using Submarine UI, we need to add one, I will file a PR later (TODO) 3) Trying to create notebook, the notebook name initially I gave is "nb_123", but I got the error, "K8s submitter: parse Job object failed by Unprocessable Entity, please try again" If we have limitation of how naming of notebook should be, we should add it to the UI (like only character or numbers are supported). >From the doc it mentioned: "Name of the notebook server. It should be unique and include no spaces." We need to update both of doc and UI. 4) When I choose the environment for notebook, I saw both my-submarine-env and notebook-env. What are the differences between the two? - I found only notebook-env works, "my-submarine-env" failed to start. We should remove the my-submarine-env from the default helm installation. 5) Also, I found even if notebook is not fully start and running, the UI indicate it is created: [image: image.png] Clicking notebook name will show you an error page, we should improve this part. 6) After waiting for ~5 mins, the notebook started, but I still cannot access the notebook UI, clicking the link on the Submarine UI tells me: HTTP ERROR 404 Problem accessing /notebook/default/notebook/. Reason: Not Found And logs for the notebook pod tells me; kubectl logs notebook-0 Conda current version is currentVersion=4.8.3;. Moving forward with env creation and activation. [I 17:45:50.056 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret [W 17:45:52.108 NotebookApp] All authentication is disabled. Anyone who can connect to this server will be able to run code. [I 17:45:52.144 NotebookApp] Serving notebooks from local directory: /home/jovyan [I 17:45:52.145 NotebookApp] Jupyter Notebook 6.1.3 is running at: [I 17:45:52.146 NotebookApp] http://notebook-0:/notebook/default/notebook/ [I 17:45:52.147 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). Is it bind to a wrong port? On Mon, Nov 2, 2020 at 9:18 AM Wangda Tan wrote: > Hi Kevin, > > Thank you so much for running this release. > > Trying to follow the helm install stage, but notebook controller is failed > to start. > > I downloaded RC1 source code, and follow the guidance: > https://github.com/apache/submarine/blob/master/docs/userdocs/k8s/helm.md > > The pods in my system: > > NAMESPACE NAME READY > STATUS RESTARTS AGE > default notebook-controller-deployment-58797bdd75-9fstx 0/1 > CrashLoopBackOff 7 12m > default pytorch-operator-75fd845678-dpc52 1/1 > Running0 12m > default submarine-database-54776644c6-x2zrm 1/1 > Running0 12m > default submarine-server-5d846f7b4f-m2fw8 1/1 > Running0 12m > default submarine-traefik-d55c689b5-cq6db 1/1 > Running0 12m > default tf-job-operator-598686fd84-2fwt9 1/1 > Running0 12m > > And notebook-controller pod has the following information: (kubectl > describe pods notebook-controller-deployment-58797bdd75-9fstx) > > Events: > Type Reason Age From > Message > -- > --- > Normal Scheduled 12m default-scheduler > Successfully assigned > default/notebook-controller-deployment-58797bdd75-9fstx to docker-desktop > Normal Pulling12m kubelet, docker-desktop > Pulling image "apache/submarine:notebook-controller-v1.1.0-g253890cb" > Normal Pulled 12m kubelet, docker-desktop > Successfully pulled image > "apache/submarine:notebook-controller-v1.1.0-g253890cb" > Normal Pulled 11m (x4 over 12m) kubelet, docker-desktop > Container image "apache/submarine:notebook-controller-v1.1.0-g253890cb" > already present on machine > Normal Created11m (x5 over 12m) kubelet, docker-desktop > Created container manager > Normal Started11m (x5 over 12m) kubelet, docker-desktop > Started container manager > Warning BackOff2m45s (x51 over 12m) kubelet, docker-desktop > Back-off restarting failed container > > Logs: (kubectl log notebook-controller-deployment-58797bdd75-9fstx)
Re: [VOTE] Submarine-0.5.0-RC1 is ready for a vote!
e
221
222 content_type = response_data.getheader('content-type')
/opt/conda/lib/python3.7/site-packages/submarine/experiment/api_client.py
in __call_api(self, resource_path, method, path_params, query_params,
header_params, body, post_params, files, response_type, auth_settings,
_return_http_data_only, collection_formats, _preload_content,
_request_timeout, _host)
215 body=body,
216
_preload_content=_preload_content,
--> 217
_request_timeout=_request_timeout)
218 except ApiException as e:
219 e.body = e.body.decode('utf-8') if six.PY3 else e.body
/opt/conda/lib/python3.7/site-packages/submarine/experiment/api_client.py
in request(self, method, url, query_params, headers, post_params, body,
_preload_content, _request_timeout)
461
_preload_content=_preload_content,
462
_request_timeout=_request_timeout,
--> 463 body=body)
464 elif method == "PUT":
465 return self.rest_client.PUT(url,
/opt/conda/lib/python3.7/site-packages/submarine/experiment/rest.py in
POST(self, url, headers, query_params, post_params, body, _preload_content,
_request_timeout)
324 _preload_content=_preload_content,
325 _request_timeout=_request_timeout,
--> 326 body=body)
327
328 def PUT(self,
/opt/conda/lib/python3.7/site-packages/submarine/experiment/rest.py in
request(self, method, url, query_params, headers, body, post_params,
_preload_content, _request_timeout)
247
248 if not 200 <= r.status <= 299:
--> 249 raise ApiException(http_resp=r)
250
251 return r
ApiException: (409)
Reason: Conflict
HTTP response headers: HTTPHeaderDict({'Date': 'Tue, 03 Nov 2020 00:08:40
GMT', 'Content-Type': 'application/json;charset=utf-8', 'Content-Length':
'140', 'Server': 'Jetty(9.4.18.v20190429)'})
HTTP response body:
{"status":"CONFLICT","code":409,"success":null,"message":"K8s submitter:
parse Job object failed by Conflict","result":null,"attributes":{}}
- Also, submitted PR: https://github.com/apache/submarine/pull/444 for
documentation-related improvements, please help to review, I think it is
important to get these issues fixed.
After restart K8s, the problem (cannot connect to notebook) is gone, I'm
not sure if there's anybody else hits the issue, it will be better if we
can add documentation to the user doc and help users to do troubleshooting.
On Mon, Nov 2, 2020 at 9:54 AM Wangda Tan wrote:
> Nevermind, please ignore the previous message. I think it is caused by
> previous installation (0.4.0), I reset K8s cluster and now everything can
> be installed, trying to follow other steps now.
>
> 1) Why by default Submarine sever login has maria_dev as default user
> name?
>
> 2) There's no doc about using Submarine UI, we need to add one, I will
> file a PR later (TODO)
>
> 3) Trying to create notebook, the notebook name initially I gave is
> "nb_123", but I got the error, "K8s submitter: parse Job object failed by
> Unprocessable Entity, please try again"
> If we have limitation of how naming of notebook should be, we should add
> it to the UI (like only character or numbers are supported).
> From the doc it mentioned: "Name of the notebook server. It should be
> unique and include no spaces." We need to update both of doc and UI.
>
> 4) When I choose the environment for notebook, I saw both my-submarine-env
> and notebook-env. What are the differences between the two?
> - I found only notebook-env works, "my-submarine-env" failed to start. We
> should remove the my-submarine-env from the default helm installation.
>
> 5) Also, I found even if notebook is not fully start and running, the UI
> indicate it is created:
>
> [image: image.png]
>
> Clicking notebook name will show you an error page, we should improve this
> part.
>
> 6) After waiting for ~5 mins, the notebook started, but I still cannot
> access the notebook UI, clicking the link on the Submarine UI tells me:
>
> HTTP ERROR 404
>
> Problem accessing /notebook/default/notebook/. Reason:
>
> Not Found
>
>
> And logs for the notebook pod tells me;
>
>
> kubectl logs notebook-0
> Conda current version is currentVersion=4.8.3;. Moving forward with env
> creation and activation.
> [I 17:45:50.056 NotebookApp] Writing notebook server cookie secret to
> /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
> [W 17:45:52.108 NotebookApp] All au
[jira] [Updated] (SUBMARINE-313) [Umbrella] Support Submarine Deployment in cloud-native way
[ https://issues.apache.org/jira/browse/SUBMARINE-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-313: - Summary: [Umbrella] Support Submarine Deployment in cloud-native way (was: [Umbrella] Submarine Cloud) > [Umbrella] Support Submarine Deployment in cloud-native way > --- > > Key: SUBMARINE-313 > URL: https://issues.apache.org/jira/browse/SUBMARINE-313 > Project: Apache Submarine > Issue Type: New Feature > Components: Submarine Cloud >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > > The Submarine Cloud module allows submarine services to run in k8s, aws, gce, > azure environments in a cloud-native way. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-310) Support golang and java mixed compilation
[ https://issues.apache.org/jira/browse/SUBMARINE-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-310: - Status: Patch Available (was: Open) > Support golang and java mixed compilation > - > > Key: SUBMARINE-310 > URL: https://issues.apache.org/jira/browse/SUBMARINE-310 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Submarine needs to support both the yarn and k8s resource scheduling systems > for job operation. > Some modules (submitter-yarn) are developed using java, and some modules > (submarine-operator / submitter-k8s) are developed using golang. > So submarine needs to support both mvn and go build compilation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-309) Make yarn service configuraiton optional in installation guide
[ https://issues.apache.org/jira/browse/SUBMARINE-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-309: - Target Version: 0.3.0 > Make yarn service configuraiton optional in installation guide > -- > > Key: SUBMARINE-309 > URL: https://issues.apache.org/jira/browse/SUBMARINE-309 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Critical > > At this time, Submarine has two runtimes for yarn clusters, yarn service and > tony. So the installation guide should works for the both two rumtimes. As > the installation of calico and registrydns is only for yarn service, they > should be marked as optional. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-309) Make yarn service configuraiton optional in installation guide
[ https://issues.apache.org/jira/browse/SUBMARINE-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-309: - Priority: Critical (was: Major) > Make yarn service configuraiton optional in installation guide > -- > > Key: SUBMARINE-309 > URL: https://issues.apache.org/jira/browse/SUBMARINE-309 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Critical > > At this time, Submarine has two runtimes for yarn clusters, yarn service and > tony. So the installation guide should works for the both two rumtimes. As > the installation of calico and registrydns is only for yarn service, they > should be marked as optional. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-310) Support golang and java mixed compilation
[ https://issues.apache.org/jira/browse/SUBMARINE-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-310: - Target Version: 0.3.0 > Support golang and java mixed compilation > - > > Key: SUBMARINE-310 > URL: https://issues.apache.org/jira/browse/SUBMARINE-310 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Submarine needs to support both the yarn and k8s resource scheduling systems > for job operation. > Some modules (submitter-yarn) are developed using java, and some modules > (submarine-operator / submitter-k8s) are developed using golang. > So submarine needs to support both mvn and go build compilation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-261) Add exception class in module common utils
[ https://issues.apache.org/jira/browse/SUBMARINE-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-261: - Status: Patch Available (was: Open) Batch update: submit patches for these JIRAs, please use "Submit Patch" instead of adding a label to indicate PR available in the future. > Add exception class in module common utils > -- > > Key: SUBMARINE-261 > URL: https://issues.apache.org/jira/browse/SUBMARINE-261 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Add several exception classes for submarine modules. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-319) kind-cluster-build.sh doesn't invoke installed kind
[ https://issues.apache.org/jira/browse/SUBMARINE-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-319: - Status: Patch Available (was: Open) Batch update: submit patches for these JIRAs, please use "Submit Patch" instead of adding a label to indicate PR available in the future. > kind-cluster-build.sh doesn't invoke installed kind > --- > > Key: SUBMARINE-319 > URL: https://issues.apache.org/jira/browse/SUBMARINE-319 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > kind-cluster-build.sh can download kind automatically. But > kind-cluster-build.sh doesn't use it. kind-cluster-build.sh invoke kind > command which is defined in PATH variable. > We need to let users to move downloaded kind and kubectl to system path > locations. So that users can directly use kind and kubectl to interact with > the local k8s cluster -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-280) Reorganize dependencies of sub-modules in pom.xml
[ https://issues.apache.org/jira/browse/SUBMARINE-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-280: - Status: Patch Available (was: Open) Batch update: submit patches for these JIRAs, please use "Submit Patch" instead of adding a label to indicate PR available in the future. > Reorganize dependencies of sub-modules in pom.xml > -- > > Key: SUBMARINE-280 > URL: https://issues.apache.org/jira/browse/SUBMARINE-280 > Project: Apache Submarine > Issue Type: Improvement >Reporter: ShuMing Li >Assignee: ShuMing Li >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Now a lot of sub-modules are using dependencies in its self pom.xml > separately which is redundant and messy, so need to reorganize dependencies > of sub-modules which may only depend the parent dependencies management. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-299) Write a 'submarine' shell script
[
https://issues.apache.org/jira/browse/SUBMARINE-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated SUBMARINE-299:
-
Status: Patch Available (was: Open)
Batch update: submit patches for these JIRAs, please use "Submit Patch" instead
of adding a label to indicate PR available in the future.
> Write a 'submarine' shell script
>
>
> Key: SUBMARINE-299
> URL: https://issues.apache.org/jira/browse/SUBMARINE-299
> Project: Apache Submarine
> Issue Type: Improvement
>Reporter: Ryan Lo
>Assignee: Ryan Lo
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Write a shell script file called "submarine" under "${SUBMARINE_HOME}/bin"
> use case:
> {code:java}
> $SUBMARINE_HOME/bin/submarine job run --name tf-job-001 \
> --framework tensorflow \
> --verbose \
> --input_path "" \
> --num_workers 2 \
> --worker_resources memory=1G,vcores=1 \
> --num_ps 1 \
> --ps_resources memory=1G,vcores=1 \
> --worker_launch_cmd "myvenv.zip/venv/bin/python mnist_distributed.py --steps
> 2 --data_dir /tmp/data --working_dir /tmp/mode" \
> --ps_launch_cmd "myvenv.zip/venv/bin/python mnist_distributed.py --steps 2
> --data_dir /tmp/data --working_dir /tmp/mode" \
> --insecure \
> --conf
> tony.containers.resources=path-to/myvenv.zip#archive,path-to/mnist_distributed.py,path-to/tony-cli-0.3.13-all.jar{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-231) [Interpreter] Introducing zeppelin via submodule
[ https://issues.apache.org/jira/browse/SUBMARINE-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-231: - Status: Patch Available (was: Open) Batch update: submit patches for these JIRAs, please use "Submit Patch" instead of adding a label to indicate PR available in the future. > [Interpreter] Introducing zeppelin via submodule > > > Key: SUBMARINE-231 > URL: https://issues.apache.org/jira/browse/SUBMARINE-231 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine Interpreter >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > Labels: pull-request-available > Fix For: 0.3.0 > > Attachments: Untitled.png > > Time Spent: 20m > Remaining Estimate: 0h > > # First, fork Apache Zeppelin's code repository in the hadoopsubmarine > account, which ensures that hadoopsubmarine/zeppelin can always follow > apache/zeppelin code updates. > # Create a submarine branch in hadoopsubmarine/zeppelin and customize the > development on this branch. > # The hadoopsubmarine/zeppelin repository is contained via submodule in the > apache/submarine repository. > # In the /submarine/submarine-workbench/submarine-interpreter in the > apache/submarine repository, we import the JAR package of our customized > zeppelin interpreter in submodule for use. > Design doc: > [https://docs.google.com/document/d/1LJc5hRTWfIs6K5f7vMgGC_FUizeQf-qqiWICqO8SoR8/edit#] > !Untitled.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-304) add project encoding
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Status: Patch Available (was: Open) Batch update: submit patches for these JIRAs, please use "Submit Patch" instead of adding a label to indicate PR available in the future. > add project encoding > > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Labels: pull-request-available > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-302) Split the submitter-yarnservice from travis into job
[ https://issues.apache.org/jira/browse/SUBMARINE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-302: - Status: Patch Available (was: Open) Batch update: submit patches for these JIRAs, please use "Submit Patch" instead of adding a label to indicate PR available in the future. > Split the submitter-yarnservice from travis into job > > > Key: SUBMARINE-302 > URL: https://issues.apache.org/jira/browse/SUBMARINE-302 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The submitter-yarnservice depend on hadoop (3.1+), but it always break other > modules. So we'd better split it into job from travis at first. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Resolved] (SUBMARINE-178) Package license and notice into artifacts
[
https://issues.apache.org/jira/browse/SUBMARINE-178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan resolved SUBMARINE-178.
--
Resolution: Done
I think this is already done, closing this Jira as "Done", please reopen if
there's anything else to do.
> Package license and notice into artifacts
> -
>
> Key: SUBMARINE-178
> URL: https://issues.apache.org/jira/browse/SUBMARINE-178
> Project: Apache Submarine
> Issue Type: Improvement
>Reporter: Zhankun Tang
>Assignee: Zac Zhou
>Priority: Major
> Labels: submarine-external-repo
>
> The current distribution build process through "mvn clean package" contains
> only 4 jars.
> If we look at the hadoop's release artifacts. It contains src, binary
> tarballs, changelog, rat txt file, release note, and their signatures.
> {code:java}
> [ztang@release-vm target]$ ls artifacts/
> CHANGELOG.md hadoop-3.1.3-site.tar.gz hadoop-3.1.3.tar.gz
> CHANGELOG.md.asc hadoop-3.1.3-site.tar.gz.asc hadoop-3.1.3.tar.gz.asc
> CHANGELOG.md.sha512 hadoop-3.1.3-site.tar.gz.sha512 hadoop-3.1.3.tar.gz.sha512
> hadoop-3.1.3-rat.txt hadoop-3.1.3-src.tar.gz RELEASENOTES.md
> hadoop-3.1.3-rat.txt.asc hadoop-3.1.3-src.tar.gz.asc RELEASENOTES.md.asc
> hadoop-3.1.3-rat.txt.sha512 hadoop-3.1.3-src.tar.gz.sha512
> RELEASENOTES.md.sha512{code}
> After SUBMARINE-169, there're several improvements we need. Some of them
> could be seperate JIRAs.
> # Build source code tarball as well
> # When building source code tarball, package file LICENSE, NOTICE, folder
> "licenses" into source code tarball
> # The binary distribution should package "LICENSE-binary", "NOTICE-binary",
> folder "licenses-binary" besides the 4 jars.
> # Sign the artifacts after they're generated. Refer to hadoop's
> dev-support/bin/create-release or spark's release dev/make-distribution.sh
> # Release note and changelog generation
> [~yuan_zac], could you please take a look at this? Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-169) Revisit submarine license and notice
[ https://issues.apache.org/jira/browse/SUBMARINE-169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-169: - Resolution: Done Status: Resolved (was: Patch Available) I think this is already done and changes addressed before spin-off, closing this Jira as "Done", please reopen if there's anything else to do. > Revisit submarine license and notice > > > Key: SUBMARINE-169 > URL: https://issues.apache.org/jira/browse/SUBMARINE-169 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zhankun Tang >Assignee: Zhankun Tang >Priority: Major > Labels: submarine-external-repo > Attachments: SUBMARINE-169.001.patch > > > Refer to recent Spark and Hadoop's revisiting of license and notice. > [https://github.com/apache/spark/pull/21640] > HADOOP-15958 > > Dependencies Analysis of submarine project > [https://docs.google.com/spreadsheets/d/1_PASM-wpc8BC52son9BpvA6qGVtlx0pEP_UAGP4YHiE/edit?usp=sharing] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-168) [WEB] Submarine workspace/create project webpage
[ https://issues.apache.org/jira/browse/SUBMARINE-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-168: - Labels: (was: submarine-external-repo) > [WEB] Submarine workspace/create project webpage > > > Key: SUBMARINE-168 > URL: https://issues.apache.org/jira/browse/SUBMARINE-168 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine Workbench >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > > h3. What is this PR for? > Adjust the existing workspace/project page to add the following features: > # Create project. > # List project. > h3. What type of PR is it? > Feature > h3. What is the Jira issue? > https://issues.apache.org/jira/browse/SUBMARINE-168 > h3. How should this be tested? > [CI Pass|https://travis-ci.org/liuxunorg/submarine/builds/586090600] > h3. Screenshots (if appropriate) > [!https://user-images.githubusercontent.com/3677382/65053543-ace10900-d99e-11e9-893c-8053e1cc4b2b.gif|width=714,height=421!|https://user-images.githubusercontent.com/3677382/65053543-ace10900-d99e-11e9-893c-8053e1cc4b2b.gif] > h3. Questions: > * Does the licenses files need update? Yes/No > * Is there breaking changes for older versions? Yes/No > * Does this needs documentation? Yes/No -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-168) [WEB] Submarine workspace/create project webpage
[ https://issues.apache.org/jira/browse/SUBMARINE-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-168: - Component/s: Submarine Workbench > [WEB] Submarine workspace/create project webpage > > > Key: SUBMARINE-168 > URL: https://issues.apache.org/jira/browse/SUBMARINE-168 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine Workbench >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > Labels: submarine-external-repo > > h3. What is this PR for? > Adjust the existing workspace/project page to add the following features: > # Create project. > # List project. > h3. What type of PR is it? > Feature > h3. What is the Jira issue? > https://issues.apache.org/jira/browse/SUBMARINE-168 > h3. How should this be tested? > [CI Pass|https://travis-ci.org/liuxunorg/submarine/builds/586090600] > h3. Screenshots (if appropriate) > [!https://user-images.githubusercontent.com/3677382/65053543-ace10900-d99e-11e9-893c-8053e1cc4b2b.gif|width=714,height=421!|https://user-images.githubusercontent.com/3677382/65053543-ace10900-d99e-11e9-893c-8053e1cc4b2b.gif] > h3. Questions: > * Does the licenses files need update? Yes/No > * Is there breaking changes for older versions? Yes/No > * Does this needs documentation? Yes/No -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Resolved] (SUBMARINE-168) [WEB] Submarine workspace/create project webpage
[ https://issues.apache.org/jira/browse/SUBMARINE-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved SUBMARINE-168. -- Fix Version/s: 0.3.0 Resolution: Fixed Patch committed, closing. > [WEB] Submarine workspace/create project webpage > > > Key: SUBMARINE-168 > URL: https://issues.apache.org/jira/browse/SUBMARINE-168 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine Workbench >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > Fix For: 0.3.0 > > > h3. What is this PR for? > Adjust the existing workspace/project page to add the following features: > # Create project. > # List project. > h3. What type of PR is it? > Feature > h3. What is the Jira issue? > https://issues.apache.org/jira/browse/SUBMARINE-168 > h3. How should this be tested? > [CI Pass|https://travis-ci.org/liuxunorg/submarine/builds/586090600] > h3. Screenshots (if appropriate) > [!https://user-images.githubusercontent.com/3677382/65053543-ace10900-d99e-11e9-893c-8053e1cc4b2b.gif|width=714,height=421!|https://user-images.githubusercontent.com/3677382/65053543-ace10900-d99e-11e9-893c-8053e1cc4b2b.gif] > h3. Questions: > * Does the licenses files need update? Yes/No > * Is there breaking changes for older versions? Yes/No > * Does this needs documentation? Yes/No -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-167) [Umbrella] Submarine Workbench Notebook
[ https://issues.apache.org/jira/browse/SUBMARINE-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-167: - Labels: (was: submarine-external-repo) > [Umbrella] Submarine Workbench Notebook > --- > > Key: SUBMARINE-167 > URL: https://issues.apache.org/jira/browse/SUBMARINE-167 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > > h1. Workbench Web + Vue JS Prototype > h2. What is Vue JS? > What Vue JS is basically a progressive framework rather than an opinionated > Framework like angular js or ember js > It was Originally created by **Evan You** as a way to be a modern js > framework Something that you could get up and running quickly and get some > real results > And at the same time maintain a lot of the things that people love about > other modern js frameworks. > h2. Main features of Vue JS > * Virtual Dom > * Supports data binding (v-bind) > * Component-based > * Computed props > * Router > * Directives > * Watchers > * Templates (HTML based) > * Vuex (similar to redux) > I have a working prototype of Workbench-web in vuejs ready with full feature > parity -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-167) [Umbrella] Submarine Workbench Notebook
[ https://issues.apache.org/jira/browse/SUBMARINE-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-167: - Component/s: Submarine Workbench > [Umbrella] Submarine Workbench Notebook > --- > > Key: SUBMARINE-167 > URL: https://issues.apache.org/jira/browse/SUBMARINE-167 > Project: Apache Submarine > Issue Type: New Feature > Components: Submarine Workbench >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > > h1. Workbench Web + Vue JS Prototype > h2. What is Vue JS? > What Vue JS is basically a progressive framework rather than an opinionated > Framework like angular js or ember js > It was Originally created by **Evan You** as a way to be a modern js > framework Something that you could get up and running quickly and get some > real results > And at the same time maintain a lot of the things that people love about > other modern js frameworks. > h2. Main features of Vue JS > * Virtual Dom > * Supports data binding (v-bind) > * Component-based > * Computed props > * Router > * Directives > * Watchers > * Templates (HTML based) > * Vuex (similar to redux) > I have a working prototype of Workbench-web in vuejs ready with full feature > parity -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-165) [Umbrella] Submarine Workbench
[ https://issues.apache.org/jira/browse/SUBMARINE-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-165: - Component/s: Submarine Workbench > [Umbrella] Submarine Workbench > -- > > Key: SUBMARINE-165 > URL: https://issues.apache.org/jira/browse/SUBMARINE-165 > Project: Apache Submarine > Issue Type: New Feature > Components: Submarine Workbench >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > > Submarine can be managed via submarine Workbench: > 1. Support multi-user and multi-role > 2. Support for allocation management of server resources > 3. Support data development, algorithm development, and model launch through > notebook > 4. Support workflow workflow orchestration > 5. Support service monitoring after the model goes online > 6. Support Notebook > 7. Support Python, Spark ... execute. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-165) [Umbrella] Submarine Workbench
[ https://issues.apache.org/jira/browse/SUBMARINE-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-165: - Labels: (was: submarine-external-repo) > [Umbrella] Submarine Workbench > -- > > Key: SUBMARINE-165 > URL: https://issues.apache.org/jira/browse/SUBMARINE-165 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > > Submarine can be managed via submarine Workbench: > 1. Support multi-user and multi-role > 2. Support for allocation management of server resources > 3. Support data development, algorithm development, and model launch through > notebook > 4. Support workflow workflow orchestration > 5. Support service monitoring after the model goes online > 6. Support Notebook > 7. Support Python, Spark ... execute. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-164) [Umbrella] Mini-submarine
[ https://issues.apache.org/jira/browse/SUBMARINE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-164: - Component/s: Mini Submarine > [Umbrella] Mini-submarine > - > > Key: SUBMARINE-164 > URL: https://issues.apache.org/jira/browse/SUBMARINE-164 > Project: Apache Submarine > Issue Type: New Feature > Components: Mini Submarine >Reporter: Xun Liu >Assignee: Zac Zhou >Priority: Major > > Submarine development and a completely independent test environment with a > single docker image of mini submarine. > # submarine IDEA debug > # hadoop env > # submarine test > # hive metastore > # Spark -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-164) [Umbrella] Mini-submarine
[ https://issues.apache.org/jira/browse/SUBMARINE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-164: - Labels: (was: submarine-external-repo) > [Umbrella] Mini-submarine > - > > Key: SUBMARINE-164 > URL: https://issues.apache.org/jira/browse/SUBMARINE-164 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Xun Liu >Assignee: Zac Zhou >Priority: Major > > Submarine development and a completely independent test environment with a > single docker image of mini submarine. > # submarine IDEA debug > # hadoop env > # submarine test > # hive metastore > # Spark -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-163) [Umbrella] workbench Integration with github and gitlab
[ https://issues.apache.org/jira/browse/SUBMARINE-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-163: - Labels: (was: submarine-external-repo) > [Umbrella] workbench Integration with github and gitlab > --- > > Key: SUBMARINE-163 > URL: https://issues.apache.org/jira/browse/SUBMARINE-163 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Xun Liu >Assignee: zhulinhao >Priority: Major > > Workbench requires version management of the notebook via github and gitlab. > 1. The user sets the authentication information of github or gitlab in the > workbench. > 2. Users can submit code directly to the github or gitlab repository for > version management in the notebook. > 3. The user can list all versions in the github or gitlab repository in the > notebook and switch to the specified version code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-164) [Umbrella] Mini-submarine
[ https://issues.apache.org/jira/browse/SUBMARINE-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997035#comment-16997035 ] Wangda Tan commented on SUBMARINE-164: -- [~yuan_zac], I think majority of mini-submarine is already done and committed to 0.3.0, if my understanding is true, I suggest to close this ticket and move the k8s + mini-submarine to a separate Jira. > [Umbrella] Mini-submarine > - > > Key: SUBMARINE-164 > URL: https://issues.apache.org/jira/browse/SUBMARINE-164 > Project: Apache Submarine > Issue Type: New Feature > Components: Mini Submarine >Reporter: Xun Liu >Assignee: Zac Zhou >Priority: Major > > Submarine development and a completely independent test environment with a > single docker image of mini submarine. > # submarine IDEA debug > # hadoop env > # submarine test > # hive metastore > # Spark -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-103) [Umbrella] Submarine RESTful interface
[
https://issues.apache.org/jira/browse/SUBMARINE-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated SUBMARINE-103:
-
Labels: (was: submarine-external-repo)
> [Umbrella] Submarine RESTful interface
> --
>
> Key: SUBMARINE-103
> URL: https://issues.apache.org/jira/browse/SUBMARINE-103
> Project: Apache Submarine
> Issue Type: New Feature
>Reporter: Xun Liu
>Assignee: Xun Liu
>Priority: Major
>
> Submarine web RESTful interface service.
> You can do the following to make submarine-web use a RESTful interface.
> 1. Package output submarine-web web site
> {code:java}
> cd submarine-web
> yarn install
> yarn run build
> {code}
> The yarn will output the submarine-web web site to the submarine-web/dist
> directory.
>
> 2. Run Submarine-Server
> Run the submarine-server project directly in the idea.
> The jetty server in the submarine-server loads the web content in the
> submarine-web/dist directory by default.
>
> 3. [http://127.0.0.1:8080|http://127.0.0.1:8080/]
> 4. input admin/admin in submarine workbench login page.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-103) [Umbrella] Submarine RESTful interface
[
https://issues.apache.org/jira/browse/SUBMARINE-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997037#comment-16997037
]
Wangda Tan commented on SUBMARINE-103:
--
[~liuxun323], any additional work for this Jira? I saw all sub JIRAs are
committed, if no, please close this ticket.
> [Umbrella] Submarine RESTful interface
> --
>
> Key: SUBMARINE-103
> URL: https://issues.apache.org/jira/browse/SUBMARINE-103
> Project: Apache Submarine
> Issue Type: New Feature
>Reporter: Xun Liu
>Assignee: Xun Liu
>Priority: Major
> Labels: submarine-external-repo
>
> Submarine web RESTful interface service.
> You can do the following to make submarine-web use a RESTful interface.
> 1. Package output submarine-web web site
> {code:java}
> cd submarine-web
> yarn install
> yarn run build
> {code}
> The yarn will output the submarine-web web site to the submarine-web/dist
> directory.
>
> 2. Run Submarine-Server
> Run the submarine-server project directly in the idea.
> The jetty server in the submarine-server loads the web content in the
> submarine-web/dist directory by default.
>
> 3. [http://127.0.0.1:8080|http://127.0.0.1:8080/]
> 4. input admin/admin in submarine workbench login page.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-103) [Umbrella] Submarine RESTful interface
[
https://issues.apache.org/jira/browse/SUBMARINE-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated SUBMARINE-103:
-
Component/s: Submarine Server
> [Umbrella] Submarine RESTful interface
> --
>
> Key: SUBMARINE-103
> URL: https://issues.apache.org/jira/browse/SUBMARINE-103
> Project: Apache Submarine
> Issue Type: New Feature
> Components: Submarine Server
>Reporter: Xun Liu
>Assignee: Xun Liu
>Priority: Major
>
> Submarine web RESTful interface service.
> You can do the following to make submarine-web use a RESTful interface.
> 1. Package output submarine-web web site
> {code:java}
> cd submarine-web
> yarn install
> yarn run build
> {code}
> The yarn will output the submarine-web web site to the submarine-web/dist
> directory.
>
> 2. Run Submarine-Server
> Run the submarine-server project directly in the idea.
> The jetty server in the submarine-server loads the web content in the
> submarine-web/dist directory by default.
>
> 3. [http://127.0.0.1:8080|http://127.0.0.1:8080/]
> 4. input admin/admin in submarine workbench login page.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-163) Workbench Integration with github and gitlab
[ https://issues.apache.org/jira/browse/SUBMARINE-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-163: - Summary: Workbench Integration with github and gitlab (was: [Umbrella] workbench Integration with github and gitlab) > Workbench Integration with github and gitlab > > > Key: SUBMARINE-163 > URL: https://issues.apache.org/jira/browse/SUBMARINE-163 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Xun Liu >Assignee: zhulinhao >Priority: Major > > Workbench requires version management of the notebook via github and gitlab. > 1. The user sets the authentication information of github or gitlab in the > workbench. > 2. Users can submit code directly to the github or gitlab repository for > version management in the notebook. > 3. The user can list all versions in the github or gitlab repository in the > notebook and switch to the specified version code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-159) [WEB] Department Select Component
[ https://issues.apache.org/jira/browse/SUBMARINE-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-159: - Component/s: Submarine Workbench > [WEB] Department Select Component > - > > Key: SUBMARINE-159 > URL: https://issues.apache.org/jira/browse/SUBMARINE-159 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine Workbench >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > > The department drop-down box is often needed in the web front end. > So package Department Select into a Component, > Will speed up development efficiency. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-163) Workbench Integration with github and gitlab
[ https://issues.apache.org/jira/browse/SUBMARINE-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997040#comment-16997040 ] Wangda Tan commented on SUBMARINE-163: -- Didn't see children ticket, so removed "umbrella" > Workbench Integration with github and gitlab > > > Key: SUBMARINE-163 > URL: https://issues.apache.org/jira/browse/SUBMARINE-163 > Project: Apache Submarine > Issue Type: New Feature > Components: Submarine Workbench >Reporter: Xun Liu >Assignee: zhulinhao >Priority: Major > > Workbench requires version management of the notebook via github and gitlab. > 1. The user sets the authentication information of github or gitlab in the > workbench. > 2. Users can submit code directly to the github or gitlab repository for > version management in the notebook. > 3. The user can list all versions in the github or gitlab repository in the > notebook and switch to the specified version code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-159) [WEB] Department Select Component
[ https://issues.apache.org/jira/browse/SUBMARINE-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-159: - Labels: (was: submarine-external-repo) > [WEB] Department Select Component > - > > Key: SUBMARINE-159 > URL: https://issues.apache.org/jira/browse/SUBMARINE-159 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > > The department drop-down box is often needed in the web front end. > So package Department Select into a Component, > Will speed up development efficiency. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-163) Workbench Integration with github and gitlab
[ https://issues.apache.org/jira/browse/SUBMARINE-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-163: - Component/s: Submarine Workbench > Workbench Integration with github and gitlab > > > Key: SUBMARINE-163 > URL: https://issues.apache.org/jira/browse/SUBMARINE-163 > Project: Apache Submarine > Issue Type: New Feature > Components: Submarine Workbench >Reporter: Xun Liu >Assignee: zhulinhao >Priority: Major > > Workbench requires version management of the notebook via github and gitlab. > 1. The user sets the authentication information of github or gitlab in the > workbench. > 2. Users can submit code directly to the github or gitlab repository for > version management in the notebook. > 3. The user can list all versions in the github or gitlab repository in the > notebook and switch to the specified version code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-151) [SDK] Support mysql as the storage backend
[ https://issues.apache.org/jira/browse/SUBMARINE-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-151: - Component/s: Submarine SDK > [SDK] Support mysql as the storage backend > -- > > Key: SUBMARINE-151 > URL: https://issues.apache.org/jira/browse/SUBMARINE-151 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine SDK >Reporter: kevin su >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-151) [SDK] Support mysql as the storage backend
[ https://issues.apache.org/jira/browse/SUBMARINE-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-151: - Labels: (was: submarine-external-repo) > [SDK] Support mysql as the storage backend > -- > > Key: SUBMARINE-151 > URL: https://issues.apache.org/jira/browse/SUBMARINE-151 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: kevin su >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-150) [SDK] Support hdfs as the storage backend
[ https://issues.apache.org/jira/browse/SUBMARINE-150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-150: - Labels: (was: submarine-external-repo) > [SDK] Support hdfs as the storage backend > - > > Key: SUBMARINE-150 > URL: https://issues.apache.org/jira/browse/SUBMARINE-150 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: kevin su >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-150) [SDK] Support hdfs as the storage backend
[ https://issues.apache.org/jira/browse/SUBMARINE-150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-150: - Component/s: Submarine SDK > [SDK] Support hdfs as the storage backend > - > > Key: SUBMARINE-150 > URL: https://issues.apache.org/jira/browse/SUBMARINE-150 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine SDK >Reporter: kevin su >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-133) [Umbrella] Submarine Database
[ https://issues.apache.org/jira/browse/SUBMARINE-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-133: - Labels: (was: submarine-external-repo) > [Umbrella] Submarine Database > - > > Key: SUBMARINE-133 > URL: https://issues.apache.org/jira/browse/SUBMARINE-133 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-133) [Umbrella] Submarine Database
[ https://issues.apache.org/jira/browse/SUBMARINE-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-133: - Component/s: Submarine Database > [Umbrella] Submarine Database > - > > Key: SUBMARINE-133 > URL: https://issues.apache.org/jira/browse/SUBMARINE-133 > Project: Apache Submarine > Issue Type: New Feature > Components: Submarine Database >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-319) kind-cluster-build.sh doesn't invoke installed kind
[ https://issues.apache.org/jira/browse/SUBMARINE-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-319: - Labels: (was: pull-request-available) > kind-cluster-build.sh doesn't invoke installed kind > --- > > Key: SUBMARINE-319 > URL: https://issues.apache.org/jira/browse/SUBMARINE-319 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > kind-cluster-build.sh can download kind automatically. But > kind-cluster-build.sh doesn't use it. kind-cluster-build.sh invoke kind > command which is defined in PATH variable. > We need to let users to move downloaded kind and kubectl to system path > locations. So that users can directly use kind and kubectl to interact with > the local k8s cluster -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-310) Support golang and java mixed compilation
[ https://issues.apache.org/jira/browse/SUBMARINE-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-310: - Labels: (was: pull-request-available) > Support golang and java mixed compilation > - > > Key: SUBMARINE-310 > URL: https://issues.apache.org/jira/browse/SUBMARINE-310 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Submarine needs to support both the yarn and k8s resource scheduling systems > for job operation. > Some modules (submitter-yarn) are developed using java, and some modules > (submarine-operator / submitter-k8s) are developed using golang. > So submarine needs to support both mvn and go build compilation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-231) [Interpreter] Introducing zeppelin via submodule
[ https://issues.apache.org/jira/browse/SUBMARINE-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-231: - Labels: (was: pull-request-available) > [Interpreter] Introducing zeppelin via submodule > > > Key: SUBMARINE-231 > URL: https://issues.apache.org/jira/browse/SUBMARINE-231 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submarine Interpreter >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > Fix For: 0.3.0 > > Attachments: Untitled.png > > Time Spent: 20m > Remaining Estimate: 0h > > # First, fork Apache Zeppelin's code repository in the hadoopsubmarine > account, which ensures that hadoopsubmarine/zeppelin can always follow > apache/zeppelin code updates. > # Create a submarine branch in hadoopsubmarine/zeppelin and customize the > development on this branch. > # The hadoopsubmarine/zeppelin repository is contained via submodule in the > apache/submarine repository. > # In the /submarine/submarine-workbench/submarine-interpreter in the > apache/submarine repository, we import the JAR package of our customized > zeppelin interpreter in submodule for use. > Design doc: > [https://docs.google.com/document/d/1LJc5hRTWfIs6K5f7vMgGC_FUizeQf-qqiWICqO8SoR8/edit#] > !Untitled.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-261) Add exception class in module common utils
[ https://issues.apache.org/jira/browse/SUBMARINE-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-261: - Labels: (was: pull-request-available) > Add exception class in module common utils > -- > > Key: SUBMARINE-261 > URL: https://issues.apache.org/jira/browse/SUBMARINE-261 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add several exception classes for submarine modules. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-304) add project encoding
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Labels: (was: pull-request-available) > add project encoding > > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-302) Split the submitter-yarnservice from travis into job
[ https://issues.apache.org/jira/browse/SUBMARINE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-302: - Labels: (was: pull-request-available) > Split the submitter-yarnservice from travis into job > > > Key: SUBMARINE-302 > URL: https://issues.apache.org/jira/browse/SUBMARINE-302 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The submitter-yarnservice depend on hadoop (3.1+), but it always break other > modules. So we'd better split it into job from travis at first. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-280) Reorganize dependencies of sub-modules in pom.xml
[ https://issues.apache.org/jira/browse/SUBMARINE-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-280: - Labels: (was: pull-request-available) > Reorganize dependencies of sub-modules in pom.xml > -- > > Key: SUBMARINE-280 > URL: https://issues.apache.org/jira/browse/SUBMARINE-280 > Project: Apache Submarine > Issue Type: Improvement >Reporter: ShuMing Li >Assignee: ShuMing Li >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Now a lot of sub-modules are using dependencies in its self pom.xml > separately which is redundant and messy, so need to reorganize dependencies > of sub-modules which may only depend the parent dependencies management. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-299) Write a 'submarine' shell script
[
https://issues.apache.org/jira/browse/SUBMARINE-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated SUBMARINE-299:
-
Labels: (was: pull-request-available)
> Write a 'submarine' shell script
>
>
> Key: SUBMARINE-299
> URL: https://issues.apache.org/jira/browse/SUBMARINE-299
> Project: Apache Submarine
> Issue Type: Improvement
>Reporter: Ryan Lo
>Assignee: Ryan Lo
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Write a shell script file called "submarine" under "${SUBMARINE_HOME}/bin"
> use case:
> {code:java}
> $SUBMARINE_HOME/bin/submarine job run --name tf-job-001 \
> --framework tensorflow \
> --verbose \
> --input_path "" \
> --num_workers 2 \
> --worker_resources memory=1G,vcores=1 \
> --num_ps 1 \
> --ps_resources memory=1G,vcores=1 \
> --worker_launch_cmd "myvenv.zip/venv/bin/python mnist_distributed.py --steps
> 2 --data_dir /tmp/data --working_dir /tmp/mode" \
> --ps_launch_cmd "myvenv.zip/venv/bin/python mnist_distributed.py --steps 2
> --data_dir /tmp/data --working_dir /tmp/mode" \
> --insecure \
> --conf
> tony.containers.resources=path-to/myvenv.zip#archive,path-to/mnist_distributed.py,path-to/tony-cli-0.3.13-all.jar{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-319) kind-cluster-build.sh doesn't invoke installed kind
[ https://issues.apache.org/jira/browse/SUBMARINE-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-319: - Component/s: Cloud-native Deployment > kind-cluster-build.sh doesn't invoke installed kind > --- > > Key: SUBMARINE-319 > URL: https://issues.apache.org/jira/browse/SUBMARINE-319 > Project: Apache Submarine > Issue Type: Sub-task > Components: Cloud-native Deployment >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > kind-cluster-build.sh can download kind automatically. But > kind-cluster-build.sh doesn't use it. kind-cluster-build.sh invoke kind > command which is defined in PATH variable. > We need to let users to move downloaded kind and kubectl to system path > locations. So that users can directly use kind and kubectl to interact with > the local k8s cluster -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-319) kind-cluster-build.sh doesn't invoke installed kind
[ https://issues.apache.org/jira/browse/SUBMARINE-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-319: - Target Version: 0.4.0 > kind-cluster-build.sh doesn't invoke installed kind > --- > > Key: SUBMARINE-319 > URL: https://issues.apache.org/jira/browse/SUBMARINE-319 > Project: Apache Submarine > Issue Type: Sub-task > Components: Cloud-native Deployment >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > kind-cluster-build.sh can download kind automatically. But > kind-cluster-build.sh doesn't use it. kind-cluster-build.sh invoke kind > command which is defined in PATH variable. > We need to let users to move downloaded kind and kubectl to system path > locations. So that users can directly use kind and kubectl to interact with > the local k8s cluster -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Resolved] (SUBMARINE-232) [Umbrella] Custom Zeppelin Submodule
[ https://issues.apache.org/jira/browse/SUBMARINE-232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved SUBMARINE-232. -- Resolution: Won't Fix Discussed with [~liuxun] this Jira is not needed anymore, closing as won't fix. > [Umbrella] Custom Zeppelin Submodule > > > Key: SUBMARINE-232 > URL: https://issues.apache.org/jira/browse/SUBMARINE-232 > Project: Apache Submarine > Issue Type: New Feature > Components: Submodule Zeppelin >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > > 2. Interpreter > Submarine provides multi-language execution for notebooks in workbench > through interpreters in a variety of programming languages (for example, > python interpreter, spark interpreter). > 2.1 Interpreter execution flow > 1. The user enters the code in the workbench notebook and clicks the execute > button. First the workbench server starts the interpreter process. > 2. The workbench server sends the user's code to the interpreter process via > the interface for execution, and gets the execution result, returning to the > workbench notebook for display. > 2.2 Integrate Zeppelin Interpreter > Zeppelin has this mature multilingual interpreter that can be introduced for > use as a Submarine interpreter, but Submarine has some specific requirements > for the use of the interpreter, such as Submarine's need to extend the > interface of the Zeppelin interpreter, etc., so Zeppelin is required. The > interface layer of the interpreter is modified to some extent. > At the same time, in order to be consistent with the development of the > Zeppelin interpreter, we used submodule to customize the Zeppelin interpreter > to a certain extent. > Design Doc: > https://docs.google.com/document/d/1LJc5hRTWfIs6K5f7vMgGC_FUizeQf-qqiWICqO8SoR8/edit# -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Resolved] (SUBMARINE-233) [SUBMODULE] Delete unused modules In hadoopsubmarine/zeppelin repo
[ https://issues.apache.org/jira/browse/SUBMARINE-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved SUBMARINE-233. -- Resolution: Won't Fix Discussed with [~liuxun] this Jira is not needed anymore, closing as won't fix. > [SUBMODULE] Delete unused modules In hadoopsubmarine/zeppelin repo > -- > > Key: SUBMARINE-233 > URL: https://issues.apache.org/jira/browse/SUBMARINE-233 > Project: Apache Submarine > Issue Type: Sub-task > Components: Submodule Zeppelin >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > > Delete unused modules In submodule zeppelin, > Avoid introducing too many zeppelin modules in submarine, > Hard to manage and will compile too long. > https://github.com/hadoopsubmarine/zeppelin -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-304) add project encoding
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Description: Update: Saw pom.xml under Tony missing the two properties, suggest to fix. !image-2019-11-27-12-09-36-148.png|width=408,height=80! was:!image-2019-11-27-12-09-36-148.png|width=408,height=80! > add project encoding > > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > Update: Saw pom.xml under Tony missing the two properties, suggest to fix. > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Issue Comment Deleted] (SUBMARINE-304) add project encoding
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Comment: was deleted (was: 不知道你想要实现什么功能。 你可以加我的微信：liuxun_org 方便一下) > add project encoding > > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > Update: Saw pom.xml under Tony missing the two properties, suggest to fix. > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-304) add project encoding
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997062#comment-16997062 ] Wangda Tan commented on SUBMARINE-304: -- Update description. > add project encoding > > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > Update: Saw pom.xml under Tony missing the two properties, suggest to fix. > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Issue Comment Deleted] (SUBMARINE-304) add project encoding
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Comment: was deleted (was: 看到Tony项目下边的pom.xml中缺少了这两个,应该在properties中添加下。) > add project encoding > > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > Update: Saw pom.xml under Tony missing the two properties, suggest to fix. > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-304) Add project encoding in pom.xml of TonY project
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Summary: Add project encoding in pom.xml of TonY project (was: add project encoding) > Add project encoding in pom.xml of TonY project > --- > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > Update: Saw pom.xml under Tony missing the two properties, suggest to fix. > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-302) Split the submitter-yarnservice from travis into job
[ https://issues.apache.org/jira/browse/SUBMARINE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-302: - Component/s: CI/CD > Split the submitter-yarnservice from travis into job > > > Key: SUBMARINE-302 > URL: https://issues.apache.org/jira/browse/SUBMARINE-302 > Project: Apache Submarine > Issue Type: Improvement > Components: CI/CD >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The submitter-yarnservice depend on hadoop (3.1+), but it always break other > modules. So we'd better split it into job from travis at first. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-302) Split the submitter-yarnservice from travis into job
[ https://issues.apache.org/jira/browse/SUBMARINE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-302: - Target Version: 0.3.0 > Split the submitter-yarnservice from travis into job > > > Key: SUBMARINE-302 > URL: https://issues.apache.org/jira/browse/SUBMARINE-302 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The submitter-yarnservice depend on hadoop (3.1+), but it always break other > modules. So we'd better split it into job from travis at first. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-302) Split the submitter-yarnservice from travis into job
[ https://issues.apache.org/jira/browse/SUBMARINE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997064#comment-16997064 ] Wangda Tan commented on SUBMARINE-302: -- Saw the Jira pending commit for a while, do we still need this Jira? [~jiwq] , [~liuxun323] > Split the submitter-yarnservice from travis into job > > > Key: SUBMARINE-302 > URL: https://issues.apache.org/jira/browse/SUBMARINE-302 > Project: Apache Submarine > Issue Type: Improvement > Components: CI/CD >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The submitter-yarnservice depend on hadoop (3.1+), but it always break other > modules. So we'd better split it into job from travis at first. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-258) [SS]Add k8s job submitter in submarine server to run TF job on k8s
[ https://issues.apache.org/jira/browse/SUBMARINE-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-258: - Component/s: K8s Submitter > [SS]Add k8s job submitter in submarine server to run TF job on k8s > -- > > Key: SUBMARINE-258 > URL: https://issues.apache.org/jira/browse/SUBMARINE-258 > Project: Apache Submarine > Issue Type: Sub-task > Components: K8s Submitter >Reporter: Zhankun Tang >Assignee: Wanqiang Ji >Priority: Major > > As a PoC, we need several below steps to run the TF job on k8s. > # The submarine server can run as an independent daemon > # tf-operator deployed in a k8s cluster. Use kubeflow/tf-operator for now > # REST API in submarine server to accept a generic TF job spec > # submitter-k8s translate the spec into tf-operator format and use k8s > client API to submit this to the cluster > # REST API to get tf job status from k8s and parse it to a standard format > to be consumed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-258) [SS]Add k8s job submitter in submarine server to run TF job on k8s
[ https://issues.apache.org/jira/browse/SUBMARINE-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997065#comment-16997065 ] Wangda Tan commented on SUBMARINE-258: -- [~jiwq], do you still need this Jira? Can we move all K8s-submitter related efforts to a new umbrella, and add the component "[K8s Submitter|https://issues.apache.org/jira/issues/?jql=project+%3D+SUBMARINE+AND+component+%3D+%22K8s+Submitter%22]";? Now it is a bit hard to track these tickets. > [SS]Add k8s job submitter in submarine server to run TF job on k8s > -- > > Key: SUBMARINE-258 > URL: https://issues.apache.org/jira/browse/SUBMARINE-258 > Project: Apache Submarine > Issue Type: Sub-task > Components: K8s Submitter >Reporter: Zhankun Tang >Assignee: Wanqiang Ji >Priority: Major > > As a PoC, we need several below steps to run the TF job on k8s. > # The submarine server can run as an independent daemon > # tf-operator deployed in a k8s cluster. Use kubeflow/tf-operator for now > # REST API in submarine server to accept a generic TF job spec > # submitter-k8s translate the spec into tf-operator format and use k8s > client API to submit this to the cluster > # REST API to get tf job status from k8s and parse it to a standard format > to be consumed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-282) Moving the JsonResponse related class to commons-utils module
[ https://issues.apache.org/jira/browse/SUBMARINE-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-282: - Component/s: Commons > Moving the JsonResponse related class to commons-utils module > - > > Key: SUBMARINE-282 > URL: https://issues.apache.org/jira/browse/SUBMARINE-282 > Project: Apache Submarine > Issue Type: Sub-task > Components: Commons >Reporter: Wanqiang Ji >Assignee: zhulinhao >Priority: Major > > # Due to the JsonResponse can be used for submarine-workbench and > submarine-server, so we should move it to commons-utils module. > # Remove the automatic annotation function of dictionary, and the dictionary > value required by the restful service in the background is returned through > SQL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-300) Add UT for submitter-k8s to verify the training result
[ https://issues.apache.org/jira/browse/SUBMARINE-300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-300: - Component/s: K8s Submitter > Add UT for submitter-k8s to verify the training result > -- > > Key: SUBMARINE-300 > URL: https://issues.apache.org/jira/browse/SUBMARINE-300 > Project: Apache Submarine > Issue Type: Sub-task > Components: K8s Submitter >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Critical > > In the submitter-k8s module missing the UT to verify the tf-operator's > training result. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-281) Remove Chinese characters from source code
[ https://issues.apache.org/jira/browse/SUBMARINE-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-281: - Labels: trivial (was: ) > Remove Chinese characters from source code > -- > > Key: SUBMARINE-281 > URL: https://issues.apache.org/jira/browse/SUBMARINE-281 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zhankun Tang >Priority: Minor > Labels: trivial > > There are mock data with Chinese characters left in source code. For > instance, SysUserRestApi#info() method leaves several Chinese characters. > We should find and remove them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-282) Moving the JsonResponse related class to commons-utils module
[ https://issues.apache.org/jira/browse/SUBMARINE-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-282: - Target Version: 0.4.0 > Moving the JsonResponse related class to commons-utils module > - > > Key: SUBMARINE-282 > URL: https://issues.apache.org/jira/browse/SUBMARINE-282 > Project: Apache Submarine > Issue Type: Sub-task > Components: Commons >Reporter: Wanqiang Ji >Assignee: zhulinhao >Priority: Major > > # Due to the JsonResponse can be used for submarine-workbench and > submarine-server, so we should move it to commons-utils module. > # Remove the automatic annotation function of dictionary, and the dictionary > value required by the restful service in the background is returned through > SQL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-304) Add project encoding in pom.xml of TonY project
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Component/s: Build > Add project encoding in pom.xml of TonY project > --- > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish > Components: Build >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > Update: Saw pom.xml under Tony missing the two properties, suggest to fix. > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-304) Add project encoding in pom.xml of TonY project
[ https://issues.apache.org/jira/browse/SUBMARINE-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-304: - Target Version: 0.4.0 > Add project encoding in pom.xml of TonY project > --- > > Key: SUBMARINE-304 > URL: https://issues.apache.org/jira/browse/SUBMARINE-304 > Project: Apache Submarine > Issue Type: Wish > Components: Build >Reporter: xzh_dz >Priority: Major > Attachments: image-2019-11-27-12-09-36-148.png > > Time Spent: 10m > Remaining Estimate: 0h > > Update: Saw pom.xml under Tony missing the two properties, suggest to fix. > !image-2019-11-27-12-09-36-148.png|width=408,height=80! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-280) Reorganize dependencies of sub-modules in pom.xml
[ https://issues.apache.org/jira/browse/SUBMARINE-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-280: - Target Version: 0.4.0 > Reorganize dependencies of sub-modules in pom.xml > -- > > Key: SUBMARINE-280 > URL: https://issues.apache.org/jira/browse/SUBMARINE-280 > Project: Apache Submarine > Issue Type: Improvement > Components: Build >Reporter: ShuMing Li >Assignee: ShuMing Li >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Now a lot of sub-modules are using dependencies in its self pom.xml > separately which is redundant and messy, so need to reorganize dependencies > of sub-modules which may only depend the parent dependencies management. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-281) Remove Chinese characters from source code
[ https://issues.apache.org/jira/browse/SUBMARINE-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-281: - Target Version: 0.4.0 > Remove Chinese characters from source code > -- > > Key: SUBMARINE-281 > URL: https://issues.apache.org/jira/browse/SUBMARINE-281 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zhankun Tang >Priority: Minor > Labels: trivial > > There are mock data with Chinese characters left in source code. For > instance, SysUserRestApi#info() method leaves several Chinese characters. > We should find and remove them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-280) Reorganize dependencies of sub-modules in pom.xml
[ https://issues.apache.org/jira/browse/SUBMARINE-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-280: - Component/s: Build > Reorganize dependencies of sub-modules in pom.xml > -- > > Key: SUBMARINE-280 > URL: https://issues.apache.org/jira/browse/SUBMARINE-280 > Project: Apache Submarine > Issue Type: Improvement > Components: Build >Reporter: ShuMing Li >Assignee: ShuMing Li >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Now a lot of sub-modules are using dependencies in its self pom.xml > separately which is redundant and messy, so need to reorganize dependencies > of sub-modules which may only depend the parent dependencies management. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-271) [Umbrella] Tony feature updating
[ https://issues.apache.org/jira/browse/SUBMARINE-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-271: - Component/s: TonY > [Umbrella] Tony feature updating > > > Key: SUBMARINE-271 > URL: https://issues.apache.org/jira/browse/SUBMARINE-271 > Project: Apache Submarine > Issue Type: Improvement > Components: TonY >Reporter: Zac Zhou >Assignee: Lisheng Sun >Priority: Major > > As tony is developed rapidly, we need to sync Tony code and support its new > features. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-276) Submarine runtime class config should be defined in the new configuration class
[ https://issues.apache.org/jira/browse/SUBMARINE-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-276: - Labels: (was: pull-request-available) > Submarine runtime class config should be defined in the new configuration > class > --- > > Key: SUBMARINE-276 > URL: https://issues.apache.org/jira/browse/SUBMARINE-276 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Fix For: 0.3.0 > > Time Spent: 20m > Remaining Estimate: 0h > > There are two SubmarineConfiguration classes in the submarine project. > Let's combine them into the new configuration class > org.apache.submarine.commons.utils.SubmarineConfiguration. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-198) Update tony submodule to support MXNet
[ https://issues.apache.org/jira/browse/SUBMARINE-198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-198: - Component/s: TonY > Update tony submodule to support MXNet > -- > > Key: SUBMARINE-198 > URL: https://issues.apache.org/jira/browse/SUBMARINE-198 > Project: Apache Submarine > Issue Type: Sub-task > Components: TonY >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Fix For: 0.3.0 > > Time Spent: 20m > Remaining Estimate: 0h > > MXNet(CPU only) is supported by tony now: > [https://github.com/linkedin/TonY/commit/014a850eec1668ef4530fbe18d8da7986fb62524] > Let's update the tony submodule. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-276) Submarine runtime class config should be defined in the new configuration class
[ https://issues.apache.org/jira/browse/SUBMARINE-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-276: - Component/s: TonY > Submarine runtime class config should be defined in the new configuration > class > --- > > Key: SUBMARINE-276 > URL: https://issues.apache.org/jira/browse/SUBMARINE-276 > Project: Apache Submarine > Issue Type: Sub-task > Components: TonY >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Fix For: 0.3.0 > > Time Spent: 20m > Remaining Estimate: 0h > > There are two SubmarineConfiguration classes in the submarine project. > Let's combine them into the new configuration class > org.apache.submarine.commons.utils.SubmarineConfiguration. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-198) Update tony submodule to support MXNet
[ https://issues.apache.org/jira/browse/SUBMARINE-198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-198: - Labels: (was: pull-request-available) > Update tony submodule to support MXNet > -- > > Key: SUBMARINE-198 > URL: https://issues.apache.org/jira/browse/SUBMARINE-198 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Fix For: 0.3.0 > > Time Spent: 20m > Remaining Estimate: 0h > > MXNet(CPU only) is supported by tony now: > [https://github.com/linkedin/TonY/commit/014a850eec1668ef4530fbe18d8da7986fb62524] > Let's update the tony submodule. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-261) Add exception class in module common utils
[ https://issues.apache.org/jira/browse/SUBMARINE-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-261: - Component/s: Commons > Add exception class in module common utils > -- > > Key: SUBMARINE-261 > URL: https://issues.apache.org/jira/browse/SUBMARINE-261 > Project: Apache Submarine > Issue Type: Improvement > Components: Commons >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add several exception classes for submarine modules. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Resolved] (SUBMARINE-271) [Umbrella] Tony feature updating
[ https://issues.apache.org/jira/browse/SUBMARINE-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved SUBMARINE-271. -- Resolution: Done All children resolved, closing as done. > [Umbrella] Tony feature updating > > > Key: SUBMARINE-271 > URL: https://issues.apache.org/jira/browse/SUBMARINE-271 > Project: Apache Submarine > Issue Type: Improvement > Components: TonY >Reporter: Zac Zhou >Assignee: Lisheng Sun >Priority: Major > > As tony is developed rapidly, we need to sync Tony code and support its new > features. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-261) Add exception class in module common utils
[ https://issues.apache.org/jira/browse/SUBMARINE-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-261: - Target Version: 0.3.0 > Add exception class in module common utils > -- > > Key: SUBMARINE-261 > URL: https://issues.apache.org/jira/browse/SUBMARINE-261 > Project: Apache Submarine > Issue Type: Improvement > Components: Commons >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add several exception classes for submarine modules. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-253) Update Submarine Docs:
[ https://issues.apache.org/jira/browse/SUBMARINE-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-253: - Summary: Update Submarine Docs: (was: Update doc) > Update Submarine Docs: > --- > > Key: SUBMARINE-253 > URL: https://issues.apache.org/jira/browse/SUBMARINE-253 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > I am following the documentation and I think it can use some help -- grammar > errors, remove ambiguity, incorrect/outdated instructions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-253) Update Submarine Docs: grammar errors, remove ambiguity, incorrect/outdated instructions.
[ https://issues.apache.org/jira/browse/SUBMARINE-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-253: - Component/s: Doc > Update Submarine Docs: grammar errors, remove ambiguity, incorrect/outdated > instructions. > - > > Key: SUBMARINE-253 > URL: https://issues.apache.org/jira/browse/SUBMARINE-253 > Project: Apache Submarine > Issue Type: Improvement > Components: Doc >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > I am following the documentation and I think it can use some help -- grammar > errors, remove ambiguity, incorrect/outdated instructions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-261) Add exception class in module common utils
[ https://issues.apache.org/jira/browse/SUBMARINE-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997069#comment-16997069 ] Wangda Tan commented on SUBMARINE-261: -- [~yuan_zac], do we still need this Jira for 0.3.0? Saw the patch pending update for a while. > Add exception class in module common utils > -- > > Key: SUBMARINE-261 > URL: https://issues.apache.org/jira/browse/SUBMARINE-261 > Project: Apache Submarine > Issue Type: Improvement > Components: Commons >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add several exception classes for submarine modules. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-253) Update Submarine Docs: grammar errors, remove ambiguity, incorrect/outdated instructions.
[ https://issues.apache.org/jira/browse/SUBMARINE-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-253: - Summary: Update Submarine Docs: grammar errors, remove ambiguity, incorrect/outdated instructions. (was: Update Submarine Docs: ) > Update Submarine Docs: grammar errors, remove ambiguity, incorrect/outdated > instructions. > - > > Key: SUBMARINE-253 > URL: https://issues.apache.org/jira/browse/SUBMARINE-253 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > I am following the documentation and I think it can use some help -- grammar > errors, remove ambiguity, incorrect/outdated instructions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-242) [E2E] Submarine integration test workbench's start script
[ https://issues.apache.org/jira/browse/SUBMARINE-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-242: - Target Version: 0.4.0 > [E2E] Submarine integration test workbench's start script > - > > Key: SUBMARINE-242 > URL: https://issues.apache.org/jira/browse/SUBMARINE-242 > Project: Apache Submarine > Issue Type: Sub-task > Components: E2E Test >Reporter: luzhonghao >Priority: Major > > In the submarine-integration module, we should test whether in any directory > we can use workbench-daemon.sh to start | stop | restart workbenchServer -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-244) Submarine Releases Milestones and Planning
[ https://issues.apache.org/jira/browse/SUBMARINE-244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997070#comment-16997070 ] Wangda Tan commented on SUBMARINE-244: -- I think we have updated [https://cwiki.apache.org/confluence/display/SUBMARINE/Roadmap, |https://cwiki.apache.org/confluence/display/SUBMARINE/Roadmap] [~tangzhankun] can you make sure the contents on [https://cwiki.apache.org/confluence/display/SUBMARINE/Roadmap] is up to date and closing this ticket? > Submarine Releases Milestones and Planning > -- > > Key: SUBMARINE-244 > URL: https://issues.apache.org/jira/browse/SUBMARINE-244 > Project: Apache Submarine > Issue Type: Improvement >Reporter: Zhankun Tang >Priority: Major > > This is a long-live thread. > We use a google spreadsheet to define the releases milestones and track the > details. Everybody can see the key tasks, where we are and where we are going. > The yellow color shows upcoming tasks. The green color shows in-progress. The > grey color shows completed tasks. The purple bar marks today. > > [https://docs.google.com/spreadsheets/d/1HXA3JKMw1u3hl8nwfuT62ZtClUcFNPiraA_q5nr6v7Y/edit?usp=sharing] > Generally the contributors needs to insert row, fill in the category, JIRA#, > estimated days and start time. The sheet can calculate the end time and draw > the bar chart automatically. > Note if the start time is in the future, please choose "Upcoming" in the > status dropdown box. If it is finished or in-progress, please choose the > corresponding status. If the the bar chart is red, it means that the time or > status is not right (or maybe the formula in cell has bug). > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-242) [E2E] Submarine integration test workbench's start script
[ https://issues.apache.org/jira/browse/SUBMARINE-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-242: - Component/s: E2E Test > [E2E] Submarine integration test workbench's start script > - > > Key: SUBMARINE-242 > URL: https://issues.apache.org/jira/browse/SUBMARINE-242 > Project: Apache Submarine > Issue Type: Sub-task > Components: E2E Test >Reporter: luzhonghao >Priority: Major > > In the submarine-integration module, we should test whether in any directory > we can use workbench-daemon.sh to start | stop | restart workbenchServer -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Commented] (SUBMARINE-225) [Umbrella] Submarine Client
[ https://issues.apache.org/jira/browse/SUBMARINE-225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997074#comment-16997074 ] Wangda Tan commented on SUBMARINE-225: -- [~jiwq], [~tangzhankun] can you update what does this mean? More description will be helpful. Thanks > [Umbrella] Submarine Client > --- > > Key: SUBMARINE-225 > URL: https://issues.apache.org/jira/browse/SUBMARINE-225 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Wanqiang Ji >Priority: Major > > This is a milestone. Maybe we can used the SC in the JIRA summary field to > mark the task related to submarine client. Such as [SC] Create the > submarine-client module. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Resolved] (SUBMARINE-202) submarine core need to support MXNet
[ https://issues.apache.org/jira/browse/SUBMARINE-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved SUBMARINE-202. -- Resolution: Duplicate Closing as dup of SUBMARINE-198, [~yuan_zac] please reopen if I misunderstood. > submarine core need to support MXNet > > > Key: SUBMARINE-202 > URL: https://issues.apache.org/jira/browse/SUBMARINE-202 > Project: Apache Submarine > Issue Type: Bug >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > > Submarine core checks the machine learning framework types which must be > tensorflow or PyTorch, MXNet is supported by tony. Submarine core should > enable MXNet as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-148) [Umbrella] Submarine-sdk tracking
[ https://issues.apache.org/jira/browse/SUBMARINE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated SUBMARINE-148: - Labels: (was: submarine-external-repo) > [Umbrella] Submarine-sdk tracking > - > > Key: SUBMARINE-148 > URL: https://issues.apache.org/jira/browse/SUBMARINE-148 > Project: Apache Submarine > Issue Type: New Feature >Reporter: kevin su >Assignee: kevin su >Priority: Major > > The submarine-sdk support Python, Scala, R language for algorithm > development, The SDK is provided to help developers use submarine's internal > data caching, data exchange, and task tracking to more efficiently improve > the development and execution of machine learning tasks. > > pysubmarine *tracking* module is a python API for logging parameters, code > versions, metrics > 1. Add the sumbrine-sdk/pysubmarine module. > 2. pysubmarine supports mysql/HDFS in two ways to store metric data. > Priority to do mysql, in order to performance requires python to directly > write mysql data, do not take the submarine-sever RESTful interface. > 3. Add metric, param instance to the submarine-server database and rest > 4. Add Metric tracking ui to submarine-web > 5. Add a metric graphical display to submarine-web -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
