[GitHub] [submarine] yuanzac commented on a change in pull request #180: SUBMARINE-379. Delete invalid .gitmodules file
yuanzac commented on a change in pull request #180: SUBMARINE-379. Delete invalid .gitmodules file URL: https://github.com/apache/submarine/pull/180#discussion_r377483473 ## File path: pom.xml ## @@ -441,10 +441,6 @@ check - - true Review comment: Do we need to delete this line? line 428 is like false. How about we just delete com/linkedin/tony/rpc/proto/*,com/linkedin/tony/events/* This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] kevin85421 commented on a change in pull request #182: SUBMARINE-360. [WEB] Implement the frontend of sign-up page with Angular
kevin85421 commented on a change in pull request #182: SUBMARINE-360. [WEB]
Implement the frontend of sign-up page with Angular
URL: https://github.com/apache/submarine/pull/182#discussion_r377482767
##
File path:
submarine-test/e2e/src/test/java/org/apache/submarine/integration/sidebarIT.java
##
@@ -66,7 +68,13 @@ public void sidebarNavigation() throws Exception {
pollingWait(By.xpath("//span[contains(text(), \"Manager\")]"),
MAX_BROWSER_TIMEOUT_SEC).click();
pollingWait(By.xpath("//a[@href='/workbench/manager/user']"),
MAX_BROWSER_TIMEOUT_SEC).click();
SubmarineITUtils.sleep( 5000, true);
+
+// Lazy-loading
+ WebDriverWait wait = new WebDriverWait( driver, 15, 5000);
+ pollingWait(By.xpath("//a[@href='/workbench/manager/user']"),
MAX_BROWSER_TIMEOUT_SEC).click();
+
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[@class='ant-breadcrumb-link
ng-star-inserted']")));
Review comment:
Thank for your review, @liuxunorg
I have fixed the problem.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] kevin85421 commented on a change in pull request #182: SUBMARINE-360. [WEB] Implement the frontend of sign-up page with Angular
kevin85421 commented on a change in pull request #182: SUBMARINE-360. [WEB]
Implement the frontend of sign-up page with Angular
URL: https://github.com/apache/submarine/pull/182#discussion_r377482667
##
File path:
submarine-workbench/workbench-web-ng/src/app/pages/user/register/register.component.ts
##
@@ -18,14 +18,86 @@
*/
import { Component, OnInit } from '@angular/core';
+import { FormBuilder, FormControl, FormGroup, Validators } from
'@angular/forms';
+import { Router } from '@angular/router';
@Component({
selector: 'submarine-register',
templateUrl: './register.component.html',
styleUrls: ['./register.component.scss']
})
export class RegisterComponent implements OnInit {
- constructor() {}
+ validateForm: FormGroup;
+ // mock data: must be removed in the future
Review comment:
Thank for your review, @liuxunorg
I have fixed the problem.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] liuxunorg commented on a change in pull request #182: SUBMARINE-360. [WEB] Implement the frontend of sign-up page with Angular
liuxunorg commented on a change in pull request #182: SUBMARINE-360. [WEB]
Implement the frontend of sign-up page with Angular
URL: https://github.com/apache/submarine/pull/182#discussion_r377470297
##
File path:
submarine-test/e2e/src/test/java/org/apache/submarine/integration/sidebarIT.java
##
@@ -66,7 +68,13 @@ public void sidebarNavigation() throws Exception {
pollingWait(By.xpath("//span[contains(text(), \"Manager\")]"),
MAX_BROWSER_TIMEOUT_SEC).click();
pollingWait(By.xpath("//a[@href='/workbench/manager/user']"),
MAX_BROWSER_TIMEOUT_SEC).click();
SubmarineITUtils.sleep( 5000, true);
+
+// Lazy-loading
+ WebDriverWait wait = new WebDriverWait( driver, 15, 5000);
+ pollingWait(By.xpath("//a[@href='/workbench/manager/user']"),
MAX_BROWSER_TIMEOUT_SEC).click();
+
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[@class='ant-breadcrumb-link
ng-star-inserted']")));
Review comment:
Wrong code indentation
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] liuxunorg commented on a change in pull request #182: SUBMARINE-360. [WEB] Implement the frontend of sign-up page with Angular
liuxunorg commented on a change in pull request #182: SUBMARINE-360. [WEB]
Implement the frontend of sign-up page with Angular
URL: https://github.com/apache/submarine/pull/182#discussion_r377469944
##
File path:
submarine-workbench/workbench-web-ng/src/app/pages/user/register/register.component.ts
##
@@ -18,14 +18,86 @@
*/
import { Component, OnInit } from '@angular/core';
+import { FormBuilder, FormControl, FormGroup, Validators } from
'@angular/forms';
+import { Router } from '@angular/router';
@Component({
selector: 'submarine-register',
templateUrl: './register.component.html',
styleUrls: ['./register.component.scss']
})
export class RegisterComponent implements OnInit {
- constructor() {}
+ validateForm: FormGroup;
+ // mock data: must be removed in the future
Review comment:
You can remind yourself by adding `TODO(kevin85421): must be removed in the
future`, all TODO will be displayed in the `TODO view` in IDEA.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-360) [WEB] Implement the frontend of sign-up page with Angular
[ https://issues.apache.org/jira/browse/SUBMARINE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SUBMARINE-360: - Labels: pull-request-available (was: ) > [WEB] Implement the frontend of sign-up page with Angular > - > > Key: SUBMARINE-360 > URL: https://issues.apache.org/jira/browse/SUBMARINE-360 > Project: Apache Submarine > Issue Type: Sub-task > Components: Workbench Web >Reporter: Kai-Hsun Chen >Assignee: Kai-Hsun Chen >Priority: Major > Labels: pull-request-available > Fix For: 0.4.0 > > Original Estimate: 4h > Remaining Estimate: 4h > > Implement the sign-up page with Angular. The page needs to include the fields > of Name, Username, Email address, Password and Re-enter Password. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] kevin85421 opened a new pull request #182: SUBMARINE-360. [WEB] Implement the frontend of sign-up page with Angular
kevin85421 opened a new pull request #182: SUBMARINE-360. [WEB] Implement the frontend of sign-up page with Angular URL: https://github.com/apache/submarine/pull/182 ### What is this PR for? Implement the frontend of the sign-up page with Angular Validators : 1. Username: (1) empty (2) existed 2. Email: (1) empty (2) existed 3. Password: (1) empty (2) length 4. Re-enter Password: (1) empty (2) must be the same with the Password 5. Sign Up button: the button can be clicked only when the form is valid. ### What type of PR is it? [Feature] ### Todos the backend of the sign-up page ### What is the Jira issue? https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-360 ### How should this be tested? https://travis-ci.org/kevin85421/hadoop-submarine/builds/648758930?utm_medium=notification_source=github_status ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need an update? No * Are there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Work logged] (SUBMARINE-360) [WEB] Implement the frontend of sign-up page with Angular
[ https://issues.apache.org/jira/browse/SUBMARINE-360?focusedWorklogId=384995=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384995 ] ASF GitHub Bot logged work on SUBMARINE-360: Author: ASF GitHub Bot Created on: 11/Feb/20 06:33 Start Date: 11/Feb/20 06:33 Worklog Time Spent: 10m Work Description: kevin85421 commented on pull request #182: SUBMARINE-360. [WEB] Implement the frontend of sign-up page with Angular URL: https://github.com/apache/submarine/pull/182 ### What is this PR for? Implement the frontend of the sign-up page with Angular Validators : 1. Username: (1) empty (2) existed 2. Email: (1) empty (2) existed 3. Password: (1) empty (2) length 4. Re-enter Password: (1) empty (2) must be the same with the Password 5. Sign Up button: the button can be clicked only when the form is valid. ### What type of PR is it? [Feature] ### Todos the backend of the sign-up page ### What is the Jira issue? https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-360 ### How should this be tested? https://travis-ci.org/kevin85421/hadoop-submarine/builds/648758930?utm_medium=notification_source=github_status ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need an update? No * Are there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 384995) Remaining Estimate: 3h 50m (was: 4h) Time Spent: 10m > [WEB] Implement the frontend of sign-up page with Angular > - > > Key: SUBMARINE-360 > URL: https://issues.apache.org/jira/browse/SUBMARINE-360 > Project: Apache Submarine > Issue Type: Sub-task > Components: Workbench Web >Reporter: Kai-Hsun Chen >Assignee: Kai-Hsun Chen >Priority: Major > Labels: pull-request-available > Fix For: 0.4.0 > > Original Estimate: 4h > Time Spent: 10m > Remaining Estimate: 3h 50m > > Implement the sign-up page with Angular. The page needs to include the fields > of Name, Username, Email address, Password and Re-enter Password. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
Re: [ANNOUNCE] Apache Submarine 0.3.0 release!
Cool stuff! From: Wangda Tan Sent: Monday, February 10, 2020 9:07:07 PM To: us...@submarine.apache.org Cc: dev@submarine.apache.org Subject: Re: [ANNOUNCE] Apache Submarine 0.3.0 release! Awesome! Thanks everybody for helping this release! Best, Wangda On Thu, Feb 6, 2020 at 6:44 AM Xun Liu wrote: > Thanks to all contributors! > Let's continue to develop better features for submarine!! > > On Thu, Feb 6, 2020 at 7:25 PM Sunil Govindan wrote: > >> Awesome! Hearty Congratulations to all of you for the first release of >> Submarine as Apache TLP! >> Kudos to all who helped in this release! >> >> - Sunil >> >> On Thu, Feb 6, 2020 at 4:36 PM Zhankun Tang wrote: >> >> > Hi folks, >> > >> > It's a great honor for me to announce that the Apache Submarine >> community >> > has released Apache Submarine 0.3.0! >> > >> > Apache Submarine 0.3.0 is the first release after the spin-off from >> Apache >> > Hadoop. It includes 196 patches since the prior version. >> > The highlighted features are: >> > - Mini-submarine (YARN) >> > - Basic Tensorflow job submission to k8s through submarine-server >> RESTful >> > API >> > - Job submission on YARN through submarine-server RPC protocol >> > >> > Tons of thanks to our contributors and community! Let's keep fighting! >> > >> > *Apache Submarine 0.3.0 released*: >> > http://submarine.apache.org/releases/submarine-release-0.3.0.html >> > *Changelog*: https://s.apache.org/4ezw1 >> > >> > BR, >> > Zhankun >> > >> >
[jira] [Updated] (SUBMARINE-360) [WEB] Implement the frontend of sign-up page with Angular
[ https://issues.apache.org/jira/browse/SUBMARINE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai-Hsun Chen updated SUBMARINE-360: Description: Implement the sign-up page with Angular. The page needs to include the fields of Name, Username, Email address, Password and Re-enter Password. (was: Implement the sign-up page with Angular. The page needs to include the fields of Name, Username, Email address, Password and Confirm Password.) > [WEB] Implement the frontend of sign-up page with Angular > - > > Key: SUBMARINE-360 > URL: https://issues.apache.org/jira/browse/SUBMARINE-360 > Project: Apache Submarine > Issue Type: Sub-task > Components: Workbench Web >Reporter: Kai-Hsun Chen >Assignee: Kai-Hsun Chen >Priority: Major > Fix For: 0.4.0 > > Original Estimate: 4h > Remaining Estimate: 4h > > Implement the sign-up page with Angular. The page needs to include the fields > of Name, Username, Email address, Password and Re-enter Password. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-360) [WEB] Implement the frontend of sign-up page with Angular
[ https://issues.apache.org/jira/browse/SUBMARINE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai-Hsun Chen updated SUBMARINE-360: Summary: [WEB] Implement the frontend of sign-up page with Angular (was: [WEB] Implement the sign-up page with Angular) > [WEB] Implement the frontend of sign-up page with Angular > - > > Key: SUBMARINE-360 > URL: https://issues.apache.org/jira/browse/SUBMARINE-360 > Project: Apache Submarine > Issue Type: Sub-task > Components: Workbench Web >Reporter: Kai-Hsun Chen >Assignee: Kai-Hsun Chen >Priority: Major > Fix For: 0.4.0 > > Original Estimate: 4h > Remaining Estimate: 4h > > Implement the sign-up page with Angular. The page needs to include the fields > of Name, Username, Email address, Password and Confirm Password. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
Re: [ANNOUNCE] Apache Submarine 0.3.0 release!
Awesome! Thanks everybody for helping this release! Best, Wangda On Thu, Feb 6, 2020 at 6:44 AM Xun Liu wrote: > Thanks to all contributors! > Let's continue to develop better features for submarine!! > > On Thu, Feb 6, 2020 at 7:25 PM Sunil Govindan wrote: > >> Awesome! Hearty Congratulations to all of you for the first release of >> Submarine as Apache TLP! >> Kudos to all who helped in this release! >> >> - Sunil >> >> On Thu, Feb 6, 2020 at 4:36 PM Zhankun Tang wrote: >> >> > Hi folks, >> > >> > It's a great honor for me to announce that the Apache Submarine >> community >> > has released Apache Submarine 0.3.0! >> > >> > Apache Submarine 0.3.0 is the first release after the spin-off from >> Apache >> > Hadoop. It includes 196 patches since the prior version. >> > The highlighted features are: >> > - Mini-submarine (YARN) >> > - Basic Tensorflow job submission to k8s through submarine-server >> RESTful >> > API >> > - Job submission on YARN through submarine-server RPC protocol >> > >> > Tons of thanks to our contributors and community! Let's keep fighting! >> > >> > *Apache Submarine 0.3.0 released*: >> > http://submarine.apache.org/releases/submarine-release-0.3.0.html >> > *Changelog*: https://s.apache.org/4ezw1 >> > >> > BR, >> > Zhankun >> > >> >
[GitHub] [submarine] lowc1012 commented on a change in pull request #174: SUBMARINE-202. submarine core need to support MXNet
lowc1012 commented on a change in pull request #174: SUBMARINE-202. submarine
core need to support MXNet
URL: https://github.com/apache/submarine/pull/174#discussion_r377434457
##
File path:
submarine-server/server-submitter/submitter-yarn/src/main/java/org/apache/submarine/server/submitter/yarn/YarnUtils.java
##
@@ -203,4 +147,118 @@ private static Resource getResource(Parameter
parametersHolder, String option)
}
return ResourceUtils.createResourceFromString(resourceStr);
}
+
+ private static void setParametersForWorker (Configuration tonyConf,
+ ParametersHolder parameters) throws YarnException, ParseException {
+tonyConf.setStrings(
+TonyConfigurationKeys.getInstancesKey(Constants.WORKER_JOB_NAME),
+parameters.getOptionValue(CliConstants.N_WORKERS));
+
+if (parameters.getOptionValue(CliConstants.WORKER_RES) != null) {
+ Resource workerResource = getResource(parameters,
CliConstants.WORKER_RES);
+
+ tonyConf.setInt(
+ TonyConfigurationKeys.getResourceKey(Constants.WORKER_JOB_NAME,
+ Constants.VCORES),
+ workerResource.getVirtualCores());
+ tonyConf.setLong(
+ TonyConfigurationKeys.getResourceKey(Constants.WORKER_JOB_NAME,
+ Constants.MEMORY),
+ ResourceUtils.getMemorySize(workerResource));
+ tonyConf.setLong(
+ TonyConfigurationKeys.getResourceKey(Constants.WORKER_JOB_NAME,
+ Constants.GPUS),
+ ResourceUtils.getResourceValue(workerResource,
+ ResourceUtils.GPU_URI));
+}
+
+if (parameters.getOptionValue(CliConstants.WORKER_DOCKER_IMAGE) != null) {
+ tonyConf.set(
+
TonyConfigurationKeys.getDockerImageKey(Constants.WORKER_JOB_NAME),
+ parameters.getOptionValue(CliConstants.WORKER_DOCKER_IMAGE));
+ tonyConf.setBoolean(TonyConfigurationKeys.DOCKER_ENABLED, true);
+}
+
+if (parameters.getOptionValue(CliConstants.WORKER_LAUNCH_CMD) != null) {
+ tonyConf.set(
+
TonyConfigurationKeys.getExecuteCommandKey(Constants.WORKER_JOB_NAME),
+ parameters.getOptionValue(CliConstants.WORKER_LAUNCH_CMD));
+}
+ }
+
+ private static void setParametersForPS (Configuration tonyConf,
+ ParametersHolder parameters) throws YarnException, ParseException {
+String jobName = Constants.PS_JOB_NAME;
+if (parameters.getFramework() == Framework.MXNET) {
+ jobName = Constants.SERVER_JOB_NAME;
+}
+
+if (parameters.getOptionValue(CliConstants.N_PS) != null) {
+ tonyConf.setStrings(
+ TonyConfigurationKeys.getInstancesKey(jobName),
+ parameters.getOptionValue(CliConstants.N_PS));
+}
+if (parameters.getOptionValue(CliConstants.PS_RES) != null) {
+ Resource psResource = getResource(parameters, CliConstants.PS_RES);
+
+ tonyConf.setInt(
+ TonyConfigurationKeys.getResourceKey(jobName,
+ Constants.VCORES),
+ psResource.getVirtualCores());
+ tonyConf.setLong(
+ TonyConfigurationKeys.getResourceKey(jobName,
+ Constants.MEMORY),
+ ResourceUtils.getMemorySize(psResource));
+}
+
+if (parameters.getOptionValue(CliConstants.PS_LAUNCH_CMD) != null) {
+ tonyConf.set(
+ TonyConfigurationKeys.getExecuteCommandKey(jobName),
+ parameters.getOptionValue(CliConstants.PS_LAUNCH_CMD));
+}
+
+if (parameters.getOptionValue(CliConstants.PS_DOCKER_IMAGE) != null) {
+ tonyConf.set(
+ TonyConfigurationKeys.getDockerImageKey(jobName),
+ parameters.getOptionValue(CliConstants.PS_DOCKER_IMAGE));
+ tonyConf.setBoolean(TonyConfigurationKeys.DOCKER_ENABLED, true);
+}
+ }
+
+ private static void setParametersForScheduler (Configuration tonyConf,
+ ParametersHolder parameters) throws YarnException, ParseException {
+if (parameters.getOptionValue(CliConstants.N_SCHEDULERS) != null) {
+ tonyConf.setStrings(
+
TonyConfigurationKeys.getInstancesKey(Constants.SCHEDULER_JOB_NAME),
+ parameters.getOptionValue(CliConstants.N_SCHEDULERS));
+}
+
+if (parameters.getOptionValue(CliConstants.SCHEDULER_RES) != null) {
+ Resource schedulerResource = getResource(parameters,
CliConstants.SCHEDULER_RES);
+
+ tonyConf.setInt(
+
TonyConfigurationKeys.getResourceKey(Constants.SCHEDULER_JOB_NAME,
+ Constants.VCORES),
+ schedulerResource.getVirtualCores());
+ tonyConf.setLong(
+
TonyConfigurationKeys.getResourceKey(Constants.SCHEDULER_JOB_NAME,
+ Constants.MEMORY),
+ ResourceUtils.getMemorySize(schedulerResource));
+ tonyConf.setLong(
+
TonyConfigurationKeys.getResourceKey(Constants.SCHEDULER_JOB_NAME,
[GitHub] [submarine] lowc1012 commented on a change in pull request #174: SUBMARINE-202. submarine core need to support MXNet
lowc1012 commented on a change in pull request #174: SUBMARINE-202. submarine
core need to support MXNet
URL: https://github.com/apache/submarine/pull/174#discussion_r377434429
##
File path:
submarine-client/src/main/java/org/apache/submarine/client/cli/runjob/RunJobCli.java
##
@@ -219,6 +224,23 @@ private void addTensorboardOptions(Options options) {
CAN_BE_USED_WITH_TF_ONLY);
}
+ private void addSchedulerOptions(Options options) {
+options.addOption(CliConstants.N_SCHEDULERS, true,
+"Number of scheduler tasks of the job. " +
+"It should be 1 or 0, by default it's 0."+
+CAN_BE_USED_WITH_MXNET_ONLY);
+options.addOption(CliConstants.SCHEDULER_DOCKER_IMAGE, true,
+"Specify docker image for scheduler, when this is not specified, " +
+"scheduler uses --" + CliConstants.DOCKER_IMAGE +
+" as default. " + CAN_BE_USED_WITH_MXNET_ONLY);
+options.addOption(CliConstants.SCHEDULER_LAUNCH_CMD, true,
+"Commandline of scheduler, arguments will be " +
+"directly used to launch the scheduler. " +
CAN_BE_USED_WITH_MXNET_ONLY);
+options.addOption(CliConstants.SCHEDULER_RES, true,
+"Resource of each scheduler, for example " +
+"memory-mb=2048,vcores=2,yarn.io/gpu=2. " +
CAN_BE_USED_WITH_MXNET_ONLY);
Review comment:
Thanks for the review! I will remove it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-384) Add EditorConfig metafile to reduce growth of the indentation mess
[ https://issues.apache.org/jira/browse/SUBMARINE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wanqiang Ji updated SUBMARINE-384: -- Status: Patch Available (was: Open) > Add EditorConfig metafile to reduce growth of the indentation mess > -- > > Key: SUBMARINE-384 > URL: https://issues.apache.org/jira/browse/SUBMARINE-384 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > EditorConfig helps maintain consistent coding styles for multiple developers > working on the same project across various editors and IDEs. Considering the > growth of community development, we need to add the file in the project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-384) Add EditorConfig metafile to reduce growth of the indentation mess
[ https://issues.apache.org/jira/browse/SUBMARINE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wanqiang Ji updated SUBMARINE-384: -- Target Version: 0.4.0 > Add EditorConfig metafile to reduce growth of the indentation mess > -- > > Key: SUBMARINE-384 > URL: https://issues.apache.org/jira/browse/SUBMARINE-384 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > EditorConfig helps maintain consistent coding styles for multiple developers > working on the same project across various editors and IDEs. Considering the > growth of community development, we need to add the file in the project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-384) Add EditorConfig metafile to reduce growth of the indentation mess
[ https://issues.apache.org/jira/browse/SUBMARINE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SUBMARINE-384: - Labels: pull-request-available (was: ) > Add EditorConfig metafile to reduce growth of the indentation mess > -- > > Key: SUBMARINE-384 > URL: https://issues.apache.org/jira/browse/SUBMARINE-384 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Labels: pull-request-available > > EditorConfig helps maintain consistent coding styles for multiple developers > working on the same project across various editors and IDEs. Considering the > growth of community development, we need to add the file in the project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Work logged] (SUBMARINE-384) Add EditorConfig metafile to reduce growth of the indentation mess
[ https://issues.apache.org/jira/browse/SUBMARINE-384?focusedWorklogId=384527=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384527 ] ASF GitHub Bot logged work on SUBMARINE-384: Author: ASF GitHub Bot Created on: 10/Feb/20 16:05 Start Date: 10/Feb/20 16:05 Worklog Time Spent: 10m Work Description: jiwq commented on pull request #181: SUBMARINE-384. Add EditorConfig metafile to reduce growth of the indentation mess URL: https://github.com/apache/submarine/pull/181 ### What is this PR for? To reduce growth of the indentation mess ### What type of PR is it? [Improvement] ### Todos ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-384 ### How should this be tested? ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 384527) Remaining Estimate: 0h Time Spent: 10m > Add EditorConfig metafile to reduce growth of the indentation mess > -- > > Key: SUBMARINE-384 > URL: https://issues.apache.org/jira/browse/SUBMARINE-384 > Project: Apache Submarine > Issue Type: New Feature >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > EditorConfig helps maintain consistent coding styles for multiple developers > working on the same project across various editors and IDEs. Considering the > growth of community development, we need to add the file in the project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] jiwq opened a new pull request #181: SUBMARINE-384. Add EditorConfig metafile to reduce growth of the indentation mess
jiwq opened a new pull request #181: SUBMARINE-384. Add EditorConfig metafile to reduce growth of the indentation mess URL: https://github.com/apache/submarine/pull/181 ### What is this PR for? To reduce growth of the indentation mess ### What type of PR is it? [Improvement] ### Todos ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-384 ### How should this be tested? ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] pingsutw commented on issue #180: SUBMARINE-379. Delete invalid .gitmodules file
pingsutw commented on issue #180: SUBMARINE-379. Delete invalid .gitmodules file URL: https://github.com/apache/submarine/pull/180#issuecomment-584189319 +1 LGTM. Thanks @liuxunorg This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Created] (SUBMARINE-384) Add EditorConfig metafile to reduce growth of the indentation mess
Wanqiang Ji created SUBMARINE-384: - Summary: Add EditorConfig metafile to reduce growth of the indentation mess Key: SUBMARINE-384 URL: https://issues.apache.org/jira/browse/SUBMARINE-384 Project: Apache Submarine Issue Type: New Feature Reporter: Wanqiang Ji EditorConfig helps maintain consistent coding styles for multiple developers working on the same project across various editors and IDEs. Considering the growth of community development, we need to add the file in the project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] pingsutw commented on a change in pull request #174: SUBMARINE-202. submarine core need to support MXNet
pingsutw commented on a change in pull request #174: SUBMARINE-202. submarine
core need to support MXNet
URL: https://github.com/apache/submarine/pull/174#discussion_r377139787
##
File path:
submarine-server/server-submitter/submitter-yarn/src/main/java/org/apache/submarine/server/submitter/yarn/YarnUtils.java
##
@@ -203,4 +147,118 @@ private static Resource getResource(Parameter
parametersHolder, String option)
}
return ResourceUtils.createResourceFromString(resourceStr);
}
+
+ private static void setParametersForWorker (Configuration tonyConf,
+ ParametersHolder parameters) throws YarnException, ParseException {
+tonyConf.setStrings(
+TonyConfigurationKeys.getInstancesKey(Constants.WORKER_JOB_NAME),
+parameters.getOptionValue(CliConstants.N_WORKERS));
+
+if (parameters.getOptionValue(CliConstants.WORKER_RES) != null) {
+ Resource workerResource = getResource(parameters,
CliConstants.WORKER_RES);
+
+ tonyConf.setInt(
+ TonyConfigurationKeys.getResourceKey(Constants.WORKER_JOB_NAME,
+ Constants.VCORES),
+ workerResource.getVirtualCores());
+ tonyConf.setLong(
+ TonyConfigurationKeys.getResourceKey(Constants.WORKER_JOB_NAME,
+ Constants.MEMORY),
+ ResourceUtils.getMemorySize(workerResource));
+ tonyConf.setLong(
+ TonyConfigurationKeys.getResourceKey(Constants.WORKER_JOB_NAME,
+ Constants.GPUS),
+ ResourceUtils.getResourceValue(workerResource,
+ ResourceUtils.GPU_URI));
+}
+
+if (parameters.getOptionValue(CliConstants.WORKER_DOCKER_IMAGE) != null) {
+ tonyConf.set(
+
TonyConfigurationKeys.getDockerImageKey(Constants.WORKER_JOB_NAME),
+ parameters.getOptionValue(CliConstants.WORKER_DOCKER_IMAGE));
+ tonyConf.setBoolean(TonyConfigurationKeys.DOCKER_ENABLED, true);
+}
+
+if (parameters.getOptionValue(CliConstants.WORKER_LAUNCH_CMD) != null) {
+ tonyConf.set(
+
TonyConfigurationKeys.getExecuteCommandKey(Constants.WORKER_JOB_NAME),
+ parameters.getOptionValue(CliConstants.WORKER_LAUNCH_CMD));
+}
+ }
+
+ private static void setParametersForPS (Configuration tonyConf,
+ ParametersHolder parameters) throws YarnException, ParseException {
+String jobName = Constants.PS_JOB_NAME;
+if (parameters.getFramework() == Framework.MXNET) {
+ jobName = Constants.SERVER_JOB_NAME;
+}
+
+if (parameters.getOptionValue(CliConstants.N_PS) != null) {
+ tonyConf.setStrings(
+ TonyConfigurationKeys.getInstancesKey(jobName),
+ parameters.getOptionValue(CliConstants.N_PS));
+}
+if (parameters.getOptionValue(CliConstants.PS_RES) != null) {
+ Resource psResource = getResource(parameters, CliConstants.PS_RES);
+
+ tonyConf.setInt(
+ TonyConfigurationKeys.getResourceKey(jobName,
+ Constants.VCORES),
+ psResource.getVirtualCores());
+ tonyConf.setLong(
+ TonyConfigurationKeys.getResourceKey(jobName,
+ Constants.MEMORY),
+ ResourceUtils.getMemorySize(psResource));
+}
+
+if (parameters.getOptionValue(CliConstants.PS_LAUNCH_CMD) != null) {
+ tonyConf.set(
+ TonyConfigurationKeys.getExecuteCommandKey(jobName),
+ parameters.getOptionValue(CliConstants.PS_LAUNCH_CMD));
+}
+
+if (parameters.getOptionValue(CliConstants.PS_DOCKER_IMAGE) != null) {
+ tonyConf.set(
+ TonyConfigurationKeys.getDockerImageKey(jobName),
+ parameters.getOptionValue(CliConstants.PS_DOCKER_IMAGE));
+ tonyConf.setBoolean(TonyConfigurationKeys.DOCKER_ENABLED, true);
+}
+ }
+
+ private static void setParametersForScheduler (Configuration tonyConf,
+ ParametersHolder parameters) throws YarnException, ParseException {
+if (parameters.getOptionValue(CliConstants.N_SCHEDULERS) != null) {
+ tonyConf.setStrings(
+
TonyConfigurationKeys.getInstancesKey(Constants.SCHEDULER_JOB_NAME),
+ parameters.getOptionValue(CliConstants.N_SCHEDULERS));
+}
+
+if (parameters.getOptionValue(CliConstants.SCHEDULER_RES) != null) {
+ Resource schedulerResource = getResource(parameters,
CliConstants.SCHEDULER_RES);
+
+ tonyConf.setInt(
+
TonyConfigurationKeys.getResourceKey(Constants.SCHEDULER_JOB_NAME,
+ Constants.VCORES),
+ schedulerResource.getVirtualCores());
+ tonyConf.setLong(
+
TonyConfigurationKeys.getResourceKey(Constants.SCHEDULER_JOB_NAME,
+ Constants.MEMORY),
+ ResourceUtils.getMemorySize(schedulerResource));
+ tonyConf.setLong(
+
TonyConfigurationKeys.getResourceKey(Constants.SCHEDULER_JOB_NAME,
[jira] [Work logged] (SUBMARINE-379) Delete invalid .gitmodules file
[ https://issues.apache.org/jira/browse/SUBMARINE-379?focusedWorklogId=384445=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384445 ] ASF GitHub Bot logged work on SUBMARINE-379: Author: ASF GitHub Bot Created on: 10/Feb/20 14:52 Start Date: 10/Feb/20 14:52 Worklog Time Spent: 10m Work Description: liuxunorg commented on pull request #180: SUBMARINE-379. Delete invalid .gitmodules file URL: https://github.com/apache/submarine/pull/180 ### What is this PR for? Deleted the tony submodule in https://issues.apache.org/jira/browse/SUBMARINE-371, but forgot to delete the .gitmodules file and configure in pom.xml. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-379 ### How should this be tested? * https://travis-ci.org/liuxunorg/submarine/builds/648431616 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 384445) Remaining Estimate: 0h Time Spent: 10m > Delete invalid .gitmodules file > --- > > Key: SUBMARINE-379 > URL: https://issues.apache.org/jira/browse/SUBMARINE-379 > Project: Apache Submarine > Issue Type: Bug > Components: Build >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Deleted the tony submodule in > https://issues.apache.org/jira/browse/SUBMARINE-371, but forgot to delete the > submodule file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-379) Delete invalid .gitmodules file
[ https://issues.apache.org/jira/browse/SUBMARINE-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SUBMARINE-379: - Labels: pull-request-available (was: ) > Delete invalid .gitmodules file > --- > > Key: SUBMARINE-379 > URL: https://issues.apache.org/jira/browse/SUBMARINE-379 > Project: Apache Submarine > Issue Type: Bug > Components: Build >Reporter: Liu Xun >Assignee: Liu Xun >Priority: Major > Labels: pull-request-available > > Deleted the tony submodule in > https://issues.apache.org/jira/browse/SUBMARINE-371, but forgot to delete the > submodule file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] liuxunorg opened a new pull request #180: SUBMARINE-379. Delete invalid .gitmodules file
liuxunorg opened a new pull request #180: SUBMARINE-379. Delete invalid .gitmodules file URL: https://github.com/apache/submarine/pull/180 ### What is this PR for? Deleted the tony submodule in https://issues.apache.org/jira/browse/SUBMARINE-371, but forgot to delete the .gitmodules file and configure in pom.xml. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-379 ### How should this be tested? * https://travis-ci.org/liuxunorg/submarine/builds/648431616 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Resolved] (SUBMARINE-378) [E2E] E2E test with Chrome on macOS.
[ https://issues.apache.org/jira/browse/SUBMARINE-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Xun resolved SUBMARINE-378. --- Resolution: Duplicate > [E2E] E2E test with Chrome on macOS. > > > Key: SUBMARINE-378 > URL: https://issues.apache.org/jira/browse/SUBMARINE-378 > Project: Apache Submarine > Issue Type: Sub-task > Components: E2E Test >Reporter: Kai-Hsun Chen >Assignee: Liu Xun >Priority: Major > Fix For: 0.4.0 > > > We cannot run E2E unit tests with Chrome on macOS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] asfgit closed pull request #178: SUBAMRINE-382. Output submarine server run log in travis
asfgit closed pull request #178: SUBAMRINE-382. Output submarine server run log in travis URL: https://github.com/apache/submarine/pull/178 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[submarine] branch master updated: SUBAMRINE-382. Output submarine server run log in travis
This is an automated email from the ASF dual-hosted git repository. liuxun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/submarine.git The following commit(s) were added to refs/heads/master by this push: new 2234648 SUBAMRINE-382. Output submarine server run log in travis 2234648 is described below commit 2234648072aa1d55c216d7a9c5150c73b1cbac73 Author: Xun Liu AuthorDate: Mon Feb 10 09:36:50 2020 +0800 SUBAMRINE-382. Output submarine server run log in travis ### What is this PR for? Outputting the log of the submarine startup in the submarine-e2e test can help us check the reason why the submarine server cannot be started normally in travis. called function `printSubmarineLog()`. like, https://api.travis-ci.org/v3/job/648240038/log.txt You can see submarine/logs/submarine.log context in travis log. ``` = logs/submarine.log BEGIN = 2020-02-10 05:04:38,668 INFO [main]: submarine.AbstractSubmarineIT (AbstractSubmarineIT.java:printSubmarineLog(183)) - SUBMARINE_SERVER_CLASSPATH: /home/travis/build/liuxunorg/submarine/submarine-dist/target/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/bin/../lib/*::/home/travis/build/liuxunorg/submarine/submarine-dist/target/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/conf log4j:ERROR Could not find value for key log4j.appender.EventCounter log4j:ERROR Could not instantiate appender named "EventCounter". 20/02/10 05:04:37 [main]: INFO utils.SubmarineConfiguration: Load configuration from file:/home/travis/build/liuxunorg/submarine/submarine-dist/target/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/conf/submarine-site.xml [INFO ] 2020-02-10 05:04:37,556 method:org.apache.submarine.server.SubmarineServer.main(SubmarineServer.java:78) Submarine server Host: 0.0.0.0 [INFO ] 2020-02-10 05:04:37,556 method:org.apache.submarine.server.SubmarineServer.main(SubmarineServer.java:80) Submarine server Port: 8080 [INFO ] 2020-02-10 05:04:37,604 method:org.eclipse.jetty.util.log.Log.initialized(Log.java:193) Logging initialized 705ms to org.eclipse.jetty.util.log.Slf4jLog [WARN ] 2020-02-10 05:04:37,896 method:org.eclipse.jetty.server.ServerConnector.setSoLingerTime(ServerConnector.java:458) Ignoring deprecated socket close linger time [INFO ] 2020-02-10 05:04:38,060 method:org.apache.submarine.server.SubmarineServer.setupWebAppContext(SubmarineServer.java:170) workbench web war file path is ../workbench-web-ng.war. [INFO ] 2020-02-10 05:04:38,445 method:org.apache.submarine.server.workbench.websocket.NotebookServer.(NotebookServer.java:76) NotebookServer instantiated: org.apache.submarine.server.workbench.websocket.NotebookServer3943a2be = logs/submarine.log END = ``` ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-382 ### How should this be tested? * https://travis-ci.org/liuxunorg/submarine/builds/648240030 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Xun Liu Closes #178 from liuxunorg/SUBMARINE-382 and squashes the following commits: c751d7f [Xun Liu] SUBAMRINE-382. Output submarine server run log in travis --- .../org/apache/submarine/AbstractSubmarineIT.java | 71 ++ .../apache/submarine/ChromeWebDriverProvider.java | 2 +- .../submarine/integration/SubmarineLogIT.java | 51 .../org/apache/submarine/integration/loginIT.java | 3 +- 4 files changed, 125 insertions(+), 2 deletions(-) diff --git a/submarine-test/e2e/src/test/java/org/apache/submarine/AbstractSubmarineIT.java b/submarine-test/e2e/src/test/java/org/apache/submarine/AbstractSubmarineIT.java index 79d3f5a..b7d26db 100644 --- a/submarine-test/e2e/src/test/java/org/apache/submarine/AbstractSubmarineIT.java +++ b/submarine-test/e2e/src/test/java/org/apache/submarine/AbstractSubmarineIT.java @@ -21,6 +21,7 @@ package org.apache.submarine; import com.google.common.base.Function; import org.apache.commons.codec.binary.Base64; import org.apache.commons.io.FileUtils; +import org.apache.commons.lang3.StringUtils; import org.openqa.selenium.By; import org.openqa.selenium.JavascriptExecutor; import org.openqa.selenium.Keys; @@ -37,7 +38,12 @@ import org.openqa.selenium.support.ui.WebDriverWait; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import java.io.BufferedReader; import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; +import
[GitHub] [submarine] liuxunorg commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine
liuxunorg commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine URL: https://github.com/apache/submarine/pull/179#discussion_r377006408 ## File path: submarine-sdk/pysubmarine/tests/client/test_client.py ## @@ -0,0 +1,161 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from mock import patch +from unittest.mock import ANY +import pytest +from submarine.client import Client + +from submarine.exceptions import SubmarineException +from submarine.proto.SubmarineServerProtocol_pb2_grpc import SubmarineServerProtocolStub +from submarine.client.params import RoleParameter, Localization, Resource +import submarine.proto.SubmarineServerProtocol_pb2 + + +def __init__(self, channel): +def mocked_submit(params): +class mocked_response: +application_id = 'application-123456789' + +return mocked_response() + +self.SubmitJob = mocked_submit + + +# Submarine client parameters +host = 'submarine' +port = 8980 +security_disabled = True +keytab = 'keytabPath' +principal = 'testPrincipal' +distribute_keytab = True +queue = 'submarine' +client = Client(host=host, port=port, security_disabled=security_disabled, keytab=keytab, +principal=principal, distribute_keytab=distribute_keytab, queue=queue) + +# Submarine job parameters +name = 'test_rpc' +docker_image = 'apache/submarine:latest' Review comment: Because submarine has many types of mirrors, such as: `submairne database`, `submarine server`, `submarine-operator`, now all use naming rules such `as apache/submarine:server-VERSION`. https://hub.docker.com/r/apache/submarine/tags If you need to use a `submarine-server` image here, you need to change it to `apache/submarine:server-latest`. Regarding the command of mirroring, we may need to discuss it in the documentation or community meetings. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] pingsutw commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine
pingsutw commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine URL: https://github.com/apache/submarine/pull/179#discussion_r377033798 ## File path: submarine-sdk/pysubmarine/tests/client/test_client.py ## @@ -0,0 +1,161 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from mock import patch +from unittest.mock import ANY +import pytest +from submarine.client import Client + +from submarine.exceptions import SubmarineException +from submarine.proto.SubmarineServerProtocol_pb2_grpc import SubmarineServerProtocolStub +from submarine.client.params import RoleParameter, Localization, Resource +import submarine.proto.SubmarineServerProtocol_pb2 + + +def __init__(self, channel): +def mocked_submit(params): +class mocked_response: +application_id = 'application-123456789' + +return mocked_response() + +self.SubmitJob = mocked_submit + + +# Submarine client parameters +host = 'submarine' +port = 8980 +security_disabled = True +keytab = 'keytabPath' +principal = 'testPrincipal' +distribute_keytab = True +queue = 'submarine' +client = Client(host=host, port=port, security_disabled=security_disabled, keytab=keytab, +principal=principal, distribute_keytab=distribute_keytab, queue=queue) + +# Submarine job parameters +name = 'test_rpc' +docker_image = 'apache/submarine:latest' Review comment: This is just an arbitrary string of docker image. Actually, we won't run this docker images in the submarine cluster. I used mock_submit, so we won't pass these parameters to RPC server This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] pingsutw commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine
pingsutw commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine URL: https://github.com/apache/submarine/pull/179#discussion_r377033839 ## File path: submarine-sdk/pysubmarine/tests/client/test_client.py ## @@ -0,0 +1,161 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from mock import patch +from unittest.mock import ANY +import pytest +from submarine.client import Client + +from submarine.exceptions import SubmarineException +from submarine.proto.SubmarineServerProtocol_pb2_grpc import SubmarineServerProtocolStub +from submarine.client.params import RoleParameter, Localization, Resource +import submarine.proto.SubmarineServerProtocol_pb2 + + +def __init__(self, channel): +def mocked_submit(params): +class mocked_response: +application_id = 'application-123456789' + +return mocked_response() + +self.SubmitJob = mocked_submit + + +# Submarine client parameters +host = 'submarine' +port = 8980 Review comment: @liuxunorg Thanks for the review. Yes, this is default submarine rpc server port This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] liuxunorg commented on issue #174: SUBMARINE-202. submarine core need to support MXNet
liuxunorg commented on issue #174: SUBMARINE-202. submarine core need to support MXNet URL: https://github.com/apache/submarine/pull/174#issuecomment-584080614 @lowc1012 Can you rebase the development branch and commit it? Need to execute the following type command. ``` git rebase upstream/master git push -f origin SUBMARINE-202:SUBMARINE-202 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] liuxunorg commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine
liuxunorg commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine URL: https://github.com/apache/submarine/pull/179#discussion_r377006408 ## File path: submarine-sdk/pysubmarine/tests/client/test_client.py ## @@ -0,0 +1,161 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from mock import patch +from unittest.mock import ANY +import pytest +from submarine.client import Client + +from submarine.exceptions import SubmarineException +from submarine.proto.SubmarineServerProtocol_pb2_grpc import SubmarineServerProtocolStub +from submarine.client.params import RoleParameter, Localization, Resource +import submarine.proto.SubmarineServerProtocol_pb2 + + +def __init__(self, channel): +def mocked_submit(params): +class mocked_response: +application_id = 'application-123456789' + +return mocked_response() + +self.SubmitJob = mocked_submit + + +# Submarine client parameters +host = 'submarine' +port = 8980 +security_disabled = True +keytab = 'keytabPath' +principal = 'testPrincipal' +distribute_keytab = True +queue = 'submarine' +client = Client(host=host, port=port, security_disabled=security_disabled, keytab=keytab, +principal=principal, distribute_keytab=distribute_keytab, queue=queue) + +# Submarine job parameters +name = 'test_rpc' +docker_image = 'apache/submarine:latest' Review comment: Because submarine has many types of mirrors, such as: `submairne database`, `submarine server`, `submarine-operator`, now all use naming rules such `as apache/submarine:server-VERSION`. If you need to use a `submarine-server` image here, you need to change it to `apache/submarine:server-latest`. Regarding the command of mirroring, we may need to discuss it in the documentation or community meetings. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] liuxunorg commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine
liuxunorg commented on a change in pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine URL: https://github.com/apache/submarine/pull/179#discussion_r377004193 ## File path: submarine-sdk/pysubmarine/tests/client/test_client.py ## @@ -0,0 +1,161 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from mock import patch +from unittest.mock import ANY +import pytest +from submarine.client import Client + +from submarine.exceptions import SubmarineException +from submarine.proto.SubmarineServerProtocol_pb2_grpc import SubmarineServerProtocolStub +from submarine.client.params import RoleParameter, Localization, Resource +import submarine.proto.SubmarineServerProtocol_pb2 + + +def __init__(self, channel): +def mocked_submit(params): +class mocked_response: +application_id = 'application-123456789' + +return mocked_response() + +self.SubmitJob = mocked_submit + + +# Submarine client parameters +host = 'submarine' +port = 8980 Review comment: `8980`, This is submarine rpc server port? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-377) [SDK] Submit a job by pysubamrine
[ https://issues.apache.org/jira/browse/SUBMARINE-377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SUBMARINE-377: - Labels: pull-request-available (was: ) > [SDK] Submit a job by pysubamrine > -- > > Key: SUBMARINE-377 > URL: https://issues.apache.org/jira/browse/SUBMARINE-377 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: Kevin Su >Assignee: Kevin Su >Priority: Major > Labels: pull-request-available > Fix For: 0.4.0 > > > Use SDK to submit a submarine job. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Work logged] (SUBMARINE-377) [SDK] Submit a job by pysubamrine
[ https://issues.apache.org/jira/browse/SUBMARINE-377?focusedWorklogId=384333=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384333 ] ASF GitHub Bot logged work on SUBMARINE-377: Author: ASF GitHub Bot Created on: 10/Feb/20 10:38 Start Date: 10/Feb/20 10:38 Worklog Time Spent: 10m Work Description: pingsutw commented on pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine URL: https://github.com/apache/submarine/pull/179 ### What is this PR for? Use python SDK to submit a submarine job. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-377 ### How should this be tested? https://travis-ci.org/pingsutw/hadoop-submarine/builds/648336769 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 384333) Remaining Estimate: 0h Time Spent: 10m > [SDK] Submit a job by pysubamrine > -- > > Key: SUBMARINE-377 > URL: https://issues.apache.org/jira/browse/SUBMARINE-377 > Project: Apache Submarine > Issue Type: Sub-task >Reporter: Kevin Su >Assignee: Kevin Su >Priority: Major > Labels: pull-request-available > Fix For: 0.4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Use SDK to submit a submarine job. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[GitHub] [submarine] pingsutw opened a new pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine
pingsutw opened a new pull request #179: SUBMARINE-377. [SDK] Submit a job by pysubamrine URL: https://github.com/apache/submarine/pull/179 ### What is this PR for? Use python SDK to submit a submarine job. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-377 ### How should this be tested? https://travis-ci.org/pingsutw/hadoop-submarine/builds/648336769 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-383) [WEB] Implement frontend of workspace in workbench with Angular
[ https://issues.apache.org/jira/browse/SUBMARINE-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HSIN YUAN, CHEN updated SUBMARINE-383: -- Description: Implement frontend of workspace in workbench with Angular. (was: Implement workspace in workbench with Angular.) > [WEB] Implement frontend of workspace in workbench with Angular > --- > > Key: SUBMARINE-383 > URL: https://issues.apache.org/jira/browse/SUBMARINE-383 > Project: Apache Submarine > Issue Type: Sub-task > Components: Workbench Web >Reporter: HSIN YUAN, CHEN >Assignee: HSIN YUAN, CHEN >Priority: Major > Fix For: 0.4.0 > > > Implement frontend of workspace in workbench with Angular. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
[jira] [Updated] (SUBMARINE-383) [WEB] Implement frontend of workspace in workbench with Angular
[ https://issues.apache.org/jira/browse/SUBMARINE-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HSIN YUAN, CHEN updated SUBMARINE-383: -- Summary: [WEB] Implement frontend of workspace in workbench with Angular (was: [WEB] Implement workspace in workbench with Angular) > [WEB] Implement frontend of workspace in workbench with Angular > --- > > Key: SUBMARINE-383 > URL: https://issues.apache.org/jira/browse/SUBMARINE-383 > Project: Apache Submarine > Issue Type: Sub-task > Components: Workbench Web >Reporter: HSIN YUAN, CHEN >Assignee: HSIN YUAN, CHEN >Priority: Major > Fix For: 0.4.0 > > > Implement workspace in workbench with Angular. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org
