[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[
https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824810#comment-16824810
]
Zhankun Tang commented on SUBMARINE-54:
---
[~snemeth], I build it this way. And there will be jars of core and runtime
under "hadoop-submarine/hadoop-submarine-core/target/" and
"hadoop-submarine/hadoop-submarine-yarnservice-runtime/target/".
{code:java}
mvn clean install -Pdist,native -DskipTests -DskipShade
-Dmaven.javadoc.skip=true
mvn install -Pdist,submarine -DskipTests -DskipShade
-Dmaven.javadoc.skip=true{code}
To run the job, first put the jars under CLASSPATH, and "yarn jar
hadoop-submarine-core.jar ..."
> Add test coverage for YarnServiceJobSubmitter and make it ready for extension
> for PyTorch
> -
>
> Key: SUBMARINE-54
> URL: https://issues.apache.org/jira/browse/SUBMARINE-54
> Project: Hadoop Submarine
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch,
> SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch,
> SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch,
> SUBMARINE-54.009.patch, SUBMARINE-54.009.patch
>
>
> This crucial class has no associated test yet. We need to improve this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SUBMARINE-52) Generate Service spec + launch script for single node PyTorch learning job
[
https://issues.apache.org/jira/browse/SUBMARINE-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated SUBMARINE-52:
Description:
Similar to what we have for Tensorflow in
{{org.apache.hadoop.yarn.submarine.runtimes.yarnservice.YarnServiceJobSubmitter}},
we need a code that generates Service spec file (json) for PyTorch.
We also need to take care of the separation of CLI/YAML arguments of TF /
PyTorch.
was:Similar to what we have for Tensorflow in
{{org.apache.hadoop.yarn.submarine.runtimes.yarnservice.YarnServiceJobSubmitter}},
we need a code that generates Service spec file (json) for PyTorch.
> Generate Service spec + launch script for single node PyTorch learning job
> --
>
> Key: SUBMARINE-52
> URL: https://issues.apache.org/jira/browse/SUBMARINE-52
> Project: Hadoop Submarine
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
>
> Similar to what we have for Tensorflow in
> {{org.apache.hadoop.yarn.submarine.runtimes.yarnservice.YarnServiceJobSubmitter}},
> we need a code that generates Service spec file (json) for PyTorch.
> We also need to take care of the separation of CLI/YAML arguments of TF /
> PyTorch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[
https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824808#comment-16824808
]
Szilard Nemeth edited comment on SUBMARINE-54 at 4/24/19 4:32 AM:
--
Hi [~tangzhankun]!
You are right, the jar is not generated at all!
Turns out I was using a wrong one, which was coming from a previous build on
trunk, see:
{code:java}
??-( szilardnemeth@snemeth-MBP[06:24:36] <0> @SUBMARINE-52-2 )--(
~/development/apache/hadoop )--
└-$ ll
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
-rw-r--r-- 1 szilardnemeth staff 114343 Mar 12 19:33
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
{code}
Also, find gives me these:
{code:java}
??-( szilardnemeth@snemeth-MBP[06:25:46] <0> @SUBMARINE-52-2 )--(
~/development/apache/hadoop )--
└-$ find . | grep ".*submarine.*jar"
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT-sources.jar
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT-test-sources.jar
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
{code}
It can happen that I'm using a wrong build command, this is what I use:
{code:java}
mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true
{code}
Is there special mvn command we should use?
So I guess we should fix the jar first in order to try it out, right?
I can look into that.
was (Author: snemeth):
Hi [~tangzhankun]!
You are right, the jar is not generated at all!
Turns out I was using a wrong one, which was coming from a previous build on
trunk, see:
{code:java}
??-( szilardnemeth@snemeth-MBP[06:24:36] <0> @SUBMARINE-52-2 )--(
~/development/apache/hadoop )--
└-$ ll
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
-rw-r--r-- 1 szilardnemeth staff 114343 Mar 12 19:33
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
{code}
Also, find gives me these:
{code:java}
??-( szilardnemeth@snemeth-MBP[06:25:46] <0> @SUBMARINE-52-2 )--(
~/development/apache/hadoop )--
└-$ find . | grep ".*submarine.*jar"
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT-sources.jar
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT-test-sources.jar
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
{code}
So I guess we should fix the jar first in order to try it out, right?
I can look into that.
> Add test coverage for YarnServiceJobSubmitter and make it ready for extension
> for PyTorch
> -
>
> Key: SUBMARINE-54
> URL: https://issues.apache.org/jira/browse/SUBMARINE-54
> Project: Hadoop Submarine
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch,
> SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch,
> SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch,
> SUBMARINE-54.009.patch, SUBMARINE-54.009.patch
>
>
> This crucial class has no associated test yet. We need to improve this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[
https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824808#comment-16824808
]
Szilard Nemeth commented on SUBMARINE-54:
-
Hi [~tangzhankun]!
You are right, the jar is not generated at all!
Turns out I was using a wrong one, which was coming from a previous build on
trunk, see:
{code:java}
??-( szilardnemeth@snemeth-MBP[06:24:36] <0> @SUBMARINE-52-2 )--(
~/development/apache/hadoop )--
└-$ ll
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
-rw-r--r-- 1 szilardnemeth staff 114343 Mar 12 19:33
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
{code}
Also, find gives me these:
{code:java}
??-( szilardnemeth@snemeth-MBP[06:25:46] <0> @SUBMARINE-52-2 )--(
~/development/apache/hadoop )--
└-$ find . | grep ".*submarine.*jar"
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT-sources.jar
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT-test-sources.jar
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/target/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
{code}
So I guess we should fix the jar first in order to try it out, right?
I can look into that.
> Add test coverage for YarnServiceJobSubmitter and make it ready for extension
> for PyTorch
> -
>
> Key: SUBMARINE-54
> URL: https://issues.apache.org/jira/browse/SUBMARINE-54
> Project: Hadoop Submarine
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch,
> SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch,
> SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch,
> SUBMARINE-54.009.patch, SUBMARINE-54.009.patch
>
>
> This crucial class has no associated test yet. We need to improve this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[ https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824763#comment-16824763 ] Zhankun Tang commented on SUBMARINE-54: --- [~snemeth], I'm helping verifying the single node training job can work with this patch. But I found that the current hadoop-submarine directory cannot generate this jar. "hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar". I'm still seeking the new way to run it. [~sunilg], did you succeed in running the jars built under the new top-level hadoop-submarine directory? It doesn't build a "hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar" now. > Add test coverage for YarnServiceJobSubmitter and make it ready for extension > for PyTorch > - > > Key: SUBMARINE-54 > URL: https://issues.apache.org/jira/browse/SUBMARINE-54 > Project: Hadoop Submarine > Issue Type: Sub-task >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch, > SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch, > SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch, > SUBMARINE-54.009.patch, SUBMARINE-54.009.patch > > > This crucial class has no associated test yet. We need to improve this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SUBMARINE-58) Submarine client needs to generate fat jar
[ https://issues.apache.org/jira/browse/SUBMARINE-58?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated SUBMARINE-58: -- Status: Patch Available (was: Open) A submarine job can be submitted using the uber jar like this: /home/hadoop/java-current/bin/java -cp /home/hadoop/hadoop-current/etc/hadoop/:/home/hadoop/zq/hadoop-submarine-standalone-0.2.0-SNAPSHOT-with-all-dependencies.jar \ org.apache.hadoop.yarn.submarine.client.cli.Cli job run \ --env DOCKER_JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre \ --env DOCKER_HADOOP_HDFS_HOME=/hadoop-current --name distributed-tf-gpu-ml4 \ --env YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK=calico-network \ --env PYTHONPATH="./submarine_algorithm:$PYTHONPATH" \ --env TZ="Asia/Shanghai" \ --input_path hdfs://hadoop-cluster/tmp/cifar-10-data \ --checkpoint_path hdfs://hadoop-cluster/user/hadoop/tf-distributed-checkpoint-ml4 \ --saved_model_path hdfs://hadoop-cluster/user/hadoop/tf-distributed-saved-model-ml4 \ --num_ps 0 \ --ps_resources memory=4G,vcores=2,gpu=0 \ --ps_launch_cmd "python cifar10_main.py --data-dir=hdfs://hadoop-cluster/tmp/cifar-10-data --job-dir=%checkpoint_path% --num-gpus=0" \ --ps_docker_image *.*.*.*:5000/tensorflow1.13.1-hadoop3.1.2-cpu:1.0.0 \ --worker_docker_image *.*.*.*:5000/tensorflow1.13.1-hadoop3.1.2-gpu:1.0.0 \ --worker_resources memory=4G,vcores=2,gpu=1 --verbose \ --num_workers 1 \ --worker_launch_cmd "python cifar10_main.py --data-dir=hdfs://hadoop-cluster/tmp/cifar-10-data --job-dir=%checkpoint_path% --train-steps=500 --eval-batch-size=16 --train-batch-size=16 --num-gpus=1" \ --localization "hdfs://hadoop-cluster/user/hadoop/estimator-model/1.10/cifar10_estimator/cifar10.py:." \ --localization "hdfs://hadoop-cluster/user/hadoop/estimator-model/1.10/cifar10_estimator/cifar10_main.py:." \ --localization "hdfs://hadoop-cluster/user/hadoop/estimator-model/1.10/cifar10_estimator/cifar10_model.py:." \ --localization "hdfs://hadoop-cluster/user/hadoop/estimator-model/1.10/cifar10_estimator/cifar10_utils.py:." \ --localization "hdfs://hadoop-cluster/user/hadoop/estimator-model/1.10/cifar10_estimator/generate_cifar10_tfrecords.py:." \ --localization "hdfs://hadoop-cluster/user/hadoop/estimator-model/1.10/cifar10_estimator/model_base.py:." \ --localization "hdfs://hadoop-cluster/user/hadoop/estimator-model/1.10/cifar10_estimator:./submarine_algorithm" \ --wait_job_finish \ --tensorboard \ --tensorboard_docker_image *.*.*.*:5000/tensorflow1.13.1-hadoop3.1.2-cpu:1.0.0 \ --keytab /home/hadoop/hadoop.keytab \ --principal hadoop/admin \ --distribute_keytab > Submarine client needs to generate fat jar > -- > > Key: SUBMARINE-58 > URL: https://issues.apache.org/jira/browse/SUBMARINE-58 > Project: Hadoop Submarine > Issue Type: Improvement >Reporter: Xun Liu >Assignee: Zac Zhou >Priority: Major > Attachments: SUBMARINE-58.001.patch > > > When submitting a job using the submarine client alone, Will encounter > package dependencies and cause execution to fail, If the submarine client can > provide a fat jar, Many development and usage issues will be avoided. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SUBMARINE-58) Submarine client needs to generate fat jar
[ https://issues.apache.org/jira/browse/SUBMARINE-58?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated SUBMARINE-58: -- Attachment: SUBMARINE-58.001.patch > Submarine client needs to generate fat jar > -- > > Key: SUBMARINE-58 > URL: https://issues.apache.org/jira/browse/SUBMARINE-58 > Project: Hadoop Submarine > Issue Type: Improvement >Reporter: Xun Liu >Assignee: Zac Zhou >Priority: Major > Attachments: SUBMARINE-58.001.patch > > > When submitting a job using the submarine client alone, Will encounter > package dependencies and cause execution to fail, If the submarine client can > provide a fat jar, Many development and usage issues will be avoided. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[
https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824087#comment-16824087
]
Szilard Nemeth commented on SUBMARINE-54:
-
Hi [~tangzhankun], [~sunilg]!
I played around with the single node training job on my cluster:
1. The command to start the Submarine job was:
{code:java}
/opt/hadoop/bin/yarn jar /home/systest/hadoop-yarn-submarine-3.3.0-SNAPSHOT.jar
job run \
--name tf-job-001 --verbose --docker_image hadoopsubmarine/tf-1.8.0-gpu:0.0.1 \
--input_path hdfs://default/dataset/cifar-10-data \
--env DOCKER_JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre \
--env DOCKER_HADOOP_HDFS_HOME=/hadoop-3.1.0 \
--num_workers 1 --worker_resources memory=5G,vcores=2 \
--worker_launch_cmd "cd /test/models/tutorials/image/cifar10_estimator &&
python cifar10_main.py --data-dir=%input_path% --job-dir=%checkpoint_path%
--train-steps=1 --eval-batch-size=16 --train-batch-size=16 --num-gpus=2
--sync" \
--tensorboard --tensorboard_docker_image wtan/tf-1.8.0-cpu:0.0.3
{code}
2. I have the following error logs from the serviceam.log file:
{code:java}
2019-04-23 05:14:42,679 [Component dispatcher] INFO
instance.ComponentInstance - [COMPINSTANCE master-0 :
container_e17_1556021556746_0001_01_02] failed without retry,
exitStatus=ContainerStatus: [ContainerId:
container_e17_1556021556746_0001_01_02, ExecutionType: GUARANTEED, State:
COMPLETE, Capability: , Diagnostics: [2019-04-23
05:14:42.182]Exception from container-launch.
Container id: container_e17_1556021556746_0001_01_02
Exit code: 1
[2019-04-23 05:14:42.229]Container exited with a non-zero exit code 1. Error
file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr.txt :
./run-PRIMARY_WORKER.sh: line 9: /hadoop-3.1.0/bin/hadoop: No such file or
directory
./run-PRIMARY_WORKER.sh: line 16: cd:
/test/models/tutorials/image/cifar10_estimator: No such file or directory
[2019-04-23 05:14:42.229]Container exited with a non-zero exit code 1. Error
file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr.txt :
./run-PRIMARY_WORKER.sh: line 9: /hadoop-3.1.0/bin/hadoop: No such file or
directory
./run-PRIMARY_WORKER.sh: line 16: cd:
/test/models/tutorials/image/cifar10_estimator: No such file or directory
, ExitStatus: 1, IP: null, Host: null, ExposedPorts: , ContainerSubState: DONE]
{code}
Could this be because of missing Docker configuration? There's no clear
indication to that.
Also I don't get what is this value for?
{code:java}
DOCKER_HADOOP_HDFS_HOME=/hadoop-3.1.0
{code}
By the way, it's not too easy to configure YARN to run submarine on a cluster
everytime we make a change in the code.
Could it be a more long-term solution to develop an integration test
environment to cover basic cases like single node / distributed TF training
jobs are starting up without issues?
I would happily work on this, but not right now of course.
Also, testing on cluster would take more time, as I need to configure docker on
the hosts.
I guess we can't proceed with SUBMARINE-52 until I have a working end-to-end
submarine job.
[~sunilg]: What's your opinion?
Thanks!
> Add test coverage for YarnServiceJobSubmitter and make it ready for extension
> for PyTorch
> -
>
> Key: SUBMARINE-54
> URL: https://issues.apache.org/jira/browse/SUBMARINE-54
> Project: Hadoop Submarine
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch,
> SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch,
> SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch,
> SUBMARINE-54.009.patch, SUBMARINE-54.009.patch
>
>
> This crucial class has no associated test yet. We need to improve this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[
https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823977#comment-16823977
]
Hadoop QA commented on SUBMARINE-54:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
2s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 21 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 15s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
39s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 16s{color} | {color:orange} hadoop-submarine: The patch generated 27 new +
27 unchanged - 12 fixed = 54 total (was 39) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-submarine-core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
0s{color} | {color:green} hadoop-submarine-yarnservice-runtime in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-SUBMARINE-Build/62/artifact/out/Dockerfile
|
| JIRA Issue | SUBMARINE-54 |
| JIRA Patch URL |
[jira] [Updated] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[ https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated SUBMARINE-54: Attachment: SUBMARINE-54.009.patch > Add test coverage for YarnServiceJobSubmitter and make it ready for extension > for PyTorch > - > > Key: SUBMARINE-54 > URL: https://issues.apache.org/jira/browse/SUBMARINE-54 > Project: Hadoop Submarine > Issue Type: Sub-task >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch, > SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch, > SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch, > SUBMARINE-54.009.patch, SUBMARINE-54.009.patch > > > This crucial class has no associated test yet. We need to improve this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[
https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823918#comment-16823918
]
Hadoop QA commented on SUBMARINE-54:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 8m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 21 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
16s{color} | {color:red} hadoop-submarine-yarnservice-runtime in trunk failed.
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
15s{color} | {color:red} hadoop-submarine-yarnservice-runtime in trunk failed.
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
38s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} hadoop-submarine-yarnservice-runtime in trunk failed.
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
11s{color} | {color:red} hadoop-submarine-yarnservice-runtime in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange} hadoop-submarine: The patch generated 28 new +
27 unchanged - 13 fixed = 55 total (was 40) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
12s{color} | {color:red} hadoop-submarine-yarnservice-runtime in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 29s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
13s{color} | {color:red} hadoop-submarine-yarnservice-runtime in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} hadoop-submarine-yarnservice-runtime in the patch
failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hadoop-submarine-core in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 13s{color}
| {color:red} hadoop-submarine-yarnservice-runtime in the patch failed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce
[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[ https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823847#comment-16823847 ] Szilard Nemeth commented on SUBMARINE-54: - Patch009 is rebased onto latest trunk! Hopefully, the build issues will be gone as well! > Add test coverage for YarnServiceJobSubmitter and make it ready for extension > for PyTorch > - > > Key: SUBMARINE-54 > URL: https://issues.apache.org/jira/browse/SUBMARINE-54 > Project: Hadoop Submarine > Issue Type: Sub-task >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch, > SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch, > SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch, > SUBMARINE-54.009.patch > > > This crucial class has no associated test yet. We need to improve this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[ https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated SUBMARINE-54: Attachment: SUBMARINE-54.009.patch > Add test coverage for YarnServiceJobSubmitter and make it ready for extension > for PyTorch > - > > Key: SUBMARINE-54 > URL: https://issues.apache.org/jira/browse/SUBMARINE-54 > Project: Hadoop Submarine > Issue Type: Sub-task >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch, > SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch, > SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch, > SUBMARINE-54.009.patch > > > This crucial class has no associated test yet. We need to improve this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUBMARINE-40) Add TonY runtime to Submarine
[ https://issues.apache.org/jira/browse/SUBMARINE-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823802#comment-16823802 ] Hudson commented on SUBMARINE-40: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16451 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16451/]) SUBMARINE-40. Add TonY runtime to Submarine. Contributed by Keqiu Hu. (ztang: rev e79a9c12c14dcecc674048ed182e74d8690e663a) * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/buider/JobStatusBuilder.java * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/buider/package-info.java * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/site/resources/css/site.css * (add) hadoop-submarine/hadoop-submarine-tony-runtime/README.md * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyJobSubmitter.java * (edit) hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/RunJobParameters.java * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/package-info.java * (edit) hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceJobSubmitter.java * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/test/java/TestTonyUtils.java * (edit) hadoop-submarine/hadoop-submarine-core/src/site/markdown/QuickStart.md * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyJobMonitor.java * (edit) hadoop-submarine/pom.xml * (edit) hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/CliConstants.java * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/site/markdown/QuickStart.md * (add) hadoop-submarine/hadoop-submarine-tony-runtime/pom.xml * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyUtils.java * (edit) hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/RunJobCli.java * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/site/site.xml * (add) hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyRuntimeFactory.java > Add TonY runtime to Submarine > - > > Key: SUBMARINE-40 > URL: https://issues.apache.org/jira/browse/SUBMARINE-40 > Project: Hadoop Submarine > Issue Type: New Feature >Affects Versions: 0.2.0 >Reporter: Keqiu Hu >Assignee: Keqiu Hu >Priority: Major > Labels: submarine, tony > Fix For: 0.2.0 > > Attachments: SUBMARINE-40.003.patch, SUBMARINE-40.004.patch, > SUBMARINE-40.005.patch, SUBMARINE-40.006.patch, SUBMARINE-40.007.patch, > SUBMARINE-40.008.patch, SUBMARINE-40.009.patch, SUBMARINE-40.010.patch, > SUBMARINE-40.patch > > Original Estimate: 504h > Remaining Estimate: 504h > > Currently Submarine depends on YARN native service, we need another runtime > to support Hadoop versions w/o native service. Plan is to add a TonY runtime > into Submarine eco system. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUBMARINE-40) Add TonY runtime to Submarine
[ https://issues.apache.org/jira/browse/SUBMARINE-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823791#comment-16823791 ] Zhankun Tang commented on SUBMARINE-40: --- [~oliverhuh...@gmail.com], LGTM. +1. Committed to trunk. Thanks! > Add TonY runtime to Submarine > - > > Key: SUBMARINE-40 > URL: https://issues.apache.org/jira/browse/SUBMARINE-40 > Project: Hadoop Submarine > Issue Type: New Feature >Affects Versions: 0.2.0 >Reporter: Keqiu Hu >Assignee: Keqiu Hu >Priority: Major > Labels: submarine, tony > Fix For: 0.2.0 > > Attachments: SUBMARINE-40.003.patch, SUBMARINE-40.004.patch, > SUBMARINE-40.005.patch, SUBMARINE-40.006.patch, SUBMARINE-40.007.patch, > SUBMARINE-40.008.patch, SUBMARINE-40.009.patch, SUBMARINE-40.010.patch, > SUBMARINE-40.patch > > Original Estimate: 504h > Remaining Estimate: 504h > > Currently Submarine depends on YARN native service, we need another runtime > to support Hadoop versions w/o native service. Plan is to add a TonY runtime > into Submarine eco system. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SUBMARINE-58) Submarine client needs to generate fat jar
[ https://issues.apache.org/jira/browse/SUBMARINE-58?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou reassigned SUBMARINE-58: - Assignee: Zac Zhou > Submarine client needs to generate fat jar > -- > > Key: SUBMARINE-58 > URL: https://issues.apache.org/jira/browse/SUBMARINE-58 > Project: Hadoop Submarine > Issue Type: Improvement >Reporter: Xun Liu >Assignee: Zac Zhou >Priority: Major > > When submitting a job using the submarine client alone, Will encounter > package dependencies and cause execution to fail, If the submarine client can > provide a fat jar, Many development and usage issues will be avoided. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUBMARINE-54) Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch
[ https://issues.apache.org/jira/browse/SUBMARINE-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823763#comment-16823763 ] Szilard Nemeth commented on SUBMARINE-54: - Hi [~sunilg], [~tangzhankun]! There we have this pom resolution failure again :( Last week, it just disappeared after a while. Can you help me out with this, please? Thanks! > Add test coverage for YarnServiceJobSubmitter and make it ready for extension > for PyTorch > - > > Key: SUBMARINE-54 > URL: https://issues.apache.org/jira/browse/SUBMARINE-54 > Project: Hadoop Submarine > Issue Type: Sub-task >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > Attachments: SUBMARINE-54.001.patch, SUBMARINE-54.002.patch, > SUBMARINE-54.003.patch, SUBMARINE-54.004.patch, SUBMARINE-54.005.patch, > SUBMARINE-54.006.patch, SUBMARINE-54.007.patch, SUBMARINE-54.008.patch > > > This crucial class has no associated test yet. We need to improve this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUBMARINE-40) Add TonY runtime to Submarine
[
https://issues.apache.org/jira/browse/SUBMARINE-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823764#comment-16823764
]
Hadoop QA commented on SUBMARINE-40:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 56s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
32s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange} hadoop-submarine: The patch generated 2 new + 25
unchanged - 0 fixed = 27 total (was 25) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 49s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
51s{color} | {color:green} hadoop-submarine in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-submarine-core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hadoop-submarine-yarnservice-runtime in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-submarine-tony-runtime in the patch passed.
{color} |
| {color:green}+1{color} | {color:green}
[jira] [Updated] (SUBMARINE-40) Add TonY runtime to Submarine
[ https://issues.apache.org/jira/browse/SUBMARINE-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keqiu Hu updated SUBMARINE-40: -- Attachment: SUBMARINE-40.010.patch > Add TonY runtime to Submarine > - > > Key: SUBMARINE-40 > URL: https://issues.apache.org/jira/browse/SUBMARINE-40 > Project: Hadoop Submarine > Issue Type: New Feature >Affects Versions: 0.2.0 >Reporter: Keqiu Hu >Assignee: Keqiu Hu >Priority: Major > Labels: submarine, tony > Fix For: 0.2.0 > > Attachments: SUBMARINE-40.003.patch, SUBMARINE-40.004.patch, > SUBMARINE-40.005.patch, SUBMARINE-40.006.patch, SUBMARINE-40.007.patch, > SUBMARINE-40.008.patch, SUBMARINE-40.009.patch, SUBMARINE-40.010.patch, > SUBMARINE-40.patch > > Original Estimate: 504h > Remaining Estimate: 504h > > Currently Submarine depends on YARN native service, we need another runtime > to support Hadoop versions w/o native service. Plan is to add a TonY runtime > into Submarine eco system. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUBMARINE-40) Add TonY runtime to Submarine
[
https://issues.apache.org/jira/browse/SUBMARINE-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823712#comment-16823712
]
Hadoop QA commented on SUBMARINE-40:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 7m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 19s{color} | {color:orange} The patch fails to run checkstyle in
hadoop-submarine {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
17s{color} | {color:red} hadoop-submarine-yarnservice-runtime in trunk failed.
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
21s{color} | {color:red} hadoop-submarine in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
16s{color} | {color:red} hadoop-submarine-yarnservice-runtime in trunk failed.
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
35s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} hadoop-submarine-yarnservice-runtime in trunk failed.
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
30s{color} | {color:red} hadoop-submarine in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} hadoop-submarine-tony-runtime in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
29s{color} | {color:red} hadoop-submarine in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s{color}
| {color:red} hadoop-submarine in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange} hadoop-submarine: The patch generated 16 new +
20 unchanged - 0 fixed = 36 total (was 20) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
33s{color} | {color:red} hadoop-submarine in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
19s{color} | {color:red} hadoop-submarine-tony-runtime in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 20s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
48s{color} | {color:red} hadoop-submarine in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color}
