[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199472#comment-16199472 ] Jian He commented on YARN-7202: --- bq. After discussion with Jian, we agree to revert back to Patch 03, Ok, I missed this comment. In that case, my above comment doesn't apply. I think Billie's comment make sense, with that, the whole bunch of if/else condition are not needed. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch, > YARN-7202.yarn-native-services.005.patch, > YARN-7202.yarn-native-services.006.patch, > YARN-7202.yarn-native-services.007.patch, > YARN-7202.yarn-native-services.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199443#comment-16199443 ] Jian He commented on YARN-7202: --- bq. This is the reason that MiniHDFSCluster and MiniYARNCluster are both required dependencies for the test to run. MiniHDFSCluster is not even used in code at all. Can you remove the TestApiService from this patch and all the changes to followup patch ? because that's what needed by followup patches. Let's focus on one thing at a time. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch, > YARN-7202.yarn-native-services.005.patch, > YARN-7202.yarn-native-services.006.patch, > YARN-7202.yarn-native-services.007.patch, > YARN-7202.yarn-native-services.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199438#comment-16199438 ] Billie Rinaldi commented on YARN-7202: -- Instead of the patch 08 changes to updateService, I suggest changing stopService to return BAD_REQUEST instead of NOT_FOUND. This has almost the same effect on return codes as the patch 08 changes, but it does not remap OK response codes to NO_CONTENT. I would also suggest that when stopService succeeds, we should return OK instead of NO_CONTENT. We can delay addressing the issue of multiple types of operations being performed in PUT until YARN-7217. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch, > YARN-7202.yarn-native-services.005.patch, > YARN-7202.yarn-native-services.006.patch, > YARN-7202.yarn-native-services.007.patch, > YARN-7202.yarn-native-services.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198976#comment-16198976
]
Eric Yang commented on YARN-7202:
-
[~jianhe] wrote:
{code}
Confused. I'm seeing the opposite.
Before the patch, it directly returns the status code based on whatever failure
conditions it is inside the method which is accurate.
After the patch, it only returns INTERNAL_SERVER_ERROR or BAD_REQUEST. e.g. the
NOT_FOUND status code for stopService method is instead converted to the
BAD_REQUEST status code. Why is this converstion needed?
{code}
In ServiceClient, YarnException is used for invalid request, i.e. Service Name
not found, Application already exist, lost contact to AM, or Service definition
is not found in HDFS. There are many other exceptions overloaded using
YarnException and all of them are general error handling. We end up with web
response that are inconsistent between Web response code and the root cause,
such as 404 NOT_FOUND, and application already exist. This is the reason the
response code is generalized into BAD_REQUEST to avoid confusion.
> End-to-end UT for api-server
>
>
> Key: YARN-7202
> URL: https://issues.apache.org/jira/browse/YARN-7202
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Eric Yang
> Attachments: YARN-7202.yarn-native-services.001.patch,
> YARN-7202.yarn-native-services.002.patch,
> YARN-7202.yarn-native-services.003.patch,
> YARN-7202.yarn-native-services.004.patch,
> YARN-7202.yarn-native-services.005.patch,
> YARN-7202.yarn-native-services.006.patch,
> YARN-7202.yarn-native-services.007.patch,
> YARN-7202.yarn-native-services.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198881#comment-16198881
]
Eric Yang commented on YARN-7202:
-
[~jianhe] I took another look at the code from patch 08, and I have answers for
this question:
{quote}
confused, isn't this code not part of this patch ? I did clean before build.
Can you try compile only this patch with the dependency removed?
{quote}
I refactored TestYarnNativeServices to move the configuration initializer code
to ServiceTestUtils java class and move ServiceTestUtils java class to
org/apache/hadoop/yarn/service/client/ServiceTestUtils. This was done to
enhance the ability to reuse the utility class for YARN service setup for test
cases. TestApiService extends ServiceTestUtils for mini yarn cluster setup.
This is the reason that MiniHDFSCluster and MiniYARNCluster are both required
dependencies for the test to run. The scope of the dependencies is set to
"test". This is the reason that you were able to compile without the
dependencies expressed, they are used during test phase. Hence, both exception
throwing, and dependency expression are non-issues. Could you take another
look at patch 08? I deleted patch 09 and 10 because the decision that we made
are not correct in yesterday's meeting.
> End-to-end UT for api-server
>
>
> Key: YARN-7202
> URL: https://issues.apache.org/jira/browse/YARN-7202
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Eric Yang
> Attachments: YARN-7202.yarn-native-services.001.patch,
> YARN-7202.yarn-native-services.002.patch,
> YARN-7202.yarn-native-services.003.patch,
> YARN-7202.yarn-native-services.004.patch,
> YARN-7202.yarn-native-services.005.patch,
> YARN-7202.yarn-native-services.006.patch,
> YARN-7202.yarn-native-services.007.patch,
> YARN-7202.yarn-native-services.008.patch,
> YARN-7202.yarn-native-services.009.patch,
> YARN-7202.yarn-native-services.010.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198013#comment-16198013
]
Hadoop QA commented on YARN-7202:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 18m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
34s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
13s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
7m 52s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
19s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
11s{color} | {color:red} hadoop-yarn-services-api in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
12s{color} | {color:red} hadoop-yarn-services-api in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 12s{color}
| {color:red} hadoop-yarn-services-api in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 8s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api:
The patch generated 2 new + 2 unchanged - 1 fixed = 4 total (was 3) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
12s{color} | {color:red} hadoop-yarn-services-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 42s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
11s{color} | {color:red} hadoop-yarn-services-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 12s{color}
| {color:red} hadoop-yarn-services-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7202 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12891174/YARN-7202.yarn-native-services.010.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux 181f123b50eb 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 4993b8a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198012#comment-16198012 ] Eric Yang commented on YARN-7202: - [~jianhe] PUT method throws YarnException for both Service Not Found, and general Yarn Framework errors in ServiceClient. When YarnException is surfaced to client, it will indicate Service Not Found, in fact, it may have failed due to backend errors. This is the reason that later patch was modified to show only BAD_REQUEST or INTERNAL_SERVER_ERROR only to make avoid misdirect user. I revert it back to your original intent. We can probably throw a file not found exception in ServiceClient to make YarnException more precise of the intent. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch, > YARN-7202.yarn-native-services.005.patch, > YARN-7202.yarn-native-services.006.patch, > YARN-7202.yarn-native-services.007.patch, > YARN-7202.yarn-native-services.008.patch, > YARN-7202.yarn-native-services.009.patch, > YARN-7202.yarn-native-services.010.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197999#comment-16197999
]
Hadoop QA commented on YARN-7202:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 37s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 9s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api:
The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 33s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
37s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 20s{color}
| {color:red} hadoop-yarn-services-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api
|
| | Write to static field
org.apache.hadoop.yarn.service.webapp.ApiServer.SERVICE_CLIENT from instance
method
org.apache.hadoop.yarn.service.webapp.ApiServer.setServiceClient(ServiceClient)
At ApiServer.java:from instance method
org.apache.hadoop.yarn.service.webapp.ApiServer.setServiceClient(ServiceClient)
At ApiServer.java:[line 88] |
| Failed junit tests | hadoop.yarn.service.TestApiServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7202 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12891173/YARN-7202.yarn-native-services.009.patch
|
| Optional Tests | asflicense compile javac javadoc
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197860#comment-16197860
]
Jian He commented on YARN-7202:
---
confused, isn't this code not part of this patch ? I did clean before build.
Can you try compile only this patch with the dependency removed?
{code}
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/ServiceClientTest.java:[29,45]
cannot find symbol
[ERROR] symbol: class YarnSolrClientTest
{code}
bq. The new code enables the logic to be handled more accurately and provide
better status report to down stream program weather the failure is caused by a
bad request, record not found, or undetermined server errors. The previous
implementation does not allow certain error to be surfaced correctly.
Confused. I'm seeing the opposite.
Before the patch, it directly returns the status code based on whatever failure
conditions it is inside the method which is accurate.
After the patch, it only returns INTERNAL_SERVER_ERROR or BAD_REQUEST. e.g.
the NOT_FOUND status code for stopService method is instead converted to the
BAD_REQUEST status code. Why is this converstion needed?
> End-to-end UT for api-server
>
>
> Key: YARN-7202
> URL: https://issues.apache.org/jira/browse/YARN-7202
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Eric Yang
> Attachments: YARN-7202.yarn-native-services.001.patch,
> YARN-7202.yarn-native-services.002.patch,
> YARN-7202.yarn-native-services.003.patch,
> YARN-7202.yarn-native-services.004.patch,
> YARN-7202.yarn-native-services.005.patch,
> YARN-7202.yarn-native-services.006.patch,
> YARN-7202.yarn-native-services.007.patch,
> YARN-7202.yarn-native-services.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197669#comment-16197669
]
Eric Yang commented on YARN-7202:
-
[~jianhe] Thank you for the review. The new code enables the logic to be
handled more accurately and provide better status report to down stream program
weather the failure is caused by a bad request, record not found, or
undetermined server errors. The previous implementation does not allow certain
error to be surfaced correctly. This could lead to unclear or incorrect error
messages. TestApiServer is unit test to make sure the REST API provide correct
response codes in all conditions. It is mostly tested with negative test case
where mocked ServiceClient will throw all kind of errors to ApiServer. This
ensure all corner cases are handled correctly from REST API point of view.
TestApiService is testing positive end to end integration test. Both type of
tests should exist. There are javadoc description for TestApiServer and
TestApiService.
When the test jar file dependencies are removed. The tests fail on my side
with class not found exception.
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
(default-testCompile) on project hadoop-yarn-services-api: Compilation failure:
Compilation failure:
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/ServiceClientTest.java:[29,45]
cannot find symbol
[ERROR] symbol: class YarnSolrClientTest
[ERROR] location: package org.apache.hadoop.yarn.service.client
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiService.java:[26,45]
cannot find symbol
[ERROR] symbol: class ServiceTestUtils
[ERROR] location: package org.apache.hadoop.yarn.service.client
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiService.java:[40,52]
cannot find symbol
[ERROR] symbol: class ServiceTestUtils
[ERROR] location: package org.apache.hadoop.yarn.service.client
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiService.java:[40,1]
static import only from classes and interfaces
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiService.java:[46,37]
cannot find symbol
[ERROR] symbol: class ServiceTestUtils
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiServer.java:[29,45]
cannot find symbol
[ERROR] symbol: class YarnSolrClientTest
[ERROR] location: package org.apache.hadoop.yarn.service.client
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiServer.java:[44,27]
cannot find symbol
[ERROR] symbol: class MatcherAssert
[ERROR] location: package org.hamcrest
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiServer.java:[44,1]
static import only from classes and interfaces
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/ServiceClientTest.java:[57,15]
cannot find symbol
[ERROR] symbol: class YarnSolrClientTest
[ERROR] location: class org.apache.hadoop.yarn.service.ServiceClientTest
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiService.java:[51,19]
cannot find symbol
[ERROR] symbol: variable NUM_NMS
[ERROR] location: class org.apache.hadoop.yarn.service.TestApiService
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiService.java:[57,5]
cannot find symbol
[ERROR] symbol: method shutdown()
[ERROR] location: class org.apache.hadoop.yarn.service.TestApiService
[ERROR]
/home/eyang/test/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api/src/test/java/org/apache/hadoop/yarn/service/TestApiService.java:[125,24]
cannot find symbol
[ERROR] symbol: method createClient()
[ERROR] location: class org.apache.hadoop.yarn.service.TestApiService
[ERROR]
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197421#comment-16197421
]
Jian He commented on YARN-7202:
---
bq. If Service config changed, save configuration.
bq. If Service should increase/decrease containers, perform flex operation.
bq. If Service should be in STARTED state, then start containers.
bq. If Service should be in STOPPED state, then stop containers.
I meant given that start and stop are anyways the last operations. Are there
any difference if I simply do this:
{code}
if (updateServiceData.getState() != null
&& updateServiceData.getState() == ServiceState.STOPPED) {
return stopService(appName, false);
}
// If a START is requested
if (updateServiceData.getState() != null
&& updateServiceData.getState() == ServiceState.STARTED) {
return startService(appName);
}
{code}
vs this,
{code}
if (updateServiceData.getState() != null
&& updateServiceData.getState() == ServiceState.STOPPED) {
Response r = stopService(appName, false);
if (r.getStatus() ==
Response.status(Status.BAD_REQUEST).build()
.getStatus() ||
r.getStatus() ==
Response.status(Status.NOT_FOUND).build()
.getStatus()) {
hasBadRequest = true;
} else if (r.getStatus() ==
Response.status(Status.INTERNAL_SERVER_ERROR).build()
.getStatus()) {
hasErrors = true;
}
}
// If a START is requested
if (updateServiceData.getState() != null
&& updateServiceData.getState() == ServiceState.STARTED) {
Response r = startService(appName);
if (r.getStatus() ==
Response.status(Status.BAD_REQUEST).build()
.getStatus()) {
hasBadRequest = true;
} else if (r.getStatus() ==
Response.status(Status.INTERNAL_SERVER_ERROR).build()
.getStatus()) {
hasErrors = true;
}
}
{code}
And that said, I'm going remove the flex operation in this updateService method.
bq. The dependency are required to import MiniCluster. For some reason, I think
the code refactoring has some issues.
I complied after removing these dependencies without any issues. Please double
check.
{code}
org.apache.hadoop
hadoop-common
test-jar
org.apache.hadoop
hadoop-hdfs
test-jar
org.apache.hadoop
hadoop-yarn-server-tests
test-jar
{code}
Can you remove the TestApiService in this jira and move it to whichever
followup jira it fits? because, I don't see any fundamental difference of
TestApiServer and TestApiService here, given that both use mocked client. (btw,
the test class name is confusing.)
> End-to-end UT for api-server
>
>
> Key: YARN-7202
> URL: https://issues.apache.org/jira/browse/YARN-7202
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Eric Yang
> Attachments: YARN-7202.yarn-native-services.001.patch,
> YARN-7202.yarn-native-services.002.patch,
> YARN-7202.yarn-native-services.003.patch,
> YARN-7202.yarn-native-services.004.patch,
> YARN-7202.yarn-native-services.005.patch,
> YARN-7202.yarn-native-services.006.patch,
> YARN-7202.yarn-native-services.007.patch,
> YARN-7202.yarn-native-services.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196544#comment-16196544
]
Eric Yang commented on YARN-7202:
-
[~jianhe] ServiceClient code is partially completed in YARN-7202.
"updateService" code is not quite testable because service operation and config
change sequence are inverse of what they are supposed to be.
Prior to this patch, the PUT method is:
If Service object should be in STARTED state, then return.
If Service object should be in STOPPED state, then return.
If Service object should increase or decrease containers, perform the operation.
The problem arises, if someone retrieve Service status object via GET
/ws/v1/services/{service_name}, and change number of container for the service.
It will not perform the container changes. The code got to this point because
there is no separation between configuration, and status. If status object is
reused to perform configuration changes, then the code doesn't perform what it
was designed to do. For clarity, flex number of container operation should be
considered as configuration changes because even existing containers may need
to have information of newly spawning containers. Hence, it is best that we
treat flexing container numbers as configuration change, and restart all
containers with in the service.
Giving the above reasons, the proper design should be:
If Service config changed, save configuration.
If Service should increase/decrease containers, perform flex operation.
If Service should be in STARTED state, then start containers.
If Service should be in STOPPED state, then stop containers.
At this point, ServiceClient is not written to support the proper sequence of
operation to perform. I only made changes to ApiServer updateService method to
contain the scope of required changes. Therefore, I use mock object here to
allow TestApiService to pass without making a failure test case to block other
development. ServiceClient will not pass the test code at this point, but
ApiServer code can pass the test with improved logic. This is the reason that
mock object is used here. In YARN-7215, after spec and status are separated
two calls. I added additional logic in ServiceClient code to have ability to
make service config changes. In YARN-7215, you can see that TestApiService is
switched to the real ServiceClient, and both positive and negative test cases
passed.
The dependency are required to import MiniCluster. For some reason, I think
the code refactoring has some issues. Hadoop-yarn-service-api is not a sibling
project to hadoop-yarn-services, or hadoop-yarn-services-core. Therefore, it
doesn't inherit MiniCluster imports. This is the reason that test jar files
are imported again.
I think YARN-7202, YARN-7216, and YARN-7215 should be one transaction. Giving
degree of complexity in the design bug, it is good that we discuss the reasons
for these changes. The issues are raised out of order due to late discovery of
the bugs in the inherited code. Hence, the course correction is only made in
later JIRAs.
Hope this clarifies that the end goal of this JIRA is met, but the proper code
will only happen after YARN-7216 and YARN-7215 are committed.
> End-to-end UT for api-server
>
>
> Key: YARN-7202
> URL: https://issues.apache.org/jira/browse/YARN-7202
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Eric Yang
> Attachments: YARN-7202.yarn-native-services.001.patch,
> YARN-7202.yarn-native-services.002.patch,
> YARN-7202.yarn-native-services.003.patch,
> YARN-7202.yarn-native-services.004.patch,
> YARN-7202.yarn-native-services.005.patch,
> YARN-7202.yarn-native-services.006.patch,
> YARN-7202.yarn-native-services.007.patch,
> YARN-7202.yarn-native-services.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196338#comment-16196338
]
Jian He commented on YARN-7202:
---
Thanks for the patch, few comments:
- STARTED and STOPPED are mutually exclusive conditions, I don't think we need
to try to serialize these two conditions. The additional if/else are
unnecessary. We can simply do return from stopService or startService
{code}
if (updateServiceData.getState() != null
&& updateServiceData.getState() == ServiceState.STOPPED) {
Response r = stopService(appName, false);
if (r.getStatus() ==
Response.status(Status.BAD_REQUEST).build()
.getStatus() ||
r.getStatus() ==
Response.status(Status.NOT_FOUND).build()
.getStatus()) {
hasBadRequest = true;
} else if (r.getStatus() ==
Response.status(Status.INTERNAL_SERVER_ERROR).build()
.getStatus()) {
hasErrors = true;
}
}
// If a START is requested
if (updateServiceData.getState() != null
&& updateServiceData.getState() == ServiceState.STARTED) {
Response r = startService(appName);
if (r.getStatus() ==
Response.status(Status.BAD_REQUEST).build()
.getStatus()) {
hasBadRequest = true;
} else if (r.getStatus() ==
Response.status(Status.INTERNAL_SERVER_ERROR).build()
.getStatus()) {
hasErrors = true;
}
}
{code}
- A bit confused with the TestApiService - It tries to use
TestYarnNativeService's MiniYarnCluster framework which is supposed for fully
end-to-end integration test. But the test overrides the ServiceClient with its
own mocked client, then what's the point of using the MiniYarnCluster framework
?
- Are these dependencies required?
{code}
org.apache.hadoop
hadoop-common
test-jar
org.apache.hadoop
hadoop-hdfs
test-jar
org.apache.hadoop
hadoop-yarn-server-tests
test-jar
{code}
> End-to-end UT for api-server
>
>
> Key: YARN-7202
> URL: https://issues.apache.org/jira/browse/YARN-7202
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Eric Yang
> Attachments: YARN-7202.yarn-native-services.001.patch,
> YARN-7202.yarn-native-services.002.patch,
> YARN-7202.yarn-native-services.003.patch,
> YARN-7202.yarn-native-services.004.patch,
> YARN-7202.yarn-native-services.005.patch,
> YARN-7202.yarn-native-services.006.patch,
> YARN-7202.yarn-native-services.007.patch,
> YARN-7202.yarn-native-services.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194957#comment-16194957
]
Hadoop QA commented on YARN-7202:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 6m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
16s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
3s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 52s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
12s{color} | {color:green} root: The patch generated 0 new + 7 unchanged - 3
fixed = 7 total (was 10) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 19s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
51s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}110m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192146#comment-16192146
]
Hadoop QA commented on YARN-7202:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
33s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
28s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
20s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 39s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
9s{color} | {color:green} root: The patch generated 0 new + 7 unchanged - 3
fixed = 7 total (was 10) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 19s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
52s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}122m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16191835#comment-16191835
]
Hadoop QA commented on YARN-7202:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
38s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 8s{color} | {color:orange} root: The patch generated 1 new + 7 unchanged -
3 fixed = 8 total (was 10) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 14s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
52s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}107m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190594#comment-16190594
]
Hadoop QA commented on YARN-7202:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
6s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
50s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 53s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 33s{color} | {color:orange} root: The patch generated 24 new + 10 unchanged
- 1 fixed = 34 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
58s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}105m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190567#comment-16190567
]
Hadoop QA commented on YARN-7202:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
3s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
49s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
58s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
21s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 7s{color} | {color:orange} root: The patch generated 24 new + 10 unchanged
- 1 fixed = 34 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 30s{color}
| {color:red} hadoop-yarn-services-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
52s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}115m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188942#comment-16188942 ] Eric Yang commented on YARN-7202: - TestYarnNativeServices and TestApiServer combination provides end-to-end unit tests coverages. After reviewing HADOOP-9122, it doesn't appear that powermock can help in Hadoop unit test. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184996#comment-16184996 ] Jian He commented on YARN-7202: --- oh, ok, didn't know we can still re-open it. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184914#comment-16184914 ] Eric Yang commented on YARN-7202: - [~jianhe] I am able to re-open HADOOP-9122. I can move powermock related changes to that JIRA for community to review. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184881#comment-16184881 ] Jian He commented on YARN-7202: --- Once the issue is closed, it cannot be reopened. If the issue is only resolved, it can be re-opened. Maybe you can make a comment there ? and open a new jira in 'hadoop-common' to express the intent? > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184870#comment-16184870 ] Eric Yang commented on YARN-7202: - [~jianhe] Sure, I will add to javadoc to indicate the purpose of test case. I tried to reopen HADOOP-7537, but JIRA won't let me. Can you reopen the issue? People didn't find a working combination of Mockito, Powermock and commons-io in the past. The outstanding concerns had been addressed in open source in the past couple years. Hence, I like to proposed to move forward with the tested version of Powermockito. As we can see from the tests, powermock did not introduce unwanted behavior to Hadoop tests. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184648#comment-16184648 ] Jian He commented on YARN-7202: --- could you also add the comments in your first comment of this jira here to each test which explained what each test is doing ? > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184640#comment-16184640 ] Jian He commented on YARN-7202: --- bq. If both mocked version of ServiceClient, and ApiServer are in agreement of the behavior. The ambiguity problem must be originated from ServiceClient. This verification confirms that unit test for ApiServer is done correctly, and problem resides in ServiceClient. It won't be easy to maintain the behavior to be consistent across ServiceClientForTest and ServiceClient, because most of the time people will not look at the UTs unless the UT failed. Any way, I agree we can do the more end-to-end test separately. Reg the UT patch, for the changes in updateService, I saw you tried to Serialize all the operations and that introduced quite a few if/else conditions. To keep it simple, for start and stop, we can just return directly？ it is expected to only do start or stop. And the " // flex a single component app" scenario code will be removed later. The TestApiServer introduced powermock. Till now, Hadoop had not been using powermock, and is using Mockito. Some historical discussion HADOOP-7537, HADOOP-9122 happened for this. To keep it consistent with rest code base and less controversial, how about changing to use Mockito ? Or any part in the UT must rely on powermock? > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182886#comment-16182886 ] Eric Yang commented on YARN-7202: - [~jianhe] In the example that you provided, it is showing what ApiServer would behave and what is expected from ServiceClient. If ServiceClient throws YarnException for NOT_FOUND, and ApiServer does not capturing YarnException to treat it as NOT_FOUND, then we look at the reason for the discrepancy. We know ServiceClient will throw YarnException for various errors, including NOT_FOUND. However, ServiceClient does not provide more details for ApiServer to identify the difference between NOT_FOUND, and INTERNAL_SERVER_ERROR by looking at YarnException. Hence, the mocked version of ServiceClient is verifying how ApiServer would behave under all conditions that ApiServer covered. The ambiguity does not come from ApiServer. If both mocked version of ServiceClient, and ApiServer are behaving correctly. The ambiguity problem must be originated from ServiceClient. This verification confirms that unit test for ApiServer is done correctly, and problem resides in ServiceClient. If we peel off layer and layer of ambiguity, then we have a good suite of unit test cases that can be repeated and reproduced. ServiceClient code is inherited from Apache Slider. There are too many integration classes with external dependencies. It won't be easy to write unit test cases in one task. I would proposed to write unit test cases in separate JIRAs as we continue to trim down the framework to make up for the technical debts from Apache Slider. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181778#comment-16181778 ] Jian He commented on YARN-7202: --- I'm just worried some ended up testing the ServiceClientForTest instead of real code. e.g. testBadGetService tests if a service exists or not, and check if the return code is NOT_FOUND. The test passes because in ServiceClientForTest it implements its own logic to throw IllegalArgumentException if service-name doesn't equal to the hardcode 'jenkins'. But this is not the case for real code. In real code, it doesn't throw IllegalArgumentException if service is not found, it throws YarnException. and the return code won't be NOT_FOUND. So the test becomes moot in this scenario. May be mock particular method (like the rpc methods) instead of overriding the entire serviceClient ? > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181706#comment-16181706 ] Eric Yang commented on YARN-7202: - [~jianhe] The test case are designed to be unit tests for ApiServer without over reaching. It also covers the input validation in ServiceClient. It is possible to write additional tests to focus on ServiceClient correctness which can increase test coverage. However, I think it is easy to mistaken unit test verse integration test. Without good coverage at interface level, integration test may not help to identify the root problems. I can't say I am familiar with ServiceClient code to write unit tests for it. I can write unit tests for new code that I am about to add to ServiceClient for storing Service config. However, it would be helpful for original author who wrote ServiceClient to add test code. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181680#comment-16181680 ] Jian He commented on YARN-7202: --- [~eyang], thanks for working on the patch. I looked at the patch, looks like the test only check whether the return code is success or failure. I think it can be more end-to-end, i.e. a fully functional rest API test, e.g. from the submission till the persistency of service config file. TestServiceCLI does that from CLI level. Also, the mockServiceClient pretty much hides all the real code, making it work with real ServiceClient will be testing the real code. Btw, there are a bunch RM web service related tests in the codebase. e.g TestRMWebServicesAppsModification, TestRMWebServices etc. Those tests actually test it from end-to-end without mock code > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181567#comment-16181567
]
Hadoop QA commented on YARN-7202:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
49s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
15s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 13s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api:
The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 30s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7202 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889144/YARN-7202.yarn-native-services.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux d1e21510392d 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 3f7a50d |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17645/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api
U:
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181251#comment-16181251
]
Hadoop QA commented on YARN-7202:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
48s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 28s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 9s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api:
The patch generated 3 new + 2 unchanged - 1 fixed = 5 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 17s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7202 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889109/YARN-7202.yarn-native-services.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux 291d82f3734c 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 3f7a50d |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17642/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-api.txt
|
| Test Results |
