[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158146#comment-16158146 ] Tsz Wo Nicholas Sze commented on HDFS-11948: Sure, will update the patch. > Ozone: change TestRatisManager to check cluster with data > - > > Key: HDFS-11948 > URL: https://issues.apache.org/jira/browse/HDFS-11948 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Labels: ozoneMerge > Attachments: HDFS-11948-HDFS-7240.20170614.patch, > HDFS-11948-HDFS-7240.20170731.patch > > > TestRatisManager first creates multiple Ratis clusters. Then it changes the > membership and closes some clusters. However, it does not test the clusters > with data. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155995#comment-16155995 ] Anu Engineer commented on HDFS-11948: - [~szetszwo] Since the HDFS-12244 is fixed, would you like to proceed with this patch? I am not sure if this is needed with the Ratis pipeline work but would appreciate your feedback on it. > Ozone: change TestRatisManager to check cluster with data > - > > Key: HDFS-11948 > URL: https://issues.apache.org/jira/browse/HDFS-11948 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Labels: ozoneMerge > Attachments: HDFS-11948-HDFS-7240.20170614.patch, > HDFS-11948-HDFS-7240.20170731.patch > > > TestRatisManager first creates multiple Ratis clusters. Then it changes the > membership and closes some clusters. However, it does not test the clusters > with data. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109952#comment-16109952 ] Tsz Wo Nicholas Sze commented on HDFS-11948: The test still fails intermittently since the static cache provided by ContainerCache does not work in Unit tests -- it cannot be shared by multiple datanodes; filed HDFS-12244. > Ozone: change TestRatisManager to check cluster with data > - > > Key: HDFS-11948 > URL: https://issues.apache.org/jira/browse/HDFS-11948 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: HDFS-11948-HDFS-7240.20170614.patch, > HDFS-11948-HDFS-7240.20170731.patch > > > TestRatisManager first creates multiple Ratis clusters. Then it changes the > membership and closes some clusters. However, it does not test the clusters > with data. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093896#comment-16093896 ] Tsz Wo Nicholas Sze commented on HDFS-11948: No, I suspect it was IntelliJ. Will try running the tests with maven directly. BTW, we should figure out why an 1-datanode uses 400+ threads. It probably is caused by some default conf properties being set to a large value. > Ozone: change TestRatisManager to check cluster with data > - > > Key: HDFS-11948 > URL: https://issues.apache.org/jira/browse/HDFS-11948 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: HDFS-11948-HDFS-7240.20170614.patch > > > TestRatisManager first creates multiple Ratis clusters. Then it changes the > membership and closes some clusters. However, it does not test the clusters > with data. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093890#comment-16093890 ] Anu Engineer commented on HDFS-11948: - Thanks for sharing this info. Do you have any sense of where the leak is happening? > Ozone: change TestRatisManager to check cluster with data > - > > Key: HDFS-11948 > URL: https://issues.apache.org/jira/browse/HDFS-11948 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: HDFS-11948-HDFS-7240.20170614.patch > > > TestRatisManager first creates multiple Ratis clusters. Then it changes the > membership and closes some clusters. However, it does not test the clusters > with data. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093867#comment-16093867 ] Tsz Wo Nicholas Sze commented on HDFS-11948: > Also in the profiler, I see around 2025 threads being launched for this > single test. Thought you might be interested in that. [~anu], I just have finished some testing and filed HDFS-12163. It turns out that an 1-datanode MiniOzoneCluster uses 400+ threads for both Ratis and non-Ratis settings. Also, there is a thread leak problem -- some threads used in earlier tests are leaked to later tests. The thread leak explains why there are around 2025 threads showed in the profiler. TestRatisManager uses 5 datanodes. It only uses a similar number of threads as other 5-datanode cases showed in [this comment|https://issues.apache.org/jira/browse/HDFS-12163?focusedCommentId=16093848=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16093848] - testTestRatisManagerNetty | (NETTY,5) | init | 6 | | (NETTY,5) | MiniOzoneCluster | 598 | | (NETTY,5) | ready | 598 | | (NETTY,5) | create RatisCluster | 690 | | (NETTY,5) | close RatisCluster | 722 | | (NETTY,5) | update RatisCluster | 807 | | (NETTY,5) | shutdown | 438 | | (NETTY,5) | sleep | 143 | - testTestRatisManagerGrpc | (GRPC,5) | init | 6 | | (GRPC,5) | MiniOzoneCluster | 600 | | (GRPC,5) | ready | 600 | | (GRPC,5) | create RatisCluster | 621 | | (GRPC,5) | close RatisCluster | 623 | | (GRPC,5) | update RatisCluster | 633 | | (GRPC,5) | shutdown | 257 | | (GRPC,5) | sleep | 137 | > Ozone: change TestRatisManager to check cluster with data > - > > Key: HDFS-11948 > URL: https://issues.apache.org/jira/browse/HDFS-11948 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: HDFS-11948-HDFS-7240.20170614.patch > > > TestRatisManager first creates multiple Ratis clusters. Then it changes the > membership and closes some clusters. However, it does not test the clusters > with data. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088042#comment-16088042 ] Tsz Wo Nicholas Sze commented on HDFS-11948: Thanks, Anu. I will see why the tests use so many threads. > Ozone: change TestRatisManager to check cluster with data > - > > Key: HDFS-11948 > URL: https://issues.apache.org/jira/browse/HDFS-11948 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: HDFS-11948-HDFS-7240.20170614.patch > > > TestRatisManager first creates multiple Ratis clusters. Then it changes the > membership and closes some clusters. However, it does not test the clusters > with data. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[
https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087993#comment-16087993
]
Anu Engineer commented on HDFS-11948:
-
[~szetszwo] Thanks for uploading the patch. From code changes perspective this
looks good. I think the unit test failure is related to this patch.
{code}
2017-07-14 13:21:13,357 [Thread-624] INFO impl.LeaderElection
(LeaderElection.java:askForVotes(127)) - 127.0.0.1:53437: begin an election in
Term 592
2017-07-14 13:21:13,367 [nioEventLoopGroup-7-4] WARN
channel.DefaultChannelPipeline (Slf4JLogger.java:warn(151)) - An
exceptionCaught() event was fired, and it reached at the tail of the pipeline.
It usually means the last handler in the pipeline did not handle the exception.
java.lang.IllegalStateException: STATE MISMATCHED: In 127.0.0.1:53431, current
state STARTING is not one of the expected states [RUNNING]
{code}
I think on the server side we are not catching some exception and it is caught
by netty default handler on the server side, hence that error is not propagated
to the client side.
Also in the profiler, I see around 2025 threads being launched for this single
test. Thought you might be interested in that.
> Ozone: change TestRatisManager to check cluster with data
> -
>
> Key: HDFS-11948
> URL: https://issues.apache.org/jira/browse/HDFS-11948
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
> Attachments: HDFS-11948-HDFS-7240.20170614.patch
>
>
> TestRatisManager first creates multiple Ratis clusters. Then it changes the
> membership and closes some clusters. However, it does not test the clusters
> with data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data
[
https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049415#comment-16049415
]
Hadoop QA commented on HDFS-11948:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
33s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
34s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
29s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
15s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 58s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
| | hadoop.ozone.scm.TestXceiverClientManager |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| Timed out junit tests |
org.apache.hadoop.ozone.container.ozoneimpl.TestRatisManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11948 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12872994/HDFS-11948-HDFS-7240.20170614.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f4b378787790 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / 0688a1c |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/19905/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19905/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19905/console |
|
