Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
Thanks Erik. 在 2019年09月24日 23:19, Erik Joelsson 写道: Sure, will do. /Erik
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
Sure, will do. /Erik On 2019-09-23 20:57, Jia Huang wrote: Hi Erik, Thank you for your review and valuable comments. Updated: http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/ - The reference to the pkcs11 README and the reviewers had been added. Please note the user in the patch. Hope you wouldn't mind it. Thanks. Could you please sponsor it? Thanks a lot. Best regards, Jia 在 2019年09月23日 23:18, Erik Joelsson 写道: I think this type of comment fits well in the top level test doc. It just provides basic instructions for setting up these tests so that they pass without going into too much detail. Perhaps a reference to the pkcs11 README for more details would be a good idea. Looks good to me. /Erik On 2019-09-23 05:54, sha.ji...@oracle.com wrote: Hi Jia, I think this isn't a general testing problem. It may not worthy of highlighting this point in the JDK testing doc. In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README Best regards, John Jiang On 2019/9/23 18:04, Jia Huang wrote: Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8231351 Webrev: http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.00/ sun/security/pkcs11/Secmod/AddTrustedCert.java failed on Ubuntu 18.04. According to the comments in JDK-8231338, it was caused by the improper NSS libs of the system. These failures are confusing and hard to diagnose. It might be better to add some notes for the pkcs11 tests. Thanks a lot. Best regards, Jia
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
Hi John Jiang, I don't care which NSS you have skipped. I am just curious about the root cause for the failure on Ubuntu. It seems more reasonable to close JDK-8231338 as won't fix. But you just closed it as can't reproduce. Thanks. Best regards, Jie On 2019/9/24 下午4:01, sha.ji...@oracle.com wrote: Hi Jie, IIRC, this test passed on your Ubuntu 18.04 with a new built NSS 3.35 libs. So, I suspected your Linux or the system built-in NSS libs had something wrong. Now that this test passed with NSS 3.35 on others' Linux, including mine, it may not make sense that this test is skipped for NSS 3.35 on all platforms (even Linux only). Certainly, you are always welcomed to re-open JDK-8231338. A bit furthermore... Some different PKCS11 test cases have been skipped due to the bugs on different NSS releases or platforms. It would be better not to make the codes more complicated to deal with this scenario. The alternative NSS libs could be specified (via system property test.nss.lib.paths) for PKCS11 tests. It doesn't have to depend on the system built-in NSS libs. Different people can use different NSS versions based on their requirements. Best regards, John Jiang On 2019/9/24 15:03, Jie Fu wrote: I can't understand why JDK-8231338 was closed as "Cannot Reproduce" since it can always be reproduced on Ubuntu 18.04.2 LTS. Reproduce on Ubuntu 18.04 is very simple: - make test TEST="jtreg:sun/security/pkcs11/Secmod/AddTrustedCert.java" CONF=re - In the fix of JDK-8180837, the NSS-3.35 was assumed to work with AddTrustedCert.java, but it failed on Ubuntu 18.04. I wonder whether there is some other reasons which may lead to the failure. So I filed JDK-8231338 hoping to find the root cause of the failure. But I was disappointed since the author of JDK-8180837 just told me: "it's hard to say what's the problem, Linux, NSS build or others, on this test case." In fact, the root cause for the failure on Ubuntu 18.04 still remains unknown. Thanks a lot. Best regards, Jie On 2019/9/24 下午2:01, Xuelei Fan wrote: I may be a little bit hesitate to add such words, "highly recommended to use the latest NSS version ...", in the general TOP doc. There are a few issues that I wary about: It is not always expected that all PKCS 11 test should be run on latest NSS version. Otherwise, there might be compatibility issues that we did not handle properly. The JDK is expected to work with as much NSS versions as possible, not just the latest version. It is good to know which version does not really work because of a specific bug. The note should be added as close as to the place where the issue happens, for maintaining and searching. I think the best place could be the test code where the failure occurs (test/jdk/sun/security/pkcs11/Secmod/AddTrustedCert.java?). However, I'm still not sure if we really want this note. JDK-8231338 is reported with NSS 3.35. Per the comment, the test could pass with NSS 3.35 on Debian and Ubuntu, and the bug submitter could pass the test with a proper build of NSS 3.35. And then the bug was closed as "Cannot Reproduce". I think we are done with the bug. It might not be necessary to add a note any more. Just my $.02. Xuelei On 9/23/2019 9:06 PM, Jia Huang wrote: Hi John Jiang, Thank you for your review. 在 2019年09月23日 20:54, sha.ji...@oracle.com 写道: In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README I'm afraid most people wouldn't see test/jdk/sun/security/pkcs11/README at all. So it makes very little sense to add the notes in it. I still prefer doc/testing.md. A reference to test/jdk/sun/security/pkcs11/README had been added in [1]. Thanks a lot. Best regards, Jia [1] http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
Hi Jie, IIRC, this test passed on your Ubuntu 18.04 with a new built NSS 3.35 libs. So, I suspected your Linux or the system built-in NSS libs had something wrong. Now that this test passed with NSS 3.35 on others' Linux, including mine, it may not make sense that this test is skipped for NSS 3.35 on all platforms (even Linux only). Certainly, you are always welcomed to re-open JDK-8231338. A bit furthermore... Some different PKCS11 test cases have been skipped due to the bugs on different NSS releases or platforms. It would be better not to make the codes more complicated to deal with this scenario. The alternative NSS libs could be specified (via system property test.nss.lib.paths) for PKCS11 tests. It doesn't have to depend on the system built-in NSS libs. Different people can use different NSS versions based on their requirements. Best regards, John Jiang On 2019/9/24 15:03, Jie Fu wrote: I can't understand why JDK-8231338 was closed as "Cannot Reproduce" since it can always be reproduced on Ubuntu 18.04.2 LTS. Reproduce on Ubuntu 18.04 is very simple: - make test TEST="jtreg:sun/security/pkcs11/Secmod/AddTrustedCert.java" CONF=re - In the fix of JDK-8180837, the NSS-3.35 was assumed to work with AddTrustedCert.java, but it failed on Ubuntu 18.04. I wonder whether there is some other reasons which may lead to the failure. So I filed JDK-8231338 hoping to find the root cause of the failure. But I was disappointed since the author of JDK-8180837 just told me: "it's hard to say what's the problem, Linux, NSS build or others, on this test case." In fact, the root cause for the failure on Ubuntu 18.04 still remains unknown. Thanks a lot. Best regards, Jie On 2019/9/24 下午2:01, Xuelei Fan wrote: I may be a little bit hesitate to add such words, "highly recommended to use the latest NSS version ...", in the general TOP doc. There are a few issues that I wary about: It is not always expected that all PKCS 11 test should be run on latest NSS version. Otherwise, there might be compatibility issues that we did not handle properly. The JDK is expected to work with as much NSS versions as possible, not just the latest version. It is good to know which version does not really work because of a specific bug. The note should be added as close as to the place where the issue happens, for maintaining and searching. I think the best place could be the test code where the failure occurs (test/jdk/sun/security/pkcs11/Secmod/AddTrustedCert.java?). However, I'm still not sure if we really want this note. JDK-8231338 is reported with NSS 3.35. Per the comment, the test could pass with NSS 3.35 on Debian and Ubuntu, and the bug submitter could pass the test with a proper build of NSS 3.35. And then the bug was closed as "Cannot Reproduce". I think we are done with the bug. It might not be necessary to add a note any more. Just my $.02. Xuelei On 9/23/2019 9:06 PM, Jia Huang wrote: Hi John Jiang, Thank you for your review. 在 2019年09月23日 20:54, sha.ji...@oracle.com 写道: In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README I'm afraid most people wouldn't see test/jdk/sun/security/pkcs11/README at all. So it makes very little sense to add the notes in it. I still prefer doc/testing.md. A reference to test/jdk/sun/security/pkcs11/README had been added in [1]. Thanks a lot. Best regards, Jia [1] http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
I can't understand why JDK-8231338 was closed as "Cannot Reproduce" since it can always be reproduced on Ubuntu 18.04.2 LTS. Reproduce on Ubuntu 18.04 is very simple: - make test TEST="jtreg:sun/security/pkcs11/Secmod/AddTrustedCert.java" CONF=re - In the fix of JDK-8180837, the NSS-3.35 was assumed to work with AddTrustedCert.java, but it failed on Ubuntu 18.04. I wonder whether there is some other reasons which may lead to the failure. So I filed JDK-8231338 hoping to find the root cause of the failure. But I was disappointed since the author of JDK-8180837 just told me: "it's hard to say what's the problem, Linux, NSS build or others, on this test case." In fact, the root cause for the failure on Ubuntu 18.04 still remains unknown. Thanks a lot. Best regards, Jie On 2019/9/24 下午2:01, Xuelei Fan wrote: I may be a little bit hesitate to add such words, "highly recommended to use the latest NSS version ...", in the general TOP doc. There are a few issues that I wary about: It is not always expected that all PKCS 11 test should be run on latest NSS version. Otherwise, there might be compatibility issues that we did not handle properly. The JDK is expected to work with as much NSS versions as possible, not just the latest version. It is good to know which version does not really work because of a specific bug. The note should be added as close as to the place where the issue happens, for maintaining and searching. I think the best place could be the test code where the failure occurs (test/jdk/sun/security/pkcs11/Secmod/AddTrustedCert.java?). However, I'm still not sure if we really want this note. JDK-8231338 is reported with NSS 3.35. Per the comment, the test could pass with NSS 3.35 on Debian and Ubuntu, and the bug submitter could pass the test with a proper build of NSS 3.35. And then the bug was closed as "Cannot Reproduce". I think we are done with the bug. It might not be necessary to add a note any more. Just my $.02. Xuelei On 9/23/2019 9:06 PM, Jia Huang wrote: Hi John Jiang, Thank you for your review. 在 2019年09月23日 20:54, sha.ji...@oracle.com 写道: In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README I'm afraid most people wouldn't see test/jdk/sun/security/pkcs11/README at all. So it makes very little sense to add the notes in it. I still prefer doc/testing.md. A reference to test/jdk/sun/security/pkcs11/README had been added in [1]. Thanks a lot. Best regards, Jia [1] http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
I may be a little bit hesitate to add such words, "highly recommended to use the latest NSS version ...", in the general TOP doc. There are a few issues that I wary about: It is not always expected that all PKCS 11 test should be run on latest NSS version. Otherwise, there might be compatibility issues that we did not handle properly. The JDK is expected to work with as much NSS versions as possible, not just the latest version. It is good to know which version does not really work because of a specific bug. The note should be added as close as to the place where the issue happens, for maintaining and searching. I think the best place could be the test code where the failure occurs (test/jdk/sun/security/pkcs11/Secmod/AddTrustedCert.java?). However, I'm still not sure if we really want this note. JDK-8231338 is reported with NSS 3.35. Per the comment, the test could pass with NSS 3.35 on Debian and Ubuntu, and the bug submitter could pass the test with a proper build of NSS 3.35. And then the bug was closed as "Cannot Reproduce". I think we are done with the bug. It might not be necessary to add a note any more. Just my $.02. Xuelei On 9/23/2019 9:06 PM, Jia Huang wrote: Hi John Jiang, Thank you for your review. 在 2019年09月23日 20:54, sha.ji...@oracle.com 写道: In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README I'm afraid most people wouldn't see test/jdk/sun/security/pkcs11/README at all. So it makes very little sense to add the notes in it. I still prefer doc/testing.md. A reference to test/jdk/sun/security/pkcs11/README had been added in [1]. Thanks a lot. Best regards, Jia [1] http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
Hi John Jiang, Thank you for your review. 在 2019年09月23日 20:54, sha.ji...@oracle.com 写道: In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README I'm afraid most people wouldn't see test/jdk/sun/security/pkcs11/README at all. So it makes very little sense to add the notes in it. I still prefer doc/testing.md. A reference to test/jdk/sun/security/pkcs11/README had been added in [1]. Thanks a lot. Best regards, Jia [1] http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
Hi Erik, Thank you for your review and valuable comments. Updated: http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.01/ - The reference to the pkcs11 README and the reviewers had been added. Please note the user in the patch. Hope you wouldn't mind it. Thanks. Could you please sponsor it? Thanks a lot. Best regards, Jia 在 2019年09月23日 23:18, Erik Joelsson 写道: I think this type of comment fits well in the top level test doc. It just provides basic instructions for setting up these tests so that they pass without going into too much detail. Perhaps a reference to the pkcs11 README for more details would be a good idea. Looks good to me. /Erik On 2019-09-23 05:54, sha.ji...@oracle.com wrote: Hi Jia, I think this isn't a general testing problem. It may not worthy of highlighting this point in the JDK testing doc. In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README Best regards, John Jiang On 2019/9/23 18:04, Jia Huang wrote: Hi all, JBS:https://bugs.openjdk.java.net/browse/JDK-8231351 Webrev: http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.00/ sun/security/pkcs11/Secmod/AddTrustedCert.java failed on Ubuntu 18.04. According to the comments in JDK-8231338, it was caused by the improper NSS libs of the system. These failures are confusing and hard to diagnose. It might be better to add some notes for the pkcs11 tests. Thanks a lot. Best regards, Jia
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
I think this type of comment fits well in the top level test doc. It just provides basic instructions for setting up these tests so that they pass without going into too much detail. Perhaps a reference to the pkcs11 README for more details would be a good idea. Looks good to me. /Erik On 2019-09-23 05:54, sha.ji...@oracle.com wrote: Hi Jia, I think this isn't a general testing problem. It may not worthy of highlighting this point in the JDK testing doc. In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README Best regards, John Jiang On 2019/9/23 18:04, Jia Huang wrote: Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8231351 Webrev: http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.00/ sun/security/pkcs11/Secmod/AddTrustedCert.java failed on Ubuntu 18.04. According to the comments in JDK-8231338, it was caused by the improper NSS libs of the system. These failures are confusing and hard to diagnose. It might be better to add some notes for the pkcs11 tests. Thanks a lot. Best regards, Jia
Re: RFR: 8231351: Add notes for PKCS11 tests in the test doc
Hi Jia, I think this isn't a general testing problem. It may not worthy of highlighting this point in the JDK testing doc. In fact, PKCS11 tests have their own doc at: test/jdk/sun/security/pkcs11/README Best regards, John Jiang On 2019/9/23 18:04, Jia Huang wrote: Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8231351 Webrev: http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.00/ sun/security/pkcs11/Secmod/AddTrustedCert.java failed on Ubuntu 18.04. According to the comments in JDK-8231338, it was caused by the improper NSS libs of the system. These failures are confusing and hard to diagnose. It might be better to add some notes for the pkcs11 tests. Thanks a lot. Best regards, Jia
RFR: 8231351: Add notes for PKCS11 tests in the test doc
Hi all, JBS:https://bugs.openjdk.java.net/browse/JDK-8231351 Webrev: http://cr.openjdk.java.net/~jiefu/8231351-huangjia/webrev.00/ sun/security/pkcs11/Secmod/AddTrustedCert.java failed on Ubuntu 18.04. According to the comments in JDK-8231338, it was caused by the improper NSS libs of the system. These failures are confusing and hard to diagnose. It might be better to add some notes for the pkcs11 tests. Thanks a lot. Best regards, Jia
