Please, review the following test change Issue : https://bugs.openjdk.java.net/browse/JDK-8145919 Webrev: http://cr.openjdk.java.net/~jbachorik/8145919/webrev.02
The previous attempt to fix this problem was focused on the fact that the test tend to fail on ARM64 platforms. This is no more true, the failure is reproducible on various platforms if using fastdebug build.
It turns out that the test is setting up SSL in a way that only RC4 cipher suites are to be used (the test keys are generated by this algo). These cipher suites, however, has been disabled (JDK-8076221).
By all means the test should be failing since the RC4 test suites were excluded. For some reason it started failing intermittently instead. I will leave the exercise of figuring out why to someone with a thorough expertise in SSL handshake.
The fix is straightforward - create new keys (and keystore and truststore) using a supported cipher suite. I opted for the default one (TLS_DHE_DSS_WITH_AES_128_GCM_SHA256) and update the ssl properties for the test to request this cipher suite. After this change the test is passing regularly (tried running it 200 times in a loop - without any failure).
Thanks, -JB-