[classlib][nio] How to deal with unstable but useful tests?

[Andrew Zhang]

Hello everybody,

I noticed such tests in DatagramChannel, which is useful, but potentially
unstable.  Consider:

public void testReadWrite_configureBlock() throws Exception {
       byte[] targetArray = new byte[2];
       // bind and connect
       this.channel1.socket().bind(localAddr2);
       this.channel1.connect(localAddr1);
       this.channel2.socket().bind(localAddr1);
       this.channel2.connect(localAddr2);
       ByteBuffer targetBuf = ByteBuffer.wrap(targetArray);

       new Thread() {
           public void run() {
               try {
                   Thread.sleep(TIME_UNIT);
                   channel1.configureBlocking(false); // Label 1
                   channel1.close(); // Label 2
               } catch (Exception e) {
                   //ignore
               }
           }
       }.start();
       try {
           this.channel1.read(targetBuf); // Label 3
           fail("should throw AsynchronousCloseException");
       } catch (AsynchronousCloseException e) {
           // ok
       }
   }
This test assumes Label 3  code should execute before Label 1 and Label 2,
which is right in most cases, because the code invokes "Thread.sleep
(TIME_UNIT)".

However, it's potentially unstable. It heavily depends on TIME_UNIT value,
test environment (Hardware, OS ...)

Indubitably, the test is very useful for testing
"AsynchronousCloseException" of DatagramChannel.read(ByteBuffer) method.

How shall we deal with such issue?  Deleting such valuable tests is not a
wise decision, while keeping them may cause build system fail.

One solution I could image is TestNG. :) i.e. Use "@dev" or something to
mark such tests, say, the tests are only for developing purpose.

Any suggestions?

Thanks!

--
Andrew Zhang
China Software Development Lab, IBM