> On Sep 21, 2018, at 4:45 PM, Jamil Nimeh <jamil.j.ni...@oracle.com> wrote:
>
> Hi Xuelei,
>
> I started getting into making the one test per run approach - having these
> controlled from command line args in the run line gets a little weird after a
> while. We have different hello messages that are byte arrays, so you have to
> map them to strings (easy), but then some test cases (in the future, not now)
> might need SSLContexts created with different alg names, might throw
> different exceptions, we may want to take slightly different actions based on
> how the processClientHello reacts to a given message, etc. Those things are
> easier to write into the body of the test.
>
> Would you be OK with an approach where the output on stdout clearly indicates
> a PASS/FAIL for each test it performs? Then if it fails one only needs to
> look at stdout to see which test went haywire and go from there.
>
It would help simplify the failure evaluation.
But there is still a problem that when we run a lot test, the debug log may be
swallowed, for example over 5000 lines. The result is that the failure output
may not appear in the debug log.
However, it is a very minor issue. We can consider make the improvement later
when we have more cycles.
I’m fine with the current code.
Thanks,
Xuelei
> --Jamil
>
>> On 9/21/2018 4:15 PM, Xuelei Fan wrote:
>>> On 9/21/2018 4:00 PM, Jamil Nimeh wrote:
>>> Are you suggesting having multiple run lines or something like that? I
>>> think we could do that.
>> I would prefer to to the run lines.
>>
>>> I would like to have it run all cases rather than short-circuit on the
>>> first failure, as each case doesn't depend on the others.
>> It should be fine to break earlier as normally the test should be passed.
>>
>>> Let me play around with the run directives and see if we can make it work
>>> more along the lines you want.
>>>
>> Thanks!
>>
>> Xuelei
>>
>>> --Jamil
>>>
>>>
>>>> On 09/21/2018 03:55 PM, Xuelei Fan wrote:
>>>> Once there is a test case failed, it may be not straightforward to
>>>> identify which one is failed. Especially, currently, the testing blog may
>>>> be swallowed if it is too long. Would you please consider one case per
>>>> test? Or break immediately if a test case failed, instead of waiting for
>>>> all to complete?
>>>>
>>>> Thanks,
>>>> Xuelei
>>>>
>>>>> On 9/21/2018 2:35 PM, Jamil Nimeh wrote:
>>>>> Hello all,
>>>>>
>>>>> This adds a test that lets us send different kinds of client hellos to a
>>>>> JSSE server. It can be extended to handle different kinds of corner cases
>>>>> for client hello extension sets as well as fuzzing test cases in the
>>>>> future. It also provides some extra test coverage for JDK-8210334 and
>>>>> JDK-8209916.
>>>>>
>>>>> Webrev: http://cr.openjdk.java.net/~jnimeh/reviews/8210918/webrev.01/
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8210918
>>>>>
>>>>> Thanks and have a good weekend,
>>>>> --Jamil
>>>
>
