Re: RFR: JDK-8156658 Common way to run jtreg tests

[Dmitry Samersoff]

Staffan,

Looks good for me.

-Dmitry


On 2016-05-10 17:22, Staffan Larsen wrote:
> Updated webrev with those changes to the hotspot
> makefile: http://cr.openjdk.java.net/~sla/8156658/webrev.04/
> 
> Thanks,
> /Staffan
> 
>> On 10 maj 2016, at 15:50, Dmitry Samersoff
>> > wrote:
>>
>> Staffan,
>>
>>> Maybe. I didn’t want to change existing behavior in this patch. Which
>>> one do you prefer?
>>
>> The code from jdk/test/Makefile (jaxp uses the same code).
>>
>> -Dmitry
>>
>>
>> On 2016-05-10 16:06, Staffan Larsen wrote:
>>>
 On 10 maj 2016, at 14:46, Dmitry Samersoff
 >
 wrote:

 Staffan,

 Long awaited changes, thank you for doing it.

 Is it possible to use the same code for hotspot/test/Makefile:128 and
 jdk/test/Makefile:84
>>>
>>> Maybe. I didn’t want to change existing behavior in this patch. Which
>>> one do you prefer?
>>>

 84   ifdef ALT_OUTPUTDIR
 85 ABS_OUTPUTDIR = $(shell $(CD) $(ALT_OUTPUTDIR) && $(PWD))
 86   else
 87 ABS_OUTPUTDIR = $(shell $(CD) $(TEST_ROOT)/.. && $(PWD))
 88   endif
 89
 90   ABS_PLATFORM_BUILD_ROOT = $(ABS_OUTPUTDIR)
 91   ABS_TEST_OUTPUT_DIR :=
 $(ABS_PLATFORM_BUILD_ROOT)/testoutput/$(UNIQUE_DIR)

 -Dmitry

 On 2016-05-10 15:21, Staffan Larsen wrote:
> For test automation purposes we want a way to run jtreg tests for
> the different components (jdk, langtools, jaxp, hotspot, nashorn)
> in a unified way from the command line. We also want each component
> to be able to define how jtreg is invoked (problemlists,
> concurrency, agentvm, ...).  There is already some support for
> running jtreg tests from the /test folder, but it is not
> general enough since it isn't possible to pick individual tests or
> specific test groups. It is also not possible to direct the output
> to a common location.
>
> The proposed solution is to be able to run jtreg tests using make
> from within the /test folder like this:
>
> $ make JT_JAVA= JT_HOME= PRODUCT_HOME= product path> \
>   TESTDIRS=../
> TEST_SELECTION= \
>   TEST_OUTPUT_DIR= jtreg_tests
>
> This will run the specified tests and collect output in
> TEST_OUTPUT_DIR/jtreg. EXTRA_JTREG_OPTIONS is also supported to add
> additional options to the jtreg command line.
>
> No existing behavior should be changed.
>
> webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/
> 
> bug: https://bugs.openjdk.java.net/browse/JDK-8156658
> 
>
> Thanks,
> /Staffan
>


 -- 
 Dmitry Samersoff
 Oracle Java development team, Saint Petersburg, Russia
 * I would love to change the world, but they won't give me the sources.
>>>
>>
>>
>> -- 
>> Dmitry Samersoff
>> Oracle Java development team, Saint Petersburg, Russia
>> * I would love to change the world, but they won't give me the sources.
> 


-- 
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* I would love to change the world, but they won't give me the sources.

Re: RFR: JDK-8156658 Common way to run jtreg tests

[Erik Joelsson]

There are several issues that I would like to bring up after this, but I 
really don't want to open that can of worms since the goal here is not 
to do a full cleanup. I'm ok with this.


/Erik

On 2016-05-10 16:22, Staffan Larsen wrote:

Updated webrev with those changes to the hotspot makefile: 
http://cr.openjdk.java.net/~sla/8156658/webrev.04/ 


Thanks,
/Staffan


On 10 maj 2016, at 15:50, Dmitry Samersoff  wrote:

Staffan,


Maybe. I didn’t want to change existing behavior in this patch. Which
one do you prefer?

The code from jdk/test/Makefile (jaxp uses the same code).

-Dmitry


On 2016-05-10 16:06, Staffan Larsen wrote:

On 10 maj 2016, at 14:46, Dmitry Samersoff  wrote:

Staffan,

Long awaited changes, thank you for doing it.

Is it possible to use the same code for hotspot/test/Makefile:128 and
jdk/test/Makefile:84

Maybe. I didn’t want to change existing behavior in this patch. Which one do 
you prefer?


84   ifdef ALT_OUTPUTDIR
85 ABS_OUTPUTDIR = $(shell $(CD) $(ALT_OUTPUTDIR) && $(PWD))
86   else
87 ABS_OUTPUTDIR = $(shell $(CD) $(TEST_ROOT)/.. && $(PWD))
88   endif
89
90   ABS_PLATFORM_BUILD_ROOT = $(ABS_OUTPUTDIR)
91   ABS_TEST_OUTPUT_DIR :=
$(ABS_PLATFORM_BUILD_ROOT)/testoutput/$(UNIQUE_DIR)

-Dmitry

On 2016-05-10 15:21, Staffan Larsen wrote:

For test automation purposes we want a way to run jtreg tests for the different 
components (jdk, langtools, jaxp, hotspot, nashorn) in a unified way from the command 
line. We also want each component to be able to define how jtreg is invoked 
(problemlists, concurrency, agentvm, ...).  There is already some support for running 
jtreg tests from the /test folder, but it is not general enough since it 
isn't possible to pick individual tests or specific test groups. It is also not 
possible to direct the output to a common location.

The proposed solution is to be able to run jtreg tests using make from within the 
/test folder like this:

$ make JT_JAVA= JT_HOME= PRODUCT_HOME= \
   TESTDIRS=../ TEST_SELECTION= \
   TEST_OUTPUT_DIR= jtreg_tests

This will run the specified tests and collect output in TEST_OUTPUT_DIR/jtreg. 
EXTRA_JTREG_OPTIONS is also supported to add additional options to the jtreg 
command line.

No existing behavior should be changed.

webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 

bug: https://bugs.openjdk.java.net/browse/JDK-8156658 


Thanks,
/Staffan



--
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* I would love to change the world, but they won't give me the sources.


--
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* I would love to change the world, but they won't give me the sources.

Re: RFR: JDK-8156658 Common way to run jtreg tests

[Staffan Larsen]

Updated webrev with those changes to the hotspot makefile: 
http://cr.openjdk.java.net/~sla/8156658/webrev.04/ 


Thanks,
/Staffan

> On 10 maj 2016, at 15:50, Dmitry Samersoff  
> wrote:
> 
> Staffan,
> 
>> Maybe. I didn’t want to change existing behavior in this patch. Which
>> one do you prefer?
> 
> The code from jdk/test/Makefile (jaxp uses the same code).
> 
> -Dmitry
> 
> 
> On 2016-05-10 16:06, Staffan Larsen wrote:
>> 
>>> On 10 maj 2016, at 14:46, Dmitry Samersoff  
>>> wrote:
>>> 
>>> Staffan,
>>> 
>>> Long awaited changes, thank you for doing it.
>>> 
>>> Is it possible to use the same code for hotspot/test/Makefile:128 and
>>> jdk/test/Makefile:84
>> 
>> Maybe. I didn’t want to change existing behavior in this patch. Which one do 
>> you prefer? 
>> 
>>> 
>>> 84   ifdef ALT_OUTPUTDIR
>>> 85 ABS_OUTPUTDIR = $(shell $(CD) $(ALT_OUTPUTDIR) && $(PWD))
>>> 86   else
>>> 87 ABS_OUTPUTDIR = $(shell $(CD) $(TEST_ROOT)/.. && $(PWD))
>>> 88   endif
>>> 89
>>> 90   ABS_PLATFORM_BUILD_ROOT = $(ABS_OUTPUTDIR)
>>> 91   ABS_TEST_OUTPUT_DIR :=
>>> $(ABS_PLATFORM_BUILD_ROOT)/testoutput/$(UNIQUE_DIR)
>>> 
>>> -Dmitry
>>> 
>>> On 2016-05-10 15:21, Staffan Larsen wrote:
 For test automation purposes we want a way to run jtreg tests for the 
 different components (jdk, langtools, jaxp, hotspot, nashorn) in a unified 
 way from the command line. We also want each component to be able to 
 define how jtreg is invoked (problemlists, concurrency, agentvm, ...).  
 There is already some support for running jtreg tests from the /test 
 folder, but it is not general enough since it isn't possible to pick 
 individual tests or specific test groups. It is also not possible to 
 direct the output to a common location. 
 
 The proposed solution is to be able to run jtreg tests using make from 
 within the /test folder like this: 
 
 $ make JT_JAVA= JT_HOME= PRODUCT_HOME=>>> path> \
   TESTDIRS=../ TEST_SELECTION=>>> test or jtreg group> \
   TEST_OUTPUT_DIR= jtreg_tests 
 
 This will run the specified tests and collect output in 
 TEST_OUTPUT_DIR/jtreg. EXTRA_JTREG_OPTIONS is also supported to add 
 additional options to the jtreg command line. 
 
 No existing behavior should be changed.
 
 webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 
 
 bug: https://bugs.openjdk.java.net/browse/JDK-8156658 
 
 
 Thanks,
 /Staffan
 
>>> 
>>> 
>>> -- 
>>> Dmitry Samersoff
>>> Oracle Java development team, Saint Petersburg, Russia
>>> * I would love to change the world, but they won't give me the sources.
>> 
> 
> 
> -- 
> Dmitry Samersoff
> Oracle Java development team, Saint Petersburg, Russia
> * I would love to change the world, but they won't give me the sources.

Re: RFR: JDK-8156658 Common way to run jtreg tests

[Dmitry Samersoff]

Staffan,

> Maybe. I didn’t want to change existing behavior in this patch. Which
> one do you prefer?

The code from jdk/test/Makefile (jaxp uses the same code).

-Dmitry


On 2016-05-10 16:06, Staffan Larsen wrote:
> 
>> On 10 maj 2016, at 14:46, Dmitry Samersoff  
>> wrote:
>>
>> Staffan,
>>
>> Long awaited changes, thank you for doing it.
>>
>> Is it possible to use the same code for hotspot/test/Makefile:128 and
>> jdk/test/Makefile:84
> 
> Maybe. I didn’t want to change existing behavior in this patch. Which one do 
> you prefer? 
> 
>>
>>  84   ifdef ALT_OUTPUTDIR
>>  85 ABS_OUTPUTDIR = $(shell $(CD) $(ALT_OUTPUTDIR) && $(PWD))
>>  86   else
>>  87 ABS_OUTPUTDIR = $(shell $(CD) $(TEST_ROOT)/.. && $(PWD))
>>  88   endif
>>  89
>>  90   ABS_PLATFORM_BUILD_ROOT = $(ABS_OUTPUTDIR)
>>  91   ABS_TEST_OUTPUT_DIR :=
>> $(ABS_PLATFORM_BUILD_ROOT)/testoutput/$(UNIQUE_DIR)
>>
>> -Dmitry
>>
>> On 2016-05-10 15:21, Staffan Larsen wrote:
>>> For test automation purposes we want a way to run jtreg tests for the 
>>> different components (jdk, langtools, jaxp, hotspot, nashorn) in a unified 
>>> way from the command line. We also want each component to be able to define 
>>> how jtreg is invoked (problemlists, concurrency, agentvm, ...).  There is 
>>> already some support for running jtreg tests from the /test folder, 
>>> but it is not general enough since it isn't possible to pick individual 
>>> tests or specific test groups. It is also not possible to direct the output 
>>> to a common location. 
>>>
>>> The proposed solution is to be able to run jtreg tests using make from 
>>> within the /test folder like this: 
>>>
>>>  $ make JT_JAVA= JT_HOME= PRODUCT_HOME=>> path> \
>>>TESTDIRS=../ TEST_SELECTION=>> test or jtreg group> \
>>>TEST_OUTPUT_DIR= jtreg_tests 
>>>
>>> This will run the specified tests and collect output in 
>>> TEST_OUTPUT_DIR/jtreg. EXTRA_JTREG_OPTIONS is also supported to add 
>>> additional options to the jtreg command line. 
>>>
>>> No existing behavior should be changed.
>>>
>>> webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 
>>> 
>>> bug: https://bugs.openjdk.java.net/browse/JDK-8156658 
>>> 
>>>
>>> Thanks,
>>> /Staffan
>>>
>>
>>
>> -- 
>> Dmitry Samersoff
>> Oracle Java development team, Saint Petersburg, Russia
>> * I would love to change the world, but they won't give me the sources.
> 


-- 
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* I would love to change the world, but they won't give me the sources.

Re: RFR: JDK-8156658 Common way to run jtreg tests

[Staffan Larsen]

> On 10 maj 2016, at 14:46, Dmitry Samersoff  
> wrote:
> 
> Staffan,
> 
> Long awaited changes, thank you for doing it.
> 
> Is it possible to use the same code for hotspot/test/Makefile:128 and
> jdk/test/Makefile:84

Maybe. I didn’t want to change existing behavior in this patch. Which one do 
you prefer? 

> 
>  84   ifdef ALT_OUTPUTDIR
>  85 ABS_OUTPUTDIR = $(shell $(CD) $(ALT_OUTPUTDIR) && $(PWD))
>  86   else
>  87 ABS_OUTPUTDIR = $(shell $(CD) $(TEST_ROOT)/.. && $(PWD))
>  88   endif
>  89
>  90   ABS_PLATFORM_BUILD_ROOT = $(ABS_OUTPUTDIR)
>  91   ABS_TEST_OUTPUT_DIR :=
> $(ABS_PLATFORM_BUILD_ROOT)/testoutput/$(UNIQUE_DIR)
> 
> -Dmitry
> 
> On 2016-05-10 15:21, Staffan Larsen wrote:
>> For test automation purposes we want a way to run jtreg tests for the 
>> different components (jdk, langtools, jaxp, hotspot, nashorn) in a unified 
>> way from the command line. We also want each component to be able to define 
>> how jtreg is invoked (problemlists, concurrency, agentvm, ...).  There is 
>> already some support for running jtreg tests from the /test folder, but 
>> it is not general enough since it isn't possible to pick individual tests or 
>> specific test groups. It is also not possible to direct the output to a 
>> common location. 
>> 
>> The proposed solution is to be able to run jtreg tests using make from 
>> within the /test folder like this: 
>> 
>>  $ make JT_JAVA= JT_HOME= PRODUCT_HOME=> path> \
>>TESTDIRS=../ TEST_SELECTION=> test or jtreg group> \
>>TEST_OUTPUT_DIR= jtreg_tests 
>> 
>> This will run the specified tests and collect output in 
>> TEST_OUTPUT_DIR/jtreg. EXTRA_JTREG_OPTIONS is also supported to add 
>> additional options to the jtreg command line. 
>> 
>> No existing behavior should be changed.
>> 
>> webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 
>> 
>> bug: https://bugs.openjdk.java.net/browse/JDK-8156658 
>> 
>> 
>> Thanks,
>> /Staffan
>> 
> 
> 
> -- 
> Dmitry Samersoff
> Oracle Java development team, Saint Petersburg, Russia
> * I would love to change the world, but they won't give me the sources.

Re: RFR: JDK-8156658 Common way to run jtreg tests

[Dmitry Samersoff]

Staffan,

Long awaited changes, thank you for doing it.

Is it possible to use the same code for hotspot/test/Makefile:128 and
 jdk/test/Makefile:84

  84   ifdef ALT_OUTPUTDIR
  85 ABS_OUTPUTDIR = $(shell $(CD) $(ALT_OUTPUTDIR) && $(PWD))
  86   else
  87 ABS_OUTPUTDIR = $(shell $(CD) $(TEST_ROOT)/.. && $(PWD))
  88   endif
  89
  90   ABS_PLATFORM_BUILD_ROOT = $(ABS_OUTPUTDIR)
  91   ABS_TEST_OUTPUT_DIR :=
$(ABS_PLATFORM_BUILD_ROOT)/testoutput/$(UNIQUE_DIR)

-Dmitry

On 2016-05-10 15:21, Staffan Larsen wrote:
> For test automation purposes we want a way to run jtreg tests for the 
> different components (jdk, langtools, jaxp, hotspot, nashorn) in a unified 
> way from the command line. We also want each component to be able to define 
> how jtreg is invoked (problemlists, concurrency, agentvm, ...).  There is 
> already some support for running jtreg tests from the /test folder, but 
> it is not general enough since it isn't possible to pick individual tests or 
> specific test groups. It is also not possible to direct the output to a 
> common location. 
> 
> The proposed solution is to be able to run jtreg tests using make from within 
> the /test folder like this: 
> 
>   $ make JT_JAVA= JT_HOME= PRODUCT_HOME= path> \
> TESTDIRS=../ TEST_SELECTION= test or jtreg group> \
> TEST_OUTPUT_DIR= jtreg_tests 
> 
> This will run the specified tests and collect output in 
> TEST_OUTPUT_DIR/jtreg. EXTRA_JTREG_OPTIONS is also supported to add 
> additional options to the jtreg command line. 
> 
> No existing behavior should be changed.
> 
> webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 
> 
> bug: https://bugs.openjdk.java.net/browse/JDK-8156658 
> 
> 
> Thanks,
> /Staffan
> 


-- 
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* I would love to change the world, but they won't give me the sources.

Re: RFR: JDK-8156658 Common way to run jtreg tests

[Staffan Larsen]

Thanks Erik!

> On 10 maj 2016, at 14:39, Erik Joelsson  wrote:
> 
> Looks good to me.
> 
> /Erik
> 
> On 2016-05-10 14:21, Staffan Larsen wrote:
>> For test automation purposes we want a way to run jtreg tests for the 
>> different components (jdk, langtools, jaxp, hotspot, nashorn) in a unified 
>> way from the command line. We also want each component to be able to define 
>> how jtreg is invoked (problemlists, concurrency, agentvm, ...).  There is 
>> already some support for running jtreg tests from the /test folder, but 
>> it is not general enough since it isn't possible to pick individual tests or 
>> specific test groups. It is also not possible to direct the output to a 
>> common location.
>> 
>> The proposed solution is to be able to run jtreg tests using make from 
>> within the /test folder like this:
>> 
>>   $ make JT_JAVA= JT_HOME= PRODUCT_HOME=> path> \
>> TESTDIRS=../ TEST_SELECTION=> test or jtreg group> \
>> TEST_OUTPUT_DIR= jtreg_tests
>> 
>> This will run the specified tests and collect output in 
>> TEST_OUTPUT_DIR/jtreg. EXTRA_JTREG_OPTIONS is also supported to add 
>> additional options to the jtreg command line.
>> 
>> No existing behavior should be changed.
>> 
>> webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 
>> 
>> bug: https://bugs.openjdk.java.net/browse/JDK-8156658 
>> 
>> 
>> Thanks,
>> /Staffan
> 

Re: RFR: JDK-8156658 Common way to run jtreg tests

[Erik Joelsson]

Looks good to me.

/Erik

On 2016-05-10 14:21, Staffan Larsen wrote:

For test automation purposes we want a way to run jtreg tests for the different 
components (jdk, langtools, jaxp, hotspot, nashorn) in a unified way from the command 
line. We also want each component to be able to define how jtreg is invoked 
(problemlists, concurrency, agentvm, ...).  There is already some support for running 
jtreg tests from the /test folder, but it is not general enough since it 
isn't possible to pick individual tests or specific test groups. It is also not 
possible to direct the output to a common location.

The proposed solution is to be able to run jtreg tests using make from within the 
/test folder like this:

   $ make JT_JAVA= JT_HOME= PRODUCT_HOME= \
 TESTDIRS=../ TEST_SELECTION= \
 TEST_OUTPUT_DIR= jtreg_tests

This will run the specified tests and collect output in TEST_OUTPUT_DIR/jtreg. 
EXTRA_JTREG_OPTIONS is also supported to add additional options to the jtreg 
command line.

No existing behavior should be changed.

webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 

bug: https://bugs.openjdk.java.net/browse/JDK-8156658 


Thanks,
/Staffan

RFR: JDK-8156658 Common way to run jtreg tests

[Staffan Larsen]

For test automation purposes we want a way to run jtreg tests for the different 
components (jdk, langtools, jaxp, hotspot, nashorn) in a unified way from the 
command line. We also want each component to be able to define how jtreg is 
invoked (problemlists, concurrency, agentvm, ...).  There is already some 
support for running jtreg tests from the /test folder, but it is not 
general enough since it isn't possible to pick individual tests or specific 
test groups. It is also not possible to direct the output to a common location. 

The proposed solution is to be able to run jtreg tests using make from within 
the /test folder like this: 

  $ make JT_JAVA= JT_HOME= PRODUCT_HOME= \
TESTDIRS=../ TEST_SELECTION= \
TEST_OUTPUT_DIR= jtreg_tests 

This will run the specified tests and collect output in TEST_OUTPUT_DIR/jtreg. 
EXTRA_JTREG_OPTIONS is also supported to add additional options to the jtreg 
command line. 

No existing behavior should be changed.

webrev: http://cr.openjdk.java.net/~sla/8156658/webrev.03/ 

bug: https://bugs.openjdk.java.net/browse/JDK-8156658 


Thanks,
/Staffan