Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: Geir Magnusson Jr. wrote: Gregory Shimansky wrote: One reason would be is that I don't know ant well enough to redesign the whole stuff all together. I used the existing setup and init targets which take care of including ancontrip and cctask jars. If you ask me, I'd prefer make in the first place, ant is just too foreign to me. There is no reason to use caps, we didn't even start to discuss how we want to see drlvm build and when WE ARE GOING TO GET RID OF IT at some point. The caps were to get your attention. I thought you had a nice way to create a standalone testbed and then hook that in. Well as I've written already, there is very much that is done in setup and init targets of drlvm build. It is checking for necessary jar files like junit, ant-contrib, cctask. Also these targets setup a lot of necessary properties. I just didn't want to duplicate all of that stuff. The fact that these targets (setup and init) also do XML transform is almost not used in jvmti tests. Yes there are 2 select which are processed in XML transform, but I can remove them when necessary. I consider this dependency on current drlvm build minor. If we decide to get rid of XML processing, the changes in jvmti tests build shall be minimal. Does it sound ok to you? Hey, the work is done :) The fact is, you can still have a dependency in init and setup to get the goodness from there. I hope to look at this on the plane home tonight. The time invested, well... I learned a lot since the last time I used ant. Maybe one day I'll be able to write something as impressive and unmaintainable as the current drlvm built :) Seriously, if we're going to change it, let's discuss it how we want it to look like and which tool we'll use. I vote for make (gnu version, that is gmake), even on win32 it exists in cygwin and mingw. I think that we should simply use the same tooling that we're using now in classlib. Ok let's decide that exactly we don't like in the current drlvm build. Probably I should start another thread. We decided that last may, didn't we? geir
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Geir Magnusson Jr. wrote: Gregory Shimansky wrote: One reason would be is that I don't know ant well enough to redesign the whole stuff all together. I used the existing setup and init targets which take care of including ancontrip and cctask jars. If you ask me, I'd prefer make in the first place, ant is just too foreign to me. There is no reason to use caps, we didn't even start to discuss how we want to see drlvm build and when WE ARE GOING TO GET RID OF IT at some point. The caps were to get your attention. I thought you had a nice way to create a standalone testbed and then hook that in. Well as I've written already, there is very much that is done in setup and init targets of drlvm build. It is checking for necessary jar files like junit, ant-contrib, cctask. Also these targets setup a lot of necessary properties. I just didn't want to duplicate all of that stuff. The fact that these targets (setup and init) also do XML transform is almost not used in jvmti tests. Yes there are 2 select which are processed in XML transform, but I can remove them when necessary. I consider this dependency on current drlvm build minor. If we decide to get rid of XML processing, the changes in jvmti tests build shall be minimal. Does it sound ok to you? The time invested, well... I learned a lot since the last time I used ant. Maybe one day I'll be able to write something as impressive and unmaintainable as the current drlvm built :) Seriously, if we're going to change it, let's discuss it how we want it to look like and which tool we'll use. I vote for make (gnu version, that is gmake), even on win32 it exists in cygwin and mingw. I think that we should simply use the same tooling that we're using now in classlib. Ok let's decide that exactly we don't like in the current drlvm build. Probably I should start another thread. -- Gregory
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
On Sunday 12 November 2006 00:39 Geir Magnusson Jr. wrote: Ok I think I've come up with a reasonable compromise. I still used the whole system of converting XML and all the stuff. It does quite a lot of things in setup and init targets and using select is convenient. I don't know how to untangle all of the setup and not do a lot of duplication in ant scripting which I am not big expert in. Why? Why do we want to persist with this system, when WE ARE GOING TO GET RID OF IT at some point? One reason would be is that I don't know ant well enough to redesign the whole stuff all together. I used the existing setup and init targets which take care of including ancontrip and cctask jars. If you ask me, I'd prefer make in the first place, ant is just too foreign to me. There is no reason to use caps, we didn't even start to discuss how we want to see drlvm build and when WE ARE GOING TO GET RID OF IT at some point. Did you also fix the silliness of having to use annotations to exclude tests? Please? :) No, the patch has an exclude example using and exclude statement in patternset in its beginning. I'm glad we have these tests, but really... I wish you hadn't invested the time integrating it into the DRLVM build system... Even if we write a new one from scratch I want the tests right now. There were several times when JVMTI was broken since there were no tests for it at all since it is a special VM mode no one usually uses. The time invested, well... I learned a lot since the last time I used ant. Maybe one day I'll be able to write something as impressive and unmaintainable as the current drlvm built :) Seriously, if we're going to change it, let's discuss it how we want it to look like and which tool we'll use. I vote for make (gnu version, that is gmake), even on win32 it exists in cygwin and mingw. -- Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: On Sunday 12 November 2006 00:39 Geir Magnusson Jr. wrote: Ok I think I've come up with a reasonable compromise. I still used the whole system of converting XML and all the stuff. It does quite a lot of things in setup and init targets and using select is convenient. I don't know how to untangle all of the setup and not do a lot of duplication in ant scripting which I am not big expert in. Why? Why do we want to persist with this system, when WE ARE GOING TO GET RID OF IT at some point? One reason would be is that I don't know ant well enough to redesign the whole stuff all together. I used the existing setup and init targets which take care of including ancontrip and cctask jars. If you ask me, I'd prefer make in the first place, ant is just too foreign to me. There is no reason to use caps, we didn't even start to discuss how we want to see drlvm build and when WE ARE GOING TO GET RID OF IT at some point. The caps were to get your attention. I thought you had a nice way to create a standalone testbed and then hook that in. Did you also fix the silliness of having to use annotations to exclude tests? Please? :) No, the patch has an exclude example using and exclude statement in patternset in its beginning. Yay! I'm glad we have these tests, but really... I wish you hadn't invested the time integrating it into the DRLVM build system... Even if we write a new one from scratch I want the tests right now. There were several times when JVMTI was broken since there were no tests for it at all since it is a special VM mode no one usually uses. That's not the issue - we all agree we need the tests right now. What I'm arguing about is doing the tests, and then doing the integration into a build system we don't want. The time invested, well... I learned a lot since the last time I used ant. Maybe one day I'll be able to write something as impressive and unmaintainable as the current drlvm built :) Seriously, if we're going to change it, let's discuss it how we want it to look like and which tool we'll use. I vote for make (gnu version, that is gmake), even on win32 it exists in cygwin and mingw. I think that we should simply use the same tooling that we're using now in classlib. geir
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory, I have checked the patch. I like it. Here are few notes. * When I used ant there were no for tag [1]. I used autogenerated ant files to run something in a loop. Solution with for takes less place and is more readable. * From my perspective 3 min timeout for a smoke test should be decreased. I think there should be no stress/reliaiblity loads during acceptance testing. The reason is simple: complex loads demonstrate unpredicable behavior and do not reveal problems with 100% accuracy, so the bad patch pass such acceptance tests sooner or later. * As for TestNG concern, I don't think we need to stick to the harness. When the time comes, we will change the harness painlessly. 1. http://mail-archives.apache.org/mod_mbox/ant-user/200410.mbox/[EMAIL PROTECTED] On 11/10/06, Gregory Shimansky [EMAIL PROTECTED] wrote: On Thursday 02 November 2006 23:24 Geir Magnusson Jr. wrote: That's an understatement. Don't feel bad. I've never seen anything like it before. The idea of generating ant scripts on teh fly is very unconventional. . You don't have enough cuts and bruises from working with the DRLVM build :) Ok I think I've come up with a reasonable compromise. I still used the whole system of converting XML and all the stuff. It does quite a lot of things in setup and init targets and using select is convenient. I don't know how to untangle all of the setup and not do a lot of duplication in ant scripting which I am not big expert in. But I managed to cut away the loop over components looking at how test target in build.xml is written. I've also converted smoke.test target to the same way because both jvmti and smoke tests are meant for a whole VM, not some component of it. This also made a weird bug go away when of smoke tests were built and run in some random subdirectory of semis instead of being in vm when they were ran separately as build smoke.test. Tests should be in their own subdirectories (main test inclusion/exclusion loop is done over them), main Java class for application has to be equal to have package and name equal to its subdirectory. Otherwise the build system won't know what to run. Other files may have any kind of names. I wrote one simple JVMTI test to start the suite. Other tests which I've experimented with I cannot submit because I didn't write them. I think they'll appear later from JIRAs like one in HARMONY-2143 which were submitted to ASF. Take a look at HARMONY-2151 and say what you think. If I don't get much opposition I'll commit the patch on this weekend. Don't shoot me. Writing even that much of Ant took a lot of time, beer and hair on my head. I said I am not an ant guru, didn't I? -- Gregory Shimansky, Intel Middleware Products Division -- Thank you, Alexei
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: On Thursday 02 November 2006 23:24 Geir Magnusson Jr. wrote: That's an understatement. Don't feel bad. I've never seen anything like it before. The idea of generating ant scripts on teh fly is very unconventional. . You don't have enough cuts and bruises from working with the DRLVM build :) Ok I think I've come up with a reasonable compromise. I still used the whole system of converting XML and all the stuff. It does quite a lot of things in setup and init targets and using select is convenient. I don't know how to untangle all of the setup and not do a lot of duplication in ant scripting which I am not big expert in. Why? Why do we want to persist with this system, when WE ARE GOING TO GET RID OF IT at some point? But I managed to cut away the loop over components looking at how test target in build.xml is written. I've also converted smoke.test target to the same way because both jvmti and smoke tests are meant for a whole VM, not some component of it. This also made a weird bug go away when of smoke tests were built and run in some random subdirectory of semis instead of being in vm when they were ran separately as build smoke.test. Did you also fix the silliness of having to use annotations to exclude tests? Please? :) Tests should be in their own subdirectories (main test inclusion/exclusion loop is done over them), main Java class for application has to be equal to have package and name equal to its subdirectory. Otherwise the build system won't know what to run. Other files may have any kind of names. I wrote one simple JVMTI test to start the suite. Other tests which I've experimented with I cannot submit because I didn't write them. I think they'll appear later from JIRAs like one in HARMONY-2143 which were submitted to ASF. Take a look at HARMONY-2151 and say what you think. If I don't get much opposition I'll commit the patch on this weekend. Don't shoot me. Writing even that much of Ant took a lot of time, beer and hair on my head. I said I am not an ant guru, didn't I? I'm glad we have these tests, but really... I wish you hadn't invested the time integrating it into the DRLVM build system... geir
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
On Thursday 02 November 2006 23:24 Geir Magnusson Jr. wrote: That's an understatement. Don't feel bad. I've never seen anything like it before. The idea of generating ant scripts on teh fly is very unconventional. . You don't have enough cuts and bruises from working with the DRLVM build :) Ok I think I've come up with a reasonable compromise. I still used the whole system of converting XML and all the stuff. It does quite a lot of things in setup and init targets and using select is convenient. I don't know how to untangle all of the setup and not do a lot of duplication in ant scripting which I am not big expert in. But I managed to cut away the loop over components looking at how test target in build.xml is written. I've also converted smoke.test target to the same way because both jvmti and smoke tests are meant for a whole VM, not some component of it. This also made a weird bug go away when of smoke tests were built and run in some random subdirectory of semis instead of being in vm when they were ran separately as build smoke.test. Tests should be in their own subdirectories (main test inclusion/exclusion loop is done over them), main Java class for application has to be equal to have package and name equal to its subdirectory. Otherwise the build system won't know what to run. Other files may have any kind of names. I wrote one simple JVMTI test to start the suite. Other tests which I've experimented with I cannot submit because I didn't write them. I think they'll appear later from JIRAs like one in HARMONY-2143 which were submitted to ASF. Take a look at HARMONY-2151 and say what you think. If I don't get much opposition I'll commit the patch on this weekend. Don't shoot me. Writing even that much of Ant took a lot of time, beer and hair on my head. I said I am not an ant guru, didn't I? -- Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Geir Magnusson Jr. wrote: Why not use junit? Junit task in implicitly assumes that you can run all tests in single JVM instance. Though it does provide fork-mode to run tests in separate JVM processes, it still does not allow per-test JVM args configuration. Which is exactly what we need for JVMTI tests, as most of the test has their own special agent to be run with. Nevertheless, we could use junit task with some benefit in creating test reports. I think this can be accomodated in Gregory's scheme by using junit task instead of java task. The most of the ant mechanics (selecting tests and looping over them for compilation and running) will be there no matter what we use for test running, junit or java.
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
On 11/2/06, Salikh Zakirov [EMAIL PROTECTED] wrote: Geir Magnusson Jr. wrote: Why not use junit? Junit task in implicitly assumes that you can run all tests in single JVM instance. Though it does provide fork-mode to run tests in separate JVM processes, it still does not allow per-test JVM args configuration. Which is exactly what we need for JVMTI tests, as most of the test has their own special agent to be run with. Salikh, the problem you described is very similar to the problem I had to solve in JIT internal testing framework: run every test in a separate VM. ( http://issues.apache.org/jira/browse/HARMONY-1531) I've created a class that runs VM, waits until the process is finished, reads its err and out streams and returns to the junit testcase. + Every native JIT test has Java prototype. Do you want to create something similar? May be we can join our solutions? -- Mikhail Fursov
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Salikh Zakirov wrote: Geir Magnusson Jr. wrote: Why not use junit? Junit task in implicitly assumes that you can run all tests in single JVM instance. Though it does provide fork-mode to run tests in separate JVM processes, it still does not allow per-test JVM args configuration. Which is exactly what we need for JVMTI tests, as most of the test has their own special agent to be run with. I understand the agent is needed, but I don't see why you can't do that that with junit. Can't you just invoke ant w/ an exec? Nevertheless, we could use junit task with some benefit in creating test reports. I think this can be accomodated in Gregory's scheme by using junit task instead of java task. The most of the ant mechanics (selecting tests and looping over them for compilation and running) will be there no matter what we use for test running, junit or java. Right - I'm just tired of adding to the mess that is, IMO, the DRLVM build. I'd rather see new things be more conventional. geir
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Geir Magnusson Jr. wrote: Why not use junit? If you mean why not use junit to loop over tests, then it is not the case. I've used junit to do this in my work. The loop which I wrote here is the loop over components in the build.xml of drlvm. If you run build smoke.test you'll see that the same tests are repeated several times (you have to be patient to see this). This is done because build loops over all known VM components like vmi, vmocode, gc, interpreter, etc. It is Ok for building, and it is done exactly for building, but it is not Ok for testing because repeating tests for the whole JVM for each of its components makes no sense. Gregory Shimansky wrote: On Wednesday 01 November 2006 19:48 Alexey Varlamov wrote: Gregory, I observed similar quirks with paths while intergrating kernel tests into build. AFAIU the Grand Design is the following: there are abstracted targets and isolated component descriptors; build system iterates through all components and tries to apply given target to each component. So there are various tricks to stop it running tests multiple times a-la recurring inclusion protection in C headers. I do not grok how it calculates dependencies though, but it is quite easy to drive it mad and it starts doing wrong sequence of targets and picks wrong components etc. So I just snipped off that fanciful machinery and made simple subant for kernel.test target - see its definition in build/make/build.xml, and compare with nearby smoke.test one. Ok I've made it though all of the interesting ant tricks and created my own jvmti.test target. I noticed the difference of how kernel.test target is included into build.xml as compared to smoke.test or unit.test. AFAIU only test target does actually work to run only once and for the required number of modes (jit, interpreter). This is done with a special workaround for test target in build.xml, it has its own processing. But inclusion of smoke.test and unit.test in build.xml right now makes it run in a loop for all components for which tests don't have any relation to. I am still experimenting with the build, it might be I'll find a solution for individual test categories so it would be possible to run them separately from the general test target. -- Gregory
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: Geir Magnusson Jr. wrote: Why not use junit? If you mean why not use junit to loop over tests, then it is not the case. I've used junit to do this in my work. The loop which I wrote here is the loop over components in the build.xml of drlvm. If you run build smoke.test you'll see that the same tests are repeated several times (you have to be patient to see this). You're joking, right? I do this for ever patch commit I do. This is done because build loops over all known VM components like vmi, vmocode, gc, interpreter, etc. It is Ok for building, and it is done exactly for building, but it is not Ok for testing because repeating tests for the whole JVM for each of its components makes no sense. You're joking here too, right? How do you know if there aren't side effects among components? geir Gregory Shimansky wrote: On Wednesday 01 November 2006 19:48 Alexey Varlamov wrote: Gregory, I observed similar quirks with paths while intergrating kernel tests into build. AFAIU the Grand Design is the following: there are abstracted targets and isolated component descriptors; build system iterates through all components and tries to apply given target to each component. So there are various tricks to stop it running tests multiple times a-la recurring inclusion protection in C headers. I do not grok how it calculates dependencies though, but it is quite easy to drive it mad and it starts doing wrong sequence of targets and picks wrong components etc. So I just snipped off that fanciful machinery and made simple subant for kernel.test target - see its definition in build/make/build.xml, and compare with nearby smoke.test one. Ok I've made it though all of the interesting ant tricks and created my own jvmti.test target. I noticed the difference of how kernel.test target is included into build.xml as compared to smoke.test or unit.test. AFAIU only test target does actually work to run only once and for the required number of modes (jit, interpreter). This is done with a special workaround for test target in build.xml, it has its own processing. But inclusion of smoke.test and unit.test in build.xml right now makes it run in a loop for all components for which tests don't have any relation to. I am still experimenting with the build, it might be I'll find a solution for individual test categories so it would be possible to run them separately from the general test target.
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Geir I actually was serious. Probably you were confused, I didn't write build test, I wrote build smoke.test. The first one works ok, the second doesn't. It happens because test (top-level test target) is handled in a different way from smoke.test (target just for smoke test category) target in build.xml. The smoke.test target just switches default processing target to smoke.test and runs process_components (which in its turn loops over all components). The test special handling of processing components escapes me, I don't quite understand how it works, but it seems to work correctly, without looping. The question I was trying to solve was, how to correctly add jvmti.test target to the build.xml so that it would run only jvmti tests but not loop over components, but when test target it called, jvmti tests category would be executed along with all other categories. Including jvmti.test target into build.xml in the same way as smoke.test doesn't make me happy. Geir Magnusson Jr. wrote: Gregory Shimansky wrote: Geir Magnusson Jr. wrote: Why not use junit? If you mean why not use junit to loop over tests, then it is not the case. I've used junit to do this in my work. The loop which I wrote here is the loop over components in the build.xml of drlvm. If you run build smoke.test you'll see that the same tests are repeated several times (you have to be patient to see this). You're joking, right? I do this for ever patch commit I do. This is done because build loops over all known VM components like vmi, vmocode, gc, interpreter, etc. It is Ok for building, and it is done exactly for building, but it is not Ok for testing because repeating tests for the whole JVM for each of its components makes no sense. You're joking here too, right? How do you know if there aren't side effects among components? geir Gregory Shimansky wrote: On Wednesday 01 November 2006 19:48 Alexey Varlamov wrote: Gregory, I observed similar quirks with paths while intergrating kernel tests into build. AFAIU the Grand Design is the following: there are abstracted targets and isolated component descriptors; build system iterates through all components and tries to apply given target to each component. So there are various tricks to stop it running tests multiple times a-la recurring inclusion protection in C headers. I do not grok how it calculates dependencies though, but it is quite easy to drive it mad and it starts doing wrong sequence of targets and picks wrong components etc. So I just snipped off that fanciful machinery and made simple subant for kernel.test target - see its definition in build/make/build.xml, and compare with nearby smoke.test one. Ok I've made it though all of the interesting ant tricks and created my own jvmti.test target. I noticed the difference of how kernel.test target is included into build.xml as compared to smoke.test or unit.test. AFAIU only test target does actually work to run only once and for the required number of modes (jit, interpreter). This is done with a special workaround for test target in build.xml, it has its own processing. But inclusion of smoke.test and unit.test in build.xml right now makes it run in a loop for all components for which tests don't have any relation to. I am still experimenting with the build, it might be I'll find a solution for individual test categories so it would be possible to run them separately from the general test target. -- Gregory
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: Geir I actually was serious. Probably you were confused, I didn't write build test, I wrote build smoke.test. The first one works ok, the second doesn't. It happens because test (top-level test target) is handled in a different way from smoke.test (target just for smoke test category) target in build.xml. The smoke.test target just switches default processing target to smoke.test and runs process_components (which in its turn loops over all components). The test special handling of processing components escapes me, I don't quite understand how it works, but it seems to work correctly, without looping. I've used them both, and targetted smoke for specific use cases (opt, etc) The question I was trying to solve was, how to correctly add jvmti.test target to the build.xml so that it would run only jvmti tests but not loop over components, but when test target it called, jvmti tests category would be executed along with all other categories. Including jvmti.test target into build.xml in the same way as smoke.test doesn't make me happy. Right - I thought you'd start a revolt and do it outside of the loop over test types system we have now. In order to keep this simple, why not just have a separate test target for now $ sh build.sh test.jvmti and we can stare at that together, and figure out how to integrate... simplest thing would be to rename the current test target to test_loop or something, and then target name=test depends=test_loop, test.jvmti / :) geir Geir Magnusson Jr. wrote: Gregory Shimansky wrote: Geir Magnusson Jr. wrote: Why not use junit? If you mean why not use junit to loop over tests, then it is not the case. I've used junit to do this in my work. The loop which I wrote here is the loop over components in the build.xml of drlvm. If you run build smoke.test you'll see that the same tests are repeated several times (you have to be patient to see this). You're joking, right? I do this for ever patch commit I do. This is done because build loops over all known VM components like vmi, vmocode, gc, interpreter, etc. It is Ok for building, and it is done exactly for building, but it is not Ok for testing because repeating tests for the whole JVM for each of its components makes no sense. You're joking here too, right? How do you know if there aren't side effects among components? geir Gregory Shimansky wrote: On Wednesday 01 November 2006 19:48 Alexey Varlamov wrote: Gregory, I observed similar quirks with paths while intergrating kernel tests into build. AFAIU the Grand Design is the following: there are abstracted targets and isolated component descriptors; build system iterates through all components and tries to apply given target to each component. So there are various tricks to stop it running tests multiple times a-la recurring inclusion protection in C headers. I do not grok how it calculates dependencies though, but it is quite easy to drive it mad and it starts doing wrong sequence of targets and picks wrong components etc. So I just snipped off that fanciful machinery and made simple subant for kernel.test target - see its definition in build/make/build.xml, and compare with nearby smoke.test one. Ok I've made it though all of the interesting ant tricks and created my own jvmti.test target. I noticed the difference of how kernel.test target is included into build.xml as compared to smoke.test or unit.test. AFAIU only test target does actually work to run only once and for the required number of modes (jit, interpreter). This is done with a special workaround for test target in build.xml, it has its own processing. But inclusion of smoke.test and unit.test in build.xml right now makes it run in a loop for all components for which tests don't have any relation to. I am still experimenting with the build, it might be I'll find a solution for individual test categories so it would be possible to run them separately from the general test target.
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Geir Magnusson Jr. wrote: Gregory Shimansky wrote: Geir I actually was serious. Probably you were confused, I didn't write build test, I wrote build smoke.test. The first one works ok, the second doesn't. It happens because test (top-level test target) is handled in a different way from smoke.test (target just for smoke test category) target in build.xml. The smoke.test target just switches default processing target to smoke.test and runs process_components (which in its turn loops over all components). The test special handling of processing components escapes me, I don't quite understand how it works, but it seems to work correctly, without looping. I've used them both, and targetted smoke for specific use cases (opt, etc) Hmm you probably know more about VM build than I do since you've been modifying it for quite some time already. I've just started to look into the whole thing. The question I was trying to solve was, how to correctly add jvmti.test target to the build.xml so that it would run only jvmti tests but not loop over components, but when test target it called, jvmti tests category would be executed along with all other categories. Including jvmti.test target into build.xml in the same way as smoke.test doesn't make me happy. Right - I thought you'd start a revolt and do it outside of the loop over test types system we have now. Well I am not an ant guru, and the current build system definitely requires some time to understand. I've tried to copy most of the stuff from other test ant files to make my own. I think I'll file a JIRA before committing it to make it possible to discuss it. In order to keep this simple, why not just have a separate test target for now $ sh build.sh test.jvmti and we can stare at that together, and figure out how to integrate... simplest thing would be to rename the current test target to test_loop or something, and then target name=test depends=test_loop, test.jvmti / :) Hmm good idea, why didn't I think of it myself?... :) -- Gregory
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: Geir Magnusson Jr. wrote: Gregory Shimansky wrote: Geir I actually was serious. Probably you were confused, I didn't write build test, I wrote build smoke.test. The first one works ok, the second doesn't. It happens because test (top-level test target) is handled in a different way from smoke.test (target just for smoke test category) target in build.xml. The smoke.test target just switches default processing target to smoke.test and runs process_components (which in its turn loops over all components). The test special handling of processing components escapes me, I don't quite understand how it works, but it seems to work correctly, without looping. I've used them both, and targetted smoke for specific use cases (opt, etc) Hmm you probably know more about VM build than I do since you've been modifying it for quite some time already. I've just started to look into the whole thing. The question I was trying to solve was, how to correctly add jvmti.test target to the build.xml so that it would run only jvmti tests but not loop over components, but when test target it called, jvmti tests category would be executed along with all other categories. Including jvmti.test target into build.xml in the same way as smoke.test doesn't make me happy. Right - I thought you'd start a revolt and do it outside of the loop over test types system we have now. Well I am not an ant guru, and the current build system definitely requires some time to understand. That's an understatement. Don't feel bad. I've never seen anything like it before. The idea of generating ant scripts on teh fly is very unconventional. I've tried to copy most of the stuff from other test ant files to make my own. I think I'll file a JIRA before committing it to make it possible to discuss it. In order to keep this simple, why not just have a separate test target for now $ sh build.sh test.jvmti and we can stare at that together, and figure out how to integrate... simplest thing would be to rename the current test target to test_loop or something, and then target name=test depends=test_loop, test.jvmti / :) Hmm good idea, why didn't I think of it myself?... :) You don't have enough cuts and bruises from working with the DRLVM build :) geir
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
On Wednesday 01 November 2006 19:48 Alexey Varlamov wrote: Gregory, I observed similar quirks with paths while intergrating kernel tests into build. AFAIU the Grand Design is the following: there are abstracted targets and isolated component descriptors; build system iterates through all components and tries to apply given target to each component. So there are various tricks to stop it running tests multiple times a-la recurring inclusion protection in C headers. I do not grok how it calculates dependencies though, but it is quite easy to drive it mad and it starts doing wrong sequence of targets and picks wrong components etc. So I just snipped off that fanciful machinery and made simple subant for kernel.test target - see its definition in build/make/build.xml, and compare with nearby smoke.test one. Ok I've made it though all of the interesting ant tricks and created my own jvmti.test target. I noticed the difference of how kernel.test target is included into build.xml as compared to smoke.test or unit.test. AFAIU only test target does actually work to run only once and for the required number of modes (jit, interpreter). This is done with a special workaround for test target in build.xml, it has its own processing. But inclusion of smoke.test and unit.test in build.xml right now makes it run in a loop for all components for which tests don't have any relation to. I am still experimenting with the build, it might be I'll find a solution for individual test categories so it would be possible to run them separately from the general test target. -- Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Geir Magnusson Jr. wrote: Gregory Shimansky wrote: I don't want to create a huge discussion out of it like most [testing] discussions become. Just want to know your arguments to create one more tests category. Because the current frameworks are... wacky. I can't turn off smoke tests without *recompiling* the test. The c-unit test rig is kinda cool, but inappropriate. Maybe kernel could be used. Ok I see your point. I'll try to create my own tests building and running category, maybe if it is good enough we'll transfer smoke tests to it in the future. it sounds like you just want to launch a set of conventional junit tests with a special invocation of java to get the agent running, right? Hmm I didn't think of using junit before your suggestion. Now that I think of it, it can probably work for me. It appears that it is possible to pass special vmarg/ to junit task. I'll give it a try. Now that I looked at the smoke tests build more closely and found a paths problem which I don't know how to fix, I am also inclined to make my own build script to have it separated. jokeI would at least know how it works and own this secret like someone who wrote smoke build script does./joke That's what I was hoping to avoid. Something conventional. JUnit or TestNG (TestNG! TestNG!), and a separate ant script invoked from main script. Hmm I am not familiar with TestNG at all. I'll try junit first. -- Gregory Shimansky, Intel Middleware Products Division
RE: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory wrote, I need is probably smoke tests category. I need to add building native code part and add a custom command line setting somewhere. +1 I believe you need one or two test with a good coverage to check your changes regularly. This is enough for acceptance testing. This doesn't inhibit the separate category - it would be quite useful for thorough testing. But from my perspective this is not the first thing to do. With best regards, Alexei Fedotov, Intel Java XML Engineering -Original Message- From: Gregory Shimansky [mailto:[EMAIL PROTECTED] Sent: Monday, October 30, 2006 11:25 PM To: harmony-dev@incubator.apache.org Subject: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks Hello JVMTI implementation is quite a big piece of drlvm code which doesn't have any tests that are ran regularly. This may create regressions in JVMTI implementation which won't be caught early. So I want to add several small tests for JVMTI which would cover most of the currently implemented functionality. The specific of such tests is that they have to have a custom command line, to specify -agentlib:agent name library. They also require to build native code. They need to be run in default (JIT) mode and on interpreter. Current tests which we have for commit checks for drlvm don't have these features out of the box. Cunit tests build native code, but don't run java executable. Smoke and kernel tests don't have native code and don't have a custom command line. So I should either create a new 4th category for tests with custom build file, or extend one of the current categories which we have. The most close to what I need is probably smoke tests category. I need to add building native code part and add a custom command line setting somewhere. Or do you think I should go with completely new tests category? -- Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: Hello JVMTI implementation is quite a big piece of drlvm code which doesn't have any tests that are ran regularly. This may create regressions in JVMTI implementation which won't be caught early. So I want to add several small tests for JVMTI which would cover most of the currently implemented functionality. Yay! The specific of such tests is that they have to have a custom command line, to specify -agentlib:agent name library. They also require to build native code. They need to be run in default (JIT) mode and on interpreter. Ok. You can do that from ant Current tests which we have for commit checks for drlvm don't have these features out of the box. Cunit tests build native code, but don't run java executable. Smoke and kernel tests don't have native code and don't have a custom command line. So I should either create a new 4th category for tests with custom build file, or extend one of the current categories which we have. The most close to what I need is probably smoke tests category. I need to add building native code part and add a custom command line setting somewhere. Or do you think I should go with completely new tests category? New category geir
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
On Tuesday 31 October 2006 00:24 Fedotov, Alexei A wrote: Gregory wrote, I need is probably smoke tests category. I need to add building native code part and add a custom command line setting somewhere. +1 I believe you need one or two test with a good coverage to check your changes regularly. This is enough for acceptance testing. This doesn't inhibit the separate category - it would be quite useful for thorough testing. But from my perspective this is not the first thing to do. Now if I just could understand the Grand Design behind that huge tangle called drlvm ant build... I cannot find a place to start with. I thought eclipse ant debugging facilities may help (after I read how to build classlib from eclipse), and spent 2 hours trying to give it properties here and there to no avail, building drlvm from eclipse didn't work for me so far. Anyway, I think I've found a path bug with running smoke tests. The paths seems to be different depending on what I run build.sh test or build.sh smoke.test. This is what I see when I run build.sh smoke.test /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/deploy/jre/bin/java -Dvm.assert_dialog=0 -classpath /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/semis/vm/interpreter/_smoke.tests/classes StackTest If you look closely you'll see that tests are taken from vm/interpreter/_smoke.tests/classes while the correct location which is built when I run build.sh test is vm/_smoke.tests/classes. There is no interpreter path in it. If someone knows how to fix it, I'd be grateful. Now that I've tried it a second time after a full rebuild the path looks like this /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/deploy/jre/bin/java -Dvm.assert_dialog=0 -classpath /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/semis/vm/gc_gen/_smoke.tests/classes StackTest Funny how it works picking up seemingly a random subdirectory in build/lnx_ia32_gcc_debug/semis. -- Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
On Tuesday 31 October 2006 02:27 Geir Magnusson Jr. wrote: So I should either create a new 4th category for tests with custom build file, or extend one of the current categories which we have. The most close to what I need is probably smoke tests category. I need to add building native code part and add a custom command line setting somewhere. Or do you think I should go with completely new tests category? New category Hmm... I shouldn't have asked, or I wouldn't receive two different answers :) I don't want to create a huge discussion out of it like most [testing] discussions become. Just want to know your arguments to create one more tests category. Now that I looked at the smoke tests build more closely and found a paths problem which I don't know how to fix, I am also inclined to make my own build script to have it separated. jokeI would at least know how it works and own this secret like someone who wrote smoke build script does./joke -- Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: On Tuesday 31 October 2006 00:24 Fedotov, Alexei A wrote: Gregory wrote, I need is probably smoke tests category. I need to add building native code part and add a custom command line setting somewhere. +1 I believe you need one or two test with a good coverage to check your changes regularly. This is enough for acceptance testing. This doesn't inhibit the separate category - it would be quite useful for thorough testing. But from my perspective this is not the first thing to do. Now if I just could understand the Grand Design behind that huge tangle called drlvm ant build... I cannot find a place to start with. I thought eclipse ant debugging facilities may help (after I read how to build classlib from eclipse), and spent 2 hours trying to give it properties here and there to no avail, building drlvm from eclipse didn't work for me so far. That's why I said new category. Do something conventional. Anyway, I think I've found a path bug with running smoke tests. The paths seems to be different depending on what I run build.sh test or build.sh smoke.test. This is what I see when I run build.sh smoke.test /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/deploy/jre/bin/java -Dvm.assert_dialog=0 -classpath /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/semis/vm/interpreter/_smoke.tests/classes StackTest If you look closely you'll see that tests are taken from vm/interpreter/_smoke.tests/classes while the correct location which is built when I run build.sh test is vm/_smoke.tests/classes. There is no interpreter path in it. If someone knows how to fix it, I'd be grateful. Now that I've tried it a second time after a full rebuild the path looks like this /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/deploy/jre/bin/java -Dvm.assert_dialog=0 -classpath /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/semis/vm/gc_gen/_smoke.tests/classes StackTest Funny how it works picking up seemingly a random subdirectory in build/lnx_ia32_gcc_debug/semis.
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory Shimansky wrote: On Tuesday 31 October 2006 02:27 Geir Magnusson Jr. wrote: So I should either create a new 4th category for tests with custom build file, or extend one of the current categories which we have. The most close to what I need is probably smoke tests category. I need to add building native code part and add a custom command line setting somewhere. Or do you think I should go with completely new tests category? New category Hmm... I shouldn't have asked, or I wouldn't receive two different answers :) I don't want to create a huge discussion out of it like most [testing] discussions become. Just want to know your arguments to create one more tests category. Because the current frameworks are... wacky. I can't turn off smoke tests without *recompiling* the test. The c-unit test rig is kinda cool, but inappropriate. Maybe kernel could be used. it sounds like you just want to launch a set of conventional junit tests with a special invocation of java to get the agent running, right? Now that I looked at the smoke tests build more closely and found a paths problem which I don't know how to fix, I am also inclined to make my own build script to have it separated. jokeI would at least know how it works and own this secret like someone who wrote smoke build script does./joke That's what I was hoping to avoid. Something conventional. JUnit or TestNG (TestNG! TestNG!), and a separate ant script invoked from main script. geir
