RE: [drlvm][unit] 100% of class library tests pass
Geir wrote, > I'd prefer if we didn't spam the lists with every good result Ok, the alternative which comes to my mind next is the following. If I understand correctly, Anton's site gets uploaded zip archives, processes them and populates a database. From the outside we can see result sets for database selections. Anton, If you add a possibility to browse and download uploaded archives, this would help to detect problems from the outside. One can compare uploaded archives and selection results. The only possible problem is if you don't keep archives after they are processed. I don't think we should keep them for long either. I think this worth to be done if you found this useful for debug feature for yourself. With best regards, Alexei Fedotov, Intel Java & XML Engineering >-Original Message- >From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED] >Sent: Friday, November 17, 2006 3:05 AM >To: harmony-dev@incubator.apache.org >Subject: Re: [drlvm][unit] 100% of class library tests pass > > > >Alexei Fedotov wrote: >> Pavel, >> The life started showing that you were correct. Today there were no >> report on http://harmonytest.org. Even if I would like to be a living >> notification, I couldn't. >> >> Vladimir, >> The thing which concerns me most is not an absence of results - I >> believe this is just a technical problem. For me the main problem is >> that without you there is no way to understand what happens. > >I don't understand that. > >The goal here is to establish the build-test framework as the thing that >everyone uses- we aren't dependent only upon Vladimir. > >I'll have a version running on 64-bit ubuntu whenever it works, and >probalby 32-bit as well reporting in... > >> >> Can we made a process of reporting more open? For example, can we tune >> CC to send a notification on upload status? If there is a successful >> notification, then the file is uploaded successfully, and we need to >> ask Anton why it is not visible. Otherwise we'll know the problem is >> in CC. > >Um. I'd prefer if we didn't spam the lists with every good result - we >only want to know when there's breakage or "fixage". > >geir > >> >> Thanks! >> >> >> >> On 11/16/06, Pavel Ozhdikhin <[EMAIL PROTECTED]> wrote: >>> Sorry to say but it actually does not work until there is no >>> notifications >>> to the mailing list and no immediate reaction to the regressions. >>> >>> thanks, >>> Pavel >>> >>> >>> On 11/16/06, Fedotov, Alexei A <[EMAIL PROTECTED]> wrote: >>> > >>> > Pavel, All, >>> > >>> > Let me add one "pro" for the second approach: it works already. >>> > Vladimir's scripts daily upload test results to >http://harmonytest.org. >>> > >>> > With best regards, >>> > Alexei Fedotov, >>> > Intel Java & XML Engineering >>> > >>> > >-Original Message- >>> > >From: Tim Ellison [mailto:[EMAIL PROTECTED] >>> > >Sent: Thursday, November 16, 2006 12:37 PM >>> > >To: harmony-dev@incubator.apache.org >>> > >Subject: Re: [drlvm][unit] 100% of class library tests pass >>> > > >>> > >Pavel Ozhdikhin wrote: >>> > >> We have to evolving systems - classlib and DRLVM. To check >>> commits to >>> > >> classlib we need a stable DRLVM which can pass 100% of HUT. >>> Otherwise >>> > >it's >>> > >> impossible to use DRLVM for pre-commit testing - you never know >>> > whether >>> > >> your >>> > >> test fail because of your patch or due to latest changes in DRLVM. >>> > >> >>> > >> I remember the time when DRLVM and Jitrino actively evolved - for >>> > some >>> > >time >>> > >> JIT had to use an older version of DRLVM which could pass all >commit >>> > >> criteria because newer versions suffered from regressions. And >>> > finally we >>> > >> came to comon strict commit criteria which prevented regressions in >>> > both >>> > >VM >>> > >> and JIT. >>> > >> >>> > >> To avoid regressions using DRLVM in classlib testing I see 3 >>> possible >>> > >> solutions: >>> > >> >>> > >> 1. Use one fixed DRLVM version whi
Re: [drlvm][unit] 100% of class library tests pass
Alexei Fedotov wrote: Pavel, The life started showing that you were correct. Today there were no report on http://harmonytest.org. Even if I would like to be a living notification, I couldn't. Vladimir, The thing which concerns me most is not an absence of results - I believe this is just a technical problem. For me the main problem is that without you there is no way to understand what happens. I don't understand that. The goal here is to establish the build-test framework as the thing that everyone uses- we aren't dependent only upon Vladimir. I'll have a version running on 64-bit ubuntu whenever it works, and probalby 32-bit as well reporting in... Can we made a process of reporting more open? For example, can we tune CC to send a notification on upload status? If there is a successful notification, then the file is uploaded successfully, and we need to ask Anton why it is not visible. Otherwise we'll know the problem is in CC. Um. I'd prefer if we didn't spam the lists with every good result - we only want to know when there's breakage or "fixage". geir Thanks! On 11/16/06, Pavel Ozhdikhin <[EMAIL PROTECTED]> wrote: Sorry to say but it actually does not work until there is no notifications to the mailing list and no immediate reaction to the regressions. thanks, Pavel On 11/16/06, Fedotov, Alexei A <[EMAIL PROTECTED]> wrote: > > Pavel, All, > > Let me add one "pro" for the second approach: it works already. > Vladimir's scripts daily upload test results to http://harmonytest.org. > > With best regards, > Alexei Fedotov, > Intel Java & XML Engineering > > >-Original Message- > >From: Tim Ellison [mailto:[EMAIL PROTECTED] > >Sent: Thursday, November 16, 2006 12:37 PM > >To: harmony-dev@incubator.apache.org > >Subject: Re: [drlvm][unit] 100% of class library tests pass > > > >Pavel Ozhdikhin wrote: > >> We have to evolving systems - classlib and DRLVM. To check commits to > >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise > >it's > >> impossible to use DRLVM for pre-commit testing - you never know > whether > >> your > >> test fail because of your patch or due to latest changes in DRLVM. > >> > >> I remember the time when DRLVM and Jitrino actively evolved - for > some > >time > >> JIT had to use an older version of DRLVM which could pass all commit > >> criteria because newer versions suffered from regressions. And > finally we > >> came to comon strict commit criteria which prevented regressions in > both > >VM > >> and JIT. > >> > >> To avoid regressions using DRLVM in classlib testing I see 3 possible > >> solutions: > >> > >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update > this > >> version from time to time. > >>Pros: + Less time to run DRLVM pre-commit tests > >> + Classlib does not suffer from regressions in DRLVM > >>Cons: - DRLVM will suffer from regressions > >> - Classlib can not use the latest DRLVM > >> - Need additional efforts to regain stability on DRLVM > >> when we want to update the version for classlib > testing > >> > >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits > causing > >> regressions > >>Pros: + Less time to run DRLVM pre-commit tests > >> + Classlib can use the latest DRLVM > >>Cons: - Classlib can suffer from DRLVM regressions (time lag > before > >> rollback) > >> - It is not always clear which commit caused a > regression > >> - Rollbacks are costly > >> > >> 3. Add HUT to the commit criteria for DRLVM > >> Pros: + Classlib always can use the latest DRLVM > >> + DRLVM has no regressions regarding to HUT > >> Cons: - More time to run DRLVM pre-commit tests (I was told that > HUT > >> take 25 minutes running in single JVM mode) > >> > >> I think that preventing a problem is better than solving it > afterwards. > >So, > >> I personally would choose the 3rd approach, don't mind against the > second > >> and dislike the first one. Probably some combination of these is > >possible. > > > >While I appreciate the desire to keep things stable, I think it is > >unreasonable to ask developers to run the entire test suite each time. > >As we have seen in the classlib code, running targeted tests befo
Re: [drlvm][unit] 100% of class library tests pass
Pavel, The life started showing that you were correct. Today there were no report on http://harmonytest.org. Even if I would like to be a living notification, I couldn't. Vladimir, The thing which concerns me most is not an absence of results - I believe this is just a technical problem. For me the main problem is that without you there is no way to understand what happens. Can we made a process of reporting more open? For example, can we tune CC to send a notification on upload status? If there is a successful notification, then the file is uploaded successfully, and we need to ask Anton why it is not visible. Otherwise we'll know the problem is in CC. Thanks! On 11/16/06, Pavel Ozhdikhin <[EMAIL PROTECTED]> wrote: Sorry to say but it actually does not work until there is no notifications to the mailing list and no immediate reaction to the regressions. thanks, Pavel On 11/16/06, Fedotov, Alexei A <[EMAIL PROTECTED]> wrote: > > Pavel, All, > > Let me add one "pro" for the second approach: it works already. > Vladimir's scripts daily upload test results to http://harmonytest.org. > > With best regards, > Alexei Fedotov, > Intel Java & XML Engineering > > >-Original Message- > >From: Tim Ellison [mailto:[EMAIL PROTECTED] > >Sent: Thursday, November 16, 2006 12:37 PM > >To: harmony-dev@incubator.apache.org > >Subject: Re: [drlvm][unit] 100% of class library tests pass > > > >Pavel Ozhdikhin wrote: > >> We have to evolving systems - classlib and DRLVM. To check commits to > >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise > >it's > >> impossible to use DRLVM for pre-commit testing - you never know > whether > >> your > >> test fail because of your patch or due to latest changes in DRLVM. > >> > >> I remember the time when DRLVM and Jitrino actively evolved - for > some > >time > >> JIT had to use an older version of DRLVM which could pass all commit > >> criteria because newer versions suffered from regressions. And > finally we > >> came to comon strict commit criteria which prevented regressions in > both > >VM > >> and JIT. > >> > >> To avoid regressions using DRLVM in classlib testing I see 3 possible > >> solutions: > >> > >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update > this > >> version from time to time. > >>Pros: + Less time to run DRLVM pre-commit tests > >> + Classlib does not suffer from regressions in DRLVM > >>Cons: - DRLVM will suffer from regressions > >> - Classlib can not use the latest DRLVM > >> - Need additional efforts to regain stability on DRLVM > >> when we want to update the version for classlib > testing > >> > >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits > causing > >> regressions > >>Pros: + Less time to run DRLVM pre-commit tests > >> + Classlib can use the latest DRLVM > >>Cons: - Classlib can suffer from DRLVM regressions (time lag > before > >> rollback) > >> - It is not always clear which commit caused a > regression > >> - Rollbacks are costly > >> > >> 3. Add HUT to the commit criteria for DRLVM > >> Pros: + Classlib always can use the latest DRLVM > >> + DRLVM has no regressions regarding to HUT > >> Cons: - More time to run DRLVM pre-commit tests (I was told that > HUT > >> take 25 minutes running in single JVM mode) > >> > >> I think that preventing a problem is better than solving it > afterwards. > >So, > >> I personally would choose the 3rd approach, don't mind against the > second > >> and dislike the first one. Probably some combination of these is > >possible. > > > >While I appreciate the desire to keep things stable, I think it is > >unreasonable to ask developers to run the entire test suite each time. > >As we have seen in the classlib code, running targeted tests before > >commit and leaving the build machines to run continuous tests ensures > >that we are productive and are notified of breakages in time to easily > >back out a patch and re-evaluate. > > > >With the amount of machine time we have running harmony tests on > >different cpu's/os's/compilers/etc we are getting better coverage than > >any individual could be expected to provide. > > > >Which is a long way of saying I think option (2) above is best -- and > >relies on the bid machines letting us know if things break, and the > >commitment from all of us to go straight in and fix it. > > > >Regards, > >Tim > > > >-- > > > >Tim Ellison ([EMAIL PROTECTED]) > >IBM Java technology centre, UK. > -- Thank you, Alexei
Re: [drlvm][unit] 100% of class library tests pass
Stefano Mazzocchi wrote: > Mikhail Loenko wrote: > >> And I hope we will have other workloads running on Harmony nightly and >> reporting regressions to the list. > > Speaking of which, is there a J9 for x86_64 available? Nope, we have not put a Harmony VME up on Developerworks for x86_64. And predicting your next question, yes it would be possible, but it's not an overnight process ;-/ Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: [drlvm][unit] 100% of class library tests pass
Mikhail Loenko wrote: Well let's see how often we will break CI systems. If we break it twice a day then pre-commit testing needs to be strengthened. Right. Exactly. Iterate and adapt. BTW if compile in release mode then all classlib tests run 35 minutes on DRLVM. Once we fix DRLVM to run with the fork mode "once" it will be even faster... Thanks, Mikhail 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: Mikhail Loenko wrote: > why not? Because the full-stack testing is appropriate for CI systems that are running full-time to catch bugs. That's what our build-test infrastructure is all about. Asking DRLVM developers (and conversely, classlib developers) to run 1+ hours of tests for even the smallest commits is a waste of time. We simply need to balance efficiency (targeted testing when you make a fix) with the dedication to have a rapid response when the CI systems find a problem. geir > > 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: >> >> >> Pavel Ozhdikhin wrote: >> > On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: >> >> >> >> Be sure to not miss anyone :) This was a great community effort, with >> >> everyone pitching in. >> >> >> >> DRLVM is now a full peer to J9 in Harmony testing. :) We still need >> >> to use J9 (and another VM that happens to work with our classlibrary), >> >> as a sanity check, but we should from now on use DRLVM in our CI >> testing >> >> framework. >> > >> > >> > On the other hand, to make sure DRLVM has no regressions regarding >> > to Classlibrary Unit Tests it makes sense to add these tests to the >> "test" >> > target of DRLVM build. >> > What do people think? >> >> Adding classlib unit tests to DRLVM test target? No thanks :) >> >> geir >> >> > >> > Thanks, >> > Pavel >> > >> > >> > >> >> geir >> >> >> >> >> >> Alexei Fedotov wrote: >> >> > Oops, I've missed: >> >> > * Andrew Zhang for reviewing class library patches and helpful >> >> discussions >> >> > >> >> > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: >> >> >> >> >> >> Folks, >> >> >> According to http://harmonytest.org, today 100% of class library >> unit >> >> >> tests pass on DRLVM. Thank you all! It takes 44 days for the great >> >> >> team we are. Thanks for your thoughtful, diligent work and deep >> >> >> inspiration. Kudos to you for the following (and not limited to >> this): >> >> >> >> >> >> * Alexey Varlamov and Elena for driving the whole process >> >> >> * Anton and Vladimir Ivanov for automating test runs >> >> >> * Geir and Gregory for checking and committing related DRLVM >> patches >> >> >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and >> >> >> committing related class library patches >> >> >> * Alexey Petrenko for becoming ICU expert and writing good JIRA >> issue >> >> >> resolution guidelines >> >> >> * Alexei Zakharov for resolving class library test issues >> >> >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey >> Ivanov for >> >> >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay >> >> >> Sidelnikov and Alexander Astapchuk for making execution engines >> work >> >> >> * Tatiana and Maxim for filing JIRA issues about test failures >> >> >> * Nikolay Kuznetsov for completing thread interruption handling and >> >> >> reverting consequences of park/unpark integration >> >> >> * Pavel Afremov for fixing exception handling >> >> >> * Boris Kuznetsov for intelligent fixing of security tests >> >> >> * Rana and Salikh for evaluating and discussing problems, reviewing >> >> >> and trying DRLVM patches >> >> >> * Pavel Pervov and Evgueni for help with DRLVM patches >> >> >> * Artem for discovering and fixing weird Windows* behavior >> >> >> * My wife for bringing hot tea to the computer during sleepless >> nights >> >> >> >> >> >> There are still open issues with reliability, multiprocessor and >> other >> >> >> special configurations, so the page >> >> >> http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains >> >> >> active. But this shouldn't prevent us from including class library >> >> >> testing into Harmony "zero regression" policy. What do you think? >> >> >> >> >> >> -- >> >> >> Thank you, >> >> >> Alexei >> >> > >> >> > >> >> > >> >> >> > >> >
Re: [drlvm][unit] 100% of class library tests pass
Well let's see how often we will break CI systems. If we break it twice a day then pre-commit testing needs to be strengthened. BTW if compile in release mode then all classlib tests run 35 minutes on DRLVM. Once we fix DRLVM to run with the fork mode "once" it will be even faster... Thanks, Mikhail 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: Mikhail Loenko wrote: > why not? Because the full-stack testing is appropriate for CI systems that are running full-time to catch bugs. That's what our build-test infrastructure is all about. Asking DRLVM developers (and conversely, classlib developers) to run 1+ hours of tests for even the smallest commits is a waste of time. We simply need to balance efficiency (targeted testing when you make a fix) with the dedication to have a rapid response when the CI systems find a problem. geir > > 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: >> >> >> Pavel Ozhdikhin wrote: >> > On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: >> >> >> >> Be sure to not miss anyone :) This was a great community effort, with >> >> everyone pitching in. >> >> >> >> DRLVM is now a full peer to J9 in Harmony testing. :) We still need >> >> to use J9 (and another VM that happens to work with our classlibrary), >> >> as a sanity check, but we should from now on use DRLVM in our CI >> testing >> >> framework. >> > >> > >> > On the other hand, to make sure DRLVM has no regressions regarding >> > to Classlibrary Unit Tests it makes sense to add these tests to the >> "test" >> > target of DRLVM build. >> > What do people think? >> >> Adding classlib unit tests to DRLVM test target? No thanks :) >> >> geir >> >> > >> > Thanks, >> > Pavel >> > >> > >> > >> >> geir >> >> >> >> >> >> Alexei Fedotov wrote: >> >> > Oops, I've missed: >> >> > * Andrew Zhang for reviewing class library patches and helpful >> >> discussions >> >> > >> >> > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: >> >> >> >> >> >> Folks, >> >> >> According to http://harmonytest.org, today 100% of class library >> unit >> >> >> tests pass on DRLVM. Thank you all! It takes 44 days for the great >> >> >> team we are. Thanks for your thoughtful, diligent work and deep >> >> >> inspiration. Kudos to you for the following (and not limited to >> this): >> >> >> >> >> >> * Alexey Varlamov and Elena for driving the whole process >> >> >> * Anton and Vladimir Ivanov for automating test runs >> >> >> * Geir and Gregory for checking and committing related DRLVM >> patches >> >> >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and >> >> >> committing related class library patches >> >> >> * Alexey Petrenko for becoming ICU expert and writing good JIRA >> issue >> >> >> resolution guidelines >> >> >> * Alexei Zakharov for resolving class library test issues >> >> >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey >> Ivanov for >> >> >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay >> >> >> Sidelnikov and Alexander Astapchuk for making execution engines >> work >> >> >> * Tatiana and Maxim for filing JIRA issues about test failures >> >> >> * Nikolay Kuznetsov for completing thread interruption handling and >> >> >> reverting consequences of park/unpark integration >> >> >> * Pavel Afremov for fixing exception handling >> >> >> * Boris Kuznetsov for intelligent fixing of security tests >> >> >> * Rana and Salikh for evaluating and discussing problems, reviewing >> >> >> and trying DRLVM patches >> >> >> * Pavel Pervov and Evgueni for help with DRLVM patches >> >> >> * Artem for discovering and fixing weird Windows* behavior >> >> >> * My wife for bringing hot tea to the computer during sleepless >> nights >> >> >> >> >> >> There are still open issues with reliability, multiprocessor and >> other >> >> >> special configurations, so the page >> >> >> http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains >> >> >> active. But this shouldn't prevent us from including class library >> >> >> testing into Harmony "zero regression" policy. What do you think? >> >> >> >> >> >> -- >> >> >> Thank you, >> >> >> Alexei >> >> > >> >> > >> >> > >> >> >> > >> >
Re: [drlvm][unit] 100% of class library tests pass
Mikhail Loenko wrote: > And I hope we will have other workloads running on Harmony nightly and > reporting regressions to the list. Speaking of which, is there a J9 for x86_64 available? If so, I could start getting a more complete testing scenario on the server that runs gump -- Stefano.
Re: [drlvm][unit] 100% of class library tests pass
Tim Ellison wrote: > Pavel Ozhdikhin wrote: >> We have to evolving systems - classlib and DRLVM. To check commits to >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise it's >> impossible to use DRLVM for pre-commit testing - you never know whether >> your >> test fail because of your patch or due to latest changes in DRLVM. >> >> I remember the time when DRLVM and Jitrino actively evolved - for some time >> JIT had to use an older version of DRLVM which could pass all commit >> criteria because newer versions suffered from regressions. And finally we >> came to comon strict commit criteria which prevented regressions in both VM >> and JIT. >> >> To avoid regressions using DRLVM in classlib testing I see 3 possible >> solutions: >> >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this >> version from time to time. >>Pros: + Less time to run DRLVM pre-commit tests >> + Classlib does not suffer from regressions in DRLVM >>Cons: - DRLVM will suffer from regressions >> - Classlib can not use the latest DRLVM >> - Need additional efforts to regain stability on DRLVM >> when we want to update the version for classlib testing >> >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing >> regressions >>Pros: + Less time to run DRLVM pre-commit tests >> + Classlib can use the latest DRLVM >>Cons: - Classlib can suffer from DRLVM regressions (time lag before >> rollback) >> - It is not always clear which commit caused a regression >> - Rollbacks are costly >> >> 3. Add HUT to the commit criteria for DRLVM >> Pros: + Classlib always can use the latest DRLVM >> + DRLVM has no regressions regarding to HUT >> Cons: - More time to run DRLVM pre-commit tests (I was told that HUT >> take 25 minutes running in single JVM mode) >> >> I think that preventing a problem is better than solving it afterwards. So, >> I personally would choose the 3rd approach, don't mind against the second >> and dislike the first one. Probably some combination of these is possible. > > While I appreciate the desire to keep things stable, I think it is > unreasonable to ask developers to run the entire test suite each time. > As we have seen in the classlib code, running targeted tests before > commit and leaving the build machines to run continuous tests ensures > that we are productive and are notified of breakages in time to easily > back out a patch and re-evaluate. > > With the amount of machine time we have running harmony tests on > different cpu's/os's/compilers/etc we are getting better coverage than > any individual could be expected to provide. > > Which is a long way of saying I think option (2) above is best -- and > relies on the bid machines letting us know if things break, and the > commitment from all of us to go straight in and fix it. Agreed, "commit then review" scales better. There is no way a single developer running tests on her/his own machines can know if it's safe to commit a patch... we are not making a release for every commit! I'm not suggesting people get sloppy in committing stuff, but I'm suggesting to be less anal about pre-emptive reviewing and let the testing infrastructure (or our users!) tell you what's wrong. We currently support two OSs (with various versions of distributions, libraries and compilers!) and three CPU architectures, but this is very much likely to grow in the future. And permutations grow quadratically, not linearly! There will be a time in the future when potentially hundreds of machines will run a head-less 'building' cruisecontrol (or equivalent) system that will dump results in real time into a common repository (at harmonytest.org probably) and such repository will perform analysis of the results and find who to blame and send an email to the person with the problem and copying the dev mail list. A developers commit could run the tests fine on their platform and break every other one... their own testing would have been completely useless anyway. -- Stefano.
Re: [drlvm][unit] 100% of class library tests pass
Mikhail Loenko wrote: why not? Because the full-stack testing is appropriate for CI systems that are running full-time to catch bugs. That's what our build-test infrastructure is all about. Asking DRLVM developers (and conversely, classlib developers) to run 1+ hours of tests for even the smallest commits is a waste of time. We simply need to balance efficiency (targeted testing when you make a fix) with the dedication to have a rapid response when the CI systems find a problem. geir 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: Pavel Ozhdikhin wrote: > On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: >> >> Be sure to not miss anyone :) This was a great community effort, with >> everyone pitching in. >> >> DRLVM is now a full peer to J9 in Harmony testing. :) We still need >> to use J9 (and another VM that happens to work with our classlibrary), >> as a sanity check, but we should from now on use DRLVM in our CI testing >> framework. > > > On the other hand, to make sure DRLVM has no regressions regarding > to Classlibrary Unit Tests it makes sense to add these tests to the "test" > target of DRLVM build. > What do people think? Adding classlib unit tests to DRLVM test target? No thanks :) geir > > Thanks, > Pavel > > > >> geir >> >> >> Alexei Fedotov wrote: >> > Oops, I've missed: >> > * Andrew Zhang for reviewing class library patches and helpful >> discussions >> > >> > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: >> >> >> >> Folks, >> >> According to http://harmonytest.org, today 100% of class library unit >> >> tests pass on DRLVM. Thank you all! It takes 44 days for the great >> >> team we are. Thanks for your thoughtful, diligent work and deep >> >> inspiration. Kudos to you for the following (and not limited to this): >> >> >> >> * Alexey Varlamov and Elena for driving the whole process >> >> * Anton and Vladimir Ivanov for automating test runs >> >> * Geir and Gregory for checking and committing related DRLVM patches >> >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and >> >> committing related class library patches >> >> * Alexey Petrenko for becoming ICU expert and writing good JIRA issue >> >> resolution guidelines >> >> * Alexei Zakharov for resolving class library test issues >> >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for >> >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay >> >> Sidelnikov and Alexander Astapchuk for making execution engines work >> >> * Tatiana and Maxim for filing JIRA issues about test failures >> >> * Nikolay Kuznetsov for completing thread interruption handling and >> >> reverting consequences of park/unpark integration >> >> * Pavel Afremov for fixing exception handling >> >> * Boris Kuznetsov for intelligent fixing of security tests >> >> * Rana and Salikh for evaluating and discussing problems, reviewing >> >> and trying DRLVM patches >> >> * Pavel Pervov and Evgueni for help with DRLVM patches >> >> * Artem for discovering and fixing weird Windows* behavior >> >> * My wife for bringing hot tea to the computer during sleepless nights >> >> >> >> There are still open issues with reliability, multiprocessor and other >> >> special configurations, so the page >> >> http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains >> >> active. But this shouldn't prevent us from including class library >> >> testing into Harmony "zero regression" policy. What do you think? >> >> >> >> -- >> >> Thank you, >> >> Alexei >> > >> > >> > >> >
Re: [drlvm][unit] 100% of class library tests pass
16 Nov 2006 17:15:14 +0600, Egor Pasko <[EMAIL PROTECTED]>: On the 0x223 day of Apache Harmony Alexey Varlamov wrote: > 2006/11/16, Tim Ellison <[EMAIL PROTECTED]>: > > Pavel Ozhdikhin wrote: > > > We have to evolving systems - classlib and DRLVM. To check commits to > > > classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise it's > > > impossible to use DRLVM for pre-commit testing - you never know whether > > > your > > > test fail because of your patch or due to latest changes in DRLVM. > > > > > > I remember the time when DRLVM and Jitrino actively evolved - for some time > > > JIT had to use an older version of DRLVM which could pass all commit > > > criteria because newer versions suffered from regressions. And finally we > > > came to comon strict commit criteria which prevented regressions in both VM > > > and JIT. > > > > > > To avoid regressions using DRLVM in classlib testing I see 3 possible > > > solutions: > > > > > > 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this > > > version from time to time. > > >Pros: + Less time to run DRLVM pre-commit tests > > > + Classlib does not suffer from regressions in DRLVM > > >Cons: - DRLVM will suffer from regressions > > > - Classlib can not use the latest DRLVM > > > - Need additional efforts to regain stability on DRLVM > > > when we want to update the version for classlib testing > > > > > > 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing > > > regressions > > >Pros: + Less time to run DRLVM pre-commit tests > > > + Classlib can use the latest DRLVM > > >Cons: - Classlib can suffer from DRLVM regressions (time lag before > > > rollback) > > > - It is not always clear which commit caused a regression > > > - Rollbacks are costly > > > > > > 3. Add HUT to the commit criteria for DRLVM > > > Pros: + Classlib always can use the latest DRLVM > > > + DRLVM has no regressions regarding to HUT > > > Cons: - More time to run DRLVM pre-commit tests (I was told that HUT > > > take 25 minutes running in single JVM mode) > > > > > > I think that preventing a problem is better than solving it afterwards. So, > > > I personally would choose the 3rd approach, don't mind against the second > > > and dislike the first one. Probably some combination of these is possible. > > > > While I appreciate the desire to keep things stable, I think it is > > unreasonable to ask developers to run the entire test suite each time. > > As we have seen in the classlib code, running targeted tests before > > commit and leaving the build machines to run continuous tests ensures > > that we are productive and are notified of breakages in time to easily > > back out a patch and re-evaluate. > > > > With the amount of machine time we have running harmony tests on > > different cpu's/os's/compilers/etc we are getting better coverage than > > any individual could be expected to provide. > > > Which is a long way of saying I think option (2) above is best -- and > > relies on the bid machines letting us know if things break, and the > > commitment from all of us to go straight in and fix it. > > I can't say it better. Thank you Tim :) > Maybe just to reinforce: > 1) We have absolutely stable model VM for classlib verification - j9 > it's name. Therefore I really don't think DRLVM can affect classlib's > progress disruptively. > 2) Yes, there are times when some component advances in leaps as > against accustomed smooth evolution. I do believe we'll be able to > manage such cases individually, w/o overburdening everyday activities. I am for (2) too. +1 And I hope we will have other workloads running on Harmony nightly and reporting regressions to the list. But a small correction: rollback is not always reasonable. We can explicitly agree if we do rollback or fix ASAP (as we did with TM and launcher). +1 Thanks, Mikhail I do not like (1) because interfaces are evolving. And I am not feeling like this process would stop in a short time. (3) is just too long to run as pre-commit, IMHO. Though, we might select some most bug-catching tests from HUT to run as pre-commit. I have nothing concrete to suggest now. We might return to this idea in future, when we have a longer history. -- Egor Pasko
Re: [drlvm][unit] 100% of class library tests pass
Alexei Fedotov wrote: Folks, According to http://harmonytest.org, today 100% of class library unit tests pass on DRLVM. Thank you all! It takes 44 days for the great team we are. Awesome! -- Paulex Yang China Software Development Lab IBM
Re: [drlvm][unit] 100% of class library tests pass
Pavel Ozhdikhin wrote: > Sorry to say but it actually does not work until there is no notifications > to the mailing list and no immediate reaction to the regressions. I agree -- we need to be alerted to failures, and respond to them. Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: [drlvm][unit] 100% of class library tests pass
Egor Pasko wrote: > I am for (2) too. But a small correction: rollback is not always > reasonable. We can explicitly agree if we do rollback or fix ASAP (as > we did with TM and launcher). Of course, thank you -- it is always better to fix it and move forward when that can be done quickly rather than be required to back code out. That is how we have been working to date and it is working well. Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: [drlvm][unit] 100% of class library tests pass
On the 0x223 day of Apache Harmony Alexey Varlamov wrote: > 2006/11/16, Tim Ellison <[EMAIL PROTECTED]>: > > Pavel Ozhdikhin wrote: > > > We have to evolving systems - classlib and DRLVM. To check commits to > > > classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise it's > > > impossible to use DRLVM for pre-commit testing - you never know whether > > > your > > > test fail because of your patch or due to latest changes in DRLVM. > > > > > > I remember the time when DRLVM and Jitrino actively evolved - for some > > > time > > > JIT had to use an older version of DRLVM which could pass all commit > > > criteria because newer versions suffered from regressions. And finally we > > > came to comon strict commit criteria which prevented regressions in both > > > VM > > > and JIT. > > > > > > To avoid regressions using DRLVM in classlib testing I see 3 possible > > > solutions: > > > > > > 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this > > > version from time to time. > > >Pros: + Less time to run DRLVM pre-commit tests > > > + Classlib does not suffer from regressions in DRLVM > > >Cons: - DRLVM will suffer from regressions > > > - Classlib can not use the latest DRLVM > > > - Need additional efforts to regain stability on DRLVM > > > when we want to update the version for classlib testing > > > > > > 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing > > > regressions > > >Pros: + Less time to run DRLVM pre-commit tests > > > + Classlib can use the latest DRLVM > > >Cons: - Classlib can suffer from DRLVM regressions (time lag before > > > rollback) > > > - It is not always clear which commit caused a regression > > > - Rollbacks are costly > > > > > > 3. Add HUT to the commit criteria for DRLVM > > > Pros: + Classlib always can use the latest DRLVM > > > + DRLVM has no regressions regarding to HUT > > > Cons: - More time to run DRLVM pre-commit tests (I was told that HUT > > > take 25 minutes running in single JVM mode) > > > > > > I think that preventing a problem is better than solving it afterwards. > > > So, > > > I personally would choose the 3rd approach, don't mind against the second > > > and dislike the first one. Probably some combination of these is possible. > > > > While I appreciate the desire to keep things stable, I think it is > > unreasonable to ask developers to run the entire test suite each time. > > As we have seen in the classlib code, running targeted tests before > > commit and leaving the build machines to run continuous tests ensures > > that we are productive and are notified of breakages in time to easily > > back out a patch and re-evaluate. > > > > With the amount of machine time we have running harmony tests on > > different cpu's/os's/compilers/etc we are getting better coverage than > > any individual could be expected to provide. > > > Which is a long way of saying I think option (2) above is best -- and > > relies on the bid machines letting us know if things break, and the > > commitment from all of us to go straight in and fix it. > > I can't say it better. Thank you Tim :) > Maybe just to reinforce: > 1) We have absolutely stable model VM for classlib verification - j9 > it's name. Therefore I really don't think DRLVM can affect classlib's > progress disruptively. > 2) Yes, there are times when some component advances in leaps as > against accustomed smooth evolution. I do believe we'll be able to > manage such cases individually, w/o overburdening everyday activities. I am for (2) too. But a small correction: rollback is not always reasonable. We can explicitly agree if we do rollback or fix ASAP (as we did with TM and launcher). I do not like (1) because interfaces are evolving. And I am not feeling like this process would stop in a short time. (3) is just too long to run as pre-commit, IMHO. Though, we might select some most bug-catching tests from HUT to run as pre-commit. I have nothing concrete to suggest now. We might return to this idea in future, when we have a longer history. -- Egor Pasko
Re: [drlvm][unit] 100% of class library tests pass
On 11/16/06, Pavel Ozhdikhin <[EMAIL PROTECTED]> wrote: Sorry to say but it actually does not work until there is no notifications to the mailing list and no immediate reaction to the regressions. Actually, no regression was detected today so no notifications were sent. thanks, Vladimir thanks, Pavel On 11/16/06, Fedotov, Alexei A <[EMAIL PROTECTED]> wrote: > > Pavel, All, > > Let me add one "pro" for the second approach: it works already. > Vladimir's scripts daily upload test results to http://harmonytest.org. > > With best regards, > Alexei Fedotov, > Intel Java & XML Engineering > > >-Original Message- > >From: Tim Ellison [mailto:[EMAIL PROTECTED] > >Sent: Thursday, November 16, 2006 12:37 PM > >To: harmony-dev@incubator.apache.org > >Subject: Re: [drlvm][unit] 100% of class library tests pass > > > >Pavel Ozhdikhin wrote: > >> We have to evolving systems - classlib and DRLVM. To check commits to > >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise > >it's > >> impossible to use DRLVM for pre-commit testing - you never know > whether > >> your > >> test fail because of your patch or due to latest changes in DRLVM. > >> > >> I remember the time when DRLVM and Jitrino actively evolved - for > some > >time > >> JIT had to use an older version of DRLVM which could pass all commit > >> criteria because newer versions suffered from regressions. And > finally we > >> came to comon strict commit criteria which prevented regressions in > both > >VM > >> and JIT. > >> > >> To avoid regressions using DRLVM in classlib testing I see 3 possible > >> solutions: > >> > >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update > this > >> version from time to time. > >>Pros: + Less time to run DRLVM pre-commit tests > >> + Classlib does not suffer from regressions in DRLVM > >>Cons: - DRLVM will suffer from regressions > >> - Classlib can not use the latest DRLVM > >> - Need additional efforts to regain stability on DRLVM > >> when we want to update the version for classlib > testing > >> > >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits > causing > >> regressions > >>Pros: + Less time to run DRLVM pre-commit tests > >> + Classlib can use the latest DRLVM > >>Cons: - Classlib can suffer from DRLVM regressions (time lag > before > >> rollback) > >> - It is not always clear which commit caused a > regression > >> - Rollbacks are costly > >> > >> 3. Add HUT to the commit criteria for DRLVM > >> Pros: + Classlib always can use the latest DRLVM > >> + DRLVM has no regressions regarding to HUT > >> Cons: - More time to run DRLVM pre-commit tests (I was told that > HUT > >> take 25 minutes running in single JVM mode) > >> > >> I think that preventing a problem is better than solving it > afterwards. > >So, > >> I personally would choose the 3rd approach, don't mind against the > second > >> and dislike the first one. Probably some combination of these is > >possible. > > > >While I appreciate the desire to keep things stable, I think it is > >unreasonable to ask developers to run the entire test suite each time. > >As we have seen in the classlib code, running targeted tests before > >commit and leaving the build machines to run continuous tests ensures > >that we are productive and are notified of breakages in time to easily > >back out a patch and re-evaluate. > > > >With the amount of machine time we have running harmony tests on > >different cpu's/os's/compilers/etc we are getting better coverage than > >any individual could be expected to provide. > > > >Which is a long way of saying I think option (2) above is best -- and > >relies on the bid machines letting us know if things break, and the > >commitment from all of us to go straight in and fix it. > > > >Regards, > >Tim > > > >-- > > > >Tim Ellison ([EMAIL PROTECTED]) > >IBM Java technology centre, UK. >
Re: [drlvm][unit] 100% of class library tests pass
Sorry to say but it actually does not work until there is no notifications to the mailing list and no immediate reaction to the regressions. thanks, Pavel On 11/16/06, Fedotov, Alexei A <[EMAIL PROTECTED]> wrote: Pavel, All, Let me add one "pro" for the second approach: it works already. Vladimir's scripts daily upload test results to http://harmonytest.org. With best regards, Alexei Fedotov, Intel Java & XML Engineering >-Original Message- >From: Tim Ellison [mailto:[EMAIL PROTECTED] >Sent: Thursday, November 16, 2006 12:37 PM >To: harmony-dev@incubator.apache.org >Subject: Re: [drlvm][unit] 100% of class library tests pass > >Pavel Ozhdikhin wrote: >> We have to evolving systems - classlib and DRLVM. To check commits to >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise >it's >> impossible to use DRLVM for pre-commit testing - you never know whether >> your >> test fail because of your patch or due to latest changes in DRLVM. >> >> I remember the time when DRLVM and Jitrino actively evolved - for some >time >> JIT had to use an older version of DRLVM which could pass all commit >> criteria because newer versions suffered from regressions. And finally we >> came to comon strict commit criteria which prevented regressions in both >VM >> and JIT. >> >> To avoid regressions using DRLVM in classlib testing I see 3 possible >> solutions: >> >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this >> version from time to time. >>Pros: + Less time to run DRLVM pre-commit tests >> + Classlib does not suffer from regressions in DRLVM >>Cons: - DRLVM will suffer from regressions >> - Classlib can not use the latest DRLVM >> - Need additional efforts to regain stability on DRLVM >> when we want to update the version for classlib testing >> >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing >> regressions >>Pros: + Less time to run DRLVM pre-commit tests >> + Classlib can use the latest DRLVM >>Cons: - Classlib can suffer from DRLVM regressions (time lag before >> rollback) >> - It is not always clear which commit caused a regression >> - Rollbacks are costly >> >> 3. Add HUT to the commit criteria for DRLVM >> Pros: + Classlib always can use the latest DRLVM >> + DRLVM has no regressions regarding to HUT >> Cons: - More time to run DRLVM pre-commit tests (I was told that HUT >> take 25 minutes running in single JVM mode) >> >> I think that preventing a problem is better than solving it afterwards. >So, >> I personally would choose the 3rd approach, don't mind against the second >> and dislike the first one. Probably some combination of these is >possible. > >While I appreciate the desire to keep things stable, I think it is >unreasonable to ask developers to run the entire test suite each time. >As we have seen in the classlib code, running targeted tests before >commit and leaving the build machines to run continuous tests ensures >that we are productive and are notified of breakages in time to easily >back out a patch and re-evaluate. > >With the amount of machine time we have running harmony tests on >different cpu's/os's/compilers/etc we are getting better coverage than >any individual could be expected to provide. > >Which is a long way of saying I think option (2) above is best -- and >relies on the bid machines letting us know if things break, and the >commitment from all of us to go straight in and fix it. > >Regards, >Tim > >-- > >Tim Ellison ([EMAIL PROTECTED]) >IBM Java technology centre, UK.
RE: [drlvm][unit] 100% of class library tests pass
Pavel, All, Let me add one "pro" for the second approach: it works already. Vladimir's scripts daily upload test results to http://harmonytest.org. With best regards, Alexei Fedotov, Intel Java & XML Engineering >-Original Message- >From: Tim Ellison [mailto:[EMAIL PROTECTED] >Sent: Thursday, November 16, 2006 12:37 PM >To: harmony-dev@incubator.apache.org >Subject: Re: [drlvm][unit] 100% of class library tests pass > >Pavel Ozhdikhin wrote: >> We have to evolving systems - classlib and DRLVM. To check commits to >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise >it's >> impossible to use DRLVM for pre-commit testing - you never know whether >> your >> test fail because of your patch or due to latest changes in DRLVM. >> >> I remember the time when DRLVM and Jitrino actively evolved - for some >time >> JIT had to use an older version of DRLVM which could pass all commit >> criteria because newer versions suffered from regressions. And finally we >> came to comon strict commit criteria which prevented regressions in both >VM >> and JIT. >> >> To avoid regressions using DRLVM in classlib testing I see 3 possible >> solutions: >> >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this >> version from time to time. >>Pros: + Less time to run DRLVM pre-commit tests >> + Classlib does not suffer from regressions in DRLVM >>Cons: - DRLVM will suffer from regressions >> - Classlib can not use the latest DRLVM >> - Need additional efforts to regain stability on DRLVM >> when we want to update the version for classlib testing >> >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing >> regressions >>Pros: + Less time to run DRLVM pre-commit tests >> + Classlib can use the latest DRLVM >>Cons: - Classlib can suffer from DRLVM regressions (time lag before >> rollback) >> - It is not always clear which commit caused a regression >> - Rollbacks are costly >> >> 3. Add HUT to the commit criteria for DRLVM >> Pros: + Classlib always can use the latest DRLVM >> + DRLVM has no regressions regarding to HUT >> Cons: - More time to run DRLVM pre-commit tests (I was told that HUT >> take 25 minutes running in single JVM mode) >> >> I think that preventing a problem is better than solving it afterwards. >So, >> I personally would choose the 3rd approach, don't mind against the second >> and dislike the first one. Probably some combination of these is >possible. > >While I appreciate the desire to keep things stable, I think it is >unreasonable to ask developers to run the entire test suite each time. >As we have seen in the classlib code, running targeted tests before >commit and leaving the build machines to run continuous tests ensures >that we are productive and are notified of breakages in time to easily >back out a patch and re-evaluate. > >With the amount of machine time we have running harmony tests on >different cpu's/os's/compilers/etc we are getting better coverage than >any individual could be expected to provide. > >Which is a long way of saying I think option (2) above is best -- and >relies on the bid machines letting us know if things break, and the >commitment from all of us to go straight in and fix it. > >Regards, >Tim > >-- > >Tim Ellison ([EMAIL PROTECTED]) >IBM Java technology centre, UK.
Re: [drlvm][unit] 100% of class library tests pass
2006/11/16, Tim Ellison <[EMAIL PROTECTED]>: Pavel Ozhdikhin wrote: > We have to evolving systems - classlib and DRLVM. To check commits to > classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise it's > impossible to use DRLVM for pre-commit testing - you never know whether > your > test fail because of your patch or due to latest changes in DRLVM. > > I remember the time when DRLVM and Jitrino actively evolved - for some time > JIT had to use an older version of DRLVM which could pass all commit > criteria because newer versions suffered from regressions. And finally we > came to comon strict commit criteria which prevented regressions in both VM > and JIT. > > To avoid regressions using DRLVM in classlib testing I see 3 possible > solutions: > > 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this > version from time to time. >Pros: + Less time to run DRLVM pre-commit tests > + Classlib does not suffer from regressions in DRLVM >Cons: - DRLVM will suffer from regressions > - Classlib can not use the latest DRLVM > - Need additional efforts to regain stability on DRLVM > when we want to update the version for classlib testing > > 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing > regressions >Pros: + Less time to run DRLVM pre-commit tests > + Classlib can use the latest DRLVM >Cons: - Classlib can suffer from DRLVM regressions (time lag before > rollback) > - It is not always clear which commit caused a regression > - Rollbacks are costly > > 3. Add HUT to the commit criteria for DRLVM > Pros: + Classlib always can use the latest DRLVM > + DRLVM has no regressions regarding to HUT > Cons: - More time to run DRLVM pre-commit tests (I was told that HUT > take 25 minutes running in single JVM mode) > > I think that preventing a problem is better than solving it afterwards. So, > I personally would choose the 3rd approach, don't mind against the second > and dislike the first one. Probably some combination of these is possible. While I appreciate the desire to keep things stable, I think it is unreasonable to ask developers to run the entire test suite each time. As we have seen in the classlib code, running targeted tests before commit and leaving the build machines to run continuous tests ensures that we are productive and are notified of breakages in time to easily back out a patch and re-evaluate. With the amount of machine time we have running harmony tests on different cpu's/os's/compilers/etc we are getting better coverage than any individual could be expected to provide. Which is a long way of saying I think option (2) above is best -- and relies on the bid machines letting us know if things break, and the commitment from all of us to go straight in and fix it. I can't say it better. Thank you Tim :) Maybe just to reinforce: 1) We have absolutely stable model VM for classlib verification - j9 it's name. Therefore I really don't think DRLVM can affect classlib's progress disruptively. 2) Yes, there are times when some component advances in leaps as against accustomed smooth evolution. I do believe we'll be able to manage such cases individually, w/o overburdening everyday activities. Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: [drlvm][unit] 100% of class library tests pass
Pavel Ozhdikhin wrote: > We have to evolving systems - classlib and DRLVM. To check commits to > classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise it's > impossible to use DRLVM for pre-commit testing - you never know whether > your > test fail because of your patch or due to latest changes in DRLVM. > > I remember the time when DRLVM and Jitrino actively evolved - for some time > JIT had to use an older version of DRLVM which could pass all commit > criteria because newer versions suffered from regressions. And finally we > came to comon strict commit criteria which prevented regressions in both VM > and JIT. > > To avoid regressions using DRLVM in classlib testing I see 3 possible > solutions: > > 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this > version from time to time. >Pros: + Less time to run DRLVM pre-commit tests > + Classlib does not suffer from regressions in DRLVM >Cons: - DRLVM will suffer from regressions > - Classlib can not use the latest DRLVM > - Need additional efforts to regain stability on DRLVM > when we want to update the version for classlib testing > > 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing > regressions >Pros: + Less time to run DRLVM pre-commit tests > + Classlib can use the latest DRLVM >Cons: - Classlib can suffer from DRLVM regressions (time lag before > rollback) > - It is not always clear which commit caused a regression > - Rollbacks are costly > > 3. Add HUT to the commit criteria for DRLVM > Pros: + Classlib always can use the latest DRLVM > + DRLVM has no regressions regarding to HUT > Cons: - More time to run DRLVM pre-commit tests (I was told that HUT > take 25 minutes running in single JVM mode) > > I think that preventing a problem is better than solving it afterwards. So, > I personally would choose the 3rd approach, don't mind against the second > and dislike the first one. Probably some combination of these is possible. While I appreciate the desire to keep things stable, I think it is unreasonable to ask developers to run the entire test suite each time. As we have seen in the classlib code, running targeted tests before commit and leaving the build machines to run continuous tests ensures that we are productive and are notified of breakages in time to easily back out a patch and re-evaluate. With the amount of machine time we have running harmony tests on different cpu's/os's/compilers/etc we are getting better coverage than any individual could be expected to provide. Which is a long way of saying I think option (2) above is best -- and relies on the bid machines letting us know if things break, and the commitment from all of us to go straight in and fix it. Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: [drlvm][unit] 100% of class library tests pass
Good work everyone! While we may tend to be enticed by the API completeness numbers, it is qualities such as compatibility, stability, and performance that we will ultimately be measured upon and thanked for. It's encouraging to see the team taking all these things so seriously. Regards, Tim Alexei Fedotov wrote: > Folks, > According to http://harmonytest.org, today 100% of class library unit tests > pass on DRLVM. Thank you all! It takes 44 days for the great team we > are. Thanks for your thoughtful, diligent work and deep inspiration. Kudos > to you for the following (and not limited to this): > > * Alexey Varlamov and Elena for driving the whole process > * Anton and Vladimir Ivanov for automating test runs > * Geir and Gregory for checking and committing related DRLVM patches > * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and > committing > related class library patches > * Alexey Petrenko for becoming ICU expert and writing good JIRA issue > resolution guidelines > * Alexei Zakharov for resolving class library test issues * Ilya Okomin, > Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for fixing class library > and tests > * Ivan, Egor, Mikhail Fursov, Nikolay Sidelnikov and Alexander Astapchuk > for > making execution engines work > * Tatiana and Maxim for filing JIRA issues about test failures > * Nikolay Kuznetsov for completing thread interruption handling and > reverting consequences of park/unpark integration > * Pavel Afremov for fixing exception handling > * Boris Kuznetsov for intelligent fixing of security tests > * Rana and Salikh for evaluating and discussing problems, reviewing and > trying DRLVM patches > * Pavel Pervov and Evgueni for help with DRLVM patches > * Artem for discovering and fixing weird Windows* behavior > * My wife for bringing hot tea to the computer during sleepless nights > > There are still open issues with reliability, multiprocessor and other > special configurations, so the page > http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains active. > But > this shouldn't prevent us from including class library testing into Harmony > "zero regression" policy. What do you think? > -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: [drlvm][unit] 100% of class library tests pass
We have to evolving systems - classlib and DRLVM. To check commits to classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise it's impossible to use DRLVM for pre-commit testing - you never know whether your test fail because of your patch or due to latest changes in DRLVM. I remember the time when DRLVM and Jitrino actively evolved - for some time JIT had to use an older version of DRLVM which could pass all commit criteria because newer versions suffered from regressions. And finally we came to comon strict commit criteria which prevented regressions in both VM and JIT. To avoid regressions using DRLVM in classlib testing I see 3 possible solutions: 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this version from time to time. Pros: + Less time to run DRLVM pre-commit tests + Classlib does not suffer from regressions in DRLVM Cons: - DRLVM will suffer from regressions - Classlib can not use the latest DRLVM - Need additional efforts to regain stability on DRLVM when we want to update the version for classlib testing 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing regressions Pros: + Less time to run DRLVM pre-commit tests + Classlib can use the latest DRLVM Cons: - Classlib can suffer from DRLVM regressions (time lag before rollback) - It is not always clear which commit caused a regression - Rollbacks are costly 3. Add HUT to the commit criteria for DRLVM Pros: + Classlib always can use the latest DRLVM + DRLVM has no regressions regarding to HUT Cons: - More time to run DRLVM pre-commit tests (I was told that HUT take 25 minutes running in single JVM mode) I think that preventing a problem is better than solving it afterwards. So, I personally would choose the 3rd approach, don't mind against the second and dislike the first one. Probably some combination of these is possible. thanks, Pavel On 11/16/06, Alexey Varlamov <[EMAIL PROTECTED]> wrote: 2006/11/16, Mikhail Loenko <[EMAIL PROTECTED]>: > why not? But what benefit it would bring? build test in DRLVM takes too much time already, I'm afraid people will just stop using it :( This is analogous to enforcing full testing in classlib for every change regardless of module. Evidently this is too expensive. > > 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: > > > > > > Pavel Ozhdikhin wrote: > > > On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: > > >> > > >> Be sure to not miss anyone :) This was a great community effort, with > > >> everyone pitching in. > > >> > > >> DRLVM is now a full peer to J9 in Harmony testing. :) We still need > > >> to use J9 (and another VM that happens to work with our classlibrary), > > >> as a sanity check, but we should from now on use DRLVM in our CI testing > > >> framework. > > > > > > > > > On the other hand, to make sure DRLVM has no regressions regarding > > > to Classlibrary Unit Tests it makes sense to add these tests to the "test" > > > target of DRLVM build. > > > What do people think? > > > > Adding classlib unit tests to DRLVM test target? No thanks :) > > > > geir > > > > > > > > Thanks, > > > Pavel > > > > > > > > > > > >> geir > > >> > > >> > > >> Alexei Fedotov wrote: > > >> > Oops, I've missed: > > >> > * Andrew Zhang for reviewing class library patches and helpful > > >> discussions > > >> > > > >> > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > > >> >> > > >> >> Folks, > > >> >> According to http://harmonytest.org, today 100% of class library unit > > >> >> tests pass on DRLVM. Thank you all! It takes 44 days for the great > > >> >> team we are. Thanks for your thoughtful, diligent work and deep > > >> >> inspiration. Kudos to you for the following (and not limited to this): > > >> >> > > >> >> * Alexey Varlamov and Elena for driving the whole process > > >> >> * Anton and Vladimir Ivanov for automating test runs > > >> >> * Geir and Gregory for checking and committing related DRLVM patches > > >> >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and > > >> >> committing related class library patches > > >> >> * Alexey Petrenko for becoming ICU expert and writing good JIRA issue > > >> >> resolution guidelines > > >> >> * Alexei Zakharov for resolving class library test issues > > >> >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for > > >> >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay > > >> >> Sidelnikov and Alexander Astapchuk for making execution engines work > > >> >> * Tatiana and Maxim for filing JIRA issues about test failures > > >> >> * Nikolay Kuznetsov for completing thread interruption handling and > > >> >> reverting consequences of park/unpark integration > > >> >> * Pavel Afremov for fixing exception handling > > >> >> * Boris Kuznetsov for intelligent fixing of security tests >
Re: [drlvm][unit] 100% of class library tests pass
2006/11/16, Mikhail Loenko <[EMAIL PROTECTED]>: why not? But what benefit it would bring? build test in DRLVM takes too much time already, I'm afraid people will just stop using it :( This is analogous to enforcing full testing in classlib for every change regardless of module. Evidently this is too expensive. 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: > > > Pavel Ozhdikhin wrote: > > On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: > >> > >> Be sure to not miss anyone :) This was a great community effort, with > >> everyone pitching in. > >> > >> DRLVM is now a full peer to J9 in Harmony testing. :) We still need > >> to use J9 (and another VM that happens to work with our classlibrary), > >> as a sanity check, but we should from now on use DRLVM in our CI testing > >> framework. > > > > > > On the other hand, to make sure DRLVM has no regressions regarding > > to Classlibrary Unit Tests it makes sense to add these tests to the "test" > > target of DRLVM build. > > What do people think? > > Adding classlib unit tests to DRLVM test target? No thanks :) > > geir > > > > > Thanks, > > Pavel > > > > > > > >> geir > >> > >> > >> Alexei Fedotov wrote: > >> > Oops, I've missed: > >> > * Andrew Zhang for reviewing class library patches and helpful > >> discussions > >> > > >> > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > >> >> > >> >> Folks, > >> >> According to http://harmonytest.org, today 100% of class library unit > >> >> tests pass on DRLVM. Thank you all! It takes 44 days for the great > >> >> team we are. Thanks for your thoughtful, diligent work and deep > >> >> inspiration. Kudos to you for the following (and not limited to this): > >> >> > >> >> * Alexey Varlamov and Elena for driving the whole process > >> >> * Anton and Vladimir Ivanov for automating test runs > >> >> * Geir and Gregory for checking and committing related DRLVM patches > >> >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and > >> >> committing related class library patches > >> >> * Alexey Petrenko for becoming ICU expert and writing good JIRA issue > >> >> resolution guidelines > >> >> * Alexei Zakharov for resolving class library test issues > >> >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for > >> >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay > >> >> Sidelnikov and Alexander Astapchuk for making execution engines work > >> >> * Tatiana and Maxim for filing JIRA issues about test failures > >> >> * Nikolay Kuznetsov for completing thread interruption handling and > >> >> reverting consequences of park/unpark integration > >> >> * Pavel Afremov for fixing exception handling > >> >> * Boris Kuznetsov for intelligent fixing of security tests > >> >> * Rana and Salikh for evaluating and discussing problems, reviewing > >> >> and trying DRLVM patches > >> >> * Pavel Pervov and Evgueni for help with DRLVM patches > >> >> * Artem for discovering and fixing weird Windows* behavior > >> >> * My wife for bringing hot tea to the computer during sleepless nights > >> >> > >> >> There are still open issues with reliability, multiprocessor and other > >> >> special configurations, so the page > >> >> http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains > >> >> active. But this shouldn't prevent us from including class library > >> >> testing into Harmony "zero regression" policy. What do you think? > >> >> > >> >> -- > >> >> Thank you, > >> >> Alexei > >> > > >> > > >> > > >> > > >
Re: [drlvm][unit] 100% of class library tests pass
why not? 2006/11/16, Geir Magnusson Jr. <[EMAIL PROTECTED]>: Pavel Ozhdikhin wrote: > On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: >> >> Be sure to not miss anyone :) This was a great community effort, with >> everyone pitching in. >> >> DRLVM is now a full peer to J9 in Harmony testing. :) We still need >> to use J9 (and another VM that happens to work with our classlibrary), >> as a sanity check, but we should from now on use DRLVM in our CI testing >> framework. > > > On the other hand, to make sure DRLVM has no regressions regarding > to Classlibrary Unit Tests it makes sense to add these tests to the "test" > target of DRLVM build. > What do people think? Adding classlib unit tests to DRLVM test target? No thanks :) geir > > Thanks, > Pavel > > > >> geir >> >> >> Alexei Fedotov wrote: >> > Oops, I've missed: >> > * Andrew Zhang for reviewing class library patches and helpful >> discussions >> > >> > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: >> >> >> >> Folks, >> >> According to http://harmonytest.org, today 100% of class library unit >> >> tests pass on DRLVM. Thank you all! It takes 44 days for the great >> >> team we are. Thanks for your thoughtful, diligent work and deep >> >> inspiration. Kudos to you for the following (and not limited to this): >> >> >> >> * Alexey Varlamov and Elena for driving the whole process >> >> * Anton and Vladimir Ivanov for automating test runs >> >> * Geir and Gregory for checking and committing related DRLVM patches >> >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and >> >> committing related class library patches >> >> * Alexey Petrenko for becoming ICU expert and writing good JIRA issue >> >> resolution guidelines >> >> * Alexei Zakharov for resolving class library test issues >> >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for >> >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay >> >> Sidelnikov and Alexander Astapchuk for making execution engines work >> >> * Tatiana and Maxim for filing JIRA issues about test failures >> >> * Nikolay Kuznetsov for completing thread interruption handling and >> >> reverting consequences of park/unpark integration >> >> * Pavel Afremov for fixing exception handling >> >> * Boris Kuznetsov for intelligent fixing of security tests >> >> * Rana and Salikh for evaluating and discussing problems, reviewing >> >> and trying DRLVM patches >> >> * Pavel Pervov and Evgueni for help with DRLVM patches >> >> * Artem for discovering and fixing weird Windows* behavior >> >> * My wife for bringing hot tea to the computer during sleepless nights >> >> >> >> There are still open issues with reliability, multiprocessor and other >> >> special configurations, so the page >> >> http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains >> >> active. But this shouldn't prevent us from including class library >> >> testing into Harmony "zero regression" policy. What do you think? >> >> >> >> -- >> >> Thank you, >> >> Alexei >> > >> > >> > >> >
Re: [drlvm][unit] 100% of class library tests pass
Pavel Ozhdikhin wrote: On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: Be sure to not miss anyone :) This was a great community effort, with everyone pitching in. DRLVM is now a full peer to J9 in Harmony testing. :) We still need to use J9 (and another VM that happens to work with our classlibrary), as a sanity check, but we should from now on use DRLVM in our CI testing framework. On the other hand, to make sure DRLVM has no regressions regarding to Classlibrary Unit Tests it makes sense to add these tests to the "test" target of DRLVM build. What do people think? Adding classlib unit tests to DRLVM test target? No thanks :) geir Thanks, Pavel geir Alexei Fedotov wrote: > Oops, I've missed: > * Andrew Zhang for reviewing class library patches and helpful discussions > > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: >> >> Folks, >> According to http://harmonytest.org, today 100% of class library unit >> tests pass on DRLVM. Thank you all! It takes 44 days for the great >> team we are. Thanks for your thoughtful, diligent work and deep >> inspiration. Kudos to you for the following (and not limited to this): >> >> * Alexey Varlamov and Elena for driving the whole process >> * Anton and Vladimir Ivanov for automating test runs >> * Geir and Gregory for checking and committing related DRLVM patches >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and >> committing related class library patches >> * Alexey Petrenko for becoming ICU expert and writing good JIRA issue >> resolution guidelines >> * Alexei Zakharov for resolving class library test issues >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay >> Sidelnikov and Alexander Astapchuk for making execution engines work >> * Tatiana and Maxim for filing JIRA issues about test failures >> * Nikolay Kuznetsov for completing thread interruption handling and >> reverting consequences of park/unpark integration >> * Pavel Afremov for fixing exception handling >> * Boris Kuznetsov for intelligent fixing of security tests >> * Rana and Salikh for evaluating and discussing problems, reviewing >> and trying DRLVM patches >> * Pavel Pervov and Evgueni for help with DRLVM patches >> * Artem for discovering and fixing weird Windows* behavior >> * My wife for bringing hot tea to the computer during sleepless nights >> >> There are still open issues with reliability, multiprocessor and other >> special configurations, so the page >> http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains >> active. But this shouldn't prevent us from including class library >> testing into Harmony "zero regression" policy. What do you think? >> >> -- >> Thank you, >> Alexei > > >
Re: [drlvm][unit] 100% of class library tests pass
On 11/16/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote: Be sure to not miss anyone :) This was a great community effort, with everyone pitching in. DRLVM is now a full peer to J9 in Harmony testing. :) We still need to use J9 (and another VM that happens to work with our classlibrary), as a sanity check, but we should from now on use DRLVM in our CI testing framework. On the other hand, to make sure DRLVM has no regressions regarding to Classlibrary Unit Tests it makes sense to add these tests to the "test" target of DRLVM build. What do people think? Thanks, Pavel geir Alexei Fedotov wrote: > Oops, I've missed: > * Andrew Zhang for reviewing class library patches and helpful discussions > > On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: >> >> Folks, >> According to http://harmonytest.org, today 100% of class library unit >> tests pass on DRLVM. Thank you all! It takes 44 days for the great >> team we are. Thanks for your thoughtful, diligent work and deep >> inspiration. Kudos to you for the following (and not limited to this): >> >> * Alexey Varlamov and Elena for driving the whole process >> * Anton and Vladimir Ivanov for automating test runs >> * Geir and Gregory for checking and committing related DRLVM patches >> * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and >> committing related class library patches >> * Alexey Petrenko for becoming ICU expert and writing good JIRA issue >> resolution guidelines >> * Alexei Zakharov for resolving class library test issues >> * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for >> fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay >> Sidelnikov and Alexander Astapchuk for making execution engines work >> * Tatiana and Maxim for filing JIRA issues about test failures >> * Nikolay Kuznetsov for completing thread interruption handling and >> reverting consequences of park/unpark integration >> * Pavel Afremov for fixing exception handling >> * Boris Kuznetsov for intelligent fixing of security tests >> * Rana and Salikh for evaluating and discussing problems, reviewing >> and trying DRLVM patches >> * Pavel Pervov and Evgueni for help with DRLVM patches >> * Artem for discovering and fixing weird Windows* behavior >> * My wife for bringing hot tea to the computer during sleepless nights >> >> There are still open issues with reliability, multiprocessor and other >> special configurations, so the page >> http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains >> active. But this shouldn't prevent us from including class library >> testing into Harmony "zero regression" policy. What do you think? >> >> -- >> Thank you, >> Alexei > > >
Re: [drlvm][unit] 100% of class library tests pass
Alexei Fedotov wrote: > Folks, > According to http://harmonytest.org, today 100% of class library unit tests > pass on DRLVM. Thank you all! It takes 44 days for the great team we > are. Thanks for your thoughtful, diligent work and deep inspiration. -- Stefano.
Re: [drlvm][unit] 100% of class library tests pass
Be sure to not miss anyone :) This was a great community effort, with everyone pitching in. DRLVM is now a full peer to J9 in Harmony testing. :) We still need to use J9 (and another VM that happens to work with our classlibrary), as a sanity check, but we should from now on use DRLVM in our CI testing framework. geir Alexei Fedotov wrote: Oops, I've missed: * Andrew Zhang for reviewing class library patches and helpful discussions On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: Folks, According to http://harmonytest.org, today 100% of class library unit tests pass on DRLVM. Thank you all! It takes 44 days for the great team we are. Thanks for your thoughtful, diligent work and deep inspiration. Kudos to you for the following (and not limited to this): * Alexey Varlamov and Elena for driving the whole process * Anton and Vladimir Ivanov for automating test runs * Geir and Gregory for checking and committing related DRLVM patches * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and committing related class library patches * Alexey Petrenko for becoming ICU expert and writing good JIRA issue resolution guidelines * Alexei Zakharov for resolving class library test issues * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay Sidelnikov and Alexander Astapchuk for making execution engines work * Tatiana and Maxim for filing JIRA issues about test failures * Nikolay Kuznetsov for completing thread interruption handling and reverting consequences of park/unpark integration * Pavel Afremov for fixing exception handling * Boris Kuznetsov for intelligent fixing of security tests * Rana and Salikh for evaluating and discussing problems, reviewing and trying DRLVM patches * Pavel Pervov and Evgueni for help with DRLVM patches * Artem for discovering and fixing weird Windows* behavior * My wife for bringing hot tea to the computer during sleepless nights There are still open issues with reliability, multiprocessor and other special configurations, so the page http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains active. But this shouldn't prevent us from including class library testing into Harmony "zero regression" policy. What do you think? -- Thank you, Alexei
Re: [drlvm][unit] 100% of class library tests pass
Alexei Fedotov wrote: Folks, According to http://harmonytest.org, today 100% of class library unit tests pass on DRLVM. Yay! There are still open issues with reliability, multiprocessor and other special configurations, so the page http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains active. But this shouldn't prevent us from including class library testing into Harmony "zero regression" policy. What do you think? +1 geir
Re: [drlvm][unit] 100% of class library tests pass
Oops, I've missed: * Andrew Zhang for reviewing class library patches and helpful discussions On 11/16/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote: Folks, According to http://harmonytest.org, today 100% of class library unit tests pass on DRLVM. Thank you all! It takes 44 days for the great team we are. Thanks for your thoughtful, diligent work and deep inspiration. Kudos to you for the following (and not limited to this): * Alexey Varlamov and Elena for driving the whole process * Anton and Vladimir Ivanov for automating test runs * Geir and Gregory for checking and committing related DRLVM patches * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and committing related class library patches * Alexey Petrenko for becoming ICU expert and writing good JIRA issue resolution guidelines * Alexei Zakharov for resolving class library test issues * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay Sidelnikov and Alexander Astapchuk for making execution engines work * Tatiana and Maxim for filing JIRA issues about test failures * Nikolay Kuznetsov for completing thread interruption handling and reverting consequences of park/unpark integration * Pavel Afremov for fixing exception handling * Boris Kuznetsov for intelligent fixing of security tests * Rana and Salikh for evaluating and discussing problems, reviewing and trying DRLVM patches * Pavel Pervov and Evgueni for help with DRLVM patches * Artem for discovering and fixing weird Windows* behavior * My wife for bringing hot tea to the computer during sleepless nights There are still open issues with reliability, multiprocessor and other special configurations, so the page http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains active. But this shouldn't prevent us from including class library testing into Harmony "zero regression" policy. What do you think? -- Thank you, Alexei -- Thank you, Alexei
