Geir wrote, > I'd prefer if we didn't spam the lists with every good result
Ok, the alternative which comes to my mind next is the following. If I understand correctly, Anton's site gets uploaded zip archives, processes them and populates a database. From the outside we can see result sets for database selections. Anton, If you add a possibility to browse and download uploaded archives, this would help to detect problems from the outside. One can compare uploaded archives and selection results. The only possible problem is if you don't keep archives after they are processed. I don't think we should keep them for long either. I think this worth to be done if you found this useful for debug feature for yourself. With best regards, Alexei Fedotov, Intel Java & XML Engineering >-----Original Message----- >From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED] >Sent: Friday, November 17, 2006 3:05 AM >To: harmony-dev@incubator.apache.org >Subject: Re: [drlvm][unit] 100% of class library tests pass > > > >Alexei Fedotov wrote: >> Pavel, >> The life started showing that you were correct. Today there were no >> report on http://harmonytest.org. Even if I would like to be a living >> notification, I couldn't. >> >> Vladimir, >> The thing which concerns me most is not an absence of results - I >> believe this is just a technical problem. For me the main problem is >> that without you there is no way to understand what happens. > >I don't understand that. > >The goal here is to establish the build-test framework as the thing that >everyone uses- we aren't dependent only upon Vladimir. > >I'll have a version running on 64-bit ubuntu whenever it works, and >probalby 32-bit as well reporting in... > >> >> Can we made a process of reporting more open? For example, can we tune >> CC to send a notification on upload status? If there is a successful >> notification, then the file is uploaded successfully, and we need to >> ask Anton why it is not visible. Otherwise we'll know the problem is >> in CC. > >Um. I'd prefer if we didn't spam the lists with every good result - we >only want to know when there's breakage or "fixage". > >geir > >> >> Thanks! >> >> >> >> On 11/16/06, Pavel Ozhdikhin <[EMAIL PROTECTED]> wrote: >>> Sorry to say but it actually does not work until there is no >>> notifications >>> to the mailing list and no immediate reaction to the regressions. >>> >>> thanks, >>> Pavel >>> >>> >>> On 11/16/06, Fedotov, Alexei A <[EMAIL PROTECTED]> wrote: >>> > >>> > Pavel, All, >>> > >>> > Let me add one "pro" for the second approach: it works already. >>> > Vladimir's scripts daily upload test results to >http://harmonytest.org. >>> > >>> > With best regards, >>> > Alexei Fedotov, >>> > Intel Java & XML Engineering >>> > >>> > >-----Original Message----- >>> > >From: Tim Ellison [mailto:[EMAIL PROTECTED] >>> > >Sent: Thursday, November 16, 2006 12:37 PM >>> > >To: harmony-dev@incubator.apache.org >>> > >Subject: Re: [drlvm][unit] 100% of class library tests pass >>> > > >>> > >Pavel Ozhdikhin wrote: >>> > >> We have to evolving systems - classlib and DRLVM. To check >>> commits to >>> > >> classlib we need a stable DRLVM which can pass 100% of HUT. >>> Otherwise >>> > >it's >>> > >> impossible to use DRLVM for pre-commit testing - you never know >>> > whether >>> > >> your >>> > >> test fail because of your patch or due to latest changes in DRLVM. >>> > >> >>> > >> I remember the time when DRLVM and Jitrino actively evolved - for >>> > some >>> > >time >>> > >> JIT had to use an older version of DRLVM which could pass all >commit >>> > >> criteria because newer versions suffered from regressions. And >>> > finally we >>> > >> came to comon strict commit criteria which prevented regressions in >>> > both >>> > >VM >>> > >> and JIT. >>> > >> >>> > >> To avoid regressions using DRLVM in classlib testing I see 3 >>> possible >>> > >> solutions: >>> > >> >>> > >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update >>> > this >>> > >> version from time to time. >>> > >> Pros: + Less time to run DRLVM pre-commit tests >>> > >> + Classlib does not suffer from regressions in DRLVM >>> > >> Cons: - DRLVM will suffer from regressions >>> > >> - Classlib can not use the latest DRLVM >>> > >> - Need additional efforts to regain stability on >DRLVM >>> > >> when we want to update the version for classlib >>> > testing >>> > >> >>> > >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits >>> > causing >>> > >> regressions >>> > >> Pros: + Less time to run DRLVM pre-commit tests >>> > >> + Classlib can use the latest DRLVM >>> > >> Cons: - Classlib can suffer from DRLVM regressions (time lag >>> > before >>> > >> rollback) >>> > >> - It is not always clear which commit caused a >>> > regression >>> > >> - Rollbacks are costly >>> > >> >>> > >> 3. Add HUT to the commit criteria for DRLVM >>> > >> Pros: + Classlib always can use the latest DRLVM >>> > >> + DRLVM has no regressions regarding to HUT >>> > >> Cons: - More time to run DRLVM pre-commit tests (I was told >that >>> > HUT >>> > >> take 25 minutes running in single JVM mode) >>> > >> >>> > >> I think that preventing a problem is better than solving it >>> > afterwards. >>> > >So, >>> > >> I personally would choose the 3rd approach, don't mind against the >>> > second >>> > >> and dislike the first one. Probably some combination of these is >>> > >possible. >>> > > >>> > >While I appreciate the desire to keep things stable, I think it is >>> > >unreasonable to ask developers to run the entire test suite each >time. >>> > >As we have seen in the classlib code, running targeted tests before >>> > >commit and leaving the build machines to run continuous tests ensures >>> > >that we are productive and are notified of breakages in time to >easily >>> > >back out a patch and re-evaluate. >>> > > >>> > >With the amount of machine time we have running harmony tests on >>> > >different cpu's/os's/compilers/etc we are getting better coverage >than >>> > >any individual could be expected to provide. >>> > > >>> > >Which is a long way of saying I think option (2) above is best -- and >>> > >relies on the bid machines letting us know if things break, and the >>> > >commitment from all of us to go straight in and fix it. >>> > > >>> > >Regards, >>> > >Tim >>> > > >>> > >-- >>> > > >>> > >Tim Ellison ([EMAIL PROTECTED]) >>> > >IBM Java technology centre, UK. >>> > >>> >>> >> >>
