Alexey, You've understood quiet right. The final goal is to point out bad patch, maybe I'll use bisect or other algorithm to find this commit. Anyway I need to collect statistics routinely each day to react fast! It's good idea to use results of cruise control not to do double work.
-----Original Message----- From: Alexey Varlamov [mailto:[EMAIL PROTECTED] Sent: Friday, October 06, 2006 9:32 PM To: harmony-dev@incubator.apache.org Subject: Re: [classlib][test] Site to collect classlib test statistics Vera, I'm not sure I understood your intention. Do you want to monitor regressions routinely a la CruiseControl? Or you need an automated failure analyzer which bisects commits and finds flawed commit? 2006/10/6, Volynets, Vera <[EMAIL PROTECTED]>: > Alexey, > Thank you for your response. > As I understood I'm working on the same problem, > I write scripts to run different tests(cunit, classlib, smoke, kernel, > specjvm98) and collect statistics. > I do it to track patch which was reason of failure of any test. So bug > propagation would be stopped. > The final result is presented like table with name and state > (passed/failed). > My next step was to use mysql database, php and Apache server to make my > system user-friendly. Also I need to think out algorithm to find > "guilty" revision. Now I run tests only on Windows and only on my > machine. > I think it's a good idea to keep in touch and collaborate! > It's useful to have this system on Harmony, > because each contributor can see how his patch effect the virtual > machine!!! > > With best regards, Vera! > > -----Original Message----- > From: Alexey Varlamov [mailto:[EMAIL PROTECTED] > Sent: Friday, October 06, 2006 4:14 PM > To: harmony-dev@incubator.apache.org > Subject: Re: [classlib][test] Site to collect classlib test statistics > > Thanks for prompt response! > Those 554 errors on interpreter are due to fork mode="once", it is a > single error affecting the whole run. Is there a way to delete an > uploaded run? > > Better yet, to be able select a range of runs to include to analysis. > The appetite comes with eating :) > Criteria to select can vary, one of many possible is execution engine > - it would be nice to have such characteristic of a run ... > > > 2006/10/6, Anton Luht <[EMAIL PROTECTED]>: > > Alexey, > > > > I've added both options - search by test name and list of tests that > > ever failed. The second one is very big because of results you've > > uploaded for DRLVM/Linux with 554 errors. Please tell me if you want > > to change anything in the interface. > > > > On 10/6/06, Alexey Varlamov <[EMAIL PROTECTED]> wrote: > > > Anton, > > > > > > Many thanks for exploring this area, it looks quite promising. > > > An idea: this service would become more valuable if presented > > > transparent search/comparison of results for particular tests over > all > > > uploaded runs. > > > Let user enter name of a test and show in one table all known > results > > > for this test, or even better, provide a table of all ever failed > > > tests with links to individual consolidated tables. > > > This would help analyzing failures very much. > > > > > > -- > > > Regards, > > > Alexey > > > > > > 2006/10/4, Anton Luht <[EMAIL PROTECTED]>: > > > > Hello, > > > > > > > > > Could you tell about it in more detail? > > > > > 1) Who can upload test results? > > > > > > > > Everyone - no authorization is needed, you can include file upload > to > > > > your regular builds if you want - authorized users can remove them > > > > later if something goes wrong :) If you want to modify uploaded > tests' > > > > data - add JIRA issues to failures, add comments, etc - please > send me > > > > login/password and I'll create an account. > > > > > > > > > 2) How are you going to develop this site, structure, data base > and > > > > > etc.? > > > > > > > > I'll try to implement all feature requests that community members > will > > > > express. One of possible directions I see is support for new test > > > > suites - currently it recognises only Harmony classlib JUnit > tests > > > > output. If you have another test suite you want to upload to that > site > > > > - just send me an archive and I'll try to add support for this > kind > > > > of tests. > > > > > > > > Sources of that app are not in SVN now - if you want to > > > > review/fix/improve something - I can send you an archive. > > > > -- > > > > Regards, > > > > Anton Luht, > > > > Intel Middleware Products Division > > > > > > > > > --------------------------------------------------------------------- > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > > > > > > > > -- > > Regards, > > Anton Luht, > > Intel Middleware Products Division > > > > --------------------------------------------------------------------- > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
