[jira] [Commented] (SOLR-12052) Schedule a BeastIt unit test report to be published every week.
[ https://issues.apache.org/jira/browse/SOLR-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413997#comment-16413997 ] Mark Miller commented on SOLR-12052: 3rd report is out this morning. I think the email may be caught in moderation or something though. > Schedule a BeastIt unit test report to be published every week. > --- > > Key: SOLR-12052 > URL: https://issues.apache.org/jira/browse/SOLR-12052 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > > This issue builds on SOLR-10032. With that issue I created a tool called > BeastIt to generate beasting test reports and kicked it off a few times > manually, which was simple, but not a job made for man over time. > I took a few months off BeastIt, but just now finished the work I needed to > do to support cron'ing a command that can start up vms, generate a report for > a git repo and branch, and then shut everything down. > This issue will track getting this automated process smooth and work out how > to best advertise the report. > Initially I've setup a report on master to run once a week on Friday night. > We will see how that goes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12052) Schedule a BeastIt unit test report to be published every week.
[ https://issues.apache.org/jira/browse/SOLR-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406549#comment-16406549 ] Mark Miller commented on SOLR-12052: Still working out all the kinks (and BitBalloon automation stopped working out of the blue). I'm sending a report to the mailing list once a week now. Once things are operating relatively smoothly, I expect that report to be delivered to the list Monday mornings. > Schedule a BeastIt unit test report to be published every week. > --- > > Key: SOLR-12052 > URL: https://issues.apache.org/jira/browse/SOLR-12052 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > > This issue builds on SOLR-10032. With that issue I created a tool called > BeastIt to generate beasting test reports and kicked it off a few times > manually, which was simple, but not a job made for man over time. > I took a few months off BeastIt, but just now finished the work I needed to > do to support cron'ing a command that can start up vms, generate a report for > a git repo and branch, and then shut everything down. > This issue will track getting this automated process smooth and work out how > to best advertise the report. > Initially I've setup a report on master to run once a week on Friday night. > We will see how that goes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12052) Schedule a BeastIt unit test report to be published every week.
[
https://issues.apache.org/jira/browse/SOLR-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384821#comment-16384821
]
Mark Miller commented on SOLR-12052:
So just as a start, I setup an email that will initially roughly be like so
(also with a link to the reports):
{noformat}
BeastIt Summary Report for Apache-Solr-Master
Number of Tests: 963
Number Passed: 935
% Passed: 97.09%
Markers
@Ignore: 4
@BadApple: 5
@AwaitsFix: 3
Worst Not Marked Test (It's your Moby Dick!):
* SharedFSAutoReplicaFailoverTest 100 mission-failed '@Nightly '
If you catch that whale, next try:
* TestNonWritablePersistFile 100 mission-failed '@Nightly '
Slowest Test:
* HdfsBasicDistributedZkTest 582.14 passed! '@Nightly ' Can you speed me up?
Worst Marked Test (How long must she suffer?):
* CdcrReplicationDistributedZkTest 121 mission-failed '@Ignore @BadApple
@Nightly Ignored - too heavy'
Non Running Tests - Coverage Holes!
* CdcrReplicationDistributedZkTest '@Ignore @BadApple @Nightly Ignored - too
heavy'
* ChaosMonkeyShardSplitTest '@Ignore '
* TestRankQueryPlugin '@Ignore '
* TestMailEntityProcessor '@Ignore '
{noformat}
> Schedule a BeastIt unit test report to be published every week.
> ---
>
> Key: SOLR-12052
> URL: https://issues.apache.org/jira/browse/SOLR-12052
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
>
> This issue builds on SOLR-10032. With that issue I created a tool called
> BeastIt to generate beasting test reports and kicked it off a few times
> manually, which was simple, but not a job made for man over time.
> I took a few months off BeastIt, but just now finished the work I needed to
> do to support cron'ing a command that can start up vms, generate a report for
> a git repo and branch, and then shut everything down.
> This issue will track getting this automated process smooth and work out how
> to best advertise the report.
> Initially I've setup a report on master to run once a week on Friday night.
> We will see how that goes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12052) Schedule a BeastIt unit test report to be published every week.
[ https://issues.apache.org/jira/browse/SOLR-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384752#comment-16384752 ] Mark Miller commented on SOLR-12052: All tests are always run in a report unless its @ignore'd. > Schedule a BeastIt unit test report to be published every week. > --- > > Key: SOLR-12052 > URL: https://issues.apache.org/jira/browse/SOLR-12052 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > > This issue builds on SOLR-10032. With that issue I created a tool called > BeastIt to generate beasting test reports and kicked it off a few times > manually, which was simple, but not a job made for man over time. > I took a few months off BeastIt, but just now finished the work I needed to > do to support cron'ing a command that can start up vms, generate a report for > a git repo and branch, and then shut everything down. > This issue will track getting this automated process smooth and work out how > to best advertise the report. > Initially I've setup a report on master to run once a week on Friday night. > We will see how that goes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12052) Schedule a BeastIt unit test report to be published every week.
[ https://issues.apache.org/jira/browse/SOLR-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384741#comment-16384741 ] Erick Erickson commented on SOLR-12052: --- I don't know if you've been following SOLR-12028 and all the discussion there, but you might want to enable BadApples on these runs. Short form: I'm trying to get to the point where failures with BadApples=false indicate a regression but still want some visibility for these tests Up to you of course. > Schedule a BeastIt unit test report to be published every week. > --- > > Key: SOLR-12052 > URL: https://issues.apache.org/jira/browse/SOLR-12052 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > > This issue builds on SOLR-10032. With that issue I created a tool called > BeastIt to generate beasting test reports and kicked it off a few times > manually, which was simple, but not a job made for man over time. > I took a few months off BeastIt, but just now finished the work I needed to > do to support cron'ing a command that can start up vms, generate a report for > a git repo and branch, and then shut everything down. > This issue will track getting this automated process smooth and work out how > to best advertise the report. > Initially I've setup a report on master to run once a week on Friday night. > We will see how that goes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12052) Schedule a BeastIt unit test report to be published every week.
[ https://issues.apache.org/jira/browse/SOLR-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384565#comment-16384565 ] Mark Miller commented on SOLR-12052: bq. work out how to best advertise the report. I'm thinking a high level summary with a pointer to the new report in an automated email to the dev list when a report is complete. > Schedule a BeastIt unit test report to be published every week. > --- > > Key: SOLR-12052 > URL: https://issues.apache.org/jira/browse/SOLR-12052 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > > This issue builds on SOLR-10032. With that issue I created a tool called > BeastIt to generate beasting test reports and kicked it off a few times > manually, which was simple, but not a job made for man over time. > I took a few months off BeastIt, but just now finished the work I needed to > do to support cron'ing a command that can start up vms, generate a report for > a git repo and branch, and then shut everything down. > This issue will track getting this automated process smooth and work out how > to best advertise the report. > Initially I've setup a report on master to run once a week on Friday night. > We will see how that goes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
