Hi Jack, I have finished the macro, but it failed for a Jenkins plugin is missing. I cc the issue email to you, which will be discussed in the infra meeting.
BR/Julien David McBride <dmcbr...@linuxfoundation.org>于2018年1月27日周六 上午12:13写道： > +Julien > > Jack, > > I like the table format and the filtering options that you proposed. Also > +1 to Serena's suggestion to break out the health check data. > > I think that Julien is working on a Jenkins macro to enable the test API > to retrieve the "jenkins id". Julien - please confirm. > > David > > On Tue, Jan 23, 2018 at 10:53 PM, chenjiankun <chenjiank...@huawei.com> > wrote: > >> Hi Serena, >> >> >> >> Thanks for your information updating. >> >> I am so glad to see TestAPI is ready to collecting deployment results, >> and it would be much more convenient. >> >> One question: does all installer will push result to DB, or just Daisy >> for now? >> >> >> >> For the Jenkins id, it means a combination of deployment + functest + >> yardstick job. I am not sure if there are a ‘jenkins_id’ for now. This >> field is for identify the combination jobs. >> >> For the second comment, maybe there are some misunderstanding, as David’s >> requirements, we will show data last x iterations(not days), so I think >> there will be no such problems. >> >> For the healthcheck column, I totally agree with you. But maybe it will >> depend on functest to upload test result independently. >> >> >> >> What’s your opinions? @David, @Serena >> >> >> >> BRs, >> >> Jack Chan >> >> >> >> *发件人:* SerenaFeng(zte) [mailto:serena.feng.711+...@gmail.com] >> *发送时间:* 2018年1月22日 18:08 >> *收件人:* chenjiankun >> *抄送:* David McBride; Serena Feng; TECH-DISCUSS OPNFV >> *主题:* Re: [opnfv-tech-discuss] [test-wg]Requirements for test resources >> collection >> >> >> >> Hi David & Jack, >> >> >> >> Thanks for mentioning the task, an interface for collecting deployment >> results has been ready in TestAPI[1], >> >> the Macro for pushing deployment result is also ready by Jelien[2], I >> believe it will facilitate all the installers' work, >> >> currently, Julien is working on pushing daisy results to TestAPI >> leveraging that Macro, I think it will be finished soon. >> >> >> >> And regarding how to show all the information in a table, I suggest we >> can take a look at Jack's proposal first. >> >> >> >> @Jack, a few comments: >> >> 1. what's the opinion of jenkins id? >> >> 2. for a scenario-installer combination, some will not run once in a >> day(trigger multiple times or run in multiple pods), >> >> in this case, a simple pass/fail will be too vague, and to facilitate >> the support of data iteration, I would suggest >> >> leveraging 8/9/10(8 passed, 9 triggered, 10total), delete the >> final statistic line >> >> 3. how about adding a healthcheck column(functest-healthcheck test >> cases), to see if the installer meets the milestone 3.0 >> >> >> >> >> >> [1]: https://gerrit.opnfv.org/gerrit/#/c/49895/ >> >> [2]: https://gerrit.opnfv.org/gerrit/#/c/48515/ >> >> >> >> BRs >> >> Serena >> >> >> >> On Mon, Jan 22, 2018 at 5:02 PM chenjiankun <chenjiank...@huawei.com> >> wrote: >> >> Thanks, David. >> >> >> >> According your descriptions, I have created a demo table as below(wish I >> do not misunderstanding your meaning): >> >> >> >> *scenario * >> >> *date * >> >> *Jenkins * >> >> *Version * >> >> *Installer * >> >> *Deployment * >> >> *Functest * >> >> *yardstick * >> >> os-nosdn-nofeature-ha >> >> 2018-01-21 >> >> Jenkins id >> >> euphrates >> >> compass >> >> pass >> >> pass >> >> pass >> >> 2018-01-21 >> >> Jenkins id >> >> euphrates >> >> compass >> >> fail >> >> not trigger >> >> not trigger >> >> statistic >> >> 8/9/10 >> >> (pass:8,triggered:9, total:10) >> >> 6/7/8 >> >> 6/7/8 >> >> >> >> >> >> This last line in table body is the statistics information, and lines >> above are the detailed information(and it can be folded). >> >> The score 8/9/10 is pass/triggered/total. Total means should run, >> triggered means actually run. >> >> Also we can add three filters: >> >> >> >> If you select installer as compass, then will show all data related to >> compass. >> >> Iterations means last x data points to be displayed. >> >> >> >> Does this table satisfied your requirements? >> >> >> >> BRs, >> >> Jack Chan >> >> *发件人:* David McBride [mailto:dmcbr...@linuxfoundation.org] >> *发送时间:* 2018年1月20日 3:07 >> *收件人:* chenjiankun >> *抄送:* TECH-DISCUSS OPNFV; tro...@redhat.com; Brattain, Ross B; Rao, >> Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com; Yuyang >> (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com); emma.l.fo...@intel.com; >> Liyin (Ace); Wangwulin (Linda); georg.k...@ericsson.com; Serena Feng; >> Julien >> *主题:* Re: [opnfv-tech-discuss][test-wg]Requirements for test resources >> collection >> >> >> >> +Serena, Julien >> >> >> >> Thanks, Jack. >> >> 1. Data reported per scenario (i.e., jenkins job, deployment, >> functest, yardstick, etc. displayed together for each scenario) instead of >> separate test silos. >> 2. Include deployment results >> 3. Include all Jenkins job results (failure to start, failure to >> complete, etc.) >> 4. Clear date/time stamps for every data point >> 5. Display the data above for the last x data points (e.g., 4, 5, 10 >> ?) >> 6. Use an easy-to-understand, unified scoring method for all test >> frameworks. >> >> As I mentioned, yesterday, Julien and Serena have been working on this, >> as well. Julien has developed a macro >> <https://gerrit.opnfv.org/gerrit/#/c/48515/> to enable consolidation of >> all results per scenario. He is intending to use the Daisy installer as a >> platform to verify the macro, which then can be adapted to other installers. >> >> >> >> In addition, Serena has agreed to help manage an intern who can assist >> with the project. I have an action to create an intern proposal for that >> purpose. >> >> >> >> David >> >> >> >> On Fri, Jan 19, 2018 at 1:23 AM, chenjiankun <chenjiank...@huawei.com> >> wrote: >> >> Hi, >> >> >> >> As we discussed last test working group weekly meeting, we want to do >> test resources aggregation. >> >> We plan to offer a new friendly web portal which contain all existing >> test resource and more functions. >> >> >> >> I have a broad classification as bellow: >> >> 1. Data analysis >> >> a) Reporting(existing, For release) >> >> b) Bitergia(existing) >> >> c) Grafana(existing, For detailed test results) >> >> d) ……(maybe we can develop more tools to show our detailed test >> results) >> >> 2. Test working group information(What information you want to see >> from test working group? Test working group event? Event of each project?) >> >> 3. Tools of each project(Need each project member to complete) >> >> 4. ……(waiting for you to improve) >> >> >> >> >> >> This email is aim at collecting requirements for test resources, so if >> you have any idea about classification, existing tools(such as reporting), >> new functions you want, please do not hesitate to comment here. >> >> As Gabriel said, he will create a new wiki page for test resources >> collection, so you can also comment there. >> >> >> >> @David, @Tim, can you repeat your advice about reporting here? I will try >> my best to implement it. >> >> @All, all requirements, advice, comment are welcome~J >> >> >> >> BRs, >> >> Jack Chan >> >> >> >> >> >> -- >> >> *David McBride* >> >> Release Manager, OPNFV >> >> Mobile: +1.805.276.8018 >> >> Email/Google Talk: dmcbr...@linuxfoundation.org >> >> Skype: davidjmcbride1 >> >> IRC: dmcbride >> >> _______________________________________________ >> opnfv-tech-discuss mailing list >> opnfv-tech-discuss@lists.opnfv.org >> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >> >> > > > -- > *David McBride* > Release Manager, OPNFV > Mobile: +1.805.276.8018 > Email/Google Talk: dmcbr...@linuxfoundation.org > Skype: davidjmcbride1 > IRC: dmcbride >
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
