Hi Jack Eventually, we hope all the installers push their deployment results to TestAPI, in the current stage, we just leverage Daisy to have a try and find a common framework to achieve that, we will promote it to other installers soon after it works.
In terms of Jenkins id, almost all the installers leverage muti-job to manage deployment and drive test projects. there is an entry job to control all the jobs, such as daisy-os-nosdn-nofeature-ha-baremetal-daily-master <https://build.opnfv.org/ci/job/daisy-os-nosdn-nofeature-ha-baremetal-daily-master/>, maybe we can leverage the combination of the Jenkins job and build_id daisy-os-nosdn-nofeature-ha-baremetal-daily-master_185 <https://build.opnfv.org/ci/job/daisy-os-nosdn-nofeature-ha-baremetal-daily-master/185/>(or jest the Jenkins job) to link to the Jenkins. As for naming, instead of 'Jenkins' I would recommend using 'Job_name', because, in zuulv3, Jenkins will be discarded, but the job idea will be left(maybe in some other way) For the status show, if only show the last and latest *one* iteration, simple pass/fail is fine, or else multi-results will be included, I don't recommend that. Personally, my suggestion is either integrate last one iteration into last x iterations, all use statistics to show the results, or separate into two pages: the current status(simple pass/fail) and the history statistic status(8/9/10). For Functest health check, Sure, Functest reports per execution result per test case separately, we can get them from TestAPI, there's no problem with that. But for health check actually includes 3 test cases(connection_check/api_check/snaps_health_check), we need to cope with that carefully. BRs Serena On Sat, Jan 27, 2018 at 11:42 PM Julien <julien...@gmail.com> wrote: > Hi Jack, > > I have finished the macro, but it failed for a Jenkins plugin is missing. > I cc the issue email to you, which will be discussed in the infra meeting. > > BR/Julien > > David McBride <dmcbr...@linuxfoundation.org>于2018年1月27日周六 上午12:13写道： > >> +Julien >> >> Jack, >> >> I like the table format and the filtering options that you proposed. >> Also +1 to Serena's suggestion to break out the health check data. >> >> I think that Julien is working on a Jenkins macro to enable the test API >> to retrieve the "jenkins id". Julien - please confirm. >> >> David >> >> On Tue, Jan 23, 2018 at 10:53 PM, chenjiankun <chenjiank...@huawei.com> >> wrote: >> >>> Hi Serena, >>> >>> >>> >>> Thanks for your information updating. >>> >>> I am so glad to see TestAPI is ready to collecting deployment results, >>> and it would be much more convenient. >>> >>> One question: does all installer will push result to DB, or just Daisy >>> for now? >>> >>> >>> >>> For the Jenkins id, it means a combination of deployment + functest + >>> yardstick job. I am not sure if there are a ‘jenkins_id’ for now. This >>> field is for identify the combination jobs. >>> >>> For the second comment, maybe there are some misunderstanding, as >>> David’s requirements, we will show data last x iterations(not days), so I >>> think there will be no such problems. >>> >>> For the healthcheck column, I totally agree with you. But maybe it will >>> depend on functest to upload test result independently. >>> >>> >>> >>> What’s your opinions? @David, @Serena >>> >>> >>> >>> BRs, >>> >>> Jack Chan >>> >>> >>> >>> *发件人:* SerenaFeng(zte) [mailto:serena.feng.711+...@gmail.com] >>> *发送时间:* 2018年1月22日 18:08 >>> *收件人:* chenjiankun >>> *抄送:* David McBride; Serena Feng; TECH-DISCUSS OPNFV >>> *主题:* Re: [opnfv-tech-discuss] [test-wg]Requirements for test resources >>> collection >>> >>> >>> >>> Hi David & Jack, >>> >>> >>> >>> Thanks for mentioning the task, an interface for collecting deployment >>> results has been ready in TestAPI[1], >>> >>> the Macro for pushing deployment result is also ready by Jelien[2], I >>> believe it will facilitate all the installers' work, >>> >>> currently, Julien is working on pushing daisy results to TestAPI >>> leveraging that Macro, I think it will be finished soon. >>> >>> >>> >>> And regarding how to show all the information in a table, I suggest we >>> can take a look at Jack's proposal first. >>> >>> >>> >>> @Jack, a few comments: >>> >>> 1. what's the opinion of jenkins id? >>> >>> 2. for a scenario-installer combination, some will not run once in a >>> day(trigger multiple times or run in multiple pods), >>> >>> in this case, a simple pass/fail will be too vague, and to >>> facilitate the support of data iteration, I would suggest >>> >>> leveraging 8/9/10(8 passed, 9 triggered, 10total), delete the >>> final statistic line >>> >>> 3. how about adding a healthcheck column(functest-healthcheck test >>> cases), to see if the installer meets the milestone 3.0 >>> >>> >>> >>> >>> >>> [1]: https://gerrit.opnfv.org/gerrit/#/c/49895/ >>> >>> [2]: https://gerrit.opnfv.org/gerrit/#/c/48515/ >>> >>> >>> >>> BRs >>> >>> Serena >>> >>> >>> >>> On Mon, Jan 22, 2018 at 5:02 PM chenjiankun <chenjiank...@huawei.com> >>> wrote: >>> >>> Thanks, David. >>> >>> >>> >>> According your descriptions, I have created a demo table as below(wish I >>> do not misunderstanding your meaning): >>> >>> >>> >>> *scenario * >>> >>> *date * >>> >>> *Jenkins * >>> >>> *Version * >>> >>> *Installer * >>> >>> *Deployment * >>> >>> *Functest * >>> >>> *yardstick * >>> >>> os-nosdn-nofeature-ha >>> >>> 2018-01-21 >>> >>> Jenkins id >>> >>> euphrates >>> >>> compass >>> >>> pass >>> >>> pass >>> >>> pass >>> >>> 2018-01-21 >>> >>> Jenkins id >>> >>> euphrates >>> >>> compass >>> >>> fail >>> >>> not trigger >>> >>> not trigger >>> >>> statistic >>> >>> 8/9/10 >>> >>> (pass:8,triggered:9, total:10) >>> >>> 6/7/8 >>> >>> 6/7/8 >>> >>> >>> >>> >>> >>> This last line in table body is the statistics information, and lines >>> above are the detailed information(and it can be folded). >>> >>> The score 8/9/10 is pass/triggered/total. Total means should run, >>> triggered means actually run. >>> >>> Also we can add three filters: >>> >>> >>> >>> If you select installer as compass, then will show all data related to >>> compass. >>> >>> Iterations means last x data points to be displayed. >>> >>> >>> >>> Does this table satisfied your requirements? >>> >>> >>> >>> BRs, >>> >>> Jack Chan >>> >>> *发件人:* David McBride [mailto:dmcbr...@linuxfoundation.org] >>> *发送时间:* 2018年1月20日 3:07 >>> *收件人:* chenjiankun >>> *抄送:* TECH-DISCUSS OPNFV; tro...@redhat.com; Brattain, Ross B; Rao, >>> Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com; Yuyang >>> (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com); emma.l.fo...@intel.com; >>> Liyin (Ace); Wangwulin (Linda); georg.k...@ericsson.com; Serena Feng; >>> Julien >>> *主题:* Re: [opnfv-tech-discuss][test-wg]Requirements for test resources >>> collection >>> >>> >>> >>> +Serena, Julien >>> >>> >>> >>> Thanks, Jack. >>> >>> 1. Data reported per scenario (i.e., jenkins job, deployment, >>> functest, yardstick, etc. displayed together for each scenario) instead >>> of >>> separate test silos. >>> 2. Include deployment results >>> 3. Include all Jenkins job results (failure to start, failure to >>> complete, etc.) >>> 4. Clear date/time stamps for every data point >>> 5. Display the data above for the last x data points (e.g., 4, 5, 10 >>> ?) >>> 6. Use an easy-to-understand, unified scoring method for all test >>> frameworks. >>> >>> As I mentioned, yesterday, Julien and Serena have been working on this, >>> as well. Julien has developed a macro >>> <https://gerrit.opnfv.org/gerrit/#/c/48515/> to enable consolidation of >>> all results per scenario. He is intending to use the Daisy installer as a >>> platform to verify the macro, which then can be adapted to other installers. >>> >>> >>> >>> In addition, Serena has agreed to help manage an intern who can assist >>> with the project. I have an action to create an intern proposal for that >>> purpose. >>> >>> >>> >>> David >>> >>> >>> >>> On Fri, Jan 19, 2018 at 1:23 AM, chenjiankun <chenjiank...@huawei.com> >>> wrote: >>> >>> Hi, >>> >>> >>> >>> As we discussed last test working group weekly meeting, we want to do >>> test resources aggregation. >>> >>> We plan to offer a new friendly web portal which contain all existing >>> test resource and more functions. >>> >>> >>> >>> I have a broad classification as bellow: >>> >>> 1. Data analysis >>> >>> a) Reporting(existing, For release) >>> >>> b) Bitergia(existing) >>> >>> c) Grafana(existing, For detailed test results) >>> >>> d) ……(maybe we can develop more tools to show our detailed test >>> results) >>> >>> 2. Test working group information(What information you want to >>> see from test working group? Test working group event? Event of each >>> project?) >>> >>> 3. Tools of each project(Need each project member to complete) >>> >>> 4. ……(waiting for you to improve) >>> >>> >>> >>> >>> >>> This email is aim at collecting requirements for test resources, so if >>> you have any idea about classification, existing tools(such as reporting), >>> new functions you want, please do not hesitate to comment here. >>> >>> As Gabriel said, he will create a new wiki page for test resources >>> collection, so you can also comment there. >>> >>> >>> >>> @David, @Tim, can you repeat your advice about reporting here? I will >>> try my best to implement it. >>> >>> @All, all requirements, advice, comment are welcome~J >>> >>> >>> >>> BRs, >>> >>> Jack Chan >>> >>> >>> >>> >>> >>> -- >>> >>> *David McBride* >>> >>> Release Manager, OPNFV >>> >>> Mobile: +1.805.276.8018 >>> >>> Email/Google Talk: dmcbr...@linuxfoundation.org >>> >>> Skype: davidjmcbride1 >>> >>> IRC: dmcbride >>> >>> _______________________________________________ >>> opnfv-tech-discuss mailing list >>> opnfv-tech-discuss@lists.opnfv.org >>> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >>> >>> >> >> >> -- >> *David McBride* >> Release Manager, OPNFV >> Mobile: +1.805.276.8018 >> Email/Google Talk: dmcbr...@linuxfoundation.org >> Skype: davidjmcbride1 >> IRC: dmcbride >> >
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
