Hi David, Thanks for your response. ☺
About the build_id, it is same as Jenkins macro, but I have checked the environment variable, I don’t see any variable can be used. Maybe I have missing something? @Julien, can you give some clarification? For the health check and fuel@x86&fuel@aarm64 issue, I think it is not had to implement. I will add them in another patches. We can improve this page step by step. BRs, Jack Chan 发件人: David McBride [mailto:dmcbr...@linuxfoundation.org] 发送时间: 2018年4月26日 11:53 收件人: Chenjiankun (JackChan) 抄送: TECH-DISCUSS OPNFV; tro...@redhat.com; Brattain, Ross B; Rao, Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com; Yuyang (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com); emma.l.fo...@intel.com; Liyin (Ace); Wangwulin (Linda); georg.k...@ericsson.com; Serena Feng; Julien 主题: Re: [opnfv-tech-discuss][test-wg]Requirements for test resources collection Hi Jack, Sorry for the slow response. I've been busy with the Fraser release. Comments 1. I didn't understand point 1 about the build_id. I thought that was the purpose of the Jenkins macro that Julien developed. What am I missing? 2. I'd like to see Health Check broken out as a separate column from Functest. 3. The installer drop-down just lists "Fuel". We need to include both Fuel@x86 and Fuel@aarm64. Thanks for your effort in pushing this forward. David On Sun, Apr 22, 2018 at 8:49 PM, Chenjiankun (JackChan) <chenjiank...@huawei.com<mailto:chenjiank...@huawei.com>> wrote: Hi David, As discussed in the previous Email, I have create a first version of scenario results table(we can call it gating reporting ☺), see: http://116.66.187.136:9998/index.html Patch see: https://gerrit.opnfv.org/gerrit/#/c/56179/ For now, I have added four filters: 1. Scenario 2. Version 3. Installer 4. Iteration(we will query the last 10 days data, the default iteration is 10, if all record is more than 10, it will only 10 record) For now, we still have some problems to solve: 1. build_id(Jenkins id), we don’t have a unified ID to mark the whole Jenkins jobs(deployment + functest + yardstick), for now, I use the build_tag(e.g. jenkins-functest-fuel-baremetal-daily-master-131, the project:functest and installer:fuel are in build_tag) to index, so we can only show the functest result. 2. Jenkins job results, depend on build_id, we can’t show all results(deployment + functest + yardstick, only functest for now). And each projects need to upload their results(UNTRIGGERED, PASS, FAIL). 3. Deployment results, for now, only daisy upload its deployment result, we need ask each installer to upload deployment results, only build_id ready, we can show all results. 4. Statistic results, depend on projects upload their results It is the first version, we will add more functions step by step after the above issue solved. @All, if you have any suggestions, please let me know. BRs, Jack Chan 发件人: chenjiankun 发送时间: 2018年1月22日 17:02 收件人: 'David McBride' 抄送: TECH-DISCUSS OPNFV; tro...@redhat.com<mailto:tro...@redhat.com>; Brattain, Ross B; Rao, Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com<mailto:mark.bei...@dell.com>; Yuyang (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com<mailto:acmor...@att.com>); emma.l.fo...@intel.com<mailto:emma.l.fo...@intel.com>; Liyin (Ace); Wangwulin (Linda); georg.k...@ericsson.com<mailto:georg.k...@ericsson.com>; Serena Feng; Julien 主题: RE: [opnfv-tech-discuss][test-wg]Requirements for test resources collection Thanks, David. According your descriptions, I have created a demo table as below(wish I do not misunderstanding your meaning): scenario date Jenkins Version Installer Deployment Functest yardstick os-nosdn-nofeature-ha 2018-01-21 Jenkins id euphrates compass pass pass pass 2018-01-21 Jenkins id euphrates compass fail not trigger not trigger statistic 8/9/10 (pass:8,triggered:9, total:10) 6/7/8 6/7/8 This last line in table body is the statistics information, and lines above are the detailed information(and it can be folded). The score 8/9/10 is pass/triggered/total. Total means should run, triggered means actually run. Also we can add three filters: If you select installer as compass, then will show all data related to compass. Iterations means last x data points to be displayed. Does this table satisfied your requirements? BRs, Jack Chan 发件人: David McBride [mailto:dmcbr...@linuxfoundation.org] 发送时间: 2018年1月20日 3:07 收件人: chenjiankun 抄送: TECH-DISCUSS OPNFV; tro...@redhat.com<mailto:tro...@redhat.com>; Brattain, Ross B; Rao, Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com<mailto:mark.bei...@dell.com>; Yuyang (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com<mailto:acmor...@att.com>); emma.l.fo...@intel.com<mailto:emma.l.fo...@intel.com>; Liyin (Ace); Wangwulin (Linda); georg.k...@ericsson.com<mailto:georg.k...@ericsson.com>; Serena Feng; Julien 主题: Re: [opnfv-tech-discuss][test-wg]Requirements for test resources collection +Serena, Julien Thanks, Jack. 1. Data reported per scenario (i.e., jenkins job, deployment, functest, yardstick, etc. displayed together for each scenario) instead of separate test silos. 2. Include deployment results 3. Include all Jenkins job results (failure to start, failure to complete, etc.) 4. Clear date/time stamps for every data point 5. Display the data above for the last x data points (e.g., 4, 5, 10 ?) 6. Use an easy-to-understand, unified scoring method for all test frameworks. As I mentioned, yesterday, Julien and Serena have been working on this, as well. Julien has developed a macro<https://gerrit.opnfv.org/gerrit/#/c/48515/> to enable consolidation of all results per scenario. He is intending to use the Daisy installer as a platform to verify the macro, which then can be adapted to other installers. In addition, Serena has agreed to help manage an intern who can assist with the project. I have an action to create an intern proposal for that purpose. David On Fri, Jan 19, 2018 at 1:23 AM, chenjiankun <chenjiank...@huawei.com<mailto:chenjiank...@huawei.com>> wrote: Hi, As we discussed last test working group weekly meeting, we want to do test resources aggregation. We plan to offer a new friendly web portal which contain all existing test resource and more functions. I have a broad classification as bellow: 1. Data analysis a) Reporting(existing, For release) b) Bitergia(existing) c) Grafana(existing, For detailed test results) d) ……(maybe we can develop more tools to show our detailed test results) 2. Test working group information(What information you want to see from test working group? Test working group event? Event of each project?) 3. Tools of each project(Need each project member to complete) 4. ……(waiting for you to improve) This email is aim at collecting requirements for test resources, so if you have any idea about classification, existing tools(such as reporting), new functions you want, please do not hesitate to comment here. As Gabriel said, he will create a new wiki page for test resources collection, so you can also comment there. @David, @Tim, can you repeat your advice about reporting here? I will try my best to implement it. @All, all requirements, advice, comment are welcome~☺ BRs, Jack Chan -- David McBride Release Manager, OPNFV Mobile: +1.805.276.8018<tel:%2B1.805.276.8018> Email/Google Talk: dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org> Skype: davidjmcbride1 IRC: dmcbride -- David McBride Release Manager, OPNFV Mobile: +1.805.276.8018<tel:%2B1.805.276.8018> Email/Google Talk: dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org> Skype: davidjmcbride1 IRC: dmcbride
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss