On 06/29/2014 03:25 PM, Ilya Shakhat wrote: > Hi! > > During last couple weeks there is an increasing demand on tracking > 3rd-party CI statuses. We at Stackalytics decided to be in trend and (with > some inspiration from Salvatore's proposal) implemented report that shows > summary on external CI status. The initial version is available for Neutron > - http://stackalytics.com/report/ci/neutron/7 > > The report shows summary of all CI jobs during specified period of time, > including: > * stats of runs on merged patch sets: > - total number of runs > - success rate (success to total ratio) > - time of the latest run > - last test result > * stats for all patch sets (the same set as for merged) > * last test results for every merged patch set grouped by days (useful to > see how different CIs correlate with each other and how often they run) > > Under "merged patch set" report means "the last patch set in the merged > change request", thus it is almost the same as the trunk code. CI > configuration is taken from DriverLog's default data > <https://git.openstack.org/cgit/stackforge/driverlog/tree/etc/default_data.json>. > Standard Stackalytics screen is also available for CIs - > http://stackalytics.com/?metric=ci, including votes breakdown and activity > log. > > Since this is the first version there are some open questions: > * Currently report shows results per CI id, but there are CIs that run > tests against multiple drivers and this case is not supported. What would > be more useful: to get stats for a driver or for CI? > * Most CIs run tests when patch set is posted. So even if change request > is merged within selected time period corresponding CI results may be > missing. > * Patterns for non-voting CIs need to be verified. For example Cisco CI > now runs 5 jobs, but DriverLog data still contains old data. > > Thanks, > Ilya > > 2014-06-16 17:52 GMT+04:00 Salvatore Orlando <sorla...@nicira.com>: > >> >> However, it would be great if we could start devising a solution for >> having "health" reports from the various CI systems. >> This report should report the following kind of information: >> - timestamp of last run >> - timestamp of last vote (a system might start job which then get aborted >> for CI infra problems) >> - % of success vs failures (not sure how important is that one but >> provides a metric of comparison with upstream jenkins) >> - % of disagreements with jenkins (this might allow us to quickly spot >> those CI systems which are randomly -1'ing patches) >> >> The percentage metrics might be taken over a 48 hours or 7 days interval, >> or both. >> Does this idea sound reasonable? >> >> > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > Hi Ilya:
I look forward to hearing more about this dashboard and ensuring you or someone else associated with this dashboard are available for questions at the third party meeting tomorrow: https://wiki.openstack.org/wiki/Meetings/ThirdParty We missed you last week. Thanks Ilya, Anita. _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev