On Fri, Oct 05, 2018 at 01:14:21PM -0700, Clark Boylan wrote: > On Wed, Oct 3, 2018, at 3:16 AM, Whaley, Graham wrote: > > Hi Infra team. > > > > First, a brief overview for folks who have not been in this loop. > > Kata Containers has a CI that runs some metrics, and spits out JSON > > results. We'd like to store those results in the OSF logstash ELK infra > > (http://logstash.openstack.org/#/dashboard/file/logstash.json), and set > > up some Kibana views and dashboards so we can monitor historical trends > > and project health (and longer term maybe some advanced trend regression > > triggers). > > > > There is a relevant github Issue/thread showing a working PoC here: > > https://github.com/kata-containers/ci/issues/60#issuecomment-426579084 > > > > I believe we are at the point where we should set up a trial index and > > keys (for the filebeat/logstash/elastic injection) etc. so we can start > > to tie the parts together. > > What do I need to make that next step happen? Do I need to open a formal > > request ticket/Issue somewhere, or can we just thrash it out here? > > Here and in code review is fine. > > > > > This email is part aimed at ClarkB :-), but obviously may involve others > > etc. Input most welcome. > > Currently all of our jobs submit jobs to the logstash processing system via > the ansible role here [0]. That is then fed through our logstash > configuration [1]. The first step in this is probably to update the logstash > configuration and update the existing Kata jobs in zuul to submit jobs to > index that information as appropriate? > > As for Kibana views I'm not sure we ever sorted that out with the in browser > version of kibana we are running. I think we can embed dashboard information > in the kibana "source" or inject it into elasticsearch? This piece would take > some investigation as I don't know what needs to be done off the top of my > head. Note that our elasticsearch can't be written to via the public > interface to it as there are no authentication and authorization controls for > elasticsearch. > > [0] > https://git.openstack.org/cgit/openstack-infra/project-config/tree/roles/submit-logstash-jobs/defaults/main.yaml > [1] > https://git.openstack.org/cgit/openstack-infra/logstash-filters/tree/filters/openstack-filters.conf > Yah, i think it would be great is we could use the JJB / grafyaml workflow here where we store the view / dashboard in yaml then write a job to push them into the application. Then we don't need to deal with public authentication.
-Paul _______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
