[OpenStack-Infra] Stepping down from core

2020-05-04 Thread David Moreau Simard
Hi everyone,

After taking some time to rest, I came to the simple (yet difficult)
realization that despite my best efforts, I can't do everything that
I'd like to do and so I'll be stepping down from infra-core.

I've somehow ended up with too many professional and familial
responsibilities on my shoulders at once and I need to take a step
back to better understand what it is that I can do without burning
myself out.

Although this week will be my last at Red Hat, I will remain involved
with OpenStack in my next adventure and I plan to help where I can as
an individual contributor.

As for ARA, I'll continue contributing to the project because it's
useful for a lot of people and I think it's fun. Thank you for hosting
it !

I've really appreciated having the opportunity to work with you and I
hope we get to collaborate again in the future.

See you around,

David Moreau Simard
dmsimard = [irc, github, twitter]


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] ARA 1.0 deployment plans

2019-07-15 Thread David Moreau Simard
On Tue, Jun 11, 2019 at 7:45 PM James E. Blair  wrote:
> Given that our direction is to remove logs.o.o from the system along
> with all of its proxy services and only serve static files from swift, I
> think we will have to continue to use the old version of ARA until the
> story with static generation is worked out for 1.0.

We're currently experimenting with a proof of concept [1] to generate
static reports in 1.x -- you can see what it looks like in the
uploaded logs here [2].

In a nutshell, it uses the offline API client to retrieve data and
then it's rendered using simple html templates.
No added dependencies, no javascript (yet?) and the only css is
included from a CDN.

The challenge with static reports remains the same, though: if you run
100 tasks against 10 hosts, it's 1000 task results.
If we keep results in their own pages to keep the playbook pages
lightweight, we end up with 1000+ small files.
Inlcuding those task results in the same pages as the playbooks
reduces the amount of files to generate but it also results in heavier
pages to load in users' browsers.

Another option is to have arguments or parameters to omit data from
the report which would result in less bloat.
For example, there could be a flag to omit (detailed) results for "OK"
and "SKIPPED" tasks or there could be a flag to only include failed
playbooks in the report. Doing something like this isn't complicated
since the API already supports filtering by playbook and task status.
I'll think about it a little bit but I'm open to suggestions if anyone
has an idea.

In the meantime, we've landed an implementation of the sqlite
middleware for the API [3].
I know it wouldn't work with Swift (unless we use something like s3ql
[4]?) but I think that an API to browse and query any job's results
could be useful.

[1]: https://review.opendev.org/#/c/670597/
[2]: 
http://logs.openstack.org/97/670597/7/check/ansible-role-ara-api-fedora-devel/1dcf203/logs/static/
[3]: https://ara.readthedocs.io/en/latest/distributed-sqlite-backend.html
[4]: https://github.com/s3ql/s3ql

David Moreau Simard
dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] ARA 1.0 deployment plans

2019-06-11 Thread David Moreau Simard
Thanks for starting this thread Ian.

On Tue, Jun 11, 2019 at 4:09 AM Ian Wienand  wrote:
> It seems ARA 1.0 has moved in some directions we're not handling right
> now.  Playing with [1] I've got ARA generating and uploading it's
> database.

Patch looks good to me.
Thanks for playing with it :)

> Currently, Apache matches an ara-report/ directory on
> logs.openstack.org and sent to the ARA wsgi application which serves
> the response from the sqlite db in that directory [2].

Correct.

> If I'm understanding, we now need ara-web [3] to display the report
> page we all enjoy.  However this web app currently only gets data from
> an ARA server instance that provides a REST interface with the info?

Correct.

> I'm not really seeing how this fits with the current middleware
> deployment? (unfortunately [4] or an analogue in the new release seems
> to have disappeared).  Do we now host a separate ARA server on
> logs.openstack.org on some known port that knows how to turn
> /*/ara-report/ URL requests into access of the .sqlite db on disk and
> thus provide the REST interface?

The sqlite middleware doesn't have an equivalent in 1.0 right now.

Although it was first implemented as somewhat of a hack to address the
lack of scalability of HTML generation, I've gotten to like the design
principle of isolating a job's result in a single database.

It easy to scale and keeps latency to a minimum compared to a central
database server.

I'm convinced that implementing a similar approach for 1.0 would make
sense. I would happily accept any input here.

> And then somehow we host a ara-web instance that knows how to
> request from this ?

Right now the API server is defined by the configuration [1].
We would need to implement a more "dynamic" way of being able to
specify an API endpoint to query.

For example, we recently added support for ara-web to prompt a login
page if the API server requires authentication. The credentials are stored
locally and then used to authenticate queries against the API.

It might not be too much of a stretch to implement a way to store the
API endpoint locally like we do for credentials.

We wouldn't want the user(s) to type in the API endpoint every time, though.
There is a little thinking to do about how to glue the things together.

The combination of the rewrite rule, the wsgi middleware and the fact that
0.x was a monolithic app definitely made this easier.

> Given I can't see us wanting to do a bunch of puppet hacking to get
> new services on logs.openstack.org, but yet also it requiring fairly
> non-trivial effort to get the extant bits and pieces on that server
> migrated to an all-Ansible environment, I think we have to give some
> thought as to how we'll roll this out (plus add in containers,
> possible logs on swift, etc ... for extra complexity :)
>
> So does anyone have thoughts on a high-level view of how this might
> hang together?

For now I've created an etherpad [2] as a starting point to summarize
some of what has been written here as well as other points which are perhaps
more Zuul-specific.

[1]: https://github.com/ansible-community/ara-web/blob/master/public/config.json
[2]: https://etherpad.openstack.org/p/ara-1.0-in-zuul

David Moreau Simard
dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Move packstack from openstack to redhat-openstack organization in github

2019-05-07 Thread David Moreau Simard
Hi,

github.com/openstack/packstack was moved successfully to
github.com/redhat-openstack/packstack.

Note that the replication needs to be set up to keep the repository up
to date. This can be done by inheriting from the upload-git-mirror [1]
job.

[1]: 
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005007.html

David Moreau Simard
dmsimard = [irc, github, twitter]

On Wed, May 1, 2019 at 1:17 PM Clark Boylan  wrote:
>
> On Tue, Apr 30, 2019, at 7:51 AM, Alfredo Moralejo Alonso wrote:
> > Hi,
> >
> > After migration to opendev an moving openstack/packstack to
> > x/packstack, mirroring to github has been lost as announced in [1].
> >
> > Could we get the repository moved to redhat-openstack organization in
> > github?, we'd like to get redirection from old repo to the new one
> > what's require to transfer the repo using github iiuc.
> >
> > Best regards,
> >
> > Alfredo
> >
> > [1]
> > http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005629.html
> >
>
> Thank you for sending this out so that we have a record of why the move was 
> made. I have no objections to making this move in GitHub. If dmsimard still 
> has time to do the move I think we can go ahead with that. Otherwise, we'll 
> likely get to it once we return to our daily lives after the summit and PTG.
>
> Clark
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Selecting New Priority Effort(s)

2018-04-03 Thread David Moreau Simard
It won't be very exciting but we really need to do one of the
following two things soon:

1) Ansiblify control plane [1]
2) Update our puppet things to puppet 4 (or 5?)

Puppet 3 has been end of life since Dec 31, 2016. [2]

The longer we draw this out, the more work it'll be :(

[1]: https://review.openstack.org/#/c/469983/
[2]: https://groups.google.com/forum/#!topic/puppet-users/IdutL5FTW7w


David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]


On Tue, Apr 3, 2018 at 4:23 PM, Clark Boylan <cboy...@sapwetik.org> wrote:
> Hello everyone,
>
> I just approved the change to mark the Zuul v3 priority effort as completed 
> in the infra-specs repo. Thank you to everyone that made that possible. With 
> Zuul v3 work largely done we can now look forward to our next priority 
> efforts.
>
> Currently the only task marked as a priority is the task-tracker spec which 
> at this point is migrating projects into storyboard. I think we can likely 
> add one or two new priority efforts to this list.
>
> After some quick initial brainstorming these were the ideas I had for getting 
> onto that list (note some may require we actually write a spec):
>
> * Gerrit upgrade to 2.14/2.15
> * Control Plane operating system upgrades to Xenial
> * Bringing wiki under config management management
>
> My bias here is I've personally been working to try and pay down some of this 
> tech debt we've built up simply due to bit rot, but I know we have other 
> specs and I'm sure we can make good arguments for why other efforts should be 
> made a priority. I'd love to get feedback on what others think would make 
> good priority efforts.
>
> Let's use this thread to identify candidates then whittle the list down to 
> one or two to focus on for the next little while.
>
> Thank you,
> Clark
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?

2018-03-29 Thread David Moreau Simard
The talk was this week and it's up on YouTube [1].

During the talk which was basically a long live demo, we...

- Sent a patch to fix a typo in the talk [2]
- Fixed a Zuul job through speculative testing [3]
- Updated the openstack-infra IRC meeting chair [4].

Oh, and we also added an item on the next meeting to talk about this talk [5].

It was fun.

[1]: https://youtu.be/6gTsL7E7U7Q
[2]: https://review.openstack.org/#/c/556738/
[3]: https://review.openstack.org/#/c/556615/
[4]: https://review.openstack.org/#/c/557095/
[5]: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting


David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]


On Sat, Mar 24, 2018 at 9:28 PM, David Moreau Simard
<dmsim...@redhat.com> wrote:
> Hi -infra,
>
> I'll be presenting a talk at a local OpenStack meetup next week [1]
> that will highlight some examples about how people can help and
> contribute to the infrastructure project.
> The talk will be recorded and should hopefully serve as a form of
> informal documentation.
>
> I'd like to disclose some semi-official numbers (as I'd personally
> pull them up) to let people have an idea of the scale our contributors
> are maintaining.
> I suppose this data is already somewhat public if you know where to
> look but I don't think it's been written down in a digestable format
> in recent history.
>
> Unless there's any objection, I'd have a slide with up to date numbers such 
> as:
> - # of projects hosted (as per git.openstack.org)
> - # of servers (in aggregate of all our regions)
> -- (Maybe some big highlights like the size of logstash, logs.o.o, Zuul)
> - Nodepool capacity (number of clouds, aggregate capacity)
> - # of jobs and Ansible playbooks per month ran by Zuul
> - Approximate number of maintained and hosted services (irc,
> gerritbot, meetbot, gerrit, git, mailing lists, wiki, ask.openstack,
> storyboard, codesearch, etc.)
> - Probably some high level numbers from Stackalytics
> - Maybe something else I haven't thought about
>
> The idea of the talk is not to brag about all the stuff we're doing
> but rather, "hey, you don't need to be a pro in OpenStack to
> contribute, we got all these different things you can help with".
>
> I realize it's a bit last minute but please let me know if you see
> anything wrong with this !
>
> [1]: https://www.meetup.com/Montreal-OpenStack/events/248344351/
>
> David Moreau Simard
> Senior Software Engineer | OpenStack RDO
>
> dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?

2018-03-26 Thread David Moreau Simard
Good point.

I'll work with that instead.

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]


On Mon, Mar 26, 2018 at 4:30 PM, James E. Blair <cor...@inaugust.com> wrote:
> David Moreau Simard <dmsim...@redhat.com> writes:
>
>> On Mon, Mar 26, 2018 at 10:20 AM, James E. Blair <cor...@inaugust.com> wrote:
>>>> - # of jobs and Ansible playbooks per month ran by Zuul
>>>
>>> I'm curious about this one -- how were you planning on defining these
>>> values and obtaining them?
>>>
>>
>> I've needed to pull statistics out of Zuul in the past for RDO (i.e,
>> justifying budget for CI resources)
>> and I use the sql reporter data to do it.
>> It looks like this:
>>
>> $range = "'2018-02-01 00:00:00' AND '2018-02-28 23:59:59'"
>> SELECT job_name,
>>result,
>>start_time,
>>end_time,
>>TIMEDIFF(end_time, start_time) as duration
>> FROM zuul_build
>> WHERE
>> start_time BETWEEN $range
>>
>> This gets me the amount of monthly *jobs* and I can extrapolate (over
>> N playbooks..)
>> by estimating a number knowing that:
>> - base and post playbooks are fairly consistently X playbooks
>> - there is at least one "run" playbook
>>
>> So pretending that 1000 jobs ran, I can say something like:
>> 1000 jobs and over [1000 * (X+1)] playbooks
>>
>> It's not a perfect number but we know we run more playbooks than that.
>>
>> What I have also been thinking about is, if I want to get a more
>> accurate number, I could do a sum of all the executor playbook results
>> (which are in graphite) but the history for those don't go too far
>> back.
>> Ex: stats.zuul.executor.ze*_openstack_org.phase.*.*
>
> The SQL query gets the number of completed jobs which are *reported*.
> It doesn't get you two other numbers, which are the jobs *launched*
> (many of which may have been aborted before completion), or the jobs
> *completed* (the results of many of which may have been discarded due to
> changes in the environment).  In reality, the system is likely to be
> significantly busier than the number of jobs reported will indicate.
>
> Both of the other values can be obtained from graphite or by parsing
> logs.  I think for this purpose, graphite might be sufficient.  (The
> only time I'd recommend going to logs is when we need to find
> project-specific resource usage information.)
>
> stats_counts.zuul.executor.*.builds should be all jobs launched.
> stats_counts.zuul.tenant.*.pipeline.*.all_jobs should be all jobs completed.
>
> -Jim

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?

2018-03-26 Thread David Moreau Simard
On Mon, Mar 26, 2018 at 10:20 AM, James E. Blair <cor...@inaugust.com> wrote:
>> - # of jobs and Ansible playbooks per month ran by Zuul
>
> I'm curious about this one -- how were you planning on defining these
> values and obtaining them?
>

I've needed to pull statistics out of Zuul in the past for RDO (i.e,
justifying budget for CI resources)
and I use the sql reporter data to do it.
It looks like this:

$range = "'2018-02-01 00:00:00' AND '2018-02-28 23:59:59'"
SELECT job_name,
   result,
   start_time,
   end_time,
   TIMEDIFF(end_time, start_time) as duration
FROM zuul_build
WHERE
start_time BETWEEN $range

This gets me the amount of monthly *jobs* and I can extrapolate (over
N playbooks..)
by estimating a number knowing that:
- base and post playbooks are fairly consistently X playbooks
- there is at least one "run" playbook

So pretending that 1000 jobs ran, I can say something like:
1000 jobs and over [1000 * (X+1)] playbooks

It's not a perfect number but we know we run more playbooks than that.

What I have also been thinking about is, if I want to get a more
accurate number, I could do a sum of all the executor playbook results
(which are in graphite) but the history for those don't go too far
back.
Ex: stats.zuul.executor.ze*_openstack_org.phase.*.*

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?

2018-03-24 Thread David Moreau Simard
Hi -infra,

I'll be presenting a talk at a local OpenStack meetup next week [1]
that will highlight some examples about how people can help and
contribute to the infrastructure project.
The talk will be recorded and should hopefully serve as a form of
informal documentation.

I'd like to disclose some semi-official numbers (as I'd personally
pull them up) to let people have an idea of the scale our contributors
are maintaining.
I suppose this data is already somewhat public if you know where to
look but I don't think it's been written down in a digestable format
in recent history.

Unless there's any objection, I'd have a slide with up to date numbers such as:
- # of projects hosted (as per git.openstack.org)
- # of servers (in aggregate of all our regions)
-- (Maybe some big highlights like the size of logstash, logs.o.o, Zuul)
- Nodepool capacity (number of clouds, aggregate capacity)
- # of jobs and Ansible playbooks per month ran by Zuul
- Approximate number of maintained and hosted services (irc,
gerritbot, meetbot, gerrit, git, mailing lists, wiki, ask.openstack,
storyboard, codesearch, etc.)
- Probably some high level numbers from Stackalytics
- Maybe something else I haven't thought about

The idea of the talk is not to brag about all the stuff we're doing
but rather, "hey, you don't need to be a pro in OpenStack to
contribute, we got all these different things you can help with".

I realize it's a bit last minute but please let me know if you see
anything wrong with this !

[1]: https://www.meetup.com/Montreal-OpenStack/events/248344351/

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] Talks for the Vancouver CFP ?

2018-01-22 Thread David Moreau Simard
Hi,

Did we want to brainstorm around topics and talks suggestions from an
openstack-infra perspective for Vancouver [1] ?

The deadline is February 8th and the tracks are the following:
- CI / CD
- Container Infrastructure
- Edge Computing
- HPC / GPU / AI
- Open Source Community
- Private & Hybrid Cloud
- Public Cloud
- Telecom & NFV

CI/CD has Zuul and Nodepool written all over it, of course.
FWIW I'm already planning on submitting a talk that covers how a
commit in an upstream project ends up being released by RDO which
includes the upstream Zuul and RDO's instance of Zuul (amongst other
things).

I started an etherpad [2], we can brainstorm there ?

[1]: https://www.openstack.org/summit/vancouver-2018/call-for-presentations/
[2]: https://etherpad.openstack.org/p/infra-vancouver-cfp

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Hostnames

2018-01-07 Thread David Moreau Simard
When I compared ze10 with ze09 today, I noticed that ze09's "hostname"
command returned "ze09" while ze10 had "ze10.openstack.org".

However, both nodes had the full fqdn when doing "hostname -f".

I didn't dig deeper since we're the weekend and all that but there might be
a clue in my experience above.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

On Jan 6, 2018 1:04 PM, "James E. Blair" <cor...@inaugust.com> wrote:

> Hi,
>
> It seems that every time we boot a new server, it either randomly has a
> hostname of foo, or foo.openstack.org.  And maybe that changes between
> the first boot and second.
>
> The result of this is that our services which require that they know
> their hostname (which is a lot, especially the complicated ones) end up
> randomly working or not.  We waste time repeating the same diagnosis and
> manual fix each time.
>
> What is the cause of this, and how do we fix this correctly?
>
> -Jim
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Xenial Upgrade Sprint Recap

2017-12-17 Thread David Moreau Simard
+1, my day to day tends to be a bit hectic so it's hard to commit to a full
sprint but I have time here and there to help move things along.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

On Dec 15, 2017 8:09 PM, "James E. Blair" <cor...@inaugust.com> wrote:

> Clark Boylan <cboy...@sapwetik.org> writes:
>
> > Hello everyone,
> >
> > Just wanted to quickly recap what we got done this week during our
> > control plane upgrade to Xenial sprint.
>
> Thanks!
>
> Now that this is over, I wonder if we should start a 'marathon' (as
> opposed to a sprint) to finish the rest of the servers?
>
> Now that we've established the pattern, perhaps if we all agreed to do
> one or two servers each per week, we'd knock the rest out in good time?
>
> I wasn't able to spend as much time dedicated to this as I would
> normally have liked, but did find that due to the latency, I could fit
> in a few minutes here and there into my schedule fairly easily.
>
> -Jim
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] grafana doesn't display result of periodic job

2017-10-03 Thread David Moreau Simard
This is a known issue of the migration to Zuul v3.

The metric data from jobs (duration, status, etc) running under Zuul v3 is
not yet available. It's something we still need to add support for.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

On Oct 3, 2017 5:14 PM, "fumihiko kakuma" <kak...@valinux.co.jp> wrote:

> Hi infra team,
>
> I noticed that grafana doesn't display the result of periodic jobs[1]
> since 9/29.
> Health results are also similar[2].
> Does the periodic job work?
>
> Thanks,
> kakuma
>
> [1] http://grafana.openstack.org/dashboard/db/neutron-failure-
> rate?panelId=4
> [2] http://status.openstack.org/openstack-health/#/job/
> periodic-tempest-dsvm-neutron-with-ryu-master-ubuntu-xenial?duration=P14D
> http://status.openstack.org/openstack-health/#/job/
> periodic-neutron-dynamic-routing-dsvm-tempest-with-ryu-
> master-scenario-ipv4?duration=P14D
>
> --
> fumihiko kakuma <kak...@valinux.co.jp>
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Infra Team PTG Dinner

2017-09-03 Thread David Moreau Simard
Do we want to set up a scheduling doodle [1] or something to get the
best possible time ?

As the PTG draws near, different events are being set up.
There's the RDO release party wednesday [2] (everyone is invited!) and
I'm trying to juggle tuesday right now.

[1]: http://doodle.com/
[2]: https://www.redhat.com/archives/rdo-list/2017-August/msg00057.html


David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]


On Wed, Aug 23, 2017 at 5:21 PM, Clark Boylan <cboy...@sapwetik.org> wrote:
> Hello,
>
> I brought this up briefly in yesterday's meeting, but are we interested
> in having a team dinner while at the PTG? I expect we are so I've tried
> to start putting information together at
> https://etherpad.openstack.org/p/infra-ptg-team-dinner.
>
> The biggest issue is there does not appear to be a lot of options near
> the PTG hotel. That said, there appears to be a good beer garden option
> about 4 miles away. Beer gardens were a hit in Darmstadt because they
> can accomodate large groups, we don't have to pre negotiate reservations
> or who is paying, and of course because beer.
>
> If interested please add your name to the list on the etherpad and the
> evenings you are available for the dinner. Also, if anyone else has
> better ideas (seriously I'm not great at this) or wants to organize go
> ahead and edit the etherpad and/or let me know.
>
> Thanks,
> Clark
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] Can we add ARA to the Zuul v3 projects ?

2017-08-26 Thread David Moreau Simard
Hi,

There are major changes coming to ARA 1.0 [1] and it currently has no
Zuul v3 test coverage at all.
I'd like to add coverage with the same approach Monty is taking here [2].

Is it possible for ARA to use Zuul v3 with an in-repo .zuul.yaml file ?

Thanks !

[1]: https://github.com/openstack/ara/compare/feature/1.0
[2]: https://review.openstack.org/#/c/498209/

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Plan for devstack Zuul v3 jobs

2017-08-25 Thread David Moreau Simard
On Fri, Aug 25, 2017 at 4:50 PM, James E. Blair <cor...@inaugust.com> wrote:
> We create a devstack-legacy job in Zuul v3 which attempts to run
> devstack-gate in the manner closest to that in which it runs today.
> This means that it will use the Zuul-provided git repos rather than
> performing its own git fetch operations, and supply config files and
> environment variables which are compatible with the way Zuul v2 works.
>
> Simultaneously, we also create a new devstack job which utilizes all the
> new features of Zuul v3 and is structured in the way we envisioned
> earlier.  We can start very simply here and avoid carrying all of the
> design baggage from the earlier job.  This will be a job that projects
> can build off of and migrate to over time, once we have completed the
> migration.

I'm OK with this.
Should we get to a point where the scripts can be used by both Zuul v2
and Zuul v3 simultaneously so that there is no "migration" or "cutover" so
to speak ?

It might mean some amount of work but it shouldn't be too bad, I think ?
Ansible (Zuul v3) will potentially end up running Ansible but we've seen
worse and it'll be temporary.

It would allow for a smooth transition because we would just essentially
stop running the v2 jobs when we are ready.

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Opendayight Internship (Jenkins job builder) Week 12 status update August 14th - August 20th 2017

2017-08-21 Thread David Moreau Simard
Hey Yolande,

I see these emails every week and I just had to send some warm thanks for
the work you've been doing on JJB.

JJB is an important piece of software that's made my life easier when
dealing with Jenkins or Zuul environments.

So thanks again for your time and dedication to the project!

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

On Aug 21, 2017 5:38 PM, "Yolande Amate" <yolandeam...@gmail.com> wrote:

Hi,

Last week I did the following:

- Included tests and complete adding support for jira_changelog_trigger[1]
- Added support for the Branch API Plugin[2]
- Began adding support for CloudBees Docker Hub Notification Plugin(draft
patch)[3]
- Updated archive based on code review to use convert_mapping_to_xml()[4]

This week, I plan to:

- Complete adding support for CloudBees Docker Hub Notification Plugin
- Add support for Email Extension Plugin
- Add support for Jira Issue Updater Plugin
- Respond to and do code review

Attached below is a link to my Jenkins job builder project proposal
document with a table of plugins(under activity details) that I have worked
on so far[5].

Cheers,
Yolande

[1]https://review.openstack.org/#/c/493336/
[2]https://review.openstack.org/#/c/494031/
[3]https://review.openstack.org/#/c/494754/
[4]https://review.openstack.org/#/c/476960/
[5]https://docs.google.com/document/d/13YGxDwt76K5mhRUlmC_bzXK_
mCRSiVGStp-CJL4VYtA/edit#

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Zuul v3: Giant pile of thoughts around zuul-web things

2017-07-26 Thread David Moreau Simard
Hi,

A bit out of my comfort zone for the content of your giant pile of
thoughts, but I'll jump on this bit:

> We have a desire to add something that knows how to
> render the job-output.json in some nice ways.

The next things on my plate for ARA are:
- Introduce the concept of API to abstract model access
- Input drivers based on the API (callback, zuul json, mqtt, etc.)
- Output drivers based on the API (html, junit, subunit, influx, graphite, etc.)

This is no small task but I'm hoping to have made significant progress
by the time the PTG rolls around.

Paul showed interest in having those job-output.json files imported into ARA.
I could see this taking a similar approach than how we are currently
importing subunit files into openstack-health.

I'll let you know if I have anything to show !

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Collected list of repos to retire

2017-07-11 Thread David Moreau Simard
I don't want to sidetrack the thread but should someone also be taking
care of cleaning up stale branches in Devstack ?
It goes all the way back to kilo.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Mon, Jul 10, 2017 at 11:05 AM, Andreas Jaeger <a...@suse.com> wrote:
> infra admins, could somebody please retire the following branches from
> repositories:
>
> openstack/sahara-extra   stable/icehousePlease retire
> openstack/sahara-extra   stable/mitaka  Please retire
> openstack/packstack  stable/kiloPlease retire
> openstack/packstack  stable/liberty Please retire
> openstack/packstack  stable/mitaka  Please retire
> openstack/bareon-ironic  stable/mitaka  Please retire
> openstack/rpm-packaging  stable/mitaka  Please retire
> openstack/training-labs  stable/mitaka  Please retire
>
> Not done in
> https://gist.githubusercontent.com/tbreeds/c99e62bf8da19380e4eb130be8783be7/raw/6d02deb40e07516ce8fc529d2ba8c74af11a5a6b/mitaka_eol_data.txt
>
> openstack/astara stable/mitaka  Please retire
>
>
> Special treatment:
> openstack/training-labs icehouse-eol (just delete branch, tag exists)
> openstack/training-labs juno-eol (delete branch, create tag instead)
>
> Note that packstack has eol tags already created.
>
> thanks,
> Andreas
> --
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] Removing EOL branches from Packstack

2017-07-10 Thread David Moreau Simard
Hi -infra,

We've tagged -eol for stable/kilo, stable/liberty as well as
stable/mitaka for Packstack.
We're a bit late! Sorry about that.

The branches can be deleted now.

Let me know if you have any questions,

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-dev] [infra][security] Encryption in Zuul v3

2017-03-21 Thread David Moreau Simard
I don't have a horse in this race or a strong opinion on the topic, in
fact I'm admittedly not very knowledgeable when it comes to low-level
encryption things.

However, I did have a question, even if just to generate discussion.
Did we ever consider simply leaving secrets out of Zuul and offloading
that "burden" to something else ?

For example, end-users could use something like git-crypt [1] to crypt
files in their git repos and Zuul could have a mean to decrypt them at
runtime.
There is also ansible-vault [2] that could perhaps be leveraged.

Just trying to make sure we're not re-inventing any wheels,
implementing crypto is usually not straightfoward.

[1]: https://www.agwa.name/projects/git-crypt/
[2]: http://docs.ansible.com/ansible/playbooks_vault.html

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Tue, Mar 21, 2017 at 12:36 PM, James E. Blair <cor...@inaugust.com> wrote:
> Hi,
>
> In working on the implementation of the encrypted secrets feature of
> Zuul v3, I have found some things that warrant further discussion.  It's
> important to be deliberate about this and I welcome any feedback.
>
> For reference, here is the relevant portion of the Zuul v3 spec:
>
> http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#secrets
>
> And here is an implementation of that:
>
> https://review.openstack.org/#/q/status:open+topic:secrets+project:openstack-infra/zuul
>
> The short version is that we want to allow users to store private keys
> in the public git repos which Zuul uses to run jobs.  To do this, we
> propose to use asymmetric cryptography (RSA) to encrypt the data.  The
> specification suggests implementing PKCS#1-OAEP, a standard for
> implementing RSA encryption.
>
> Note that RSA is not able to encrypt a message longer than the key, and
> PKCS#1 includes some overhead which eats into that.  If we use 4096 bit
> RSA keys in Zuul, we will be able to encrypt 3760 bits (or 470 bytes) of
> information.
>
> Further, note that value only holds if we use SHA-1.  It has been
> suggested that we may want to consider using SHA-256 with PKCS#1.  If we
> do, we will be able to encrypt slightly less data.  However, I'm not
> sure that the Python cryptography library allows this (yet?).  Also, see
> this answer for why it may not be necessary to use SHA-256 (and also,
> why we may want to anyway):
>
> https://security.stackexchange.com/questions/112029/should-sha-1-be-used-with-rsa-oaep
>
> One thing to note is that the OpenSSL CLI utility uses SHA-1.  Right
> now, I have a utility script which uses that to encrypt secrets so that
> it's easy for anyone to encrypt a secret without installing many
> dependencies.  Switching to another hash function would probably mean we
> wouldn't be able to use that anymore.  But that's also true for other
> systems (see below).
>
> In short, PKCS#1 pros: Simple, nicely packaged asymmetric encryption,
> hides plaintext message length (up to its limit).  Cons: limited to 470
> bytes (or less).
>
> Generally, when faced with the prospect of encrypting longer messages,
> the advice is to adopt a hybrid encryption scheme (as opposed to, say,
> chaining RSA messages together, or increasing the RSA key size) which
> uses symmetric encryption with a single-use key for the message and
> asymmetric encryption to hide the key.  If we want Zuul to support the
> encryption of longer secrets, we may want to adopt the hybrid approach.
> A frequent hybrid approach is to encrypt the message with AES, and then
> encrypt the AES key with RSA.
>
> The hiera-eyaml work which originally inspired some of this is based on
> PKCS#7 with AES as the cipher -- ultimately a hybrid approach.  An
> interesting aspect of that implementation is that the use of PKCS#7 as a
> message passing format allows for multiple possible underlying ciphers
> since the message is wrapped in ASN.1 and is self-descriptive.  We might
> have simply chosen to go with that except that there don't seem to be
> many good options for implementing this in Python, largely because of
> the nightmare that is ASN.1 parsing.
>
> The system we have devised for including encrypted content in our YAML
> files involves a YAML tag which specifies the encryption scheme.  So we
> can evolve our use to add or remove systems as needed in the future.
>
> So to break this down into a series of actionable questions:
>
> 1) Do we want a system to support encrypting longer secrets?  Our PKCS#1
> system supports up to 470 bytes.  That should be sufficient for most
> passwords and API keys, but unlikely to be sufficient for some
> certificate related systems, etc.
>
> 2) If so, what system should we use?
>
>2.1a) GPG?  This has hybr

[OpenStack-Infra] Need to be added to new project core group

2017-02-02 Thread David Moreau Simard
Hi,

The project "ansible-role-ara" was created recently [1].

Could you please add me ( dms AT redhat.com ) in the core and release groups ?
I'll take care of the rest of the permissions.

Thanks,

[1]: https://review.openstack.org/#/c/426818/

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] New StackForge Project dash-stack

2016-11-13 Thread David Moreau Simard
I'm all for diversity but I'm still curious.

In what way does Horizon not already provide what you are looking for and
why could it not be improved to get what you feel is lacking ?

I think this answer could translate into valuable feedback for the Horizon
team.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

On Nov 12, 2016 7:47 AM, "Ferhat Ozkasgarli" <ozkasga...@gmail.com> wrote:

> Hello to All Infra team,
>
> We have been developing a new dashboard for OpenStack. But recently we
> have decided to use StackForge to benefit all the advantages of OpenStack
> development environment and system.
>
> I would be glad if any of you guys can review following change so we can
> continue developing:
>
> https://review.openstack.org/#/c/394010/
>
> Sincerely,
> Ferhat Ozkasgarli
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] Thanks for enabling a great demo!

2016-11-03 Thread David Moreau Simard
It was definitely an awesome way of showing how OpenStack
contributions are done and tested while giving some warm thanks to the
providers donating resources.
Giving exposure to the workflow between Git, Gerrit, Zuul and Nodepool
was great.

Thank you and everyone else involved for getting that up on the keynote stage !

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Thu, Nov 3, 2016 at 12:26 PM, Jonathan Bryce <jonat...@openstack.org> wrote:
> Hi Infra team,
>
> I wanted to send a note to say thanks for helping us demo the power of a 
> global footprint of clouds powered by OpenStack in Barcelona. If you weren’t 
> there, you can watch the video as Lyz coaches me through adding some regions 
> to nodepool and Clark and fungi push the change out (with a 
> behind-the-scenes) +2 from jeblair): 
> http://www.openstack.org/videos/video/demoing-the-worlds-largest-multi-cloud-ci-application
>
> We received a lot of really positive feedback, and I was very happy to be 
> able to show off the awesome work you all have done over the last few years 
> building out these tools. As I said in Barcelona, thanks again to everyone 
> for letting us play with your systems live and in production!
>
> Jonathan
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] MeetBot taking unauthorised vacation

2016-08-17 Thread David Moreau Simard
I love this.

But on a serious note, the bot has to be restarted when commits are
done to it and usually cores are quite dilligent about not merging
reviews while there are meetings in progress.
Maybe something slipped this time ?

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Wed, Aug 17, 2016 at 6:14 AM, Stig Telfer <stig.openst...@telfer.org> wrote:
> Hi All -
>
> We just had a Scientific WG meeting in #openstack-meeting but the MeetBot 
> left half way through.  Now the meeting cannot end (I’m sure it often feels 
> that way).
>
> Best wishes,
> Stig
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] [openstack-dev] [StoryBoard] Thanks for the bugsquash, plus a new-things roundup

2016-07-05 Thread David Moreau Simard
I appreciate all the great work that's been going into StoryBoard lately.

Thanks everyone involved.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Tue, Jul 5, 2016 at 12:50 PM, Zara Zaimeche
<zara.zaime...@codethink.co.uk> wrote:
> Hi all,
>
> A big thank you to everyone who came and helped out in the spectacular*
> StoryBoard bug squash! We look forward to the next. :) Here are some
> hilights from the last couple of weeks:
>
> * BEAUTIFUL NEW COMMENTS AND EVENTS TIMELINE
>
> It's so beautiful, it requires all-caps. SotK has transformed the barebones
> events timeline into an elegant swan. Well, that's a weird mixed metaphor,
> but it *is* lovely! Furthermore, this magnificent gentleman has removed
> pagination so that comments are no longer lost on the second page of the
> results, and has made it possible to link comments directly. Extra thanks to
> ttx for fixing some of the css during the bugsquash! :) Here's an example:
>
> https://storyboard.openstack.org/#!/story/2000464#comment-7029
>
> There is a WIP patch in review for editing one's own comments, for anyone
> interested in trying it out and giving feedback:
>
> https://review.openstack.org/#/c/333418/
>
>
> * Email threading
>
> The kindly pedroalvarez has worked some magic on the emails StoryBoard
> sends, so that they are threaded according to story. It should now be easier
> to see what an email refers to at a glance.
>
>
> * API Docs example commands
>
> anteaya has made it easier for people to interact with StoryBoard via the
> API with these examples. This should be good news for anyone who wants to
> use scripts with StoryBoard. You can see them here:
>
> http://docs.openstack.org/infra/storyboard/webapi/v1.html#stories
>
>
> * Gerrit integration for storyboard-dev
>
> Review-dev can now post comments on storyboard-dev (our test instance)!
> Thanks so much, zaro! You can see an example patch here:
> https://review-dev.openstack.org/#/c/5454/
>
>
> * Tags search upgraded
>
> Tags search now suggests existing tags! This should making searching-by-tag
> much easier.
>
> I hope to build on this to change task-statuses in the next couple of weeks.
>
>
>
> It's been a pretty busy time... which is why I'm over a week late with this
> email \o/. Anyway, yes, thanks again to everyone who helped out. If you'd
> like to get involved in the project, we're always available in #storyboard
> on freenode; the project is a mix of python and angularjs. We have a
> worklist of stories that contain easy tasks here:
> https://storyboard.openstack.org/#!/worklist/76 , so you can see if anything
> takes your interest, then it's best to ask in the channel for the specifics.
> :)
>
> Hope to see you there! If I've missed anything, please let me know.
>
> Best Wishes,
>
> Zara
>
> *I haven't personally written any interesting patches of late, so I am
> allowed to call it 'spectacular'. :)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] Ubuntu cloud archive (UCA) AFS mirror now live

2016-06-30 Thread David Moreau Simard
At what frequency does the upstream UCA repository change and how
quickly does the AFS repository pick them up ?

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Wed, Jun 29, 2016 at 2:57 PM, Paul Belanger <pabelan...@redhat.com> wrote:
> Greetings,
>
> Thanks to the work done by Jesse Pretorius (odyssey4me)[1] we now have an UCA
> AFS mirror[2] online.
>
> Feel free to update your project testing to use them if needed. As always, if
> there is problems feel free to reach out to us in #openstack-infra.
>
> [1] https://review.openstack.org/#/c/330152/
> [2] http://mirror.dfw.rax.openstack.org/ubuntu-cloud-archive/
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] An idea to help browse and troubleshoot Ansible Playbook runs

2016-05-21 Thread David Moreau Simard
Hi again openstack-infra,

We've worked quite a bit on ARA since I first told you guys about it here.
We're pretty much out of PoC and Alpha territory by now and I've made
a formal public announcement about it [1].

Following up on the questions from the thread now that we've progressed a bit:
- Usability with things like ElasticSearch:
  The ARA CLI client uses Cliff and thus has a very similar look and
feel as python-openstackclient.
  Cliff provides several output formats out of the box such as YAML or JSON.
  You could use the client to output the information you want in JSON
format and then send that off to ElasticSearch with already-indexed
fields.

- Static file generation
  This is very high on the list of things to do.
  We will make it possible for the ARA CLI client to generate a static
version of the interface for a single playbook run.
  So far we're aiming more towards a html "tree" rather than just a
single page to keep the same browsing experience.

I see a great opportunity for ARA to be used by the OpenStack community.
The project officially needs a home for code reviews and I think
openstack or openstack-infra with the usual gerrit workflow would be a
great fit if you believe it makes sense.

What do you think ?

Please let me know !

Thanks,

[1]: 
https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Tue, May 10, 2016 at 9:24 PM, David Moreau Simard <d...@redhat.com> wrote:
> Hi openstack-infra,
>
> I've been hacking on an idea since last friday and I was thinking it
> could be of interest for you guys as Ansible users, but also
> considering the next generation of Zuul jobs will be powered by
> Ansible.
>
> The project I'm working on is called ARA [1]. It is very much an alpha
> at this point but it is hopefully fleshed out enough at this point for
> you to see where I am going with this.
> I could go on about what the project looks like but I've actually
> built a sane amount of docs [2] and even put up a video of what the
> interface looks like on YouTube [3].
>
> ARA was born out of necessity because the vast majority of RDO CI [4]
> is driven by Ansible, either through projects like TripleO-Quickstart
> [5] or WeIRDO [6].
>
> We are equipped in a fairly similar way than upstream, we have Jenkins
> console logs that are large enough to crash your browser and logs that
> are collected and made browsable like on logs.o.o.
> In a nutshell, we're running CI jobs that install and test OpenStack
> deployments through various installers: TripleO, Packstack,
> Puppet-OpenStack, and soon: Kolla and Chef-OpenStack.
>
> The sad reality is that the RDO community has limited resources and
> ARA aims at making it easier and faster to see what failed, where and
> why to make everyone more efficient.
> Another sad truth is that I am much more a system administrator than a
> developer and as such I am not exactly awesome in either Python or
> UI/UX frontends.
>
> As such, if you are interested in the project, I would certainly
> welcome not only feedback but contributions if you feel you could use
> this.
>
> Let me know if you have any questions !
>
> [1]: https://github.com/dmsimard/ara
> [2]: http://ara.readthedocs.io/en/latest/
> [3]: https://www.youtube.com/watch?v=K3jTqgm2YuY
> [4]: https://ci.centos.org/view/rdo/view/promotion-pipeline/
> [5]: https://github.com/openstack/tripleo-quickstart
> [6]: https://github.com/redhat-openstack/weirdo
>
> David Moreau Simard
> Senior Software Engineer | Openstack RDO
>
> dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] An idea to help browse and troubleshoot Ansible Playbook runs

2016-05-11 Thread David Moreau Simard
On Wed, May 11, 2016 at 9:29 AM, Tristan Cacqueray <tdeca...@redhat.com> wrote:
> Though it seems odd to use another service to display that information..
> would it be possible to render the html interface as static files and
> figures out a way to index each run in elastic search ?

There''s already a TODO [1] for static file generation for "standalone" viewing.
As far as features go right now, I'm really opened to anything and
everything as long as it's in the scope.

Like I mention in the docs, I don't want ARA to be doing things like
controlling ansible, it's inventory or other things like that.
As long as it's around recording ansible runs and providing an
intuitive interface to browse them, I'm definitely open to be doing
things like that.

In the context of elastic search, we could definitely export to JSON
so that it could be easily indexed into fields.

Sky is the limit :)

[1]: https://github.com/dmsimard/ara/issues/3

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


[OpenStack-Infra] An idea to help browse and troubleshoot Ansible Playbook runs

2016-05-10 Thread David Moreau Simard
Hi openstack-infra,

I've been hacking on an idea since last friday and I was thinking it
could be of interest for you guys as Ansible users, but also
considering the next generation of Zuul jobs will be powered by
Ansible.

The project I'm working on is called ARA [1]. It is very much an alpha
at this point but it is hopefully fleshed out enough at this point for
you to see where I am going with this.
I could go on about what the project looks like but I've actually
built a sane amount of docs [2] and even put up a video of what the
interface looks like on YouTube [3].

ARA was born out of necessity because the vast majority of RDO CI [4]
is driven by Ansible, either through projects like TripleO-Quickstart
[5] or WeIRDO [6].

We are equipped in a fairly similar way than upstream, we have Jenkins
console logs that are large enough to crash your browser and logs that
are collected and made browsable like on logs.o.o.
In a nutshell, we're running CI jobs that install and test OpenStack
deployments through various installers: TripleO, Packstack,
Puppet-OpenStack, and soon: Kolla and Chef-OpenStack.

The sad reality is that the RDO community has limited resources and
ARA aims at making it easier and faster to see what failed, where and
why to make everyone more efficient.
Another sad truth is that I am much more a system administrator than a
developer and as such I am not exactly awesome in either Python or
UI/UX frontends.

As such, if you are interested in the project, I would certainly
welcome not only feedback but contributions if you feel you could use
this.

Let me know if you have any questions !

[1]: https://github.com/dmsimard/ara
[2]: http://ara.readthedocs.io/en/latest/
[3]: https://www.youtube.com/watch?v=K3jTqgm2YuY
[4]: https://ci.centos.org/view/rdo/view/promotion-pipeline/
[5]: https://github.com/openstack/tripleo-quickstart
[6]: https://github.com/redhat-openstack/weirdo

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra