Re: [OpenStack-Infra] Swift bitrot jobs
James E. Blair wrote: > Mark McLoughlin writes: >> Could you take openstack-stable-maint off the notification list for >> swift bitrot jobs? >> >> The stable-maint team doesn't have any involvement with swift (yet?) ... >> I didn't even know it had a stable/folsom branch > > The bitrot jobs are templated such that all stable/folsom bitrot jobs > email the same address. However, if you don't care about the > stable/* branches of swift, it's unlikely anyone does, so we could > probably just stop running bitrot jobs for swift. > > https://review.openstack.org/18901 Hmm, that's not exactly true. The security team theoretically needs to be able to land patches on Swift's stable/folsom branch, and letting it drift too much in a sad state is not really a good idea. That said, I agree running those jobs while having nobody looking after their failure is useless. The trouble here is that nobody (stable team, security team, Swift devs) cares enough about stable/folsom Swift to maintain it in good shape... so I guess we'll have to wait for some security issue to pop up and the theoretical need to become a real one. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Talking is the first step.
Monty Taylor wrote: > The more branches there are, the more possibility there is for developer > confusion. Should a dev start dev work on openstack/master? Or > openstack/smokestack? Then where do they send their patch to land? +10k Having a single branch to centralize development has been one of the key things that brought clarity and focus to our development. We refrained from doing "release branches" and "development branches" for that reason. I also don't really see how cascading gates would work practically. What happens when master has A..B..C..D (with D fixing a bug in B) and smokestack rejects B (because it has a bug ?). How would you reconcile that without manually building wildly different trees ? Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Running databases in the cloud
James E. Blair wrote: > Monty Taylor writes: >> This isn't super urgent - but it sprung out of the "make sure we're >> backing up MySQL databases" bug - which then seemed like something we >> should, in fact, _not_ be dealing with. > > I'm not sure what you're saying here, but I very strongly believe that > we should be dealing with database backups. I really don't want to lose > the data we've created in the wiki, gerrit, etherpads, etc., and backups > are necessary for more than simple technical failures. I suspect Monty's point is that "backups" should be taken care of automatically by the cloud database infrastructure ? I don't really know what the DBaaS offerings currently support, but if their backup function is done properly maybe we can leverage it for proper backups covering your design goals... (which may involve additional copying of backup data to another provider). -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Polling system
Jaromir Coufal wrote: > Hi everybody, > > I would like to start voting across UX group members about future > discussion tool. For voting itself I can use some 3rd party tools or > just mailing list, but I'd like to ask you, if you can suggest me the > best way, since I am sure you already did some name voting or similar > things for OpenStack. Only problem is that I won't have users' e-mails > so I can't invite them directly. I can just share link and whoever comes > to the page might vote. We've been using Launchpad polls for insignificant things (all members of a specific Launchpad group can vote in a group poll). You don't have to know the emails and each member can only vote once, so it you set the group right it's quite usable. Drawback is that it's a "pick one" direct poll, not something fancy like Condorcet. For serious stuff (Condorcet method) we've been using CIVS (which requires you have the precise list of voters emails). Regards, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Log storage/serving
Joshua Hesketh wrote: > Great overview and plan James, thanks for that :-). > > So it seems to me that we're duplicating the job of swift a little bit > by writing a program to accept an object over http and store it on disk. > If our end-game is logs stored in swift then why not make jenkins (and > other workers) push the logs straight to swift? I suspect the additional benefit is the token-based security which means the slaves can have limited access to storage (they basically get a token that can only be used for one very limited - and one-time - task). That said I agree that reinventing HTTP POST storage sounds overkill and storing directly to swift would be a lot simpler. Does Swift include complex access rules that we could leverage to implement the same kind of security that James described in his original post ? If not, would it make sense to actually add what we need to Swift itself ? (after all, it's, you know, an OpenStack project) > [...] > *For example, depending on the size of the logs (and therefore the > job/worker) we could actually use javascript to serve up the swift > object with CORS further reducing the infrastructure requirement and > utilising the powerful CPU's and javascript engines we all have. I > already started doing that as a crude way to serve and add formatting to > my logs[1]. Basically javascript grabs a file and runs some regex for > highlighting. So my worker only reports > http://laughing-spice/logviewer/?q=http://worker/job/index.html where > the index.html contains links to logs like > http://laughing-spice/logviewer/?q=http://worker/job/mysqllog.txt etc. > [...] Sidenote: I like that idea too, but I'm not sure how well that would work with our huge logs. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Suggestion for helping gate users deal with crisis...
James E. Blair wrote: > Clint Byrum writes: > >> Hello infra rockstars. First and foremost, thank you for keeping the >> well oiled machinery of the OpenStack infrastructure running. It is a >> marvel of modern engineering, and I am not just saying that because I >> am prone to hyperbole. >> >> Last night while the gate was exploding a few of us noticed, and >> weren't really sure what to do. ttx was whitelisted in the statusbot, >> but untrained in how to handle it. I dug through jenkins configs and >> logs but I am completely ignorant of zuul and thus would have done >> more damage than not had I been able to coax anything out of the system >> (luckily I am also completely unprivileged.. good job :). > > Cool, we can do something about that. Thierry, please see: > > http://ci.openstack.org/irc.html#statusbot I actually read it during the "crisis" but thought that the commands had to be issued by PRIVMSG :) Then that lead me to look for the nick statusbot was running on, which was a red herring. Proposed doc clarification @ https://review.openstack.org/48209 -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] List of properties that use Launchpad OpenID
David Stanek wrote: > On Tue, Sep 24, 2013 at 6:18 PM, Stefano Maffulli <mailto:stef...@openstack.org>> wrote: > > We're still in the initial phases of the project so there is plenty of > room to make suggestions. The reason of my original question is to have > a full understanding of what systems are currently consuming Launchpad > ID and make sure that whatever system we pick doesn't break them and > eventually leaves space for improvements. > > > I don't think you should pick any OpenID provider. Let the user pick > the one that they already use. +1 I think the idea is to support any OpenID provider, rather than forcing everyone to use Launchpad. The Foundation /could/ run its own provider as an option for people who don't have an OpenID yet, but you should still be able to use your own OpenID provider on Foundation systems (including Launchpad's one for those attached to it). As long as we depend on Launchpad (which ONLY supports Launchpad's OpenID provider) there is no way to migrate to something else anyway... so this is more a "future plans" thing. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Speeding up storyboard (was Re: On being an OpenID consumer instead of an OpenID producer.)
Stefano Maffulli wrote: > On 09/24/2013 04:47 PM, Monty Taylor wrote: >> To be clear, I think it's a TERRIBLE idea to move half-way while we >> still require launchpad ids for bugs. > > Let's hurry up with Storyboard then: the Foundation is setting the > budget for 2014, although I wonder if we can outsource that > development... ideas? I don't think we should "outsource" it, since managing that outsourcing so that it produces the desired results would be a waste of time for the key people involved (compared to openly developing it). I would rather use the money (if any) to fund some positions that would free some time from the people that already work on it. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Fwd: RFC: Workflow for gerrit security patches
Zaro wrote: > We are getting close to standing up a private gerrit instance for > security reviews (https://bugs.launchpad.net/openstack-ci/+bug/1083101). > As mentioned in the bug our plan is to run a second gerrit to > facilitate code review for embargoed patches. But we will not run an > entire second shadow environment (too much effort for ~50 patches a year). That's awesome ! Thanks for working on it. > A few of the implementation details were unclear so I would like to make > a proposal and get feedback before continuing to work on the bug. > > A few members of infra team had a discussion on the workflow and fungi > proposed the following.. > 1. git clone from security gerrit (review-security.o.o) > 2. git review patch to security gerrit > 3. The patch is review-able by vmt member, change owner and any > manually added reviewer. > 4. patch is reviewed and approved on review-security.o.o > 5. patch is copied from security gerrit to public gerrit.. > a. git review -d from review-security.o.o > b*. git review to review.o.o > > *Note - security review information (votes/notes/etc..) will not get > copied to review.o.o > > Does this seem like the right workflow for approving and integrating > security patches? That sounds perfectly acceptable. A few questions: - would it be possible to reuse a classic repo clone (from git.o.o) and add an option to git review to submit to the security gerrit ? Otherwise we have to teach security patch developers to use specific clones from security gerrit before they work on security patches ? - do we have the ability to run check tests (no gate tests) ? - could you briefly explain how you addressed the potential leaks identified in gerrit (ability to download arbitrary changes from a guessed review number, etc) ? > We are also proposing that, instead of syncing accounts from public > gerrit to security gerrit, we shoud keep the accounts independent for > the following reasons: > 1. It would make it a little harder to unintentionally push to the wrong > gerrit. > 2. Some people might want to configure their account profiles > differently on each gerrit (something like project watches). > 3. It will minimize the potential for leakage. > 4. Syncing accounts requires more management overhead. > > With the above proposals we would replicate the gerrit git repository > from review.o.o to review-security.o.o but NOT the gerrit database. Would that mean that everyone has to specifically create an account in the security Gerrit (i.e. log in there once) before they can be subscribed to an arbitrary review ? Or just that the accounts are kept separate ? It might be inconvenient to have to ask everyone to join first, in order to get all the right people available for review help later... Cheers, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] IRC channel services for Fuel
Matthew Mosesohn wrote: > I work with Mirantis and we want to add channels #openstack-fuel and > #openstack-fuel-dev and use it for day-to-day communication and try to > reach any interested contributors. We currently collaborate elsewhere, > but want to fit in on Freenode IRC. Jeremy Stanley (fungi) proposed I > suggest this topic here to discuss the pros and cons of permitting > unofficial projects in #openstack-* namespace in IRC. I know the Foundation is getting concerned with the abuse of the "OpenStack" name for non-official projects. People announcing a "new openstack project" and calling it "openstack FOO". This grey area serves the marketing of a lot of companies well, but creates confusion and dilutes whatever impression of quality and integration we may have created on our users. I fear that adding #openstack-BAR channels adds to that confusion. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Stackforge projects: integration launchpad with openstack gerrit.
Evgeniy L wrote: > Dmitry, > > We tried but it didn't help. > https://review.openstack.org/#/c/2/ > https://bugs.launchpad.net/fuel/+bug/1248960 You might need to update the mapping between the LP project name and the git repo names. See at: https://github.com/openstack-infra/jeepyb/blob/master/jeepyb/projects.py#L134 -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] taskflow requirements
Monty Taylor wrote: > There is a change up to add taskflow to the global requirements. I have > no problem with this in principle, but it's one more that's in the set > of things like pecan, wsme and friends that are in the set of things > that Sean talked about in preemptively gate the universe. > > I'd like to not add it until we have a plan for at least assymetrical > gating, so that changes to taskflow at least can't break cinder and friends. +1 > Further, I think we might need to discuss how to include libraries such > as this. Should taskflow be a part of oslo? That would make sense... Sounds generic enough to me, and I know Joshua has plans for it to be included in multiple projects at this point. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] The future of milestone-proposed branches
Hi!
During the summit I mentioned we could try to get rid of
milestone-proposed branch handling around intermediary milestones, and
replace it by a simple tag on master that would be applied on the
milestone "delivery" date.
I looked into it in more details and I'm not sure it really simplifies
anything. I'd still have to track which the "proposed" SHA for the
milestone for each project is, and the most convenient way to do that is
actually to maintain a milestone-proposed reference pointing to that SHA...
There were actually two problems I was trying to solve:
1- Avoid reusing "milestone-proposed" branches as it creates confusion
when people end up reusing them at a later point
2- Script the release process parts that are not yet automated (like
milestone-proposed branch creation / deletion)
I think we can definitely solve both issues while keeping
milestone-proposed-like branches around.
To avoid reusing milestone-proposed branches we can just use
proposed/$MILESTONE instead ("proposed/icehouse-1", then
"proposed/icehouse" for the RCs and final) and apply [access
"refs/heads/proposed/*"] rules on every project. Since I think
nova-milestone-proposed.tar.gz still looks better (and would create less
clutter) than nova-proposed-icehouse-1.tar.gz, we could special-case the
rename in the {name}-branch-tarball job template shell script (in
modules/openstack_project/files/jenkins_job_builder/config/python-jobs.yaml).
The merge_tags.sh script also special-cases milestone-proposed and would
need to look into proposed/* too. Do you see any other adherence we'd
need to fix ?
To script milestone-proposed branch creation, it looks like granting
createReference to "Release managers" on refs/heads/proposed/* would do
the job (although maybe the current "push" permission on
"refs/for/refs/*" might already be enough). Then I suspect I could "git
branch proposed/icehouse-1 $SHA && git push gerrit proposed/icehouse-1".
To script milestone-proposed branch deletion... looks like we'd have to
allow push +force on the same refs. Not exactly sure how you git-push
the deletion of a branch, but the git/gerrit experts on this list should
be able to tell me.
In the future we could automate this deletion so that it would run as a
post-release job. It's a bit tricky though, since proposed/icehouse-1
needs to be deleted at the end of pre-release, while proposed/icehouse
(the branch that will hold all the RCs and get turned into
stable/icehouse) should stay alive until final release and its
conversion to stable/*.
If this all sounds sane I'll get to work and propose something, maybe in
time for icehouse-1.
--
Thierry Carrez (ttx)
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] The future of milestone-proposed branches
James E. Blair wrote:
> Thierry Carrez writes:
>
>> To script milestone-proposed branch creation, it looks like granting
>> createReference to "Release managers" on refs/heads/proposed/* would do
>> the job (although maybe the current "push" permission on
>> "refs/for/refs/*" might already be enough). Then I suspect I could "git
>> branch proposed/icehouse-1 $SHA && git push gerrit proposed/icehouse-1".
>>
>> To script milestone-proposed branch deletion... looks like we'd have to
>> allow push +force on the same refs. Not exactly sure how you git-push
>> the deletion of a branch, but the git/gerrit experts on this list should
>> be able to tell me.
>
> The thing we don't want to do is grant push (with or without force) on a
> branch itself ('refs/for/...' is probably fine). That is dangerous as
> it allows you to bypass code review.
Right, as far as I can tell that prevents us from deleting branches... I
never succeeded to grant "Push+Force" to refs/for/* (suggestions welcome
on how to express that in the UI), but I suspect that would not work for
branch deletion.
> Also, it's probably worth looking into the state of the REST api, both
> in 2.4 (unlikely) and 2.8 or 2.9.
Delete branch is in 2.8 REST API.
So my suggestion would be to quickly implement branch creation using git
push (Release managers already have "Create Reference" on refs/* so it
probably already works). I can try that for icehouse-1
Second step would be to switch from milestone-proposed to proposed/*,
which involves (afaict):
- for every project, modify refs/heads/milestone-proposed access rules
so that they apply to refs/heads/proposed/* instead
- special-case proposed/* in {name}-branch-tarball job template shell
script (in
modules/openstack_project/files/jenkins_job_builder/config/python-jobs.yaml)
so that it still builds generic PROJECT-milestone-proposed.tar.gz tarballs
- update merge_tags.sh so that it looks into proposed/* rather than
milestone-proposed
- anything I missed
The goal would be to have that done for icehouse-2.
Then as a 3rd step we can wait for Gerrit 2.8 before implementing branch
deletion using the REST API. Or we can wrap it in an automatically-run
post-release job. As mentioned in my original email, that job would need
to differentiate between intermediary milestone branches (which need to
be deleted at the end of pre-release) and the release branch (which
should should stay alive until final release and be converted to stable/*).
--
Thierry Carrez (ttx)
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Week-end project
Hi! If anyone is looking for a limited-scope infra project to get more familiar with OpenStack Infra, I have a suggestion: Gerrit-powered meetings agenda Currently people modify a wiki page and some other people (me) are subscribed to page changes and manually reflect them in a Google Calendar, which can be downloaded/imported as .ics. This is extremely error-prone and quite time-consuming. The idea would be to let people propose Gerrit changes to a simplified YAML description of the meetings, and have *that* produce a .ics file on merge, and publish it to an accessible URL. We'd also publish a human-readable meetings list somewhere (to replace the wiki page). The format of the YAML file would limit errors (all times in UTC, limited set of event repetition options etc.) and be much easier to manipulate than raw iCalendar. There is a icalendar Python module that can be used to write .ics but you still need to learn about that file format. Bonus points for testing for room occupation conflicts automatically pre-merging! This one has been sitting at the bottom of my TODO list for quite a while, so I figure I should let someone else have a shot at it: I think it's a great way to visit a number of aspects of OpenStack infra & development: gerrit, check tests, merge jobs, docs publication... all in a small package ! Let me now what you think, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Week-end project
Chmouel Boudjnah wrote: > On 12 Dec 2013, at 10:21, Thierry Carrez wrote: > >> The format of the YAML file would limit errors (all times in UTC, > > What would be the YAML format ? something like this ? [1]: > > openstack-meeting: > - nova: > - time: 21:00 > - occurrence: weekly > - day: tuesday > - swift: > - time: 20:00 > - occurrence: bi-weekly > - day: wedenesday > > openstack-meeting-alt: > -trove: > - time: 21:00 > - day: wedenedsay > - occurrence: bi-weekly > > Chmouel > > [1] Just kicking the discussion, didn’t enrol for that (yet) :) Looking at the meetings wiki page and what we need to recreate it, probably more like: - Nova team meeting: - day: Thursday - time: 21:00 - repeats: weekly - chair: Russell Bryant - Translations team meeting: - day: Wednesday - time: 00:00 - alternatetime: 16:00 - firstdate: 2013-12-10 - repeats: rotatingweekly - channel: #openstack-meeting-alt - wiki: Meetings/Translations Duration defaults to 1h, channel defaults to openstack-meeting, etc. -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Week-end project
Sean Dague wrote: > Honestly, translating to ical recurrence relationships is going to be > wonky enough (I used to maintain the ruby icalendar gem, and have done a > bunch of fixing of the drupal implementation), I'd just say do this in > ical native with a validator. > > ical is ugly, but it's mostly understandable (and I'd be happy to mentor > on it). For the intents and purposes for this it would probably be fine. Hmm. Not sure requiring anyone who wants to have a meeting to submit properly-formatted iCal is going to be seen as a usability improvement over "editing a wiki page". I fear the barrier to entry will be just too high and people will start having untracked meetings announced on ML posts instead. I also don't really look forward reviewing ical changes, so I would definitely leave that task to anyone who finds it funny. One benefit of coming up with our own format is that we can seriously limit the options (weekly, bi-weekly, monthly with or without rotating time), which I think cover all our current cases... while remaining user-friendly enough. -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Week-end project
Stefano Maffulli wrote: > On 12/12/2013 05:43 AM, Thierry Carrez wrote: >> One benefit of coming up with our own format is that we can seriously >> limit the options (weekly, bi-weekly, monthly with or without rotating >> time), which I think cover all our current cases... while remaining >> user-friendly enough. > > There's gotta be a way to solve the problem with tools already proven. > Like Sean said, dealing with calendars is absolutely a nightmare. > There's gotta be a way to share access to a calendar server among > different people and still keep control of who is allowed to change the > meeting. That's the trick. You want everyone to be able to change the calendar, and have a minimal code review (like we do for openstack-planet) to check that the addition/modification is sane. I don't know of any tool that lets anyone propose a change to a calendar, but requires approval from a review group before it appears. > Wouldn't something like owncloud (or radicale.org) be enough, so we can > ditch Google cal too? Simply have the PTLs, you, I and some others > allowed to edit the calendar so we can have each event with alarms (no > more reminders to the mlist) and notes attached to it automatically. Am > I getting too excited? Google Cal or ownCloud are not really the issue. Both can consume the .ics file and let you add alarms to events. The trick is how you build the calendar in the first place. None of them lets you have an open-but-reviewed calendar. "the PTLs, you, I and some others" are far from being the only ones requesting changes to the official calendar. It's a VERY diverse group, as you can see at https://wiki.openstack.org/w/index.php?title=Meetings&action=history ... So I would much rather let everyone propose and have some way of reviewing the change and then get the .ics autogenerated. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Week-end project
Mathew R Odden wrote: > Not to hijack the topic, but I have a team of university students that I > am looking for an OpenStack related blueprint/project for them to work > on. I think this would be something that wouldn't be too hard for them > to jump into and achieve some results for the duration of their team > project. > > I also think it would be an extremely useful utility for managing the > growing amount of meetings. I think that would be a good project for them (limited scope, self-contained), unless you want them to get familiar with OpenStack itself (rather than our development infrastructure). They might need mentoring though, especially to understand gerrit/zuul/jobs which is a pretty essential part of the process (check jobs to check availability and file syntax, post-merge jobs to refresh the ICS and get it published in human-readable fashion). I can help as the "customer" expressing feature requests, but can't spend too much time mentoring. Let us know if they grab it so that we don't spend more time on it. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Week-end project
Sean Dague wrote: > I think there are 2 approaches that I see being fruitful, depending on > the kind of problem the team is going after. > > 1) the yaml -> ical converter. > > Bulk of invention is going to be on the converter, especially > translating into ical recurrance rules. Also will probably want / need > to build an HTML UI for the end result so people can actually see it on > a webpage as well. > > 2) drupal + calendar + workflow > > I was actually thinking about what ttx said about no tool existing out > there to be able to take calendar updates into an approval queue. I > think you could actually build that pretty easily with drupal base site > (logins connected to lp openid) + calendar modules + workflow module > (that allows for approval queues on changes). > > Different set of things to learn (more on the drupal side), however the > advantages would be that a lot of the UI and ical bits would be handled > already. So.. approach 2 would definitely be more friendly for non-devs, but what I like about option 1 (in addition to its lack of specific infrastructure setup) is that you can check the proposed change for future conflicts before it is even reviewed and at merge time, using the same check/gate mechanisms we have for everything else. It sounds a lot more difficult to set up such automated verification in the drupal case, so that would push the conflict validation onto the human reviewing the change... -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] IRC op team
Hi! There have been a few channel trolling situations in the past (in European mornings) where nobody present on the channel could solve the situation, and it made us look a bit bad. Current IRC chanserv op rights are limited to the infra-root team (4 people in the US, 3 of them on the west coast), while this is a relatively low-trust task for which geographic coverage and IRC presence should be the defining factor. In the short term, since adding ops is a slightly painful process, I'd like to suggest we add one person in Europe and one in India/China to that team to improve our op coverage. I'm volunteering for the Europe position. I've started using IRC in 1993, if that counts as experience, and I'm firmly entrenched in Europe CET :) Once we get the channel-configuring bot up and running, we can add more people to that specific team, and also fine-tune the rights associated with every single person (most people will just need op/akick/quiet for channel policing). While we are at it, we should also try to get an "openstack" Freenode group set up, if they ever reopen them. That would allow us to control #openstack* channels, and also to give out shiny openstack cloaks. Perl coders out there might be interested in speeding that up by helping out on http://dev.freenode.net/GMS development :) -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] IRC op team
James E. Blair wrote: > Thierry Carrez writes: > >> In the short term, since adding ops is a slightly painful process, I'd >> like to suggest we add one person in Europe and one in India/China to >> that team to improve our op coverage. I'm volunteering for the Europe >> position. I've started using IRC in 1993, if that counts as experience, >> and I'm firmly entrenched in Europe CET :) > > Under the circumstances, this seems reasonable. Is there a volunteer in > APAC? What channels would you like to cover? I currently follow #openstack-dev #openstack-infra #openstack-nova #openstack-packaging #openstack #openstack-foundation #openstack-fr and #openstack-meeting[-alt]. That said, there is value in being able to help in case someone from #openstack-foo comes begging for troll handling help in #openstack-infra at 0900 UTC (true story). So having a minimal number of #openstack* ops around the world is also helpful... But to avoid manual updating all the channels, maybe we should wait for the permission handling bot before establishing that * team ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] suggestions for gate optimizations
Sean Dague wrote: > I have a couple of ideas about things we should do based on what I've > seen in the gate during this wedge. > [...] All good ideas. I would add a couple of.. more aggressive suggestions to the mix: = Parallel assume-1st-fail pipeline = John Dickinson's suggestion. Assume first one fails and have a parallel pipe proactively anticipating that case, switching over in case that prediction verifies itself. = Run multiple OR tests in parallel for the same change = This one is more aggressive. The idea would be to run two sets of test jobs for the same change. For a given job, if any of the two runs succeed, consider it a PASS. Still track the failure as a data point to find and fix non-deterministic bugs. This would let non-deterministic bugs have less impact on the gate and not be blocking anymore. Yes, it lets NEW non-deterministic bugs more easily in, but at this point I would argue that the gate is blocking more legitimate patches than preventing bad ones from coming in. This would mostly be anticipating on what we do anyway in that case (reverify). Both of those options are costly in terms of node consumption. That said I think it doesn't really make sense to build a 100-long gate queue and assigning resources to that, when we know for almost-certain there is no way that 100th test will end up being significant. The gate queue could be 25-50 deep and work the same. The changes at the top of the queue are the ones that matter and which should be given extra resources and parallel testing. -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [StoryBoard] Wireframes for MVP0 (v. 0.2.0)
Jaromir Coufal wrote:
> I am sending you location for StoryBoard wireframes:
>
> http://people.redhat.com/~jcoufal/openstack/storyboard/
>
> I put together MPV0, which you can see as the only file there at the
> moment. I tried to reflect everything what was discussed in Brussels.
Great work, thanks !
> Please, feel free to discuss, comment or give feedback on everything
> what is in your mind.
Two remarks:
Stories:
About the tasks column (item 2), I think what makes the most sense from
a user perspective is to know the completion of the story, rather than
how much is not completed. I'm reading "1/4" like the story is 25%
completed, while with the current meaning (opened/total), it's actually
75% completed. So I would switch to a closed/total meaning ("3/4"), or
switch to a total / % complete approach ("4 (75% completed)") or even
use some visual fixed-width hint to convey both the number of items and
the completion of them (see [1] for an example).
[1] http://ubuntuone.com/0C879pLwfV1NCfrZP4CFU5
Story detail:
I don't really like the separation between task list, discussion and
activity. I still prefer to have everything on the same page: activity
mixed with comments in a single event log (like in the POC). I would
allow to submit comments when changing status. I would also use icons to
convey the event nature (task created, comment added... etc.). To see
how the POC handled it, see [2].
[2] http://ubuntuone.com/2FhNSgGhPQwaumBABO0gVk
Regards,
--
Thierry Carrez (ttx)
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Moving gantt to stackforge
Russell Bryant wrote: > The gantt repository was originally put under openstack/. I would like > to move gantt to stackforge to better reflect the current status of the > project. > > We don't actually expect anyone to use gantt as-is. The consensus seems > to be that we have quite a bit of work left to do inside of Nova before > we can regenerate the gantt repo and try for the forklift again. In the > meantime, there are some that are still interested in experimenting with > the current gantt repo. > > How do we go about doing this move? Should I file a bug to track it? > What can I do to help? I'm not sure the move is really needed. openstack/* shouldn't carry any notion of usability/stability, it just reflects that the project is an openstack project. Since gantt belongs to the official Compute program, it should be under openstack/*. The move to stackforge would make sense if it was developed by a nascent separate group, but I don't think that's the case here ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Moving gantt to stackforge
Joe Gordon wrote: >> The move to stackforge would make sense if it was developed by a nascent >> separate group, but I don't think that's the case here ? > > I think this is the case, currently the core group on gantt is > nova-core and Robert Collins, while the main commiter is Donald Dugger > (who asked for gantt to be kept around to play with). But now that we > have decided that we have no use for the gantt repo, I don't think it > makes sense for nova-core to take on the burdon of reviewing gantt > patches. I know that I won't be anymore (I think I have reviewed > every gantt patch to date). That's a fair point. In the end the granularity of a program is linked to the interest of its team in general and its core reviewers team in particular. If those don't overlap, you're better off with not having that code repo under your program. >From what you're saying that seems to be a "nascent separate group" forming, in which case I agree stackforge is probably the best bet. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Announcing a new infrastructure project, Vinz code review system.
Philip Schwartz wrote: > I am a member of a new release engineering tools team at Rackspace. Our > goal is to help build the next generation of tools to clean up our > internal development process along with the OpenStack development > process. With this in mind, I recently broached the topic of a > replacement system to Gerrit with jeblair, fungi, and mordred. This will > be a complete from the ground up replacement of Gerrit built completely > in python following the standards devised for the OpenStack projects. > The targeted name for the system will be Vinz in order to match the > naming of Zuul. > [...] Being fully aware of the irony, I'll play the devil's advocate: Could you detail why improving Gerrit is not an option ? At first glance its development seemed to be open enough, and it didn't appear fundamentally wrong (just needing a few incremental changes). -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Announcing a new infrastructure project, Vinz code review system.
Monty Taylor wrote: > Which means, although I tend to have a side which agrees with Clint and > Clark that replacing gerrit is a bit of a potential giant rathole - I > also think that making a scalable thing that architecturally fits with > the other things we've got would be nice. It would be nice. But I think there are two major differences between StoryBoard and Vinz from a development effort perspective... First, task tracking is fundamentally simple. As my POC proved, it's just a few database tables -- the complexity you add on top of it is just pure bonus. I see the basic code review functionality and the manipulation of associated git repositories as a much more complex endeavor. Second, I think Launchpad created a lot of developer itches that StoryBoard hopefully will allow them to scratch. So we can hope to see our developers help with StoryBoard in the future... I'm not sure Gerrit created enough pain so far. Yes it's ugly but it gets the work done pretty well. Could you point to a specific shortcoming that you can't address in the legacy app and that would be the killer feature ? In StoryBoard, that would be the ability to track cross-project features with tons of tasks. Don't get me wrong, I don't want to prevent anyone from working on anything they would like to. I'm just afraid that this is a large project and I don't see it attracting enough contributors to be a long-term sustainable alternative... potentially making it a gigantic distraction. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Moving some projects around
James E. Blair wrote: > We've been discussing the thoroughly interesting problem of project > taxonomy recently, and I'd like to describe what I think we've mostly > decided we should do and make sure we're on the same page. If we are, > I'll invite the TC to weigh in and if they think we're completely wrong, > we can work on some policy. > > Move the following projects to the openstack-attic/ org (because they > are no longer active projects though they had some degree of > officialness about them at the time they were active): > > * openstack-dev/openstack-qa > * openstack/melange > * openstack/python-melangeclient > * openstack/openstack-chef > > We should make these read-only as well. > > Leave openstack-dev/sandbox where it is. It's a useful tool for new > openstack developers and third party CI systems -- it should be > considered part of the openstack development process. OK > Move openstack/python-openstackclient to stackforge. It's apparently a > project without an official program. (Incidentally, this makes me sad.) > > Leave openstack/gantt where it is, but make it read-only. The current > state of development is considered a dead end, but there is likely to be > a future attempt (and therefore future openstack/gantt repository). > Rather than rename it and rename it back, or rename it to something like > "openstack-attic/gant-mark-one", we think mothballing it and then > replacing the entire branch with a merge commit for the second forklift > attempt (like we did with keystone-lite) is the least silly option and > actually faithfully represents development history. On one hand I think openstack/python-openstackclient could temporarily be left where it is until we have a clear idea of what its future is. On the other moving it to stackforge could release enough energy for a proper program request to be submitted. It's not a quick and easy discussion though: there are competing approaches, the question of whether all client libraries should also live under that program, and the question of whether it should be lumped in a bigger end-user UX program. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] establishing a standing policy for approving new oslo library imports
Doug Hellmann wrote: > Can we establish a standing policy that imports for new oslo libraries > won't happen until I vote +1 the changeset? I can try to keep up and > -1 until the team has a chance to review it, but I'm worried that I > may miss an update to a changeset. You mean for the requirements repository ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] establishing a standing policy for approving new oslo library imports
Doug Hellmann wrote: >> Doug Hellmann wrote: >>> Can we establish a standing policy that imports for new oslo libraries >>> won't happen until I vote +1 the changeset? I can try to keep up and >>> -1 until the team has a chance to review it, but I'm worried that I >>> may miss an update to a changeset. >> >> You mean for the requirements repository ? > > No, sorry I wasn't clear. I mean for the config repo for changes that create > new oslo source repositories. Ah! makes sense, yes. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] About openstack branching model and openstack branch stable/icehouse
Le Tian Ren wrote: > After review the 2 models, I have several questions: As OpenStack release manager (and author of said "OpenStack model") I'll try to answer those. > 1. OpenStack's master branch works as both the development and > master(production ready) branch of the NVIE model together, right? I > wonder 2 main branches vs 1 main branch, which one is better and why? We don't have an equivalent of NVIE's "master" branch. We use a set of stable branches instead. The main reason is that it maps better to our downstream consumers (OpenStack distributions), which typically would only issue bugfixes as updates, and jump from branch to branch when they issue a new version of the distribution. For that reason we don't have a single release channel (master) but multiple ones (stable/*). > 2. I noticed that there are stable/icheouse, stable/havana branches on > github web GUI of many openstack projects like nova, heat, ironic. To > what branches mentioned in the above 2 models can these stable/* mapped, > supporting release branches? hot fix branches? They would be like overlapping "master" branches. We apply hotfixes for them, and we generally support them longer then just until the next release. > BTW, if I git clone these > projects, I cannot see those stable/* branches with git branch.. why? See Jeremy's answer for that. > And when were they created in a OpenStack release cycle and what for? They are created at release time. Note that we are changing the process this cycle, though. We'll probably have a proposed/juno branch (NVIE's release branch) which then turns into stable/juno (~ NVIE's master branch). Hope this helps, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] proposed/* or stable/* for pre-release branches
Hi Infra, At the last summit we decided to drop milestone-proposed branches in favor of named release branches. There are two implementation options. * Use "stable/juno" starting from RC1 The main benefit of this approach is that the infrastructure should "just work" by applying testing to the pre-release branch just as it applies testing to the stable branch. The main drawback of this approach is that pre-release and stable branches generate different expectations (only the post-release stable/foo is actually "stable"), and have different processes, rules and teams responsible for them. * Use "proposed/juno" from RC1 to release and rename the branch "stable/juno" at release time The main benefit of this approach is that we can apply different ACLs, and document different processes and rules based on the name of the branch. The main drawback is that the infrastructure must be taught to use proposed/foo when stable/foo is not around. In various post-summit discussions, consumers of those branches generally expressed preference in keeping them named differently depending on the stage of the release we were in. Infra people generally agreed that supporting proposed/foo was not very complex, or at least not as weird as switching ACLs for stable/foo at release time. So it looks like we should proceed with supporting proposed/foo on prerelease, but I wanted to do a last-minute check to make sure everyone was still OK with that. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] What is the final manila merge point in Kilo?
liuxinguo wrote: > We want to add a manila driver in Kilo. > > I want to know what is the latest(final) time point we should submit our > driver or get our dirver merged in kilo, while guaranteeing our driver > can be merged in kilo successfully? > > Should we must submit our driver or get our dirver merged in kilo before > Kilo-1 at 18/12/2014 or we can do this later just before Kilo-2 at > 05/02/2014? Manila being an incubated project, the rules are slightly less strict than with a project that would already be integrated. I would ask directly to the PTL (Ben, CC-ed) to see how he wanted to organize his development cycle. Regards, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] introduce myself, wanna hack git-review
Thomas Koch wrote: > On Thursday, November 27, 2014 02:32:07 PM Thomas Koch wrote: >> What's the status of git-review development? There are over 300 open review >> requests on gerrit. Can you use help with git-review development? > > Sorry. Added status:open now to my search and found 14 open requests. I'm pretty sure they would still welcome the help :) Most git-review hackers are out for Thanksgiving, so don't expect an answer on your email until Monday... but that doesn't mean they aren't interested ! FWIW git-review bugs / feature requests are tracked using our experimental task tracker at: https://storyboard.openstack.org/#!/project/719 Welcome! -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Fwd: [openstack-dev] Session length on wiki.openstack.org
Cross-posting from -dev FWIW I agree with the request, the wiki session duration seems to have been dramatically lowered a few months ago. Message transféré Sujet : [openstack-dev] Session length on wiki.openstack.org Date : Fri, 5 Dec 2014 12:03:49 +1100 De : Tony Breeds Répondre à : OpenStack Development Mailing List (not for usage questions) Pour : openstack-...@lists.openstack.org Hello Wiki masters, Is there anyway to extend the session length on the wiki? In my current work flow I login to the wiki do work and then get distracted by code/IRC when I go back to the wiki I'm almost always logged out (I'm guessing due to inactivity). It feels like this is about 30mins but I could be wrong. Is there anyway for me to tweak this session length for myself? If not can it be increased to say 2 hours? Yours Tony. signature.asc Description: OpenPGP digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Issue with swift 2.2.1.rc1 tarball creation (setuptools)
Hi infra, I have been pushing the Swift 2.2.1.rc1 tag this morning in preparation for Swift 2.2.1 release this week, and it resulted in a swift-2.2.1c1.tar.gz tarball being generated (instead of the swift-2.2.1.rc1.tar.gz tarball that my scripts expected). Tracing back to the logs at: https://jenkins05.openstack.org/job/swift-tarball/1/consoleFull it appears that setuptools is warning against using 2.2.1.rc1 as a version, and then just takes 2.2.1c1 as version instead. So the "warning" is a bit... definitive: 2014-12-15 09:19:00.162 | venv runtests: commands[0] | python setup.py sdist 2014-12-15 09:19:00.561 | /home/jenkins/workspace/swift-tarball/.tox/venv/local/lib/python2.7/site-packages/setuptools/dist.py:289: UserWarning: The version specified requires normalization, consider using '2.2.1c1' instead of '2.2.1.rc1'. [...] 2014-12-15 09:19:00.948 | running check 2014-12-15 09:19:00.948 | creating swift-2.2.1c1 2014-12-15 09:19:00.949 | creating swift-2.2.1c1/bin At this point I have the tag in git and no tarball to upload to Launchpad and announce. Let me know what I can do (should we remove and re-tag with something different ? should we fix the -tarball job and retrigger it manually ?) Staying put, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] creating branch in gerrit using commandline
Christian Berendt wrote: > On 01/21/2015 08:17 PM, Vinay Mahuli wrote: >> We generally create branches in gerrit using the web ui accessing admin >> -> projects -> select project -> branches >> >> Is it possible to do it using command line? >> I need to automate creation of branch in gerrit with a script. > > I have not tested it, but this should be possible by using the Gerrit > API as documented at > https://gerrit-review.googlesource.com/Documentation/rest-api-projects.html#create-branch. > > A client to access the Gerrit API is available at > https://pypi.python.org/pypi/gerritlib. It's also possible to simply push the branch, if you have the required permissions in Gerrit. That's what the release script which creates proposed/$SERIES branch operates: http://git.openstack.org/cgit/openstack-infra/release-tools/tree/rccut.sh#n77 -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Cross-Repo Dependencies in Zuul
James E. Blair wrote:
> We have added support for cross-repo dependencies (CRD) in Zuul. The
> important bits:
>
> * To use them, include "Depends-On: " in the footer of
> your commit message. Use the full Change-ID ('I' + 40 characters).
Thanks for that!
I know that should make some StoryBoard devs quite happy.
--
Thierry Carrez (ttx)
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Biting the bullet on issue tracking
Monty Taylor wrote: > [...] > We're looking at what our options are, and Thierry is examining them to > see how tolerable their differences would be to our community. > > I propose that we have a solid answer and migration plan to put in front > of people by Vancouver at the latest. StoryBoard was started initially as a proof-of-concept of what we'd actually love to see in the tool we use for task tracking. I was writing requirements documents to help the Infra team look into alternate solutions, and finally decided that code could be worth a thousand words. At that point, people got very excited at the idea of having a tool that would precisely map our workflows and processes, rather than having a tool you have to adapt your workflows and processes to. Let's use the POC today! Since that initial excitement, three things happened. (1) We realized coding the task tracking part is not really long or difficult. It's all the boring infrastructure that is long and painful: subscriptions, configurable email notifications, ACLs... (2) We did not get the surge in contributors we expected to get. The StoryBoard API server is built like an OpenStack API server to increase dev familiarity, but HP and Mirantis were the only ones to dedicate headcount to the effort. (3) With StoryBoard being developed not as fast as we hoped, time passed, some requirements changed, new teams have needs, and those are not necessarily easily served with a beta tool. So we are standing at the crossroads again. First, we need to determine if we are ready to accept to use a tool that is not tailored-fit to our workflows, in exchange for more immediate gratification. That is what the "Biting the bullet" from Monty is about. It's not an easy thing to do... after all the reason we want to move away from Launchpad is the pain derived from using a tool that does not fit our workflows and processes. Second, we need to see which solution is the closest to "being usable for us", and therefore should be the way forward. When you work on a single project team, it's easy to overlook that we have a pretty unique set of needs in OpenStack -- the ability to efficiently track tasks across a large set of projects and branches (for our horizontal teams sanity). Not all tools can be used like that. In fact, before we started StoryBoard, Launchpad was still the best tool for that. Personally I'm not totally convinced StoryBoard is out of the race. We may realize that the amount of custom development needed to bring existing solutions to a point where we can use them for task tracking in OpenStack is superior to the amount of development needed to bring StoryBoard at an acceptable usability level. But then, I don't personally wield any significant development headcount, so I can't make the choice: I can only define what "usable for OpenStack" minimally means. It's also worth noting that Launchpad is not (yet) out of the game. It could grow the set of features we need (ability to auth using OpenStackID, multi-task features with tags and comments, task dashboards, no more silly timeouts, comprehensive API...). Unlikely, but still possible :) I look forward to that discussion in Vancouver on the future of task tracking in OpenStack. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Biting the bullet on issue tracking
JP Maxwell wrote: > I would be happy to deploy an instance on a server at Rackspace if you'd > like to login and have a look around. I'd like to have a look around yes -- if you could set up some instance and send me credentials that are admin-enough to be able to set up new projects and play around, that would be great. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Orphaned repositories in openstack* namespace
Andreas Jaeger wrote: > We noticed last week that bashate was not part of any project in the > governance repo and wondered on IRC which other repositories are > orphaned - meaning added in one of the openstack namespaces but not > mentioned in governance/reference/. Yes, I usually periodically run a script to check consistency, but that run was long overdue! Thanks for looking into that. > I did some quick checking and came up with the following list and > added current proposed changes and my suggestions: > > openstack-dev/bashate > Proposed for QA: > https://review.openstack.org/#/c/168012/ > > openstack-infra/ansible-puppet > Proposed for Infra: > https://review.openstack.org/#/c/166820/ > > openstack/gantt > openstack/python-ganttclient > Suggestion nova: > https://review.openstack.org/#/c/168682/ > > openstack/os-client-config > Proposed for openstackclient: > https://review.openstack.org/#/c/167730/ > > openstack-dev/ci-sandbox > openstack-dev/sandbox > openstack-infra/vinz > openstack-infra/vinz-webclient >Suggestion: Infrastructure > > openstack/ossa >Suggestion: Release cycle management > > openstack-dev/devstack-vagrant >Suggestion: QA > > openstack-dev/specs-cookiecutter >Suggestion: Oslo > > Do you agree with my suggestions? If yes, I can sent patches for the > governance repo. Yes. Thanks! -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Orphaned repositories in openstack* namespace
Jeremy Stanley wrote: > On 2015-03-30 07:32:44 +0200 (+0200), Andreas Jaeger wrote: > [...] >> openstack-dev/ci-sandbox >> openstack-dev/sandbox > > Adding these to an official project means we would convey > contributor status upon anyone merging a change to one of them. > Since they're configured to allow any registered user to > approve/merge changes this seems like it would be a quick loophole > for achieving ATC. Right, those were intentionally kept in limbo. Let's keep it tis way. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Orphaned repositories in openstack* namespace
Doug Hellmann wrote: > Excerpts from Andreas Jaeger's message of 2015-03-30 07:32:44 +0200: >> We noticed last week that bashate was not part of any project in the >> governance repo and wondered on IRC which other repositories are >> orphaned - meaning added in one of the openstack namespaces but not >> mentioned in governance/reference/. >> >> I did some quick checking and came up with the following list and >> added current proposed changes and my suggestions: > > [snip] >> >> openstack-dev/specs-cookiecutter >> Suggestion: Oslo > > The specs cookiecutter seems to be outside of the Oslo mission. > Maybe the release management team should own it? Works for me. Or we can make it a TC-owned repo. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking
erface with Launchpad, which is why
a complete and clean API was such a significant part in StoryBoard
design. It looks like the Maniphest story on APIs is actually worse than
Launchpad.
Could do better: Tracking feature development
-
A feature is a complex task for which you want specific rules to apply
(like tasks may only affect master branch). It's good to visually
separate them too, so that feature-frozen period clearly excludes them.
Communicating the % of feature completion is also valuable information.
Being able to order subtasks is a nice add too.
Maniphest has no concept of task types, although it could be emulated
with a "project" (we'd tag all features with a "feature" project), or we
could use a custom field for that. It doesn't have a concept of feature
completion status either. Ideally that would be derived from the number
of completed subtasks, or set by the user on the top task. The absence
of a story (or task group) might bite us here (since in Maniphest
everything is a task, and completion only makes sense on a task group).
Also you can't order subtasks in Maniphest at all.
Not sure how to solve this one, and improvements around feature tracking
are the #1 reason for StoryBoard to exist. But one could argue that it
won't be worse than using Launchpad anyway.
Could do better: ACLs for Vulnerability management
--
Vulnerability management is about the ability to file a report that is
originally only seen by its reporter and a specific team. Then we
gradually add people to the ACL as we pull them in to help with
resolution, up to the point where the bug is open.
While Maniphest has task ACL support, it is pretty crude. There seem to
be no way to encourage people to file security reports privately (they
have to find out and make use of the "Visible To" and "Editable By"
dropboxes by themselves). Then there is no way short of defining a
"Custom policy" for a user to say ("me and the VMT"). Then there is no
way (short of expanding the custom policy) for the VMT to add
individuals to the ACL.
For this one to work, we'd need a way to encourage people to file
vulnerabilities the right way (some sort of task creation template, or
could be a separate website for vulnerability reports), then a way to
make the bug visible/editable to "VMT and all subscribers". That way
people we add to the CC end up being in the ACL without having to play
with Custom Policies. A visual indicator of such bugs would also be nice
(embargoed issues should not be discussed openly, and sometimes it's
hard to remember the thing you are on is one of those restricted issues).
We'd probably work around the limitation: Release reporting
---
One of the things we use Launchpad for is to produce release pages
listing what features and bugfixes landed in a given release tag. There
is no support for that (at all) in Maniphest. So we'd have to switch to
git-log generated changelogs, but that may not be a crazy thing to do
anyway. Release reporting is quite separate from task tracking,
Launchpad just accidentally did both.
Hope this helps,
--
Thierry Carrez (ttx)
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Scheduling a monthly meeting
Tom Fifield wrote: > A couple of things I'm working on at the moment would like to meet on a > monthly basis. > > From irc-meetings: " Options for the `frequency` are `weekly`, > `biweekly-even`, and `biweekly-odd` at the moment.", and in > yaml2ical/recurrence.py [1] it's pretty clear that there's no support > for Monthly. > > So ... first question: is there some enormous trap that means adding > support to the code for monthly meetings is best avoided/really hard? It is somewhat by design. "Once-a-month" meetings basically consume as much meeting space as "weekly" ones in terms of the conflicts they generate. So not supporting it encourages people to pick biweekly options, which just consume biweekly space. I guess we could support "once-every-4-weeks" but that would likely be confusing. Better pick a biweekly spot and skip meetings liberally. > According to ICAL examples[2], it looks like monthly is basically done > as based on a day, and a week-of-month position: > [...] Right, there is nothing technical preventing to encode it in ICAL -- it just generates so much conflicts that we so far preferred to not offer the option. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Tracking repo/branches in Maniphest tasks, episode 1
Hi, I've been investigating how we could track the repository and the branch where a given task should land in Maniphest. This is a feature gap we have compared to Launchpad, and being able to clearly track complex stories with subtasks that affect different projects/branches is a key requirement for OpenStack task tracking. Ideally the targeted project/branch would appear in the subtask lists, as well as be searchable. In this episode, I looked into how we could leverage CustomFields to achieve that. In particular, CustomFields have a renderOnListItem() method which promised to give you a handle on how tasks are represented in lists. However, I quickly realized that this method is actually never called for ManiphestCustomFields, so a custom field can't affect how a task appears in list views. I then tried to see how we could alter the views so that the content of a specific custom field of a task would appear in the displayed task name, but that is non-trivial due to the way custom fields are hooked into an object. The object itself basically can't directly access the value of custom fields (Maniphest does it from the presentation layer), which makes it rather difficult to display the value of a custom field of a linked object (like the project/branch of a subtask). I'm still investigating this, and I suspect it is possible, but I fear it will be a heavy patch that we'd likely not carry (and that upstream is not likely to accept). Stay tuned for episode 2... -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Tracking repo/branches in Maniphest tasks, episode 2
Previously in Phabriland:
In episode 1, our hero realized that the Phabricator custom field
framework could not be leveraged to display repository and branch in
task lists and subtask lists, and was stalled coming up with a
reasonable patch to implement it.
Spent some more time on this, and it's actually simpler than I thought
to patch Phabricator to support this. It may even be a reasonable patch
to carry. See attached.
The remaining issue is that the "repository" field is likely to contain
hundreds of potential values, and the built-in "select" custom fields do
not display them in a very convenient manner (infinite dropdown in edit
views, hundreds of checkboxes in search views). Also manually listing
all potential values in custom field configuration is impractical. For
all those reasons, implementation would have to go through a proper
CustomField class, which is likely to take a bit more time, but still be
possible.
So at this point the question is more... can we represent target
repositories and branches using Phabricator "projects", or should we
rely on custom fields ? Wikimedia restricts creation of "projects" to a
trusted set of users to keep the taxonomy there under strict rules[1].
So that might render them usable, at the price of extra bureaucracy to
create them. I still need to wrap my head around what this would look
like for the 3 typical cases: a simple backport request, an
issue/feature affecting multiple projects, and a security bug (which
basically combines both).
Random additional note: in Phabricator, you file a bug at the top level,
and there is an explicit triage phase to get it on the right team work
pile. The bug filing is not "per project" (like in Launchpad). It is
possible to add a project when you file a bug but it's totally optional.
This creates a problem in OpenStack where there is no default
cross-project "triaging" team. And I'm pretty sure we would have a hard
time to find volunteers for this one. Not sure how to solve this one.
While I was looking into Wikimedia: for security, our friends there
implemented something pretty close to what we need, as an extension:
https://git.wikimedia.org/tree/phabricator%2Fextensions%2Fsecurity.git
Next up on my list, looking into the CLI and API.
[1]
https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects#Type_of_project
--
Thierry Carrez (ttx)
diff --git a/src/applications/maniphest/phid/ManiphestTaskPHIDType.php b/src/applications/maniphest/phid/ManiphestTaskPHIDType.php
index 719b4a8..bb219a4 100644
--- a/src/applications/maniphest/phid/ManiphestTaskPHIDType.php
+++ b/src/applications/maniphest/phid/ManiphestTaskPHIDType.php
@@ -33,9 +33,10 @@ final class ManiphestTaskPHIDType extends PhabricatorPHIDType {
$task = $objects[$phid];
$id = $task->getID();
$title = $task->getTitle();
+ $repo = $task->getRepoPrefix();
$handle->setName("T{$id}");
- $handle->setFullName("T{$id}: {$title}");
+ $handle->setFullName("T{$id}: {$repo}{$title}");
$handle->setURI("/T{$id}");
if ($task->isClosed()) {
diff --git a/src/applications/maniphest/storage/ManiphestTask.php b/src/applications/maniphest/storage/ManiphestTask.php
index 7fbc806..49f17f6 100644
--- a/src/applications/maniphest/storage/ManiphestTask.php
+++ b/src/applications/maniphest/storage/ManiphestTask.php
@@ -153,6 +153,32 @@ final class ManiphestTask extends ManiphestDAO
return $this->assertAttached($this->edgeProjectPHIDs);
}
+ public function getRepoPrefix() {
+$viewer = PhabricatorUser::getOmnipotentUser();
+
+$field_list = PhabricatorCustomField::getObjectFields(
+ $this,
+ PhabricatorCustomField::ROLE_EDIT);
+
+$field_list
+ ->setViewer($viewer)
+ ->readFieldsFromStorage($this);
+
+$field_value = null;
+foreach ($field_list->getFields() as $field) {
+ $field_key = $field->getFieldKey();
+
+ if ($field_key == 'std:maniphest:openstack:repository') {
+$field_value = $field->getValueForStorage();
+break;
+ }
+}
+if ($field_value == '') {
+return '';
+}
+return "[{$field_value}] ";
+ }
+
public function attachProjectPHIDs(array $phids) {
$this->edgeProjectPHIDs = $phids;
return $this;
diff --git a/src/applications/maniphest/view/ManiphestTaskListView.php b/src/applications/maniphest/view/ManiphestTaskListView.php
index 05adced..0fa9548 100644
--- a/src/applications/maniphest/view/ManiphestTaskListView.php
+++ b/src/applications/maniphest/view/ManiphestTaskListView.php
@@ -61,7 +61,7 @@ final class ManiphestTaskListView extends ManiphestView {
->setUser($this->getUser())
->setObject($task)
->
Re: [OpenStack-Infra] Tracking repo/branches in Maniphest tasks, episode 3
In the first two episodes, our hero concentrated on modeling complex tasks (the ones that involve multiple projects and branches) using Phabricator, with varying degrees of success. Today I did some more investigation on that front and summarized my findings on the wiki page: https://wiki.openstack.org/wiki/Phabricator TL;DR: is: I think we could make it work using a hefty dose of custom development and patch-carrying. Today I also looked into the CLI and API. The CLI is extremely limited as far as Maniphest goes -- you can query tasks assigned to you and that's about it. You can also use it as a proxy to send API calls. The API is pretty weak at this point, and that has me worried. Partial API support is one of the reasons we want to move off Launchpad, and Phabricator seems even worse in that respect. For example, while you can create a task, subscribe people, set status etc. using the API, you can't define the relationship between the tasks. That means we won't be able to bypass some of the repo/branch tracking oddness by writing helper tools that will do the boring work of setting up multiple interrelated tasks for you. It's a known issue, apparently not trivial to fix: https://secure.phabricator.com/T5873 I wouldn't say this is a blocker (and I expect the Phabricator folks to come round and fix it one day), but something we need to take into account in our decision. This concludes the time I allocated to investigate how far Phabricator is from covering OpenStack needs. Hope this was helpful. Cheers, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] New storyboard core reviewers
Hi! As you know, StoryBoard development was mostly abandoned by its original development team, following the Infra team decision to no longer make it its long-term strategy for OpenStack task tracking. However, development was recently rebooted by two new contributors: Adam Coldrick (SotK) and Zara Zaimeche (Zara_). Their contributions are now blocked by the lack of core reviewers. I approved what I could (in openstack-infra/storyboard). Their changes and reviews sound solid. But that is not really the question: since they are the only two active developers on StoryBoard today, I think both should be core reviewers for those repositories. The alternative is to put them into a world of change approval misery as they try to convince people who have moved away from StoryBoard to pay enough attention to it to allow them to merge their changes. There is only so much pain you can endure: it will ultimately result in them forking their future development on another platform. To avoid that, I'd rather reboot the team completely and give them the keys. Thoughts ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] New storyboard core reviewers
James E. Blair wrote: > I think that sounds good. My main concern is that storyboard stay > stable and usable at least as long as we are using it. If they can > support that, I'm happy to add them to core. I suspect this won't be a > problem since they are also using it in production. I think it's fair for infra-core to quickly revert commits that would break our production instance. That sounds like a reasonable compromise: let devs make fast progress but still keep our continuous deployment to storyboard.o.o. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Migrating existing projects in the stackforge namespace
James E. Blair wrote: > As mentioned previously[1], we are retiring the stackforge/ namespace > for git repositories and creating new projects in openstack/. This is > largely a cosmetic change and does not change the governance model for > new projects. > > As part of this we want to move all of the projects that are currently > in the stackforge/ namespace into openstack/ to make it easier for them > to become official OpenStack projects in the future while reducing the > impact to the community that the current practice of sporadic renaming > causes. > > To that end, I propose the following process: > > 1) We choose a date upon which to perform a mass migration of all > stackforge/ projects into openstack/. > > I suggest either October 17 or November 7 (both Saturdays), as least > likely to interfere with the release or summit. > > 2) We create a wiki page for all such projects to either sign up for > that migration or indicate that they are no longer maintained. > > 3) Any stackforge projects that do not sign up for the migration within > a certain time are placed on the list of projects that are no longer > maintained. > > 4) We attempt to contact, by way of posts to the openstack-dev mailing > list, announcements at the cross project meeting, and direct emails to > the individuals who initially requested repository creation, people who > might be responsible for projects which have not responded and ensure > that they have a chance to respond. We will freeze the list of projects > and portions of the project-config repository several days before the > migration, to facilitate creating and reviewing the necessary change. > > 5) On the migration date, the Infrastructure team will move all of the > projects at once. We will generate the changes needed to do so > automatically, individual projects will not need to do anything except > approve .gitreview changes and possibly help fix any CI jobs that break > as a result of the moves. > > 6) For the projects that are no longer maintained, we will merge changes > to them that indicate that and make them read-only. > > We will schedule a move in early September for the projects that have > already requested moves as part of becoming official OpenStack projects. > Please don't propose any more changes to move projects before the mass > migration. > > While most new projects are being created directly in the openstack/ > namespace, we will continue to create additional git repositories > associated with existing projects in the stackforge/ namespace so that > the constituent repositories associated with those projects are not > split across namespaces. We will happily move those projects along with > the rest as part of the mass migration. > > Please reply with any feedback or suggested improvements to this plan. > If we can achieve consensus on the approach, we will make further > announcements as to specifics soon. Sounds good. I like that we take the opportunity to clean up abandoned projects: since there is no "project team owner" for those projects, it's easy to create one and abandon it. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [infra] Infra Design Summit Schedule
Anita Kuno wrote: > On 10/14/2015 07:13 PM, Jeremy Stanley wrote: >> Based on feedback from our last meeting and the discussion and >> subsequent voting in our session ideas pad, I've taken a first pass >> at scheduling: >> >> http://mitakadesignsummit.sched.org/type/Infrastructure >> >> I did the best I could to avoid obvious conflicts where necessary >> participants were likely to be involved in other summit tracks or >> conference talks, but due to the compressed nature of this round >> there's some unavoidable overlap (notably with QA, Docs, Ansible, >> Oslo, TripleO, and Ironic). >> >> Also, since it ended up making more sense to collapse the Storyboard >> and Maniphest workroom sessions into the Task Tracking fishbowl >> session (individually they had fewer votes, and otherwise we either >> had to conflict with QA or do the corresponding workrooms before the >> fishbowl), we really only had three workroom topics so I've proposed >> spending our back-to-back Wednesday sessions on the most popular of >> the three: Masterless Puppet. I have confidence you'll let me know >> if this is insane, and in that case provide alternative suggestions! The Task tracking session is at the same time as the talk Doug and myself are giving on the conference side... I'd really like to be able to attend that session though -- any chance you could move that one to another of the fishbowl slots ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] new-project swauth
Lisak, Peter wrote: > Definitely we also try to apply to start a new project team (our > developers with original project author). You might also want to consider developing on it within the existing Swift project team (a given project team can produce multiple repositories). You might want to get in touch with John Dickinson (the Swift project team lead) and discuss that with him (if you haven't already). Regards, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] list description
Spencer Krum wrote: On Mon, Feb 8, 2016, at 01:18 PM, Elizabeth K. Joseph wrote: On Mon, Feb 8, 2016 at 11:54 AM, Jeremy Stanley wrote: On 2016-02-08 11:11:39 -0800 (-0800), Elizabeth K. Joseph wrote: [...] "Development and maintenance of the project infrastructure and tooling used by contributors to develop OpenStack." I'm cool going with that as a new list description, though I'm not entirely convinced that newcomers skimming quickly looking for a place to get help with general OpenStack problems won't still confuse "the project infrastructure" with "the bits you're trying to install to run your OpenStack-based cloud infrastructure" since the word "infrastructure" is thrown around in so many contexts now it's essentially a meaningless industry fluff term. I guess what I was looking for was a non-circular definition for our particular use of "infrastructure" but that might just be expecting too much. Short of changing the name of the team itself (nooo) I'm not sure we will ever get away with people not understanding at a glance :) I think renaming the team to something that is more of a proper noun is a good idea. It would help with a lot of confusion. You could generalize usage of "Infra" as a team name and phase out usage of "Infrastructure". That would avoid a lot of renaming. """ The "Infra" team is the horizontal team developing and maintaining the systems and tooling used by contributors to develop OpenStack itself. """ -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Feedback after the infra midcycle
Spencer Krum wrote: Things that went well: * Many people expressed that focusing on a single topic made the event great. I think that's a very good point. I think we had three types of midcycle events. In the first one you meet to get together to know each other better and converge on a common goal. I call them "team building" midcycles. In the second one you have a specific objective and leverage high-bandwidth and increased focus to get there a lot faster than you would online. I call those "sprints". The third one was a bit specific to the infra team, where you would train people on the art of infra... I would call that "onboarding" or "training". My hope is that with the event split we won't need as much "team building" ones, but I can certainly see "sprints" (online or f2f) and onboarding events to persist. The key benefit is that it's easier to decide if you should come when you precisely know what will be covered and decided there. For example I opted to skip the infracloud sprint, while I may have felt compelled to attend a more generic "Infra midcycle" for fear of missing out on critical decisions. [...] * This is the first infra sprint that wasn't focused on 'infra 101' training for brand new contributors. Several attendees said they felt they got a lot more work done at this sprint than when bootstrapping was a topic. Well, we actually had a "storyboard"-focused sprint some time ago that was also not about training, and was also very productive :) -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack
Jeremy Stanley wrote: On 2016-03-25 10:51:57 -0700 (-0700), Elizabeth K. Joseph wrote: [...] 1. Spammer moved How_To_Contribute to a 555-5p4m-number-woo page 2. Spammer replaced the content with spam 3. SmitSpam deleted the page as spam [...] Great point, we need to be mindful of page moves too, so that does complicate matters. Also we should probably be marking high-value pages like How_To_Contribute locked to wiki admins anyway (we already do that for some of them since long before the current rash of defacements). Great point... We don't have much reference content left on the wiki, but we should probably protect whatever is left (while we continue the effort of replacing those pages by properly peer-reviewed sites). -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [kolla] making stable/mitaka == stable/liberty
Michał Jastrzębski wrote: So reason I don't really like having 2 different versions of openstack is because it's messy. That means having optional 2 different paths of deployment, and while they are mostly the same, there are subtle differences. My initial patchset actually did deploy 2 versions of openstack, but that require manual labor of configuring build config and lots of "if liberty, else" which lowered both code readibility and reliability, as it's additional logic. Then this is policy decision as we, kolla community, generally want to deploy N relase, so liberty deploys liberty and so forth. If we create 2 deploys per release, that will cause mess. Another thing is we specifically don't want people to deploy current stable/liberty because it is well, not stable. And it's not stable in potentially very destructive way. We want to discourage anyone from deploying current stable liberty to a point of actually removing this branch in favor of mitaka code. Doug, while I understand your reluctance, it is ugly thing, this is where we are and our first (I don't know if I can speak for everyone, but at least for me) priority is quality of deployment we provide. Bending the rules and policies is worth it if the improvement is this big, and potentially can save people from catastrophic failure and data loss. The thing is, supporting two versions of OpenStack in a single branch of Kolla *is* what you're doing here, only in catastrophic mode. You are basically rebuilding a liberty branch from mitaka code + 1-3 patches, because stable/mitaka is closer where you want to be than stable/liberty. So in essence you are using a single branch with conditionals, you are just using branches rather than if statements. My main gripe is that I don't see why this situation would not happen again, and why the same solution wouldn't be have to be applied again... Could you explain the steps you are taking that would prevent such a situation to happen in the future, to the point where maintaining a single code base wouldn't be a better solution ? I don't mind that much for stable/liberty, since it predates the "officialness" of Kolla: I'm fine with any solution there. I'm more concerned about stable/mitaka and going forward. Stable branches are supposed to be known quantities, slow moving and safe changes. What you're proposing wouldn't be an option for stable/mitaka -- so I'd like to make sure we don't find ourselves in a similar situation in the future. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [kolla] making stable/mitaka == stable/liberty
Steven Dake (stdake) wrote: I don't mind that much for stable/liberty, since it predates the "officialness" of Kolla: I'm fine with any solution there. I'm more concerned about stable/mitaka and going forward. Stable branches are supposed to be known quantities, slow moving and safe changes. What you're proposing wouldn't be an option for stable/mitaka -- so I'd like to make sure we don't find ourselves in a similar situation in the future. Going forward this will not happen again. I agree a stable branch should be a slow rate of change repository but also a safe place to rely on. At present stable/liberty is completely unsafe, so much so that I personally wrote a document in our docs.oo pages that says "DON'T UE IT". We are only backporting high/critical bugs which are legitimate bugs and not features to our stable branches. Note we will be backporting any gate changes, (which could be considered "features", but this is to simplify our lives so we don't have different gates for different versions of OpenStack. The main reason this won't happen again is we have decoupled the ansible 1.9 pin on docker 1.8.2. Now we can use any version of Docker we need, which is 1.10 and greater. Hope the context is useful. OK, as I said, liberty is pre-official so I'm fine with any solution here. We seem to be in agreement that the proposed solution should not be applied on stable/mitaka and later. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Infra priorities and spec cleanup
Jeremy Stanley wrote: On 2016-06-21 17:22:15 +1000 (+1000), Joshua Hesketh wrote: Good update, thanks fungi. Just a thought, given the pain we felt yesterday when static.o.o was down, we should consider if a log solution needs to be a priority. Using afs (or swift) could allow us to scale static.o.o horizontally. Yes, and it's not the only lengthy static.o.o outage we've had over the past month either. I agree that solving it is a good candidate for prioritization, but we need to go back and choose between a couple of options on the table there. It hurts a lot when it's down because of so many services being served from it. We could also separate the published websites (status.o.o, governance.o.o, security.o.o, releases.o.o...) which require limited resources and grow slowly, from the more resource-hungry storage sites (logs.o.o, tarballs.o.o...). -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Infra priorities and spec cleanup
Jeremy Stanley wrote: On 2016-06-21 17:34:07 + (+), Jeremy Stanley wrote: On 2016-06-21 18:16:49 +0200 (+0200), Thierry Carrez wrote: It hurts a lot when it's down because of so many services being served from it. We could also separate the published websites (status.o.o, governance.o.o, security.o.o, releases.o.o...) which require limited resources and grow slowly, from the more resource-hungry storage sites (logs.o.o, tarballs.o.o...). Agreed, that's actually a pretty trivial change, comparatively speaking. Oh, though it bears mention that the most recent extended outage (and by far longest we've experienced in a while) would have been just as bad either way. It had nothing to do with recovering attached volumes/filesystems, but rather was a host outage at the provider entirely outside our sphere of control. That sort of issue can potentially happen with any of our servers/services no matter how much we split them up. I don't think it would have been just as bad... Even in the unlucky case where the VMs end up on the same machine and are all affected, IIUC rebuilding some of them would have been much faster if they were split up (less data to rsync) ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Tri-Weekly IRC meetings?
Stig Telfer wrote: > In the Scientific WG we now have 3 co-chairs - one in Australia, one in the > USA and one in the UK. We are interested in arranging IRC meetings in 3 time > zones, ie tri-weekly time slots. Is that achievable? > > One potential issue I can see is that scheduling a tri-weekly slot would > conflict with other meetings on a bi-weekly spacing. > > Any thoughts on doing this? I assume it isn’t the first time this request > has come up. Our scheduling system currently only supports biweekly or weekly meetings. We don't currently support monthly or triweekly meetings (mostly to avoid having to detect complex conflicts with biweekly meetings). I guess we could evolve openstack-infra/yaml2ical to detect more complex conflicts. We'd gain in scheduling flexibility, but lose in our ability to pack the schedule tightly and make it pretty difficult to find empty slots manually. Any chance you could express what you are after using only biweekly meetings ? Like schedule odd Mondays, odd Thursdays and even Tuesdays ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Tri-Weekly IRC meetings?
Tony Breeds wrote: > On Wed, Nov 09, 2016 at 10:56:38AM +0100, Thierry Carrez wrote: > >> Our scheduling system currently only supports biweekly or weekly >> meetings. We don't currently support monthly or triweekly meetings >> (mostly to avoid having to detect complex conflicts with biweekly meetings). >> >> I guess we could evolve openstack-infra/yaml2ical to detect more complex >> conflicts. We'd gain in scheduling flexibility, but lose in our ability >> to pack the schedule tightly and make it pretty difficult to find empty >> slots manually. > > Yeah I think we should add monthly (which would more-or-less cover triweekly) > In that you could have a monthly-1st, monthly-3rd and monthly-4th events. > --- > - time: '1400' > day:Monday > irc:openstack-meeting > frequency: monthly-1st # 1400 on the 1st Monday of the month > - time: '2100' > day:Thursday > irc:openstack-meeting-alt > frequency: monthly-3rd # 2100 on the 3rd Thursday of the > month > - time: '0930' > day:Thursday > irc:openstack-meeting-alt > frequency: monthly-4th # 0930 on the 4th Thursday of the > month > --- > > It isn't quite a tri-weekly rotation but may be close enough. The issue with this solution is that any "monthly" slot ends up being exactly the same as a "weekly" slot: you can't schedule any weekly/biweekly meetings at the same time and location. Monthly meetings are therefore paradoxically more wasteful than biweekly meetings, and there is therefore little point in adding them to the software since you can emulate them with "weekly" bookings and just skip most meetings. Instead of monthly-1st/2nd/3rd/4th we could have monthly-1st-odd/1st-even/2nd-odd/2nd-even. That would make them slightly less wasteful (monthly-*-odd things do not conflict with biweekly-even things). But they would be pretty complex to interpret and then why not just use biweekly-* things and skip one every two meetings ? > Of course adding this would mean adding tools to grok the data ad see if a > meeting slot is open > > search-open --day Monday --time 1000 --freq weekly ? > > Which could list the available irc channels or nothing. Later it could be > extended to list "nearby options" We'll probably have to add a meeting channel as well, since that will create wasted slots (places where you can't put a weekly or biweekly meeting). -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Tri-Weekly IRC meetings?
Blair Bethwaite wrote: > On 10 Nov 2016 8:56 PM, "Thierry Carrez" <mailto:thie...@openstack.org>> wrote: >> >> The issue with this solution is that any "monthly" slot ends up being >> exactly the same as a "weekly" slot: you can't schedule any >> weekly/biweekly meetings at the same time and location. Monthly meetings >> are therefore paradoxically more wasteful than biweekly meetings, and >> there is therefore little point in adding them to the software since you >> can emulate them with "weekly" bookings and just skip most meetings. > > That seems to be based on the assumption that other groups wouldn't want > monthly scheduling? > > Also, I'd suggest we avoid "monthly" altogether and instead use > "4weekly", that would seem to allow biweekly meetings to use the same > day&time on the alternate (odd/even) week. 4weekly sounds like a good idea! Less conflicts ftw. If we really really want monthly stuff, we could create a channel dedicated to monthly meetings, to avoid creating conflicts on every channels. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Tri-Weekly IRC meetings?
Tony Breeds wrote: > I feel like there'is a tension here that we need to work through > > is yaml2ical a library for others (outside OpenStack) to use? If it is and > *they* want monthly meetings then I really think we shoudl include them. The > conflict deection isn't *that* hard and the need for other recurrence types > would tidy the code. So form this POV I htink we shoudl certainly allow for > this. I think I heard of one other group using yaml2ical... But the request to better support infrequent meetings is coming from within OpenStack at this stage. > Within OpenStack, we have several groups that want Monthly meetings so we > shoudl work out a way to support that. > > What we do right now is schedule a weekly meeting and then add a description > to > say "we only hold this on thr nth $day" [1] > > We could create a 4weekly which would reduce the wastage. It might be our best bet. > I'm also open to > creating #openstack-meeting-monthly and just moving all the monthly meetings > there. I feel like we get asked for a solution to this often enough that we > need to define an answer. That could work too, but somehow would encourage teams to only meet every month (since that is where it will be easier to schedule a meeting) which is not necessarily a best practice in short development cycles like in OpenStack. My preference would be to create a "4weekly" recurrence. The only drawback I see is that it's not trivial to determine when the next meeting is. Blair wrote: > At least for the scientific-wg, what we're actually looking for is > 4weekly rather than monthly. I'd be surprised if there's a big call > for monthly given peoples calendars generally work on a weekly basis > for other things...? Maybe we should engage with other currently "monthly" meeting chairs and ask them how happy they would be with a 4weekly ? Because if they don't plan to switch to it, there is little point in pursuing that option ? -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] How to take over a project? (tap-as-a-service)
Takashi Yamamoto wrote: i want to take over tap-as-a-service project. originally the project had two leaders. but one of them (vinay) has not been responding these days. another (anil) recently told us he might not work on it anymore. this email has cc: to them. i think gerrit is ok as i'm already a member of tap-as-a-service-core group. however, for launchpad, i'm not sure who can change the permissions. can the infra team help? the LP project is this one: https://launchpad.net/tap-as-a-service The infra team can't help much with that. The Launchpad project is owned by Anil Rao (although Vinay Yadhav might be able to change that as the original registrant), and the OpenStack Administrators was never added as a co-owner. Anil: any chance you could update the maintainer ? Thanks in advance, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [infra] Behavior change in Zuul post pipeline
James E. Blair wrote: [...] 2) Races in artifact build jobs will no longer result in old versions of documentation being published because they ran on a slightly faster node than the newer version. Great news! That was a standing issue for releases.openstack.org around release time :) -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Retiring some repos?
Andreas Jaeger wrote: On 2018-08-28 23:50, Clark Boylan wrote: On Sun, Aug 19, 2018, at 9:47 AM, Andreas Jaeger wrote: A quick search with codesearch suggests that these can be retired since I couldn't find a user: [...] openstack-infra/releasestatus (together with puppet-releasestatus, not listed above) AFAIK we can remove these two as well - looking at: https://review.openstack.org/254817 Yes releasestatus is not used anymore. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [Release-job-failures] Release of openstack-infra/jenkins-job-builder failed
z...@openstack.org wrote: Build failed. - trigger-readthedocs-webhook http://logs.openstack.org/22/22151762d1147da9bbbe9353fe52c6995ab8b658/release/trigger-readthedocs-webhook/48b0996/ : FAILURE in 1m 32s - release-openstack-python http://logs.openstack.org/22/22151762d1147da9bbbe9353fe52c6995ab8b658/release/release-openstack-python/6d983bb/ : SUCCESS in 3m 51s - announce-release http://logs.openstack.org/22/22151762d1147da9bbbe9353fe52c6995ab8b658/release/announce-release/a9ca9ee/ : SUCCESS in 3m 50s - propose-update-constraints http://logs.openstack.org/22/22151762d1147da9bbbe9353fe52c6995ab8b658/release/propose-update-constraints/453d5a1/ : SUCCESS in 3m 21s Looks like the readthedocs integration for JJB is misconfigured, causing the trigger-readthedocs-webhook to fail ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] OpenDev git hosting migration and Gerrit downtime April 19, 2019
Andreas Jaeger wrote: On 17/04/2019 02.14, Clark Boylan wrote: Fungi has generated a master list of project renames for the openstack namespaces: http://paste.openstack.org/show/749402/. If you have a moment please quickly review these planned renames for any obvious errors or issues. I wonder about these three, are they missing in some governance repo list? openstack/governance-website -> x/... This one is driving governance.openstack.org/ so it is an oversight that it does not appear anywhere. We can add it to the TC-owned repos, or to the Meta SIG repos, so that it can be kept under openstack/. openstack/openstack-map -> x/openstack-map This one is owned by the OSF to drive the openstack.org/software website and the OpenStack Map content. I would move it to osf/openstack-map. A few others: openstack/four-opens should also be under osf/. openstack/edge-computing-group should probably be added as a FEMDC SIG repo and stay under openstack/. openstack/arch-wg should be added under TC-owned repos and stay under openstack/. If that works for everyone, I'll propose the corresponding changes. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] OpenDev git hosting migration and Gerrit downtime April 19, 2019
Clark Boylan wrote: Fungi has generated a master list of project renames for the openstack namespaces: http://paste.openstack.org/show/749402/. If you have a moment please quickly review these planned renames for any obvious errors or issues. One thing that bothers me is the massive openstack-infra/ -> openstack/ rename, with things like: openstack-infra/gerrit -> openstack/gerrit Shouldn't that be directly moved to opendev/gerrit? Moving it to openstack/ sounds like a step backward. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] OpenDev git hosting migration and Gerrit downtime April 19, 2019
James E. Blair wrote: My understanding is this is due to openstack-infra being TC-governed and opendev not quite having gotten around to establishing an official non-TC governance yet. I think the intent is to eventually do that. We could probably anticipate that a bit if we would like and go ahead and sort openstack-infra things into different buckets. At the end of this, I think we will all have more hats, with overlap between opendev and openstack. Some current infra activities and repos are openstack-specific and should be re-homed into openstack; others serve all projects and should be in opendev; yet more are just things that are incidentally related to what we do and should be on their own. I've produced a list based on my estimation of what things will look like at the end of the process. This is just a starting point if we would like to explore this option. We could refine the list and use it, or we could choose to stick with the status quo temporarily and move the infra repos out of openstack at a later time when things are more clear. https://etherpad.openstack.org/p/6CmVhW40m0 I understand it conflates two separate issues (rename and governance), and we clearly don't want to delay the rename due to premature governance discussions. My personal view on this is that we can split openstack-infra things between openstack/ and opendev/ following your etherpad plan, and officialize the governance part later. Technically, things split under opendev/ would still be under the OpenStack infra team and the OpenStack TC until the governance is officially split out. That avoids unnecessary renames and crowding our new clean openstack/ space with stuff we know will move away soon. We'll obviously have to edit the projects.yaml to account for the repo renames under the Infrastructure team, but we have to do that anyway. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [release][infra] Supporting rget in our release process
Jeremy Stanley wrote: [...] For artifacts we upload to third-party services like PyPI and Docker Hub on the other hand, assuming I've digested (pun intended) the relevant literature correctly, it might make more sense for the maintainers of those services to do something similar as they tend to perform a fair amount of URL indirection and so trying to keep up historical data for those URLs ourselves could be tricky. On the other hand if those third-party services were to integrate rget updating as part of their infrastructure it would be a lot more seamless (especially if they similarly integrated CT checks into the corresponding client-side tooling). Another challenge I see is that, due to the fact that most of what we host is source code, and most consumers of our source code are obtaining it via Git rather than release artifacts, rget wouldn't really do much for them as far as I can see... though once Git completes its planned transition to SHA2-256 in the coming years, I could see a call for some solution to publish known object hashes to a CT log in a similar fashion. I suppose it could be done now by re-checksumming all content over a Git object and submitting a certificate for that, but it seems a bit heavy-weight and I'll admit to not having thought through it completely so there are likely hidden gotchas with that idea. I agree with Jeremy, it seems to cover a limited amount of use cases (people who download tarball source releases from releases.o.o). But covering only a few use cases is not a reason not to do it: we should support it for the same reason we provide signatures for released artifacts right now. Furthermore it is an initiative I'm fine being early adopters of this idea, if only so that one day we may find it covering other ways to retrieve our deliverables (pypi, git). -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Retiring stale repositories from the OpenStack org on GitHub
Hi everyone, The migration of our infrastructure to the Opendev domain gave us the opportunity to no longer have everything under "openstack" and stop the confusion around what is a part of OpenStack and what is just hosted on the same infrastructure. To that effect, in April we transferred non-OpenStack repositories to their own organization names on Opendev, with the non-claimed ones being kept for the time being under a "x" default organization. One consequence of that transition is that non-OpenStack repositories that were previously mirrored to GitHub under the "openstack" organization are now stale and no longer updated, which is very misleading. Those should now be retired, with a clear pointer to the original repository on Opendev. Jim and I volunteered to build tools to do handle that retirement and we are now ready to run those Thursday. This will not affect OpenStack repositories or repositories that were already retired or migrated off the OpenStack org on GitHub (think openstack-infra, opendev, airship...). That will only clean up no-longer-mirrored, stale, non-openstack repositories still present in the OpenStack GitHub organization. If you own a non-openstack repository on Opendev and would like to enable GitHub mirroring (to a GitHub org of your choice), it is possible to configure it as part of your Zuul jobs. You can follow instructions at: http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005007.html Cheers, -- Jim and Thierry ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Retiring stale repositories from the OpenStack org on GitHub
Jim and Thierry wrote: [...] One consequence of that transition is that non-OpenStack repositories that were previously mirrored to GitHub under the "openstack" organization are now stale and no longer updated, which is very misleading. Those should now be retired, with a clear pointer to the original repository on Opendev. Jim and I volunteered to build tools to do handle that retirement and we are now ready to run those Thursday. This will not affect OpenStack repositories or repositories that were already retired or migrated off the OpenStack org on GitHub (think openstack-infra, opendev, airship...). That will only clean up no-longer-mirrored, stale, non-openstack repositories still present in the OpenStack GitHub organization. If you own a non-openstack repository on Opendev and would like to enable GitHub mirroring (to a GitHub org of your choice), it is possible to configure it as part of your Zuul jobs. You can follow instructions at: http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005007.html The stale repositories have now been retired. Let us know if you have any question. -- Jim and Thierry ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] ask.openstack.org down for some days
Sorin Sbarnea wrote:
Having a StackOverflow kind Q&A site is of great value but only if there is
enough critical mass to maintain it. Clearly ask.openstack.org is far from reaching
that mass.
I would personally see more useful to redirect users to two existing platforms
that already passed the critical mass and which have active moderators, like:
- https://stackoverflow.com for developers
- https://serverfault.com for operators
+1
We could look at making ask read-only for a while, explicitly
redirecting to serverfault ("openstack" tag) for new questions.
I know that on OpenStack we have good track of DIY on anything but we need to
realize about what is achievable or not. Sometimes is better to focus on core
business and outsource some services.
We are also quite bad at stopping doing things :)
--
Thierry
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Checking release approvals automatically
Hi infra-folk, During the PTG in Shanghai the Release Management team discussed solving one of the remaining pain points in reviewing release requests: checking that the PTL or designated liaisons have actually +1 the request, before casting our own +2 vote. Since they change every 6 months, it's hard to remember names for the 60+ teams we have, so this manual check currently involves each of us diving into test logs, scrolling down to the place where PTLs and liaisons are listed, and then comparing them with current approvals in Gerrit. What if... we could automate that ? My proposed solution for this would be to create a specific pipeline that would trigger on specific comments (including "CodeReview"), and vote Label-PTL-approved on success. Then create a job that would run for openstack/releases changes altering deliverables/** files, and check the current change approvals against the list of PTLs and release liaisons. The job should be lightweight enough to run on the executor. With all those safeguards in place, I do not expect it to trigger significant additional load. Let me know if the idea generally sounds good or bad, or if you see simpler ways to achieve a similar results. Regards, -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Checking release approvals automatically
Jeremy Stanley wrote: On 2019-11-14 17:14:35 + (+), Jeremy Stanley wrote: On 2019-11-14 17:54:45 +0100 (+0100), Thierry Carrez wrote: [...] "checking that the PTL or designated liaisons have actually +1 the request, before casting our own +2 vote." [...] Not that I have a better alternative implementation to suggest, but just to clarify it seems like the underlying problem statement is really "block merging a release request without confirmation by a corresponding PTL or release liaison" (and the Gerrit votes themselves are more of an implementation detail). Or is it maybe "avoid spending release team review time on release requests until a corresponding PTL/liaison has confirmed them" rather than blocking merge? I'd say "Have a way to tell when the PTL/liaison approved the request without having to manually check who the PTL and liaisons are". The vote should not be blocking since we'd likely override it in corner cases. Regarding implementation, the benefit of using a specific label is that it would (also) appear in dashboards. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] OpenDev Independence and Governance
Clark Boylan wrote: [...] In James Blair's winterscale email [0] he suggested that we create a governing council made up of the OpenDev PTL and a representative from each of the OpenStack Foundation's official projects that currently consume OpenDev resources (currently OpenStack itself, Airship, and Zuul). This suggestion creates two levels of governance for the OpenDev team. The first is the position of PTL for the OpenDev project. If we want to continue to manage this position as we've managed it for the OpenStack Infra team, then we can have elections for the position every 6 months. The nominee pool and electorate would be individuals that have contributed changes to OpenDev in the last 12 months. That sounds good. Only comment: "PTL" meaning "project team lead", it's a bit of an OpenStack-ism which might not make perfect sense in the Opendev context. For the council, membership would be small, but I think demands on this group would also be minimal. Ideally the OpenDev team would be left to figure out technical details for services and this council would be used as a check on service changes or other behavioral updates that affect how OpenDev's users interact with the system. Since this group would be starting with an even number of individuals we may need to determine tie breaker requirements upfront. Also, we may want to consider if the "else" group of OpenDev users need a voice. Individuals representing constituent projects should be nominated by their project leadership. I feel like this group should more of an advisory board (to get feedback) than a governance council (to vote on motions on a one project = one vote basis). If you go for a governance structure, it creates a number of issues imho, like tie breaking, or the fact that OpenStack's vote is arguably more important than StarlingX's (being a pilot project) or Kata's (only using very few of the Opendev services). Choosing an advisory board style, there is no formal vote, just official feedback on priorities and proposals, which can then be properly weighed by the OpenDev lead and contributors. You can integrate additional seats to represent "else" opendev users without having to over-think how their voice compares to an OSF project voice. I'm also wondering if the advisory board should not also include seats for the infrastructure donors... Since we should definitely be making sure Opendev goes in a direction that encourages them to continue investing in (or increase) the resources that they give us. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] OpenDev Independence and Governance
Jeremy Stanley wrote: On 2019-12-04 09:45:48 -0500 (-0500), Mohammed Naser wrote: On Wed, Dec 4, 2019 at 5:47 AM Thierry Carrez wrote: [...] I'm also wondering if the advisory board should not also include seats for the infrastructure donors... Since we should definitely be making sure Opendev goes in a direction that encourages them to continue investing in (or increase) the resources that they give us. I wanted to bring this up but indeed, I think that as an infrastructure donor, there is a significant investment from our side and knowing where and how that's going is important Yep, you mentioned it at the PTG and I think it's a great idea. Not only does it provide a means for technical representatives from our resource donors to give more direct feedback and even debate topics between one another, it also gives the OpenDev sysadmins a clearer point of contact when they need to reach out to those same donors. Combining with Thierry's idea, perhaps there are two advisory boards for OpenDev, one for the projects participating in it and one for the resource donors? Or would they be better combined into a single advisory board? For the sake of simplicity I'd suggest having a single stakeholders/advisory board, especially if we don't expect those boards to formally vote (one seat = one vote style) on motions. The main idea, as you mentioned, is to have clear contact points and get their feedback on priorities and direction. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] OpenDev Independence and Governance
Clark Boylan wrote: [...] The OpenDev project will be governed by two entities. The first is the service coordinator. Responsibilities for the service coordinator are essentially the same of the existing Infra team PTL. They coordinate work of contributors and act as a tie breaker when clear consensus isn't found. The service coordinator is elected every 6 months. The nominee pool and electorate are individuals that have contributed changes to OpenDev in the last 12 months. The second is an advisory board made up of representatives from OpenDev's user base of projects and organizations that contribute compute resources. This advisory board provides a formal location for both our users and contributing orgs to express their needs to the OpenDev project. This creates a clear contact point for feedback on priorities and direction. Their input will help ensure that the OpenDev project is a good steward of the resources provided to it and that user needs are being addressed. Contributing orgs and user projects are not required to join the advisory board, but are encouraged to do so. Individuals on the board would be selected by their own constituency as that constituency feels is appropriate. The advisory board will also serve as a point of contact for the OpenDev project when making changes that may be disruptive. The intent is to create bidirectional communication between OpenDev and the advisory board. How does this look? Loving it. -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Checking release approvals automatically
I moved to implementation on this, but I hit an issue with the original plan: [...] The job should be lightweight enough to run on the executor. With all those safeguards in place, I do not expect it to trigger significant additional load. My current implementation is a python script run from tox. But I can't use the standard tox jobs as they have "hosts:all" tasks all over them which are bypassed[1] if the job is run on the executor. [1] See https://zuul.opendev.org/t/openstack/build/4056ca3ee8b247ebbe1cbb1474191c16/console Ideally I would define my own narrow playbook to run the script, without inheriting from the standard tox job. However the current script requires some dependencies to be installed (openstack-governance, yaml...). Here are the options I see: 1- reimplementing most of the unittests/tox job logic in "hosts:localhost" playbook(s) -- would mean lots of copypaste, does not rhyme so well with "lightweight", and increases execution times significantly 2- rewrite the Python script so that it can run on stdlib -- not sure I want to write a YAML parser from scratch though 3- Abandon the idea of running on the executor -- the trick is that for such a short job the overhead of requesting a test node is a bit heavy, and we want to run the job on every vote change on release requests Other ideas? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Checking release approvals automatically
James E. Blair wrote: [...] But back on the first hand, I think that installing python packages in a virtualenv is too heavyweight for a job to run on the executor. The candidates we usually look for are things that can run with what's already installed. Happily, yaml is already installed, because it's kind of a big deal on the executor. Unhappily, openstack-governance is not merely a repo you need to have on-disk, but is actually a python package you need installed (wow, when did that happen?). We were so close. If you just needed to run a python script that imported yaml and read a file out of governance, I'd say it would be a great candidate for running on the executor. But I think the installation of openstack-governance (which has its own requirements that are not installed on the executor) pushes this over the line, and we should run it on a full node. Actually the script only uses openstack-governance to parse YAML files that are in the governance repository... So if YAML is available and the contents of the governance repo are accessible, that can easily work. The only drawback compared to using the governance lib is that it will not survive a change in the YAML format of governance files... but then it's not the only thing that would break if we did that. So it looks like a simple Python script that only imports yaml would work on the executor. The script uses requests as well, but I can make it use urllib instead (unless requests is pre-installed on the executor too ?) Thanks for the full analysis, I learned a couple of things :) -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
