Re: [OpenStack-Infra] Development environments for infra's puppet modules
My tool of choice is vagrant - it's super easy to create a Vagrantfile, write a quick shell script that idempotently installs any required puppet modules, and then run puppet itself. An older example is the puppet-storyboard module: git checkout, vagrant up, and you're off to the races. https://github.com/openstack-infra/puppet-storyboard/blob/master/Vagrantfile That approach can be extended to simulating any openstack infra node including slaves, servers, etc https://krotscheck.net/2016/06/01/how-to-simulate-an-openstack-infra-slave.html Michael ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] New storyboard core reviewers
On Fri, Jul 10, 2015 at 8:29 AM Michael Krotscheck krotsch...@gmail.com wrote: It is quite telling (and I predicted this, just check the storyboard meeting logs form last April) that there has been zero progress on any storyboard alternatives since Vancouver. I stand corrected on this (via IRC). There appears to be movement on Maniphest, with some work outstanding for the OAuth integration. Michael ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] New storyboard core reviewers
I think we need to actually define the project's relationship to OpenStack first. As of Vancouver, there was no decision made on whether to move forward on StoryBoard, Maniphest, or something else. This lack of any kind of forward commitment, both in resources and mandate, or even the lack of ability of anyone to make a decision on this, is exactly why I left in disgust. So let me rehash the questions that were left unanswered in Vancouver. - Is OpenStack going to use StoryBoard? If not, where is the alternative and who is working on it? - Who is actually going to work on storyboard? - Is StoryBoard going to continue under infra? It is quite telling (and I predicted this, just check the storyboard meeting logs form last April) that there has been zero progress on any storyboard alternatives since Vancouver. So: If StoryBoard is not going to be used for OpenStack moving forward, then no: We should not approve new cores, instead we should move the project to the attic. Since OpenStack isn't using it, we should not waste cycles or resources on it, though I encourage the CodeThink team to fork their own. If StoryBoard _is_ going to be used for OpenStack, then I personally feel it should NOT be under the infra program, as membership of that program has done nothing but slow down progress. In fact, I'm pretty certain that had StoryBoard been incubated in stackforge instead, it would have make much more progress, and I would still be working on it. Michael On Fri, Jul 10, 2015 at 7:10 AM Thierry Carrez thie...@openstack.org wrote: Hi! As you know, StoryBoard development was mostly abandoned by its original development team, following the Infra team decision to no longer make it its long-term strategy for OpenStack task tracking. However, development was recently rebooted by two new contributors: Adam Coldrick (SotK) and Zara Zaimeche (Zara_). Their contributions are now blocked by the lack of core reviewers. I approved what I could (in openstack-infra/storyboard). Their changes and reviews sound solid. But that is not really the question: since they are the only two active developers on StoryBoard today, I think both should be core reviewers for those repositories. The alternative is to put them into a world of change approval misery as they try to convince people who have moved away from StoryBoard to pay enough attention to it to allow them to merge their changes. There is only so much pain you can endure: it will ultimately result in them forking their future development on another platform. To avoid that, I'd rather reboot the team completely and give them the keys. Thoughts ? -- Thierry Carrez (ttx) ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking
This proposal is all well and good, however (no offense intended) Monty's got a history of putting out neat proposals and leaving someone else to support it. Without identifying a dedicated person/resource that will maintain phabricator, any discussion is moot. Michael On Fri, Apr 3, 2015 at 8:16 AM Thierry Carrez thie...@openstack.org wrote: Monty Taylor wrote: As a follow up to my previous email about thinking about ceasing to use storyboard, I have done some work towards investigating using phabricator. [...] For the record, I'll repost some of the analysis I did for mordred on this. Main blocker: Ability to track tasks across projects/branches - One of the main features we need from a task tracker in OpenStack is the ability to track the completion of tasks across projects and branches. That affects vulnerability management, bug backports, and generally cross-project work. The fact that Launchpad has a limitation there (on the number of tasks it supports before timeouting) is issue #2 with LP. The problem is stated once, then a number of tasks affecting different groups are created, and the issue is closed when the last task is completed. In Maniphest as it stands, that would translate into one of those: 1- Create a single task that affects multiple projects 2- Create a main task and multiple subtasks, one for each project The main issue with (1) is that you can't track partial completion (i.e. which groups have completed their task and which haven't), which is the main goal of using a common task tracker. The main issue with (2) is that it's difficult to see what groups are affected from the main task (since it only lists task names), and the main task is not autoclosed when the last subtask is closed (requiring manual intervention). In both cases, the UI allows multiple projects to apply to a given task, which is great for tagging, but not so great when you want to track tasks per affected repository and branches. Finally, there is no concept of branches -- we would have to create one project for every repo/branch combinations. Finally, projects have no relationship between them. So even if you associated a task to project openstack/nova--master you would have to also manually associate it to project nova (as in openstack/nova + openstack/python-novaclient) so that Nova team members could also find it there. There is basically no projectgroup concept, which is issue #3 we have with Launchpad. In summary, I think Phabricator's projects concept is a (really) neat *tagging* system, with associated dashboard and all. It's just not meant to cover multiple projects in the OpenStack sense. Our projects have a few properties which make them difficult to emulate with Phabricator's projects: there should be only one per task, they have an affected branch (master, juno...), and they are grouped into higher-level structures that can be searched and browsed as well. Phabricator seems to be sized for single separate projects, and it's project concept is just a way to organize work within that single project. It is pretty obvious when you see what happens if you click one of their projects: you end up in a dashboard with one item per bug. Imagine clicking openstack/nova and landing on a dashboard with thousands of bugs: it's just not meant to be used for that. It is excellent for tracking a subset of tasks and organizing them, but not as a higher-level structure. So in order to make it work, we'd need a whole new concept added, let's call it repository. Each repository may have multiple branches associated with it. When you create a task, you select a repository and a branch (and only one). That field is clearly shown in the task name (no need to drill down to the subtask to find which repo/branch is actually affected by the task). Clicking on that will bring you to a list of bugs affecting that repository. Repositories can be grouped arbitrarily into repogroups which let you search and navigate issues in a set of repositories, rather than force everyone to rebuild queries to find, say, all infra issues. Then we could encode a rule so that when the last subtask of a repo-less task is closed, the parent task is auto-closed. Then that /could/ all work for task coordination for OpenStack, although it would be slightly less convenient than Launchpad or StoryBoard (which do not require you to drill down to update subtasks, and keep the discussion in one single place). Could do a lot better: CLI/API -- One of the main pain points with Launchpad is its partial API. Phabricator comes with the arc tool, but it is designed to drive the DVCS git-like featureset. There is no CLI for Maniphest, only a shortcut to make raw Conduit (API) calls. The Conduit API itself looks mostly undocumented and (2) making Conduit more robust isn't currently a
Re: [OpenStack-Infra] Biting the bullet on issue tracking
Hey everyone! It's quite rough to realize that the thing you've been advocating, working on, and desperately trying to recruit contributors for is DoA, and that's what I've been struggling with for the past few weeks. Even so, I was part and parcel to coming up with Monty's recommendation, so this wasn't a surprise. Don't get me wrong: I love the project. I love the team, I love our technological vision, I love all these things. There are features that I feel are unique - Federation, process-agnosticism, clean api/ui separation, etc. - and given the opportunity (and proper resources) I would love to continue working on it. However for all the excitement that I've received over the past year, very little has solidified into any kind of concrete contribution. It was time to call the bluff. And yet... StoryBoard has been a fantastic test bed for JavaScript as a first class citizen in OpenStack, and I'm going to continue moving that forward. There are some missing parts of our infrastructure, some of our existing tools need to be refined and documented, and there are some sticky policy items that need to be proposed to the TC. I see no reason not to continue supporting StoryBoard as that test bed, especially since the infrastructure team is still using it. Once a reasonable sunset has been reached (assuming new contributors don't magically materialize), my plan is to dive into the other UI components in OpenStack. A very special thank you Yolanda Robla-Mota, Mike Heald, Riccardo Cruz, Nikita Konovalov, Aleksey Ripinien, Tom Pollard, and all the others who have contributed over the past year. Y'all are awesome. Michael On Mon, Mar 23, 2015 at 4:03 PM Monty Taylor mord...@inaugust.com wrote: Hi everybody, First, some background: A year and a half ago, Infra started down the road of of writing a replacement for the pieces of Launchpad that OpenStack continues to use. There were several reasons, but notable amongst them are: - Desire to use the forthcoming openstackid OpenID/Oauth as an SSO - Delay in long-standing bugs that affect OpenStack getting fixed The existing open source offerings that we investigated did not have adequate feature parity in the key data model areas that made Launchpad particularly compelling as a choice for us, and adding what we needed to the existing offerings would amount to substantial rewrites ... so we decided that we had no real choice but to write our own. Where we're at We've gotten far enough to get Infra moved on to storyboard, but the project has never really gotten resourced to the level it needs to be to truly responsive to the needs of our community. We're making good progress towards meeting the initial set of goals we set, but in the mean time several new requests have come in - such as from the UX team and the Product Management Working Group - that we cannot meet today and which at our current rate I do not believe we would be able to meet in a reasonable timeframe. At the same time, the state of the art around us has improved since we started. A year and a half ago, I was able to very honestly say that we needed to work on this effort because we simply had no other choice. That is no longer true. Existing Open Source offerings not only can represent a large portion of our data needs, but additionally can support the additional features that have been requested by our community today out of the box. The combination of the two of those makes the likelihood of us being able to convince people to pony up more resources seem more and more far fetched. I could be wrong, of course - it's possible that in response to this someone will start jumping up and down and commit engineers to the effort ... but I'm not holding my breath. Biting the bullet I think we should get out of the business of writing our own bug tracker. It's not an easy thing to say, and I don't say it lightly. There are things that storyboard models well that continue to be things that simply are not modeled elsewhere. However, I think it's important to know when good enough will do, and I think it's important to be able to step up and say that we tried valiantly, and everyone involved did a great job, and yet the world has moved on and writing a bug tracker is not, at the end of the day, what we're all here to do. We're looking at what our options are, and Thierry is examining them to see how tolerable their differences would be to our community. I propose that we have a solid answer and migration plan to put in front of people by Vancouver at the latest. Finally, I'd like to say thank you to the storyboard team for attacking a very hard problem with not enough resources. Monty ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list
[OpenStack-Infra] [infra] [storyboard] Nominating Yolanda Robla for StoryBoard Core
Hello everyone! StoryBoard is the much anticipated successor to Launchpad, and is a component of the Infrastructure Program. The storyboard-core group is intended to be a superset of the infra-core group, with additional reviewers who specialize in the field. Yolanda has been working on StoryBoard ever since the Atlanta Summit, and has provided a diligent and cautious voice to our development effort. She has consistently provided feedback on our reviews, and is neither afraid of asking for clarification, nor of providing constructive criticism. In return, she has been nothing but gracious and responsive when improvements were suggested to her own submissions. Furthermore, Yolanda has been quite active in the infrastructure team as a whole, and provides valuable context for us in the greater realm of infra. Please respond within this thread with either supporting commentary, or concerns about her promotion. Since many western countries are currently celebrating holidays, the review period will remain open until January 9th. If the consensus is positive, we will promote her then! Thanks, Michael References: https://review.openstack.org/#/q/reviewer:%22yolanda.robla+%253Cinfo%2540ysoft.biz%253E%22,n,z http://stackalytics.com/?user_id=yolanda.roblametric=marks ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [storyboard] Paging results
Hey there, Jay- The use case is telling someone where they are in a result set, not necessarily the ability to jump pages. Say, for example, a PTL who is reviewing all the stories in their project to determine which of them are no longer relevant. To them, a very simple You are on #X of #YYY gives them an idea of how much work they have left, and allows them to split their work with other cores: Hey, can you go review 100-200 please?. A good example is the Infra Bug Day that happens fairly frequently. A different, but similar case is a hidden influencer trying to keep tabs on their own team's work backlog, which stretches across different projects. I encourage everyone to brainstorm more, I'm sure you can find them. There is a rule in UX, which is quite simple: Always tell the user exactly where they are. This rule was first clearly stated in Don't make me think, and there's been quite a bit of development on the concept since. A TL/DR version: There's a nuanced difference between paging through data and consuming a feed. In the former, a location indicator is critical. In the latter, less so. This is the case no matter how large the data set gets - Google still provides pages, even though they don't mean much given the size of their results. The best way I've found to conceptualize these two is as follows: If you put all your data into a spreadsheet, and removed the row labels, would your user care? I would characterize your use case as an individual contributor who prefers to treat their task list like a queue. This is one valid use case, and marker based paging solves your problem. However, the other use cases above think of stories more like a spreadsheet, and they require an indicator of where they are relative to the rest of entire set. From a practical perspective, most usage would never go past the second page, so I am not really concerned about the performance implications of simply passing offset straight through to the SQL query, and I would hope that storyboard's filtering capabilities are powerful enough that our users can reduce the size of the result set to a manageable size. However, we ALSO get all that with offset based paging, and we win the the ability to support all the other use cases without sacrificing anything. In short, using offset based paging requires a small investment of time, costs a performance hit for a small subset of users, and allows us to meet the needs of all of our use cases. Michael ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [storyboard] Paging results
For API clients that require access to large parts of the data set, I've actually proposed a solution that doesn't involve the REST API at all. It's likely that this particular spec will also be the backbone of storyboard's federation (assuming we don't argue it into oblivion). https://review.openstack.org/#/c/105252/ Michael On Thu, Nov 20, 2014 at 11:39 AM, Robert Collins robe...@robertcollins.net wrote: However for API usage, complete scans of result sets are much more common, and they caused substantial remedial performance work to be needed in Launchpad (which was also built with the idea that only the first few pages need to be fast). If I may be so bold, the rule of thumb 'if we can't do it fast, don't do it' is a very sensible one for many API implementations, and one that I rather suspect will apply here. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] [infra] [storyboard] Goodbye Infra on Launchpad, Hello Infra on StoryBoard
The OpenStack Infrastructure team has successfully migrated all of the openstack-infra project bugs from LaunchPad to StoryBoard. With the exception of openstack-ci bugs tracked by elastic recheck, all bugs, tickets, and work tracked for OpenStack Infrastructure projects must now be submitted and accessed at https://storyboard.openstack.org. If you file a ticket on LaunchPad, the Infrastructure team no longer guarantees that it will be addressed. Note that only the infrastructure projects have moved, no other OpenStack projects have been migrated. This is part of a long-term plan to migrate OpenStack from Launchpad to StoryBoard. At this point we feel that StoryBoard meets the needs of the OpenStack infrastructure team and plan to use this migration to further exercise the project while we continue its development. As you may notice, Development on StoryBoard is ongoing, and we have not yet reached feature parity with those parts of LaunchPad which are needed for the rest of OpenStack. Contributions are always welcome, and the team may be contacted in the #storyboard or #openstack-infra channels on freenode, via the openstack-dev list using the [storyboard] subject, or via StoryBoard itself by creating a story. Feel free to report any bugs, ask any questions, or make any improvement suggestions that you come up with at: https://storyboard.openstack.org/#!/project/456 https://storyboard.openstack.org/#!/project/456 We are always looking for more contributors! If you have skill in AngularJS or Pecan, or would like to fill in some of our documentation for us, we are happy to accept patches. If your project is interested in moving to StoryBoard, please contact us directly. While we are hesitant to move new projects to storyboard at this point, we would love working with you to determine which features are needed to support you. Relevant links: • Storyboard: https://storyboard.openstack.org https://storyboard.openstack.org/ • Team Wiki: https://wiki.openstack.org/wiki/StoryBoard https://wiki.openstack.org/wiki/StoryBoard___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [storyboard] Paging results
Jay- My own UX tests have demonstrated a need for both page jumping, and being able to communicate to a user where they are in their list. I'd be happy to show you the videos if you have a few hours. Michael ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] best way to add handlebars.js to infra?
On Sep 29, 2014, at 3:24 PM, Monty Taylor mord...@inaugust.com wrote: I am a big fan of the tooling pattern in storyboard-webclient - since it's very much aimed at solving this. Some combination of krotscheck and I will make an example patch we can all point at. I’ve added a WIP patch that starts to add the javascript toolchain. More work is needed https://review.openstack.org/#/c/124927/ The builds that execute the javascript toolchain itself are found here: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/javascript.yaml___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
