Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon
As the former Horizon PTL, I have a great respect for the importance of the contributions the distro maintainers/developers make to Horizon and OpenStack as a whole. From how many bugs the distros manage to find, to their diligence in vetting the software that we as Horizon developers provide, to their overall passion for the work they do. It is interesting to me to see the level to which the distros have not had to address this problem before. It shows a significant disconnect between the Node/JavaScript community and the distros as a whole (not surprising since node.js wasn't packaged on many distros until quite recently). I'm not excited to see the OpenStack community leading the charge on resolving packaging issues that ought to be settled between the JS community and the distros. Yet, if we have to in order to move forward I would urge us to reach out to the NPM maintainers, major library maintainers, etc. to try and make decisions that will benefit everyone for years to come. It's also interesting to see how strongly people take sides in this debate... who is OpenStack for? How crucial are the distros? Obviously if you work for RedHat or Canonical the distros are the end-all; OpenStack has to be packaged. Other companies? Opinions vary. Flexibility on this issue is not consistent, as has been pointed out already. Monty and Adam Young have, I think, voiced a good moderate viewpoint, which is that the distros are a core part of OpenStack's success and we have to ensure that they can package our software, but that the distros also cannot dictate the tools we use in order to produce the best possible product. Distros are ultimately middle-men, they provide value to their users and they make sure more people use the software we produce. We can all agree that we want to maximize the number of people we can reach. So, I propose that a group gets together and defines criteria: we need to accept that the Horizon team (and those knowledgeable about web-app development) know best what tools they need, and they need to produce such a list as a starting point. We then need packagers and maintainers to examine that list and evaluate it for problems (non-free software, irresolvable dependencies, etc.). They're looking strictly for things which are un-packageable, not commenting on the necessity of said software. And we need people (thanks, Monty) willing to build new tools to find a way to turn that list into something the package maintainers can consume without burdening either side. Make sense? - Gabriel -Original Message- From: Thomas Goirand [mailto:z...@debian.org] Sent: Friday, November 14, 2014 11:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon On 11/14/2014 10:10 PM, Martin Geisler wrote: Of course, I need to run tests. That's a big part of the QA work, and I will certainly not give-up on that. You will have a hard time convincing anyone within the OpenStack community that it's OK to not run unit tests. That's not what I said: the OpenStack developers will continue to tests the software. I personally don't think it's the job of the downstream packagers to do this QA work. (It's of course cool to run the tests on the system installed by your packages -- that test run would then install the needed tools using npm and bower and whatnot if that's how the upstream has setup the test framework.) What happens is that the environment within the distribution, inevitably, will be different from the one ran on the gate. There's going to be different versions of many components and so on. So it is very important for me to also run these unit tests, to make sure that everything continues to work. Yes, the build-dependencies will pull the same components as pulled by npm/bower, though they may be installed in different path, and maybe using different versions. On 11/14/2014 10:21 PM, Jeremy Stanley wrote: On 2014-11-14 15:10:59 +0100 (+0100), Martin Geisler wrote: [...] Just to quibble on this particular point... distro packagers are also developers. They often (more often than we'd like, and we do try to find ways to help avoid this where possible) need to carry their own patches to tweak the software to fit their deployment and operation model. Being able to rerun tests in-place with packaged versions of everything including their patches helps them confirm that what they distribute still works as intended. Further, the distro users are well within their rights to modify and respin these packages themselves, and will potentially want to be able to run these tests for the same reasons. We distribute our tests as part of our software because our tests *are* part of our software. Exactly! Let me give a few examples... In Jessie, Nova carries patches so that there is support for Ceph. Until a few days,
Re: [openstack-dev] [Fuel] Waiting for Haproxy backends
Good plan, but I really hate the name of this blueprint. I think we should stop lumping different unrelated HA improvements into a single blueprint with a generic name like that, especially when we already had a blueprint with essentially the same name (ha-pacemaker-improvements). There's nothing wrong with having 4 trivial but specific blueprints instead of one catch-all. On Wed, Nov 12, 2014 at 4:10 AM, Aleksandr Didenko adide...@mirantis.com wrote: HI, in order to make sure some critical Haproxy backends are running (like mysql or keystone) before proceeding with deployment, we use execs like [1] or [2]. We're currently working on a minor improvements of those execs, but there is another approach - we can replace those execs with puppet resource providers and move all the iterations/loops/timeouts logic there. Also we should fail catalog compilation/run if those resource providers are not able to ensure needed Haproxy backends are up and running. Because there is no point to proceed with deployment if keystone is not running, for example. If no one objects, I can start implementing this for Fuel-6.1. We can address it as a part of pacemaker improvements BP [3] or create a new BP. [1] https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/osnailyfacter/manifests/cluster_ha.pp#L551-L572 [2] https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/openstack/manifests/ha/mysqld.pp#L28-L33 [3] https://blueprints.launchpad.net/fuel/+spec/pacemaker-improvements Regards, Aleksandr Didenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Dmitry Borodaenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [api] Summit summary
Hi All, Here’s a summary of what happened at the Summit from the API Working Group perspective. Etherpad: https://etherpad.openstack.org/p/kilo-crossproject-api-wg The 2 design summit sessions on Tuesday were very well attended, maybe 100ish people I’m guessing. I got the impression there were developers from a diverse set of projects just from the people who spoke up during the session. We spent pretty much all of these 2 sessions discussing the working group itself. Some action items of note: Update the wiki page [1] with the decisions made during the discussion Add an additional meeting time [2] to accommodate EU time Email the WG about the Nova (and Neutron?) API microversions effort and how it might be a strategy for moving forward with API changes Review the rest of the action items in the etherpad to get a better picture. The follow up session on Thursday (last slot of the day) was attended by about half the people of the Tuesday sessions. We reviewed what happened on Tuesday and then got to work. We ran through the workflow of creating a guideline. We basically did #1 and #2 of How to Contribute [3] but instead of first taking notes on the API Improvement in the wiki we just discussed it in the session. We then submitted the patch for a new guideline [4]. As you can see there’s still a lot of work to be done in that review. It may even be that we need a fresh start with it. But it was a good exercise for everyone present to walk through the process together for the first time. I think it really helped put everyone on the same page for working together as a group. Thanks, Everett [1] https://wiki.openstack.org/wiki/API_Working_Group [2] https://wiki.openstack.org/wiki/Meetings/API-WG [3] https://wiki.openstack.org/wiki/API_Working_Group#How_to_Contribute [4] https://review.openstack.org/#/c/133087/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Waiting for Haproxy backends
+1 for ha-pacemaker-improvements -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Fri, Nov 14, 2014 at 11:51 PM, Dmitry Borodaenko dborodae...@mirantis.com wrote: Good plan, but I really hate the name of this blueprint. I think we should stop lumping different unrelated HA improvements into a single blueprint with a generic name like that, especially when we already had a blueprint with essentially the same name (ha-pacemaker-improvements). There's nothing wrong with having 4 trivial but specific blueprints instead of one catch-all. On Wed, Nov 12, 2014 at 4:10 AM, Aleksandr Didenko adide...@mirantis.com wrote: HI, in order to make sure some critical Haproxy backends are running (like mysql or keystone) before proceeding with deployment, we use execs like [1] or [2]. We're currently working on a minor improvements of those execs, but there is another approach - we can replace those execs with puppet resource providers and move all the iterations/loops/timeouts logic there. Also we should fail catalog compilation/run if those resource providers are not able to ensure needed Haproxy backends are up and running. Because there is no point to proceed with deployment if keystone is not running, for example. If no one objects, I can start implementing this for Fuel-6.1. We can address it as a part of pacemaker improvements BP [3] or create a new BP. [1] https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/osnailyfacter/manifests/cluster_ha.pp#L551-L572 [2] https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/openstack/manifests/ha/mysqld.pp#L28-L33 [3] https://blueprints.launchpad.net/fuel/+spec/pacemaker-improvements Regards, Aleksandr Didenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Dmitry Borodaenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] NTP settings.
If NTP server is not reachable on the first boot of the master node, it should be disabled by bootstrap_admin_node, that eliminates the possibility of it spontaneously coming to life and changing the clock for fuel master node and all target nodes in the middle of a deployment. Then all Nailgun needs to do is pop a warning notification that no NTP server is configured on the master node, and it should be fixed manually before starting any deployments. No need to block deployment altogether: if the user doesn't need need global time at all (e.g. in an off-the-grid bunker 2 miles beneath Fort Knox), synchronizing clocks on all environments just to the Fuel master will still work. On Wed, Nov 12, 2014 at 7:28 AM, Matthew Mosesohn mmoses...@mirantis.com wrote: Andrew, That just shifts the error into Nailgun which is forced to examine NTP settings of the host. With docker containerization, it makes detecting this much more difficult. Erroring during fuelmenu is the only chance where a user can effectively set the NTP parameters correctly. We could write a ton of infrastructure around this capability, but all it wins us is a more beautiful error that blocks a user from proceeding. On Wed, Nov 12, 2014 at 7:21 PM, Andrew Woodward xar...@gmail.com wrote: Should we just block the deployment until the NTP is fixed so they know they need to fix it? On Wed, Nov 12, 2014 at 6:06 PM, Stanislaw Bogatkin sbogat...@mirantis.com wrote: Hi all, Today we have a next workflow about NTP in Fuel: 1. When someone deploy a master node, we need to somehow operate with NTP and set upstream NTP servers to Fuel NTP daemon on master node. 2. NTP will enabled by default and set to default upstream values (from ntp.org pool). 3. If user will enter to fuelmenu then then he can change that values and it will stored as new ones. That can lead to some errors at deploy, some as: 1. User set upstream NTP (or use defaults). 2. By some reasons that NTP servers get unreacheable (i.e. network down). 3. User creates environment and push 'deploy' button. 4. In the middle of deploy process upstream NTP become reacheable and master node sync with them. If time on master node before that sync will have big shift regarding to real time, then we will have 2 problems: 1. Deploy can fail, cause slave nodes will sync with master one time only - right after provisioning and if master node resync with upstream in the middle of deploy - some services can stop works (e.g. mcollective). 3. It will be more complicate to understand logs, cause it will shifted too. As I see, we have two ways to solve this: 1. Check upstream NTP reachability and disable upstream servers if they unreachable at fuelmenu step. It will cost us some shift with real time in the end. or 2. Disable NTP upstream servers if they unreachable right after user press 'deploy' button and enable them after deploy is over. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Andrew Mirantis Ceph community ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Dmitry Borodaenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] NTP settings.
On 11/14/2014 04:01 PM, Dmitry Borodaenko wrote: If NTP server is not reachable on the first boot of the master node, it should be disabled by bootstrap_admin_node, that eliminates the possibility of it spontaneously coming to life and changing the clock for fuel master node and all target nodes in the middle of a deployment. Then all Nailgun needs to do is pop a warning notification that no NTP server is configured on the master node, and it should be fixed manually before starting any deployments. No need to block deployment altogether: if the user doesn't need need global time at all (e.g. in an off-the-grid bunker 2 miles beneath Fort Knox), synchronizing clocks on all environments just to the Fuel master will still work. I thought NTP (well ntpd) had an option to tell it to only ever slew the clock, never step it? Or is that only some implementations of NTP? rick jones ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Default templates for Sahara feature in 6.0
Oops, the last line should be read as On the other side, it is a nice UX feature we really want to have 6.0 Dmitry 2014-11-15 3:50 GMT+03:00 Dmitry Mescheryakov dmescherya...@mirantis.com: Dmitry, Lets review the CR from the point of danger to current deployment process: in the essence it is 43 lines of change in puppet module. The module calls a shell script which always returns 0. So whatever happens inside, the deployment will not fail. The only changes (non-get requests) the script does, it does to Sahara. It tries to upload cluster and node-group templates. That is not dangerous operation for Sahara - in the worst case the templates will just not be created and that is all. It will not affect Sahara correctness in any way. On the other side, it is a nice UX feature we really want to have 5.1.1. Thanks, Dmitry 2014-11-15 3:04 GMT+03:00 Dmitry Borodaenko dborodae...@mirantis.com: +286 lines a week after Feature Freeze, IMHO it's too late to make an exception for this one. On Wed, Nov 12, 2014 at 7:37 AM, Dmitry Mescheryakov dmescherya...@mirantis.com wrote: Hello fuelers, I would like to request you merging CR [1] which implements blueprint [2]. It is a nice UX feature we really would like to have in 6.0. On the other side, the implementation is really small: it is a small piece of puppet which runs a shell script. The script always exits with 0, so the change should not be dangerous. Other files in the change are used in the shell script only. Please consider reviewing and merging this though we've already reached FF. Thanks, Dmitry [1] https://review.openstack.org/#/c/132196/ [2] https://blueprints.launchpad.net/mos/+spec/sahara-create-default-templates ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Dmitry Borodaenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Default templates for Sahara feature in 6.0
Dmitry, Lets review the CR from the point of danger to current deployment process: in the essence it is 43 lines of change in puppet module. The module calls a shell script which always returns 0. So whatever happens inside, the deployment will not fail. The only changes (non-get requests) the script does, it does to Sahara. It tries to upload cluster and node-group templates. That is not dangerous operation for Sahara - in the worst case the templates will just not be created and that is all. It will not affect Sahara correctness in any way. On the other side, it is a nice UX feature we really want to have 5.1.1. Thanks, Dmitry 2014-11-15 3:04 GMT+03:00 Dmitry Borodaenko dborodae...@mirantis.com: +286 lines a week after Feature Freeze, IMHO it's too late to make an exception for this one. On Wed, Nov 12, 2014 at 7:37 AM, Dmitry Mescheryakov dmescherya...@mirantis.com wrote: Hello fuelers, I would like to request you merging CR [1] which implements blueprint [2]. It is a nice UX feature we really would like to have in 6.0. On the other side, the implementation is really small: it is a small piece of puppet which runs a shell script. The script always exits with 0, so the change should not be dangerous. Other files in the change are used in the shell script only. Please consider reviewing and merging this though we've already reached FF. Thanks, Dmitry [1] https://review.openstack.org/#/c/132196/ [2] https://blueprints.launchpad.net/mos/+spec/sahara-create-default-templates ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Dmitry Borodaenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron] Core/Vendor code decomposition
Hello, As follow-up action after the Design Summit Session on Core/Vendor split, please find the proposal outlined here: https://review.openstack.org/#/c/134680/ I know that Anita will tell me off since I asked for reviews on the ML, but I felt that it was important to raise awareness, even more than necessary :) I also want to stress the fact that this proposal was not going to be possible without the help of everyone we talked to over the last few weeks, and gave us constructive feedback. Finally, a special thanks goes to Maru Newby and Kevin Benton who helped with most parts of the proposal. Let the review tango begin! Cheers, Armando ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon
On 15 November 2014 00:58, Radomir Dopieralski openst...@sheep.art.pl wrote: On 14/11/14 13:02, Richard Jones wrote: I think that it boils down to whether it'is possible that distributions: 1. package the node-based tools (grunt, karma, protractor, ...) as installable programs, and 2. xstatic-package the bower-based packages that we use (probably a couple of dozen at least). We might even be able to get away without using grunt, though an alternative to its LiveReload facility (and one that doesn't then also depend on another node program like django-livereload does) would be required. I believe tox and django's runserver (and other manage commands) could suffice to replace the other functionality typically provided by grunt. We don't really need Xstatic for that. The packagers can as well package the bower-based packages directly. We can use anything, really, as long as we follow a process that makes sure that Horizon can be packaged into the different distributions. That is, we need: xstatic provides additional meta-data that makes it much easier to integrate the static bundle into an application - specifically it automatically provides the application with file locations and endpoints needed to be inserted into the application HTML. That stuff would have to be done manually without xstatic, and would be a configuration pain. 1. All dependencies explicit (with tests failing if a dependency is missing), xstatic provides us with a dependency mechanism through standard Python setuptools facilities. 2. explicit version ranges, xstatic is done through requirements.txt, so yep :) 3. no multiple versions of the same library, This is not a thing in bower/client-side land. It's really only an issue for npm-based packages, and as I've mentioned, the things we're using there should be packaged as tools by the linux vendor, rather than accessed as packages by our system. Except on non-Linux, of course, where we'll just use npm ;) So I guess the big open question is about how distros are going to deal with npm tool packaging. I can't really answer that question for them (except to say: don't try to replace npm's dependency solution with your own; it simply won't work because you'll probably never find a set of versions that satisfy even just the few tools we're looking at for this project). 4. additions and upgrades of libraries moderated by the packagers, Is there already some mechanism for handling the creation and management of xstatic packages and how they interact with linux packagers? Sorry if this is a noob question. 5. ability to replace the development environment with packaged libraries from the system, I would very much like to still use bower for rapid development and testing, with xstatic coming in once the experimentation is over. But if there was a tool to automatically generate an xstatic package from a bower one, that would be eaiser... 6. ability to build and test our software with the tools that can be used by all the distributions. Yep, I think that just implies that the xstatic stuff is done, and that the distros package the few npm tools we use. On 15 November 2014 01:05, Thomas Goirand z...@debian.org wrote: On 11/11/2014 03:02 PM, Richard Jones wrote: json3 es5-shim angular angular-route angular-cookies angular-animate angular-sanitize angular-smart-table angular-local-storage angular-bootstrap angular-translate font-awesome boot underscore ng-websocket Just FYI, in Debian, the libjs-angularjs already carries: angular-route.js angular-cookies.js angular-animate.js angular-sanitize.js We also already have packaged: font-awesome underscore So, basically, I'd have to package: json3 es5-shim boot angular-smart-table angular-local-storage angular-translate ng-websocket That's a reasonable amount of work. Multiply this by 2 for the xstatic packages (if we keep using that), that's about 14 new packages. The issue is integration with the Django application. How do we know what the file path is to the entry point for that that code, and also how it differs from other packagers? xstatic takes care of that for us (in much the same way that bower does), so is valuable. By the way, can't we use libjs-sockjs instead of ng-websocket? They offer different functionality, as far as I can tell. sockjs is a compatibility layer thing, I think. I'm not sure. The description is a little vague. On the other hand, ng-websocket is all about providing an interface for angular applications over the top of websockets (and a handy mock). Last, I'm ok if we add all these, but please, let's do this in the beginning of the Kilo cycle. It was really hard to cope with it at the end of the freeze for Juno. I'd imagine this should be reasonable, barring any additional dependencies we decide to include along the way.
