Re: [openstack-dev] [Magnum] Adding opensuse as new driver to Magnum
Michal, The right place for drivers is the /drivers folder. Take a look at the existing drivers as an examples. You'll also need to update this file https://github.com/openstack/magnum/blob/master/setup.cfg#L60 and add a new entry point for the driver. I would encourage you to hold off on this patch. We are currently working on using stevedore to load drivers and moving all the heat stack creation and update operations to each driver. -Murali From: Michal JuraSent: Thursday, August 4, 2016 3:26 AM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [Magnum] Adding opensuse as new driver to Magnum Hi, I would like to put for discussion adding new driver to Magnum. We would like to propose opensuse with kubernetes as driver. I did some initial work in bug https://launchpad.net/bugs/1600197 and changes https://review.openstack.org/339480 https://review.openstack.org/349994 I've got also some comments from you about how this should be proceed. As maintainer for this change I can propose myself. I have couple question about moving this driver to /contrib directory. If I will do this how this driver should be installed from there? Thank you for all answers and help with doing this, Best regards, Michal __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [magnum] Document adding --memory option to create containers
+1 Anything with default values should be ignored in the quickstart guide. -Murali From: Steven Dake (stdake)Sent: Thursday, October 8, 2015 2:54 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Document adding --memory option to create containers Quickstart guide should be dead dead dead dead simple. The goal of the quickstart guide isn't to tach people best practices around Magnum. It is to get a developer operational to give them that sense of feeling that Magnum can be worked on. The goal of any quickstart guide should be to encourage the thinking that a person involving themselves with the project the quickstart guide represents is a good use of the person's limited time on the planet. Regards -steve From: Hongbin Lu > Reply-To: "OpenStack Development Mailing List (not for usage questions)" > Date: Thursday, October 8, 2015 at 9:00 AM To: "OpenStack Development Mailing List (not for usage questions)" > Subject: [openstack-dev] [magnum] Document adding --memory option to create containers Hi team, I want to move the discussion in the review below to here, so that we can get more feedback https://review.openstack.org/#/c/232175/ In summary, magnum currently added support for specifying the memory size of containers. The specification of the memory size is optional, and the COE won't reserve any memory to the containers with unspecified memory size. The debate is whether we should document this optional parameter in the quickstart guide. Below is the positions of both sides: Pros: · It is a good practice to always specifying the memory size, because containers with unspecified memory size won't have QoS guarantee. · The in-development autoscaling feature [1] will query the memory size of each container to estimate the residual capacity and triggers scaling accordingly. Containers with unspecified memory size will be treated as taking 0 memory, which negatively affects the scaling decision. Cons: · The quickstart guide should be kept as simple as possible, so it is not a good idea to have the optional parameter in the guide. Thoughts? [1] https://blueprints.launchpad.net/magnum/+spec/autoscale-bay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [magnum] keystone pluggable model
?Hi All, In the IRC meeting yesterday, I brought up this new blueprint I opened. https://blueprints.launchpad.net/magnum/+spec/pluggable-keystone-model? The goal of this blueprint is to allow magnum operators to integrate with their version of keystone easily with downstream patches. The goal is NOT to implement support for keystone version 2 upstream, but to make it easy for operators to integrate with V2 if they need to. Most of the work required for this is already done in this patch. https://review.openstack.org/#/c/218699 However, we didn't want to address this change in the same review. We just need to refactor the code a little further and isolate all version specific keystone code to one file. See my comments in the following review for details on what this change entails. https://review.openstack.org/#/c/218699/5/magnum/common/clients.py https://review.openstack.org/#/c/218699/5/magnum/common/keystone.py Thanks, Murali __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum][app-catalog] [ Supporting swift downloads for operator languagepacks
Kevin\Keith, Yes, we would like to use the catalog for globally available artifacts, such as operator languagepacks. More specifically the catalog would be a great place to store metadata about publicly available artifacts to make them searchable and easy to discover. The catalog would point to the 'built' artifact, not the 'unbuilt' dockerfile in github. The point of languagepacks is to reduce the amount of time the solum CI pipeline spends building the users application container. We shouldn't build the languagepack from scratch each time. -Murali From: Keith Bray keith.b...@rackspace.com Sent: Wednesday, June 17, 2015 2:10 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Solum][app-catalog] [ Supporting swift downloads for operator languagepacks Hi Kevin, We absolute envision languagepack artifacts being made available via apps.openstack.org (ignoring for a moment that the name may not be a perfect fit, particularly for things like vanilla glance images ... Is it an OS or an App? ... catalog.openstack.org might be more fitting). Anyway, there are two stages for language packs, unbuilt, and built. If it's in an unbuilt state, then it's really a Dockerfile + any accessory files that the Dockerfile references. If it's in a built state, then it's a Docker image (same as what is found on Dockerhub I believe). I think there will need to be more discussion to know what users prefer, built vs. unbuilt, or both options (where unbuilt is often a collection of files, best managed in a repo like github vs. built which are best provided as direct links so a single source like Dockerhub). -Keith On 6/17/15 1:58 PM, Fox, Kevin M kevin@pnnl.gov wrote: This question may be off on a tangent, or may be related. As part of the application catalog project, (http://apps.openstack.org/) we're trying to provide globally accessible resources that can be easily consumed in OpenStack Clouds. How would these global Language Packs fit in? Would the url record in the app catalog be required to point to an Internet facing public Swift system then? Or, would it point to the source git repo that Solum would use to generate the LP still? Thanks, Kevin From: Randall Burt [randall.b...@rackspace.com] Sent: Wednesday, June 17, 2015 11:38 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Solum] Supporting swift downloads for operatorlanguagepacks Yes. If an operator wants to make their LP publicly available outside of Solum, I was thinking they could just make GET's on the container public. That being said, I'm unsure if this is realistically do-able if you still have to have an authenticated tenant to access the objects. Scratch that; http://blog.fsquat.net/?p=40 may be helpful. On Jun 17, 2015, at 1:27 PM, Adrian Otto adrian.o...@rackspace.com wrote: To be clear, Randall is referring to a swift container (directory). Murali has a good idea of attempting to use swift client first, as it has performance optimizations that can speed up the process more than naive file transfer tools. I did mention to him that wget does have a retiree feature, and that we could see about using curl instead to allow for chunked encoding as additional optimizations. Randall, are you suggesting that we could use swift client for both private and public LP uses? That sounds like a good suggestion to me. Adrian On Jun 17, 2015, at 11:10 AM, Randall Burt randall.b...@rackspace.com wrote: Can't an operator make the target container public therefore removing the need for multiple access strategies? Original message From: Murali Allada Date:06/17/2015 11:41 AM (GMT-06:00) To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Solum] Supporting swift downloads for operator languagepacks Hello Solum Developers, When we were designing the operator languagepack feature for Solum, we wanted to make use of public urls to download operator LPs, such as those available for CDN backed swift containers we have at Rackspace, or any publicly accessible url. This would mean that when a user chooses to build applications on top of a languagepack provided by the operator, we use a url to 'wget' the LP image. Recently, we have started noticing a number of failures because of corrupted docker images downloaded using 'wget'. The docker images work fine when we download them manually with a swift client and use them. The corruption seem to be happening when we try to download a large image using 'wget' and there are dropped packets or intermittent network issues. My thinking is to start using the swift client to download operator LPs by default instead of wget. The swift client already implements retry logic, downloading large images in chunks, etc. This means we would not get
Re: [openstack-dev] [Solum][app-catalog] [ Supporting swift downloads for operator languagepacks
Thanks Christopher. Will do for sure. -Murali From: Christopher Aedo ca...@mirantis.com Sent: Wednesday, June 17, 2015 3:24 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Solum][app-catalog] [ Supporting swift downloads for operator languagepacks On Wed, Jun 17, 2015 at 12:53 PM, Murali Allada murali.all...@rackspace.com wrote: Kevin\Keith, Yes, we would like to use the catalog for globally available artifacts, such as operator languagepacks. More specifically the catalog would be a great place to store metadata about publicly available artifacts to make them searchable and easy to discover. The catalog would point to the 'built' artifact, not the 'unbuilt' dockerfile in github. The point of languagepacks is to reduce the amount of time the solum CI pipeline spends building the users application container. We shouldn't build the languagepack from scratch each time. Murali, this is great to hear. It fits perfectly with where I'd like to see the app catalog head - which is to more easily host anything that could be run on OpenStack. Hopefully you can join us in the weeks to come (IRC or mailing list) and we can start sketching out the changes we'll need to make to allow the catalog to expand in this directly. I'm looking forward to seeing Solum assets in there! -Christopher -Murali From: Keith Bray keith.b...@rackspace.com Sent: Wednesday, June 17, 2015 2:10 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Solum][app-catalog] [ Supporting swift downloads for operator languagepacks Hi Kevin, We absolute envision languagepack artifacts being made available via apps.openstack.org (ignoring for a moment that the name may not be a perfect fit, particularly for things like vanilla glance images ... Is it an OS or an App? ... catalog.openstack.org might be more fitting). Anyway, there are two stages for language packs, unbuilt, and built. If it's in an unbuilt state, then it's really a Dockerfile + any accessory files that the Dockerfile references. If it's in a built state, then it's a Docker image (same as what is found on Dockerhub I believe). I think there will need to be more discussion to know what users prefer, built vs. unbuilt, or both options (where unbuilt is often a collection of files, best managed in a repo like github vs. built which are best provided as direct links so a single source like Dockerhub). -Keith On 6/17/15 1:58 PM, Fox, Kevin M kevin@pnnl.gov wrote: This question may be off on a tangent, or may be related. As part of the application catalog project, (http://apps.openstack.org/) we're trying to provide globally accessible resources that can be easily consumed in OpenStack Clouds. How would these global Language Packs fit in? Would the url record in the app catalog be required to point to an Internet facing public Swift system then? Or, would it point to the source git repo that Solum would use to generate the LP still? Thanks, Kevin From: Randall Burt [randall.b...@rackspace.com] Sent: Wednesday, June 17, 2015 11:38 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Solum] Supporting swift downloads for operatorlanguagepacks Yes. If an operator wants to make their LP publicly available outside of Solum, I was thinking they could just make GET's on the container public. That being said, I'm unsure if this is realistically do-able if you still have to have an authenticated tenant to access the objects. Scratch that; http://blog.fsquat.net/?p=40 may be helpful. On Jun 17, 2015, at 1:27 PM, Adrian Otto adrian.o...@rackspace.com wrote: To be clear, Randall is referring to a swift container (directory). Murali has a good idea of attempting to use swift client first, as it has performance optimizations that can speed up the process more than naive file transfer tools. I did mention to him that wget does have a retiree feature, and that we could see about using curl instead to allow for chunked encoding as additional optimizations. Randall, are you suggesting that we could use swift client for both private and public LP uses? That sounds like a good suggestion to me. Adrian On Jun 17, 2015, at 11:10 AM, Randall Burt randall.b...@rackspace.com wrote: Can't an operator make the target container public therefore removing the need for multiple access strategies? Original message From: Murali Allada Date:06/17/2015 11:41 AM (GMT-06:00) To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Solum] Supporting swift downloads for operator languagepacks Hello Solum Developers, When we were designing the operator languagepack feature for Solum, we wanted to make use
[openstack-dev] [Solum] Supporting swift downloads for operator languagepacks
Hello Solum Developers, When we were designing the operator languagepack feature for Solum, we wanted to make use of public urls to download operator LPs, such as those available for CDN backed swift containers we have at Rackspace, or any publicly accessible url. This would mean that when a user chooses to build applications on to??p of a languagepack provided by the operator, we use a url to 'wget' the LP image. Recently, we have started noticing a number of failures because of corrupted docker images downloaded using 'wget'. The docker images work fine when we download them manually with a swift client and use them. The corruption seem to be happening when we try to download a large image using 'wget' and there are dropped packets or intermittent network issues. My thinking is to start using the swift client to download operator LPs by default instead of wget. The swift client already implements retry logic, downloading large images in chunks, etc. This means we would not get the niceties of using publicly accessible urls. However, the feature will be more reliable and robust. The implementation would be as follows: * ?We'll use the existing service tenant configuration available in the solum config file to authenticate and store operator languagepacks using the swift client. We were using a different tenant to build and host LPs, but now that we require the tenants credentials in the config file, it's best to reuse the existing service tenant creds. Note: If we don't, we'll have 3 separate tenants to maintain. * ?Service tenant * Operator languagepack tenant * Global admin tenant * I'll keep the option to download the operator languagepacks from a publicly available url. I'll allow operators to choose which method they want to use by changing a setting in the solum config file. FYI: In my tests, I've noticed that downloading an image using the swift client is twice as fast as downloading the same image using 'wget' from a CDN url. Thanks, Murali __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Why do app names have to be unique?
I think app names should be unique per tenant. My opinion is that this should be solums default behavior and does not require an extra setting in the config file. I feel the pain of non unique names when I play with my own vagrant environment and deploy multiple 'test' apps. To identify the right app, I either need to save the uuid in a notepad to refer to later, or I just deploy them with unique names, which makes non unique names unnecessary. I wouldn't want to list my apps and see multiple apps with the same name. If I do, and don't have the uuid saved somewhere, I'm totally lost. On Jun 15, 2015, at 9:37 AM, Devdatta Kulkarni devdatta.kulka...@rackspace.commailto:devdatta.kulka...@rackspace.com wrote: Hi Adrian, The new app resource that is being implemented (https://review.openstack.org/#/c/185147/) does not enforce name uniqueness. This issue was discussed here sometime back. Earlier thread: http://lists.openstack.org/pipermail/openstack-dev/2015-March/058858.html The main argument for requiring unique app names within a tenant was usability. In customer research it was found that users were more comfortable with using names than UUIDs. Without the unique app name constraint, users would have to resort to using UUIDs when they create multiple apps with the same name. As a way to accommodate both requirements -- unique names when desired, and making them optional when its not an issue -- we had said that we could make app uniqueness per tenant configurable. So I would be okay if we open a new blueprint to track this work. Thanks, - Devdatta From: Adrian Otto adrian.o...@rackspace.commailto:adrian.o...@rackspace.com Sent: Friday, June 12, 2015 6:57 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Solum] Why do app names have to be unique? Team, While triaging this bug, I got to thinking about unique names: https://bugs.launchpad.net/solum/+bug/1434293 Should our app names be unique? Why? Should I open a blueprint for a new feature to make name uniqueness optional, and default it to on. If not, why? Thanks, Adrian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.orgmailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Should logs be deleted when we delete an app?
I agree, users should have a mechanism to keep logs around. I implemented the logs deletion feature after we got a bunch of requests from users to delete logs once they delete an app, so they don't get charged for storage once the app is deleted. My implementation deletes the logs by default and I think that is the right behavior. Based on user requests, that is exactly what they were asking for. I'm planning to add a --keep-logs flag in a follow up patch. The command will look as follows Solum delete app MyApp --keep-logs -Murali On Jun 15, 2015, at 11:19 PM, Keith Bray keith.b...@rackspace.commailto:keith.b...@rackspace.com wrote: Regardless of what the API defaults to, could we have the CLI prompt/warn so that the user easily knows that both options exist? Is there a precedent within OpenStack for a similar situation? E.g. solum app delete MyApp Do you want to also delete your logs? (default is Yes): [YES/no] NOTE, if you choose No, application logs will remain on your account. Depending on your service provider, you may incur on-going storage charges. Thanks, -Keith From: Devdatta Kulkarni devdatta.kulka...@rackspace.commailto:devdatta.kulka...@rackspace.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Monday, June 15, 2015 9:56 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Solum] Should logs be deleted when we delete an app? Yes, the log deletion should be optional. The question is what should be the default behavior. Should the default be to delete the logs and provide a flag to keep them, or keep the logs by default and provide a override flag to delete them? Delete-by-default is consistent with the view that when an app is deleted, all its artifacts are deleted (the app's meta data, the deployment units (DUs), and the logs). This behavior is also useful in our current state when the app resource and the CLI are in flux. For now, without a way to specify a flag, either to delete the logs or to keep them, delete-by-default behavior helps us clean all the log files from the application's cloud files container when an app is deleted. This is very useful for our CI jobs. Without this, we end up with lots of log files in the application's container, and have to resort to separate scripts to delete them up after an app is deleted. Once the app resource and CLI stabilize it should be straightforward to change the default behavior if required. - Devdatta From: Adrian Otto adrian.o...@rackspace.commailto:adrian.o...@rackspace.com Sent: Friday, June 12, 2015 6:54 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Solum] Should logs be deleted when we delete an app? Team, We currently delete logs for an app when we delete the app[1]. https://bugs.launchpad.net/solum/+bug/1463986 Perhaps there should be an optional setting at the tenant level that determines whether your logs are deleted or not by default (set to off initially), and an optional parameter to our DELETE calls that allows for the opposite action from the default to be specified if the user wants to override it at the time of the deletion. Thoughts? Thanks, Adrian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.orgmailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Should app names be unique?
The only reason this came up yesterday is because we wanted Solums 'app create' behavior to be consistent with other openstack services. However, if heat has a unique stack name constraint and glance\nova don't, then the argument of consistency does not hold. I'm still of the opinion that we should have a unique name constraint for apps and languagepacks within a tenants namespace, as it can get very confusing if a user creates multiple apps with the same name. Also, customer research done here at Rackspace has shown that users prefer using 'names' rather than 'UUIDs'. -Murali From: Devdatta Kulkarni devdatta.kulka...@rackspace.com Sent: Wednesday, March 11, 2015 2:48 PM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [Solum] Should app names be unique? Hi Solum team, In yesterday's team meeting the question of whether Solum should enforce unique app name constraint within a tenant came up. As a recollection, in Solum one can create an 'app' using: solum app create --plan-file plan-file --name app-name Currently Solum does support creating multiple apps with the same name. However, in yesterday's meeting we were debating/discussing whether this should be the case. The meeting log is available here: http://eavesdrop.openstack.org/meetings/solum_team_meeting/2015/solum_team_meeting.2015-03-10-21.00.log.html To set the context for discussion, consider the following: - heroku does not allow creating another app with the same name as that of an already existing app - github does not allow creating another repository with the same name as that of an already existing repo Thinking about why this might be in case for heroku, one aspect that comes to mind is the setting of a 'remote' using the app name. When we do a 'git push', it happens to this remote. When we don't specify a remote in 'git push' command, git defaults to using the 'origin' remote. Even if multiple remotes with the same name were to be possible, when using an implicit command such as 'git push', in which some of the input comes from the context, the system will not be able to disambiguate which remote to use. So requiring unique names ensures that there is no ambiguity when using such implicit commands. This might also be the reason why on github we cannot create repository with an already existing name. But this is just a guess for why unique names might be required. I could be totally off. I think Solum's use case is similar. Agreed that Solum currently does not host application repositories and so there is no question of Solum generated remotes. But by allowing non-unique app names it might be difficult to support this feature in the future. As an aside, I checked what position other Openstack services take on this issue. 1) Heat enforces unique stack-name constraint. 2) Nova does not enforce this constraint. So it is clear that within Openstack there is no consistency on this issue. What should Solum do? Thoughts? Best regards, Devdatta __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Addition to solum core
+1 From: Pierre Padrixe pierre.padr...@gmail.commailto:pierre.padr...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Tuesday, December 30, 2014 at 5:58 PM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Solum] Addition to solum core +1 2014-12-27 19:02 GMT+01:00 Devdatta Kulkarni devdatta.kulka...@rackspace.commailto:devdatta.kulka...@rackspace.com: +1 From: James Y. Li [yuel...@gmail.commailto:yuel...@gmail.com] Sent: Saturday, December 27, 2014 9:03 AM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Solum] Addition to solum core +1! -James Li On Dec 27, 2014 2:02 AM, Adrian Otto adrian.o...@rackspace.commailto:adrian.o...@rackspace.com wrote: Solum cores, I propose the following addition to the solum-core group[1]: + Ed Cranford (ed--cranford) Please reply to this email to indicate your votes. Thanks, Adrian Otto [1] https://review.openstack.org/#/admin/groups/229,members Current Members ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] reviews for the new API
Angus/Julien, I would disagree that we should expose the mistral DSL to end users. What if we decide to use something other than Mistral in the future? We should be able to plug in any workflow system we want without changing what we expose to the end user. To me, the pipeline DSL is similar to our plan file. We don't expose a heat template to our end users. Murali On Jun 4, 2014, at 10:58 AM, Julien Vey vey.jul...@gmail.commailto:vey.jul...@gmail.com wrote: Hi Angus, I really agree with you. I would insist on #3, most of our users will use the default workbook, and only advanced users will want to customize the workflow. advanced users should easily understand a mistral workbook, cause they are advanced To add to the cons of creating our own DSL, it will require a lot more work, more design discussions, more maintenance... We might end up doing what mistral is already doing. If we have some difficulties with Mistral's DSL, we can talk with the team, and contribute back our experience of using Mistral. Julien 2014-06-04 14:11 GMT+02:00 Angus Salkeld angus.salk...@rackspace.commailto:angus.salk...@rackspace.com: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all I have posted this series and it has evoked a surprising (to me) amount of discussion so I wanted to clarify some things and talk to some of the issues so we can move forward. So first note is this is an early posting and still is not tested much. https://review.openstack.org/#/q/status:open+project:stackforge/solum+branch:master+topic:new-api,n,z First the terminology I have a pipeline meaning the association between a mistral workbook, a trigger url and a plan. This is a running entity not just a different workbook. The main issue seems to be the extent to which I am exposing the mistral workbook. Many of you expected a simpler workflow DSL that would be converted into the mistral workbook. The reason for me doing it this way are: 1) so we don't have to write much code 2) this is an iterative process. Let's try it the simple way first and only make it more complicated if we really need to (the agile way?). 3) to be consistent in the way we treat heat templates, mistral workbooks and language packs - i.e. we provide standard ones and allow you to customize down to the underlying openstack primitives if you want (we should aim for this to be only a small percentage of our users). eg. pipeline == (check-build-deploy mistral workbook + basic-docker heat template + custom plan) here the user just choose the heat template and workbook from a list of options. 4) if the mistral dsl is difficult for our users to work with we should give the mistral devs a chance to improve it before working around it. 5) our users are primary developers and I don't think the current mistral DSL is tricky to figure out for someone that can code. 6) doing it this way we can make use of heat and mistral's horizon plugins and link to them from the pipeline instead of having to redo all of the same pages. In a similar why that heat links to servers/volumes etc from a running stack. - -Angus Some things to note: - - the entire mistral engine can be replaced with an engine level plugin -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTjwz2AAoJEFrDYBLxZjWoEr0H/3nh66Umdw2nGUEs+SikigXa XAN90NHHPuf1ssEqrF/rMjRKg+GvrLx+31x4oFfHEj7oplzGeVA9TJC0HOp4h6dh iCeXAHF7KX+t4M4VuZ0y9TJB/jLxfxg4Qge7ENJpNDD/gggjMYSNhcWzBG87QBE/ Mi4YAvxNk1/C3/YZYx2Iujq7oM+6tflTeuoG6Ld72JMHryWT5/tdYZrCMnuD4F7Q 8a6Ge3t1dQh7ZlNHEuRDAg3G5oy+FInXyFasXYlYbtdpTxDL8/HbXegyAcsw42on 2ZKRDYBubQr1MJKvSV5I3jjOe4lxXXFylbWpYpoU8Y5ZXEKp69R4wrcVISF1jQQ= =P0Sl -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] reviews for the new API
The problem with exposing Mistral DSL the way it is, is that there are many things the user should not be aware of. Lets take the example Mistral DSL created here, https://review.openstack.org/#/c/95709/4/etc/solum/workbooks/build.yaml Why should the end user know and define things like auth_token, base_image_id (which is the language pack id in the plan file), on-success actions, etc. Users should not be concerned with these things. All they should be able to do is say 'yes/no' to a task which Solum supports, and if Yes, configure it further. Dependency between tasks, internal solum/glance/keystone id's etc should be hidden, but are required by the mistral DSL. On Jun 4, 2014, at 1:10 PM, Julien Vey vey.jul...@gmail.commailto:vey.jul...@gmail.com wrote: Murali, Roshan. I think there is a misunderstood. By default, the user wouldn't see any workflow dsl. If the user does not specify anything, we would use a pre-defined mistral workbook defined by Solum, as Adrian described If the user needs more, mistral is not so complicated. Have a look at this example Angus has done https://review.openstack.org/#/c/95709/4/etc/solum/workbooks/build.yaml We can define anything as solum actions, and the users would just have to call one of this actions. Solum takes care of the implementation. If we have comments about the DSL, Mistral's team is willing to help. Our end-users will be developers, and a lot of them will need a custom workflow at some point. For instance, if Jenkins has so many plugins, it's because there are as many custom build workflows, specific to each company. If we add an abstraction on top of mistral or any other workflow engine, we will lock developers in our own decisions, and any additional feature would require a new development in Solum, whereas exposing (when users want it) mistral, we would allow for any customization. Julien 2014-06-04 19:50 GMT+02:00 Roshan Agrawal roshan.agra...@rackspace.commailto:roshan.agra...@rackspace.com: Agreeing with what Murali said below. We should make things really simple for the 99 percentile of the users, and not force the complexity needed by the minority of the “advanced users” on the rest of the 99 percentile users. Mistral is a generic workflow DSL, we do not need to expose all that complexity to the Solum user that wants to customize the pipeline. “Non-advanced” users will have a need to customize the pipeline. In this case, the user is not necessarily the developer persona, but typically an admin/release manager persona. Pipeline customization should be doable easily, without having the understand or author a generic workflow DSL. For the really advanced user who needs to have a finer grained need to tweak the mistral workflow DSL (I am not sure if there will be a use case for this if we have the right customizations exposed via the pipeline API), we should have the “option” for the user to tweak the mistral DSL directly, but we should not expect 99.9% (or more) of the users to deal with a generic workflow. From: Murali Allada [mailto:murali.all...@rackspace.commailto:murali.all...@rackspace.com] Sent: Wednesday, June 04, 2014 12:09 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [solum] reviews for the new API Angus/Julien, I would disagree that we should expose the mistral DSL to end users. What if we decide to use something other than Mistral in the future? We should be able to plug in any workflow system we want without changing what we expose to the end user. To me, the pipeline DSL is similar to our plan file. We don't expose a heat template to our end users. Murali On Jun 4, 2014, at 10:58 AM, Julien Vey vey.jul...@gmail.commailto:vey.jul...@gmail.com wrote: Hi Angus, I really agree with you. I would insist on #3, most of our users will use the default workbook, and only advanced users will want to customize the workflow. advanced users should easily understand a mistral workbook, cause they are advanced To add to the cons of creating our own DSL, it will require a lot more work, more design discussions, more maintenance... We might end up doing what mistral is already doing. If we have some difficulties with Mistral's DSL, we can talk with the team, and contribute back our experience of using Mistral. Julien 2014-06-04 14:11 GMT+02:00 Angus Salkeld angus.salk...@rackspace.commailto:angus.salk...@rackspace.com: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all I have posted this series and it has evoked a surprising (to me) amount of discussion so I wanted to clarify some things and talk to some of the issues so we can move forward. So first note is this is an early posting and still is not tested much. https://review.openstack.org/#/q/status:open+project:stackforge/solum+branch:master+topic:new-api,n,z First the terminology I have a pipeline meaning the association between a mistral workbook
Re: [openstack-dev] [solum] use of the plan for m1
+1 for using a simple plan file for M1. I agree, the language pack id would need to be part of the plan file. -Murali On Mar 4, 2014, at 9:20 PM, devdatta kulkarni devdatta.kulka...@rackspace.com wrote: I support this approach. Customization of build and deploy lifecycle actions depends on the ability to register different kinds of services for performing these actions. I can imagine that operators would want to provide such services as part of their Solum install. Then, app developers would be able to find about such services and refer to them in their application descriptor (may be a plan file, may be something else). However, for m1, I agree that we should go with the view that build and deploy services are not externalized, but are available as default services in Solum. About the proposed simpler descriptor -- the only question I have is about the language-pack to use to build the app. Won't we need it in the application descriptor? So I propose: artifacts: - name: My Python App artifact_type: application.heroku content: { href: http://github.com/some/project } language-pack: language-pack-id - Devdatta -Original Message- From: Angus Salkeld angus.salk...@rackspace.com Sent: Tuesday, March 4, 2014 8:52pm To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [solum] use of the plan for m1 Hi all I just wanted to clarify our use of the camp plan file (esp. for m1). Up until now I have been under the impression that use the plan to describe the app lifecycle (build/test/deploy) and the contents of the app. After attempting to write code that converts plans like this into heat templates started to think that this is not a good idea as it is mixing two ideas from very different areas. It also makes the resulting plan complex. I suggest we move from some of the plans suggested here: https://etherpad.openstack.org/p/solum-demystified to a very simple: artifacts: - name: My Python App artifact_type: application.heroku content: { href: http://github.com/some/project } For m1 we can assume a lifecycle of build and deploy. After that we can figure out how we would want to expose the lifecycle choices/customization to the user. Thoughts? -Angus ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Proposed changes to solum-core
+1 On Jan 25, 2014, at 9:04 AM, Roshan Agrawal roshan.agra...@rackspace.com wrote: +1 From: Rajesh Ramchandani [rajesh.ramchand...@cumulogic.com] Sent: Friday, January 24, 2014 9:35 PM To: OpenStack Development Mailing List (not for usage questions) Cc: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Solum] Proposed changes to solum-core +1 On Jan 24, 2014, at 5:35 PM, Adrian Otto adrian.o...@rackspace.com wrote: Solum Core Reviewers, I propose the following changes to solum-core: +asalkeld +noorul -mordred Thanks very much to mordred for helping me to bootstrap the reviewer team. Please reply with your votes. Thanks, Adrian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Devstack gate is failing
I'm ok with making this non-voting for Solum until this gets fixed. -Murali On Jan 7, 2014, at 8:53 PM, Noorul Islam Kamal Malmiyoda noo...@noorul.com wrote: Hi team, After merging [1] devstack gate started failing. There is already a thread [2] related to this in mailing list. Until this gets fixed shall we make this job non-voting? Regards, Noorul [1] https://review.openstack.org/64226 [2] http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg12440.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Solum] API worker architecture
Hi all, I'm working on a new blueprint to define the API service architecture. https://blueprints.launchpad.net/solum/+spec/api-worker-architecture It is still a draft for now. I've proposed a simple architecture for us to get started with implementing a few useful use cases. Please chime in on the mailing list with your thoughts. Thanks, Murali ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] API worker architecture
No, I was Infact thinking about making all responses synchronous to keep things simple in the beginning. Mostly because I can't think of any use case that requires asynchronous processing right now. But you're right, we might want to support async right from the start. On Nov 27, 2013, at 1:36 PM, Roshan Agrawal roshan.agra...@rackspace.commailto:roshan.agra...@rackspace.com wrote: We should probably include support for asynchronous responses right from the beginning to handle calls that need a long time to process. Is this in line with what you were thinking ? I am referring to your comment in the blueprint “To start things off, we can implement workflow #1 shown above and make all requests synchronous” From: Murali Allada [mailto:murali.all...@rackspace.com] Sent: Wednesday, November 27, 2013 12:43 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Solum] API worker architecture Hi all, I'm working on a new blueprint to define the API service architecture. https://blueprints.launchpad.net/solum/+spec/api-worker-architecture It is still a draft for now. I've proposed a simple architecture for us to get started with implementing a few useful use cases. Please chime in on the mailing list with your thoughts. Thanks, Murali ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] SFO Design Workshop
If you want to drive to the Rackspace office, here's a list of parking garages in the area. Street parking will be hard to find. http://goo.gl/maps/yuoWW On Nov 18, 2013, at 11:50 AM, Adrian Otto adrian.o...@rackspace.commailto:adrian.o...@rackspace.com wrote: No, it does not have any parking. I suggest coming in on BART and exit at the Montgomery station and coming south on 2nd to Folsom by foot. The Courtyard hotel across the street has parking but that is very expensive and may be full. Parking at the meters on the street is not a great option either. -- Adrian Original message From: Clayton Coleman Date:11/18/2013 9:43 AM (GMT-08:00) To: OpenStack Development Mailing List (not for usage questions) Cc: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Solum] SFO Design Workshop Is there easy parking at the rackspace office? On Nov 18, 2013, at 7:33 AM, Adrian Otto adrian.o...@rackspace.commailto:adrian.o...@rackspace.com wrote: No. Registration is for in person attendance only. -- Adrian Original message From: Rajdeep Dua Date:11/18/2013 5:39 AM (GMT-08:00) To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Solum] SFO Design Workshop Do we need to register on event brite if joining over IRC / Google Hangout? On Saturday, November 16, 2013 3:43 AM, Roshan Agrawal roshan.agra...@rackspace.commailto:roshan.agra...@rackspace.com wrote: We are all set for the Solum design workshops at SFO on Nov 19,20. The etherpad page below has schedule and agenda details https://etherpad.openstack.org/p/SolumSFOCommunityWorkshop Please sign up as topic leads/scribe for the sessions listed in the agenda. We also have etherpad pages setup for each track of discussion, and we can fill in content right away to prep for the lively discussions next week. We also have a happy hour planned on the evening of Tuesday, and have lunch arrangements on the house. Looking forward to a fun and productive two days at SFO! Thanks Regards, Roshan Agrawal Direct:512.874.1278 Mobile: 512.354.5253 roshan.agra...@rackspace.commailto:roshan.agra...@rackspace.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Version scheme
I'm not a big fan of using date information in the version number. Is there an advantage to doing that? Using a model like 0.0.1 makes it easier to communicate. A better approach might be to use Major.Minor.Revision.Build. If we want to use dates, Year.Month.Day.Build or Year.Minor.Revision.Build might be a better approach. Do any openstack projects use the build number in the version? or is there a way for the build process to insert the build number in there? Thanks, Murali On Nov 14, 2013, at 8:23 AM, Noorul Islam K M noo...@noorul.commailto:noo...@noorul.com wrote: Hello all, We need to decide on version scheme that we are going to use. Monty Taylor said the following in one of the comments for review [1]: Setting a version here enrolls solum in managing its version in a pre-release versioning manner, such that non-tagged versions will indicated that they are leading up to 0.0.1. If that's the model solum wants to do (similar to the server projects) then I recommend replacing 0.0.1 with 2014.1.0. Regards, Noorul [1] https://review.openstack.org/#/c/56130/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Version scheme
Thanks for the clear explanation Monty. -Murali On Nov 14, 2013, at 10:08 AM, Monty Taylor mord...@inaugust.com wrote: On 11/14/2013 10:36 AM, Murali Allada wrote: I'm not a big fan of using date information in the version number. Is there an advantage to doing that? Using a model like 0.0.1 makes it easier to communicate. A better approach might be to use *Major.Minor.Revision.Build*. If we want to use dates, *Year.Month.Day.Build or Year.**Minor.Revision.Build *might be a better approach.. Do any openstack projects use the build number in the version? or is there a way for the build process to insert the build number in there? To be clear, this isn't really a call to design a versioning scheme from scratch - there are two schemes currently in use, and solum should use one of them. The main reason to do 2014.1.0 is to align with OpenStack, so it depends on intent a little bit. The advantage to the Year.Minor.Revision is that, since OpenStack is on a date-based release cycle, it communicates that fact. The main reason to do a semver style Major.Minor.Patch scheme is to communicate api changes across releases. This is the reason we release our libraries using that scheme. In terms of mechanics, the way it works for both schemes is that the version produced is based on git tags. If a revision is tagged, that is the version that is produced in the tarball. If a version is NOT tagged, there are two approaches. Since the date-based versions have a predictable next version, we have intermediary versions marked as leading up to that version. Specifically, the form is: %(version_in_setup_cfg)s.dev%(num_revisions_since_last_tag)s.g%(git_short_sha) the dev prefix is a PEP440 compliant indiciation that this is a development version that is leading towards the version indicated. For semver-based versions, intermediary versions are marked as following the previous release. So we get: %(most_recent_tag)s.%(num_revisions_since_last_tag)s.g%(git_short_sha)s I would honestly recommend aligning with OpenStack and putting 2014.1.0 into the setup.cfg version line for solum itself and doing date-based releases. For python-solumclient, since it's a library, I recommend not listing a version in setup.cfg and doing semver-based versions. This way you'll be operating in the same way as the rest of the project. On Nov 14, 2013, at 8:23 AM, Noorul Islam K M noo...@noorul.com mailto:noo...@noorul.com wrote: Hello all, We need to decide on version scheme that we are going to use. Monty Taylor said the following in one of the comments for review [1]: Setting a version here enrolls solum in managing its version in a pre-release versioning manner, such that non-tagged versions will indicated that they are leading up to 0.0.1. If that's the model solum wants to do (similar to the server projects) then I recommend replacing 0.0.1 with 2014.1.0. Regards, Noorul [1] https://review.openstack.org/#/c/56130/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev