Re: [openstack-dev] [sahara] summit wrap-up: backward compat
On 29 May 2014, at 18:43, Matthew Farrellee wrote: > i do not think we should release any images that have a root password set > (essentially a backdoor). > > for K we should deprecate the hadoop1 versions and thus significantly cut the > size of the new image artifact. > Agree don’t publish images with root password. This option is made for debug purposes and if needed users may build its own image for that. Regards, Alexander Ignatov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
On 05/29/2014 10:22 AM, Sergey Lukjanov wrote: So, it looks like we have an agreement on all question. There is only one technical question - keeping release images means that we need to keep the whole matrix of images: plugin X version X OSy [X root-passwdord]. I'll take a look on total size of them and ability to publish them on OS infra. that's definitely an upper bound. in practice it will be considerably less. for juno we'd have - . vanilla hadoop1 fedora . vanilla hadoop1 ubuntu . vanilla hadoop1 centos6 . ?vanilla hadoop1 centos7? . vanilla hadoop2 fedora . vanilla hadoop2 ubuntu . vanilla hadoop2 centos6 . ?vanilla hadoop2 centos7? . hdp hadoop1 centos . hdp hadoop2 centos . spark ubuntu . ?spark fedora? . ?spark centos? i do not think we should release any images that have a root password set (essentially a backdoor). for K we should deprecate the hadoop1 versions and thus significantly cut the size of the new image artifact. best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
So, it looks like we have an agreement on all question. There is only one technical question - keeping release images means that we need to keep the whole matrix of images: plugin X version X OSy [X root-passwdord]. I'll take a look on total size of them and ability to publish them on OS infra. On Thu, May 29, 2014 at 5:54 PM, Trevor McKay wrote: > Catching up... > > On Thu, 2014-05-29 at 15:59 +0400, Alexander Ignatov wrote: >> On 28 May 2014, at 17:14, Sergey Lukjanov wrote: >> > 1. How should we handle addition of new functionality to the API, >> > should we bump minor version and just add new endpoints? >> >> Agree with most of folks. No new versions on adding new endpoints. >> Semantic changes require new major version of rest api. > > +1 this and previous comments. I don't think we'll generate too many > semantic changes (but I could be wrong :) ) > > I agree with Mike that we should have simple version numbers, v1, v2, v3 > >> > 2. For which period of time should we keep deprecated API and client for >> > it? >> >> One release cycle for deprecation period. > > +1. If we give folks N cycles, they will always wait until the Nth > cycle to move away. Might as well be 1. > >> >> > 3. How to publish all images and/or keep stability of building images >> > for plugins? >> > >> >> We should keep all images for all plugins (non-deprecated as Matt mentioned) >> for each release. In addition we could keep at least one image which could >> be >> downloaded and used with master branch of Sahara. Plugin vendors could keep >> its own set of images and we can reflect it in the docs. > > I agree with keeping all images grouped with a release for all supported > plugins in that release. > > Are we suggesting here that there are 2 places to find images, one in > the Sahara releases and a second in a vendor repo listed in the docs? > >> Regards, >> Alexander Ignatov >> >> >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
Bunch of responses/thoughts: > API I'm ok that semantics additions could be done in one API version w/o increasing minor version. I like the idea to keep only major API versions starting from the next API version. RE backward compat period, for now 1-2 cycles is ok. > Images Agreed that we should publish release images for non-deprecated plugins (somehow, all on infra or vendor-based). On Thu, May 29, 2014 at 3:59 PM, Alexander Ignatov wrote: > > On 28 May 2014, at 17:14, Sergey Lukjanov wrote: >> 1. How should we handle addition of new functionality to the API, >> should we bump minor version and just add new endpoints? > > Agree with most of folks. No new versions on adding new endpoints. > Semantic changes require new major version of rest api. > >> 2. For which period of time should we keep deprecated API and client for it? > > One release cycle for deprecation period. > >> 3. How to publish all images and/or keep stability of building images >> for plugins? >> > > We should keep all images for all plugins (non-deprecated as Matt mentioned) > for each release. In addition we could keep at least one image which could be > downloaded and used with master branch of Sahara. Plugin vendors could keep > its own set of images and we can reflect it in the docs. > > Regards, > Alexander Ignatov > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
Catching up... On Thu, 2014-05-29 at 15:59 +0400, Alexander Ignatov wrote: > On 28 May 2014, at 17:14, Sergey Lukjanov wrote: > > 1. How should we handle addition of new functionality to the API, > > should we bump minor version and just add new endpoints? > > Agree with most of folks. No new versions on adding new endpoints. > Semantic changes require new major version of rest api. +1 this and previous comments. I don't think we'll generate too many semantic changes (but I could be wrong :) ) I agree with Mike that we should have simple version numbers, v1, v2, v3 > > 2. For which period of time should we keep deprecated API and client for it? > > One release cycle for deprecation period. +1. If we give folks N cycles, they will always wait until the Nth cycle to move away. Might as well be 1. > > > 3. How to publish all images and/or keep stability of building images > > for plugins? > > > > We should keep all images for all plugins (non-deprecated as Matt mentioned) > for each release. In addition we could keep at least one image which could > be > downloaded and used with master branch of Sahara. Plugin vendors could keep > its own set of images and we can reflect it in the docs. I agree with keeping all images grouped with a release for all supported plugins in that release. Are we suggesting here that there are 2 places to find images, one in the Sahara releases and a second in a vendor repo listed in the docs? > Regards, > Alexander Ignatov > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
On 28 May 2014, at 17:14, Sergey Lukjanov wrote: > 1. How should we handle addition of new functionality to the API, > should we bump minor version and just add new endpoints? Agree with most of folks. No new versions on adding new endpoints. Semantic changes require new major version of rest api. > 2. For which period of time should we keep deprecated API and client for it? One release cycle for deprecation period. > 3. How to publish all images and/or keep stability of building images > for plugins? > We should keep all images for all plugins (non-deprecated as Matt mentioned) for each release. In addition we could keep at least one image which could be downloaded and used with master branch of Sahara. Plugin vendors could keep its own set of images and we can reflect it in the docs. Regards, Alexander Ignatov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
On 05/28/2014 03:50 PM, Andrew Lazarev wrote: for juno we should just have a v1 api (there can still be a v1.1 endpoint, but it should be deprecated), and maybe a v2 api +1 any semantic changes require new major version number +1 api should only have a major number (no 1.1 or 2.1) In this case we will end up with new major number each release. Even if no major changes were done. a semantic addition (e.g. adding EDP and v1.1) doesn't warrant a new version. so more specifically: +1 any change in existing semantics requires a new major version number but maybe i'm missing why we'd end up w/ a new version per release best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
> for juno we should just have a v1 api (there can still be a v1.1 endpoint, > but it should be deprecated), and maybe a v2 api > > +1 any semantic changes require new major version number > > +1 api should only have a major number (no 1.1 or 2.1) > In this case we will end up with new major number each release. Even if no major changes were done. we should only be producing images for the currently supported plugin > versions. images for deprecated versions can be found with the releases > where the version wasn't deprecated. > agree. We just need to store all images for previous releases somewhere. Andrew. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
On 05/28/2014 01:59 PM, Michael McCune wrote: - Original Message - Open questions 1. How should we handle addition of new functionality to the API, should we bump minor version and just add new endpoints? I think we should not include the minor revision number in the url. Looking at some of the other projects (nova, keystone) it looks like the preference to make the version endpoint able to return information about the specific version implemented. I think going forward, if we choose to bump the minor version for small features we can just change what the version endpoint returns. Any client would then be able to decide if they can use newer features based on the version reported from the return value. If we maintain a consistent version api endpoint then I don't see an issue with increasing the minor version based on new features being added. But, I only endorse this if we decide to solidify the version endpoint (e.g. /v2, not /v2.1). I realize this creates some confusion as we already have /v1 and /v1.1. I'm guessing we might subsume v1.1 at a point in time where we choose to deprecate. i agree, no minor version number. we should even collapse v1.1 and v1 for juno. i don't think we need a capability discovery step. the api should already properly response w/ 404 for endpoints that do not exist. the concern about only discovering a function isn't available until a few steps into a call sequence can be addressed with upfront endpoint detection. and i think this is an extremely rare corner case. 2. For which period of time should we keep deprecated API and client for it? Not sure what the standard for OpenStack project is, but I would imagine we keep the deprecated API version for one release to give users time to migrate. i'd say 1-2 cycles. pragmatically, we will probably never be able to remove api versions. 3. How to publish all images and/or keep stability of building images for plugins? This is a good question, I don't have a strong opinion at this time. My gut feeling is that we should maintain official images somewhere, but I realize this introduces more work in maintenance. for each release we should distribute images for the non-deprecated plugin versions. best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
On 05/28/2014 09:14 AM, Sergey Lukjanov wrote: Hey folks, it's a small wrap-up for the two topics "Sahara backward compat" and " "Hadoop cluster backward compatibility", both were discussed on design summit, etherpad [0] contains info about them. There are some open questions listed in the end of email, please, don't skip them :) Sahara backward compat Keeping released APIs stable since the Icehouse release. So, for now we have one stable API v1.1 (and v1.0 as a subset for it). Any changes to existing semantics requires new API version, additions handling is a question. As part of API stability decision python client should work with all previous Sahara versions. API of python-saharaclient should be stable itself, because we aren't limiting the client version for OpenStack release, so, the client v123 shouldn't change own API exposed to user that is working with stable release REST API versions. for juno we should just have a v1 api (there can still be a v1.1 endpoint, but it should be deprecated), and maybe a v2 api +1 any semantic changes require new major version number +1 api should only have a major number (no 1.1 or 2.1) Hadoop cluster backward compat It was decided to at least keep released versions of cluster (Hadoop) plugins for the next release, so, It means if we have vanilla-2.0.1 released as part of Icehouse, than we could remove it's support only after releasing it as part of Juno with note that it's deprecated and will not be available in the next release. Additionally, we've decided to add some docs with upgrade recommendations. we should only be producing images for the currently supported plugin versions. images for deprecated versions can be found with the releases where the version wasn't deprecated. best, matt Open questions 1. How should we handle addition of new functionality to the API, should we bump minor version and just add new endpoints? 2. For which period of time should we keep deprecated API and client for it? 3. How to publish all images and/or keep stability of building images for plugins? [0] https://etherpad.openstack.org/p/juno-summit-sahara-relmngmt-backward Thanks. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] summit wrap-up: backward compat
- Original Message - > > Open questions > > 1. How should we handle addition of new functionality to the API, > should we bump minor version and just add new endpoints? I think we should not include the minor revision number in the url. Looking at some of the other projects (nova, keystone) it looks like the preference to make the version endpoint able to return information about the specific version implemented. I think going forward, if we choose to bump the minor version for small features we can just change what the version endpoint returns. Any client would then be able to decide if they can use newer features based on the version reported from the return value. If we maintain a consistent version api endpoint then I don't see an issue with increasing the minor version based on new features being added. But, I only endorse this if we decide to solidify the version endpoint (e.g. /v2, not /v2.1). I realize this creates some confusion as we already have /v1 and /v1.1. I'm guessing we might subsume v1.1 at a point in time where we choose to deprecate. > 2. For which period of time should we keep deprecated API and client for it? Not sure what the standard for OpenStack project is, but I would imagine we keep the deprecated API version for one release to give users time to migrate. > 3. How to publish all images and/or keep stability of building images > for plugins? This is a good question, I don't have a strong opinion at this time. My gut feeling is that we should maintain official images somewhere, but I realize this introduces more work in maintenance. regards, mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [sahara] summit wrap-up: backward compat
Hey folks, it's a small wrap-up for the two topics "Sahara backward compat" and " "Hadoop cluster backward compatibility", both were discussed on design summit, etherpad [0] contains info about them. There are some open questions listed in the end of email, please, don't skip them :) > Sahara backward compat Keeping released APIs stable since the Icehouse release. So, for now we have one stable API v1.1 (and v1.0 as a subset for it). Any changes to existing semantics requires new API version, additions handling is a question. As part of API stability decision python client should work with all previous Sahara versions. API of python-saharaclient should be stable itself, because we aren't limiting the client version for OpenStack release, so, the client v123 shouldn't change own API exposed to user that is working with stable release REST API versions. > Hadoop cluster backward compat It was decided to at least keep released versions of cluster (Hadoop) plugins for the next release, so, It means if we have vanilla-2.0.1 released as part of Icehouse, than we could remove it's support only after releasing it as part of Juno with note that it's deprecated and will not be available in the next release. Additionally, we've decided to add some docs with upgrade recommendations. > Open questions 1. How should we handle addition of new functionality to the API, should we bump minor version and just add new endpoints? 2. For which period of time should we keep deprecated API and client for it? 3. How to publish all images and/or keep stability of building images for plugins? [0] https://etherpad.openstack.org/p/juno-summit-sahara-relmngmt-backward Thanks. -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
