Re: [openstack-dev] [Keystoneclient] [Keystone] [Solum] Last released version of keystoneclient does not work with python33
On Wed, Dec 4, 2013 at 6:44 PM, Adrian Otto wrote: > Jamie, > > Thanks for the guidance here. I am checking to see if any of our > developers might take an interest in helping with the upstream work. At the > very least, it might be nice to have some understanding of how much work > there is to be done in HTTPretty. (Dolph correct me if I am wrong, but...) I don't think that there is much work to be done beyond getting that pull request merged upstream. Dolph ran the tests using the code from the pull request somewhat successfully. The errors that we saw were just in keystoneclient code. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][keystone] Keystoneclient tests to tempest
On Sun, Dec 8, 2013 at 3:37 PM, Matt Riedemann wrote: > > > On Sunday, December 08, 2013 11:26:07 AM, Brant Knudson wrote: > >> >> We'd like to get the keystoneclient tests out of keystone. They're >> serving a useful purpose of catching problems with non-backwards >> compatible changes in keystoneclient so we still want them run. >> Problem is they're running at the wrong time -- only on changes to >> keystone and not changes to keystoneclient. >> >> The tests need to be run: >> >> When keystoneclient changes >> - run the tests against the change >> >> When the tests change >> - run the change against the current keystoneclient and also old clients >> >> When keystone changes >> - run the tests against the change with current client >> >> So here's what I think we need to do to get keystone client tests out >> of keystone: >> >> 1) Figure out where to put the tests - is it tempest or something else? >> 2) Write up a test and put it there >> 3) Have a job that when there's a change in the tests it runs against >> current client lib >> 4) Expand the job to also run against old clients >> - or is there 1 job per version? >> - what versions? (keystone does master, essex-3, and 0.1.1) >> - e.g. tox -e master,essex-3,0.1.1 >> - suggest start with these versions and then consider what to use >> in future >> 5) Now we can start adding tests >> 6) Have a job that when there's a change in keystoneclient it runs >> these tests against the change >> 7) When there's a change in keystone, run these tests against the change >> 8) Copy the keystoneclient tests from keystone to the new location -- >> will require some changes >> 9) Remove the tests from keystone \o/ >> 10) Move tests back to keystone where makes sense -- use webtest like >> v3 tests >> >> I created an etherpad with this same info so it's easier to discuss: >> https://etherpad.openstack.org/p/KeystoneTestsToTempest >> >> - Brant >> >> > I'll ask the super obvious question, why not move the keystoneclient tests > to keystoneclient? > > I believe Brant is talking about the tests that use different versions of the keystone client against the keystone server. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Hierarchicical Multitenancy Discussion
That's why I love this site: http://www.timeanddate.com/worldclock/fixedtime.html?iso=20140130T2100 On Thu, Jan 30, 2014 at 1:46 PM, Vishvananda Ishaya wrote: > Thanks Soren, you are correct! Yay Timezones > > Vish > > On Jan 30, 2014, at 10:39 AM, Soren Hansen wrote: > > 2100 UTC is 1 PM Pacific. :-) > Den 29/01/2014 17.01 skrev "Vishvananda Ishaya" : > >> I apologize for the confusion. The Wiki time of 2100 UTC is the correct >> time (Noon Pacific time). We can move tne next meeting to a different >> day/time that is more convienient for Europe. >> >> Vish >> >> >> On Jan 29, 2014, at 1:56 AM, Florent Flament < >> florent.flament-...@cloudwatt.com> wrote: >> >> > Hi Vishvananda, >> > >> > I would be interested in such a working group. >> > Can you please confirm the meeting hour for this Friday ? >> > I've seen 1600 UTC in your email and 2100 UTC in the wiki ( >> https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting). >> As I'm in Europe I'd prefer 1600 UTC. >> > >> > Florent Flament >> > >> > - Original Message - >> > From: "Vishvananda Ishaya" >> > To: "OpenStack Development Mailing List (not for usage questions)" < >> openstack-dev@lists.openstack.org> >> > Sent: Tuesday, January 28, 2014 7:35:15 PM >> > Subject: [openstack-dev] Hierarchicical Multitenancy Discussion >> > >> > Hi Everyone, >> > >> > I apologize for the obtuse title, but there isn't a better succinct >> term to describe what is needed. OpenStack has no support for multiple >> owners of objects. This means that a variety of private cloud use cases are >> simply not supported. Specifically, objects in the system can only be >> managed on the tenant level or globally. >> > >> > The key use case here is to delegate administration rights for a group >> of tenants to a specific user/role. There is something in Keystone called a >> "domain" which supports part of this functionality, but without support >> from all of the projects, this concept is pretty useless. >> > >> > In IRC today I had a brief discussion about how we could address this. >> I have put some details and a straw man up here: >> > >> > https://wiki.openstack.org/wiki/HierarchicalMultitenancy >> > >> > I would like to discuss this strawman and organize a group of people to >> get actual work done by having an irc meeting this Friday at 1600UTC. I >> know this time is probably a bit tough for Europe, so if we decide we need >> a regular meeting to discuss progress then we can vote on a better time for >> this meeting. >> > >> > >> https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting >> > >> > Please note that this is going to be an active team that produces code. >> We will *NOT* spend a lot of time debating approaches, and instead focus on >> making something that works and learning as we go. The output of this team >> will be a MultiTenant devstack install that actually works, so that we can >> ensure the features we are adding to each project work together. >> > >> > Vish >> > >> > ___ >> > OpenStack-dev mailing list >> > OpenStack-dev@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> > ___ >> > OpenStack-dev mailing list >> > OpenStack-dev@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance] IRC logging
It's also important to remember that IRC channels are typically not private and are likely already logged by dozens of people anyway. On Tue, Jan 6, 2015 at 1:22 PM, Christopher Aedo wrote: > On Tue, Jan 6, 2015 at 2:49 AM, Flavio Percoco wrote: > > Fully agree... I don't see how enable logging should be a limitation > > for freedom of thought. We've used it in Zaqar since day 0 and it's > > bee of great help for all of us. > > > > The logging does not remove the need of meetings where decisions and > > more relevant/important topics are discussed. > > Wanted to second this as well. I'm strongly in favor of logging - > looking through backlogs of chats on other channels has been very > helpful to me in the past, and it sure to help others in the future. > I don't think there is danger of anyone pointing to a logged IRC > conversation in this context as some statement of record. > > -Christopher > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] Nominating Brad Topol for Keystone-Spec core
+1 On Sun, Jan 18, 2015 at 2:11 PM, Morgan Fainberg wrote: > Hello all, > > I would like to nominate Brad Topol for Keystone Spec core (core reviewer > for Keystone specifications and API-Specification only: > https://git.openstack.org/cgit/openstack/keystone-specs ). Brad has been > a consistent voice advocating for well defined specifications, use of > existing standards/technology, and ensuring the UX of all projects under > the Keystone umbrella continue to improve. Brad brings to the table a > significant amount of insight to the needs of the many types and sizes of > OpenStack deployments, especially what real-world customers are demanding > when integrating with the services. Brad is a core contributor on pycadf > (also under the Keystone umbrella) and has consistently contributed code > and reviews to the Keystone projects since the Grizzly release. > > Please vote with +1/-1 on adding Brad to as core to the Keystone Spec > repo. Voting will remain open until Friday Jan 23. > > Cheers, > Morgan Fainberg > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Proposal: add local hacking for oslo-incubator
On Mon, May 5, 2014 at 5:28 PM, Doug Hellmann wrote: > > > > The assert ones do seem to fit the best practices as I understand them, > but > > I suspect there's going to be quite a bit of work to get projects > compliant. > > I've seen some work being done on that already, but I don't know how > strongly we care about those specific rules as an overall project. > > I created the Keystone blueprint[1] to automate the things we already check for in reviews. My motivation was to make it faster for contributors to contribute because they would get feedback before getting a bunch of -1s in Gerrit. I also wanted to free up core dev resources so that we can focus on more important parts of reviews. I'd be happy to start putting some of these in hacking, but I don't know which rules would be acceptable to all projects. Maybe there is a way to make optional checks that can be enabled in the tox.ini 1. https://blueprints.launchpad.net/keystone/+spec/more-code-style-automation -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Barbican][OSSG][Keystone] Mid-Cycle Meetup
On Thu, May 22, 2014 at 10:48 AM, Jarret Raim wrote: > > This should make travel a bit easier for everyone as people won't need Hey Jarret, I'm going to be at the Keystone meetup for sure, but I'm also thinking about going to the Barbican meetup too. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Redesign of Keystone Federation
On Thu, May 29, 2014 at 12:59 PM, Morgan Fainberg wrote: > > > David and Kristy, the slides and summit session are a great starting place > for this work. Now we need to get the proposal drafted up in the new > Keystone-Specs repository ( > https://git.openstack.org/cgit/openstack/keystone-specs ) so we can keep > this conversation on track. Having the specification clearly outlined and > targeted will help us address any concerns with the proposal/redesign as we > move into implementation. > I can't tell from the linked slides what exactly will change. When a spec is proposed I'd love to see at least the following things: - A description/diagram of the layers and how data flows between them. - A description/diagram showing structural code changes that need to happen to support the proposal. - A breakdown of user facing changes for things that can't be completely backward compatible. The talk at the summit seemed to revolve around a few methods and a class that had SAML in the name. I think some of the recent work is making this more generic and there is at least one proposal to add protocols. It would be nice to see why this refactoring isn't enough. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [All] API standards working group
On Wed, Sep 24, 2014 at 11:42 AM, Dean Troyer wrote: > On Tue, Sep 23, 2014 at 5:18 PM, Jay Pipes wrote: > >> Yes, I'd be willing to head up the working group... or at least >> participate in it. >> > > I'll bring an API consumer's perspective. > > > I would love to participate too. I have an interest in RESTful API design and the surrounding architecture. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] external AuthN Identity Backend
On Thu, Oct 16, 2014 at 2:54 PM, Dave Walker wrote: > Hi Steve, > > Thanks for your response. I am talking generally about the external > auth support. One use case is Kerberos, but for the sake of argument > this could quite easily be Apache Basic auth. The point is, we have > current support for entrusting AuthN outside of Keystone. > > What I was trying to outline is that it seems that the current design > of external auth is that keystone is not in the auth pipeline as we > trust auth at the edge. However, we then do additional auth within > keystone. > > With external auth and SQL, we drop the user provided username and > password on the floor and use what was provided in REMOTE_USER (set by > the webserver). > > Therefore the check as it currently stands in SQL is basically 'is > this username in the database'. The LDAP plugin does Authentication > via username and password, which is clearly not sufficient for > external auth. The LDAP plugin could be made to check in a similar > manner to SQL 'is this a valid user' - but this would seem to be a > duplicate check, as we already did this at the edge. > > If the webserver granted access to keystone, the user has already been > checked to see if they are a valid user. However, your response seems > to suggest that current external auth should be formally deprecated? I may be missing something, but can you use the external auth method with the LDAP backend? -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] Alternative federation mapping
On Tue, Nov 4, 2014 at 10:30 AM, John Dennis wrote: > O.K. group assignment is the final goal in Keystone. I suppose the > relevant question then is the functionality in the current Keystone > mapper sufficiently rich such that you can present to it an arbitrary > set of values and yield a group assessment? It's been a while since I > looked at the mapper, things might have changed, but it seemed to me it > had a lot of baked in assumptions about the data (assertion) it would > receive. As long as those assumptions held true all is good. > There are probably a few other assumptions, but the main one is that the mapper expects the incoming data to be a dictionary where the value is a string. If there are multiple values we expect them to be delimited with a semicolon in the string. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Question on decorators in Ceilometer pecan framework
It looks like maybe WSME or Pecan is inspecting the method signature. Have you tried to change the order of the decorators? On Aug 8, 2014, at 9:16, Pendergrass, Eric wrote: > Wrong link again, this is embarrassing L > https://review.openstack.org/#/c/112137/3 > > From: Pendergrass, Eric > Sent: Friday, August 08, 2014 7:15 AM > To: openstack-dev@lists.openstack.org > Subject: RE: [Ceilometer] Question on decorators in Ceilometer pecan framework > > Sorry, wrong BP review link below. Here is the correct one: > https://review.openstack.org/#/c/112127/3. Please disregard the wiki link. > > From: Pendergrass, Eric > Sent: Friday, August 08, 2014 6:50 AM > To: openstack-dev@lists.openstack.org > Cc: Giannetti, Fabio > Subject: [Ceilometer] Question on decorators in Ceilometer pecan framework > > Hi, > > We have been struggling to get a decorator working for proposed new RBAC > functionality in ceilometer-api. We’re hitting a problem where GET request > query parameters are mucked up by our decorator. Here’s an example call: > > curl -H "X-Auth-Token:$TOKEN" > 'http://localhost:8777/v2/meters?q.field=project_id&q.value=8c678720fb5b4e3bb18dee222d7d7933' > > And here’s the decorator method (we’ve tried changing the kwargs, args, etc. > with no luck): > > _ENFORCER = None > > def protected(controller_class): > > global _ENFORCER > if not _ENFORCER: > _ENFORCER = policy.Enforcer() > > def wrapper(f): > @functools.wraps(f) > def inner(self, **kwargs): > pdb.set_trace() > self._rbac_context = {} > if not _ENFORCER.enforce('context_is_admin', > {}, > {'roles': > pecan.request.headers.get('X-Roles', "").split(",")}): > self._rbac_context['project_id'] = > pecan.request.headers.get('X-Project-Id') > self._rbac_context['user_id'] = > pecan.request.headers.get('X-User-Id') > return f(self, **kwargs) > return inner > return wrapper > > tried this too: > > _ENFORCER = None > > def protected(*args): > > controller_class = 'meter' > global _ENFORCER > if not _ENFORCER: > _ENFORCER = policy.Enforcer() > > def wrapper(f, *args): > def inner(self, *args): > pdb.set_trace() > #self._rbac_context = {} > #if not _ENFORCER.enforce('context_is_admin', > # {}, > # {'roles': > pecan.request.headers.get('X-Roles', "").split(",")}): > #self._rbac_context['project_id'] = > pecan.request.headers.get('X-Project-Id') > #self._rbac_context['user_id'] = > pecan.request.headers.get('X-User-Id') > #return f(*args) > f(self, *args) > return inner > return wrapper > > and here’s how it’s used: > > class MetersController(rest.RestController): > """Works on meters.""" > > _rbac_context = {} > @pecan.expose() > def _lookup(self, meter_name, *remainder): > return MeterController(meter_name), remainder > > @wsme_pecan.wsexpose([Meter], [Query]) > @rbac_validate.protected('meters') > def get_all(self, q=None): > """Return all known meters, based on the data recorded so far. > > :param q: Filter rules for the meters to be returned. > """ > q = q or [] … > > > but we get errors similar to below where the arg parser cannot find the query > parameter because the decorator doesn’t take a q argument as > MetersController.get_all does. > > Is there any way to get a decorator to work within the v2 API code and wsme > framework or should we consider another approach? Decorators would really > simplify the RBAC idea we’re working on, which is mostly code-implemented > save for this fairly major problem. > > I have a WIP registered BP on this > athttps://blueprints.launchpad.net/ceilometer/+spec/ready-ceilometer-rbac-keystone-v3. > > If I can provide more details I’ll be happy to. > > Thanks > Eric > > /usr/local/bin/ceilometer-api(10)() > -> sys.exit(api()) > /opt/stack/ceilometer/ceilometer/cli.py(96)api() > -> srv.serve_forever() > /usr/lib/python2.7/SocketServer.py(227)serve_forever() > -> self._handle_request_noblock() > /usr/lib/python2.7/SocketServer.py(284)_handle_request_noblock() > -> self.process_request(request, client_address) > /usr/lib/python2.7/SocketServer.py(310)process_request() > -> self.finish_request(request, client_address) > /usr/lib/python2.7/SocketServer.py(323)finish_request() > -> self.RequestHandlerClass(request, client_address, self) > /usr/lib/python2.7/SocketServer.py(638)__init__() > -> self.handle() > /usr/lib/python2.7/wsgiref/simple_server.py(124)handle() > -> handler.run(self.server.get_app()) > /usr/lib/python2.7/wsgiref/handlers.py(85)run() > -> self.res
Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest
On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague wrote: > On 10/18/2013 12:04 PM, Brant Knudson wrote: > >> >> 2) "git clone"ing the keystoneclient doesn't work well with parallel >> testing (we have a similar problem in our tests with our "pristine" >> database backup) >> > > Can you go into the specifics of why? We use unsafe paths for the test SQLite database and test config files. Instead of using something like tempfile we are using hardcoded paths. When the setUp method is run in parallel it will stomp on other tests. I believe the 'git clone' is the same way. The clone happens in the setUp so if you have 2 test methods in that test class one of the cloning operations will break. I have a bug filed for the DB/config file issue already. The cloning issue may solved by putting it into the setupClass instead of setUp. I'd have to try it. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] RFC - Icehouse logging harmonization
On Wed, Oct 23, 2013 at 3:26 PM, Robert Collins wrote: > On 24 October 2013 08:14, Dolph Mathews wrote: > > > > On Wed, Oct 23, 2013 at 1:20 PM, Sean Dague wrote: > > > > > Deprecation warnings! > > > > Based on the approach we're taking in the patch below, we'll be able to > > notate how imminently a feature is facing deprecation. Right now, they're > > just landing in WARNING, but I think we'll surely a desire to silence, > > prioritize or redirect those messages using different log levels (for > > example, based on how imminently a feature is facing deprecation). > > > > https://review.openstack.org/#/c/50486/ > > Huh, I did not see that go by. Python already has built in signalling > for deprecated features; I think we should be using that. We can of > course wrap it with a little sugar to make it easy to encode future > deprecations. > The initial patch used the the warnings modules instead of using the deprecated logging provided by oslo. We talked about it IRC and I switched because there existed a way to do deprecation messaging is olso and it had a configurable way to turn warnings into exceptions. I intend to submit a patch to oslo-incubator based off of the above patch. I'd love to get some feedback about the needs of the other projects. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Remove vim modelines?
On Thu, Oct 24, 2013 at 2:48 PM, Robert Collins wrote: > > > *) They help casual contributors *more* than long time core > contributors : and those are the folk that are most likely to give up > and walk away. Keeping barriers to entry low is an important part of > making OpenStack development accessible to new participants. > I'm not sure I buy the low barrier to entry in this case. We could easily provide a more comprehensive vimscript snippet that experienced vim users can include into their vimrc. Not only that, but we can do the same for any other editor. If a user is not familiar with vim, I don't think we can/should do much to teach users how to use their editor. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Bad review patterns
On Wed, Nov 6, 2013 at 7:21 PM, Day, Phil wrote: > > > > Leaving a mark. > > === > > > > You review a change and see that it is mostly fine, but you feel that > since you > > did so much work reviewing it, you should at least find > > *something* wrong. So you find some nitpick and -1 the change just so > that > > they know you reviewed it. > > > > This is quite obvious. Just don't do it. It's OK to spend an hour > reviewing > > something, and then leaving no comments on it, because it's simply fine, > or > > because we had to means to test someting (see the first pattern). > > > > > > Another one that comes into this category is adding a -1 which just says > "I agree with > the other -1's in here". If you have some additional perspective and can > expand on > it then that's fine - otherwise it adds very little and is just review > count chasing. > > It's an unfortunate consequence of counting and publishing review stats > that having > such a measure will inevitable also drive behavour. I agree that having no comment with a +1 is OK. If I think the code looks good I'll +1 it. If I think the code could be better I'll -1 it and leave comments. I don't think that leaving a -1 with a 'what they said' comment is a bad thing. As the developer writing the patch it's helpful to know there is a consensus and it's not just one person requesting a change. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [style] () vs \ continuations
On Mon, Nov 18, 2013 at 8:23 PM, Vishvananda Ishaya wrote: > > > +1 to sticking something in hacking. FWIW I would probably do the following > to avoid the debate altogether: > > result = self._path_file_exists(ds_browser, folder_path, file_name) > folder_exists, file_exists, file_size_in_kb, disk_extents = result > > Vish > > I'm working on some test code and came up with what I think is a valid use of a line ending in \. I'm not escaping the newline from a Python syntax point of view. I'm doing in inside of a string literal. Example: mismatches_description = textwrap.dedent("""\ expected = http://docs.openstack.org/identity/api/v2.0 "> actual = http://docs.openstack.org/identity/api/v2.0";> """) This can be written in several different ways, but I think that none of them is as clear as the trailing slash. This is, I think, the clearest alternative example: mismatches_description = textwrap.dedent(""" expected = http://docs.openstack.org/identity/api/v2.0 "> actual = http://docs.openstack.org/identity/api/v2.0";> """).lstrip() Would this use be forbidden too? -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
On Thu, Jul 11, 2013 at 5:20 AM, Mark McLoughlin wrote: > > But I think what you're saying is missing is the stack trace from the > underlying exception. > > As I understood it, Python doesn't have a way of chaining exceptions > like this but e.g. Java does. A little bit more poking right now shows > up this: > > http://www.python.org/dev/peps/pep-3134/ > > i.e. we can't do the right thing until Python 3, where we'd do: > > def download_image(host, port, path): > try: > s = socket.create_connection((host, port)) > except socket.error as e: > raise ImageDownloadFailure(host, port, path, e.strerror) from e > > I haven't read the PEP in detail yet, though. > > You can actually do this in Python 2 and keep the original context: def download_image(host, port, path): try: s = socket.create_connection((host, port)) except socket.error as e: raise ImageDownloadFailure, e, sys.exc_info()[-1] This will keep the original message and stack trace, but change the type. You can also change the message if you want my mucking with e's message. I've done that to add a string like " (socket.error)" at the end of the exception message so I could see the original type. If you really, really wanted to use a bare except you could also do something like: try: do_something_that_raises_an_exception() except: exc_value, exc_tb = sys.exc_info()[1:] raise MyException, exc_value, exc_tb -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Devstack] is dogpile.cache a requirement?
On Wed, Sep 4, 2013 at 10:23 AM, Dolph Mathews wrote: > > On Wed, Sep 4, 2013 at 9:14 AM, Salvatore Orlando wrote: > >> whenever I run devstack keystone falies to start because dogpile.cache is >> not installed; this is easily solved by installing it, but I wonder if it >> should be in requirements.txt >> Also, since the cache appears to be disabled by default (and I'm not >> enabling it in my localrc), I'm not sure why I am hitting this error, as I >> would expect the caching module to not be loaded at all. >> >> > That sounds like a bug! It should only be a hard requirement if > keystone.conf [cache] enabled=True > > > Currently keystone.assignment.core imports keystone.common.cache with ends up depending on dogpile. The current implementation does depend on dogpile even if caching isn't being used. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][all] Move from active distrusting model to trusting model
On Mon, Nov 23, 2015 at 11:52 AM Dmitry Tantsur wrote: > On 11/23/2015 05:42 PM, Morgan Fainberg wrote: > > Hi everyone, > > > [snip,snip] > > > > This type of policy is an actively distrustful policy. > I don't see it quite like that. I don't think the policy is there because I'm not trusted to make good decisions, I see it more as protecton from the appearance that I am doing it. I trust all of the Keystone cores and would have no issue lifting the policy. The question is if that opens us up to scrutiny (or anything else) from outside our development group. I don't know the history behind the policy, so I can't answer that question. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][all] Move from active distrusting model to trusting model
On Mon, Nov 23, 2015 at 6:06 PM David Chadwick wrote: [snip] > > > > This is just a vote for distrusting the community. If you think there's > > "power" in being able to merge things, and that organizations will abuse > > this power, then you vote for distrust. > > No, rather for the abuse of power by organisations, which is rather > different :-) We have multiple examples of these all the time. > I don't know if it's fair to say that voting for the current, strict rules is the same as voting for distrust. I also don't know if it's fair to say that we have multiple examples of organizations abusing power without clarifying if you mean in our community. As I said earlier I can see these rules as being protective. Similar to how organizations have lots of rules about getting gifts from suppliers, customers, etc. and a variety of other 'conflict of interest' situations. Much of that is useful to avoid the appearance of impropriety. Simple rules can keep you out of some bad situations. Is that valuable at all within our community? Is that valuable to protect our community from outside forces? -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] orchestration and db_sync
On Fri, May 27, 2016 at 12:08 PM, Ryan Hallisey wrote: Theses changes do not all happen at the same times for an OpenStack installation. > - Create the service's users and add a password into the databse Should only happen once during installation. > - Sync the service with the database Should happen during installation and for every upgrade. > - Start the service > > I was wondering if for some services they could be aware of whether or not > they need > to sync with the database at startup. Or maybe the service runs a db_sync > every time > is starts? I figured I would start a thread about this because Keystone has > some > flexibility when running N+1 in a cluster of N. If Keystone could have that > that ability maybe Keystone could db_sync each time it starts without harming > the > cluster? This isn't something I would want to see for a few reasons. The most important one is that I think the decision to run db_sync needs to be explicit. An operator should run it when they are ready (maybe they need to shut something down, ensure up-to-date backups, etc.). Another issue is database modification permissions. The user running the application, as well as the DB user the application uses, shouldn't have access to DML for security reasons. Little Bobby Tables' mom found this out the hard way[1]. 1. https://xkcd.com/327/ -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Placement API WSGI code -- let's just use flask
On Tue, Jun 21, 2016 at 08:00:50AM -0400, Sean Dague wrote: > > Keystone - custom WSGI with Routes / Paste > Keystone is moving toward Flask. I have an experimental patch that moves us in that direction. I'm in the process to rebasing and fixing it up since it's wildly out of date. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [security] [horizon] Security implications of exposing a keystone token to a JS client
On 06/29 at 21:10, Timur Sufiev wrote: > Hello, vigilant folks of OpenStack Security team! > > The commit(s) I'd like you to take a look at introduces a new Horizon > feature, Create (Glance) Image using CORS (AKA Cross-Origin Resource > Sharing) [1]. > > The main idea is to bypass Horizon web-server when uploading large local > image and to send it directly to Glance server, thus saving network > bandwidth and disk space on the controller node where Horizon web-server is > deployed. However there is one possible security trade-off I had to make so > that Glance service would allow me to upload an image - I'm passing the > Keystone token to the Horizon JS runtime [2], and then pass it to Glance > service [3] or [4] (different links here correspond to different versions > of new Create Image - Django and Angular). This trade-off made Horizon > community somewhat hesitant if we should push these changes forward, but > nobody yet voiced a viable alternative, so here I'm writing this letter to > you. > > The usual Horizon workflow for working with Keystone tokens is the > following: retrieve scoped token and put it into web-server session, which > is itself not exposed to browser (unless SESSION_STORAGE signed_cookies > backend was chosen, but even in that case session contents are encrypted in > some way), but is kept on web-server and referenced using the session key > which is kept in browser cookies - so one may say that in existing setup > keystone token never leaks to browser. > > On the other hand, in some not so far (I hope) future, when more logic is > moved to client-side UI (i.e. browser), the issue of browser authenticating > to some OpenStack services directly would become more widespread, it just > happened that this work on Create Image in Horizon is pioneering this area > (AFAIK). So, what do you think of possible security implications of this > setup? > > Just for the reference, three patches mentioned in [1-3] implement most of > the logic of new Create Image feature. > > [1] > https://blueprints.launchpad.net/horizon/+spec/horizon-glance-large-image-upload > [2] > https://review.openstack.org/#/c/317365/15/openstack_dashboard/api/glance.py@215 > [3] > https://review.openstack.org/#/c/230434/37/horizon/static/horizon/js/horizon.modals.js@212 > [4] > https://review.openstack.org/#/c/317456/16/openstack_dashboard/static/app/core/openstack-service-api/glance.service.js@151 Since tokens are bearer tokens any leak could possibly lead to a security issue. I don't see allowing the JS application to have access to the token as being a terrible thing. We just need to make sure we do it as safely as we can in order to prevent the token from lingering around after the web session has completed. For example, putting the token in redirect URLs may cause it to end up in browser history, putting it in the source of page that could be cached may write it to disk, etc, etc. -- David Stanek web: http://dstanek.com blog: http://traceback.org __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Syntribos Error : AttributeError: 'tuple' object has no attribute 'headers'
On 07/02 at 15:52, OpenStack Mailing List Archive wrote: > Link: https://openstack.nimeyo.com/89478/?show=89478#q89478 > From: run2obtain > > > Hi... I tried to use OpenStack Syntribos today for security testing against my > devstack kilo cloud. I followed installation and configuration instructions > provided at the openstack syntribos repo .Unfortunately, I received some > errors > after running the command : syntribos keystone.config .opencafe/templates/ > keystone/roles_get.txt . The errors are File "/usr/local/lib/python2.7/ > dist-packages/syntribos/extensions/identity/client.py", line 146, in > get_token_v3 return r.headers["X-Subject-Token"] AttributeError: 'tuple' > object > has no attribute 'headers'. ' I have not been successful at discovering what > could be wrong or how to resolve this issue, even after googling. Does anyone > have a hint as to how to resolve this issue. Many thanks for your anticipated > response. > A quick look at the code[1] show that the HTTP client used by the identity client actually returns a tuple instead of a response object. The tuple contains two items: a response object and a signal handler object. This is the first I've heard of this project, so I was very disappointed to not find any docs for it. 1. https://github.com/openstack/syntribos/blob/master/syntribos/clients/http/base_http_client.py#L143 -- David Stanek web: http://dstanek.com blog: http://traceback.org __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [security] [horizon] Security implications of exposing a keystone token to a JS client
On 07/01 at 19:41, Fox, Kevin M wrote: > Hi David, > > How do you feel about the approach here: > https://review.openstack.org/#/c/311189/ > > Its lets the existing angular js module: > horizon.app.core.openstack-service-api.keystone > > access the current token via getCurrentUserSession().token > Hey Kevin, It's hard to tell without a lot of the context. From what I can tell the token is pulled down as part of the data of an API request. As long as that's not cached I think you are OK. -- David Stanek web: http://dstanek.com blog: http://traceback.org __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][ptl][keystone] Proposal to split authentication part out of Keystone to separated project
On Wed, Apr 6, 2016 at 3:26 PM Boris Pavlovic wrote: > > 2) This will reduce scope of Keystone, which means 2 things > 2.1) Smaller code base that has less issues and is simpler for testing > 2.2) Keystone team would be able to concentrate more on fixing > perf/scalability issues of authorization, which is crucial at the moment > for large clouds. > I'm not sure that this is entirely true. If we truly just split up the project, meaning we don't remove functionality, then we'd have the same number of bugs and work. It would just be split across two projects. I think the current momentum to get out of the authn business is still our best bet. As Steve mentioned this is ongoing work. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-keystoneclient] Return request-id to caller
On Wed, Apr 13, 2016 at 3:26 AM koshiya maho wrote: > > My request to all keystone cores to give their suggestions about the same. > > I'll test this a little and see if I can see how it breaks. Overall I'm not really a fan of this design. It's just a hack to add attributes where they don't belong. Long term I think this will be hard to maintain. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] summit tools
On Wed, Apr 20, 2016 at 12:14 PM Neil Jerram wrote: > A couple of questions about our Austin-related planning tools... > > - Can one's calendar at > > https://www.openstack.org/summit/austin-2016/summit-schedule/#day=2016-04-25 > be exported as .ics, or otherwise integrated into a wider calendaring > system? > I put together a tool[1] that can do this. There is some manual work involved because you have to save the print view of your calendar from your browser. It was a quick and dirty hackso patches accepted and encouraged. 1. https://gist.github.com/dstanek/83d536af9461eb26d7a19ff73a7391c2 -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] re-introducing twisted to global-requirements
On Thu, Jan 7, 2016 at 3:01 PM, Jim Rollenhagen wrote: > We'd be using this for functional tests, not unit, where we can't really > inject mocks. The idea is that we could run a full functional suite > against either mimic or a full ironic environment, just by changing a > test setting. > I'm assuming that this would be for client functional tests because it wouldn't make sense for testing a server. How is the interface created? It seems like it would be possible for the mocked API to not match the actual API. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [trove][neutron][cinder][swift][ceilometer][nova][keystone][sahara][glance][neutron-lbaas][imm] stylistic changes to code, how do we handle them?
On Tue, Jan 12, 2016 at 8:51 AM, Amrith Kumar wrote: > I've tagged this message with the projects impacted by a series of change > sets: > > [trove] https://review.openstack.org/#/c/266220/ > [neutron] https://review.openstack.org/#/c/266156/1 > [cinder] https://review.openstack.org/#/c/266099/2 > [swift] https://review.openstack.org/#/c/266185/1 > [ceilometer] https://review.openstack.org/#/c/266211/1 > [nova] https://review.openstack.org/#/c/266143/2 > [keystone] https://review.openstack.org/#/c/266203/2 > [sahara] https://review.openstack.org/#/c/266230/1 > [glance] https://review.openstack.org/#/c/266192/1 > [neutron-lbaas] https://review.openstack.org/#/c/266181/1 > > I would like the guidance of the developer community in figuring out how > to proceed with this change, and changes like this. > > The change, in essence changes a construct of the kind: > > if var > 0: > ... something ... > > To > > if var: > ... something ... > > In a couple of cases, it also changes messages from (for example, in Trove) > > "Limit value must be > 0" to > "Limit value must be greater than zero" > > My question to the ML is this, should stylistic changes of this kind be > handled in a consistent way across all projects, maybe with a hacking rule > and some discussion on the ML first? After all, if this change is > worthwhile, it is worth ensuring that this construct that we are seeking to > eliminate, does not reenter the code base. > The code above is not stylistic in nature. It actually changes the semantics of the check. This should not be done blindly across the board. In the first case it looks for a value to be greater than a constant. In the second case it looks for a value to have a truthy value. > For what it is worth, I agree with Vitaly Grindev [sahara, in review > https://review.openstack.org/#/c/266230/1]. I think the code before the > change was more intuitive and readable. > Only if the code is correct. In that review I saw you changing "len(x) > 0" to "len(x)" and that's fine. But you also changed a check looking at the status code from a conductor call to no longer allow negative return values. I have no idea if it returns negative numbers, but in this case I would guess the "> 0" had a purpose. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [trove][neutron][cinder][swift][ceilometer][nova][keystone][sahara][glance][neutron-lbaas][imm] stylistic changes to code, how do we handle them?
On Tue, Jan 12, 2016 at 9:13 AM, Amrith Kumar wrote: > Chris, Ihar, > > I assumed that this was stylistic based on the fact that in the places > where I was seeing it, it seemed to be the case that the LHS was > intuitively positive (a length, for example). I did not exhaustively verify > this but yes, you are correct, the construct > > If var > > is the same as (I believe) > > if var is not None Those two statements are also not the same. 'if var' will be True is 'var' is *only* True values. 'if var is not None' will be True *everything* except None, so 0, '', [], etc. will not pass the first test, but will pass the second. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Fernet] HA SQL backend for Fernet keys
On Mon, Aug 3, 2015 at 7:14 AM, Davanum Srinivas wrote: > agree. "Native HA solution" was already ruled out in several email > threads by keystone cores already (if i remember right). This is a > devops issue and should be handled as such was the feedback. > I'm sure you are right. I'm not sure why we would want to add that much complexity into Keystone. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Fernet] HA SQL backend for Fernet keys
On Sat, Aug 1, 2015 at 8:03 PM, Boris Bobrov wrote: > On Sat, Aug 1, 2015 at 3:41 PM, Clint Byrum wrote: > > > This too is overly complex and will cause failures. If you replace key 0, > > > you will stop validating tokens that were encrypted with the old key 0. > > > > No. Key 0 is replaced after rotation. > > > > Also, come on, does http://paste.openstack.org/show/406674/ look overly > complex? (it should be launched from Fuel master node). > I'm reading this on a small phone, so I may have it wrong, but the script appears to be broken. It will ssh to node-1 and rotate. In the simplest case this takes key 0 and moves it to the next highest key number. Then a new key 0 is generated. Later there is a loop that will again ssh into node-1 and run the rotation script. If there is a limit set on the number of keys and you are at that limit a key will be deleted. This extra rotation on node-1 means that it's possible that it has a different set of keys than are on node-2 and node-3. What's the issue with just a simple rsync of the directory? -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] FFE Request for completion of data driven assignment testing in Keystone
On Thu, Sep 3, 2015 at 3:44 PM Henry Nash wrote: > > I would like to request an FFE for the remaining two patches that are > already in review (https://review.openstack.org/#/c/153897/ and > https://review.openstack.org/#/c/154485/). These contain only test code > and no functional changes, and increase our test coverage - as well as > enable other items to be re-use the list_role_assignment backend method. > Do we need a FFE for changes to tests? -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] creating new users with invalid mail addresses possible
On Fri, Sep 11, 2015 at 8:26 AM, Christian Berendt wrote: > At the moment it is possible to create new users with invalid mail > addresses. I pasted the output of my test at > http://paste.openstack.org/show/456642/. (the listing of invalid mail > addresses is available at > http://codefool.tumblr.com/post/15288874550/list-of-valid-and-invalid-email-addresses > ). > > Is it intended that addresses are not be validated? > > Does it makes sense to validate addresses (e.g. with > https://github.com/mailgun/flanker)? > I don't know the complete history of this (I'm sure others can chime in later), but since Keystone doesn't use the email address for anything it was never really considered a first class attribute. It is just something we accept and return through the API. It doesn't even have its own column in the database. I don't like this for a variety of reasons and we do have a bug[1] for fixing this. Last Thursday several of us were discussing making a database column for the email address as part of the fix for that bug. 1. https://bugs.launchpad.net/keystone/+bug/1218682 -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] PTL non-candidacy
On Thu, Sep 10, 2015 at 5:40 PM, Morgan Fainberg wrote: > While I will be changing my focus to spend more time on the general needs > of OpenStack and working on the Public Cloud story, I am confident in those > who can, and will, step up to the challenges of leading development of > Keystone and the associated projects. I may be working across more > projects, but you can be assured I will be continuing to work hard to see > the initiatives I helped start through. I wish the best of luck to the next > PTL. It's been an honor and a privilege to work with you. You've done a great job and I'm sorry to see you go. Fortunately you're not going too far! Good luck with your future OpenStack adventures! -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone] PTL Candidacy
Hello everyone, I'd like to announce my candidacy for the Keystone PTL for the Mitaka cycle. The project has had great leadership the entire time I have been involved and I want to keep up the tradition. I've been working on Keystone for the past 2 years and I'd like to step up and take more of a leadership role. I have a long track record of technical leadership that can be directly applied to Keystone. This includes everything from project management to application architecture and many things in between. My thoughts on the direction of the project really boil down to landing the features we have already committed to, taking our experimental features and making them stable, improving general project stability and expanding our community. Goals for this cycle: * Land the features Several features that are very beneficial to the community are currently in development. We need to get them stable and shipped for Mitaka. This includes, but is not limited to federation, reseller, and centralized policy. There is also additional work that needs to be done with federation to take it from experimental to stable, like helping other projects adapt to the new requirements and making it a first class OpenStack feature. * Fix the bugs We have 293 bugs (at the time of this writing) that need some attention. Many have patches that need some work or just need reviews. Others need to be investigated and fixed. I'd like to cut this number in half. To do that I'd like to get people to focus more on them through activities such as bug reviews and bug days (more on that below). * Expand out testing Over the last two cycles we have made some significant advances in our testing practices. We need to expand on this cultural shift and even expand the focus on testing. The full test runtime needs to be cut by at least 50% to improve developer workflow. Near immediate test feedback is important for not breaking flow when writing code. This can be accomplished by refactoring test code to reduce unnecessary setup and focus on the code being tested. * Expand our community Being PTL isn't about me making all of the decisions or calling all of the shots. It's about facilitating the design and development of Keystone by working with the community. Through mentoring we can get more developers ready to be core to speed up our review pace. We need to work together to find ways to give more people the ability to contribute upstream. I do believe it's possible to make our thriving community even better. Thank you for voting for me as your PTL for the Mitaka release cycle. -- David Stanek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
I thoughts below mention Keystone, but in reality I would apply the same logic to any OpenStack service. On Fri, Sep 18, 2015 at 10:32 AM, Boris Bobrov wrote: > There are 2 dimensions this discussion should happen in: web server and > application server. Now we use apache2 as web server and mod_wsgi as app > server. > This is exactly true and Keystone should be deployable in an WSGI compliant application server. If it's not I would consider it a bug. > > I don't have a specific opinion on the app server (mod_wsgi vs uwsgi) and I > don't really care. > > Regarding apache2 vs nginx. I don't see any reasons for the switch. > Apache2 is > well known to deployers and sysadmins. It is very rich for modules. I > wonder > if there are customer-written modules. > Keystone doesn't use or require Apache. We recommend that it is deployed using Apache, but there is no requirement to do that if you don't need to use any Apache modules. For example, at least one of my devstack nodes happily runs Keystone's manage_all. [snip] > There are things in keystone that work under apache. They are not tested. > They > were written to work under apache because it's the simplest and the most > standard way to do. Making them work in nginx means forcing developers > write > some code. You're ready to do that? > This should only be true for optional features and currently require Apache modules. > > > May be someone does not need something that apache supports and nginx not > > and needs nginx features which apache does not support. Let's let our > users > > decide what they want. > > > > And the first step should be simple here - support for uwsgi. > > Why uwsgi? Why not gunicorn? Cherrypy? Twisted? > uwsgi and gunicorn should both work fine, as should any WSGI application server. CherryPy and Twister are more framework than application server, so I would not expect them to work. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [releases][requirements][keystone]something incompatible with our requirements
On Fri, Sep 18, 2015 at 3:32 PM, Robert Collins wrote: > I know this is terrible timing with the release and all, but > constraints updates are failing. This is the first evidence - and it > doesn't look like a race to me: > > http://logs.openstack.org/57/221157/10/check/gate-tempest-dsvm-full/18eb440/logs/devstacklog.txt.gz#_2015-09-18_13_51_46_902 > > https://review.openstack.org/#/c/221157/ is the updated review to > bring it all together. I'm worried that the incompatibility is going > to impact distributors and/or may even be from one of our own recent > library releases. > This looks like the issue I just ran into. The newest os-client-config depends on keystoneauth1 and this breaks openstackclient since it registers its plugins under the keystoneclient entrypoint and not the keystoneauth1 entry point. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On Fri, Sep 25, 2015 at 8:25 AM Adam Heczko wrote: > Are we discussing mod_wsgi and Keystone or OpenStack as a general? > If Keystone specific use case, then probably Apache provides broadest > choice of tested external authenticators. > I'm not against uwsgi at all, but to be honest expectation that nginx > could substitute Apache in terms of authentication providers is simply > unrealistic. > uwsgi isn't a replacement for Apache. It's a replacement for mod_wsgi. It just so happens that it does let you use user web servers if that's what your usecase dictates. As a Keystone developer I don't want to tell deployers that they have to use Apache. It should be their choice. Since Apache is the most common web server in our community I think we should continue to provide example configurations and guidance for it. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] service catalog: TNG
On Fri, Oct 9, 2015 at 1:28 PM, Jonathan D. Proulx wrote: > On Fri, Oct 09, 2015 at 01:01:20PM -0400, Shamail wrote: > :> On Oct 9, 2015, at 12:28 PM, Monty Taylor wrote: > :> > :>> On 10/09/2015 11:21 AM, Shamail wrote: > :>> > :>> > :>>> On Oct 9, 2015, at 10:39 AM, Sean Dague wrote: > :>>> > :>>> It looks like some great conversation got going on the service catalog > :>>> standardization spec / discussion at the last cross project meeting. > :>>> Sorry I wasn't there to participate. > :>> Apologize if this is a question that has already been address but why > can't we just leverage something like consul.io? > :> > :> It's a good question and there have actually been some discussions > about leveraging it on the backend. However, even if we did, we'd still > need keystone to provide the multi-tenancy view on the subject. consul > wasn't designed (quite correctly I think) to be a user-facing service for > 50k users. > :> > :> I think it would be an excellent backend. > :Thanks, that makes sense. I agree that it might be a good backend but > not the overall solution... I was bringing it up to ensure we consider > existing options (where possible) and spend cycles on the unsolved bits. > > As an operator I'd be happy to use SRV records to define endpoints, > though multiple regions could make that messy. > > would we make subdomins per region or include region name in the > service name? > > _compute-regionone._tcp.example.com >-vs- > _compute._tcp.regionone.example.com > > Also not all operators can controll their DNS to this level so it > couldn't be the only option. > > Or are you talking about using an internal DNS implementation private > to the OpenStack Deployment? I'm actually a bit less happy with that > idea. > I was able to put together an implementation[1] of DNS-SD loosely based on RFC-6763[2]. It'd really a proof of concept, but we've talked so much about it that I decided to get something working. Although if this seems like a viable option then there's still much work to be done. I'd love feedback. 1. https://gist.github.com/dstanek/093f851fdea8ebfd893d 2. https://tools.ietf.org/html/rfc6763 -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone] Let's get together and fix all the bugs
I would like to start running a recurring bug squashing day. The general idea is to get more focus on bugs and stability. You can find the details here: https://etherpad.openstack.org/p/keystone-office-hours -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] Midcycle planning
On Sat, May 30, 2015, 12:32 Adam Young wrote: I've started a Trello Board to manage the details. https://trello.com/b/SXrl6UQ5/midcycle-planning. It is world readable, but you need to be a member to be able to edit it. Let me know if you feel the need to be able to edit it, or to receive notifications. I'll be there. If you give me write access I'll add some more hotel details. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][security] Enable user password complexity verification
On Wed, Jun 3, 2015 at 6:04 AM liusheng wrote: > Thanks for this topic, also, I think it is similar situation when talking > about keystone users, not only the instances's password. > > In the past we've talked about having more advanced password management features in Keystone (complexity checks, rotation, etc). The end result is that we are not adding them because we would like to get away from managing users in Keystone that way. Instead we are pushing for users to integrate Keystone with more fully featured identity products. > > 在 2015/6/3 17:48, 郑振宇 写道: > > Hi All, > > The current OpenStack does not provide user password complexity > verification option. > > > When performing actions such as create instances, evacuate instances, > rebuild instances, rescue instances and update instances' admin password. > The complexity of user provided admin password has not been verified. This > can cause security problems. > > One solution will be adding a configuration option: > using_complex_admin_password = True, if this option is set in configure > file by administrator, then Nova will perform password complexity checks, > the check standards can be set to following the IT industry general > standard, if the provided admin password is not complex enough, an > exception will be throw. If this option is not set in configure file, then > the complexity check will be skipped. > > When the user dose not provide admin password, generate_password() in > utils.py is used to generate an admin password. Generate_password() now > uses two password symbol groups: default and easier, the default symbol > group contains numbers, upper case letters and small case letters. the > easier symbol group contains only numbers and upper case letters. The > generated password is not complex enough and can also cause security > problems. > > One possible solution is to add a new symbol group: > STRONGER_PASSWORD_SYMBOLS which contains numbers, upper case letters, lower > case letters and also special characters such as `~!@#$%^&*()-_=+ and > space. Then adding a new option in configuration file: > generate_strong_password = True, when this option is set, nova will > generate password using STRONGER_PASSWORD_SYMBOLS symbol group and with > longer password length. If this option is not set, the password will be > generated using the default symbol group and default length. > > AWS allows the selection of password policy to configure which kind of > password complexity is used in the cloud. Please see: > > http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_ManagingPasswordPolicies.html > > And about the standard of complexity, Microsoft also have an advise > about it, please see: > https://technet.microsoft.com/en-us/library/hh994562%28v=ws.10%29.aspx > > Thanks, > BR, > Zhenyu Zheng > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][reseller] New way to get a project scoped token by name
On Thu, Jun 4, 2015 at 10:10 AM Rodrigo Duarte wrote: > > Also, if we are going to use a delimiter, we need to update the way > projects names are returned in the GET v3/projects API to include the > hierarchy so the user (or client) knows how to request a token using the > project name. > This comment made me think of something (maybe crazy)... I'm hoping that we are not expecting a client to understand how to create the hierarchical representation of the fully qualified project name and instead feed it to them in the JSON responses. If this is the case we can probably control the delimiter on the server side without the client even knowing that there is a delimiter. We could have the first character of this new property actually contain the delimiter used. ".A.B.C" - would mean that "." is the delimiter - vs - "#A#B#C" - would mean that "#" is the delimiter The challenge left is to figure out which delimiter to use. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra] eventlet 0.18.1 not on PyPi anymore
On Wed, Feb 17, 2016 at 6:58 PM Sean Dague wrote: > > Question. Are we only tripping this up in unit tests because the tests > are doing things we'd never really do in real life? > I think that some of the issues have been real. Keystone had issues with 0.18.0 because it dropped methods from subprocess after patching it. They called it out in their release notes as an incompatible change. Version 0.18.2 broke the Keystone tests for Python 34. The poll() method seemed to be in some sort of deadlock and the tests timed out. This may have been a testing problem, but for whatever reason Python27 didn't have the issue. I want to say that we've seen other issues after 0.17.4, but I can't find definitive evidence. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [magnum] High Availability
On Fri, Mar 18, 2016 at 4:03 PM Douglas Mendizábal < douglas.mendiza...@rackspace.com> wrote: > [snip] > > > > Regarding the Keystone solution, I'd like to hear the Keystone team's > feadback on that. It definitely sounds to me like you're trying to put a > square peg in a round hole. > > > > I believe that using Keystone for this is a mistake. As mentioned in the blueprint, Keystone is not encrypting the data so magnum would be on the hook to do it. So that means that if security is a requirement you'd have to duplicate more than just code. magnum would start having a larger security burden. Since we have a system designed to securely store data I think that's the best place for data that needs to be secure. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] Proposing Marek Denis for the Keystone Core Team
+1 On Tue, Feb 10, 2015 at 12:51 PM, Morgan Fainberg wrote: > Hi everyone! > > I wanted to propose Marek Denis (marekd on IRC) as a new member of the > Keystone Core team. Marek has been instrumental in the implementation of > Federated Identity. His work on Keystone and first hand knowledge of the > issues with extremely large OpenStack deployments has been a significant > asset to the development team. Not only is Marek a strong developer working > on key features being introduced to Keystone but has continued to set a > high bar for any code being introduced / proposed against Keystone. I know > that the entire team really values Marek’s opinion on what is going in to > Keystone. > > Please respond with a +1 or -1 for adding Marek to the Keystone core team. > This poll will remain open until Feb 13. > > -- > Morgan Fainberg > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Output on stderr
On Wed, Mar 4, 2015 at 6:50 AM, Abhishek Talwar/HYD/TCS < abhishek.tal...@tcs.com> wrote: > While working on a bug for keystoneclient I have replaced sys.exit with > return. However, the code reviewers want that the output should be on > stderr(as sys.exit does). So how can we get the output on stderr. The print function allows you to specify a file: >>> from __future__ import print_function >>> import sys >>> print('something to', file=sys.stderr) The __future__ import is needed for Python 2.6/2.7 because print was changed to a function in Python 3. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone]ON DELETE RESTRICT VS ON DELETE CASCADE
On Sun, Mar 8, 2015 at 1:37 PM, Mike Bayer wrote: > can you elaborate on your reasoning that FK constraints should be used less > overall? or do you just mean that the client side should be mirroring the > same > rules that would be enforced by the FKs? > I don't think he means that we will use them less. Our SQL backends are full of them. What Keystone can't do is rely on them because not all implementations of our backends support FKs. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone]ON DELETE RESTRICT VS ON DELETE CASCADE
On Sun, Mar 8, 2015 at 10:28 PM, Chen, Wei D wrote: > +1, > > > > I am fan of checking the constraints in the controller level instead of > relying on FK constraints itself, thanks. > The Keystone controllers shouldn't do any business logic. This should be in the managers. The controllers should do nothing more that take web stuff and convert it for use by the managers. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][fernet] Fernet tokens sync
On Fri, Mar 27, 2015 at 10:14 AM, Boris Bobrov wrote: > As you know, keystone introduced non-persistent tokens in kilo -- Fernet > tokens. These tokens use Fernet keys, that are rotated from time to time. A > great description of key rotation and replication can be found on [0] and > [1] > (thanks, lbragstad). In HA setup there are multiple nodes with Keystone and > that requires key replication. How do we do that with new Fernet tokens? > > Please keep in mind that the solution should be HA -- there should not be > any > "master" server, pushing keys to slave servers, because master server > might go > down. > In my test environment I was using ansible to sync the keys across multiple nodes. Keystone should probably provide some guidance around this process, but I don't think it should deal with the actual syncing. I think that's better left to an installation's existing configuration management tools. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] SQLite support (migrations, work-arounds, and more), is it worth it?
Exactly. This is the direction I have been going. Functional tests are written using the public APIs using the client. I would also add that I don't like that the Keystone unit tests are so database heavy. I would not want MySQL or ant RDBMS to be a requirement for running the tests. On Mon, Apr 6, 2015, 12:42 Morgan Fainberg wrote: > > > > On Apr 6, 2015, at 09:20, Mike Bayer wrote: > > > > > > > >> On 4/6/15 12:06 PM, Clint Byrum wrote: > >> Excerpts from Boris Bobrov's message of 2015-04-03 18:29:08 -0700: > On Saturday 04 April 2015 03:55:59 Morgan Fainberg wrote: > I am looking forward to the Liberty cycle and seeing the special > casing we > do for SQLite in our migrations (and elsewhere). My inclination is > that we > should (similar to the deprecation of eventlet) deprecate support for > SQLite in Keystone. In Liberty we will have a full functional test > suite > that can (and will) be used to validate everything against much more > real > environments instead of in-process “eventlet-like” > test-keystone-services; > the “Restful test cases” will no longer be part of the standard unit > tests > (as they are functional testing). With this change I’m inclined to say > SQLite (being the non-production usable DB) what it is we should look > at > dropping migration support for SQLite and the custom work-arounds. > > Most deployers and developers (as far as I know) use devstack and > MySQL or > Postgres to really suss out DB interactions. > > I am looking for feedback from the community on the general stance for > SQLite, and more specifically the benefit (if any) of supporting it in > Keystone. > >>> +1. Drop it and clean up tons of code used for support of sqlite only. > >>> > >>> Doing tests with mysql is as easy, as with sqlite ("mysqladmin drop -f; > >>> mysqladmin create" for "reset"), and using it by default will finally > make > >>> people test their code on real rdbmses. > >> Please please please be careful with that and make sure the database > >> name is _always_ random in tests... or even better, write a fixture to > >> spin up a mysqld inside a private tempdir. That would be a really cool > >> thing for oslo.db to provide actually. > >> > >> I'm just thinking some poor sap runs the test suite with the wrong > >> .my.cnf in the wrong place and there went keystone's db. > > The standard approach here is that tests based on the oslo.db approach > by default connect using a username openstack_citest on localhost, which is > then used to create databases of random names. If you base your database > tests on oslo.db, you get that right now. This username/host/etc is also > configurable through environment variables. I don't see how my.cnf is > involved in this nor how it would impact someone's keystone database, > unless we're talking about tests that happen to load down and/or crash the > whole database server. > > > > spinning up a whole mysqld seems really heavy-handed and unnecessary. > Not to mention the tests run on other backends as well such as Postgresql. > > > > The reasons outlined by both Clint and Mike are why we won't be attempting > this until we are happy with our functional test suite. But once we are > happy dropping SQLite is on the table. The way I see it the functional > tests should be performed against a real keystone server, even if it is one > spun up for testing specifically. > > Per test db creation / other such stuff will be part of that discussion. > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] SQLite support (migrations, work-arounds, and more), is it worth it?
On Mon, Apr 6, 2015 at 2:41 PM, Mike Bayer wrote: > > > On 4/6/15 12:49 PM, David Stanek wrote: > > Exactly. This is the direction I have been going. Functional tests are > written using the public APIs using the client. > > I would also add that I don't like that the Keystone unit tests are so > database heavy. I would not want MySQL or ant RDBMS to be a requirement for > running the tests. > > is that because you'd prefer that the unit tests were more isolated, or > just that an external service is being used? > There are two types of testing that I'm interested in really promoting in Keystone. I want to see functional tests that only use the published APIs to trigger and observe behavior. I also want to see unit tests that are small, fast and that test the business logic of the system. I started working[1] on functional testing in the Kilo cycle. The spec[2] talks about standing up Keystone in different configurations (MySQL, LDAP, Federated, etc.) and then running a set of tests against that instance. This will allow us the test any backend we want and confirm it works the way we expect Keystone to work. On the other hand I want the unit tests to be smaller and faster. The job of the unit tests should be to test small isolated bits of code and help guide/evaluate the code design. Except to test the SQL backends I don't see a need for using a database and even then I'd like to get by without it. > I've done some work with extensive mocking of SQL databases; specifically > mocking at the ORM level. It is nice when you get it to run, but it's also > a much bigger job to write fine-grained mocks like that, the mocks can be > brittle in relation to the code they're targeting, and you also need to > come up with some solution to actually functional test your database access > code. > > I tend to think that having a mysqld service running is the lesser of two > evils and you get a lot more code coverage by going all the way out to the > DB. > The simplicity of the Keystone design is actually very good for dealing with this (controller -> manager -> backend). Since backends are where the actually query work happens we can easily provide a fake for each subsystem that we want to test. This would allow us the test the business rules in the manager without a database. The reason that is good is that we'd probably be better off requiring LDAP (if anything) for some of the subsystems since that is what they are typically configured to use. I would rather just leave that level of detail to the functional tests. 1. https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:functonal-testing,n,z 2. https://review.openstack.org/#/c/153300/l -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][code quality] Voting coverage job (-1 if coverage get worse after patch)
On Sat, Apr 18, 2015 at 9:30 PM, Boris Pavlovic wrote: > Code coverage is one of the very important metric of overall code quality > especially in case of Python. It's quite important to ensure that code is > covered fully with well written unit tests. > > One of the nice thing is coverage job. > I really like the idea of adding the coverage job everywhere so that developers can view the results be using a link in Gerrit. I think this would make it easier for many. I don't like the idea of checking that coverage is increased. There are many issues with that. The two biggest one for me are: 1. It will either lead to people doing things to game the system or overuse of the #no-coverage-check tag you mentioned. 2. It really doesn't tell you too much. A core developer should really be looking at the tested use cases to see if they are all there. Line coverage and even branch coverage won't tell you that. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone] Adding foreign keys between subsystems
[tl;dr I want to remove the artificial restriction of not allowing FKs between subsystems and I want to stop FK enforcement in code.] The keystone code architecture is pretty simple. The data and functionality are divided up into subsystems. Each subsystem can be configured to use a different backend datastore. Of course, there are always exceptions to the rule like how the federation and identity subsystems are highly coupled in the data model. On the surface this flexible model sounds good, but there are some interesting consequences. First, you can't tell from looking at the data model that there actually is a lot of coupling between the subsystems. So instead of looking at our sqlalchemy models to see relationships, we must look throughout the code for where a particular primary key is used and look for enforcement. (Hopefully we enforce it in all of the right places.) Additionally, a DBA/data architect/ whenever can't see the relationship at all by looking at the database. Second, this has polluted our code and causes erroneous API errors. We have added lots of get_*() calls in our code that checks for the existence of IDs in another subsystem. In some cases we probably don't do the check and thus would allow bad data to be stored. The check often causes 404s instead of 400s when bad data is provided. So I'd like us to be more deliberate in defining which parts of the data model are truly independent and a separate backend datastore would make sense. For instance, we know we want to support LDAP for identity (although this still puts identity info into a SQL database) and catalog is very isolated from the rest of the data model. As a side effect this means that if deployers wished to use a separate backend for something they would need to also implement it for the other highly coupled subsystems. They would also have to provide any FK enforcement that their own datastore does not provide. Thoughts? -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Adding foreign keys between subsystems
On 12-Apr 14:30, Rodrigo Duarte wrote: > Just to illustrate the discussion, we have a bug fix that currently tries > to drop a FK between the federation and identity subsystems [1]. > > [1] https://review.openstack.org/#/c/445505/ I think this highlights one of my problems with the current architecture. I see that you've removed the FK and added delete logic to do what the data layer would be doing for you. I didn't see any added get_user() checks to make sure the user_id being used in creates/updates is valid. Are we already checking that somewhere else or is this introducing a new bug? -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Adding foreign keys between subsystems
On 12-Apr 15:25, Rodrigo Duarte wrote: > On Wed, Apr 12, 2017 at 2:47 PM, David Stanek wrote: > > > On 12-Apr 14:30, Rodrigo Duarte wrote: > > > Just to illustrate the discussion, we have a bug fix that currently tries > > > to drop a FK between the federation and identity subsystems [1]. > > > > > > [1] https://review.openstack.org/#/c/445505/ > > > > I think this highlights one of my problems with the current architecture. > > I see that > > you've removed the FK and added delete logic to do what the data layer > > would be doing > > for you. I didn't see any added get_user() checks to make sure the user_id > > being used > > in creates/updates is valid. Are we already checking that somewhere else > > or is this > > introducing a new bug? > > > > The review [1] is dropping the idp_id and idp_id + protocol_id FKs, not the > user_id one. > Right, misremembered. Just run s/user_id/idp_id/ and s/get_user/get_idp/ and you'll have the same issues. -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] Multi-factor Auth with Keystone and TOTP
On Mon, Jul 18, 2016 at 9:13 AM, Adrian Turjak wrote: > We need an MFA solution, and this doesn't seem like too terrible an option. One thing to note here is that the credentials for TOTP stored in the keystone credentials backend are not encrypted. So a breach of your database could expose those to an attacker. This is a review[1] to fix this issue that is close to merging. 1. https://review.openstack.org/#/c/317169/ -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects
On Thu, Aug 25 at 13:13 -0400, Steve Martinelli wrote: > The keystone team is pursuing a trigger-based approach to support rolling, > zero-downtime upgrades. The proposed operator experience is documented here: > > http://docs.openstack.org/developer/keystone/upgrading.html > I wanted to mention a few things. One of the reasons I suggested this approach for keystone is that I've had success in the past using a combination of triggers and code to do live, online migrations. Many times using completely different schemas. In keystone we are just talking about some simple data transformations between columns and things like that. The triggers themselves shouldn't get too complicated. If there are cases where triggers won't work, then we won't force them. (A current example of this is encrypting credentials.) The online migrations are not required. Operators can still go the old route and db_sync while others help test out the cutting edge features. The triggers are not there during the entire lifecycle of the application. The expand phase adds them and the contract removes them. -- David Stanek web: http://dstanek.com blog: http://traceback.org __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects
On Wed, Aug 31 at 17:18 -0500, Monty Taylor wrote: > > Nova and Neutron have an approach for this. It may or may not be ideal - > but it exists right now. While it can be satisfying to discount the > existing approach and write a new one, I do not believe that is in the > best interests of OpenStack as a whole. To diverge in _keystone_ - which > is one of the few projects that must exist in every OpenStack install - > when there exists an approach in the two other most commonly deployed > projects - is such a terrible example of the problems inherent in > Conway's Law that it makes me want to push up a proposal to dissolve all > of the individual project teams and merge all of the repos into a single > repo. That's a bit overly dramatic. I think having some innovation is a good thing. Specifically in this case where our needs appear to be a little simpler than those of nova. > > Make the oslo libraries Nova and Neutron are using better. Work with the > Nova and Neutron teams on a consolidated approach. We need to be driving > more towards an OpenStack that behaves as if it wasn't written by > warring factions of developers who barely communicate. I believe we tried to keep with the same extract/migrate/contract patterns. Sure our implementation differs, but I don't see operators caring about that as long as it works. > > Even if the idea was one I thought was good technically, the above would > still trump that. Work with Nova and Neutron. Be more similar. > > PLEASE > > BUT - I also don't think it's a good technical solution. That isn't > because triggers don't work in MySQL (they do) - but because we've spent > the last six years explicitly NOT writing raw SQL. We've chosen an > abstraction layer (SQLAlchemy) which does its job well. > > IF this were going to be accompanied by a corresponding shift in > approach to not support any backends by MySQL and to start writing our > database interactions directly in SQL in ALL of our projects - I could > MAYBE be convinced. Even then I think doing it in triggers is the wrong > place to put logic. > > "Database triggers are obviously a new challenge for developers to > write, honestly challenging to debug (being side effects), and are made > even more difficult by having to hand write triggers for MySQL, > PostgreSQL, and SQLite independently (SQLAlchemy offers no assistance in > this case)" > > If you look at: > > https://review.openstack.org/#/c/355618/40/keystone/common/sql/expand_repo/versions/002_add_key_hash_and_encrypted_blob_to_credential.py > > You will see the three different SQL dialects this. Not only that, but > some of the more esoteric corners of those backends. We can barely get > _indexes_ right in our database layers ... now we think we're going to > get triggers right? Consistently? And handle things like Galera? > > The other option is app level, which is what nova and neutron are doing. > It's a good option, because it puts the logic in python, which is a > thing we have 2500 developers fairly well versed in. It's also scalable, > as the things executing whatever the logic is are themselves a scale-out > set of servers. Finally, it's a known and accepted pattern in large > scale MySQL shops ... Roll out a new version of the app code which > understands both the old and the new schema version, then roll out a > no-downtime additive schema change to the database, then have the app > layer process and handle on the fly transformation if needed. I've done both types of migrations in the past, but with one imporant exception. We could roll out our application on Tuesday and then the cleanup on Thursday. We didn't carry baggage for 6 months to a year. My fear with keystone is that we'd slow development even more by adding more cruft and cruft on top of cuft. > > SO ... > > Just do what Nova and Neutron are doing - and if it's not good enough, > fix it. Having some projects use triggers and other projects not use > triggers is one of the more epically crazypants things I've heard around > here ... and I lived through the twisted/eventlet argument. -- David Stanek web: http://dstanek.com blog: http://traceback.org __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] new core reviewer (rderose)
On Thu, Sep 01 at 10:44 -0400, Steve Martinelli wrote: > > Thanks for all your hard work Ron, we sincerely appreciate it. > Contrats! Well deserved for sure! -- David Stanek web: http://dstanek.com blog: http://traceback.org __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Custom ProjectID upon creation
On 05-Dec 15:14, Lance Bragstad wrote: > I put myself in Boris' camp on this one. This can open up the opportunity > for negative user-experience, purely based on where I authenticate and > which token I happen to authenticate with. A token would no longer be > something I can assume to be properly validated against any node in my > deployment. Now, when I receive a 401 Unauthorized, is it because the token > is actually invalid, did I use the wrong endpoint, or did I use a token > with the wrong scope for the endpoint I wanted to interact with? > I agree. I think having different behavior for tokens based on scope will not only lead to bad user experiences, but will lead to baking in those rules into the client. Someone will propose this as soon as they get confused by the token 401ing unexpectedly. -- david stanek web: http://www.dstanek.com blog: http://www.traceback.org __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] webob 1.7
On 18-Jan 08:59, Chuck Short wrote: > Hi > > We have been expericing problems with newer versions of webob (webob 1.7). > Reading the changelog, it seems that the upstream developers have > introduced some backwards incompatibility with previous versions of webob > that seems to be hitting keystone and possibly other projects as well > (nova/glance in particular). For keystone this bug has been reported in bug > #1657452. I would just like to get more developer's eyes on this particular > issue and possibly get a fix. I suspect its starting to hit other distros > as well or already have hit. > I've confirmed that this is an issue. I'll work on a fix. We can take further discussion to the bug tracker. -- david stanek web: https://www.dstanek.com twitter: https://twitter.com/dstanek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone]PKI token VS Fernet token
On 15-Feb 18:16, 王玺源 wrote: > Hello everyone, > PKI/PKIZ token has been removed from keystone in Ocata. But recently our > production team did some test about PKI and Fernet token (With Keystone > Mitaka). They found that in large-scale production environment, Fernet > token's performance is not as good as PKI. Here is the test data: > > https://docs.google.com/document/d/12cL9bq9EARjZw9IS3YxVmYsGfdauM25NzZcdzPE0fvY/edit?usp=sharing This is nice to see. Thanks. > > From the data, we can see that: > 1. In large-scale concurrency test, PKI is much faster than Fernet. > 2. PKI token revoke can't immediately make the token invalid. So it has the > revoke issue. https://wiki.openstack.org/wiki/OSSN/OSSN-0062 > > But in our production team's opinion, the revoke issue is a small problem, > and can be avoided by some periphery ways. (More detail solution could be > explained by them in the follow email). > They think that the performance issue is the most important thing. Maybe > you can see that in some production environment, performance is the first > thing to be considered. I'd like to hear solutions to this if you have already come up with them. This issue, however, isn't the only one that led us to remove PKI tokens. > > So here I'd like to ask you, especially the keystone experts: > 1. Is there any chance to bring PKI/PKIZ back to Keystone? I would guess that, at least in the immediate future, we would not want to put it back into keystone until someone can fix the issues. Also ideally running the token provider in production. > 2. Has Fernet token improved the performance during these releases? Or any > road map so that we can make sure Fernet is better than PKI in all side. > Otherwise, I don't think that remove PKI in Ocata is the right way. Or > even, we can keep the PKI token in Keystone for more one or two cycles, > then remove it once Fernet is stable enough. > 3. Since I'll be in Atalanta next week, if it is possible, I'd like to > bring this topic to Keystone PTG. can I? Sure. We have a pretty packed calendar, but I'm sure you could steal a few minutes somewhere. > > It is a real production problem and I really need your feedback. > Have you tried playing with the crypt_strength[1]? If the slowness is the crypto (which it was in the past) then you can tune it a little bit. Another option might be to keep the same token flow and find a faster method for hashing a token. 1. http://git.openstack.org/cgit/openstack/keystone/tree/etc/keystone.conf.sample#n67 -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Signing off
On 30-May-2018 08:45, Henry Nash wrote: > Hi > > It is with a somewhat heavy heart that I have decided that it is time to hang > up my keystone core status. Having been involved since the closing stages of > Folsom, I've had a good run! When I look at how far keystone has come since > the > v2 days, it is remarkable - and we should all feel a sense of pride in that. > > Thanks to all the hard work, commitment, humour and support from all the > keystone folks over the years - I am sure we will continue to interact and > meet > among the many other open source projects that many of us are becoming > involved > with. Ad astra! > Hey Henry! It's good to hear from you! You were always fun to work with and I got a lot out of our chats about crazy, enterprisey things. I guess the world with have to wait for another episode of "Stanek & Nash". -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev