Re: [openstack-dev] [nova] Expand resource name allowed characters
Hi Chris, Thanks for bring it up here, -Original Message- From: Chris St. Pierre [mailto:stpie...@metacloud.com] Sent: Saturday, September 13, 2014 2:53 AM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [nova] Expand resource name allowed characters We have proposed that the allowed characters for all resource names in Nova (flavors, aggregates, etc.) be expanded to all printable unicode characters and horizontal spaces: https://review.openstack.org/#/c/119741 Currently, the only allowed characters in most resource names are alphanumeric, space, and [.-_]. We have proposed this change for two principal reasons: 1. We have customers who have migrated data forward since Essex, when no restrictions were in place, and thus have characters in resource names that are disallowed in the current version of OpenStack. This is only likely to be useful to people migrating from Essex or earlier, since the current restrictions were added in Folsom. 2. It's pretty much always a bad idea to add unnecessary restrictions without a good reason. While we don't have an immediate need to use, for example, the ever-useful http://codepoints.net/U+1F4A9 in a flavor name, it's hard to come up with a reason people *shouldn't* be allowed to use it. That said, apparently people have had a need to not be allowed to use some characters, but it's not clear why: https://bugs.launchpad.net/nova/+bug/977187 So I guess if anyone knows any reason why these printable characters should not be joined in holy resource naming, speak now or forever hold your peace. I also could not find the reason of current restriction on the bug report, and I'd like to know it as the history. On v2 API(not v2.1), each resource name contains the following restriction for its name: Resource | Length | Pattern ---+-+-- aggregate | 1-255 | nothing backup| nothing | nothing flavor| 1-255 | '^[a-zA-Z0-9. _-]*[a-zA-Z0-9_-]+ | | [a-zA-Z0-9. _-]*$' keypair | 1-255 | '^[a-zA-Z0-9 _-]+$' server| 1-255 | nothing cell | 1-255 | don't contain . and ! On v2.1 API, we have applied the same restriction rule[1] for whole resource names for API consistency, so maybe we need to consider this topic for whole names. [1]: https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L44 Thanks Ken'ichi Ohmichi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Contribution work flow
Hi, I am about to submit my first patch. I saw the contributions guidelines in the documentations. Just to make it clear, is it that I issue a pull request in GitHub, which automatically pushes my patch to gerrit? Also, I found something called change-Id in the commit message. Is it the hash code for the git commit? If yes, should we prefix a 'I' in the beginning of hash code? Thanks, Sharan Kumar M ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Expand resource name allowed characters
On 09/13/2014 02:28 AM, Kenichi Oomichi wrote: Hi Chris, Thanks for bring it up here, -Original Message- From: Chris St. Pierre [mailto:stpie...@metacloud.com] Sent: Saturday, September 13, 2014 2:53 AM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [nova] Expand resource name allowed characters We have proposed that the allowed characters for all resource names in Nova (flavors, aggregates, etc.) be expanded to all printable unicode characters and horizontal spaces: https://review.openstack.org/#/c/119741 Currently, the only allowed characters in most resource names are alphanumeric, space, and [.-_]. We have proposed this change for two principal reasons: 1. We have customers who have migrated data forward since Essex, when no restrictions were in place, and thus have characters in resource names that are disallowed in the current version of OpenStack. This is only likely to be useful to people migrating from Essex or earlier, since the current restrictions were added in Folsom. 2. It's pretty much always a bad idea to add unnecessary restrictions without a good reason. While we don't have an immediate need to use, for example, the ever-useful http://codepoints.net/U+1F4A9 in a flavor name, it's hard to come up with a reason people *shouldn't* be allowed to use it. That said, apparently people have had a need to not be allowed to use some characters, but it's not clear why: https://bugs.launchpad.net/nova/+bug/977187 So I guess if anyone knows any reason why these printable characters should not be joined in holy resource naming, speak now or forever hold your peace. I also could not find the reason of current restriction on the bug report, and I'd like to know it as the history. On v2 API(not v2.1), each resource name contains the following restriction for its name: Resource | Length | Pattern ---+-+-- aggregate | 1-255 | nothing backup| nothing | nothing flavor| 1-255 | '^[a-zA-Z0-9. _-]*[a-zA-Z0-9_-]+ | | [a-zA-Z0-9. _-]*$' keypair | 1-255 | '^[a-zA-Z0-9 _-]+$' server| 1-255 | nothing cell | 1-255 | don't contain . and ! On v2.1 API, we have applied the same restriction rule[1] for whole resource names for API consistency, so maybe we need to consider this topic for whole names. [1]: https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L44 Honestly, I bet this had to do with how the database used to be set up. That being said, i'm pro letting names be 'utf8'. The id fields that are strings (like flavor_id) I think we should keep constrained, as we do actually do joins on them. (And as jay said, the current utf8 schema is actually highly inefficient and bloaty.) -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini
On 09/12/2014 07:14 PM, Doug Hellmann wrote: On Sep 12, 2014, at 12:03 PM, Sean Dague s...@dague.net wrote: On 09/12/2014 11:52 AM, Doug Hellmann wrote: On Sep 12, 2014, at 11:21 AM, Mike Bayer mba...@redhat.com wrote: On Sep 12, 2014, at 7:39 AM, Sean Dague s...@dague.net wrote: I assume you, gentle OpenStack developers, often find yourself in a hair tearing out moment of frustration about why local unit tests are doing completely insane things. The code that it is stack tracing on is no where to be found, and yet it fails. And then you realize that part of oslo doesn't exist any more except there are still pyc files laying around. Gah! I've proposed the following to Nova and Python novaclient - https://review.openstack.org/#/c/121044/ Which sets PYTHONDONTWRITEBYTECODE=true in the unit tests. my VPN was down and I didn’t get this thread just now, but I am strongly -1 on this as added to tox.ini, my response is http://lists.openstack.org/pipermail/openstack-dev/2014-September/045873.html. Short answer: if you want this feature, put PYTHONDONTWRITEBYTECODE into *your* environment. Don’t force it on our automated tests or on my environment. .pyc files make a difference in behavior, and if we banish them from all testing, then our code is never tested within the environment that it will normally be run in after shipment. I’d far prefer a simple script added to tox.ini which deletes orphaned .pyc files only, if a change to tox.ini must be made. I have to agree with Mike here. Cleaning up our dev environments using a little automation is better than disabling a feature of the interpreter that may have unforeseen consequences in behavior or performance. The more we introduce unusual settings like this into our environments and tools, the more edge cases and weirdness we’re going to find in those tools that keep us from doing the work we really want to be doing. We could use a git hook (see my earlier message in this thread) or we could add a command to tox to remove them before starting the tests. Neither of those solutions would affect the runtime behavior in a way that makes our dev environments fundamentally different from a devstack or production deployment. You believe that unit tests are going to change in the way they run so dramatically with this change that it invalidates their use? Do we have examples of what changes if you do and don't have pyc files there? Remember, we're not changing integration testing with this. This is solely unit testing. The reason I don't like just fix it in your local env is you are then exporting the complexity to developers. For something that they should really not have to get bitten by... a lot. Adding a command to tox to remove the files would be less intrusive than disabling their creation. We have had bad experiences mixing features to produce unusual dev environments that are different from the way the software really runs. All of the issues we had with namespace packages were caused by a bug in that implementation exposed because we were doing something unusual in devstack, for example. Adding some variation of “find $(python setup.py --name) --name ‘*.pyc’ | xargs rm -f” to tox.ini before running testr solves the problem you have identified without introducing any side-effects. This sounds saner to me so, I'd definitely go for this one. (FWIW, I'm use to clean up `pyc` files by myself) Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Contribution work flow
2014-09-13 7:09 GMT-03:00 Sharan Kumar M sharan.monikan...@gmail.com: Hi, I am about to submit my first patch. I saw the contributions guidelines in the documentations. Just to make it clear, is it that I issue a pull request in GitHub, which automatically pushes my patch to gerrit? Also, I found something called change-Id in the commit message. Is it the hash code for the git commit? If yes, should we prefix a 'I' in the beginning of hash code? Thanks, Sharan Kumar M ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Hi Sharan, In order to submit your contributions you have to configure your git repo as stated in [0]. We don't directly issue pull requests, we use git review instead. If you didn't install it yet, install it with 'pip install git-review', and then simply 'git review' your change. So, the workflow is: - Code, code, code - Make sure you have your changes in a branch called bug/bug-no or bp/bp-id. If you started working in master, simply do a 'git checkout -b bug/bug-no or bp/bp-id' - Commit the change - Write the commit message [1] - And finally 'git review' The change-id is auto generated, you don't have to add it by hand. You can check more details about the workflow in [2]. [0] https://wiki.openstack.org/wiki/Gerrit_Workflow#Project_Setup [1] https://wiki.openstack.org/wiki/GitCommitMessages [2] https://wiki.openstack.org/wiki/Gerrit_Workflow#Normal_Workflow Hope it's clear, and if not, feel free to reach me on IRC at irc.freenode.org at #openstack-101. My IRC handle is vkmc. My best, ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Contribution work flow
On 2014-09-13 15:39:30 +0530 (+0530), Sharan Kumar M wrote: I am about to submit my first patch. Welcome to our contributor community! I'm going to apologize in advance for the convoluted way in which our projects accept code contributions, but it's streamlined for high-volume contributors (which it's very efficient at enabling) but can be a bit of an initial hassle for first-time or casual contributors. I saw the contributions guidelines in the documentations. It doesn't sound like you did. Please *carefully* follow the instructions here: https://wiki.openstack.org/wiki/How_To_Contribute#If_you.27re_a_developer There is a legal agreement you'll need to confirm if you want to contribute patches to most of our official projects, which as a prerequisite requires that you first join the OpenStack Foundation as a (free) member. The process is basically instantaneous, but if you skip steps or do them out of order you're going to end up confused very quickly. Just to make it clear, is it that I issue a pull request in GitHub, which automatically pushes my patch to gerrit? We do not use Github to run the OpenStack Project. We are a free project, and Github is not free software. Also, I found something called change-Id in the commit message. Is it the hash code for the git commit? If yes, should we prefix a 'I' in the beginning of hash code? This is an implementation detail of our code review system, which tools should create for you (it's normally generated by a Git hook which is installed in your local repository configuration the first time you run 'git review' and then gets inserted into commit messages for you on each subsequent commit message edit in that repository). Please do not manually construct the Change-Id for a commit. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Octavia] Which threading library?
On Sep 12, 2014, at 5:53 PM, Eichberger, German german.eichber...@hp.com wrote: Hi, I think the “standard” threading library for OpenStack is eventlet – however, it seems that Oslo is spearheading efforts to get to a more compatible one (see http://techs.enovance.com/6562/asyncio-openstack-python3) I am now wondering since we are starting fresh if we should embrace (a potential) future or stick with eventlet and all its flaws? the so-called “flaws” of implicit async are up for debate.A “flaw” of asyncio is that it requires a full rewrite of code that uses it, as well as all libraries which it consumes that also happen to use IO.This includes the particularly salient topic that none of the Python database APIs that exist, other than psycopg2, have any support for true non-blocking code at the IO level, without using monkeypatching. the standard Python DBAPI (pep 249) has no support for explicit async, so any APIs that do so are ad-hoc (see http://initd.org/psycopg/docs/advanced.html#async-support for psycopg2’s very well done, but entirely non-standard API in this regard). Once you’re on explicit async, simple imperative code that happens to make calls which imply IO is no longer possible. This is kind of a blocker to end-to-end integration of asyncio in all of Openstack, rather than just making use of it in those areas where it is already directly applicable. This has been discussed in depth at lists.openstack.org/pipermail/openstack-dev/2014-July/039291.html. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] What's holding nova development back?
Just an observation from the last week or so... The biggest problem nova faces at the moment isn't code review latency. Our biggest problem is failing to fix our bugs so that the gate is reliable. The number of rechecks we've done in the last week to try and land code is truly startling. I know that some people are focused by their employers on feature work, but those features aren't going to land in a world in which we have to hand walk everything through the gate. Michael -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Contribution work flow
Shar, Hi! 1) install git-review and set it up (poke around openstack docs) 2) after crafting your patch in a new branch (git branch name-of-branch-you-are-working-on), commit the changes (git add -a), craft a commit message, save it, and then type git review If everything is correct, it will submit to openstack's ci machine (review.openstack.org) and you can track CI testing + reviews and such. Also - cool to see you working on OpenStack! Cheers, Patrick On 09/13/2014 03:09 AM, Sharan Kumar M wrote: Hi, I am about to submit my first patch. I saw the contributions guidelines in the documentations. Just to make it clear, is it that I issue a pull request in GitHub, which automatically pushes my patch to gerrit? Also, I found something called change-Id in the commit message. Is it the hash code for the git commit? If yes, should we prefix a 'I' in the beginning of hash code? Thanks, Sharan Kumar M ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] What's holding nova development back?
Michael, I am so glad that you started this topic. I really like idea of of taking a pause with features and concentrating on improvement of current code base. Even if the 1 k open bugs https://bugs.launchpad.net/nova are vital issue, there are other things that could be addressed to improve Nova team throughput. Like it was said in another thread: Nova code is current too big and complex to be understand by one person. It produces 2 issues: A) There is hard to find person who can observer full project and make global architecture decisions including work on cross projects interactions (So project doesn't have straight direction of development) B) It's really hard to find cores, and current cores are under too heavy load (because of project complexity) I believe that whole current Nova functionality can be implemented in much simpler manner. Basically, complexity was added during the process of adding a lot of features for years, that didn't perfectly fit to architecture of Nova. And there wasn't much work on refactoring the architecture to cleanup these features. So maybe it's proper time to think about what, why and how we are doing. That will allows us to find simpler solutions for current functionality. Best regards, Boris Pavlovic On Sun, Sep 14, 2014 at 1:07 AM, Michael Still mi...@stillhq.com wrote: Just an observation from the last week or so... The biggest problem nova faces at the moment isn't code review latency. Our biggest problem is failing to fix our bugs so that the gate is reliable. The number of rechecks we've done in the last week to try and land code is truly startling. I know that some people are focused by their employers on feature work, but those features aren't going to land in a world in which we have to hand walk everything through the gate. Michael -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] oslo.vmware 0.6.0 released
The Oslo team is pleased to release version 0.6.0 of oslo.vmware. This release includes: $ git log --oneline --no-merges 0.5.0..0.6.0 f36cd7f Add DuplicateName exception 81ef9d4 Add 'details' property to VMwareDriverException 5571e9f Enable oslo.i18n for oslo.vmware 6c24953 Add API to enable calling module to register an exception b72ab3e Imported Translations from Transifex ffd9a6d Add docs target and generate api docs 4938dff Updated from global requirements e2f0469 Work toward Python 3.4 support and testing 9c6a20e warn against sorting requirements 6c5e449 Add exception for TaskInProgress d51fdbe Updated from global requirements 9273388 Refactoring to reduce noise in log files c4437af Imported Translations from Transifex a3f8146 Add missing session parameter to get_summary 74832f4 Updated from global requirements d9ada2a Switch off caching to prevent cache poisoning by local attacker abb4e82 Support for copying streamOptimized disk to file e434d1b Add support for the DatastoreURL object e5c22fa Add methods to the Datastore objects d33c195 Imported Translations from Transifex 788e944 Add Pylint testenv environment 6316a6f Port the Datastore and DatastorePath objects d76620b Log additional details of suds faults 9a9649f Fix seek and tell in BlockingQueue Please report issues to the bug tracker: https://bugs.launchpad.net/oslo.vmware -- dims -- Davanum Srinivas :: https://twitter.com/dims ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
