Re: [openstack-dev] [neutron] Engine facade

2017-04-02 Thread Gary Kotton 
Yes, sorry my bad for not adding it - 
http://logs.openstack.org/39/452539/2/check/gate-vmware-nsx-python27-ubuntu-xenial/14c019c/testr_results.html.gz
Please see the test test_create_port_dns_name
Thanks
Gary

From: Kevin Benton 
Reply-To: OpenStack List 
Date: Monday, April 3, 2017 at 12:56 AM
To: OpenStack List 
Subject: Re: [openstack-dev] [neutron] Engine facade

Do you have a link to a traceback?

On Apr 2, 2017 09:25, "Gary Kotton" 
> wrote:
Hi,
The change https://review.openstack.org/#/c/402750/ has broken the vmware-nsx 
plugin. I am not sure if this has had effect on any other decomposed plugins.
One of the issues that we have is when we create a PortDNS object under a 
transaction we get an exception: DBReferenceError: (sqlite3.IntegrityError) 
FOREIGN KEY constraint failed [SQL: u'INSERT INTO portdnses (port_id, 
current_dns_name, current_dns_domain, previous_dns_name, previous_dns_domain, 
dns_name) VALUES (?, ?, ?, ?, ?, ?)'] [parameters: 
('2f2039ac-e7e6-4cc3-a8a0-3298089d4afb', u'', u'', u'', u'', u'port-dns-name')]
Any ideas?
Thanks
Gary

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Removing BDM devices from POST requests

2017-04-02 Thread Matt Riedemann 

On 4/2/2017 8:08 PM, Feodor Tersin wrote:

Can you give some more details? How does this actually "merge" or



replace BDMs defined in the image metadata? Is this because we use
device name as part of a hack for a primary key in the database [1]? I
assume it is. The lack of unique constraint in the BDM data model has
also always been a source of bugs, especially with cells v1 and why we
talk about adding a uuid column to that table every few months.


Right. This is not pure merging, but replacing, based on device names as
a primary key. Important point is that it is a user defined, human
readable key.


I've never actually heard of this use case but it looks like it's been
baked in since the legacy BDM behavior in the code, which is itself a
bunch of technical debt that I'd love to remove at some point.


A couple of years ago when Dipanov accidentally broke the merging, we
asked him to fix it here in ML, and he did it. If it's important, i'll
find these reviews and ML thread.


The history isn't as important to me. What's frustrating is this is 
something people care about, and has been regressed before, and we don't 
have good enough test coverage or documentation to know it's something 
people do care about. I think before we move forward with any of this, 
we'll want a test added to Tempest that validates this use case so we 
avoid regressing it again.





I guess what I'm trying to get at is, is this just important for EC2
compatibility? Or do non-AWS OpenStack users care about this use case,
because we definitely don't advertise this anywhere in our
documentation, or test it in any of our integrated testing (Tempest). So
just because it happens to work by chance of a poor data model doesn't
make me want to bend over backward to keep this working.


Well, i cannot estimate the importance in absolute measurement, but in
comparison with OpenStack this use case is more important in AWS. Volume
backed images (EBS images) are used in AWS much more widely than in
OpenStack. There are some difficulties in Nova and Cinder because that
users try to avoid using volume backed images in favor of disk based
(instance-store) ones. This  explain why this use case is less important
for pure OpenStack users.


If we wanted to support updating/overriding a specific image BDM during
server create, I'd think we could do something more straight-forward in
the compute API, like add an "image_bdm_override=True" field to
block_device_mapping_v2, something like that. What do you think about
that alternative?


Since you want to delete the only natural key from bdm, how to refer on
a certain bdm in parameters? Honestly, without real merging implemented
in Nova, such workarounds fused into bdm structure do not look good for
me. It looks like a crutch for something unfinished. Another
disadvantages is that this new field may be added into DB, etc., because
all other bdm fields are. Perhaps more appropriate way is to add a new
'ignore_image_bdms' parameter to  run instance method. This brings
another questions, but does not directly affect bdms at least.


I'm not sure what the best design is for maintaining this in the API 
without using the device name. I rattled off a few options in the spec 
review, but they aren't great options, and some simply won't work for 
older BDM or image resources, so are non-starters. I'm hoping some 
others with a fresh perspective on this can chime in with ideas, 
otherwise I'll bring it up in the nova API subteam meeting this week so 
we can talk about the best way forward.


Thanks again Feodor.

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Removing BDM devices from POST requests

2017-04-02 Thread Feodor Tersin 
> Can you give some more details? How does this actually "merge" or

> replace BDMs defined in the image metadata? Is this because we use
> device name as part of a hack for a primary key in the database [1]? I
> assume it is. The lack of unique constraint in the BDM data model has
> also always been a source of bugs, especially with cells v1 and why we
> talk about adding a uuid column to that table every few months.

Right. This is not pure merging, but replacing, based on device names as a 
primary key. Important point is that it is a user defined, human readable key.

> I've never actually heard of this use case but it looks like it's been
> baked in since the legacy BDM behavior in the code, which is itself a
> bunch of technical debt that I'd love to remove at some point.

A couple of years ago when Dipanov accidentally broke the merging, we asked him 
to fix it here in ML, and he did it. If it's important, i'll find these reviews 
and ML thread.

> I guess what I'm trying to get at is, is this just important for EC2
> compatibility? Or do non-AWS OpenStack users care about this use case,
> because we definitely don't advertise this anywhere in our
> documentation, or test it in any of our integrated testing (Tempest). So
> just because it happens to work by chance of a poor data model doesn't
> make me want to bend over backward to keep this working.

Well, i cannot estimate the importance in absolute measurement, but in 
comparison with OpenStack this use case is more important in AWS. Volume backed 
images (EBS images) are used in AWS much more widely than in OpenStack. There 
are some difficulties in Nova and Cinder because that users try to avoid using 
volume backed images in favor of disk based (instance-store) ones. This  
explain why this use case is less important for pure OpenStack users.

> If we wanted to support updating/overriding a specific image BDM during
> server create, I'd think we could do something more straight-forward in
> the compute API, like add an "image_bdm_override=True" field to
> block_device_mapping_v2, something like that. What do you think about
> that alternative?

Since you want to delete the only natural key from bdm, how to refer on a 
certain bdm in parameters? Honestly, without real merging implemented in Nova, 
such workarounds fused into bdm structure do not look good for me. It looks 
like a crutch for something unfinished. Another disadvantages is that this new 
field may be added into DB, etc., because all other bdm fields are. Perhaps 
more appropriate way is to add a new 'ignore_image_bdms' parameter to  run 
instance method. This brings another questions, but does not directly affect 
bdms at least.

Thanks,
Feodor Tersin.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][devstack][kuryr][fuxi][zun] Consolidate docker installation

2017-04-02 Thread Davanum Srinivas 
Hongbin,

Nice. +1 in theory :) the etcd one i have a WIP for the etcd/DLM,
please see here https://review.openstack.org/#/c/445432/

-- Dims

On Sun, Apr 2, 2017 at 8:13 PM, Hongbin Lu  wrote:
> Hi devstack team,
>
>
>
> Please find my proposal about consolidating docker installation into one
> place that is devstack tree:
>
>
>
> https://review.openstack.org/#/c/452575/
>
>
>
> Currently, there are several projects that installed docker in their
> devstack plugins in various different ways. This potentially introduce
> issues if more than one such services were enabled in devstack because the
> same software package will be installed and configured multiple times. To
> resolve the problem, an option is to consolidate the docker installation
> script into one place so that all projects will leverage it. Before
> continuing this effort, I wanted to get early feedback to confirm if this
> kind of work will be accepted. BTW, etcd installation might have a similar
> problem and I would be happy to contribute another patch to consolidate it
> if that is what will be accepted as well.
>
>
>
> Best regards,
>
> Hongbin
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa][devstack][kuryr][fuxi][zun] Consolidate docker installation

2017-04-02 Thread Hongbin Lu 
Hi devstack team,

Please find my proposal about consolidating docker installation into one place 
that is devstack tree:

https://review.openstack.org/#/c/452575/

Currently, there are several projects that installed docker in their devstack 
plugins in various different ways. This potentially introduce issues if more 
than one such services were enabled in devstack because the same software 
package will be installed and configured multiple times. To resolve the 
problem, an option is to consolidate the docker installation script into one 
place so that all projects will leverage it. Before continuing this effort, I 
wanted to get early feedback to confirm if this kind of work will be accepted. 
BTW, etcd installation might have a similar problem and I would be happy to 
contribute another patch to consolidate it if that is what will be accepted as 
well.

Best regards,
Hongbin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Removing BDM devices from POST requests

2017-04-02 Thread Matt Riedemann 

On 4/2/2017 12:35 PM, Feodor Tersin wrote:

Device merging is used by EC2 API at least. This allows us (EC2 API
team) to implement a use case of changing attached volume parameters
when launching an instance from a volume backed image. If the image's
bdms contain several volume descriptions, but a user wants to change
some of them for a new instance, he can [1] E.g. the user may want to
increase root volume size, skip another volume.


Nova API doesn't support this ability fine (we must specify full bdm
rather than changes only), but it works at least. If you remove device
names from API, what alternative way are you going to provide instead
for this purpose?


[1] 
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html#Using_OverridingAMIBDM


Thanks,

Feodor Tersin.



Can you give some more details? How does this actually "merge" or 
replace BDMs defined in the image metadata? Is this because we use 
device name as part of a hack for a primary key in the database [1]? I 
assume it is. The lack of unique constraint in the BDM data model has 
also always been a source of bugs, especially with cells v1 and why we 
talk about adding a uuid column to that table every few months.


I've never actually heard of this use case but it looks like it's been 
baked in since the legacy BDM behavior in the code, which is itself a 
bunch of technical debt that I'd love to remove at some point.


I guess what I'm trying to get at is, is this just important for EC2 
compatibility? Or do non-AWS OpenStack users care about this use case, 
because we definitely don't advertise this anywhere in our 
documentation, or test it in any of our integrated testing (Tempest). So 
just because it happens to work by chance of a poor data model doesn't 
make me want to bend over backward to keep this working.


If we wanted to support updating/overriding a specific image BDM during 
server create, I'd think we could do something more straight-forward in 
the compute API, like add an "image_bdm_override=True" field to 
block_device_mapping_v2, something like that. What do you think about 
that alternative?


[1] 
https://github.com/openstack/nova/blob/468916ee57fede4d0c89fbf9c776269943e2cb44/nova/db/sqlalchemy/api.py#L4170-L4175


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Engine facade

2017-04-02 Thread Kevin Benton 
Do you have a link to a traceback?

On Apr 2, 2017 09:25, "Gary Kotton"  wrote:

> Hi,
>
> The change https://review.openstack.org/#/c/402750/ has broken the
> vmware-nsx plugin. I am not sure if this has had effect on any other
> decomposed plugins.
>
> One of the issues that we have is when we create a PortDNS object under a
> transaction we get an exception: DBReferenceError: (sqlite3.IntegrityError)
> FOREIGN KEY constraint failed [SQL: u'INSERT INTO portdnses (port_id,
> current_dns_name, current_dns_domain, previous_dns_name,
> previous_dns_domain, dns_name) VALUES (?, ?, ?, ?, ?, ?)'] [parameters:
> ('2f2039ac-e7e6-4cc3-a8a0-3298089d4afb', u'', u'', u'', u'',
> u'port-dns-name')]
>
> Any ideas?
>
> Thanks
>
> Gary
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kubernetes][go] External OpenStack Cloud Provider for Kubernetes

2017-04-02 Thread Monty Taylor 
On 03/29/2017 03:39 PM, Steve Gordon wrote:
> - Original Message -
>> From: "Davanum Srinivas" 
>> To: "Chris Hoge" 
>> Cc: "OpenStack Development Mailing List (not for usage questions)" 
>> ,
>> "kubernetes-sig-openstack" 
>> Sent: Wednesday, March 29, 2017 2:28:29 PM
>> Subject: Re: [openstack-dev] [kubernetes][go] External OpenStack Cloud 
>> Provider for Kubernetes
>>
>> Team,
>>
>> Repo is ready:
>> http://git.openstack.org/cgit/openstack/k8s-cloud-provider
>>
>> I've taken the liberty of updating it with the latest changes in the
>> kubernetes/kubernetes repo:
>> https://review.openstack.org/#/q/project:openstack/k8s-cloud-provider is
>> ready
>>
>> So logical next step would be to add CI jobs to test in OpenStack
>> Infra. Anyone interested?
> 
> One question I have around this - do we have a shared view of what the ideal 
> matrix of tested combinations would like? E.g. kubernetes master on openstack 
> project's master, kubernetes master on openstack project's stable branches 
> (where available), do we also need/want to test kubernetes stable milestones, 
> etc.
> 
> At a high level my goal would be the same as Chris's "k8s gating on OpenStack 
> in the same ways that it does on AWS and GCE." which would imply reporting 
> results on PRs proposed to K8S master *before* they merge but not sure we all 
> agree on what that actually means testing against in practice on the 
> OpenStack side of the equation?

I think we want to have jobs that have the ability to test:

1) A proposed change to k8s-openstack-provider against current master of
OpenStack
2) A proposed change to k8s-openstack-provider against a stable release
of OpenStack
3) A proposed change to OpenStack against current master of
k8s-openstack-provider
4) A proposed change to OpenStack against stable release of
k8s-openstack-provider

Those are all easy now that the code is in gerrit, and it's well defined
what triggers and where it reports.

Additionally, we need to test the surface area between
k8s-openstack-provider and k8s itself. (if we wind up needing to test
k8s against proposed changes to OpenStack then we've likely done
something wrong in life)

5) A proposed change to k8s-openstack-provider against current master of k8s
6) A proposed change to k8s-openstack-provider against a stable release
of k8s
7) A proposed change to k8s against current master of k8s-openstack-provider
8) A proposed change to k8s against stable release of k8s-openstack-provider

5 and 6 are things we can do right now. 7 and 8 will have to wait for GH
support to land in zuul (without which we can neither trigger test jobs
on proposed changes to k8s nor can we report the results back to anyone)

I would recommend that we make 5 and 6 non-voting until such a time as
we are reporting on 7 and 8 back to k8s and have a reasonable
expectation someone will pay attention to failures - otherwise k8s will
be able to wedge the k8s-openstack-provider gate.

>> On Sat, Mar 25, 2017 at 12:10 PM, Chris Hoge  wrote:
>>>
>>>
>>> On Friday, March 24, 2017 at 8:46:42 AM UTC-7, Antoni Segura Puimedon
>>> wrote:



 On Friday, March 24, 2017 at 3:59:18 PM UTC+1, Graham Hayes wrote:
>
> On 24/03/17 10:27 -0400, Davanum Srinivas wrote:
>> Folks,
>>
>> As discussed in the etherpad:
>> https://etherpad.openstack.org/p/go-and-containers
>>
>> Here's a request for a repo in OpenStack:
>> https://review.openstack.org/#/c/449641/
>>
>> This request pulls in the existing code from kubernetes/kubernetes
>> repo and preserves the git history too
>> https://github.com/dims/k8s-cloud-provider
>>
>> Anyone interested? please ping me on Slack or IRC and we can continue
>> this work.
>
> Yeah - I would love to continue the provider work on gerrit :)
>
> Is there a way for us to make sure changes in the k8 master don't
> break our plugin? Or do we need to periodic jobs on the provider repo
> to catch breakages in the plugin interface?


 I suppose the options are either:

 ask k8s to add select external cloud providers in the CI
 Have a webhook in the k8s repo that triggered CI on the OSt infra
>>>
>>>
>>> Yes please to these. My preference is for the provider to remain upstream
>>> in
>>> k8s, but it's development has stalled out a bit. I want the best provider
>>> possible, but also want to make sure it's tested and visible to the k8s
>>> community that want to run on OpenStack. I've mentioned before that one of
>>> my goals is to have k8s gating on OpenStack in the same ways that it does
>>> on
>>> AWS and GCE.
>>>
>>> -Chris
>>>
>>>
>
>
> Thanks, Graham
>
>
>>> __
>> OpenStack Development Mailing List (not for usage

Re: [openstack-dev] [kubernetes][go] External OpenStack Cloud Provider for Kubernetes

2017-04-02 Thread Monty Taylor 
On 04/02/2017 02:53 PM, Chris Hoge wrote:
> Now that the provider has a repository in the OpenStack project
> namespace, we need to move over the existing set of issues and pull
> requests and create an initial work list for migrating patches and
> fixing existing issues.
> 
> I've started up an etherpad where we can track that work[1]. In the longer
> run we should migrate over to Launchpad or Storyboard. One question,
> to help preserve continuity with the K8S community workflow: do we want
> to investigate ways to allow for issue creation in the OpenStack
> namespace on GitHub?

I do not think this is a thing we want to do. While I understand the
urge, a project needs to live somewhere (in this case we've chosen
OpenStack) and should behave as projects do in that location. When I
work on Ansible, I do issues on github. When I deal with tox, I file
issues on bitbucket. Back when I dealt with Jenkins I filed issues in
their Jira. I do not think that filing an issue in the issue tracker for
a project is too onerous of a request to make of someone.

We have issues turned off in all of our github mirrors, so it's highly
unlikely someone will accidentally attempt to file an issue like the.
(it's too bad we can't similarly turn off pull requests, but oh well)


> [1] https://etherpad.openstack.org/p/k8s-provider-issue-migration
> 
> On Friday, March 24, 2017 at 7:27:09 AM UTC-7, Davanum Srinivas wrote:
> 
> Folks,
> 
> As discussed in the etherpad:
> https://etherpad.openstack.org/p/go-and-containers
> 
> 
> Here's a request for a repo in OpenStack:
> https://review.openstack.org/#/c/449641/
> 
> 
> This request pulls in the existing code from kubernetes/kubernetes
> repo and preserves the git history too
> https://github.com/dims/k8s-cloud-provider
> 
> 
> Anyone interested? please ping me on Slack or IRC and we can
> continue this work.
> 
> Thanks,
> Dims
> 
> -- 
> Davanum Srinivas :: https://twitter.com/dims
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kubernetes][go] External OpenStack Cloud Provider for Kubernetes

2017-04-02 Thread Chris Hoge 
Now that the provider has a repository in the OpenStack project
namespace, we need to move over the existing set of issues and pull
requests and create an initial work list for migrating patches and
fixing existing issues.

I've started up an etherpad where we can track that work[1]. In the longer
run we should migrate over to Launchpad or Storyboard. One question,
to help preserve continuity with the K8S community workflow: do we want
to investigate ways to allow for issue creation in the OpenStack
namespace on GitHub?

-Chris

[1] https://etherpad.openstack.org/p/k8s-provider-issue-migration

On Friday, March 24, 2017 at 7:27:09 AM UTC-7, Davanum Srinivas wrote:
>
> Folks, 
>
> As discussed in the etherpad: 
> https://etherpad.openstack.org/p/go-and-containers 
>
> Here's a request for a repo in OpenStack: 
> https://review.openstack.org/#/c/449641/ 
>
> This request pulls in the existing code from kubernetes/kubernetes 
> repo and preserves the git history too 
> https://github.com/dims/k8s-cloud-provider 
>
> Anyone interested? please ping me on Slack or IRC and we can continue this 
> work. 
>
> Thanks, 
> Dims 
>
> -- 
> Davanum Srinivas :: https://twitter.com/dims 
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Removing BDM devices from POST requests

2017-04-02 Thread Feodor Tersin 
Device merging is used by EC2 API at least. This allows us (EC2 API team) to 
implement a use case of changing attached volume parameters when launching an 
instance from a volume backed image. If the image's bdms contain several volume 
descriptions, but a user wants to change some of them for a new instance, he 
can [1] E.g. the user may want to increase root volume size, skip another 
volume.


Nova API doesn't support this ability fine (we must specify full bdm rather 
than changes only), but it works at least. If you remove device names from API, 
what alternative way are you going to provide instead for this purpose?


[1] 
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html#Using_OverridingAMIBDM


Thanks,

Feodor Tersin.


From: Matt Riedemann 
Sent: Sunday, April 2, 2017 6:59:12 PM
To: Matt Riedemann; openstack-dev@lists.openstack.org; 
openstack-operat...@lists.openstack.org
Subject: Re: [openstack-dev] [nova] Removing BDM devices from POST requests

On 4/2/2017 10:13 AM, Matt Riedemann wrote:
> On 4/1/2017 1:01 PM, Matt Riedemann wrote:
>> I know we've talked about this over and over and another bug [1]
>> reminded me of it. We have long talked about removing the ability to
>> specify a block device name when creating a server or attaching a volume
>> because we can't honor the requested device name anyway and trying to do
>> so just causes issues. That's part of the reason why the libvirt driver
>> stopped honoring the block device name in requests back in Liberty [2].
>>
>> I think we all agree on removing the device name from the API, but I'm
>> having a hard time remembering if someone signed up to write a spec for
>> this. I could have sworn this came up recently and someone said they'd
>> write a spec, but I can't remember.
>>
>> So this is my attempt at remembering and if it's all a dream, then is
>> anyone interested in owning this? If not, I'll start writing the spec
>> this week.
>>
>> [1] https://bugs.launchpad.net/nova/+bug/1648323
>>
>
> One thing I thought about with this is if you want to identify the block
> device after it's attached to the server, you can use device tags to do
> that (introduced in microversion 2.32). But that's only at server create
> time until [1] is merged. So removing the device from attach volume
> requests is probably dependent on completing Artom's blueprint first for
> tagging volumes when you attach them to an existing server.
>
> [1]
> https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/virt-device-tagged-attach-detach.html
>
>

Spec started here:

https://review.openstack.org/#/c/452546/

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] Engine facade

2017-04-02 Thread Gary Kotton 
Hi,
The change https://review.openstack.org/#/c/402750/ has broken the vmware-nsx 
plugin. I am not sure if this has had effect on any other decomposed plugins.
One of the issues that we have is when we create a PortDNS object under a 
transaction we get an exception: DBReferenceError: (sqlite3.IntegrityError) 
FOREIGN KEY constraint failed [SQL: u'INSERT INTO portdnses (port_id, 
current_dns_name, current_dns_domain, previous_dns_name, previous_dns_domain, 
dns_name) VALUES (?, ?, ?, ?, ?, ?)'] [parameters: 
('2f2039ac-e7e6-4cc3-a8a0-3298089d4afb', u'', u'', u'', u'', u'port-dns-name')]
Any ideas?
Thanks
Gary
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Removing BDM devices from POST requests

2017-04-02 Thread Matt Riedemann 

On 4/2/2017 10:13 AM, Matt Riedemann wrote:

On 4/1/2017 1:01 PM, Matt Riedemann wrote:

I know we've talked about this over and over and another bug [1]
reminded me of it. We have long talked about removing the ability to
specify a block device name when creating a server or attaching a volume
because we can't honor the requested device name anyway and trying to do
so just causes issues. That's part of the reason why the libvirt driver
stopped honoring the block device name in requests back in Liberty [2].

I think we all agree on removing the device name from the API, but I'm
having a hard time remembering if someone signed up to write a spec for
this. I could have sworn this came up recently and someone said they'd
write a spec, but I can't remember.

So this is my attempt at remembering and if it's all a dream, then is
anyone interested in owning this? If not, I'll start writing the spec
this week.

[1] https://bugs.launchpad.net/nova/+bug/1648323



One thing I thought about with this is if you want to identify the block
device after it's attached to the server, you can use device tags to do
that (introduced in microversion 2.32). But that's only at server create
time until [1] is merged. So removing the device from attach volume
requests is probably dependent on completing Artom's blueprint first for
tagging volumes when you attach them to an existing server.

[1]
https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/virt-device-tagged-attach-detach.html




Spec started here:

https://review.openstack.org/#/c/452546/

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Removing BDM devices from POST requests

2017-04-02 Thread Matt Riedemann 

On 4/1/2017 1:01 PM, Matt Riedemann wrote:

I know we've talked about this over and over and another bug [1]
reminded me of it. We have long talked about removing the ability to
specify a block device name when creating a server or attaching a volume
because we can't honor the requested device name anyway and trying to do
so just causes issues. That's part of the reason why the libvirt driver
stopped honoring the block device name in requests back in Liberty [2].

I think we all agree on removing the device name from the API, but I'm
having a hard time remembering if someone signed up to write a spec for
this. I could have sworn this came up recently and someone said they'd
write a spec, but I can't remember.

So this is my attempt at remembering and if it's all a dream, then is
anyone interested in owning this? If not, I'll start writing the spec
this week.

[1] https://bugs.launchpad.net/nova/+bug/1648323



Oops I forgot the reference for the Liberty change.

[2] https://review.openstack.org/#/c/189632/

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Removing BDM devices from POST requests

2017-04-02 Thread Matt Riedemann 

On 4/1/2017 1:01 PM, Matt Riedemann wrote:

I know we've talked about this over and over and another bug [1]
reminded me of it. We have long talked about removing the ability to
specify a block device name when creating a server or attaching a volume
because we can't honor the requested device name anyway and trying to do
so just causes issues. That's part of the reason why the libvirt driver
stopped honoring the block device name in requests back in Liberty [2].

I think we all agree on removing the device name from the API, but I'm
having a hard time remembering if someone signed up to write a spec for
this. I could have sworn this came up recently and someone said they'd
write a spec, but I can't remember.

So this is my attempt at remembering and if it's all a dream, then is
anyone interested in owning this? If not, I'll start writing the spec
this week.

[1] https://bugs.launchpad.net/nova/+bug/1648323



One thing I thought about with this is if you want to identify the block 
device after it's attached to the server, you can use device tags to do 
that (introduced in microversion 2.32). But that's only at server create 
time until [1] is merged. So removing the device from attach volume 
requests is probably dependent on completing Artom's blueprint first for 
tagging volumes when you attach them to an existing server.


[1] 
https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/virt-device-tagged-attach-detach.html


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Idempotence of the deployment process

2017-04-02 Thread Dan Prince 
On Fri, 2017-03-31 at 17:21 -0600, Alex Schultz wrote:
> Hey folks,
> 
> I wanted to raise awareness of the concept of idempotence[0] and how
> it affects deployment(s).  In the puppet world, we consider this very
> important because since puppet is all about ensuring a desired state
> (ie. a system with config files + services). That being said, I feel
> that it is important for any deployment tool to be aware of this.
> When the same code is applied to the system repeatedly (as would be
> the case in a puppet master deployment) the subsequent runs should
> result in no changes if there is no need.  If you take a configured
> system and rerun the same deployment code you don't want your
> services
> restarting when the end state is supposed to be the same. In the case
> of TripleO, we should be able deploy an overcloud and rerun the
> deployment process should result in no configuration changes and 0
> services being restarted during the process. The second run should
> essentially be a noop.
> 
> We have recently uncovered various bugs[1][2][3][4] that have
> introduced service disruption due to a lack of idempotency causing
> service restarts. So when reviewing or developing new code what is
> important about the deployment is to think about what happens if I
> run
> this bit of code twice.  There are a few common items that come up
> around idempotency. Things like execs in puppet-tripleo should be
> refreshonly or use unless/onlyif to prevent running again if
> unnecessary.  Additionally in the TripleO configuration it's
> important
> to understand in which step a service is configured and if it
> possibly
> would get deconfigured in another step.  For example, we configure
> apache and some wsgi services in step 3. But we currently configure
> some additional wsgi openstack services in step 4 which is resulting
> in excessive httpd restarts and possible service unavailability[5]
> when updates are applied.
> 
> Another important place to understand this concept is in upgrades
> where we currently allow for ansible tasks to be used. These should
> result in an idempotent action when puppet is subsequently run which
> means that the two bits of code essentially need to result in the
> same
> configuration. For example in the nova-api upgrades for Newton to
> Ocata we needed to run the same commands[6] that would later be run
> by
> puppet to prevent clashing configurations and possible idempotency
> problems.
> 
> Idempotency issues can cause service disruptions, longer deployment
> times for end users, or even possible misconfigurations.  I think it
> might be beneficial to add an idempotency periodic job that is
> basically a double run of the deployment process to ensure no service
> or configuration changes on the second run. Thoughts?  Ideally one in
> the gate would be awesome but I think it would take to long to be
> feasible with all the other jobs we currently run.

How would we verify that services aren't getting changed/restarted
even? Checking process runtimes perhaps or something?

If you used the multinode jobs or perhaps the new undercloud_deploy
installer (single node) it might be feasible to add this into the gate.
I would avoid adding this to the OVB queue as it is already too full
and we can probably gain the coverage we need without that type of
testing.

Dan

> 
> Thanks,
> -Alex
> 
> [0] http://binford2k.com/content/2015/10/idempotence-not-just-big-sca
> ry-word
> [1] https://bugs.launchpad.net/tripleo/+bug/1664650
> [2] https://bugs.launchpad.net/puppet-nova/+bug/1665443
> [3] https://bugs.launchpad.net/tripleo/+bug/1665405
> [4] https://bugs.launchpad.net/tripleo/+bug/1665426
> [5] https://review.openstack.org/#/c/434016/
> [6] https://review.openstack.org/#/c/405241/
> 
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] - Adding Miguel Lavalle to neutron-drivers team

2017-04-02 Thread Gary Kotton 
+1

On 3/31/17, 12:55 AM, "Ihar Hrachyshka"  wrote:

On Thu, Mar 30, 2017 at 2:11 PM, Kevin Benton  wrote:
> Hi,
>
> I would like to add Miguel Lavalle (mlavalle) to the Neutron drivers team
> [1].

It's not completely clear if you seek support here, but in case you do, +1.

Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev