[Openstack-operators] leaving Openstack mailing lists

[Saverio Proto]

Hello,

I will be leaving this mailing list in a few days.

I am going to a new job and I will not be involved with Openstack at
least in the short term future.
Still, it was great working with the Openstack community in the past few years.

If you need to reach me about any bug/patch/review that I submitted in
the past, just write directly to my email. I will try to give answers.

Cheers

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Routing deployments + Storage networks

[Saverio Proto]

Hello,

we route the Ceph storage network in the same fabric. We did not have
problems with that so far.

Cheers

Saverio

Il giorno gio 16 ago 2018 alle ore 10:43 Paul Browne 
ha scritto:
>
> Hi operators,
>
> I had a quick question for those operators who use a routed topology for 
> their OpenStack deployments, whether routed spine-leaf or routed underlay 
> providing L2 connectivity in tunnels;
>
> Where using one, would the storage network (e.g. Ceph public network) also be 
> routed on the same fabric, or would separate fabric be employed here to 
> reduce hops?
>
> Many thanks,
> Paul Browne
>
> --
> ***
> Paul Browne
> Research Computing Platforms
> University Information Services
> Roger Needham Building
> JJ Thompson Avenue
> University of Cambridge
> Cambridge
> United Kingdom
> E-Mail: pf...@cam.ac.uk
> Tel: 0044-1223-746548
> ***
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova] ask deployment question

[Saverio Proto]

Hello Rambo,

you can find information about other deployments reading the User Survey:
https://www.openstack.org/user-survey/survey-2018/landing

For blog posts with experiences from other operators check out:
https://superuser.openstack.org/ and http://planet.openstack.org/

Cheers

Saverio

Il giorno gio 16 ago 2018 alle ore 11:59 Rambo 
ha scritto:
>
> Hi,all
>I have some questions about deploy the large scale openstack 
> cloud.Such as
>1.Only in one region situation,How many physical machines are the 
> biggest deployment scale in our community?
>Can you tell me more about these combined with own practice? Would you 
> give me some methods to learn it?Such as the website,blog and so on. Thank 
> you very much!Looking forward to hearing from you.
>
>
>
>
>
>
>
>
> Best Regards
> Rambo
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Openstack Version discovery with the cli client.

[Saverio Proto]

Thanks !

I think the command I was looking for is:
"openstack versions show"

But for example for Neutron I get just version v2.0 from Newton to
Pike, that tells me very little.

The use case is when testing Kubernetes on Openstack, a lot of
kubernetes users cannot tell easily the version of Openstack they are
testing on. Because things like the LBaaS are so different between
Openstack releases that version v2.0 tells too little.
Often it is good to know what is the version of the Openstack cloud to
identify bugs on launchpad.

Cheers,

Saverio




Il giorno mar 7 ago 2018 alle ore 15:48 George Mihaiescu
 ha scritto:
>
> Hi Saverio,
>
> I think only the API versions supported by some of the endpoint are 
> discoverable, as described here: 
> https://wiki.openstack.org/wiki/VersionDiscovery
>
> curl https://x.x.x.x:9292/image
> curl https://x.x.x.x:8774/compute
>
>
> Cheers,
> George
>
> On Tue, Aug 7, 2018 at 9:30 AM, Saverio Proto  wrote:
>>
>> Hello Jimmy,
>>
>> thanks for your help. If I understand correctly the answer you linked,
>> that helps if you operate the cloud and you have access to the
>> servers. Then of course you can call nova-manage.
>>
>> But being a user of a public cloud without having access the the
>> infrastructure servers ... how do you do that ?
>>
>> thanks
>>
>> Saverio
>>
>>
>>
>> Il giorno mar 7 ago 2018 alle ore 15:09 Jimmy McArthur
>>  ha scritto:
>> >
>> > Hey Saverio,
>> >
>> > This answer from ask.openstack.org should have what you're looking for:
>> > https://ask.openstack.org/en/question/45513/how-to-find-out-which-version-of-openstack-is-installed/at
>> >
>> > Once you get the release number, you have to look it up here to match
>> > the release date: https://releases.openstack.org/
>> >
>> > I had to use this the other day when taking the COA.
>> >
>> > Cheers,
>> > Jimmy
>> >
>> > Saverio Proto wrote:
>> > > Hello,
>> > >
>> > > This is maybe a super trivial question bit I have to admit I could not
>> > > figure it out.
>> > >
>> > > Can the user with the openstack cli client discover the version of
>> > > Openstack that is running ?
>> > >
>> > > For example in kubernetes the kubectl version command returns the
>> > > version of the client and the version of the cluster.
>> > >
>> > > For Openstack I never managed to discover the backend version, and
>> > > this could be useful when using public clouds.
>> > >
>> > > Anyone knows how to do that ?
>> > >
>> > > thanks
>> > >
>> > > Saverio
>> > >
>> > > ___
>> > > OpenStack-operators mailing list
>> > > OpenStack-operators@lists.openstack.org
>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> >
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Openstack Version discovery with the cli client.

[Saverio Proto]

Hello Jimmy,

thanks for your help. If I understand correctly the answer you linked,
that helps if you operate the cloud and you have access to the
servers. Then of course you can call nova-manage.

But being a user of a public cloud without having access the the
infrastructure servers ... how do you do that ?

thanks

Saverio



Il giorno mar 7 ago 2018 alle ore 15:09 Jimmy McArthur
 ha scritto:
>
> Hey Saverio,
>
> This answer from ask.openstack.org should have what you're looking for:
> https://ask.openstack.org/en/question/45513/how-to-find-out-which-version-of-openstack-is-installed/at
>
> Once you get the release number, you have to look it up here to match
> the release date: https://releases.openstack.org/
>
> I had to use this the other day when taking the COA.
>
> Cheers,
> Jimmy
>
> Saverio Proto wrote:
> > Hello,
> >
> > This is maybe a super trivial question bit I have to admit I could not
> > figure it out.
> >
> > Can the user with the openstack cli client discover the version of
> > Openstack that is running ?
> >
> > For example in kubernetes the kubectl version command returns the
> > version of the client and the version of the cluster.
> >
> > For Openstack I never managed to discover the backend version, and
> > this could be useful when using public clouds.
> >
> > Anyone knows how to do that ?
> >
> > thanks
> >
> > Saverio
> >
> > ___
> > OpenStack-operators mailing list
> > OpenStack-operators@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Neutron not adding iptables rules for metadata agent

[Saverio Proto]

Hello,

I would suggest to open a bug on launchpad to track this issue.

thank you

Saverio

2018-06-18 12:19 GMT+02:00 Radu Popescu | eMAG, Technology
:
> Hi,
>
> We're using Openstack Ocata, deployed using Openstack Ansible v15.1.7.
> Neutron server is v10.0.3.
> I can see enable_isolated_metadata and enable_metadata_network only used for
> isolated networks that don't have a router which is not our case.
> Also, I checked all namespaces on all our novas and only affected 6 out of
> 66 ..and only 1 namespace / nova. Seems like isolated case that doesn't
> happen very often.
>
> Can it be RabbitMQ? I'm not sure where to check.
>
> Thanks,
> Radu
>
> On Fri, 2018-06-15 at 17:11 +0200, Saverio Proto wrote:
>
> Hello Radu,
>
>
> yours look more or less like a bug report. This you check existing
>
> open bugs for neutron ? Also what version of openstack are you running
>
> ?
>
>
> how did you configure enable_isolated_metadata and
>
> enable_metadata_network options ?
>
>
> Saverio
>
>
> 2018-06-13 12:45 GMT+02:00 Radu Popescu | eMAG, Technology
>
> :
>
> Hi all,
>
>
> So, I'm having the following issue. I'm creating a VM with floating IP.
>
> Everything is fine, namespace is there, postrouting and prerouting from the
>
> internal IP to the floating IP are there. The only rules missing are the
>
> rules to access metadata service:
>
>
> -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp
>
> --dport 80 -j REDIRECT --to-ports 9697
>
> -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp
>
> --dport 80 -j MARK --set-xmark 0x1/0x
>
>
> (this is taken from another working namespace with iptables-save)
>
>
> Forgot to mention, VM is booting ok, I have both the default route and the
>
> one for the metadata service (cloud-init is running at boot time):
>
> [   57.150766] cloud-init[892]: ci-info:
>
> ++--+--+---+---+---+
>
> [   57.150997] cloud-init[892]: ci-info: | Device |  Up  |   Address|
>
> Mask | Scope | Hw-Address|
>
> [   57.151219] cloud-init[892]: ci-info:
>
> ++--+--+---+---+---+
>
> [   57.151431] cloud-init[892]: ci-info: |  lo:   | True |  127.0.0.1   |
>
> 255.0.0.0   |   .   | . |
>
> [   57.151627] cloud-init[892]: ci-info: | eth0:  | True | 10.240.9.186 |
>
> 255.255.252.0 |   .   | fa:16:3e:43:d1:c2 |
>
> [   57.151815] cloud-init[892]: ci-info:
>
> ++--+--+---+---+---+
>
> [   57.152018] cloud-init[892]: ci-info:
>
> +++Route IPv4
>
> info
>
> [   57.152225] cloud-init[892]: ci-info:
>
> +---+-++-+---+---+
>
> [   57.152426] cloud-init[892]: ci-info: | Route |   Destination   |
>
> Gateway   | Genmask | Interface | Flags |
>
> [   57.152621] cloud-init[892]: ci-info:
>
> +---+-++-+---+---+
>
> [   57.152813] cloud-init[892]: ci-info: |   0   | 0.0.0.0 |
>
> 10.240.8.1 | 0.0.0.0 |eth0   |   UG  |
>
> [   57.153013] cloud-init[892]: ci-info: |   1   |10.240.1.0   |
>
> 0.0.0.0   |  255.255.255.0  |eth0   |   U   |
>
> [   57.153202] cloud-init[892]: ci-info: |   2   |10.240.8.0   |
>
> 0.0.0.0   |  255.255.252.0  |eth0   |   U   |
>
> [   57.153397] cloud-init[892]: ci-info: |   3   | 169.254.169.254 |
>
> 10.240.8.1 | 255.255.255.255 |eth0   |  UGH  |
>
> [   57.153579] cloud-init[892]: ci-info:
>
> +---+-++-+---+---+
>
>
> The extra route is there because the tenant has 2 subnets.
>
>
> Before adding those 2 rules manually, I had this coming from cloud-init:
>
>
> [  192.451801] cloud-init[892]: 2018-06-13 12:29:26,179 -
>
> url_helper.py[WARNING]: Calling
>
> 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]:
>
> request error [('Connection aborted.', error(113, 'No route to host'))]
>
> [  193.456805] cloud-init[892]: 2018-06-13 12:29:27,184 -
>
> url_helper.py[WARNING]: Calling
>
> 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [1/120s]:
>
> request error [('Connection aborted.', error(113, 'No route to host'))]
>
> [  194.461592] cloud-init[892]: 2018-06-13 12:29:28,189 -
>
> url_helper.py[WARNING]: Calling
>
> 'http://169.254.169.254/2009-04-04/meta-data

Re: [Openstack-operators] Neutron not adding iptables rules for metadata agent

[Saverio Proto]

Hello Radu,

yours look more or less like a bug report. This you check existing
open bugs for neutron ? Also what version of openstack are you running
?

how did you configure enable_isolated_metadata and
enable_metadata_network options ?

Saverio

2018-06-13 12:45 GMT+02:00 Radu Popescu | eMAG, Technology
:
> Hi all,
>
> So, I'm having the following issue. I'm creating a VM with floating IP.
> Everything is fine, namespace is there, postrouting and prerouting from the
> internal IP to the floating IP are there. The only rules missing are the
> rules to access metadata service:
>
> -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp
> --dport 80 -j REDIRECT --to-ports 9697
> -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp
> --dport 80 -j MARK --set-xmark 0x1/0x
>
> (this is taken from another working namespace with iptables-save)
>
> Forgot to mention, VM is booting ok, I have both the default route and the
> one for the metadata service (cloud-init is running at boot time):
> [   57.150766] cloud-init[892]: ci-info:
> ++--+--+---+---+---+
> [   57.150997] cloud-init[892]: ci-info: | Device |  Up  |   Address|
> Mask | Scope | Hw-Address|
> [   57.151219] cloud-init[892]: ci-info:
> ++--+--+---+---+---+
> [   57.151431] cloud-init[892]: ci-info: |  lo:   | True |  127.0.0.1   |
> 255.0.0.0   |   .   | . |
> [   57.151627] cloud-init[892]: ci-info: | eth0:  | True | 10.240.9.186 |
> 255.255.252.0 |   .   | fa:16:3e:43:d1:c2 |
> [   57.151815] cloud-init[892]: ci-info:
> ++--+--+---+---+---+
> [   57.152018] cloud-init[892]: ci-info:
> +++Route IPv4
> info
> [   57.152225] cloud-init[892]: ci-info:
> +---+-++-+---+---+
> [   57.152426] cloud-init[892]: ci-info: | Route |   Destination   |
> Gateway   | Genmask | Interface | Flags |
> [   57.152621] cloud-init[892]: ci-info:
> +---+-++-+---+---+
> [   57.152813] cloud-init[892]: ci-info: |   0   | 0.0.0.0 |
> 10.240.8.1 | 0.0.0.0 |eth0   |   UG  |
> [   57.153013] cloud-init[892]: ci-info: |   1   |10.240.1.0   |
> 0.0.0.0   |  255.255.255.0  |eth0   |   U   |
> [   57.153202] cloud-init[892]: ci-info: |   2   |10.240.8.0   |
> 0.0.0.0   |  255.255.252.0  |eth0   |   U   |
> [   57.153397] cloud-init[892]: ci-info: |   3   | 169.254.169.254 |
> 10.240.8.1 | 255.255.255.255 |eth0   |  UGH  |
> [   57.153579] cloud-init[892]: ci-info:
> +---+-++-+---+---+
>
> The extra route is there because the tenant has 2 subnets.
>
> Before adding those 2 rules manually, I had this coming from cloud-init:
>
> [  192.451801] cloud-init[892]: 2018-06-13 12:29:26,179 -
> url_helper.py[WARNING]: Calling
> 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]:
> request error [('Connection aborted.', error(113, 'No route to host'))]
> [  193.456805] cloud-init[892]: 2018-06-13 12:29:27,184 -
> url_helper.py[WARNING]: Calling
> 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [1/120s]:
> request error [('Connection aborted.', error(113, 'No route to host'))]
> [  194.461592] cloud-init[892]: 2018-06-13 12:29:28,189 -
> url_helper.py[WARNING]: Calling
> 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [2/120s]:
> request error [('Connection aborted.', error(113, 'No route to host'))]
> [  195.466441] cloud-init[892]: 2018-06-13 12:29:29,194 -
> url_helper.py[WARNING]: Calling
> 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [3/120s]:
> request error [('Connection aborted.', error(113, 'No route to host'))]
>
> I can see no errors in neither nova or neutron services.
> In the mean time, I've searched all our nova servers for this kind of
> behavior and we have 1 random namespace missing those rules on 6 of our 66
> novas.
>
> Any ideas would be greatly appreciated.
>
> Thanks,
> Radu
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev][publiccloud-wg][k8s][octavia] OpenStack Load Balancer APIs and K8s

[Saverio Proto]

Hello Flint,

what version of Kubernetes are you deploying on top of Openstack ?

are you using the external Openstack cloud controller ? I tested it an
it works only if you have at least v.1.10.3

Look at this page:
https://github.com/kubernetes/cloud-provider-openstack/tree/master/examples/loadbalancers

Please test that you can make a SSL termination on the loadbalancer,
describing it with Kubernetes yaml files. That is important for
production operation. Test also if you have downtime when you have to
renew SSL certificates.

You will also want to check that traffic that hits your pods has the
HTTP header X-Forwarded-For, or even better the IP packets you receive
at the Pods have the source IP address of the original client.

If needed test everything also with IPv6

I personally decided not to use Octavia, but to go for the Kubernetes
ingress-nginx
https://github.com/kubernetes/ingress-nginx

The key idea is that instead of Openstack controlling the LoadBalancer
having Octavia spinning up a VM running nginx, you have Kubernetes
controlling the LoadBalancer, running a nginx-container.
At the end you need a nginx to reverse proxy, you have to decided if
this resource is managed by Openstack or Kubernetes.

Keep in mind that if you go for a kubernetes ingress controller you
can avoid using nginx. There is already an alternative ha-proxy
implementation:
https://www.haproxy.com/blog/haproxy_ingress_controller_for_kubernetes/

Cheers,

Saverio

2018-05-28 19:09 GMT+02:00 Flint WALRUS :
> Hi everyone, I’m currently deploying Octavia as our global LBaaS for a lot
> of various workload such as Kubernetes ingress LB.
>
> We use Queens and plan to upgrade to rocky as soon as it reach the stable
> release and we use the native Octavia APIv2 (Not a neutron redirect etc).
>
> What do you need to know?
>
> Le lun. 28 mai 2018 à 14:50, Saverio Proto  a écrit :
>>
>> Hello Chris,
>>
>> I finally had the time to write about my deployment:
>>
>> https://cloudblog.switch.ch/2018/05/22/openstack-horizon-runs-on-kubernetes-in-production-at-switch/
>>
>> in this blog post I explain why I use the kubernetes nginx-ingress
>> instead of Openstack LBaaS.
>>
>> Cheers,
>>
>> Saverio
>>
>>
>> 2018-03-15 23:55 GMT+01:00 Chris Hoge :
>> > Hi everyone,
>> >
>> > I wanted to notify you of a thread I started in openstack-dev about the
>> > state
>> > of the OpenStack load balancer APIs and the difficulty in integrating
>> > them
>> > with Kubernetes. This in part directly relates to current public and
>> > private
>> > deployments, and any feedback you have would be appreciated. Especially
>> > feedback on which version of the load balancer APIs you deploy, and if
>> > you
>> > haven't moved on to Octavia, why.
>> >
>> >
>> > http://lists.openstack.org/pipermail/openstack-dev/2018-March/128399.html
>> > <http://lists.openstack.org/pipermail/openstack-dev/2018-March/128399.html>
>> >
>> > Thanks in advance,
>> > Chris
>> > ___
>> > OpenStack-operators mailing list
>> > OpenStack-operators@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev][publiccloud-wg][k8s][octavia] OpenStack Load Balancer APIs and K8s

[Saverio Proto]

Hello Chris,

I finally had the time to write about my deployment:
https://cloudblog.switch.ch/2018/05/22/openstack-horizon-runs-on-kubernetes-in-production-at-switch/

in this blog post I explain why I use the kubernetes nginx-ingress
instead of Openstack LBaaS.

Cheers,

Saverio


2018-03-15 23:55 GMT+01:00 Chris Hoge :
> Hi everyone,
>
> I wanted to notify you of a thread I started in openstack-dev about the state
> of the OpenStack load balancer APIs and the difficulty in integrating them
> with Kubernetes. This in part directly relates to current public and private
> deployments, and any feedback you have would be appreciated. Especially
> feedback on which version of the load balancer APIs you deploy, and if you
> haven't moved on to Octavia, why.
>
> http://lists.openstack.org/pipermail/openstack-dev/2018-March/128399.html 
> 
>
> Thanks in advance,
> Chris
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] attaching network cards to VMs taking a very long time

[Saverio Proto]

Glad to hear it!
Always monitor rabbitmq queues to identify bottlenecks !! :)

Cheers

Saverio

Il gio 24 mag 2018, 11:07 Radu Popescu | eMAG, Technology <
radu.pope...@emag.ro> ha scritto:

> Hi,
>
> did the change yesterday. Had no issue this morning with neutron not being
> able to move fast enough. Still, we had some storage issues, but that's
> another thing.
> Anyway, I'll leave it like this for the next few days and report back in
> case I get the same slow neutron errors.
>
> Thanks a lot!
> Radu
>
> On Wed, 2018-05-23 at 10:08 +, Radu Popescu | eMAG, Technology wrote:
>
> Hi,
>
> actually, I didn't know about that option. I'll enable it right now.
> Testing is done every morning at about 4:00AM ..so I'll know tomorrow
> morning if it changed anything.
>
> Thanks,
> Radu
>
> On Tue, 2018-05-22 at 15:30 +0200, Saverio Proto wrote:
>
> Sorry email went out incomplete.
>
> Read this:
>
> https://cloudblog.switch.ch/2017/08/28/starting-1000-instances-on-switchengines/
>
>
> make sure that Openstack rootwrap configured to work in daemon mode
>
>
> Thank you
>
>
> Saverio
>
>
>
> 2018-05-22 15:29 GMT+02:00 Saverio Proto <ziopr...@gmail.com>:
>
> Hello Radu,
>
>
> do you have the Openstack rootwrap configured to work in daemon mode ?
>
>
> please read this article:
>
>
> 2018-05-18 10:21 GMT+02:00 Radu Popescu | eMAG, Technology
>
> <radu.pope...@emag.ro>:
>
> Hi,
>
>
> so, nova says the VM is ACTIVE and actually boots with no network. We are
>
> setting some metadata that we use later on and have cloud-init for different
>
> tasks.
>
> So, VM is up, OS is running, but network is working after a random amount of
>
> time, that can get to around 45 minutes. Thing is, is not happening to all
>
> VMs in that test (around 300), but it's happening to a fair amount - around
>
> 25%.
>
>
> I can see the callback coming few seconds after neutron openvswitch agent
>
> says it's completed the setup. My question is, why is it taking so long for
>
> nova openvswitch agent to configure the port? I can see the port up in both
>
> host OS and openvswitch. I would assume it's doing the whole namespace and
>
> iptables setup. But still, 30 minutes? Seems a lot!
>
>
> Thanks,
>
> Radu
>
>
> On Thu, 2018-05-17 at 11:50 -0400, George Mihaiescu wrote:
>
>
> We have other scheduled tests that perform end-to-end (assign floating IP,
>
> ssh, ping outside) and never had an issue.
>
> I think we turned it off because the callback code was initially buggy and
>
> nova would wait forever while things were in fact ok, but I'll  change
>
> "vif_plugging_is_fatal = True" and "vif_plugging_timeout = 300" and run
>
> another large test, just to confirm.
>
>
> We usually run these large tests after a version upgrade to test the APIs
>
> under load.
>
>
>
>
> On Thu, May 17, 2018 at 11:42 AM, Matt Riedemann <mriede...@gmail.com>
>
> wrote:
>
>
> On 5/17/2018 9:46 AM, George Mihaiescu wrote:
>
>
> and large rally tests of 500 instances complete with no issues.
>
>
>
> Sure, except you can't ssh into the guests.
>
>
> The whole reason the vif plugging is fatal and timeout and callback code was
>
> because the upstream CI was unstable without it. The server would report as
>
> ACTIVE but the ports weren't wired up so ssh would fail. Having an ACTIVE
>
> guest that you can't actually do anything with is kind of pointless.
>
>
> ___
>
>
> OpenStack-operators mailing list
>
>
> OpenStack-operators@lists.openstack.org
>
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
>
> ___
>
> OpenStack-operators mailing list
>
> OpenStack-operators@lists.openstack.org
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
> ___
>
> OpenStack-operators mailing list
>
> OpenStack-operators@lists.openstack.org
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] attaching network cards to VMs taking a very long time

[Saverio Proto]

Sorry email went out incomplete.
Read this:
https://cloudblog.switch.ch/2017/08/28/starting-1000-instances-on-switchengines/

make sure that Openstack rootwrap configured to work in daemon mode

Thank you

Saverio


2018-05-22 15:29 GMT+02:00 Saverio Proto <ziopr...@gmail.com>:
> Hello Radu,
>
> do you have the Openstack rootwrap configured to work in daemon mode ?
>
> please read this article:
>
> 2018-05-18 10:21 GMT+02:00 Radu Popescu | eMAG, Technology
> <radu.pope...@emag.ro>:
>> Hi,
>>
>> so, nova says the VM is ACTIVE and actually boots with no network. We are
>> setting some metadata that we use later on and have cloud-init for different
>> tasks.
>> So, VM is up, OS is running, but network is working after a random amount of
>> time, that can get to around 45 minutes. Thing is, is not happening to all
>> VMs in that test (around 300), but it's happening to a fair amount - around
>> 25%.
>>
>> I can see the callback coming few seconds after neutron openvswitch agent
>> says it's completed the setup. My question is, why is it taking so long for
>> nova openvswitch agent to configure the port? I can see the port up in both
>> host OS and openvswitch. I would assume it's doing the whole namespace and
>> iptables setup. But still, 30 minutes? Seems a lot!
>>
>> Thanks,
>> Radu
>>
>> On Thu, 2018-05-17 at 11:50 -0400, George Mihaiescu wrote:
>>
>> We have other scheduled tests that perform end-to-end (assign floating IP,
>> ssh, ping outside) and never had an issue.
>> I think we turned it off because the callback code was initially buggy and
>> nova would wait forever while things were in fact ok, but I'll  change
>> "vif_plugging_is_fatal = True" and "vif_plugging_timeout = 300" and run
>> another large test, just to confirm.
>>
>> We usually run these large tests after a version upgrade to test the APIs
>> under load.
>>
>>
>>
>> On Thu, May 17, 2018 at 11:42 AM, Matt Riedemann <mriede...@gmail.com>
>> wrote:
>>
>> On 5/17/2018 9:46 AM, George Mihaiescu wrote:
>>
>> and large rally tests of 500 instances complete with no issues.
>>
>>
>> Sure, except you can't ssh into the guests.
>>
>> The whole reason the vif plugging is fatal and timeout and callback code was
>> because the upstream CI was unstable without it. The server would report as
>> ACTIVE but the ports weren't wired up so ssh would fail. Having an ACTIVE
>> guest that you can't actually do anything with is kind of pointless.
>>
>> ___
>>
>> OpenStack-operators mailing list
>>
>> OpenStack-operators@lists.openstack.org
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] attaching network cards to VMs taking a very long time

[Saverio Proto]

Hello Radu,

do you have the Openstack rootwrap configured to work in daemon mode ?

please read this article:

2018-05-18 10:21 GMT+02:00 Radu Popescu | eMAG, Technology
:
> Hi,
>
> so, nova says the VM is ACTIVE and actually boots with no network. We are
> setting some metadata that we use later on and have cloud-init for different
> tasks.
> So, VM is up, OS is running, but network is working after a random amount of
> time, that can get to around 45 minutes. Thing is, is not happening to all
> VMs in that test (around 300), but it's happening to a fair amount - around
> 25%.
>
> I can see the callback coming few seconds after neutron openvswitch agent
> says it's completed the setup. My question is, why is it taking so long for
> nova openvswitch agent to configure the port? I can see the port up in both
> host OS and openvswitch. I would assume it's doing the whole namespace and
> iptables setup. But still, 30 minutes? Seems a lot!
>
> Thanks,
> Radu
>
> On Thu, 2018-05-17 at 11:50 -0400, George Mihaiescu wrote:
>
> We have other scheduled tests that perform end-to-end (assign floating IP,
> ssh, ping outside) and never had an issue.
> I think we turned it off because the callback code was initially buggy and
> nova would wait forever while things were in fact ok, but I'll  change
> "vif_plugging_is_fatal = True" and "vif_plugging_timeout = 300" and run
> another large test, just to confirm.
>
> We usually run these large tests after a version upgrade to test the APIs
> under load.
>
>
>
> On Thu, May 17, 2018 at 11:42 AM, Matt Riedemann 
> wrote:
>
> On 5/17/2018 9:46 AM, George Mihaiescu wrote:
>
> and large rally tests of 500 instances complete with no issues.
>
>
> Sure, except you can't ssh into the guests.
>
> The whole reason the vif plugging is fatal and timeout and callback code was
> because the upstream CI was unstable without it. The server would report as
> ACTIVE but the ports weren't wired up so ssh would fail. Having an ACTIVE
> guest that you can't actually do anything with is kind of pointless.
>
> ___
>
> OpenStack-operators mailing list
>
> OpenStack-operators@lists.openstack.org
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Receipt to transfer the ownership of an instance

[Saverio Proto]

Hello Massimo,

what we suggest to our users, is to migrate a volume, and to create a
new VM from that volume.
https://help.switch.ch/engines/documentation/migrating-resources/

the bad thing is that the new VM has a new IP address, so eventually
DNS records have to be updated by the users.

Cheers,

Saverio


2018-04-23 10:17 GMT+02:00 Massimo Sgaravatto :
> As far as I understand there is not a clean way to transfer the ownership of
> an instance from a user to another one (the implementation of the blueprint
> https://blueprints.launchpad.net/nova/+spec/transfer-instance-ownership was
> abandoned).
>
>
> Is there at least a receipt (i.e. what needs to be changed in the database)
> that operators can follow to implement such use case ?
>
> Thanks, Massimo
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Nova resources are out of sync in ocata version

[Saverio Proto]

It works for me in Newton.
Try it at your own risk :)

Cheers,

Saverio

2018-04-09 13:23 GMT+02:00 Anwar Durrani <durrani.an...@gmail.com>:
> No this is different one. should i try this one ? if it works ?
>
> On Mon, Apr 9, 2018 at 4:11 PM, Saverio Proto <ziopr...@gmail.com> wrote:
>>
>> Hello Anwar,
>>
>> are you talking about this script ?
>>
>> https://github.com/openstack/osops-tools-contrib/blob/master/nova/nova-libvirt-compare.py
>>
>> it does not work for you ?
>>
>> Saverio
>>
>> 2018-04-09 11:53 GMT+02:00 Anwar Durrani <durrani.an...@gmail.com>:
>> > Hi All,
>> >
>> > Nova resources are out of sync in ocata version, what values are showing
>> > on
>> > dashboard are mismatch of actual running instances, i do remember i had
>> > script for auto sync resources but this script is getting fail in this
>> > case,
>> > Kindly help here.
>> >
>> > --
>> > Thanks & regards,
>> > Anwar M. Durrani
>> > +91-9923205011
>> >
>> >
>> >
>> > ___
>> > OpenStack-operators mailing list
>> > OpenStack-operators@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> >
>
>
>
>
> --
> Thanks & regards,
> Anwar M. Durrani
> +91-9923205011
>
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ocata security groups don't work with LBaaS v2 ports

[Saverio Proto]

Hello Ignazio,

it would interesting to know how this works. For instances ports,
those ports are created by openvswitch on the compute nodes, where the
neutron-agent will take care of the security groups enforcement (via
iptables or openvswitch rules).

the LBaaS is a namespace that lives where the neutron-lbaasv2-agent is running.

The question is if the neutron-lbaasv2-agent is capable for setting
iptables rules. I would start to read the code there.

Cheers,

Saverio


2018-03-23 13:51 GMT+01:00 Ignazio Cassano :
> Hi all,
> following the ocata documentation, I am trying to apply security group to a
> lbaas v2 port but
> it seems not working because any filter is applyed.
> The Port Security Enabled is True on lbaas port, so I expect applying
> security group should work.
> Is this a bug ?
> Regards
> Ignazio
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova] about rebuild instance booted from volume

[Saverio Proto]

My idea is that if delete_on_termination flag is set to False the
Volume should never be deleted by Nova.

my 2 cents

Saverio

2018-03-14 15:10 GMT+01:00 Tim Bell :
> Matt,
>
> To add another scenario and make things even more difficult (sorry (), if the 
> original volume has snapshots, I don't think you can delete it.
>
> Tim
>
>
> -Original Message-
> From: Matt Riedemann 
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
> 
> Date: Wednesday, 14 March 2018 at 14:55
> To: "openstack-...@lists.openstack.org" , 
> openstack-operators 
> Subject: Re: [openstack-dev] [nova] about rebuild instance booted from volume
>
> On 3/14/2018 3:42 AM, 李杰 wrote:
> >
> >  This is the spec about  rebuild a instance booted from
> > volume.In the spec,there is a
> >question about if we should delete the old root_volume.Anyone who
> > is interested in
> >booted from volume can help to review this. Any suggestion is
> > welcome.Thank you!
> >The link is here.
> >Re:the rebuild spec:https://review.openstack.org/#/c/532407/
>
> Copying the operators list and giving some more context.
>
> This spec is proposing to add support for rebuild with a new image for
> volume-backed servers, which today is just a 400 failure in the API
> since the compute doesn't support that scenario.
>
> With the proposed solution, the backing root volume would be deleted and
> a new volume would be created from the new image, similar to how boot
> from volume works.
>
> The question raised in the spec is whether or not nova should delete the
> root volume even if its delete_on_termination flag is set to False. The
> semantics get a bit weird here since that flag was not meant for this
> scenario, it's meant to be used when deleting the server to which the
> volume is attached. Rebuilding a server is not deleting it, but we would
> need to replace the root volume, so what do we do with the volume we're
> replacing?
>
> Do we say that delete_on_termination only applies to deleting a server
> and not rebuild and therefore nova can delete the root volume during a
> rebuild?
>
> If we don't delete the volume during rebuild, we could end up leaving a
> lot of volumes lying around that the user then has to clean up,
> otherwise they'll eventually go over quota.
>
> We need user (and operator) feedback on this issue and what they would
> expect to happen.
>
> --
>
> Thanks,
>
> Matt
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [horizon] collectstatic with custom theme is broken at least since Ocata

[Saverio Proto]

Hello Mateusz,

thanks for your input.

I just want to confirm that a patch was merged in master and backported
all the way back to Ocata to fix the bug.

details here:
https://bugs.launchpad.net/horizon/+bug/1744239

thank you

Saverio


On 05.02.18 14:54, Mateusz Kowalski wrote:
> Hi,
> 
> We are running Horizon in Pike and cannot confirm having the same problem as 
> you describe. We are using a custom theme however the folder structure is a 
> bit different than the one you presented in the bug report.
> In our case we have
> 
> - /usr/share/openstack-dashboard/openstack_dashboard/themes
> |-- cern
> |-- default
> |-- material
> 
> what means we do not modify at all files inside "default". Let me know if you 
> want to compare more deeply our changes to see where the problem comes from, 
> as for us "theme_file.split('/templates/')" does not cause the trouble.
> 
> Cheers,
> Mateusz
> 
>> On 5 Feb 2018, at 14:44, Saverio Proto <saverio.pr...@switch.ch> wrote:
>>
>> Hello,
>>
>> I have tried to find a fix to this:
>>
>> https://ask.openstack.org/en/question/107544/ocata-theme-customization-with-templates/
>> https://bugs.launchpad.net/horizon/+bug/1744239
>> https://review.openstack.org/#/c/536039/
>>
>> I am upgrading from Newton to Pike.
>>
>> Here the real question is: how is it possible that this bug was found so
>> late ???
>>
>> There is at least another operator that documented hitting this bug in
>> Ocata.
>>
>> Probably this bug went unnoticed because you hit it only if you have
>> customizations for Horizon. All the automatic testing does not notice
>> this bug.
>>
>> What I cannot undestand is.
>> - are we two operators hitting a corner case ?
>> - No one else uses Horizon with custom themes in production with
>> version newer than Newton ?
>>
>> This is all food for your brainstorming about LTS,bugfix branches,
>> release cycle changes
>>
>> Cheers,
>>
>> Saverio
>>
>>
>> -- 
>> SWITCH
>> Saverio Proto, Peta Solutions
>> Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
>> phone +41 44 268 15 15, direct +41 44 268 1573
>> saverio.pr...@switch.ch, http://www.switch.ch
>>
>> http://www.switch.ch/stories
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] AggregateMultiTenancyIsolation with multiple (many) projects

[Saverio Proto]

> If you’re willing to, I could share with you a way to get a FrankeinCloud
> using a Docker method with kolla to get a pike/queens/whatever cloud at the
> same time that your Ocata one.

I am interested in knowing more about this. If you have any link /
blog post please share them :)

thank you

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] [horizon] collectstatic with custom theme is broken at least since Ocata

[Saverio Proto]

Hello,

I have tried to find a fix to this:

https://ask.openstack.org/en/question/107544/ocata-theme-customization-with-templates/
https://bugs.launchpad.net/horizon/+bug/1744239
https://review.openstack.org/#/c/536039/

I am upgrading from Newton to Pike.

Here the real question is: how is it possible that this bug was found so
late ???

There is at least another operator that documented hitting this bug in
Ocata.

Probably this bug went unnoticed because you hit it only if you have
customizations for Horizon. All the automatic testing does not notice
this bug.

What I cannot undestand is.
 - are we two operators hitting a corner case ?
 - No one else uses Horizon with custom themes in production with
version newer than Newton ?

This is all food for your brainstorming about LTS,bugfix branches,
release cycle changes

Cheers,

Saverio


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] LTS pragmatic example

[Saverio Proto]

Hello !

thanks for accepting the patch :)

It looks like the best is always to send an email and have a short
discussion together, when we are not sure about a patch.

thank you

Cheers,

Saverio

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] LTS pragmatic example

[Saverio Proto]

> You seem to be interested in a policy shift toward more of a "bugfix"
> branch where any fix should be allowed to land, and where branch age
> should not be a factor. It would be interesting to assess if that is a
> general view. I know that distros in general are happy with more of a
> "stable" approach.

Hello Thierry,

you are right. A bugfix branch is very important for us. I cannot keep a
UX bug in production, when I have a user that opens a support ticket
about this. I have to avoid the second user opening the second ticket
for the very same problem.
If merging to a common bugfix branch is not possible, I will have to
carry local patches.
Please be aware that most Openstack Ubuntu packages do carry local
patches in the debian/patches/ folder.
I am pretty sure that this patch could land without problems in the
Ubuntu packages for Newton/Horizon.

> 
>> But merging a patch that changes a log file in Nova back to Newton was
>> OKAY few weeks ago.
> Could you provide a link to that one ?

sure, here it is:
https://review.openstack.org/#/q/If525313c63c4553abe8bea6f2bfaf75431ed18ea

Thank you

Saverio



-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] LTS pragmatic example

[Saverio Proto]

Hello all,

I am again proposing a change due to operations experience. I am
proposing a clean and simple cherry-pick to Ocata.

"it depends" works pretty bad as policy for accepting patches.

Now I really dont understand what is the issue with the Stable Policy
and this patch:

https://review.openstack.org/#/c/539439/

This is a UX problem. Horizon is giving the wrong information to the user.

I got this answer:
Ocata is the second phase of stable branches [1]. Only critical bugfixes
and security patches are acceptable. I don't think it belongs to the
category.

But merging a patch that changes a log file in Nova back to Newton was
OKAY few weeks ago.

I will not be able to be in person at the PTG, but please talk about
this. People just give up upstreaming stuff like this.

thank you

Saverio


On 15.11.17 03:37, Matt Riedemann wrote:
> On 11/14/2017 10:58 AM, Davanum Srinivas wrote:
>> Let's focus our energy on the etherpad please
>>
>> https://etherpad.openstack.org/p/LTS-proposal
>>
>> On Wed, Nov 15, 2017 at 3:35 AM, Davanum Srinivas <dava...@gmail.com>
>> wrote:
>>> Saverio,
>>>
>>> Please see this :
>>> https://docs.openstack.org/project-team-guide/stable-branches.html for
>>> current policies.
>>>
>>> On Wed, Nov 15, 2017 at 3:33 AM, Saverio Proto
>>> <saverio.pr...@switch.ch> wrote:
>>>>> Which stable policy does that patch violate?  It's clearly a bug
>>>>> because the wrong information is being logged.  I suppose it goes
>>>>> against the string freeze rule? Except that we've stopped translating
>>>>> log messages so maybe we don't need to worry about that in this case,
>>>>> since it isn't an exception.
>>>>
>>>> Well, I also would like to understand more about stable policy
>>>> violations.
>>>> When I proposed such patches in the past for the release N-2 I have
>>>> always got the answer: it is not a security issue so it will not be
>>>> merged.
>>>>
>>>> This is a good example of how things have been working so far:
>>>>
>>>> https://review.openstack.org/#/q/677eb1c4160c08cfce2900495741f0ea15f566fa
>>>>
>>>>
>>>> This cinder patch was merged in master. It was then merged in Mitaka.
>>>> But it was not merged in Liberty just because "only security fixes"
>>>> were
>>>> allowed at that point.
>>>>
>>>> You can read that in the comments:
>>>> https://review.openstack.org/#/c/306610/
>>>>
>>>> Is this kind of things going to change after the discussion in Sydney ?
>>>> The discussion is not enough ? what we need to get done then ?
>>>>
>>>> thank you
>>>>
>>>> Saverio
>>>>
>>>>
>>>> __
>>>>
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> -- 
>>> Davanum Srinivas :: https://twitter.com/dims
>>
>>
>>
> 
> Heh, I'm reading this thread after approving all of those patches.
> 
> The answer as to whether it's appropriate or not, is "it depends".
> Depends on the patch, depends on the age of the branch, etc.
> 
> In this case, newton is in phase 3 so normally it's only security or
> critical fixes allowed, but in this case it's so trivial and so
> obviously wrong that I was OK with approving it just to get it in before
> we end of life the branch.
> 
> So, it depends. And because it depends, that's also why we don't
> automate the backport of every fix made on master. Because guess what,
> we also backport "fixes" that introduce regressions, and when you do
> that to n-1 (Pike at this point) then you still have a lot of time to
> detect that and fix it upstream, but regressing things on the oldest
> branch leaves very little time to (1) have it detected and (2) get it
> fixed before end of life.
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] Snapshot from Cinder to Glance doing a ceph rbd clone

[Saverio Proto]

Hello ops,

we have this working for Nova ephemeral images already, but Cinder did
not implement this spec:
https://specs.openstack.org/openstack/cinder-specs/specs/liberty/optimze-rbd-copy-volume-to-image.html

Is anyone carrying an unmerged patch that implements this spec ?
I could not believe my eyes this morning when I figured out this was
working only for nova.

thanks

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

> 3.34.0 is a queens series release, which makes it more likely that more
> other dependencies would need to be updated. Even backporting the
> changes to the Ocata branch and releasing it from there would require
> updating several other libraries.
> 

That is what I was fearing. Consider that our upgrade schedule is now to
have Pike by the end of 2018. Unless we try to skip a release.

> Are you using packages from Canonical, or are you building them
> yourself?

I am using the packages from Canonical, but I am familiar patching those
packages and merge my changes upstream back to Canonical.
If the problem is just dependencies with the ".deb" packages, I can
handle that. But if the problem is really python code not working
together across multiple components, then I have little hope to fix this
in Newton.

Thanks for the support, if I manage to make some progress on this I will
send an update on this thread on the mailing list.

Cheers,

Saverio

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

Hello Doug,

I have run the script:
here is my output:

http://paste.openstack.org/show/650913/

At this point I have some questions. Can I upgrade just oslo.log library
keeping the rest of the stuff in Newton ?
The versions of oslo.log have a different numbering scheme than other
openstack projects, so I cannot understand the versions compatibility.

As far as I understand 3.34.0 should be enough for me ? :

 git tag --contains 1b012d0fc6811f00e032e52ed586fe37e157584d
3.34.0
3.35.0
3.36.0

thank you

Saverio


On 22.01.18 23:20, Doug Hellmann wrote:
> Excerpts from Saverio Proto's message of 2018-01-22 18:45:15 +0100:
>> Hello Doug,
>>
>> in the extra session I see just {"project": "unknown", "version": "unknown"}
>>
>> here a full line from nova-api:
>>
>> {"thread_name": "MainThread", "extra": {"project": "unknown", "version":
>> "unknown"}, "process": 31142, "relative_created": 3459415335.4091644,
>> "module": "wsgi", "message":
>> "2001:xxx::8100::80,2001:xxx::81ff::b0 \"GET
>> /v2/64b5b50eb21d4efe9783eb1d81a9ec65/os-services HTTP/1.1\" status: 200
>> len: 1812 time: 0.1813300", "hostname": "nova-0", "filename": "wsgi.py",
>> "levelno": 20, "lineno": 555, "asctime": "2018-01-22 18:37:02,312",
>> "msg": "2001:xxx::8100::80,2001:xxx::81ff::b0 \"GET
>> /v2/64b5b50eb21d4efe9783eb1d81a9ec65/os-services HTTP/1.1\" status: 200
>> len: 1812 time: 0.1813300", "args": [], "process_name": "MainProcess",
>> "name": "nova.osapi_compute.wsgi.server", "thread": 140414249163824,
>> "created": 1516642622.312235, "traceback": null, "msecs":
>> 312.23511695861816, "funcname": "handle_one_response", "pathname":
>> "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", "levelname": "INFO"}
> 
> It looks like you're running into a limitation of the older version of
> the library where the context was only logged from openstack source
> code. This particular log message is coming from the eventlet library.
> 
> Try running the script below and saving the output to a pastebin.
> 
> Under the newton version of oslo.log, I get
> http://paste.openstack.org/show/650566/ and under the queens version I
> get http://paste.openstack.org/show/650569/ which shows me that the
> "extra" handling is working more or less the same way but the "context"
> handling is improved in the newer version (lots of the values are null
> because I don't fully set up the context, but the request_id field has a
> valid value).
> 
> Doug
> 
> 
> #!/usr/bin/env python
> 
> from __future__ import print_function
> 
> import logging
> 
> from oslo_context import context
> from oslo_log import formatters, log
> 
> 
> ch = logging.StreamHandler()
> ch.setLevel(logging.DEBUG)
> 
> formatter = formatters.JSONFormatter()
> ch.setFormatter(formatter)
> 
> LOG = logging.getLogger()
> LOG.setLevel(logging.DEBUG)
> LOG.addHandler(ch)
> 
> ctx = context.RequestContext(request_id='the-request-id')
> 
> LOG.debug('without extra')
> print()
> 
> LOG.debug('with extra', extra={'context': ctx})
> print()
> 
> log.getLogger().debug('via KeywordArgumentAdapter', context=ctx)
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

ice looks
>>>> like.
>>>>
>>>> Doug
>>>
>>> Earlier in the thread you mentioned running the newton versions of
>>> neutron and oslo.log. The newton release has been marked end-of-life
>>> and is not supported by the community any longer. You may find
>>> support from your vendor, but if you're deploying on your own we'll
>>> have to work something else out. If we determine that this is a bug
>>> in the newton version of the library I won't have any way to give
>>> you a new release because the branch is closed.
>>>
>>> It should be possible for you to update just oslo.log to a more
>>> recent (and supported), although to do so you would have to get the
>>> package separately or build your own and that may complicate your
>>> deployment.
>>>
>>> More recent versions of the JSON formatter change the structure of
>>> the data to include the entire context (including the request id)
>>> in a separate key.  Are you updating to newton as part of upgrading
>>> further than that?  If so, we probably want to wait to debug this
>>> until you hit the latest supported version you're planning to deploy,
>>> in case the problem is already fixed there.
>>>
>>> Doug
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

Hello,

I figured out a bug is already open since a long time :(
https://bugs.launchpad.net/oslo.log/+bug/1564931

And there is already a review:
https://review.openstack.org/#/c/367514/

it looks like the review was not merged, and it went to abandoned
because of no progress on it for long time.

I rebased that code on the current master:
https://review.openstack.org/536149

Saverio


On 18.01.18 18:14, Doug Hellmann wrote:
> Excerpts from Doug Hellmann's message of 2018-01-18 11:45:28 -0500:
>> Excerpts from Saverio Proto's message of 2018-01-18 14:49:21 +0100:
>>> Hello all,
>>>
>>> well this oslo.log library looks like a core thing that is used by
>>> multiple projects. I feel scared hearing that bugs opened on that
>>> project are probably just ignored.
>>>
>>> should I reach out to the current PTL of OSLO ?
>>> https://github.com/openstack/governance/blob/master/reference/projects.yaml#L2580
>>>
>>> ChangBo Guo are you reading this thread ? Do you think this is a bug or
>>> a missing feature ? And moreover is really nobody looking at these
>>> oslo.log bugs ?
>>
>> The Oslo team is small, but we do pay attention to bug reports. I
>> don't think this issue is going to rise to the level of "drop what
>> you're doing and help because the world is on fire", so I think
>> Sean is just encouraging you to have a little patience.
>>
>> Please do go ahead and open a bug and attach (or paste into the
>> description) an example of what the log output for your service looks
>> like.
>>
>> Doug
> 
> Earlier in the thread you mentioned running the newton versions of
> neutron and oslo.log. The newton release has been marked end-of-life
> and is not supported by the community any longer. You may find
> support from your vendor, but if you're deploying on your own we'll
> have to work something else out. If we determine that this is a bug
> in the newton version of the library I won't have any way to give
> you a new release because the branch is closed.
> 
> It should be possible for you to update just oslo.log to a more
> recent (and supported), although to do so you would have to get the
> package separately or build your own and that may complicate your
> deployment.
> 
> More recent versions of the JSON formatter change the structure of
> the data to include the entire context (including the request id)
> in a separate key.  Are you updating to newton as part of upgrading
> further than that?  If so, we probably want to wait to debug this
> until you hit the latest supported version you're planning to deploy,
> in case the problem is already fixed there.
> 
> Doug
> 
> __________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

Hello all,

well this oslo.log library looks like a core thing that is used by
multiple projects. I feel scared hearing that bugs opened on that
project are probably just ignored.

should I reach out to the current PTL of OSLO ?
https://github.com/openstack/governance/blob/master/reference/projects.yaml#L2580

ChangBo Guo are you reading this thread ? Do you think this is a bug or
a missing feature ? And moreover is really nobody looking at these
oslo.log bugs ?

thanks

Saverio


On 18.01.18 12:37, Sean Dague wrote:
> A bug would be fine. I'm not sure how many people are keeping an eye on
> oslo.log bugs at this point, so be realistic in when it might get looked at.
> 
> On 01/18/2018 03:06 AM, Saverio Proto wrote:
>> Hello Sean,
>> after the brief chat we had on IRC, do you think I should open a bug
>> about this issue ?
>>
>> thank you
>>
>> Saverio
>>
>>
>> On 13.01.18 07:06, Saverio Proto wrote:
>>>> I don't think this is a configuration problem.
>>>>
>>>> Which version of the oslo.log library do you have installed?
>>>
>>> Hello Doug,
>>>
>>> I use the Ubuntu packages, at the moment I have this version:
>>>
>>> python-oslo.log   3.16.0-0ubuntu1~cloud0
>>>
>>> thank you
>>>
>>> Saverio
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
> 
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

Hello Sean,
after the brief chat we had on IRC, do you think I should open a bug
about this issue ?

thank you

Saverio


On 13.01.18 07:06, Saverio Proto wrote:
>> I don't think this is a configuration problem.
>>
>> Which version of the oslo.log library do you have installed?
> 
> Hello Doug,
> 
> I use the Ubuntu packages, at the moment I have this version:
> 
> python-oslo.log   3.16.0-0ubuntu1~cloud0
> 
> thank you
> 
> Saverio
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

> I don't think this is a configuration problem.
> 
> Which version of the oslo.log library do you have installed?

Hello Doug,

I use the Ubuntu packages, at the moment I have this version:

python-oslo.log   3.16.0-0ubuntu1~cloud0

thank you

Saverio

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

> Which service's logs are missing the request_id?
> 
If I look at neutron-server logs with my current setup I get two files:

neutron-server.log
neutron-server.json

the standard log file has in all neutron.wsgi lines information like:

 neutron.wsgi [req-4fda8017-50c7-40eb-9e7b-710e7fba0d01
97d349b9499b4bd29c5e167c65ca1fb3 d447c836b6934dfab41a03f1ff96d879 - - -]

where req-UUID is the request ID, and the other two values are the user
UUID and the keystone project UUID.

when I look at the same line in the JSON output this information is missing.

I am starting neutron-server with the command line option
--log-config-append=/etc/neutron/logging.neutron-server.conf

where the conf file looks like

[loggers]
keys = root, neutron

[handlers]
keys = logfile, jsonfile, null

[formatters]
keys = context, json, default

[logger_root]
level = WARNING
handlers = null

[logger_neutron]
level = INFO
handlers = logfile, jsonfile
qualname = neutron

[handler_logfile]
class = handlers.WatchedFileHandler
args = ('/var/log/neutron/neutron-server.log',)
formatter = context

[handler_jsonfile]
level = INFO
class = handlers.WatchedFileHandler
args = ('/var/log/neutron/neutron-server.json',)
formatter = json

[handler_null]
class = logging.NullHandler
formatter = default
args = ()

[formatter_context]
class = oslo_log.formatters.ContextFormatter

[formatter_json]
class = oslo_log.formatters.JSONFormatter

[formatter_default]
format = %(message)s


I had a look at nova-api and I have the same problem. Anyway in my
Kibana I never so a req-UUID what so ever, so this looks like a problem
with all the openstack services.

Is it a problem with my logging configuration ?

thank you

Saverio





-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] oslo.log JSON logs are missing the request ID

[Saverio Proto]

Hello,

probably someone here is using stuff like Kibana to look at Openstack
logs. We are trying here to use the json logging, and we are surprised
that the request-id is not printed in the json output.

I wrote this email to the devs:
http://lists.openstack.org/pipermail/openstack-dev/2018-January/126144.html

Does anyone have this working, or the field is really missing in the
code of the oslo.log json formatter ?

thanks a lot

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID

[Saverio Proto]

Hello,

we recently enabled the JSON logging to feed a Kibana dashboard and look
at the logs with modern tooling.

however it looks like in our Openstack Newton deployment that some
information in the JSON files is missing.

most important missing bit is the request-id, that we use to track an
event across multiple log files on multiple hosts.

Looking at the code it really looks like the request ID is there for the
context formatter and not for the json formatter.

https://github.com/openstack/oslo.log/blob/master/oslo_log/formatters.py#L208

https://github.com/openstack/oslo.log/blob/master/oslo_log/formatters.py#L460

I am an operator and a very bad python developer, so can anyone confirm
that is really missing in the code, and it is not me configuring stuff
wrongly ?

If it is really missing the request-id in the json log formatter, should
I open a bug about this ?

thank you

Saverio


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Mixed service version CI testing

[Saverio Proto]

it makes sense and it is very valuable !
thanks

Saverio

Il 19 dic 2017 4:59 PM, "Matt Riedemann"  ha scritto:

> During discussion in the TC channel today [1], we got talking about how
> there is a perception that you must upgrade all of the services together
> for anything to work, at least the 'core' services like
> keystone/nova/cinder/neutron/glance - although maybe that's really just
> nova/cinder/neutron?
>
> Anyway, I posit that the services are not as tightly coupled as some
> people assume they are, at least not since kilo era when microversions
> started happening in nova.
>
> However, with the way we do CI testing, and release everything together,
> the perception is there that all things must go together to work.
>
> In our current upgrade job, we upgrade everything to N except the
> nova-compute service, that remains at N-1 to test rolling upgrades of your
> computes and to make sure guests are unaffected by the upgrade of the
> control plane.
>
> I asked if it would be valuable to our users (mostly ops for this right?)
> if we had an upgrade job where everything *except* nova were upgraded. If
> that's how the majority of people are doing upgrades anyway it seems we
> should make sure that works.
>
> I figure leaving nova at N-1 makes more sense because nova depends on the
> other services (keystone/glance/cinder/neutron) and is likely the harder
> / slower upgrade if you're going to do rolling upgrades of your compute
> nodes.
>
> This type of job would not run on nova changes on the master branch, since
> those changes would not be exercised in this type of environment. So we'd
> run this on master branch changes to keystone/cinder/glance/neutron
> /trove/designate/etc.
>
> Does that make sense? Would this be valuable at all? Or should the
> opposite be tested where we upgrade nova to N and leave all of the
> dependent services at N-1?
>
> Really looking for operator community feedback here.
>
> [1] http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23op
> enstack-tc.2017-12-19.log.html#t2017-12-19T15:14:15
>
> --
>
> Thanks,
>
> Matt
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] Security Groups and Metadata Service

[Saverio Proto]

Hello,

we have this recurring problem with our users.

An advanced user deletes all the default security groups to create his
own. This user will define only ingress rules.

Because there is no egress rule, the cloud-init will fail to open a
connection to the metadata service.

The user will open a ticket that he cant login into the VM, because of
corse the SSH key was not injected.

Does anyone has a good solution to prevent the user from setting the
system in a such a way that does not work ??

thank you

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [Openstack-operators] LTS pragmatic example

[Saverio Proto]

> Which stable policy does that patch violate?  It's clearly a bug
> because the wrong information is being logged.  I suppose it goes
> against the string freeze rule? Except that we've stopped translating
> log messages so maybe we don't need to worry about that in this case,
> since it isn't an exception.

Well, I also would like to understand more about stable policy violations.
When I proposed such patches in the past for the release N-2 I have
always got the answer: it is not a security issue so it will not be merged.

This is a good example of how things have been working so far:

https://review.openstack.org/#/q/677eb1c4160c08cfce2900495741f0ea15f566fa

This cinder patch was merged in master. It was then merged in Mitaka.
But it was not merged in Liberty just because "only security fixes" were
allowed at that point.

You can read that in the comments:
https://review.openstack.org/#/c/306610/

Is this kind of things going to change after the discussion in Sydney ?
The discussion is not enough ? what we need to get done then ?

thank you

Saverio


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] LTS pragmatic example

[Saverio Proto]

Hello,

here an example of a trivial patch that is important for people that
do operations, and they have to troubleshoot stuff.

with the old Stable Release thinking this patch would not be accepted
on old stable branches.

Let's see if this gets accepted back to stable/newton

https://review.openstack.org/#/q/If525313c63c4553abe8bea6f2bfaf75431ed18ea

Please note that a developers/operators that make the effort of fixing
this in master, should do also all the cherry-pickes back. We dont
have any automatic procudure for this.

thank you

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] LTS pragmatic example

[Saverio Proto]

Hello,

here an example of a trivial patch that is important for people that
do operations, and they have to troubleshoot stuff.

with the old Stable Release thinking this patch would not be accepted
on old stable branches.

Let's see if this gets accepted back to stable/newton

https://review.openstack.org/#/q/If525313c63c4553abe8bea6f2bfaf75431ed18ea

Please note that a developers/operators that make the effort of fixing
this in master, should do also all the cherry-pickes back. We dont
have any automatic procudure for this.

thank you

Saverio

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Upstream LTS Releases

[Saverio Proto]

The 1 year release cycle makes a lot of sense to me too. +1

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [cinder] volume in-use attached to None?

[Saverio Proto]

Hello Christopher,

check out this:
https://ask.openstack.org/en/question/66918/how-to-delete-volume-with-available-status-and-attached-to/

Saverio

2017-10-16 20:45 GMT+02:00 Christopher Hull :
> Running Liberty.
> I'd like to be able to create new volumes from old ones.  Launching
> instances from volumes results in the volume being "root attached", and
> therefore, it seems, forever wed to the instance.  It can not be copied.
> So I tried deleting the instance.  Still no good.  The volume is now in-use
> by None.   So now the volume is completely useless.
> How do I force Cinder to detach from either a root mounted , or of all
> things, None??
>
> -Chris
>
>
>
>
> - Christopher T. Hull
>
> http://faq.chrishull.com
> Sunnyvale CA. 94085
> (415) 385 4865
> chrishul...@gmail.com
> http://chrishull.com
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Guest crash and KVM unhandled rdmsr

[Saverio Proto]

Hello Blair,

I found this link in my browser history:
https://bugs.launchpad.net/ubuntu/+source/kvm/+bug/1583819

Is it the same messages that you are seeing in Xenial ?

Saverio



2017-10-12 23:26 GMT+02:00 Blair Bethwaite :
> Hi all,
>
> Has anyone seen guest crashes/freezes associated with KVM unhandled rdmsr
> messages in dmesg on the hypervisor?
>
> We have seen these messages before but never with a strong correlation to
> guest problems. However over the past couple of weeks this is happening
> almost daily with consistent correlation for a set of hosts dedicated to a
> particular HPC workload. So far as I know the workload has not changed, but
> we have just recently moved the hypervisors to Ubuntu Xenial (though they
> were already on the Xenial kernel previously) and done minor guest (CentOS7)
> updates. CPU mode is host-passthrough. Currently trying to figure out if the
> CPU flags in the guest have changed since the host upgrade...
>
> Cheers,
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Cinder quota per volume types in the dashboard

[Saverio Proto]

So we had a bad feedback on the bug:
https://bugs.launchpad.net/horizon/+bug/1717342

Do we have anything pushed to Gerrit that we can at least use carrying
a local patch ?

thanks

Saverio


2017-09-26 20:21 GMT+02:00 Massimo Sgaravatto <massimo.sgarava...@gmail.com>:
> ok, I will
>
> 2017-09-26 14:43 GMT+02:00 Arne Wiebalck <arne.wieba...@cern.ch>:
>>
>> Massimo,
>>
>> Following Rob’s comment on
>> https://bugs.launchpad.net/horizon/+bug/1717342, would you
>> be willing to write up a blueprint? Mateusz would then prepare our code
>> and submit it to
>> gerrit as a partial implementation (as we only have the user facing part,
>> not the admin panel).
>>
>> Cheers,
>>  Arne
>>
>>
>> On 25 Sep 2017, at 10:46, Arne Wiebalck <arne.wieba...@cern.ch> wrote:
>>
>> Ah, nice, wasn’t aware. Mateusz is one of the Horizon experts here at CERN
>> I was referring to :)
>>
>> On 25 Sep 2017, at 10:41, Massimo Sgaravatto
>> <massimo.sgarava...@gmail.com> wrote:
>>
>> Just found that there is already this one:
>>
>> https://bugs.launchpad.net/horizon/+bug/1717342
>>
>> 2017-09-25 10:28 GMT+02:00 Saverio Proto <ziopr...@gmail.com>:
>>>
>>> Yes I am interested. Are you going to push them to gerrit ?
>>> Should we open a bug to track this change into Horizon ?
>>>
>>> massimo do you want to open the bug on Launchpad ? So if Arne pushed
>>> the patches on gerrit we can link them to the bug. I pointed
>>> robcresswell to this thread, he is reading us.
>>>
>>> thanks !
>>>
>>> Saverio
>>>
>>> 2017-09-25 10:13 GMT+02:00 Arne Wiebalck <arne.wieba...@cern.ch>:
>>> > Massimo, Saverio,
>>> >
>>> > We faced the same issue and have created patches for Horizon to display
>>> > - the per volume quota in the volume request panel, and also
>>> > - additional information about the volume type (like IOPS and
>>> > throughput limits, intended usage etc.)
>>> >
>>> > The patches will need some polishing before being sent upstream (I’ll
>>> > need
>>> > need to cross-check with our Horizon experts), but we use them in prod
>>> > since
>>> > quite a while and are happy to already share patch files if you’re
>>> > interested.
>>> >
>>> > Cheers,
>>> >  Arne
>>> >
>>> >
>>> >
>>> >> On 25 Sep 2017, at 09:58, Saverio Proto <ziopr...@gmail.com> wrote:
>>> >>
>>> >> I am pinging on IRC robcresswell from the Horizon project. He is still
>>> >> PTL I think.
>>> >> If you are on IRC please join #openstack-horizon.
>>> >>
>>> >> We should ask the Horizon PTL how to get this feature request into
>>> >> implementation.
>>> >>
>>> >> With the command line interface, can you already see the two different
>>> >> quotas for the two different volume types ? Can you paste an example
>>> >> output from the CLI ?
>>> >>
>>> >> thank you
>>> >>
>>> >> Saverio
>>> >>
>>> >>
>>> >> 2017-09-25 9:55 GMT+02:00 Massimo Sgaravatto
>>> >> <massimo.sgarava...@gmail.com>:
>>> >>> We are currently running Mitaka (preparing to update to Ocata). I see
>>> >>> the
>>> >>> same behavior on an Ocata based testbed
>>> >>>
>>> >>> Thanks, Massimo
>>> >>>
>>> >>> 2017-09-25 9:50 GMT+02:00 Saverio Proto <ziopr...@gmail.com>:
>>> >>>>
>>> >>>> Hello Massimo,
>>> >>>>
>>> >>>> what is your version of Openstack ??
>>> >>>>
>>> >>>> thank you
>>> >>>>
>>> >>>> Saverio
>>> >>>>
>>> >>>> 2017-09-25 9:13 GMT+02:00 Massimo Sgaravatto
>>> >>>> <massimo.sgarava...@gmail.com>:
>>> >>>>> Hi
>>> >>>>>
>>> >>>>>
>>> >>>>> In our OpenStack cloud we have two backends for Cinder (exposed
>>> >>>>> using
>>> >>>>> two
>>> >>>>> volume types), and we set different quotas for these two volume
>>> >>>>&

Re: [Openstack-operators] [openstack-dev] [nova] reset key pair during rebuilding

[Saverio Proto]

Hello,

I agree this feature of injecting a new keypair is something of great
use. We are always dealing with users that cant access their VMs
anymore.

But AFAIU here we are talking about injecting a new key at REBUILD. So
it does not fit the scenario of a staff member that leaves !

We hardly never use the rebuild feature in our workflow. Our users
just use create and delete.

I think it would be more useful a feature where you can reinject a new
keypair in the VM at any time. Ahhh it makes the users happy but of
course it is a security nightmare :D

Cheers

Saverio



2017-09-27 11:15 GMT+02:00 Marcus Furlong :
> On 27 September 2017 at 09:23, Michael Still  wrote:
>>
>> Operationally, why would I want to inject a new keypair? The scenario I can
>> think of is that there's data in that instance that I want, and I've lost
>> the keypair somehow. Unless that data is on an ephemeral, its gone if we do
>> a rebuild.
>
> This is quite a common scenario - staff member who started the
> instance leaves, and you want to access data on the instance, or
> maintain/debug the service running on the instance.
>
> Hitherto, I have used direct db calls to update the key, so it would
> be nice if there was an API call to do so.
>
> Cheers,
> Marcus.
> --
> Marcus Furlong
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [Openstack-operators] [skip-level-upgrades][fast-forward-upgrades] PTG summary

[Saverio Proto]

Hello,

sorry I could not make it to the PTG.

I have an idea that I want to share with the community. I hope this is a
good place to start the discussion.

After years of Openstack operations, upgrading releases from Icehouse to
Newton, the feeling is that the control plane upgrade is doable.

But it is also a lot of pain to upgrade all the compute nodes. This
really causes downtime to the VMs that are running.
I can't always make live migrations, sometimes the VMs are just too big
or too busy.

It would be nice to guarantee the ability to run an updated control
plane with compute nodes up to N-3 Release.

This way even if we have to upgrade the control plane every 6 months, we
can keep a longer lifetime for compute nodes. Basically we can never
upgrade them until we decommission the hardware.

If there are new features that require updated compute nodes, we can
always organize our datacenter in availability zones, not scheduling new
VMs to those compute nodes.

To my understanding this means having compatibility at least for the
nova-compute agent and the neutron-agents running on the compute node.

Is it a very bad idea ?

Do other people feel like me that upgrading all the compute nodes is
also a big part of the burden regarding the upgrade ?

thanks !

Saverio


On 28.09.17 17:38, arkady.kanev...@dell.com wrote:
> Erik,
> 
> Thanks for setting up a session for it.
> 
> Glad it is driven by Operators.
> 
> I will be happy to work with you on the session and run it with you.
> 
> Thanks,
> 
> Arkady
> 
>  
> 
> *From:*Erik McCormick [mailto:emccorm...@cirrusseven.com]
> *Sent:* Thursday, September 28, 2017 7:40 AM
> *To:* Lee Yarwood <lyarw...@redhat.com>
> *Cc:* OpenStack Development Mailing List
> <openstack-dev@lists.openstack.org>; openstack-operators
> <openstack-operat...@lists.openstack.org>
> *Subject:* Re: [openstack-dev] [Openstack-operators]
> [skip-level-upgrades][fast-forward-upgrades] PTG summary
> 
>  
> 
>  
> 
> On Sep 28, 2017 4:31 AM, "Lee Yarwood" <lyarw...@redhat.com
> <mailto:lyarw...@redhat.com>> wrote:
> 
> On 20-09-17 14:56:20, arkady.kanev...@dell.com
> <mailto:arkady.kanev...@dell.com> wrote:
> > Lee,
> > I can chair meeting in Sydney.
> > Thanks,
> > Arkady
> 
> Thanks Arkady!
> 
> FYI I see that emccormickva has created the following Forum session to
> discuss FF upgrades:
> 
> http://forumtopics.openstack.org/cfp/details/19
> 
> You might want to reach out to him to help craft the agenda for the
> session based on our discussions in Denver.
> 
> .
> 
> I just didn't want to risk it not getting in, and it was on our etherpad
> as well. I'm happy to help, but would love for you guys to lead.
> 
>  
> 
> Thanks,
> 
> Erik
> 
>  
> 
> 
> Thanks again,
> 
> Lee
> 
> --
> Lee Yarwood                 A5D1 9385 88CB 7E5F BE64  6618 BCA6 6E33
> F672 2D76
> 
> ___
> OpenStack-operators mailing list
> openstack-operat...@lists.openstack.org
> <mailto:openstack-operat...@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> 
>  
> 
> 
> 
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Cinder quota per volume types in the dashboard

[Saverio Proto]

I am pinging on IRC robcresswell from the Horizon project. He is still
PTL I think.
 If you are on IRC please join #openstack-horizon.

We should ask the Horizon PTL how to get this feature request into
implementation.

With the command line interface, can you already see the two different
quotas for the two different volume types ? Can you paste an example
output from the CLI ?

thank you

Saverio


2017-09-25 9:55 GMT+02:00 Massimo Sgaravatto <massimo.sgarava...@gmail.com>:
> We are currently running Mitaka (preparing to update to Ocata). I see the
> same behavior on an Ocata based testbed
>
> Thanks, Massimo
>
> 2017-09-25 9:50 GMT+02:00 Saverio Proto <ziopr...@gmail.com>:
>>
>> Hello Massimo,
>>
>> what is your version of Openstack ??
>>
>> thank you
>>
>> Saverio
>>
>> 2017-09-25 9:13 GMT+02:00 Massimo Sgaravatto
>> <massimo.sgarava...@gmail.com>:
>> > Hi
>> >
>> >
>> > In our OpenStack cloud we have two backends for Cinder (exposed using
>> > two
>> > volume types), and we set different quotas for these two volume types.
>> >
>> > The problem happens when a user, using the dashboard, tries to create a
>> > volume using a volume type for which the project quota is over:
>> >
>> > - the reported error message simply reports "unable to create volume",
>> > without mentioning that the problem is with quota
>> >
>> > - (by default) the dashboard only shows the overall Cinder quota (and
>> > not
>> > the quota per volume)
>> >
>> >
>> > Do you know if it possible in some to expose on the dashboard the cinder
>> > quota per volume type ?
>> >
>> >
>> > Thanks, Massimo
>> >
>> >
>> >
>> >
>> >
>> >
>> > ___
>> > OpenStack-operators mailing list
>> > OpenStack-operators@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> >
>
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Cinder quota per volume types in the dashboard

[Saverio Proto]

Hello Massimo,

what is your version of Openstack ??

thank you

Saverio

2017-09-25 9:13 GMT+02:00 Massimo Sgaravatto :
> Hi
>
>
> In our OpenStack cloud we have two backends for Cinder (exposed using two
> volume types), and we set different quotas for these two volume types.
>
> The problem happens when a user, using the dashboard, tries to create a
> volume using a volume type for which the project quota is over:
>
> - the reported error message simply reports "unable to create volume",
> without mentioning that the problem is with quota
>
> - (by default) the dashboard only shows the overall Cinder quota (and not
> the quota per volume)
>
>
> Do you know if it possible in some to expose on the dashboard the cinder
> quota per volume type ?
>
>
> Thanks, Massimo
>
>
>
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Is there any reason to exclude originally failed build hosts during live migration?

[Saverio Proto]

> The actual fix for this is trivial:
>
> https://review.openstack.org/#/c/505771/

Why the change is called:
Ignore original retried hosts when live migrating
?

Isn't it implementing the opposite ? Dont Ignore ?

thanks

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Instances not getting DHCP Lease

[Saverio Proto]

> checking http://169.254.169.254/2009-04-04/instance-id
> failed 1/20: up 188.93. request failed
> failed 2/20: up 191.21. request failed
> failed 3/20: up 193.36. request failed
> failed 4/20: up 195.54. request failed
> failed 5/20: up 197.68. request failed
> failed 6/20: up 199.83. request failed
> failed 7/20: up 201.97. request failed
> failed 8/20: up 204.13. request failed
> failed 9/20: up 206.31. request failed
> failed 10/20: up 208.48. request failed
> failed 11/20: up 210.63. request failed
> failed 12/20: up 212.89. request failed
> failed 13/20: up 215.06. request failed
> failed 14/20: up 217.21. request failed

Hello,

this is the metadata service failing, it is not DHCP. Does the
instance get an ip address from DHCP or not ?

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Instances not getting DHCP Lease

[Saverio Proto]

Hello,

using:

openstack console log show 

you can check if it is really the dhclient failing to have an address.

it is usually also good to have a look at the nova-compute.log file in
the compute node where the instance is scheduled.

Saverio



2017-08-31 19:55 GMT+02:00 Divneet Singh :
> I realize this is a common question , and I've looked through more than a
> couple of links , s, but I'm afraid I'm new enough to OpenStack that I
> wasn't able to make sense of many of the replies and was unable to solve my
> problem after searching.
> I have a multi-node deployment with one controller/network node and one
> compute node. When I create a cirros image on the Compute node, I see in the
> Horizon interface that a DHCP address is assigned, but when I run ifconfig
> in the VNC console on the Horizon dashboard, I see the instance has no IP
> address
>
> Can anyone suggest possible fixes? As I said, I'm pretty new to OpenStack.
> I'm sure that I haven't provided all of the information needed to solve this
> problem; if someone could tell me what further information would be useful.
>
> Thanks !
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [l2gw] How to handle correctly unknown-dst traffic

[Saverio Proto]

> The ucast-mac-remote table is filled with information that don't match
> your comments. In my environment, I have created only one neutron
> network, one l2gw instance and one l2gw connection. However, the mac
> reflected in that table corresponds to the dhcp port of the Neutron
> network (I've checked the mac on the dhcp namespace and it's the same).
> I've created several VMs in different compute nodes and there is only
> one line there. Could you check again the MAC address?

Hello,

I confirm I have one line per VM

root@l2gw-0:/# vtep-ctl list-remote-macs
ee87db33-1b3a-42e9-bc09-02747f8a0ad5
ucast-mac-remote
  fa:16:3e:6b:6e:d1 -> vxlan_over_ipv4/10.1.1.161
  fa:16:3e:7f:18:77 -> vxlan_over_ipv4/10.1.0.126
  fa:16:3e:90:7f:f9 -> vxlan_over_ipv4/10.1.1.177
  fa:16:3e:c2:7b:da -> vxlan_over_ipv4/10.1.1.167
  fa:16:3e:ca:ad:c6 -> vxlan_over_ipv4/10.1.0.126
  fa:16:3e:f0:21:01 -> vxlan_over_ipv4/10.1.1.175

I have the indication of the mac address of the VM, and the IP address
of the hypervisor where it is hosted.


> In my setup I get this automatically:
> 
> mcast-mac-remote
>   unknown-dst -> vxlan_over_ipv4/192.0.2.6 
> 
> If you're using the agent, it might be a bug.

I am running Openstack Newton, on what version is based your PoC ?

thank you

Saverio


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] possible race condition with nova instance and neutron ports

[Saverio Proto]

Thank you.
It worked with the new settings. Now the instances are correctly in
ERROR state of network is not functional.

To solve the performance problem and have 0 instances in ERROR state I
had to enable the neutron-rootwrap-daemon. Without it the dhcp agents
were not fast enough to consume the rabbit queues.

thanks

Saverio


On 09/08/17 14:40, Sławomir Kapłoński wrote:
> With such settings nova is not waiting for info from neutron and that is why 
> Your instances starting without network ready.
> If You will change this timeout to some value higher than 0 then instance 
> will be paused and nova will wait for info from neutron that port is active 
> (You should also check credentials config in neutron server)
> If You will set also vif_plugging_is_fatal=True then nova will put instance 
> in ERROR state if port will not be active after timeout time.
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] possible race condition with nova instance and neutron ports

[Saverio Proto]

Hello,

thanks for the tip.
I checked on a compute node, in the nova.conf file I have the following:

vif_plugging_is_fatal=False
vif_plugging_timeout=0

both options are in the [DEFAULT] section.
I guess these settings were never changed since we were running Icehouse.

Should I change to the Ocata default values ? (True and 300)

I will try that today.
Thank you

Saverio


On 09/08/17 09:23, Sławomir Kapłoński wrote:
> Hello,
> 
> Do You have configured in nova-compute: vif_plugging_timeout and 
> vif_plugging_is_fatal options?
> With those options nova should pause VM until port will be set to ACTIVE in 
> Neutron.
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] possible race condition with nova instance and neutron ports

[Saverio Proto]

Hello,

I see this in Openstack Newton

I start 200 instances with a oneliner.

openstack server create \
--image "Ubuntu Xenial 16.04 (SWITCHengines)" \
--flavor c1.small \
--network demonetwork \
--user-data cloud-init.txt \
--key-name macsp \
--min 200 \
--max 200 test

When I do this I see a problem where all the instances boot and are in
Running state, before all neutron ports are ACTIVE.

I see with `openstack port list` neutron ports still in BUILD state. It
takes a longer time to make them all ACTIVE, the instances that use
those ports boot up much faster.

In rabbitmq queues I see growing the dhcp_agent. queues.

At the very end all neutron ports will be ACTIVE and rabbitmq queues
back to normal. But many VMs failed getting an address via DHCP because
at the time they were trying the dnsmasq process did not have a
corresponding entry for the instance in the `host` file. Unfortunately
the instance gives up trying obtaining an address via DHCP after 5 minutes.

When I start 200 instances I always end up with 15 to 30 instances
without IP address. I check carefully using cloud-init: I make them
phone-home to check if they are alive.

Should I open a bug for this ?? It looks like a race condition where
nova boots the instance before the neutron port is really ready.

thank you for your feedback.

Saverio





-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Experience with Cinder volumes as root disks?

[Saverio Proto]

Hello Conrad,

I jump late on the conversation because I was away from the mailing
lists last week.

We run Openstack with both nova ephemeral root disks and cinder volume
boot disks. Both are with ceph rbd backend. It is the user that flags
"boot from volume" in Horizon when starting an instance.

Everything works in both cases, and there are pros and cons as you see
from the many answers you had.

But, if I have to give you a suggestion, I would choose only 1 way to
go and stick with it.

Having all this flexibility is great for us, operators that understand
Openstack internals. But it is a nightmare for the openstack users.

First of all looking at Horizon it is very difficult to understand the
kind of root volume being used.
It is also difficult to understand that you have a snapshot of the
nova instance, and a snapshot of the cinder volume.
We have different snapshot procedures depending on the type for root
disk, and users always get confused.
When the root disk is cinder, if you snapshot from the instance page,
you will get a 0 byte glance image connected to a cinder snapshot.

A user that has a instance with a disk, should not have to understand
if the disk is managed by nova or cinder to finally backup his data
with a snapshot.
Looking at Cloud usability, I would say that mixing the two solutions
is not the best. Probably this explains the Amazon and Azure choices
you described earlier.

Cheers,

Saverio



2017-08-01 16:50 GMT+02:00 Kimball, Conrad :
> In our process of standing up an OpenStack internal cloud we are facing the
> question of ephemeral storage vs. Cinder volumes for instance root disks.
>
>
>
> As I look at public clouds such as AWS and Azure, the norm is to use
> persistent volumes for the root disk.  AWS started out with images booting
> onto ephemeral disk, but soon after they released Elastic Block Storage and
> ever since the clear trend has been to EBS-backed instances, and now when I
> look at their quick-start list of 33 AMIs, all of them are EBS-backed.  And
> I’m not even sure one can have anything except persistent root disks in
> Azure VMs.
>
>
>
> Based on this and a number of other factors I think we want our user normal
> / default behavior to boot onto Cinder-backed volumes instead of onto
> ephemeral storage.  But then I look at OpenStack and its design point
> appears to be booting images onto ephemeral storage, and while it is
> possible to boot an image onto a new volume this is clumsy (haven’t found a
> way to make this the default behavior) and we are experiencing performance
> problems (that admittedly we have not yet run to ground).
>
>
>
> So …
>
> · Are other operators routinely booting onto Cinder volumes instead
> of ephemeral storage?
>
> · What has been your experience with this; any advice?
>
>
>
> Conrad Kimball
>
> Associate Technical Fellow
>
> Chief Architect, Enterprise Cloud Services
>
> Application Infrastructure Services / Global IT Infrastructure / Information
> Technology & Data Analytics
>
> conrad.kimb...@boeing.com
>
> P.O. Box 3707, Mail Code 7M-TE
>
> Seattle, WA  98124-2207
>
> Bellevue 33-11 bldg, office 3A6-3.9
>
> Mobile:  425-591-7802
>
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] UDP Buffer Filling

[Saverio Proto]

It is merged in Mitaka but your glance images must be decorated with:

hw_vif_multiqueue_enabled='true'

when you do "openstack image show uuid"

in the property you should see this, and then you will have multiqueue

Saverio



2017-07-28 14:50 GMT+02:00 John Petrini <jpetr...@coredial.com>:

> Hi Saverio,
>
> Thanks for the info. The parameter is missing completely:
>
> 
>   
>   
>   
>   
>function='0x0'/>
> 
>
> I've came across the blueprint for adding the image property
> hw_vif_multiqueue_enabled. Do you know if this feature is available in
> Mitaka?
>
> John Petrini
>
> Platforms Engineer   //   *CoreDial, LLC*   //   coredial.com   //   [image:
> Twitter] <https://twitter.com/coredial>   [image: LinkedIn]
> <http://www.linkedin.com/company/99631>   [image: Google Plus]
> <https://plus.google.com/104062177220750809525/posts>   [image: Blog]
> <http://success.coredial.com/blog>
> 751 Arbor Way, Hillcrest I, Suite 150, Blue Bell, PA 19422
> *P:* 215.297.4400 x232 <(215)%20297-4400>   //   *F: *215.297.4401
> <(215)%20297-4401>   //   *E: *jpetr...@coredial.com
>
> On Fri, Jul 28, 2017 at 3:59 AM, Saverio Proto <ziopr...@gmail.com> wrote:
>
>> Hello John,
>>
>> a common problem is packets being dropped when they pass from the
>> hypervisor to the instance. There is bottleneck there.
>>
>> check the 'virsh dumpxml' of one of the instances that is dropping
>> packets. Check for the interface section, should look like:
>>
>> 
>>   
>>   
>>   
>>   
>>   
>>   
>>   > function='0x0'/>
>> 
>>
>> how many queues you have ??? Usually if you have only 1 or if the
>> parameter is missing completely is not good.
>>
>> in Mitaka nova should use 1 queue for every instance CPU core you
>> have. It is worth to check if this is set correctly in your setup.
>>
>> Cheers,
>>
>> Saverio
>>
>>
>>
>> 2017-07-27 17:49 GMT+02:00 John Petrini <jpetr...@coredial.com>:
>> > Hi List,
>> >
>> > We are running Mitaka with VLAN provider networking. We've recently
>> > encountered a problem where the UDP receive queue on instances is
>> filling up
>> > and we begin dropping packets. Moving instances out of OpenStack onto
>> bare
>> > metal resolves the issue completely.
>> >
>> > These instances are running asterisk which should be pulling these
>> packets
>> > off the queue but it appears to be falling behind no matter the
>> resources we
>> > give it.
>> >
>> > We can't seem to pin down a reason why we would see this behavior in
>> KVM but
>> > not on metal. I'm hoping someone on the list might have some insight or
>> > ideas.
>> >
>> > Thank You,
>> >
>> > John
>> >
>> > ___
>> > OpenStack-operators mailing list
>> > OpenStack-operators@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> >
>>
>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] UDP Buffer Filling

[Saverio Proto]

Hello John,

a common problem is packets being dropped when they pass from the
hypervisor to the instance. There is bottleneck there.

check the 'virsh dumpxml' of one of the instances that is dropping
packets. Check for the interface section, should look like:


  
  
  
  
  
  
  


how many queues you have ??? Usually if you have only 1 or if the
parameter is missing completely is not good.

in Mitaka nova should use 1 queue for every instance CPU core you
have. It is worth to check if this is set correctly in your setup.

Cheers,

Saverio



2017-07-27 17:49 GMT+02:00 John Petrini :
> Hi List,
>
> We are running Mitaka with VLAN provider networking. We've recently
> encountered a problem where the UDP receive queue on instances is filling up
> and we begin dropping packets. Moving instances out of OpenStack onto bare
> metal resolves the issue completely.
>
> These instances are running asterisk which should be pulling these packets
> off the queue but it appears to be falling behind no matter the resources we
> give it.
>
> We can't seem to pin down a reason why we would see this behavior in KVM but
> not on metal. I'm hoping someone on the list might have some insight or
> ideas.
>
> Thank You,
>
> John
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [masakari][nova] Allow evacuation of instances in resized state

[Saverio Proto]

Hello,

as far as I know from the Openstack survey there is a very small
number of production environments with more than 100 servers running
already Ocata.

you probably will have to find your self that information and report
back to the list.

thank you

Saverio


2017-07-10 10:27 GMT+02:00 Kekane, Abhishek <abhishek.kek...@nttdata.com>:
> Hi Saverio,
>
> Thank you for reply.
>
> Currently we are using Ocata release for Openstack.
>
> Please let me know if you get any update.
>
> Thank you,
>
> Abhishek
>
> -----Original Message-
> From: Saverio Proto [mailto:ziopr...@gmail.com]
> Sent: Monday, July 10, 2017 12:52 PM
> To: Kekane, Abhishek
> Cc: OpenStack Development Mailing List (not for usage questions); 
> openstack-operators@lists.openstack.org
> Subject: Re: [Openstack-operators] [openstack-dev] [masakari][nova] Allow 
> evacuation of instances in resized state
>
> Hello Abhishek,
>
> I am sorry I dont have an answer for your question. I would have to try my 
> self everything to give answer because I never experienced this use case you 
> describe.
>
> I would suggest also to specify what version of Openstack you are working 
> with. Because the behaviour can change a lot in different versions.
>
> Cheers,
>
> Saverio
>
>
> 2017-07-10 9:00 GMT+02:00 Kekane, Abhishek <abhishek.kek...@nttdata.com>:
>> Hi Operators,
>>
>> Could you please let me know your opinion on below scenario? It will help me 
>> to proceed my work.
>>
>> Thank you,
>>
>> Abhishek
>>
>> -Original Message-
>> From: Kekane, Abhishek [mailto:abhishek.kek...@nttdata.com]
>> Sent: Tuesday, July 04, 2017 2:52 PM
>> To: OpenStack Development Mailing List (not for usage questions);
>> openstack-operators@lists.openstack.org
>> Subject: Re: [Openstack-operators] [openstack-dev] [masakari][nova]
>> Allow evacuation of instances in resized state
>>
>> Hi operators,
>>
>> I want to know how evacuation of resized instances is handled in real 
>> environment.
>> For example if the vm is in resized state and if the compute host on which 
>> the vm is resized goes down, then how will operator evacuate the vm.
>>
>> One possible way is to reset that vm state to error and then evacuate it to 
>> new compute host.
>> Please refer below scenario for reference:
>>
>> Scenario:
>> =
>>
>> Pre-conditions:
>> 
>> 1. Config option allow_resize_to_same_host is False.
>> 2. Instance path is not mounted on shared storage.
>> 3. Three compute nodes: "compute node A", "compute node B" and "compute node 
>> C"
>>
>> Steps:
>> 
>> 1. Boot an instance on "compute node A".
>> 2. User tries to resize the newly created instance and nova-scheduler 
>> selects "compute node B" as a destination node for resize.
>>In this case nova creates a instance directory on destination "compute 
>> node B" and mark the instance directory which is present on the source 
>> "compute node A" as "*_resize".
>>
>> Note that the resize operation is yet not confirmed and "compute node B" 
>> goes down.
>>
>> 3. Reset instance state to ERROR as nova allows evacuation only if instance 
>> state is 'ACTIVE', 'STOPPED' or 'ERROR'.
>> 4. Evacuate the instance to "compute node C" using target_host option.
>>As a result, instance files which were on "compute node B" will be 
>> cleaned up after compute service on it is up again, but instance files which 
>> were on "compute node A" marked as "*_resize" will never be cleaned up. As 
>> of now there is no periodic task in nova to perform cleanup of these kinds 
>> of scenarios.
>>
>>
>> Questions:
>> 1. is this the only possible way of evacuating the resized instances in real 
>> world scenario?
>> 2. If yes is there any way to cleanup unused (*_resize) instance files from 
>> the source compute node other than cleaning up it manually?
>> 3. Should we add support of evacuating of resized instances in nova?
>>
>> Please let me know your opinions about the same.
>>
>>
>> Thank you,
>>
>> Abhishek Kekane
>>
>>
>>
>> -Original Message-
>> From: Kekane, Abhishek [mailto:abhishek.kek...@nttdata.com]
>> Sent: Thursday, June 29, 2017 5:57 PM
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] [masakari]

Re: [Openstack-operators] [openstack-dev] [masakari][nova] Allow evacuation of instances in resized state

[Saverio Proto]

Hello Abhishek,

I am sorry I dont have an answer for your question. I would have to
try my self everything to give answer because I never experienced this
use case you describe.

I would suggest also to specify what version of Openstack you are
working with. Because the behaviour can change a lot in different
versions.

Cheers,

Saverio


2017-07-10 9:00 GMT+02:00 Kekane, Abhishek :
> Hi Operators,
>
> Could you please let me know your opinion on below scenario? It will help me 
> to proceed my work.
>
> Thank you,
>
> Abhishek
>
> -Original Message-
> From: Kekane, Abhishek [mailto:abhishek.kek...@nttdata.com]
> Sent: Tuesday, July 04, 2017 2:52 PM
> To: OpenStack Development Mailing List (not for usage questions); 
> openstack-operators@lists.openstack.org
> Subject: Re: [Openstack-operators] [openstack-dev] [masakari][nova] Allow 
> evacuation of instances in resized state
>
> Hi operators,
>
> I want to know how evacuation of resized instances is handled in real 
> environment.
> For example if the vm is in resized state and if the compute host on which 
> the vm is resized goes down, then how will operator evacuate the vm.
>
> One possible way is to reset that vm state to error and then evacuate it to 
> new compute host.
> Please refer below scenario for reference:
>
> Scenario:
> =
>
> Pre-conditions:
> 
> 1. Config option allow_resize_to_same_host is False.
> 2. Instance path is not mounted on shared storage.
> 3. Three compute nodes: "compute node A", "compute node B" and "compute node 
> C"
>
> Steps:
> 
> 1. Boot an instance on "compute node A".
> 2. User tries to resize the newly created instance and nova-scheduler selects 
> "compute node B" as a destination node for resize.
>In this case nova creates a instance directory on destination "compute 
> node B" and mark the instance directory which is present on the source 
> "compute node A" as "*_resize".
>
> Note that the resize operation is yet not confirmed and "compute node B" goes 
> down.
>
> 3. Reset instance state to ERROR as nova allows evacuation only if instance 
> state is 'ACTIVE', 'STOPPED' or 'ERROR'.
> 4. Evacuate the instance to "compute node C" using target_host option.
>As a result, instance files which were on "compute node B" will be cleaned 
> up after compute service on it is up again, but instance files which were on 
> "compute node A" marked as "*_resize" will never be cleaned up. As of now 
> there is no periodic task in nova to perform cleanup of these kinds of 
> scenarios.
>
>
> Questions:
> 1. is this the only possible way of evacuating the resized instances in real 
> world scenario?
> 2. If yes is there any way to cleanup unused (*_resize) instance files from 
> the source compute node other than cleaning up it manually?
> 3. Should we add support of evacuating of resized instances in nova?
>
> Please let me know your opinions about the same.
>
>
> Thank you,
>
> Abhishek Kekane
>
>
>
> -Original Message-
> From: Kekane, Abhishek [mailto:abhishek.kek...@nttdata.com]
> Sent: Thursday, June 29, 2017 5:57 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [masakari][nova] Allow evacuation of instances 
> in resized state
>
> Hi Chris,
>
> IMO we cannot perform auto-confirm as confirming or reverting is user's 
> choice, whereas reverting is not possible as the node where the instance is 
> resized is down.
> As suggested by you allowing this in nova require additional work. It is 
> possible if we take power-state into consideration for evacuation operation, 
> i.e. while evacuation if instance vmstate is resized and power-state is 
> shutoff then we can stop that instance after evacuation.
>
> Thank you,
>
> Abhishek Kekane
>
> -Original Message-
> From: Chris Friesen [mailto:chris.frie...@windriver.com]
> Sent: Wednesday, June 28, 2017 8:54 PM
> To: openstack-...@lists.openstack.org
> Subject: Re: [openstack-dev] [masakari][nova] Allow evacuation of instances 
> in resized state
>
> On 06/28/2017 05:50 AM, Kekane, Abhishek wrote:
>
>> In masakari, we are setting an instance to an error state if the
>> vmstate is resized before evacuating it to a new host.
>
> Arguably the instance should be set to an error state as soon as you notice 
> that the compute node is down.
>
>> Once an instance (which was in
>> resized state) is evacuated then it becomes active on the new host.
>> The main problem with this implementation from user’s point of view is
>> the instance goes into active state after evacuation, it should be in
>> stopped state if the prior action on the instance before resizing was
>> stop. In masakari, It’s possible to set the vm state to stopped state
>> after evacuation but for a short period the instance will go into the active 
>> state which is unacceptable.
>
> That's a valid point, I think.
>
>> *Proposing changes to Nova:*
>>
>> In the current nova code, 

Re: [Openstack-operators] OpenStack User Survey Now Open

[Saverio Proto]

Allison I clicked on "Add Deployment" and I got a 404 page (with a cat)

The URL I was redirected to is:
https://www.openstack.org/%7B$Controller.Link%7D%7B$CurrentStep.Template.Title%7D/add-entity

Saverio

2017-06-26 23:44 GMT+02:00 Allison Price :
> Hi everyone,
>
> If you’re running OpenStack, please participate in the OpenStack User
> Survey. If you have already completed the survey before, you can simply
> login to update your deployment details. Please note that if your survey
> response has not been updated in 12 months, it will expire, so we encourage
> you to take this time to update your existing profile so your deployment can
> be included in the upcoming analysis.
>
> As a member of our community, please help us spread the word. We're trying
> to gather as much real-world deployment data as possible to share back with
> you. We have made it easier to complete, and the survey is now available in
> 7 languages—English, German, Indonesian, Japanese, Korean, traditional
> Chinese and simplified Chinese.
>
> The information provided is confidential and will only be presented in
> aggregate unless you consent to making it public.
>
> The deadline to complete the survey and be part of the next report is
> Friday, August 11 at 23:59 UTC.
>
> You can login and complete the OpenStack User Survey here:
> http://www.openstack.org/user-survey
> If you’re interested in joining the OpenStack User Survey Working Group to
> help with the survey analysis, please complete this form:
> https://openstackfoundation.formstack.com/forms/user_survey_working_group
> Help us promote the User Survey:
> https://twitter.com/OpenStack/status/879434563134652416
>
>
> Please let me know if you have any questions.
>
> Cheers,
> Allison
>
>
> Allison Price
> OpenStack Foundation
> alli...@openstack.org
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ocata instance snapshot

[Saverio Proto]

Hello Ignazio,

do you mean the instance has booted from volume ?

when is booted from volume this 0 bytes glance image is created
together with a cinder snapshot.

I wrote about it for my openstack users here:
https://help.switch.ch/engines/documentation/backup-with-snapshots/

I think that is by design. Also my opinion is that the current design
with booting from volumes or images and different snapshotting paths
is very confusing for the users. That is why I had to write that page.

2017-06-23 10:29 GMT+02:00 Ignazio Cassano :
> I created the instance from an image and the instance has an attached volume
> (it is NOT ephemeral).
> I checked that in this case it makes a volume snapshot (probably because the
> istance is attached to a volume) but also an empty  image (why ?)
> In the case the instance is ethemeral (no volumes attached) , a not empty
> image is created.
> Regards
> Ignazio
>
> 2017-06-23 10:10 GMT+02:00 Ahmed Mostafa :
>>
>> What do you mean by the instance has an attached volume ?
>>
>> Have you started the instance from a volume ?
>>
>> On Fri, Jun 23, 2017 at 8:06 AM, Ignazio Cassano
>>  wrote:
>>>
>>> Hi All,
>>> using ocata dashboard I am trying to create instance snapshots.
>>> If the instance has an attached volume, the image created is empty.
>>> If the instance does not have an attached volume, the image created is
>>> not empty.
>>> This is by design or I must change my configuration ?
>>>
>>> Regards
>>> Ignazio
>>>
>>> ___
>>> OpenStack-operators mailing list
>>> OpenStack-operators@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>
>>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [l2gw] How to handle correctly unknown-dst traffic

[Saverio Proto]

Hello,

I try again. Any l2gw plugin user that wants to comment on my email ?

thank you

Saverio


On 29/05/17 16:54, Saverio Proto wrote:
> Hello,
> 
> I have a question about the l2gw. I did a deployment, I described the
> steps here:
> https://review.openstack.org/#/c/453209/
> 
> The unicast traffic works fine, but I dont understand what is the idea
> behind the handling of the broadcast traffic.
> 
> Looking at openvswitch:
> 
> I obtain the uuid with `vtep-ctl list-ls`
> 
> vtep-ctl list-remote-macs 
> 
> In this output I get an entry for each VM that has an interface in the
> L2 network I am bridging:
> 
> 
> # vtep-ctl list-remote-macs 
> ucast-mac-remote
>   fa:16:3e:c2:7b:da -> vxlan_over_ipv4/10.1.1.167
> 
> mcast-mac-remote
> -
> 
> The ucast-mac-remote entry is created by Openstack when I start a VM.
> (Also it is never removed when I delete the instance, is this a bug ? )
> Note that 10.1.1.167 is the IP address of the hypervisor where the VM is
> running.
> 
> But mcast-mac-remote is empty. So this means that ARP learning for
> example works only in 1 way. The VM in openstack does not receive any
> broadcast traffic, unless I do manually:
> 
> vtep-ctl add-mcast-remote ee87db33-1b3a-42e9-bc09-02747f8a0ad5
> unknown-dst  10.1.1.167
> 
> This creates an entry in the table mcast-mac-remote and everything works
> correctly.
> 
> 
> Now I read here http://networkop.co.uk/blog/2016/05/21/neutron-l2gw/
> about sending add-mcast-remote to the network nodes and then doing some
> magic I dont really understand. But I am confused because in my setup
> the tenant does not have a L3 router, so there is not a qrouter
> namespace for this network, I was planning to keep the network node out
> of the game.
> 
> Is anyone running this in production and can shed some light ?
> 
> thanks
> 
> Saverio
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Neutron not clearing up deleted routers

[Saverio Proto]

I patched this back in liberty.
What version of Openstack are you using ?

Saverio


2017-06-08 19:36 GMT+02:00 Grant Morley :

> Ignore that now all,
>
> Managed to fix it by restarting the l3-agent. Looks like it must have been
> cached in memory.
>
> Thanks,
>
> On 08/06/17 18:07, Grant Morley wrote:
>
> Hi All,
>
> We have  noticed in our neutron-l3-agent logs that there are a number of
> routers that neutron seems to think exist ( but they no longer exist in the
> data base )  and it is constantly trying to delete them. However because
> they don't appear to exist,  so neutron seems to get stuck in a loop of
> trying to delete  routers that are no longer there.
>
> We originally noticed that neutron was trying to delete the
> "qrouter--xxx-" and that didn't exist. So we added the router
> manually with "ip netns add qrouter-xxx-xxx-xxx-xxx" which then deletes the
> router however we then get the following error messages:
>
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent [-] Error while
> deleting router da7b633a-233b-46d1-ba3d-315b3bee6a61
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent Traceback (most
> recent call last):
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/agent.py", line 359, in _safe_router_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self._router_removed(router_id)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/agent.py", line 377, in _router_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> ri.delete(self)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/ha_router.py", line 380, in delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> super(HaRouter, self).delete(agent)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 361, in delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self.process_delete(agent)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/common/utils.py", line 385, in call
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent self.logger(e)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/oslo_utils/excutils.py", line 220, in __exit__
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self.force_reraise()
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/oslo_utils/excutils.py", line 196, in force_reraise
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> six.reraise(self.type_, self.value, self.tb)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/common/utils.py", line 382, in call
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent return
> func(*args, **kwargs)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 972, in process_delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self._process_external_on_delete(agent)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 794, in
> _process_external_on_delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self._process_external_gateway(ex_gw_port, agent.pd)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 693, in
> _process_external_gateway
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self.external_gateway_removed(self.ex_gw_port, interface_name)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/ha_router.py", line 371, in
> external_gateway_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> interface_name)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 668, in
> external_gateway_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> ip_addr['prefixlen']))
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> 

[openstack-dev] [l2gw] How to handle correctly unknown-dst traffic

[Saverio Proto]

Hello,

I have a question about the l2gw. I did a deployment, I described the
steps here:
https://review.openstack.org/#/c/453209/

The unicast traffic works fine, but I dont understand what is the idea
behind the handling of the broadcast traffic.

Looking at openvswitch:

I obtain the uuid with `vtep-ctl list-ls`

vtep-ctl list-remote-macs 

In this output I get an entry for each VM that has an interface in the
L2 network I am bridging:


# vtep-ctl list-remote-macs 
ucast-mac-remote
  fa:16:3e:c2:7b:da -> vxlan_over_ipv4/10.1.1.167

mcast-mac-remote
-

The ucast-mac-remote entry is created by Openstack when I start a VM.
(Also it is never removed when I delete the instance, is this a bug ? )
Note that 10.1.1.167 is the IP address of the hypervisor where the VM is
running.

But mcast-mac-remote is empty. So this means that ARP learning for
example works only in 1 way. The VM in openstack does not receive any
broadcast traffic, unless I do manually:

vtep-ctl add-mcast-remote ee87db33-1b3a-42e9-bc09-02747f8a0ad5
unknown-dst  10.1.1.167

This creates an entry in the table mcast-mac-remote and everything works
correctly.


Now I read here http://networkop.co.uk/blog/2016/05/21/neutron-l2gw/
about sending add-mcast-remote to the network nodes and then doing some
magic I dont really understand. But I am confused because in my setup
the tenant does not have a L3 router, so there is not a qrouter
namespace for this network, I was planning to keep the network node out
of the game.

Is anyone running this in production and can shed some light ?

thanks

Saverio











-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Routed provider networks...

[Saverio Proto]

> We use provider networks to essentially take neutron-l3 out of the equation. 
> Generally they are shared on all compute hosts, but usually there aren't huge 
> numbers of computes.

Hello,

we have a datacenter completely L3, routing to the host.

to implement the provider networks we are using the neutron-l2gw plugin.

basically there is a Switch where we bridge a physical port attached
to the provider network, with a virtual VXLAN tenant network. The
config of the switch is managed by neutron using the l2gw-plugin.

Cheers,

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Neutron Issues

[Saverio Proto]

is the instance scheduled to an hypervisor ? check this with openstack
server show uuid
(admin credential)

if yes check nova-compute.log on the hypervisor. maybe you find some good
information to debug

Saverio

Il 03 mag 2017 2:16 AM, "Steve Powell"  ha
scritto:

[ml2]

type_drivers = flat,vlan,vxlan

tenant_network_types = vxlan

mechanism_drivers = linuxbridge,l2population

extension_drivers = port_security



[ml2_type_flat]

flat_networks = provider









*From:* Neil Jerram [mailto:n...@tigera.io]
*Sent:* Tuesday, May 2, 2017 7:58 PM
*To:* Steve Powell ; Chris Sarginson <
csarg...@gmail.com>; openstack-operators@lists.openstack.org

*Subject:* Re: [Openstack-operators] Neutron Issues



I think you probably need to say what ML2 mechanism driver(s) you are using.





On Tue, May 2, 2017 at 10:29 PM Steve Powell 
wrote:

No OVS here. Thanks though! This one has me stumped!



*From:* Chris Sarginson [mailto:csarg...@gmail.com]
*Sent:* Tuesday, May 2, 2017 5:24 PM
*To:* Steve Powell ; openstack-operators@lists.
openstack.org
*Subject:* Re: [Openstack-operators] Neutron Issues



If you're using openvswitch, with Newton there was a change to the default
agent for configuring openvswitch to be the python ryu library, I think
it's been mentioned on here recently, so probably worth having a poke
through the archives for more information.  I'd check your neutron
openvswitch agent logs for errors pertaining to openflow configuration
specifically, and if you see anything, it's probably worth applying the
following config to your ml2 ini file under the [OVS] section:



of_interface = ovs-ofctl



https://docs.openstack.org/mitaka/config-reference/
networking/networking_options_reference.html



Then restart the neutron openvswitch agent, watch the logs, hopefully this
is of some use to you.



On Tue, 2 May 2017 at 21:30 Steve Powell  wrote:

I forgot to mention I’m running Newton and my neutron.conf file is below
and I’m running haproxy.



[DEFAULT]

core_plugin = ml2

service_plugins = router

allow_overlapping_ips = True

notify_nova_on_port_status_changes = True

notify_nova_on_port_data_changes = True

transport_url = rabbit://openstack:#@x.x.x.x

auth_strategy = keystone



[agent]

root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf



[cors]



[cors.subdomain]



[database]

connection = mysql+pymysql://neutron:###@
10.10.6.220/neutron



[keystone_authtoken]

auth_url = http://x.x.x.x:35357/v3

auth_uri = https://xxx..xxx:5000/v3

memcached_servers = x.x.x.x:11211

auth_type = password

project_domain_name = Default

user_domain_name = Default

project_name = service

username = neutron

password = ##





[matchmaker_redis]



[nova]



auth_url = http://x.x.x.x:35357/v3

auth_type = password

project_domain_name = Default

user_domain_name = Default

region_name = RegionOne

project_name = service

username = nova

password = ###



[oslo_concurrency]



[oslo_messaging_amqp]



[oslo_messaging_notifications]



[oslo_messaging_rabbit]



[oslo_messaging_zmq]



[oslo_middleware]

enable_proxy_headers_parsing = True

enable_http_proxy_to_wsgi = True



[oslo_policy]



[qos]



[quotas]



[ssl]



*From:* Steve Powell [mailto:spow...@silotechgroup.com]
*Sent:* Tuesday, May 2, 2017 4:16 PM
*To:* openstack-operators@lists.openstack.org
*Subject:* [Openstack-operators] Neutron Issues



This sender failed our fraud detection checks and may not
be who they appear to be. Learn about spoofing


Feedback 

Hello Ops!



I have a major issue slapping me in the face and seek any assistance
possible. When trying to spin up and instance whether from the command
line, manually in Horizon, or with a HEAT template I receive the following
error in nova and, where applicable, heat logs:



Failed to allocate the network(s), not rescheduling.



I see in the neutron logs where the request make it through to completion
but that info is obviously not making it back to nova.



INFO neutron.notifiers.nova [-] Nova event response: {u'status':
u'completed', u'code': 200, u'name': u'network-changed', u'server_uuid':
u'6892bb9e-4256-4fc9-a313-331f0c576a03'}



What am I missing? Why would the response from neutron not make it back to
nova?







___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Mitaka - Heat] HTTPInternalServerError: ERROR: Remote error: BadRequest Expecting to find domain in user

[Saverio Proto]

Edgar,

as admin, and using V3, do the following command:

openstack user show heat

in the field domain_id do you see default or Default with capital D ?

in the heat config you pasted you have capital D, is that correct ?

ciao,

Saverio



2017-05-02 21:47 GMT+02:00 Edgar Magana <edgar.mag...@workday.com>:
> Hi,
>
> Yes, all that is in place. This is why #openstack stack list works.
> The openstack client is:
>
> # yum list installed |grep client
> python-openstackclient.noarch  3.2.0-2.el7  @3rdParty7
> python2-heatclient.noarch  1.5.0-1.el7  @3rdParty7
>
> Edgar
>
> On 5/2/17, 12:39 PM, "Saverio Proto" <ziopr...@gmail.com> wrote:
>
> Hello Edgar,
>
> what is the version of the openstack client ?
>
> did you export this?
>
> export OS_IDENTITY_API_VERSION=3
>
> also the auth URL should have the v3, like;
>
> export 
> OS_AUTH_URL=https://urldefense.proofpoint.com/v2/url?u=https-3A__hostname-3A5000_v3=DwIBaQ=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ=bwlIMYJQM6k26RrwtU0-iHi3-63kP3vUZLLHvFNSWZ0=aS1zOIR-pRfqhzkrTpilZrUBGwKku7fn_O5OjaYOV3g=
>
> check the client configuration and give us feedback,
>
> ciao,
>
> Saverio
>
> 2017-05-02 21:15 GMT+02:00 Edgar Magana <edgar.mag...@workday.com>:
> > Hello Ops,
> >
> >
> >
> > In our Mitaka Deployment with SSL and Keystone v3, we are having the 
> below
> > errors while trying to create a stack but they do not show while listing
> > them:
> >
> >
> >
> > # openstack stack list
> >
> > WARNING: openstackclient.common.utils is deprecated and will be removed
> > after Jun 2017. Please use osc_lib.utils
> >
> >
> >
> > [root@wpc-controller ~]# openstack stack create --template 
> instance-hot.yaml
> > hot-instance
> >
> > ERROR: Remote error: BadRequest Expecting to find domain in user - the
> > server could not comply with the request since it is either malformed or
> > otherwise incorrect. The client is assumed to be in error. (HTTP 400)
> > (Request-ID: req-e281f8bd-c7cd-486f-87be-dd6401b25cd3)
> >
> > [u'
> >
> >
> >
> > Our Heat Conf:
> >
> > [DEFAULT]
> >
> > log_dir = /var/log/heat
> >
> > heat_metadata_server_url =
> > 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wpc-2Dcontroller.cedev1.d002.eng.pdx.wd-3A8000=DwIBaQ=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ=bwlIMYJQM6k26RrwtU0-iHi3-63kP3vUZLLHvFNSWZ0=JENHqqh-HFBKqgqYOoBuhULMNKV1XIV3lrsM2GKWfio=
> >
> > heat_waitcondition_server_url =
> > 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wpc-2Dcontroller.cedev1.d002.eng.pdx.wd-3A8000_v1_waitcondition=DwIBaQ=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ=bwlIMYJQM6k26RrwtU0-iHi3-63kP3vUZLLHvFNSWZ0=3ja1hV81lm9YdBA3c9asFSLij9cYr9toU8oZ4giTHAQ=
> >
> > heat_watch_server_url = 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wpc-2Dcontroller.cedev1.d002.eng.pdx.wd-3A8003=DwIBaQ=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ=bwlIMYJQM6k26RrwtU0-iHi3-63kP3vUZLLHvFNSWZ0=X_TRvJLmYhjJXyOPwbfGan5uYINzsBwkyc6VfmIlZNE=
> >
> > region_name_for_services = RegionOne
> >
> > num_engine_workers = 5
> >
> > auth_encryption_key = orchestration_auth_encryption_32
> >
> >
> >
> > # Domain
> >
> > stack_domain_admin_password = service_pass
> >
> > stack_domain_admin = heat_domain_admin
> >
> > stack_user_domain = d41933e4c1314a9e8f589db66cc9f844
> >
> > deferred_auth_method=trusts
> >
> >
> >
> >
> >
> > [oslo_messaging_notifications]
> >
> > driver = heat.openstack.common.notifier.rpc_notifier
> >
> >
> >
> > [keystone_authtoken]
> >
> > auth_type = v3password
> >
> > username = heat
> >
> > project_name = service
> >
> > project_domain_name = Default
> >
> > user_domain_name = Default
> >
> > auth_url = 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wpc-2Dcontroller.cedev1.d002.eng.pdx.wd-3A5000_v3=DwI

Re: [openstack-dev] [heat] change in behavior from Mitaka to Newton, is this a bug ?

[Saverio Proto]

Hello,

I confirm that setting:

[oslo_middleware]
enable_proxy_headers_parsing = true

Fixes the problem, and the 'links' property when I create new stacks
gets again the https:// URL.

THANK YOU !!! :)

Anyway we should document this change in the Release notes for people
upgrading to Newton.

I never had:
secure_proxy_ssl_header=X-Forwarded-Proto

in my Mitaka config, because that was the default setting, it was not
necessary to set it explicit.

Also according to:
https://docs.openstack.org/newton/config-reference/orchestration/api.html
the setting is deprecated but not removed.

Adding the enable_proxy_headers_parsing parameter we are also changing
the default behavior, and this is bad if it is not written in the heat
release notes.

https://docs.openstack.org/releasenotes/heat/liberty.html
https://docs.openstack.org/releasenotes/heat/mitaka.html
https://docs.openstack.org/releasenotes/heat/newton.html

Please correct me if I am wrong. But I cannot find anything in the
release (upgrade) notes that warns you should flag this to true if you
are running behind a proxy.

Is it possible to integrate the release notes for Newton now adding a
file in releasenotes/notes/ ? How does it work ?

thank you

Saverio



On 13/04/17 09:52, Rabi Mishra wrote:
> On Thu, Apr 13, 2017 at 1:04 PM, Saverio Proto <saverio.pr...@switch.ch
> <mailto:saverio.pr...@switch.ch>> wrote:
> 
> Hello,
> 
> I am looking at a strange change in default behavior in heat, after the
> upgrade from Mitaka to Newton.
> 
> when you do
> 
> openstack stack show uuid
> 
> you get a table, and there is a property called 'links'
> 
> In Mitaka the value was something like
> 
> - href: https://URL
>   rel: self
> 
> After I upgraded to Newton the 'links' field changed to
> 
> http://URL (self)
> 
> Also in the process the 's' from https went missing.
> 
> 
> Though not sure, may be related to http_proxy_to_wsgi middleware related
> change as mentioned here[1] in newton?
> 
> [1] https://bugs.launchpad.net/heat/+bug/1630778
> 
> May be add the below in heat.conf and check.
> 
> [oslo_middleware]
> enable_proxy_headers_parsing = true
> 
>  
> 
> This broke first of all our rally tests, that now fail with "Prohibited
> endpoint redirect"
> 
> It also broke Horizon, because if I click on the stack name, I am not
> able to get to the page with the stack property, but I get a red box
> "Error: Unable to retrieve stack.".
> 
> Before opening a strange bug that will be marked as "Invalid" I would
> like to understand better what is this 'links' property of the stack,
> and why it changed from Mitaka to Newton ? Is possible this is a
> regression bug ?
> 
> Thank you
> 
> Saverio
> 
> 
> 
> 
> 
> --
> SWITCH
> Saverio Proto, Peta Solutions
> Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
> phone +41 44 268 15 15, direct +41 44 268 1573
> saverio.pr...@switch.ch <mailto:saverio.pr...@switch.ch>,
> http://www.switch.ch
> 
> http://www.switch.ch/stories
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> 
> -- 
> Regards,
> Rabi Misra
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] Openstack Heat https links problems upgrading to Newton

[Saverio Proto]

Hello ops,

if anyone is interested I have problems with Heat and the Newton upgrade.

I sent an email about this here:
http://lists.openstack.org/pipermail/openstack-dev/2017-April/115412.html

If anyone already faced this issue any help would be appreciated !

thank you

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] [heat] change in behavior from Mitaka to Newton, is this a bug ?

[Saverio Proto]

Hello,

I am looking at a strange change in default behavior in heat, after the
upgrade from Mitaka to Newton.

when you do

openstack stack show uuid

you get a table, and there is a property called 'links'

In Mitaka the value was something like

- href: https://URL
  rel: self

After I upgraded to Newton the 'links' field changed to

http://URL (self)

Also in the process the 's' from https went missing.

This broke first of all our rally tests, that now fail with "Prohibited
endpoint redirect"

It also broke Horizon, because if I click on the stack name, I am not
able to get to the page with the stack property, but I get a red box
"Error: Unable to retrieve stack.".

Before opening a strange bug that will be marked as "Invalid" I would
like to understand better what is this 'links' property of the stack,
and why it changed from Mitaka to Newton ? Is possible this is a
regression bug ?

Thank you

Saverio





-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] about the R release name

[Saverio Proto]

Hello Ops,

I got the mail about the Poll for OpenStack R Release Naming

I am shocked that there are proposed names like Raspberry or Root !

Think about troubleshooting and searching on google:

Openstack Raspberry "string of log file"

The Raspberry or Root words are anti-google words that will pollute
the search with a lot of results from other context domain 

Please be smart when voting ! We need names that can identify easily
the releases when using search engines !!!

thank you

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] [neutron][networking-l2gw] bug in case of switch replacement

[Saverio Proto]

Hello,

I am testing the neutron l2gw plugin for production use.

I found a bug, that I think is either a design problem or feature that
is not implemented.

When we configure the l2gateways with the Openstack API, some state is
stored in the mysql database, and the neutron-l2gateway-agent implements
this changes in the OVSDB of the l2gw node.

AFAIK the neutron-l2gateway-agent acts only when triggered by an API
call. If you reboot this service, there is no check between the state
stored in the mysql database, and the actual state of the l2gw node.

Now if the l2gw node is a real switch, and it brakes so that you have to
replace it with a new chassis (real production use-case), you will end
up with some state in the mysql database, and a empty OVSDB on the
switch, and there is no way to resync the OVSDB.

I even tried to delete stuff using the API commands 'neutron
l2-gateway-delete' or 'neutron l2-gateway-connection-delete', but I get
python stacktraces in logs, because it tries to delete stuff from the
OVSDB of the switch that is already empty.. and you are stuck.

Can anyone clarify if what I say is right ?
Do you think I should open a bug about this ?

thanks for the feedback

Saverio


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-l2gw] openstack vtep setup missing docs

[Saverio Proto]

Hello Armando,

I managed to implement the L2GW setup purely in software, without an
hardware appliance.

I documented in the README file, please look at this review
https://review.openstack.org/453209

I have a question: do we have a name for this node where the actually
bridging happens between a VXLAN tenant network and a physical L2 network ?
Is it okay to call it the l2gw node ?

The l2gw plugin it self runs on the controller, so also the
neutron-l2gw-plugin agent runs on the controller.

I think it necessary to clarify this naming, because before trying the
software I did the mistake of thinking that the neutron-l2gw-agent had
to run on the switch where the actual briding happens.

thank you

Saverio




On 30/03/17 18:40, Armando M. wrote:
> 
> 
> On 30 March 2017 at 08:47, Saverio Proto <saverio.pr...@switch.ch
> <mailto:saverio.pr...@switch.ch>> wrote:
> 
> Hello,
> 
> I am trying to use the neutron l2gw plugin, but I am not using a bare
> metal switch to bridge.
> 
> I am using a server with Openvswitch.
> 
> 
> I am not aware of any effort to implement L2GW purely in software, in
> fact this was one key missing pieces that prevented the project to have
> CI solely dealt with the upstream infra resources. Perhaps OVN may come
> to the rescue here, I recall at some point the team was looking at the
> L2GW API.
> 
> Thanks,
> Armando
>  
> 
> 
> Following this documentation:
> 
> http://networkop.co.uk/blog/2016/05/21/neutron-l2gw/
> <http://networkop.co.uk/blog/2016/05/21/neutron-l2gw/>
> 
> At one point there is this command:
> 
> sudo vtep-bootstrap L5 10.0.0.5 192.168.91.21 --no_encryption
> 
> This vtep-bootstrap is specific for Cumulux Linux
> 
> Anybody has documentation with normal vtep-ctl commands ?
> 
> So far on the Ubuntu server I did the following:
> 
> apt-get install openvswitch-vtep
> ovsdb-tool create /etc/openvswitch/vtep.db
> /usr/share/openvswitch/vtep.ovsschema
> 
> Anyone has more complete documentation ?
> 
> I did not understand if the vtep-openvswitch controlled by the l2gw
> plugin will make vxlan tunnels to all the compute nodes, to bridge the
> tenant network with a physical l2 network ? Or all this traffic has to
> pass to the network node also because the vtep openvswitch is not able
> to talk to the compute nodes ?
> 
> thank you
> 
> Saverio
> 
> 
> --
> SWITCH
> Saverio Proto, Peta Solutions
> Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
> phone +41 44 268 15 15, direct +41 44 268 1573
> saverio.pr...@switch.ch <mailto:saverio.pr...@switch.ch>,
> http://www.switch.ch
> 
> http://www.switch.ch/stories
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> 
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-l2gw] openstack vtep setup missing docs

[Saverio Proto]

I managed to make it work without an appliance.

It is possible to use just a Ubuntu Server with OVS, and run both the
vtep.db and ovs.db
Basically as you have a Openstack network node, you can have a openstack
l2gatewaynode ... but it does not really need any openstack software on
it. You just need OVS.

It is okay of I submit a git review that changes this file:
https://github.com/openstack/networking-l2gw/blob/master/README.rst

in the getting started session ?

or there is a better place where to integrate this documentation ?

is it okay to put a reference to the blogs where I found the information
to make the all thing work ?

Saverio

On 30/03/17 18:40, Armando M. wrote:
> 
> 
> On 30 March 2017 at 08:47, Saverio Proto <saverio.pr...@switch.ch
> <mailto:saverio.pr...@switch.ch>> wrote:
> 
> Hello,
> 
> I am trying to use the neutron l2gw plugin, but I am not using a bare
> metal switch to bridge.
> 
> I am using a server with Openvswitch.
> 
> 
> I am not aware of any effort to implement L2GW purely in software, in
> fact this was one key missing pieces that prevented the project to have
> CI solely dealt with the upstream infra resources. Perhaps OVN may come
> to the rescue here, I recall at some point the team was looking at the
> L2GW API.
> 
> Thanks,
> Armando
>  
> 
> 
> Following this documentation:
> 
> http://networkop.co.uk/blog/2016/05/21/neutron-l2gw/
> <http://networkop.co.uk/blog/2016/05/21/neutron-l2gw/>
> 
> At one point there is this command:
> 
> sudo vtep-bootstrap L5 10.0.0.5 192.168.91.21 --no_encryption
> 
> This vtep-bootstrap is specific for Cumulux Linux
> 
> Anybody has documentation with normal vtep-ctl commands ?
> 
> So far on the Ubuntu server I did the following:
> 
> apt-get install openvswitch-vtep
> ovsdb-tool create /etc/openvswitch/vtep.db
> /usr/share/openvswitch/vtep.ovsschema
> 
> Anyone has more complete documentation ?
> 
> I did not understand if the vtep-openvswitch controlled by the l2gw
> plugin will make vxlan tunnels to all the compute nodes, to bridge the
> tenant network with a physical l2 network ? Or all this traffic has to
> pass to the network node also because the vtep openvswitch is not able
> to talk to the compute nodes ?
> 
> thank you
> 
> Saverio
> 
> 
> --
> SWITCH
> Saverio Proto, Peta Solutions
> Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
> phone +41 44 268 15 15, direct +41 44 268 1573
> saverio.pr...@switch.ch <mailto:saverio.pr...@switch.ch>,
> http://www.switch.ch
> 
> http://www.switch.ch/stories
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> 
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][networking-l2gw] openstack vtep setup missing docs

[Saverio Proto]

Hello,

I am trying to use the neutron l2gw plugin, but I am not using a bare
metal switch to bridge.

I am using a server with Openvswitch.

Following this documentation:

http://networkop.co.uk/blog/2016/05/21/neutron-l2gw/

At one point there is this command:

sudo vtep-bootstrap L5 10.0.0.5 192.168.91.21 --no_encryption

This vtep-bootstrap is specific for Cumulux Linux

Anybody has documentation with normal vtep-ctl commands ?

So far on the Ubuntu server I did the following:

apt-get install openvswitch-vtep
ovsdb-tool create /etc/openvswitch/vtep.db
/usr/share/openvswitch/vtep.ovsschema

Anyone has more complete documentation ?

I did not understand if the vtep-openvswitch controlled by the l2gw
plugin will make vxlan tunnels to all the compute nodes, to bridge the
tenant network with a physical l2 network ? Or all this traffic has to
pass to the network node also because the vtep openvswitch is not able
to talk to the compute nodes ?

thank you

Saverio


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][networking-l2gw] database tables for neutron l2gw plugin

[Saverio Proto]

Hello,

I am testing the neutron l2gw in our staging env.

Because I cant redeploy everything, to retry the installation of the
l2gw, to clean the database I drop the following tables from the neutron
database:

drop table physical_ports;
drop table physical_locators;
drop table physical_switches;
drop table logical_switches;
drop table ucast_macs_remotes;
drop table ucast_macs_locals;
drop table vlan_bindings;
drop table pending_ucast_macs_remotes;
drop table l2gatewayinterfaces;
drop table l2gatewaydevices;
drop table l2gatewayconnections;
drop table l2gateways;
drop table l2gw_alembic_version;

I would strongly suggest to have a common prefix like l2gw_ for all the
tables that belong to the same neutron plugin.

How can I figure out if I missed a table without reading all the code ?

Thank you

Saverio



-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] backup to object store - tool recommendations

[Saverio Proto]

Hello all,

we use rclone a lot, and we are happy with it.

the real problem I would say is that a lot of these tools use the
latest AWS4 signature.

AFAIK the radosgw with Ceph Jewel and Openstack keystone integration
supports only AWS2 signature because of this bug:
http://tracker.ceph.com/issues/19056

is anyone else hitting this ?

Saverio

2017-03-27 22:11 GMT+02:00 John Dickinson :
>
>
> On 27 Mar 2017, at 4:39, Blair Bethwaite wrote:
>
>> Hi all,
>>
>> Does anyone have any recommendations for good tools to perform
>> file-system/tree backups and restores to/from a (Ceph RGW-based)
>> object store (Swift or S3 APIs)? Happy to hear about both FOSS and
>> commercial options please.
>>
>> I'm interested in:
>> 1) tools known to work or not work at all for a basic file-based data backup
>
> There's a bunch of backup tools that will work with the Swift API and/or the 
> S3 API.
>
> Veritas, Commvault, Trilio, and CloudBerry all work. There's other companies 
> too that can back specific stuff up to Swift (e.g. Percona with MySQL).
>
> (The above list taken from https://www.swiftstack.com/solutions/backup [my 
> employer] because it's the first linkable place I knew of to answer your 
> question.)
>
>
> --John
>
>
>
>
>>
>> Plus these extras:
>> 2) preserves/restores correct file metadata (e.g. owner, group, acls etc)
>> 3) preserves/restores xattrs
>> 4) backs up empty directories and files
>> 5) supports some sort of snapshot/versioning/differential
>> functionality, i.e., will keep a copy or diff or last N versions of a
>> file or whole backup set, e.g., so that one can restore yesterday's
>> file/s or last week's but not have to keep two full copies to achieve
>> it
>> 6) is readily able to restore individual files
>> 7) can encrypt/decrypt client side
>> 8) anything else I should be considering?
>>
>> --
>> Cheers,
>> ~Blairo
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [neutron] Modify Default Quotas

[Saverio Proto]

Hello,

floating IPs is the real issue.

When using horizon it is very easy for users to allocate floating ips
but it is also very difficult to release them.

In our production cloud we had to change the default from 50 to 2. We
have to be very conservative with floatingips quota because our
experience is that the user will never release a floating IP.

A good starting point is to set the quota for the floatingips at the
the same quota for nova instances.

Saverio


2017-03-22 16:46 GMT+01:00 Morales, Victor :
> Hey there,
>
>
>
> I noticed that Ihar started working on a change to increase the default
> quotas values in Neutron[1].  Personally, I think that makes sense to change
> it but I’d like to complement it.  So, based on your experience, what should
> be the most common quota value for networks, subnets, ports, security
> groups, security rules, routers and Floating IPs per tenant?
>
>
>
> Regards/Saludos
>
> Victor Morales
>
> irc: electrocucaracha
>
>
>
> [1] https://review.openstack.org/#/c/444030
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] [lbaas][neutron] Is possible the lbaas commands are missing completely from openstack cli ?

[Saverio Proto]

Client version: openstack 3.9.0

I cant find any lbaas commands. I have to use the 'neutron' client.

Everycommand I get:
neutron CLI is deprecated and will be removed in the future. Use
openstack CLI instead.

Is LBaaS even going to be implemented in the unified openstack client ?

thank you

Saverio

-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] ceph rbd root disk unexpected deletion

[Saverio Proto]

Hello Mike,

what version of openstack ?
is the instance booting from ephemeral disk or booting from cinder volume ?

When you boot from volume, that will be the root disk of your
instance. The user could have clicked on "Delete Volume on Instance
Delete". It can be selected when creating a new instance.

Saverio

2017-03-13 15:47 GMT+01:00 Mike Lowe :
> Over the weekend a user reported that his instance was in a stopped state and 
> could not be started, on further examination it appears that the vm had 
> crashed and the strange thing is that the root disk is now gone.  Has anybody 
> come across anything like this before?
>
> And why on earth is it attempting deletion of the rbd device without deletion 
> of the instance?
>
> 2017-03-12 10:59:07.591 3010 WARNING nova.virt.libvirt.storage.rbd_utils [-] 
> rbd remove 4367a2e4-d704-490d-b3a6-129b9465cd0d_disk in pool ephemeral-vms 
> failed
> 2017-03-12 10:59:17.613 3010 WARNING nova.virt.libvirt.storage.rbd_utils [-] 
> rbd remove 4367a2e4-d704-490d-b3a6-129b9465cd0d_disk in pool ephemeral-vms 
> failed
> 2017-03-12 10:59:26.143 3010 WARNING nova.virt.libvirt.storage.rbd_utils [-] 
> rbd remove 4367a2e4-d704-490d-b3a6-129b9465cd0d_disk in pool ephemeral-vms 
> failed
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [neutron][lbaasv2] Migrate LBaaS instance

[Saverio Proto]

Hello there,

I am just back from the Ops Midcycle where Heidi Joy Tretheway reported
some data from the user survey.

So if we look at deployments with more than 100 servers NO ONE IS USING
NEWTON yet. And I scream this loud. Everyone is still in Liberty or Mitaka.

I am just struggling to upgrade to LBaaSv2 to hear that is already going
into deprecation. The feature Zhi is proposing is important also for me
once I go to production.

I would encourage devs to listen more to operators feedback. Also you
devs cant just ignore that users are running still Liberty/Mitaka so you
need to change something in this way of working or all the users will
run away.

thank you

Saverio


On 16/03/17 16:26, Kosnik, Lubosz wrote:
> Hello Zhi,
> Just one small information. Yesterday on Octavia weekly meeting we
> decided that we’re gonna add new features to LBaaSv2 till Pike-1 so the
> windows is very small.
> This decision was made as LBaaSv2 is currently Octavia delivery, not
> Neutron anymore and this project is going into deprecations stage.
> 
> Cheers,
> Lubosz
> 
>> On Mar 16, 2017, at 5:39 AM, zhi <changzhi1...@gmail.com
>> <mailto:changzhi1...@gmail.com>> wrote:
>>
>> Hi, all
>> Currently, LBaaS v2 doesn't support migration. Just like router
>> instances, we can remove a router instance from one L3 agent and add
>> it to another L3 agent.
>>
>> So, there is a single point failure in LBaaS agent. As far as I know,
>> LBaaS supports " allow_automatic_lbaas_agent_failover ". But in many
>> cases, we want to migrate LBaaS instances manually. Do we plan to do this?
>>
>> I'm doing this right now. But I meet a question. I define a function
>> in agent_scheduler.py like this:
>>
>> def remove_loadbalancer_from_lbaas_agent(self, context, agent_id,
>> loadbalancer_id):
>> self._unschedule_loadbalancer(context, loadbalancer_id, agent_id)
>>
>> The question is, how do I notify LBaaS agent? 
>>
>> Hope for your reply.
>>
>>
>>
>> Thanks
>> Zhi Chang
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org
>> <mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] Thank you for the ops-midcycle in Milano

[Saverio Proto]

Hello !

thank you for the great event. Mariano & all the people from Milano
made an excellent work.

thanks all to all of you that helped moderating the sessions and
contributing to the etherpads.

it was really a great event and I am looking forward for the next ones.

Cheers,

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] OpenStack Operators Milan - Neutron Session

[Saverio Proto]

Hello Edgar,

there was a discussion about this here:
http://eavesdrop.openstack.org/meetings/ops_meetup_team/2017/ops_meetup_team.2017-03-07-20.16.log.txt

we had several sponsor requests to make presentations. We agreed that as
long as the sponsor presentations are strongly technical everybody will be
happy.

ciao,

Saverio



2017-03-13 18:21 GMT+01:00 Edgar Magana :

> WOW! Something has changed from previous Ops Meet-ups. I remember clearly
> that we (Operators), wanted to have a free-from-vendors environment and it
> seems that now we want them to come and provide updates on their offerings.
>
> I think vendors have plenty opportunities to show their features and
> demos, for instance the Summits and OpenStack Days around the world.
>
>
>
> Thanks,
>
>
>
> Edgar
>
>
>
> *From: *Christoph Andreas Torlinsky 
> *Date: *Monday, March 13, 2017 at 10:11 AM
> *To: *"randy.perry...@dell.com" 
> *Cc: *"OpenStack-operators@lists.openstack.org" <
> OpenStack-operators@lists.openstack.org>
> *Subject: *Re: [Openstack-operators] OpenStack Operators Milan - Neutron
> Session
>
>
>
> I'll be there from Nuage Networks, as far as I know.
>
>
>
>
>
>
> Christoph Andreas Torlinski
>
>
>
> christ...@nuagenetworks.net
>
>
>
> On 9 March 2017 at 13:36,  wrote:
>
> Here is the etherpad for the Neutron Session in Milan
>
> https://etherpad.openstack.org/p/MIL-ops-neutron
> 
>
>
>
> Please add to it anything you would like to cover.
>
>
>
> Also will anyone from SDN Vendors be there?  Please come and provide a 5
> minute update to your offering.
>
>
>
> Thank You
>
>
>
> *Randy Perryman*
>
> Technical Staff, Cloud Architect
> Certified OpenStack Administrator
>
>
> * Dell* *EMC* | OpenSource Solutions Group
>
> *office* + 01 603 249 7710
>
> *mobile *+01 603 321 5611
>
> email randy.perry...@dell.com
>
> http://www.dell.com/cloud
> 
>
>
>
> Please consider the environment before printing this email.
>
>
>
> Confidentiality Notice | This e-mail message, including any attachments,
> is for the sole use of the intended recipient(s) and may contain
> confidential or proprietary information. Any unauthorized review, use,
> disclosure or distribution is prohibited. If you are not the intended
> recipient, immediately contact the sender by reply e-mail and destroy all
> copies of the original message.
>
>
>
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> 
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] OpenStack Operators Milan

[Saverio Proto]

> Can someone please put together a list of urls to all the sessions, there are 
> so many urls flying around.

Hi,
you can find the list here:
https://etherpad.openstack.org/p/MIL-ops-meetup

Cheers

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [Neutron][LBaaS] - Best release to upgrade from LBaaS v1 to v2

[Saverio Proto]

On 10/03/17 17:49, Michael Johnson wrote:
> Yes, folks have recently deployed the dashboard with success.  I think you
> had that discussion on the IRC channel, so I won't repeat it here.
> 
> Please note, the neutron-lbaas-dashboard does not support LBaaS v1, you must
> have LBaaS v2 deployed for the neutron-lbaas-dashboard to work.  If you are
> trying to use LBaaS v1, you can use the legacy panels included in the older
> versions of horizon.
> 
[..CUT..]
> If you think there is an open bug for the dashboard, please report it in
> https://bugs.launchpad.net/neutron-lbaas-dashboard



Hello,
I updated the bug
https://bugs.launchpad.net/neutron-lbaas-dashboard/+bug/1621403

Can anyone clarify the version matrix to use between the horizon version
and the neutron-lbaas-dashboard panels versions ?

can anyone confirm that both files
_1481_project_ng_loadbalancersv2_panel.py and file
_1480_project_loadbalancersv2_panel.py need to be installed ?

Is it okay to use branch master of neutron-lbaas-dashboard with horizon
stable/newton ?

thank you

Saverio
















__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] need input on log translations

[Saverio Proto]

Hello there,
I am Italian speaking.
Does not make any sense to have the log messages translated. I think
everything has already being said.

Saverio


2017-03-11 9:20 GMT+01:00 George Shuklin :
> Whole idea with log translation is half-backed anyway. About the half of
> important log messages contain output of things outside openstack. Libvirt,
> ip, sudo, kernel, etc. In any i18n installation there going to be some
> amount of untranslated messages. This kills whole idea of localization.
>
> Modern operator ought to know English at 'technical reading' level anyway.
> Therefore, localization does not achieve it goal, but cause pain instead:
> search segmentation, slightly misleading translation (f.e. 'stream' and
> 'thread' both translate into Russian 'поток', which brings ambiguity),
> different system may use slightly different translation, causing even more
> mess.
>
> As Russian speaker and openstack operator I definitely don't want to have
> logs translation.
>
> On Mar 10, 2017 4:42 PM, "Doug Hellmann"  wrote:
>
> There is a discussion on the -dev mailing list about the i18n team
> decision to stop translating log messages [1]. The policy change means
> that we may be able to clean up quite a lot of "clutter" throughout the
> service code, because without anyone actually translating the messages
> there is no need for the markup code used to tag those strings.
>
> If we do remove the markup from log messages, we will be effectively
> removing "multilingual logs" as a feature. Given the amount of work
> and code churn involved in the first roll out, I would not expect
> us to restore that feature later.
>
> Therefore, before we take what would almost certainly be an
> irreversible action, we would like some input about whether log
> message translations are useful to anyone. Please let us know if
> you or your customers use them.
>
> Thanks,
> Doug
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [Neutron][LBaaS] - Best release to upgrade from LBaaS v1 to v2

[Saverio Proto]

I spent the all day trying to deploy an Horizon instance with working
panels for LBaaSv2.
https://github.com/openstack/neutron-lbaas-dashboard

I tried stable/ocata and I am never able to list existing load balancers
or create a new loadbalancer.

Looks like I am not the only one with this issue:
https://ask.openstack.org/en/question/96790/lbaasv2-dashboard-issues/

Is there anyone that has a working setup ?

Should I open a bug here?
https://bugs.launchpad.net/octavia/+filebug

Thanks

Saverio


On 09/03/17 16:19, Saverio Proto wrote:
> Hello,
> 
> I managed to do the database migration.
> 
> I had to skip this logic:
> https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py#L342-L353
> 
> I had to force flag=True
> 
> That code obviously breaks if you have LBaaS used by more than 1 tenant.
> 
> What was the goal ? to make sure that a given healthmonitor is not
> reused in multiple pools ?
> 
> Should the right approach be to check if these two values are the same ?:
> 
> select count(DISTINCT monitor_id) from poolmonitorassociations;
> select count(monitor_id) from poolmonitorassociations;
> 
> Second question: should the old tables from LBaaSV1 be dropped ?
> 
> Please give me feedback so I can fix the code and submit a review.
> 
> thank you
> 
> Saverio
> 
> 
> On 09/03/17 13:38, Saverio Proto wrote:
>>> I would recommend experimenting with the database-migration-from-v1-to-v2.py
>>> script and working with your vendor (if you are using a vendor load
>>> balancing engine) on a migration path.
>>
>>
>> Hello,
>> there is no vendor here to help us :)
>>
>> I made a backup of the current DB.
>>
>> I identified this folder on our Neutron server:
>>
>> /usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration ; tree
>> .
>> |-- alembic_migrations
>> |   |-- env.py
>> |   |-- env.pyc
>> |   |-- __init__.py
>> |   |-- __init__.pyc
>> |   |-- README
>> |   |-- script.py.mako
>> |   `-- versions
>> |   |-- 364f9b6064f0_agentv2.py
>> |   |-- 364f9b6064f0_agentv2.pyc
>> |   |-- 4b6d8d5310b8_add_index_tenant_id.py
>> |   |-- 4b6d8d5310b8_add_index_tenant_id.pyc
>> |   |-- 4ba00375f715_edge_driver.py
>> |   |-- 4ba00375f715_edge_driver.pyc
>> |   |-- 4deef6d81931_add_provisioning_and_operating_statuses.py
>> |   |-- 4deef6d81931_add_provisioning_and_operating_statuses.pyc
>> |   |-- CONTRACT_HEAD
>> |   |-- EXPAND_HEAD
>> |   |-- kilo_release.py
>> |   |-- kilo_release.pyc
>> |   |-- lbaasv2.py
>> |   |-- lbaasv2.pyc
>> |   |-- lbaasv2_tls.py
>> |   |-- lbaasv2_tls.pyc
>> |   |-- liberty
>> |   |   |-- contract
>> |   |   |   |-- 130ebfdef43_initial.py
>> |   |   |   `-- 130ebfdef43_initial.pyc
>> |   |   `-- expand
>> |   |   |-- 3345facd0452_initial.py
>> |   |   `-- 3345facd0452_initial.pyc
>> |   |-- mitaka
>> |   |   `-- expand
>> |   |   |-- 3426acbc12de_add_flavor_id.py
>> |   |   |-- 3426acbc12de_add_flavor_id.pyc
>> |   |   |-- 3543deab1547_add_l7_tables.py
>> |   |   |-- 3543deab1547_add_l7_tables.pyc
>> |   |   |-- 4a408dd491c2_UpdateName.py
>> |   |   |-- 4a408dd491c2_UpdateName.pyc
>> |   |   |-- 62deca5010cd_add_tenant_id_index_for_l7_tables.py
>> |   |   |-- 62deca5010cd_add_tenant_id_index_for_l7_tables.pyc
>> |   |   |-- 6aee0434f911_independent_pools.py
>> |   |   `-- 6aee0434f911_independent_pools.pyc
>> |   |-- start_neutron_lbaas.py
>> |   `-- start_neutron_lbaas.pyc
>> |-- __init__.py
>> `-- __init__.pyc
>>
>> 7 directories, 40 files
>>
>> Now here it says: "Create a revision file"
>>
>> https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py#L30
>>
>> There is some specific openstack-dev documentation to "Create a revision
>> file" or I should just learn the Alembic tool ? I never used it before.
>>
>> So far I did copy the alembic.ini from here:
>> https://github.com/openstack/neutron/blob/master/neutron/db/migration/alembic.ini
>>
>> into /usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration
>>
>> then I did run the command:
>>
>> alembic revision -m "migrate to LBaaSv2"
>>
>> as a result it created the file:
>> /usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration/alembic_migrations/versions/242

Re: [openstack-dev] [Neutron][LBaaS] - Best release to upgrade from LBaaS v1 to v2

[Saverio Proto]

Hello,

I managed to do the database migration.

I had to skip this logic:
https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py#L342-L353

I had to force flag=True

That code obviously breaks if you have LBaaS used by more than 1 tenant.

What was the goal ? to make sure that a given healthmonitor is not
reused in multiple pools ?

Should the right approach be to check if these two values are the same ?:

select count(DISTINCT monitor_id) from poolmonitorassociations;
select count(monitor_id) from poolmonitorassociations;

Second question: should the old tables from LBaaSV1 be dropped ?

Please give me feedback so I can fix the code and submit a review.

thank you

Saverio


On 09/03/17 13:38, Saverio Proto wrote:
>> I would recommend experimenting with the database-migration-from-v1-to-v2.py
>> script and working with your vendor (if you are using a vendor load
>> balancing engine) on a migration path.
> 
> 
> Hello,
> there is no vendor here to help us :)
> 
> I made a backup of the current DB.
> 
> I identified this folder on our Neutron server:
> 
> /usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration ; tree
> .
> |-- alembic_migrations
> |   |-- env.py
> |   |-- env.pyc
> |   |-- __init__.py
> |   |-- __init__.pyc
> |   |-- README
> |   |-- script.py.mako
> |   `-- versions
> |   |-- 364f9b6064f0_agentv2.py
> |   |-- 364f9b6064f0_agentv2.pyc
> |   |-- 4b6d8d5310b8_add_index_tenant_id.py
> |   |-- 4b6d8d5310b8_add_index_tenant_id.pyc
> |   |-- 4ba00375f715_edge_driver.py
> |   |-- 4ba00375f715_edge_driver.pyc
> |   |-- 4deef6d81931_add_provisioning_and_operating_statuses.py
> |   |-- 4deef6d81931_add_provisioning_and_operating_statuses.pyc
> |   |-- CONTRACT_HEAD
> |   |-- EXPAND_HEAD
> |   |-- kilo_release.py
> |   |-- kilo_release.pyc
> |   |-- lbaasv2.py
> |   |-- lbaasv2.pyc
> |   |-- lbaasv2_tls.py
> |   |-- lbaasv2_tls.pyc
> |   |-- liberty
> |   |   |-- contract
> |   |   |   |-- 130ebfdef43_initial.py
> |   |   |   `-- 130ebfdef43_initial.pyc
> |   |   `-- expand
> |   |   |-- 3345facd0452_initial.py
> |   |   `-- 3345facd0452_initial.pyc
> |   |-- mitaka
> |   |   `-- expand
> |   |   |-- 3426acbc12de_add_flavor_id.py
> |   |   |-- 3426acbc12de_add_flavor_id.pyc
> |   |   |-- 3543deab1547_add_l7_tables.py
> |   |   |-- 3543deab1547_add_l7_tables.pyc
> |   |   |-- 4a408dd491c2_UpdateName.py
> |   |   |-- 4a408dd491c2_UpdateName.pyc
> |   |   |-- 62deca5010cd_add_tenant_id_index_for_l7_tables.py
> |   |   |-- 62deca5010cd_add_tenant_id_index_for_l7_tables.pyc
> |   |   |-- 6aee0434f911_independent_pools.py
> |   |   `-- 6aee0434f911_independent_pools.pyc
> |   |-- start_neutron_lbaas.py
> |   `-- start_neutron_lbaas.pyc
> |-- __init__.py
> `-- __init__.pyc
> 
> 7 directories, 40 files
> 
> Now here it says: "Create a revision file"
> 
> https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py#L30
> 
> There is some specific openstack-dev documentation to "Create a revision
> file" or I should just learn the Alembic tool ? I never used it before.
> 
> So far I did copy the alembic.ini from here:
> https://github.com/openstack/neutron/blob/master/neutron/db/migration/alembic.ini
> 
> into /usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration
> 
> then I did run the command:
> 
> alembic revision -m "migrate to LBaaSv2"
> 
> as a result it created the file:
> /usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration/alembic_migrations/versions/24274573545b_migrate_to_lbaasv2.py
> 
> Then I added the script to that file:
> wget -O -
> https://raw.githubusercontent.com/openstack/neutron-lbaas/master/tools/database-migration-from-v1-to-v2.py
>>> alembic_migrations/versions/24274573545b_migrate_to_lbaasv2.py
> 
> I tried now:
> neutron-db-manage upgrade heads
> 
> but it fails with a easy stacktrace. I get stuck here:
> https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py#L346
> 
> of course the query:
> select tenant_id from pools;
> 
> returns more than 1 tenant_id
> 
> it means I cannot migrate if I have more than 1 tenant using LBaaS v1 ?
> 
> am I doing something completely wrong here ? am I on the right direction ?
> 
> thank you
> 
> Saverio
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _

Re: [openstack-dev] [Neutron][LBaaS] - Best release to upgrade from LBaaS v1 to v2

[Saverio Proto]

> I would recommend experimenting with the database-migration-from-v1-to-v2.py
> script and working with your vendor (if you are using a vendor load
> balancing engine) on a migration path.


Hello,
there is no vendor here to help us :)

I made a backup of the current DB.

I identified this folder on our Neutron server:

/usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration ; tree
.
|-- alembic_migrations
|   |-- env.py
|   |-- env.pyc
|   |-- __init__.py
|   |-- __init__.pyc
|   |-- README
|   |-- script.py.mako
|   `-- versions
|   |-- 364f9b6064f0_agentv2.py
|   |-- 364f9b6064f0_agentv2.pyc
|   |-- 4b6d8d5310b8_add_index_tenant_id.py
|   |-- 4b6d8d5310b8_add_index_tenant_id.pyc
|   |-- 4ba00375f715_edge_driver.py
|   |-- 4ba00375f715_edge_driver.pyc
|   |-- 4deef6d81931_add_provisioning_and_operating_statuses.py
|   |-- 4deef6d81931_add_provisioning_and_operating_statuses.pyc
|   |-- CONTRACT_HEAD
|   |-- EXPAND_HEAD
|   |-- kilo_release.py
|   |-- kilo_release.pyc
|   |-- lbaasv2.py
|   |-- lbaasv2.pyc
|   |-- lbaasv2_tls.py
|   |-- lbaasv2_tls.pyc
|   |-- liberty
|   |   |-- contract
|   |   |   |-- 130ebfdef43_initial.py
|   |   |   `-- 130ebfdef43_initial.pyc
|   |   `-- expand
|   |   |-- 3345facd0452_initial.py
|   |   `-- 3345facd0452_initial.pyc
|   |-- mitaka
|   |   `-- expand
|   |   |-- 3426acbc12de_add_flavor_id.py
|   |   |-- 3426acbc12de_add_flavor_id.pyc
|   |   |-- 3543deab1547_add_l7_tables.py
|   |   |-- 3543deab1547_add_l7_tables.pyc
|   |   |-- 4a408dd491c2_UpdateName.py
|   |   |-- 4a408dd491c2_UpdateName.pyc
|   |   |-- 62deca5010cd_add_tenant_id_index_for_l7_tables.py
|   |   |-- 62deca5010cd_add_tenant_id_index_for_l7_tables.pyc
|   |   |-- 6aee0434f911_independent_pools.py
|   |   `-- 6aee0434f911_independent_pools.pyc
|   |-- start_neutron_lbaas.py
|   `-- start_neutron_lbaas.pyc
|-- __init__.py
`-- __init__.pyc

7 directories, 40 files

Now here it says: "Create a revision file"

https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py#L30

There is some specific openstack-dev documentation to "Create a revision
file" or I should just learn the Alembic tool ? I never used it before.

So far I did copy the alembic.ini from here:
https://github.com/openstack/neutron/blob/master/neutron/db/migration/alembic.ini

into /usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration

then I did run the command:

alembic revision -m "migrate to LBaaSv2"

as a result it created the file:
/usr/lib/python2.7/dist-packages/neutron_lbaas/db/migration/alembic_migrations/versions/24274573545b_migrate_to_lbaasv2.py

Then I added the script to that file:
wget -O -
https://raw.githubusercontent.com/openstack/neutron-lbaas/master/tools/database-migration-from-v1-to-v2.py
>> alembic_migrations/versions/24274573545b_migrate_to_lbaasv2.py

I tried now:
neutron-db-manage upgrade heads

but it fails with a easy stacktrace. I get stuck here:
https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py#L346

of course the query:
select tenant_id from pools;

returns more than 1 tenant_id

it means I cannot migrate if I have more than 1 tenant using LBaaS v1 ?

am I doing something completely wrong here ? am I on the right direction ?

thank you

Saverio














__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] Milano session on Upgrades Patches and Packaging

[Saverio Proto]

Hello,

I prepared the skeleton for the session about Upgrades Patches and Packaging

https://etherpad.openstack.org/p/MIL-ops-upgrades-patches-packaging

Please contribute to the etherpad with stuff like:

 * patches that you are carrying in production
 * patches that you dont manage to get merged upstream
 * upgrade problems
 * regression bugs once you land on the new release

This is a session that is very useful if you have to do soon an
openstack upgrade :)

ciao,

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [Neutron][LBaaS] - Best release to upgrade from LBaaS v1 to v2

[Saverio Proto]

Hello Michael,

thanks. Using your email I read this page:
https://docs.openstack.org/ocata/networking-guide/config-lbaas.html

It is still not clear to me if the command:

neutron-db-manage --subproject neutron-lbaas upgrade head

Will make the necessary database migrations from LBaaS v1 to v2 ?

Does this command triggers the execution of this code ?
https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py

On what openstack version should I run that neutron-db-manage command ?
I am currently in Mitaka.

Here I read:
https://docs.openstack.org/releasenotes/neutron-lbaas/newton.html
LBaaS API v1 has been removed. Do not upgrade before migrating to LBaaS
API v2.

This means I have to run 'neutron-db-manage --subproject neutron-lbaas
upgrade head' before upgrading ?

am I missing the page where the migration from V1 to V2 is explained ?

thank you

Saverio


On 07/03/17 17:33, Michael Johnson wrote:
> Hi Saverio,
> 
> I think the confusion is coming from neutron/neutron-lbaas/octavia.
> 
> Neutron-lbaas, prior to the Ocata series was a sub-project of neutron and as
> such has it's own release notes:
> https://docs.openstack.org/releasenotes/neutron-lbaas/
> 
> As of Ocata, neutron-lbaas is part of the Octavia project
> (https://governance.openstack.org/tc/reference/projects/octavia.html) and is
> no longer a sub-project of neutron.  In fact, we are actively working to
> merge the neutron-lbaas v2 API into the Octavia API to create a combined
> project.
> 
> Going forward you will probably want to monitor both neutron-lbaas and the
> octavia release notes:
> https://docs.openstack.org/releasenotes/neutron-lbaas/
> https://docs.openstack.org/releasenotes/octavia/
> 
> To answer your original question, the LBaaS v1 API as removed in the newton
> release of neutron-lbaas
> (https://docs.openstack.org/releasenotes/neutron-lbaas/newton.html).
> 
> Michael
> 
> 
> -Original Message-
> From: Saverio Proto [mailto:saverio.pr...@switch.ch] 
> Sent: Tuesday, March 7, 2017 1:09 AM
> To: OpenStack Development Mailing List (not for usage questions)
> <openstack-dev@lists.openstack.org>
> Subject: [openstack-dev] [Neutron][LBaaS] - Best release to upgrade from
> LBaaS v1 to v2
> 
> Hello,
> 
> I am upgrading from Mitaka to Newton.
> 
> our Openstack cloud has in production LBaaSv1.
> 
> I read all the following release notes:
> 
> https://docs.openstack.org/releasenotes/neutron/liberty.html
> https://docs.openstack.org/releasenotes/neutron/mitaka.html
> https://docs.openstack.org/releasenotes/neutron/newton.html
> 
> In the liberty release notes I read:
> "The LBaaS V1 API is marked as deprecated and is planned to be removed in a
> future release. Going forward, the LBaaS V2 API should be used."
> 
> But which one is the release that drops LBaaS V1 ?
> 
> I see this script is merged in stable/newton:
> 
> https://review.openstack.org/#/c/289595/
> 
> Can I still use LBaaS V1 in newton and do the migration before upgrading to
> Ocata ?
> 
> The cherry-pick to Mitaka was abandoned:
> https://review.openstack.org/#/c/370103/
> 
> The Ocata release notes again dont say anything about LBaaS:
> https://docs.openstack.org/releasenotes/neutron/ocata.html
> 
> thank you
> 
> Saverio
> 
> 
> 
> --
> SWITCH
> Saverio Proto, Peta Solutions
> Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland phone +41 44 268 15 15,
> direct +41 44 268 1573 saverio.pr...@switch.ch, http://www.switch.ch
> 
> http://www.switch.ch/stories
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron][LBaaS] - Best release to upgrade from LBaaS v1 to v2

[Saverio Proto]

Hello,

I am upgrading from Mitaka to Newton.

our Openstack cloud has in production LBaaSv1.

I read all the following release notes:

https://docs.openstack.org/releasenotes/neutron/liberty.html
https://docs.openstack.org/releasenotes/neutron/mitaka.html
https://docs.openstack.org/releasenotes/neutron/newton.html

In the liberty release notes I read:
"The LBaaS V1 API is marked as deprecated and is planned to be removed
in a future release. Going forward, the LBaaS V2 API should be used."

But which one is the release that drops LBaaS V1 ?

I see this script is merged in stable/newton:

https://review.openstack.org/#/c/289595/

Can I still use LBaaS V1 in newton and do the migration before upgrading
to Ocata ?

The cherry-pick to Mitaka was abandoned:
https://review.openstack.org/#/c/370103/

The Ocata release notes again dont say anything about LBaaS:
https://docs.openstack.org/releasenotes/neutron/ocata.html

thank you

Saverio



-- 
SWITCH
Saverio Proto, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
phone +41 44 268 15 15, direct +41 44 268 1573
saverio.pr...@switch.ch, http://www.switch.ch

http://www.switch.ch/stories

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] nova mitaka->newton online_data_migrations

[Saverio Proto]

Hello there,

before doing the upgrade, when still in Mitaka (tag 13.1.3), I am
running the folllwing command:

nova-manage db online_data_migrations

I get this output:

Option "verbose" from group "DEFAULT" is deprecated for removal.  Its
value may be silently ignored in the future.
Option "notification_topics" from group "DEFAULT" is deprecated. Use
option "topics" from group "oslo_messaging_notifications".
Running batches of 50 until complete

I always get the same output.
 I also tried :
nova-manage db online_data_migrations --max-count 5000

In this case I only get the warnings about the option without 'Running
batches of 50 until complete'.

When I do not use max-count I will always get 'Running batches of 50
until complete'.

How do I understand when all the online_data_migrations are done ?

I never get anything printed like 'all done'.

thank you

Saverio

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Migration to StoryBoard

[Saverio Proto]

Hello Kendall Nelson,

thanks for you email about Story Board. I did not understand if as
Openstack operators, are we supposed to open bugs on storyboard
instead of Launchpad ? When this starts ?

thank you

Saverio


2017-03-03 20:49 GMT+01:00 Kendall Nelson :
> Hello!
>
> As you may or may not know, OpenStack is in the process of migrating from
> using Launchpad as our task tracker to StoryBoard. This has been a long time
> effort and we are finally getting projects lined up to make the switch.
>
> To keep you all in the loop about the new tool, I wanted to take a moment to
> share with you the information we have been sharing with the projects
> themselves. We have a variety of blogposts we have been writing to help
> educate OpenStack users and developers about the differences between
> Launchpad[1][2] and Storyboard and posts about why we are making the
> switch[3].
>
> As storyboard is being worked on by OpenStack community members we always
> welcome feedback on the tool so if you are interested in playing around with
> the sandbox environment[4], want to share thoughts in one of our weekly
> meetings[5], are interested in the documentation[6], or just want to see the
> real thing and how its being used [7] please do!
>
> - Kendall Nelson (diablo_rojo) & the Storyboard team
>
> [1] https://storyboard-blog.io/things-that-storyboard-does-differently.html
> [2] https://storyboard-blog.io/mapping-launchpad-to-storyboard.html
> [3] https://storyboard-blog.io/why-storyboard-for-openstack.html
> [4] https://storyboard-dev.openstack.org/
> [5] https://wiki.openstack.org/wiki/StoryBoard
> [6] http://docs.openstack.org/infra/storyboard/
> [7] https://storyboard.openstack.org/
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Unable to launch instances because of glance errors

[Saverio Proto]

Everything LTGM

but if you do:

openstack image show Ubuntu-16.04

then you get again the error about the multiple names ?

if you go to the glance database and you do:

select * from images where name=''Ubuntu-16.04";

how many rows you have ?

after we collect this information we can try to understand if this is nova
or glance bug.

thank you

Saverio


2017-03-02 17:01 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:

> sure no problem:
>
> openstack image show d5d43ba0-82e9-43de-8883-5ebaf07bf3e3
> +--+
> --+
> | Field| Value
>
> |
> +--+
> --+
> | checksum | 6008d3fdf650658965b132b547416d
> 83
> |
> | container_format | bare
>
> |
> | created_at   | 2016-12-07T16:00:54Z
>
> |
> | disk_format  | raw
>
> |
> | file | /v2/images/d5d43ba0-82e9-43de-
> 8883-5ebaf07bf3e3/file
> |
> | id   | d5d43ba0-82e9-43de-8883-
> 5ebaf07bf3e3
> |
> | min_disk | 0
>
> |
> | min_ram  | 0
>
> |
> | name | Ubuntu-16.04
>
> |
> | owner| 4a6213a64312482896130efc304719
> 5c
> |
> | properties   | direct_url='rbd://742cd163-
> 32ac-4121-8060-d440aa7345d6/images/d5d43ba0-82e9-43de-8883-5ebaf07bf3e3/snap'
> |
> | protected| False
>
> |
> | schema   | /v2/schemas/image
>
> |
> | size | 2361393152
>
> |
> | status   | active
>
> |
> | tags |
>
> |
> | updated_at   | 2016-12-07T16:01:45Z
>
> |
> | virtual_size | None
>
> |
> | visibility   | public
>
> |
> +--+
> --+
>
> On 02/03/17 15:56, Saverio Proto wrote:
>
> Can you share with us the output of:
>
> openstack image show 
>
> for that image ?
>
> Saverio
>
>
> 2017-03-02 13:54 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:
>
>> Unfortunately not, I still get the same error.
>>
>> Grant
>>
>> On 02/03/17 12:54, Saverio Proto wrote:
>>
>> If you pass the uuid of the image does it work ?
>>
>> Saverio
>>
>> 2017-03-02 13:49 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:
>>
>>> Hi Saverio,
>>>
>>> We are running Mitaka - sorry forgot to mention that.
>>>
>>> Grant
>>>
>>> On 02/03/17 12:45, Saverio Proto wrote:
>>>
>>> What version of Openstack are we talking about ?
>>>
>>> Saverio
>>>
>>> 2017-03-02 12:11 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:
>>>
>>>> Hi All,
>>>>
>>>> Not sure if anyone can help, but as of today we are unable to launch
>>>> any instances and I have traced back the error to glance. Whenever I try
>>>> and launch an instance I get the below returned:
>>>>
>>>> openstack server create --flavor g1.small --image "Ubuntu-16.04"
>>>> --key-name ib-desktop --security-group default --nic
>>>> net-id=e2d925f0-63c1-442f-9158-6a138e5847ce gmtest
>>>> Could not find resource Ubuntu-16.04
>>>>
>>>> The image exists:
>>>>
>>>> openstack image list | grep Ubuntu-16.04
>>>>
>>>> | d5d43ba0-82e9-43de-8883-5ebaf07bf3e3 | Ubuntu-16.04   | active |
>>>>
>>>> The traceback gives a really odd message about Multiple choices ( All
>>>> images have unique names )
>>>>
>>>> Below is the stack trace - not sure if anyone has come across this
>>>> before? I am stumped at the moment.
>>>>
>>>> 2017-03-02 11:10:16.842 2107 INFO eventlet.wsgi.server [-]
>>>> 10.6.2.255,10.6.0.39 - - [02/Mar/2017 11:10:16] "GET
>>>> /images/detail?is_public=none=20 HTTP/1.1" 300 787 0.000769
>>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>>> [req-3b498a7e-2d6a-4337-a314-12abfbac0117
>>>> 4c91f07132454a97b21fff35402b7825 4a6213a64312482896130efc3047195c - -
>>>> -] Registry client request GET /images/detail raised MultipleChoices
>>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>>> Traceback (most recent call last):
>>>> 2017-03-02 11:10:16.843 2104 ERROR glance

Re: [Openstack-operators] Unable to launch instances because of glance errors

[Saverio Proto]

Can you share with us the output of:

openstack image show 

for that image ?

Saverio


2017-03-02 13:54 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:

> Unfortunately not, I still get the same error.
>
> Grant
>
> On 02/03/17 12:54, Saverio Proto wrote:
>
> If you pass the uuid of the image does it work ?
>
> Saverio
>
> 2017-03-02 13:49 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:
>
>> Hi Saverio,
>>
>> We are running Mitaka - sorry forgot to mention that.
>>
>> Grant
>>
>> On 02/03/17 12:45, Saverio Proto wrote:
>>
>> What version of Openstack are we talking about ?
>>
>> Saverio
>>
>> 2017-03-02 12:11 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:
>>
>>> Hi All,
>>>
>>> Not sure if anyone can help, but as of today we are unable to launch any
>>> instances and I have traced back the error to glance. Whenever I try and
>>> launch an instance I get the below returned:
>>>
>>> openstack server create --flavor g1.small --image "Ubuntu-16.04"
>>> --key-name ib-desktop --security-group default --nic
>>> net-id=e2d925f0-63c1-442f-9158-6a138e5847ce gmtest
>>> Could not find resource Ubuntu-16.04
>>>
>>> The image exists:
>>>
>>> openstack image list | grep Ubuntu-16.04
>>>
>>> | d5d43ba0-82e9-43de-8883-5ebaf07bf3e3 | Ubuntu-16.04   | active |
>>>
>>> The traceback gives a really odd message about Multiple choices ( All
>>> images have unique names )
>>>
>>> Below is the stack trace - not sure if anyone has come across this
>>> before? I am stumped at the moment.
>>>
>>> 2017-03-02 11:10:16.842 2107 INFO eventlet.wsgi.server [-]
>>> 10.6.2.255,10.6.0.39 - - [02/Mar/2017 11:10:16] "GET
>>> /images/detail?is_public=none=20 HTTP/1.1" 300 787 0.000769
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> [req-3b498a7e-2d6a-4337-a314-12abfbac0117 4c91f07132454a97b21fff35402b7825
>>> 4a6213a64312482896130efc3047195c - - -] Registry client request GET
>>> /images/detail raised MultipleChoices
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> Traceback (most recent call last):
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> File "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/
>>> glance/registry/client/v1/client.py", line 123, in do_request
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> **kwargs)
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> File 
>>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>>> line 70, in wrapped
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> return func(self, *args, **kwargs)
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> File 
>>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>>> line 373, in do_request
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> headers=copy.deepcopy(headers))
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> File 
>>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>>> line 87, in wrapped
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> return func(self, method, url, body, headers)
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> File 
>>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>>> line 534, in _do_request
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> raise exception.MultipleChoices(body=read_body(res))
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> MultipleChoices: The request returned a 302 Multiple Choices. This
>>> generally means that you have not included a version indicator in a request
>>> URI.
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client The
>>> body of response returned:
>>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>>> {"versions": [{"status&quo

Re: [Openstack-operators] Unable to launch instances because of glance errors

[Saverio Proto]

If you pass the uuid of the image does it work ?

Saverio

2017-03-02 13:49 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:

> Hi Saverio,
>
> We are running Mitaka - sorry forgot to mention that.
>
> Grant
>
> On 02/03/17 12:45, Saverio Proto wrote:
>
> What version of Openstack are we talking about ?
>
> Saverio
>
> 2017-03-02 12:11 GMT+01:00 Grant Morley <gr...@absolutedevops.io>:
>
>> Hi All,
>>
>> Not sure if anyone can help, but as of today we are unable to launch any
>> instances and I have traced back the error to glance. Whenever I try and
>> launch an instance I get the below returned:
>>
>> openstack server create --flavor g1.small --image "Ubuntu-16.04"
>> --key-name ib-desktop --security-group default --nic
>> net-id=e2d925f0-63c1-442f-9158-6a138e5847ce gmtest
>> Could not find resource Ubuntu-16.04
>>
>> The image exists:
>>
>> openstack image list | grep Ubuntu-16.04
>>
>> | d5d43ba0-82e9-43de-8883-5ebaf07bf3e3 | Ubuntu-16.04   | active |
>>
>> The traceback gives a really odd message about Multiple choices ( All
>> images have unique names )
>>
>> Below is the stack trace - not sure if anyone has come across this
>> before? I am stumped at the moment.
>>
>> 2017-03-02 11:10:16.842 2107 INFO eventlet.wsgi.server [-]
>> 10.6.2.255,10.6.0.39 - - [02/Mar/2017 11:10:16] "GET
>> /images/detail?is_public=none=20 HTTP/1.1" 300 787 0.000769
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> [req-3b498a7e-2d6a-4337-a314-12abfbac0117 4c91f07132454a97b21fff35402b7825
>> 4a6213a64312482896130efc3047195c - - -] Registry client request GET
>> /images/detail raised MultipleChoices
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> Traceback (most recent call last):
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> File "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/
>> glance/registry/client/v1/client.py", line 123, in do_request
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> **kwargs)
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> File 
>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>> line 70, in wrapped
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> return func(self, *args, **kwargs)
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> File 
>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>> line 373, in do_request
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> headers=copy.deepcopy(headers))
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> File 
>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>> line 87, in wrapped
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> return func(self, method, url, body, headers)
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> File 
>> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
>> line 534, in _do_request
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> raise exception.MultipleChoices(body=read_body(res))
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> MultipleChoices: The request returned a 302 Multiple Choices. This
>> generally means that you have not included a version indicator in a request
>> URI.
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client The
>> body of response returned:
>> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
>> {"versions": [{"status": "CURRENT", "id": "v2.3", "links": [{"href":
>> "http://10.6.0.3:9191/v2/; <http://10.6.0.3:9191/v2/>, "rel": "self"}]},
>> {"status": "SUPPORTED", "id": "v2.2", "links": [{"href":
>> "http://10.6.0.3:9191/v2/; <http://10.6.0.3:9191/v2/>, "rel": "self"}]},
>> {"status": "SUPPORTED", "id": "v2.1", "links": [{"href":
>> "http://10.6.0.3:9191/v2/; <http://10.6.0.3:9191/v2/>, "rel&

Re: [Openstack-operators] Unable to launch instances because of glance errors

[Saverio Proto]

What version of Openstack are we talking about ?

Saverio

2017-03-02 12:11 GMT+01:00 Grant Morley :

> Hi All,
>
> Not sure if anyone can help, but as of today we are unable to launch any
> instances and I have traced back the error to glance. Whenever I try and
> launch an instance I get the below returned:
>
> openstack server create --flavor g1.small --image "Ubuntu-16.04"
> --key-name ib-desktop --security-group default --nic
> net-id=e2d925f0-63c1-442f-9158-6a138e5847ce gmtest
> Could not find resource Ubuntu-16.04
>
> The image exists:
>
> openstack image list | grep Ubuntu-16.04
>
> | d5d43ba0-82e9-43de-8883-5ebaf07bf3e3 | Ubuntu-16.04   | active |
>
> The traceback gives a really odd message about Multiple choices ( All
> images have unique names )
>
> Below is the stack trace - not sure if anyone has come across this before?
> I am stumped at the moment.
>
> 2017-03-02 11:10:16.842 2107 INFO eventlet.wsgi.server [-]
> 10.6.2.255,10.6.0.39 - - [02/Mar/2017 11:10:16] "GET
> /images/detail?is_public=none=20 HTTP/1.1" 300 787 0.000769
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> [req-3b498a7e-2d6a-4337-a314-12abfbac0117 4c91f07132454a97b21fff35402b7825
> 4a6213a64312482896130efc3047195c - - -] Registry client request GET
> /images/detail raised MultipleChoices
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> Traceback (most recent call last):
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> File "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/
> glance/registry/client/v1/client.py", line 123, in do_request
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> **kwargs)
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> File 
> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
> line 70, in wrapped
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> return func(self, *args, **kwargs)
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> File 
> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
> line 373, in do_request
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> headers=copy.deepcopy(headers))
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> File 
> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
> line 87, in wrapped
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> return func(self, method, url, body, headers)
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> File 
> "/openstack/venvs/glance-13.3.8/lib/python2.7/site-packages/glance/common/client.py",
> line 534, in _do_request
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> raise exception.MultipleChoices(body=read_body(res))
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> MultipleChoices: The request returned a 302 Multiple Choices. This
> generally means that you have not included a version indicator in a request
> URI.
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client The
> body of response returned:
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> {"versions": [{"status": "CURRENT", "id": "v2.3", "links": [{"href":
> "http://10.6.0.3:9191/v2/; , "rel": "self"}]},
> {"status": "SUPPORTED", "id": "v2.2", "links": [{"href":
> "http://10.6.0.3:9191/v2/; , "rel": "self"}]},
> {"status": "SUPPORTED", "id": "v2.1", "links": [{"href":
> "http://10.6.0.3:9191/v2/; , "rel": "self"}]},
> {"status": "SUPPORTED", "id": "v2.0", "links": [{"href":
> "http://10.6.0.3:9191/v2/; , "rel": "self"}]},
> {"status": "SUPPORTED", "id": "v1.1", "links": [{"href":
> "http://10.6.0.3:9191/v1/; , "rel": "self"}]},
> {"status": "SUPPORTED", "id": "v1.0", "links": [{"href":
> "http://10.6.0.3:9191/v1/; , "rel": "self"}]}]}
> 2017-03-02 11:10:16.843 2104 ERROR glance.registry.client.v1.client
> 2017-03-02 11:10:16.844 2104 ERROR glance.common.wsgi
> [req-3b498a7e-2d6a-4337-a314-12abfbac0117 4c91f07132454a97b21fff35402b7825
> 4a6213a64312482896130efc3047195c - - -] Caught error: The request
> returned a 302 Multiple Choices. This generally means that you have not
> included a version indicator in a request URI.
>
> Any help would be great.
>
> Regards,
> --
> Grant Morley
> Cloud Lead
> Absolute DevOps Ltd
> Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
> www.absolutedevops.io gr...@absolutedevops.io  0845
> 874 0580
>
> ___
> OpenStack-operators mailing list
> 

Re: [Openstack-operators] [Large deployments] Neutron issues in Openstack Large deployment using DVR

[Saverio Proto]

Hello Satya,

I would fill a bug on launchpad for this issue.
114 VMs is not much. Can you identify how to trigger the issue to
reproduce it ? or it just happens randomly ?

When you say rebooting the network node, do you mean the server
running the neutron-server process ?

what version and distribution of openstack are you using ?

thank you

Saverio


2017-02-21 13:54 GMT+01:00 Satyanarayana Patibandla
:
> Hi All,
>
> We are trying to deploy Openstack in our production environment. For
> networking we are using DVR with out L3 HA. We are able to create 114 VMs
> with out any issue. After creating 114 VMs we are getting the below error.
>
> Error: 504 Gateway Time-out The server didn't respond
> in time. 
>
> Neutron services are getting freezed up due to a persistent lock on the
> agents table. it seems one of the network node is holding the lock on the
> table. After rebooting the network node, the Neutron CLI was responsive
> again.
>
> Neutron agent and neutron server is throwing below errors.
>
> Neutron-server errors:
> ERROR oslo_db.sqlalchemy.exc_filters "Can't reconnect until invalid "
> ERROR oslo_db.sqlalchemy.exc_filters InvalidRequestError: Can't reconnect
> until invalid transaction is rolled back
> ERROR neutron.api.v2.resource [req-24fa6eaa-a9e0-4f55-97e0-59db203e72c6
> 3eb776587c9c40569731ebe5c3557bc7 f43e8699cd5a46e89ffe39e3cac75341 - - -]
> index failed: No details.
> ERROR neutron.api.v2.resource DBError: Can't reconnect until invalid
> transaction is rolled back
>
>
> Neutron agents errors:
> MessagingTimeout: Timed out waiting for a reply to message ID
> 40638b6bf12c44cd9a404ecaa14a9909
>
> Could you please provide us your valuable inputs or suggestions for above
> errors.
>
> Thanks,
> Satya.P
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Instances are not creating after adding 3 additional nova nodes

[Saverio Proto]

Well,
I have no idea from this log file. Trying to make nova-compute more
verbose if you dont find anything in the logs

Saverio

2017-02-20 7:50 GMT+01:00 Anwar Durrani <durrani.an...@gmail.com>:
>
> On Thu, Feb 16, 2017 at 1:44 PM, Saverio Proto <ziopr...@gmail.com> wrote:
>>
>> openstack server show uuid
>
>
> Hi Saverio,
>
> I have investigated and progressed the case as per your saying, i got to
> know that instance was supposed to be launched on one of the nova node,
> where i dig and tried find out log as you mentioned, following output i have
> seen as below :
>
> tail -f /var/log/nova/nova-compute.log
>
> 2017-02-20 10:40:19.318 5365 WARNING nova.compute.manager
> [req-34fa4448-061d-44ad-b6e9-6ff0d1fd072f - - - - -] While synchronizing
> instance power states, found 4 instances in the database and 0 instances on
> the hypervisor.
>
> and
>
> other log where i have found following :
>
> tail -f /var/log/nova/nova-manage.log
>
> 2017-02-15 16:42:42.896 115003 TRACE nova   File
> "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 34, in
> sleep
>
> 2017-02-15 16:42:42.896 115003 TRACE nova hub.switch()
>
> 2017-02-15 16:42:42.896 115003 TRACE nova   File
> "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
>
> 2017-02-15 16:42:42.896 115003 TRACE nova return self.greenlet.switch()
>
> 2017-02-15 16:42:42.896 115003 TRACE nova   File
> "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 346, in run
>
> 2017-02-15 16:42:42.896 115003 TRACE nova self.wait(sleep_time)
>
>
> Thanks
>
>
>
> --
> Thanks & regards,
> Anwar M. Durrani
> +91-9923205011
>
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Instances are not creating after adding 3 additional nova nodes

[Saverio Proto]

The three new compute nodes that you added are empty, so most likely
the new instances are scheduled to those three (3 attempts) and
something goes wrong.

with admin rights do:
openstack server show uuid

this should give you the info about the compute node where the
instance was scheduled. Check the nova-compute.log of that one, you
should find some debug info.

Saverio


2017-02-16 8:40 GMT+01:00 Anwar Durrani :
> Hi team,
>
> I am running Kilo setup with 1 Controller and 7 nova nodes, i have added 3
> additional nova nodes, where i can see, in system information under compute
> nodes that nodes are up and running, but when i am trying to launch instance
> then it is prompting below error :
>
> Error: Failed to perform requested operation on instance "test", the
> instance has an error status: Please try again later [Error: No valid host
> was found. Exceeded max scheduling attempts 3 for instance
> 969e71d4-845a-40da-be2a-d4f2619cbc68. Last exception: [u'Traceback (most
> recent call last):\n', u' File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2248, in
> _do].
>
>
> Thanks in advance.
>
>
> --
> Thanks & regards,
> Anwar M. Durrani
> +91-9923205011
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators