Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

;> >> Kris Lindgren >> >> Senior Linux Systems Engineer >> >> GoDaddy >> >> >> >> *From: *Dmitry Mescheryakov >> *Date: *Thursday, July 28, 2016 at 6:17 AM >> *To: *Sam Morrison >> *Cc: *OpenStack Operators >> *Subject

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

something that could contribute > to our rabbitmq issues. > > > > > > ___ > > Kris Lindgren > > Senior Linux Systems Engineer > > GoDaddy > > > > *From: *Dmitry Mescheryakov > *Date: *Thursday, July 28, 2016 at 6:17 AM

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

> On 28 Jul 2016, at 10:17 PM, Dmitry Mescheryakov > wrote: > > > > 2016-07-27 2:20 GMT+03:00 Sam Morrison >: > >> On 27 Jul 2016, at 4:05 AM, Dmitry Mescheryakov > > wrote: >> >> >> >> 2016-07-26 2:15 GMT+03:00 Sam Morrison >

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

. ___ Kris Lindgren Senior Linux Systems Engineer GoDaddy From: Dmitry Mescheryakov Date: Thursday, July 28, 2016 at 6:17 AM To: Sam Morrison Cc: OpenStack Operators Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty 2016-07-27 2:20 GMT+03:00 Sam Morrison

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

Kevin From: Davanum Srinivas [dava...@gmail.com] Sent: Thursday, July 28, 2016 5:31 AM To: Dmitry Mescheryakov Cc: Fox, Kevin M; OpenStack Operators Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty Dima, Kevin, There are PreStop

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

c: Sam Morrison; OpenStack Operators Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty 2016-07-26 21:20 GMT+03:00 Fox, Kevin M mailto:kevin@pnnl.gov>>: It only relates to Kubernetes in that Kubernetes can do automatic rolling upgrades by destroying/re

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

an up on shutdown in some > external place. > > Thanks, > > Dmitry > >> >> Thanks, >> Kevin >> >> From: Dmitry Mescheryakov [dmescherya...@mirantis.com] >> Sent: Tuesday, July 26, 2016 11:01 AM >> To: Fox, Kevin M >> Cc: Sam Mor

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

hanks, Dmitry > Thanks, > Kevin > -- > *From:* Dmitry Mescheryakov [dmescherya...@mirantis.com] > *Sent:* Tuesday, July 26, 2016 11:01 AM > *To:* Fox, Kevin M > *Cc:* Sam Morrison; OpenStack Operators > > *Subject:* Re: [Openstack-operators] [oslo]

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

2016-07-27 2:20 GMT+03:00 Sam Morrison : > > On 27 Jul 2016, at 4:05 AM, Dmitry Mescheryakov < > dmescherya...@mirantis.com> wrote: > > > > 2016-07-26 2:15 GMT+03:00 Sam Morrison : > >> The queue TTL happens on reply queues and fanout queues. I don’t think it >> should happen on fanout queues. The

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

> On 27 Jul 2016, at 4:05 AM, Dmitry Mescheryakov > wrote: > > > > 2016-07-26 2:15 GMT+03:00 Sam Morrison >: > The queue TTL happens on reply queues and fanout queues. I don’t think it > should happen on fanout queues. They should auto delete. I can understand the

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

t: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty 2016-07-25 18:47 GMT+03:00 Fox, Kevin M mailto:kevin@pnnl.gov>>: Ah. Interesting. The graceful shutdown would really help the Kubernetes situation too. Kubernetes can do easy rolling upgrades and

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

2016-07-26 2:15 GMT+03:00 Sam Morrison : > The queue TTL happens on reply queues and fanout queues. I don’t think it > should happen on fanout queues. They should auto delete. I can understand > the reason for having them on reply queues though so maybe that would be a > way to forward? > > Or am

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

> *From:* Dmitry Mescheryakov [dmescherya...@mirantis.com] > *Sent:* Monday, July 25, 2016 3:47 AM > *To:* Sam Morrison > *Cc:* OpenStack Operators > *Subject:* Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues > moving to Liberty > > Sam, >

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

The queue TTL happens on reply queues and fanout queues. I don’t think it should happen on fanout queues. They should auto delete. I can understand the reason for having them on reply queues though so maybe that would be a way to forward? Or am I missing something and it is needed on fanout que

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

1:27 PM To: OpenStack Operators Subject: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty We recently upgraded to Liberty and have come across some issues with queue build ups. This is due to changes in rabbit to set queue expiries as opposed to queue auto delete.

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

have to be added to all projects using it? Thanks, Kevin From: Dmitry Mescheryakov [dmescherya...@mirantis.com] Sent: Monday, July 25, 2016 3:47 AM To: Sam Morrison Cc: OpenStack Operators Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

I have filed a bug in oslo.messaging to track the issue [1] and my colleague Kirill Bespalov posted a fix for it [2]. We have checked the fix and it is working for neutron-server, l3-agent and dhcp-agent. It does not work for openvswitch-agent and metadata-agent meaning they do not stop RPC server

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

Sam, For your case I would suggest to lower rabbit_transient_queues_ttl until you are comfortable with volume of messages which comes during that time. Setting the parameter to 1 will essentially replicate bahaviour of auto_delete queues. But I would suggest not to set it that low, as otherwise yo

[Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

We recently upgraded to Liberty and have come across some issues with queue build ups. This is due to changes in rabbit to set queue expiries as opposed to queue auto delete. See https://bugs.launchpad.net/oslo.messaging/+bug/1515278 for more information. The fix for this bug is in liberty and