Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi George On 30/05/15 02:01, George Shuklin wrote: >> Today we've discover a very serious bug in juno: >> https://bugs.launchpad.net/nova/+bug/1459726 >> >> In short: if you're using syslog, and restart rsyslog, all APIs >> processes will eventua

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

On 05/28/2015 07:56 PM, George Shuklin wrote: Hello. Today we've discover a very serious bug in juno: https://bugs.launchpad.net/nova/+bug/1459726 In short: if you're using syslog, and restart rsyslog, all APIs processes will eventually stuck with 100% CPU usage without doing anything. I

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

On 28 May 2015 at 22:45, Nick Jones wrote: > > On 28 May 2015, at 19:10, Tim Bell wrote: > > Using UDP is a great workaround but it does not feel like a fix... can't > the daemons realise that the syslog socket is not alive and reconnect. > Given it affects most of the OpenStack projects, a fix i

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

> On 28 May 2015, at 19:10, Tim Bell wrote: > > Using UDP is a great workaround but it does not feel like a fix... can't the > daemons realise that the syslog socket is not alive and reconnect. Given it > affects most of the OpenStack projects, a fix inside one of the oslo logging > libraries

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

> Using UDP is a great workaround but it does not feel like a fix... Additionally, it's not possible to use TLS/SSL with syslog and UDP -- TCP is required. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.opensta

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

> -Original Message- > From: Christian Schwede [mailto:cschw...@redhat.com] > Sent: 28 May 2015 20:03 > To: George Shuklin; openstack-operators@lists.openstack.org > Subject: Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted > > On 28.05.15 18:56,

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

On 28.05.15 18:56, George Shuklin wrote: > Today we've discover a very serious bug in juno: > https://bugs.launchpad.net/nova/+bug/1459726 > > In short: if you're using syslog, and restart rsyslog, all APIs > processes will eventually stuck with 100% CPU usage without doing > anything. > > Is a

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

Hi, As I commented on the bug, it sounds similar with: https://bugs.launchpad.net/ubuntu/+source/python-eventlet/+bug/1452312 https://github.com/eventlet/eventlet/issues/192 2015-05-29 1:56 GMT+09:00 George Shuklin : > Hello. > > Today we've discover a very serious bug in juno: > https://bugs.lau

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

I bet you can workaround this by switching to UDP remote syslog. You'd loose messages, but your processes *should* "fire and forget" > On May 28, 2015, at 9:56 AM, George Shuklin wrote: > > Hello. > > Today we've discover a very serious bug in juno: > https://bugs.launchpad.net/nova/+bug/1459

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

Hello, Yeah, I ran into it last fall: http://www.gossamer-threads.com/lists/openstack/operators/41876 Good to know that this issue still exists in Juno (we're still on Icehouse). Thanks for the note. :) Joe On Thu, May 28, 2015 at 10:56 AM, George Shuklin wrote: > Hello. > > Today we've disc

[Openstack-operators] 100% CPU and hangs if syslog is restarted

Hello. Today we've discover a very serious bug in juno: https://bugs.launchpad.net/nova/+bug/1459726 In short: if you're using syslog, and restart rsyslog, all APIs processes will eventually stuck with 100% CPU usage without doing anything. Is anyone hits this bug before? It looks like very