Thanks for the suggestion - that hasn't improved matters, and in fact makes the CPU ramp-up quicker (60% CPU after 1 hour). This implies it is something to do with the batching mechanism.
Unfortunately, that code is quite complex, and so hard for us to work out exactly what is going on. We'll try and get a stand-alone testcase Simon From: Roshan Naik <[email protected]> Sent: 16 April 2018 10:06 To: [email protected] Subject: Re: Storm using all the CPU in disruptor-flush-task-pool Strange indeed. However since you have slow throughput scenario you could try to see if setting the disruptor batch size to 1 works around the issue. Sent from Yahoo Mail for iPhone<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Foverview.mail.yahoo.com%2F%3F.src%3DiOS&data=02%7C01%7Csimon.cooper%40featurespace.co.uk%7C14c551e72e9743b0f35308d5a3796a3a%7C19e863aab068484d9f9f990b545c5a0f%7C0%7C0%7C636594664232884141&sdata=gHMUGV3ct1gMPPadbWdPXnnS6iiT4r004m1lQEvSX6k%3D&reserved=0> On Monday, April 16, 2018, 1:50 AM, Simon Cooper <[email protected]<mailto:[email protected]>> wrote: Hi, We've been encountering a problem for some time now, where the storm CPU usage is continually going up, until it's using all the CPU on the system. This is despite the topology not really doing much - 1-2 tuples a second. We first noticed it on 1.1.0, but it has continued after upgrading to 1.2.1. We've isolated the problem to the disruptor-flush-task-pool. Changing the size of the pool doesn't really seem to change things; after a couple of days of 'idling', those threads are working all the time inside DisruptorQueue.ThreadLocalBatcher and DisruptorQueue.Flusher, checking _overflow.isEmpty() and running _batchers.values().iterator() Has anyone else seen similar problems? Is there likely to be a slow-burning task proliferation issue inside DisruptorQueue? Thanks, SimonC This message, and any files/attachments transmitted together with it, is intended for the use only of the person (or persons) to whom it is addressed. It may contain information which is confidential and/or protected by legal privilege. Accordingly, any dissemination, distribution, copying or use of this message, or any part of it or anything sent together with it, other than by intended recipients, may constitute a breach of civil or criminal law and is hereby prohibited. Unless otherwise stated, any views expressed in this message are those of the person sending it and not the sender's employer. No responsibility, legal or otherwise, of whatever nature, is accepted as to the accuracy of the contents of this message or for the completeness of the message as received. Anyone who is not the intended recipient of this message is advised to make no use of it and is requested to contact Featurespace Limited as soon as possible. Any recipient of this message who has knowledge or suspects that it may have been the subject of unauthorised interception or alteration is also requested to contact Featurespace Limited.
