Hi, As I am migrating from 0.9.6 to 2.4.0. I have currently 10 boxes for 0.9.6 and I am migrating to 5 boxes for 2.4.0. So starting with i have figured out worker process time for 2.4.0 with 1 server is less than worker process time for 0.9.6 with 2 servers by adjusting confs as mentioned in above email. However when I am testing for 5 servers of 2.4.0 and 10 servers of 0.9.6 with confs it look longer. One thing I have observed is that CPU utl for 0.9.6 all servers are 50%+ . That means worker process is distributed across the cluster. CPU utl for 2.4.0 servers is very less. So I think worker process is not distributed across the cluster. Is my guess right? How we increase parallelism across cluster in apache storm 2.4.0?
Regards, Viraj Kadu On Mon, 28 Aug, 2023, 10:14 pm v.s kadu, <[email protected]> wrote: > Hi Aishwarya, > Yes that was helpful. > I have increased supervisor core and memory to 8 core and 8192 > respectively. > Also decreased worker Xmx to 8g as it is sufficient for that worker > process . > CPU utilization has come down drastically. > > Regards, > Viraj Kadu > > On Mon, 28 Aug, 2023, 7:04 pm Aishwarya Soni, <[email protected]> > wrote: > >> I also observed that we do need more CPU power since 2.x versions. Try >> increasing the supervisor CPU and its service memory/RAM. Since you are >> giving 20Gb XmX for workers, we need supervisors to be in a much beefier >> state to provide that heap. We are a streaming app and for us, the >> supervisor CPU utilization can go close to 16 CPU cores+. >> >> On Mon, Aug 28, 2023 at 5:13 PM v.s kadu <[email protected]> wrote: >> >>> Hi All, >>> I have figured out a way to deal with this. >>> Just need to increase the parallelism hint of the component which is >>> taking longer to process or it's Capacity is near to one in storm ui. This >>> helped me to generate files faster. >>> >>> However another question which I have is regarding CPU utilization. >>> I have executed my topologies with storm 0.9.6 and 2.4.0. >>> Storm 0.9.6 uses ~30 % of 16 core RHEL 8 server to process. >>> However storm 2.4.0 uses ~65% to process the same . >>> The difference is huge. >>> Please note we are emitting 1000 records from Spout and this is coded in >>> java. >>> Storm configurations used in both versions >>> >>> topology.max.spout.pending:200 >>> topology.message.timeout.secs:300 >>> storm.messaging transport:"backtype.storm.messaging.netty.Context" >>> nimbus.task.timeout.secs:240 >>> nimbus.thrift.max_buffer_size:20480000 >>> nimbus.childopts:"-Xmx2048m -Djava.net.preferIPv4Stack=true" >>> >>> worker. childopts:"-Xmx20g -XX:+UseG1GC -Djava.net.preferIPv4Stack=true" >>> >>> supervisor.childopts:"-Xmx2048m -Djava.net.preferIPv4Stack=true" >>> >>> How we could trace why storm 2.4.0 is taking such high CPU utilization? >>> Any configuration changes needed for storm 2.4.0 while migrating from >>> 0.9.6? >>> >>> >>> >>> Regards, >>> Viraj Kadu >>> >>> On Thu, 24 Aug, 2023, 1:49 am v.s kadu, <[email protected]> wrote: >>> >>>> Hello Team, >>>> >>>> I am migrating to apache storm 2.4.0 from apache storm 0.9.6 But the >>>> performance in apache storm 2.4.0 is slower. >>>> >>>> I read that wait strategy plays role in performance Exiting application >>>> on apache storm 0.9.6 has following wait strategy configuration >>>> >>>> topology.spout.wait.strategy: backtype.storm.spout.SleepWaitStrategy >>>> topology.disruptor.wait.strategy: >>>> com.lmax.disruptor.BlockingWaitStrategy >>>> topology.disruptor.wait.timeout.millis: 1000 >>>> topology.sleep.spout.wait.strategy.time.ms: 1 >>>> >>>> Tried tweaking wait strategy parameter in apache storm 2.4.0. Following >>>> tests i had performed >>>> 1. >>>> Kept all defaults as it is >>>> >>>> 2. >>>> topology.bolt.wait.progressive.level1.count:0 >>>> topology.bolt.wait.progressive.level2.count:0 >>>> topology.bolt.wait.progressive.level3.sleep.millis:0 >>>> topology.backpressure.wait.progressive.level1.count:0 >>>> topology.backpressure.wait.progressive.level2.count:0 >>>> topology.backpressure.wait.progressive.level3.sleep.millis:0 >>>> >>>> 3. >>>> topology.spout.wait.strategy:org.apache.storm.policy.WaitStrategy >>>> topology.backpressure.wait.strategy:org.apache.storm.policy.WaitStrategy >>>> topology.bolt.wait.strategy:org.apache.storm.policy.WaitStrategy >>>> topology.bolt.wait.microsec:0 >>>> >>>> But performance was lot slower in all scenarios. What other factors we >>>> can check for performance testing while migrating? How to improve >>>> performance in apache storm 2.4.0? >>>> >>>> Regards, >>>> Viraj Kadu >>>> >>>
