FIPS compliance for Storm 2.4.0

[Aishwarya Soni]

Hello,

Could someone please let me know if Apache Storm version 2.4.0 is FIPS
(Federal Information Processing Standard) compliant? I did not find any
online info about this.

Regards,
Aishwarya Soni

Re: Apache storm 2.4.0 working slower than 0.9.6

[Aishwarya Soni]

I also observed that we do need more CPU power since 2.x versions. Try
increasing the supervisor CPU and its service memory/RAM. Since you are
giving 20Gb XmX for workers, we need supervisors to be in a much beefier
state to provide that heap. We are a streaming app and for us, the
supervisor CPU utilization can go close to 16 CPU cores+.

On Mon, Aug 28, 2023 at 5:13 PM v.s kadu  wrote:

> Hi All,
> I have figured out a way to deal with this.
> Just need to increase the parallelism hint of the component which is
> taking longer to process or it's Capacity is near to one in storm ui. This
> helped me to generate files faster.
>
> However another question which I have is regarding CPU utilization.
> I have executed my topologies with storm 0.9.6 and 2.4.0.
> Storm 0.9.6 uses ~30 % of 16 core RHEL  8 server to process.
> However storm 2.4.0 uses ~65% to process the same .
> The difference is huge.
> Please note we are emitting 1000 records from Spout and this is coded in
> java.
> Storm configurations used in both versions
>
> topology.max.spout.pending:200
> topology.message.timeout.secs:300
> storm.messaging transport:"backtype.storm.messaging.netty.Context"
> nimbus.task.timeout.secs:240
> nimbus.thrift.max_buffer_size:2048
> nimbus.childopts:"-Xmx2048m -Djava.net.preferIPv4Stack=true"
>
> worker. childopts:"-Xmx20g -XX:+UseG1GC -Djava.net.preferIPv4Stack=true"
>
> supervisor.childopts:"-Xmx2048m -Djava.net.preferIPv4Stack=true"
>
> How we could trace why storm 2.4.0 is taking such high CPU utilization?
> Any configuration changes needed for storm 2.4.0 while migrating from
> 0.9.6?
>
>
>
> Regards,
> Viraj Kadu
>
> On Thu, 24 Aug, 2023, 1:49 am v.s kadu,  wrote:
>
>> Hello Team,
>>
>> I am migrating to apache storm 2.4.0 from apache storm 0.9.6 But the
>> performance in apache storm 2.4.0 is slower.
>>
>> I read that wait strategy plays role in performance Exiting application
>> on apache storm 0.9.6 has following wait strategy configuration
>>
>> topology.spout.wait.strategy: backtype.storm.spout.SleepWaitStrategy
>> topology.disruptor.wait.strategy: com.lmax.disruptor.BlockingWaitStrategy
>> topology.disruptor.wait.timeout.millis: 1000
>> topology.sleep.spout.wait.strategy.time.ms: 1
>>
>> Tried tweaking wait strategy parameter in apache storm 2.4.0. Following
>> tests i had performed
>> 1.
>> Kept all defaults as it is
>>
>> 2.
>> topology.bolt.wait.progressive.level1.count:0
>> topology.bolt.wait.progressive.level2.count:0
>> topology.bolt.wait.progressive.level3.sleep.millis:0
>> topology.backpressure.wait.progressive.level1.count:0
>> topology.backpressure.wait.progressive.level2.count:0
>> topology.backpressure.wait.progressive.level3.sleep.millis:0
>>
>> 3.
>> topology.spout.wait.strategy:org.apache.storm.policy.WaitStrategy
>> topology.backpressure.wait.strategy:org.apache.storm.policy.WaitStrategy
>> topology.bolt.wait.strategy:org.apache.storm.policy.WaitStrategy
>> topology.bolt.wait.microsec:0
>>
>> But performance was lot slower in all scenarios. What other factors we
>> can check for performance testing while migrating? How to improve
>> performance in apache storm 2.4.0?
>>
>> Regards,
>> Viraj Kadu
>>
>

Re: Re: storm nimbus cannot be successfully started

[Aishwarya Soni]

Did you try cleaning up the old storm folder/znode inside the zookeeper and
deploying the upgraded build again?

On Tue, Aug 22, 2023 at 7:44 AM 姜林海  wrote:

> Hi Bipin,
>
> Yes, zookeeper is there and we are using version 3.6.4. everything is
> working well on storm 1.1.3.
>
>
>
> Thanks,
> Linhai
>
>
>
> At 2023-08-21 23:38:08, "Bipin Prasad via user" 
> wrote:
>
> Did you also deploy Zookeeper? If yes, what version?
>
>
> Sent from Yahoo Mail for iPhone
> 
>
> On Monday, August 21, 2023, 8:12 AM, 姜林海  wrote:
>
> Hi,
>
> i am upgrading storm from 1.1.3 to 2.4.0/2.5.0. I am deploying the storm
> in K8S using statefulset for all components, i.e. zookeeper, storm
> supervisor. Everything is good on storm 1.1.3. When i upgrade to 2.4.0 or
> 2.5.0. Nimbus cannot be started, and I can see only attached logs when
> starting nimbus. The command just stopped without any error. I am totally
> lost and need your help. Any tips are well appreciated.
>
> Thanks,
> Linhai
>
>
>
>