Hi everyone, We've been banging our heads against the wall trying to get reasonable performance out of a small storm cluster.
Setup after stripping down trying to debug: - All servers on EC2 m3.larges - 2 Kestrel 2.4.1 queue servers - 3 Storm Servers (1 running ui + nimbus, all running supervisors and thus workers) - 2 workers per instance, workers get 2GB of RAM max - 1 topology with 2 KestrelSpouts We measure performance by doing the following: - loading up the queues with a couple million items in each - deploying the topology - pulling up the storm ui and tracking the changes in ack counts over time on the spouts to compute average throughputs With acking enabled on our spouts we were getting around 4k messages/second With acking disabled on our spouts, we were seeing around 6k messages/second Adding a few bolts with acking quickly bring performance down below 800 messages/second - pretty dreadful. Based on the reports many other people have posted about their Storm clusters, I find these numbers really disappointing. We've tried tuning the worker jvm options, number of workers/executors with this simple setup but could not squeeze anything more out. Does anyone have any further suggestions about where we should be looking? We are about set to pull storm out of production and roll our own processor. Thanks, Gary
