Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/2289#discussion_r135073224 --- Diff: examples/storm-loadgen/README.md --- @@ -0,0 +1,195 @@ +# Storm Load Generation Tools + +A set of tools to place an artificial load on a storm cluster to compare against a different storm cluster. This is particularly helpful when making changes to the data path in storm to see what if any impact the changes had. This is also useful for end users that want to compare different hardware setups to see what the trade-offs are, although actually running your real topologies is going to be more accurate. + +## Methodology +The idea behind all of these tools is to measure the trade-offs between latency, throughput, and cost when processing data using Apache Storm. + +When processing data you typically will know a few things. First you will know about how much data you are going to be processing. This will typically be a range of values that change throughput the day. You also will have an idea of how quickly you need the data processed by. Often this is measured in terms of the latency it takes to process data at the some percentile or set of percentiles. This is because of most use cases the value of the data declines over time, and being able to react to the data quickly is more valuable. You probably also have a budget for how much you are willing to spend to be able to process this data. There are always trade-offs in how quickly you can process some data and how efficiently you can processes that data both in terms of resource usage (cost) and latency. These tools are designed to help you explore that space. --- End diff -- Couple nits: 1. ``` Often this is measured in terms of the latency it takes to process data at the some percentile or set of percentiles. ``` `the some` 2. ```This is because of most use cases ...``` *in* most use cases? *for* most use cases?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---