Re: effect of high IOWait on KStream app?

Could this delay be contributing to the various issues Im seeing? - extended / repeating rebalances - broker connection drops On Sat, Dec 17, 2016 at 10:27 AM, Eno Thereska wrote: > Yeah, the numbers for the streams tests seem to be low. For reference, > here is what I

Re: effect of high IOWait on KStream app?

Yeah, the numbers for the streams tests seem to be low. For reference, here is what I get when I run it on my laptop, with Kafka co-located (Macbook pro, 16GB, SSD). These are rounded up with no decimal places: > Producer Performance [MB/sec write]: 40 > Consumer Performance [MB/sec read]: 126

Re: effect of high IOWait on KStream app?

stateDir=/tmp/kafka-streams-simple-benchmark numRecords=1000 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/ec2-user/kafka/streams/build/dependant-libs-2.10.6/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found

Re: effect of high IOWait on KStream app?

I'd be happy to but the AWS AMI (default) i'm using is fighting this at every turn. Will keep trying. On Sat, Dec 17, 2016 at 2:46 AM, Eno Thereska wrote: > Jon, > > It's hard to tell. Would you be willing to run a simple benchmark and > report back the numbers? The

Re: effect of high IOWait on KStream app?

Jon, It's hard to tell. Would you be willing to run a simple benchmark and report back the numbers? The benchmark is called SimpleBenchmark.java, it's included with the source, and it will start a couple of streams apps. It requires a ZK and a broker to be up. Then you run it:

effect of high IOWait on KStream app?

Looking for reasons why my installations seem to be generating so many issues: Starting an app which is stream->aggregate->filter->foreach While it's running the system in question (AWS) averages >10% IOWait with spikes to 60-70%. The CPU load is in the range of 3/2/1 (8 core machine w/ 16G