We are using percentiles computed via Kamon 0.6.8. In a very low request rate environment like this, it takes roughly 1 super slow request/second to throw off the percentiles (which is what I think is happening).
On Mon, Oct 23, 2017 at 4:20 PM, Viktor Klang <viktor.kl...@gmail.com> wrote: > What definition of latency are you using? (i.e. how is it derived) > > On Mon, Oct 23, 2017 at 10:11 PM, Gary Malouf <malouf.g...@gmail.com> > wrote: > >> Hi Konrad, >> >> Our real issue is that we can not reproduce the results. The web server >> we are having latency issues with is under peak load of 10-15 >> requests/second - obviously not much to deal with. >> >> When we use load tests (https://github.com/apigee/apib), it's easy for >> us to throw a few thousand requests/second at it and get latencies in the ~ >> 3 ms range. We use kamon to track internal metrics - what we see is that >> our 95th and 99th percentiles only look bad under the production traffic >> but not under load tests. >> >> I've since used kamon to print out the actual requests trying to find any >> pattern in them to hint at what's wrong in my own code, but they seem to be >> completely random. What we do know is that downgrading to spray gets us >> 99.9th percentile latencies under 2ms, so something related to the upgrade >> is allowing this. >> >> Thanks, >> >> Gary >> >> On Tuesday, October 17, 2017 at 12:07:51 PM UTC-4, Konrad Malawski wrote: >>> >>> Step 1 – don’t panic ;-) >>> Step 2 – as I already asked for, please share actual details of the >>> benchmarks. It is not good to discuss benchmarks without any insight into >>> what / how exactly you’re measuring. >>> >>> -- >>> Cheers, >>> Konrad 'ktoso <http://kto.so>' Malawski >>> Akka <http://akka.io/> @ Lightbend <http://lightbend.com/> >>> >>> On October 12, 2017 at 15:31:19, Gary Malouf (malou...@gmail.com) wrote: >>> >>> We have a web service that we just finished migrating from spray 1.3 to >>> Akka-Http 10.0.9. While in most cases it is performing well, we are seeing >>> terrible 99th percentile latencies 300-450ms range) starting from a very >>> low request rate (10/second) on an ec2 m3.large. >>> >>> Our service does not do anything complicated - it does a few Map lookups >>> and returns a response to a request. In spray, even 99th percentile >>> latencies were on the order of 1-3 ms, so we are definitely concerned. >>> Connections as with many pixel-type servers are short-lived -> we actually >>> pass the Connection: Close header intentionally in our responses. >>> >>> Is there any obvious tuning that should be done on the server >>> configuration that others have found? >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >>> urrent/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/grou >>> p/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >> urrent/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Cheers, > √ > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to a topic in the > Google Groups "Akka User List" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/akka-user/-_C9jCPDwts/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
