Thanks Amnon & Hi Jesper, You're correct. Most of the time is actually spent in *ioutil.ReadAll()* . Trying to debug network issues related to GCP, GKE & the functionality of ISTIO sidecar reverse proxy, but seems okay so far couldn't find anything.
But, I've performed an interesting experiment. Tried the same API in Ruby with Sinatra by mirroring the traffic. When instrumented the API Response Times, all were <= 100 ms. Any clues on how to debug this?I can't seem to find much on inner workings of http.Request.Body -> io.ReadCloser Also, If we're are looking for flaky end-user connections i.e., mobile wireless devices etc., the connection termination is at reverse proxy itself and the lag in reading request shouldn't be observed at the application level. Correct me if I'm wrong. Regards On Thu, Apr 30, 2020 at 2:34 PM Jesper Louis Andersen < jesper.louis.ander...@gmail.com> wrote: > On Tue, Apr 28, 2020 at 6:48 PM Sarath Prabath Redlapalli Jaya < > harrysarath2...@gmail.com> wrote: > >> We've instrument the above code block and reading request body bytes is >> taking very long i.e., upto 5 secs sometimes reaching upto 20 seconds at the >> throughput of ~4-5K RPM >> >> >> The Request Body Size Metrics are as follows >> >> Average: 73190 Bytes >> 90th percentile: 170862 Bytes >> 99th percentile: 467638 Bytes >> >> > Let us assume we have 500 kilobytes of data. At 20 seconds processing > time, you are looking at something like 25 bytes per millisecond, which > seems rather low for a decent CPU. Try measuring the time it takes to > ioutil.ReadAll from the Reader I assume you have access to. My hunch is you > are spending most of the time there, and relatively little on decoding the > JSON data you have. Once you've done this, you will have some more > knowledge of where to look: the network, or the JSON decoder. > > If this is a web request, you should also look at how time it takes the > other end to feed data to the GCP process. Chances are that the problem is > in the other end, and you are looking at a slow sender, especially if you > have a large set of mobile devices out there with variety in their internet > connections. > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CADOH-f9mhKy0e3B9exCe8P3BNmgO5OMCXwGsNN1VjWSg3ofNqQ%40mail.gmail.com.
