A somewhat common culprit seems to be the following case: 1. In order for the GC to switch from marking to sweeping, it needs all cores to agree. This requires a "barrier" in the system and thus we have to wait on all CPUs. 2. The barrier check happens on a function call. 3. A CPU core is currently executing a long-running operation inside a function, but is never calling another function in doing so. Usually there is a loop involved. 4. The system is now practically stalled.
There may be other reasons, but the above scheme is somewhat common. A workaround is to make sure you break long-running loops every N iterations and coerce the system to consider a garbage collection by returning from a function, say. Work is being done to automate this test so low latency can be achieved automatically without you having to intervene[0] To check for this assumption, set GODEBUG=gctrace=1 and look at the pauses of the different phases. It should tell you if this is the culprit or not. It could easily be something else, for instance pressure on the GC itself due to heavy allocation. [0] This will come at a slight cost to throughput, but it is probably a valid trade-off to make in a modern world of highly parallelized systems. On Thu, Apr 20, 2017 at 3:49 PM Lee Armstrong <lesm...@gmail.com> wrote: > See attached graph which shows the GC pauses of an application we have. > > I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 and > have a large map that is frequently accessed and items are removed and > added to it. These can be of some size. > > Is there a way to get these pauses down at all? Would forcing a GC() > after removing a batch of elements help at all? > > Alongside the pauses I see some substantial CPU usage showing up in traces > for the GC scan. > > Thanks in advance! > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
