I am using sarama "golang" kafka 1.8.1 client ( https://github.com/Shopify/sarama) to send messages to message queue once in 3secs and this task drives cpu consumption to 130% on the quad-cpu blade; The number stays this high regardless of number of partitions/consumers.... According to results of profiling with 'pprof', most of the cycles are spent in producer.go/NewBrokerProducer; Total: 47297 samples 5947 12.6% 12.6% 15013 31.7% selectgo 3617 7.6% 20.2% 3617 7.6% runtime.xchg 2819 6.0% 26.2% 28506 60.3% github.com/Shopify/sarama.func·008 <http://github.com/Shopify/sarama.func%C2%B7008> ---------------------------------------------------------------------- NewBrokerProducer ----------------------------------------------------------- 229: go func() { . . 230: timer := time.NewTimer(p.config.MaxBufferTime) . . 231: var shutdownRequired bool . . 232: wg.Done() . . 233: for { 208 17598 234: select { 153 749 235: case <-bp.flushNow: . . 236: if shutdownRequired = bp.flush(p); shutdownRequired { . . 237: goto shutdown . . 238: } 2230 3144 239: case <-timer.C: 144 3606 240: if shutdownRequired = bp.flushIfAnyMessages(p); shutdownRequired { . . 241: goto shutdown
wonder if there's some known issue with this method or if anyone already seen it before... Thank you,
