On Thu, 6 Nov 2014 09:30:48 -0500 Tom Cameron <tcame...@dyn.com> wrote:
> But, I suspect the > cost of disassembling packets is higher than the math you're going to do. > This is made more true by the fact that you need shared (locked) structures > for your counts. i was thinking of having N workers, where every worker is responsible for a certain subset of the key space. e.g. using consistent hashing, the processing for all metric lines relating to an individual key requires no outside data. But at the same time, I think you're right that reading from UDP is the slowest aspect (which still surprises me) so this optimization would probably be pointless. > > Were these connections to be TCP, there would be more value. But since it's > UDP, which has no concept of a stream, I suspect it'll be a mostly wasted > effort. hmm, what do you mean with this? are udp packets slower to process than tcp? how so? i thought reassembling a tcp stream was more expensive. > Better to speed up the net library than try to distribute the work > and potentially have to drag a SK_buf across NUMA nodes. do you mean that the golang net library could be made more performant? thanks, Dieter _______________________________________________ Heka mailing list Heka@mozilla.org https://mail.mozilla.org/listinfo/heka
