Jun, Jay: Thanks for the info and the patch.
~Thai On Thu, Oct 27, 2011 at 12:54 AM, Jay Kreps <[email protected]> wrote: > Jun is correct about the request size. One thing you point out that is a > concern is that the send of the request size is actually turning into a > seperate packet (because we write it separately and have enabled tcpnodelay. > > I have implemented a patch which turns this into a single write using > vector i/o (and actually shortens code too). > > So far I don't see any perf improvement from this. Will post the results I > see so far. Let's move the discussion for a JIRA: > https://issues.apache.org/jira/browse/KAFKA-171 > > On Wed, Oct 26, 2011 at 12:12 AM, Bao Thai Ngo <[email protected]>wrote: > >> Hi, >> >> We have just made inspection on the message format and found some >> information unclear to us. Before the investigation, I did know that each >> Kafka message has an overhead of only 10 bytes. >> >> Below is what we have done with Kafka 0.7 RC2 on a CentOS 5.6 server: >> - start kafka server on port 4444 >> - tcpdump -i lo -w test.pcap port 4444 and host localhost -vvvvv >> - start kafka producer >> - have kafka producer send some sort of message (hello, aaa) to the server >> >> We found 2 important items: >> 1. Before sending an actual message, kafka producer do send a (control) >> message of 4 bytes to the server. Kafka producer always does this action >> before send some message to the server. >> 2. Sending a message of 15 bytes (10 bytes for overhead + 5 bytes for the >> message payload) from the producer to the server gives us an IP packet of 83 >> bytes (IP header: 20, TCP header: 32, Data: 31 bytes). The data of the IP >> packet is 31 instead of 15 bytes >> >> Could you please give us a detailed explanation about these 2 >> items? Please also see the attachment for further information. >> >> Thanks, >> ~Thai >> >> >> >> >> >
