And there you have it - four separate sends for logically associated
data, when there should have been just the one sendmsg or writev call.
Given the sizes involved, that is roughly three to four times as much
CPU consumed to send that data is should have been - for "small"
packets you can handwave that the CPU cost is "per-packet" rather than
"per-byte" (OK, not _strictly_ true for say that 566 byte chunk, but
for the rest basically true)
We're kind of stuck using gsoap. We like the toolkit and have a lot of
code using it. Fortunately, none of our packets will be going on the
internet.
Doesn't really matter since the stack will still behave as it they might.
The same problem exists using the latest version of gsoap, so they are
still utilizing small packets. I am planning on mentioning this to them
occasionally on their list.
I would suggest reporting it as a _bug_ not just as a simple mention.
Sitting on my monitor is a 19" long piece of 2x2 calle the "HP Attitude
Adjustor" HP P/N 19511-80014 (yes, it actually as a part number !-) (*)
You are hereby licenced to duplicate that IP and employ it in a suitable
manner in getting this issue resolved.
We noticed an interesting dynamic in the way the TCP ACK delay is
applied to our transactions. The gsoap toolkit has an internal
"keep_alive" variable that regulates the number of transactions a client
can make to the server on a given connection. We've always used the
default of 100.
In the tcpdump, we see that the first 40 ms. ACK delay is applied just
after the TCP window ramps up on a new connection. We get additional ACK
delays applied for the next 22 transactions, then it stops--no more
delays are seen in the remaining 77 transactions. Then the keep_alive
has counted to zero, and the client is forced to reconnect to the
server, and the process repeats--23 ACK delays, then 77 unmolested
transactions.
Changing the keep_alive to 10,000 (we write all our clients, not worried
about abuse) drops the ACK delay count to about 27/13,000 transactions,
much better than the 3000 delays originally seen. Another option is to
sysctl tcp_abc off, if we have to.
It _would_ be very interesting to see if disabling tcp_abc does
workaround the problem.
I wonder if client applications are being charged for the initial SYN
packets, or something else odd is happening in the ABC algorithm as the
TCP window is growing?
If I've read everything correctly, the Linux TCP stack uses a "packet
based" congestion window. Virtually all of the RFC's are written to a
"byte based" congestion window. The mapping of the ABC byte-based cwnd
to the Linux packet-based cwnd has the cwnd increasing by one packet
only after an MSS-worth of data has been transferred across the connection.
The initial congestion window is (IIRC) 3 segments. Until the
congestion window can grow to accomodate those separate small sends per
transaction, you are stuck waiting for congestion window, which is only
available after the remot4e ACKs some traffic. Since the entire request
has not made it to the remote, the ACK only comes after the standalone
ACK timer expires.
Once enough transactions have gone across to get the cwnd grown, those
small sends are no longer held-up by the congestion window and so are
sent without further delay.
Once that application software decides to close and reopen the
connection, it starts with a fresh initial congestion window and the
entire process starts over again.
rick jones
on netdev, no need to cc me
---
John
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
