On Tue, Sep 28, 2010 at 05:01:21PM +0200, Lars Ellenberg wrote:
> # HG changeset patch
> # User Lars Ellenberg <l...@linbit.com>
> # Date 1285685263 -7200
> # Node ID beedb189b8a9c47c793f898755d0e301aacce4d4
> # Parent 94f4ee6025502c7ef710775d458ef766c51ad6db
> Medium: adjust socket buffers when adjusting ipc queue length
>
> If a channel adjusts its ipc queue length,
> it is probably a good idea to also change
> the socket buffers on the system level.
>
> The scaling qlen -> buffsize is somewhat arbitrary.
>
> diff -r 94f4ee602550 -r beedb189b8a9 lib/clplumbing/ipcsocket.c
> --- a/lib/clplumbing/ipcsocket.c Tue Sep 28 16:01:48 2010 +0200
> +++ b/lib/clplumbing/ipcsocket.c Tue Sep 28 16:47:43 2010 +0200
> @@ -1715,6 +1715,36 @@
> return socket_get_recv_fd(ch);
> }
>
> +static void
> +socket_adjust_buf(struct IPC_CHANNEL *ch, int optname, unsigned q_len)
> +{
> + const char *direction = optname == SO_SNDBUF ? "snd" : "rcv";
> + int fd = socket_get_send_fd(ch);
> + unsigned byte;
> +
> + /* Arbitrary scaling.
> + * DEFAULT_MAX_QLEN is 64, default socket buf is often 64k to 128k,
> + * at least on those linux I checked.
> + * Keep that ratio, and allow for some overhead. */
> + if (qlen == 0)
> + /* client does not want anything,
> + * reduce system buffers as well */
> + byte = 4096;
> + else if (q_len < 512)
> + byte = (32 + q_len) * 1024;
> + else
> + byte = q_len * 1024;
> +
> + if (0 == setsockopt(fd, SOL_SOCKET, optname, &byte, sizeof(byte))) {
> + cl_log(LOG_DEBUG, "adjusted %sbuf size to %u", direction,
> bytes);
> + } else {
> + /* If this fails, you may need to adjust net.core.rmem_max,
> + * ...wmem_max, or equivalent */
> + cl_log(LOG_DEBUG, "adjust %sbuf size to %u failed: %s",
> + direction, byte, strerror(errno));
Why not make this a higher severity? When do you expect
setsockopt to fail? Would it keep failing very often?
While we're at it, we'll need some kind of logging interface
which would issue a log message every now and then. Some messages
may under some circumstances be logged very often.
Dejan
> + }
> +}
> +
> static int
> socket_set_send_qlen (struct IPC_CHANNEL* ch, int q_len)
> {
> @@ -1722,9 +1752,9 @@
> if (ch->send_queue == NULL) {
> return IPC_FAIL;
> }
> + socket_adjust_buf(ch, SO_SNDBUF, q_len);
> ch->send_queue->max_qlen = q_len;
> - return IPC_OK;
> -
> + return IPC_OK;
> }
>
> static int
> @@ -1734,7 +1764,7 @@
> if (ch->recv_queue == NULL) {
> return IPC_FAIL;
> }
> -
> + socket_adjust_buf(ch, SO_RCVBUF, q_len);
> ch->recv_queue->max_qlen = q_len;
> return IPC_OK;
> }
> _______________________________________________________
> Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
> Home Page: http://linux-ha.org/
_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
