On 21.07.2016 13:37, Ilya Maximets wrote:
>
>
> On 21.07.2016 13:13, Yuanhan Liu wrote:
>> On Thu, Jul 21, 2016 at 12:45:32PM +0300, Ilya Maximets wrote:
>>> On 21.07.2016 12:37, Yuanhan Liu wrote:
>>>> On Thu, Jul 21, 2016 at 11:21:15AM +0300, Ilya Maximets wrote:
>>>>> If something abnormal happened to QEMU, 'connect()' can block calling
>>>>> thread (e.g. main thread of OVS) forever or for a really long time.
>>>>> This can break whole application or block the reconnection thread.
>>>>>
>>>>> Example with OVS:
>>>>>
>>>>> ovs_rcu(urcu2)|WARN|blocked 512000 ms waiting for main to quiesce
>>>>> (gdb) bt
>>>>> #0 connect () from /lib64/libpthread.so.0
>>>>> #1 vhost_user_create_client (vsocket=0xa816e0)
>>>>> #2 rte_vhost_driver_register
>>>>> #3 netdev_dpdk_vhost_user_construct
>>>>> #4 netdev_open (name=0xa664b0 "vhost1")
>>>>> [...]
>>>>> #11 main
>>>>>
>>>>> Fix that by setting non-blocking mode for client sockets for connection.
>>>>>
>>>>> Fixes: 64ab701c3d1e ("vhost: add vhost-user client mode")
>>>>
>>>> Thanks for spotting and fixing yet another bug!
>>>>
>>>>>
>>>>> +static int
>>>>> +vhost_user_connect_nonblock(int fd, struct sockaddr *un, size_t sz)
>>>>
>>>> I don't quite understand why this is needed: connect() with O_NONBLOCK
>>>> flag set is not enough?
>>>
>>> There is a little issue with non-blocking connect() call. Connection
>>> establishing may be started but '-1' returned with 'errno = EINPROGRESS'.
>>> In this case we must wait on fd until it will be available for writing.
>>> After that we need to check current status of connection using getsockopt().
>>>
>>> I don't sure that we're able to get such situation, but it's documented,
>>> and, I think, we should handle it.
>>>
>>> See 'man connect' for details.
>>
>> I see. Thanks.
>>
>> But basically, I don't like the way of introduing yet another
>> fdset here. I'm wondering we could leverage current fdset code
>> to achieve that. This might need some work though.
>>
>> So how about making it simple and stupid at this stage: sleep a
>> while (maybe 1ms, or maybe 1s) when that happens, and give up
>> when the connection is still not established?
>
> Hmm, how about this fixup:
> ------------------------------------------------------------------------------
> diff --git a/lib/librte_vhost/vhost_user/vhost-net-user.c
> b/lib/librte_vhost/vhost_user/vhost-net-user.c
> index 8626d13..b0f45e6 100644
> --- a/lib/librte_vhost/vhost_user/vhost-net-user.c
> +++ b/lib/librte_vhost/vhost_user/vhost-net-user.c
> @@ -537,18 +537,7 @@ vhost_user_connect_nonblock(int fd, struct sockaddr *un,
> size_t sz)
> errno = EINVAL;
>
> ret = connect(fd, un, sz);
> - if (ret == -1 && errno != EINPROGRESS)
> - return -1;
> - if (ret == 0)
> - goto connected;
> -
> - FD_ZERO(&fdset);
> - FD_SET(fd, &fdset);
> -
> - ret = select(fd + 1, NULL, &fdset, NULL, &tv);
> - if (!ret)
> - errno = ETIMEDOUT;
> - if (ret != 1)
> + if (ret < 0 && errno != EISCONN)
> return -1;
>
> ret = getsockopt(fd, SOL_SOCKET, SO_ERROR, &so_error, &len);
> @@ -558,7 +547,6 @@ vhost_user_connect_nonblock(int fd, struct sockaddr *un,
> size_t sz)
> return -1;
> }
>
> -connected:
> flags = fcntl(fd, F_GETFL, 0);
> if (flags < 0) {
> RTE_LOG(ERR, VHOST_CONFIG,
> ------------------------------------------------------------------------------
> ?
>
> We will not check the EINPROGRESS, but subsequent 'connect()' will return
> EISCONN if connection already established. getsockopt() is kept just in
> case. Subsequent 'connect()' will happen on the next iteration of
> reconnection cycle (1 second sleep).
I've sent v2 with this changes.
Best regards, Ilya Maximets.
