Re: [PATCH RFC net-next] net: epoll support for busy poll
On 26/08/2013 09:03, Eliezer Tamir wrote: > On 26/08/2013 00:30, Amir Vadai wrote: >> I'm on vacation, will test and have some inputs later this week when I be >> back. >> > Hello Amir, > > Ping me when you get back and I will send you my latest so you can > play with it. > Hi Eliezer, You can send the latest version, so I can play with it and get some numbers. Thanks, Amir -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
On 26/08/2013 00:30, Amir Vadai wrote: > I'm on vacation, will test and have some inputs later this week when I be > back. > Hello Amir, Ping me when you get back and I will send you my latest so you can play with it. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
Eliezer Tamir wrote: > I'm not sure I understand your claim that epoll has a higher latency > than poll. Nevermind, I usually live in an EPOLLONESHOT bubble so I use more epoll_ctl than "normal" epoll apps :x Anyways, good to know epoll wins over poll. Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
On 22/08/2013 23:11, Eric Wong wrote: > Eliezer Tamir wrote: >> Performance: >> using sockperf, Intel X520 NICs, >> Supermicro 6026TT-BTF systems with E5-2690 Xeon CPUs >> 100 UDP sockets avg. latency 5.756 (std-dev 0.510) >> 1k UDP sockets avg. latency 5.780 (std-dev 0.536) >> 10k UDP sockets avg. latency 6.269 (std-dev 0.611) > > How does this compare to with normal poll on this system? > > In other words, what advantage is there to using epoll instead of poll > when busy looping? > > epoll and busy_poll seem to be opposites. epoll inherently has higher > latency than normal poll, but provides stable performance with many more > FDs. I'm not sure I understand your claim that epoll has a higher latency than poll. Some quick (and very unscientific) testing seem to indicate that while poll is slightly faster than epoll when polling one socket, when polling on three sockets epoll is already slightly faster. (what's the point of using poll if you only need to monitor one socket?) With a larger socket set epoll is a clear winner. Both select and poll on 1k UDP sockets take over 120us on a similar system. Poll on 10 sockets takes about 6.8us, on 100 socket about 16.5us. Note that I'm using a slightly old NIC that has a high HW latency. For comparison, a busy poll on a socket read (using netperf) is about 5.7us. I expect Amir is seeing better numbers, but he's not telling. ;) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
On 22/08/2013 23:14, David Miller wrote: > From: Eliezer Tamir > Date: Wed, 21 Aug 2013 13:39:54 +0300 > >> +SK_LL_STATE_MISS_2, /* data came through napi poll twice in a row */ > > Please rename this to SK_LL_STATE_MISS_MULTI or something like that. > > Thanks. OK -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
From: Eliezer Tamir Date: Wed, 21 Aug 2013 13:39:54 +0300 > + SK_LL_STATE_MISS_2, /* data came through napi poll twice in a row */ Please rename this to SK_LL_STATE_MISS_MULTI or something like that. Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
Eliezer Tamir wrote: > Performance: > using sockperf, Intel X520 NICs, > Supermicro 6026TT-BTF systems with E5-2690 Xeon CPUs > 100 UDP sockets avg. latency 5.756 (std-dev 0.510) > 1k UDP sockets avg. latency 5.780 (std-dev 0.536) > 10k UDP sockets avg. latency 6.269 (std-dev 0.611) How does this compare to with normal poll on this system? In other words, what advantage is there to using epoll instead of poll when busy looping? epoll and busy_poll seem to be opposites. epoll inherently has higher latency than normal poll, but provides stable performance with many more FDs. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
On 21/08/2013 16:53, Eric Dumazet wrote: > On Wed, 2013-08-21 at 13:39 +0300, Eliezer Tamir wrote: > >> >> Instead of remembering the napi_id for all the sockets in an epoll, >> we only track the first socket we see with any unique napi_id. >> The rational for this is that while there may be many thousands of >> sockets tracked by a single epoll, we expect to only see a handful >> of unique napi_ids in most cases. > > This looks buggy assumption to me. > > We use epoll() with in the order of millions of fds per epoller, and > typically one napi_id per cpu. > > With your model, we would have to use nr_cpu epollers and make sure that > sockets are properly placed on those epollers by their napi_ids. There is no assumption here on how the user configures epoll. All we are doing is reducing the list of sockets monitored by an epoll instance to a smaller subset where each NAPI_ID appears only once. The user is free to have an epoll monitor sockets from queues associated to any CPU. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH RFC net-next] net: epoll support for busy poll
On Wed, 2013-08-21 at 13:39 +0300, Eliezer Tamir wrote: > > Instead of remembering the napi_id for all the sockets in an epoll, > we only track the first socket we see with any unique napi_id. > The rational for this is that while there may be many thousands of > sockets tracked by a single epoll, we expect to only see a handful > of unique napi_ids in most cases. This looks buggy assumption to me. We use epoll() with in the order of millions of fds per epoller, and typically one napi_id per cpu. With your model, we would have to use nr_cpu epollers and make sure that sockets are properly placed on those epollers by their napi_ids. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
