On 9/28/15 4:48 AM, Wei Fang wrote: > On 2015/9/25 2:06, William Allen Simpson wrote: >> Do any of these make a system call? Then they are not useful. >> This thread pool has been designed to have only one system call >> per invocation -- or better yet continue processing the next >> task without a system call. > > Yes, there has a system call in shed_setaffinity. But I don't understand > why they can't make a system call.
This is a userland program, the whole point is to run with limited system calls. Otherwise, we'd just use the Linux nfsd.... > Do you mean that it may lower > performance of threads? I think it depends on how much work the thread > is going to do. > There seems to be some confusion here. Every system call limits the performance. It has nothing what-so-ever to do with how much work the thread is going to do. If the thread has a little to do, you'll have serious latency issues and cap the ops-per-second. This is already the case, and we're trying to fix it. If the thread has a lot to do, the system calls may be amortized over the longer work load. But the latency remains. > I do have tested it on Linux, and obtained some performance benefits. > What's the increase in ops-per-second? What environment? > May those users use the same work pool simultaneously? I thought every > user has it's own pool, and every pool can set it's own CPU affinity > separately. > All users share this same work pool. So each connection will grab a worker. And each request over the connection will grab another worker, although that worker should continue using the same thread for many subsequent requests. What happened is you tested without RDMA (the primary user of this pool). Moreover, likely many requests were over one connection. So there was only one main user. But that's not a typical workload. >> Your current code has them oddly counting themselves. No idea >> how this would work. > > I count the threads created in the pool and use it to split the threads to > different CPUs. There's problem I mentioned on the TODO list in my patch, > and should do more work on this later. > Looks like you're depending on serendipity for affinity. As I mentioned in the next message, perhaps combined with Malahal's lanes gives us some affinity per connection. But my biggest objection is the additional system overhead. Pin the CPU assignment algorithmically at thread creation. Minimize system calls. ------------------------------------------------------------------------------ _______________________________________________ Nfs-ganesha-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
