Re: Updated scalable urandom patchkit

2015-10-21 Thread George Spelvin
> Can we just use my multi pool patch for now? It works and is actually scalable > and does not require any new "cryptographic research" or other risks. To clarify, it's actually quite easy to come up with something that works, cryptographically. All the discussion is: 1) Erring on the side of pu

Re: Updated scalable urandom patchkit

2015-10-21 Thread Andi Kleen
On Wed, Oct 21, 2015 at 04:27:43AM -0400, George Spelvin wrote: > After having worked out the basics of a non-blocking /dev/urandom read, > (patches poated; I'm hoping someone will comment on them!), I need to > work out the crypto. Can we just use my multi pool patch for now? It works and is ac

Re: Updated scalable urandom patchkit

2015-10-21 Thread George Spelvin
After having worked out the basics of a non-blocking /dev/urandom read, (patches poated; I'm hoping someone will comment on them!), I need to work out the crypto. Patch 3/4 of that series showed how to have concurrent pool readers not separated by mixback operations, but it was a stub, not really

Re: Updated scalable urandom patchkit

2015-10-13 Thread Andi Kleen
> Off hand, do you know how large a read each operation is? I want to > reduce mixback from once per 10 bytes to once per read, and the size > ratio will give me some idea of how large an improvement to expect. My test reads 64 bytes using the syscall. -Andi -- a...@linux.intel.com -- Speaking

Re: Updated scalable urandom patchkit

2015-10-13 Thread George Spelvin
> Ted's patch peaks at 350K with four CPUs, but then quickly degrades to > 50K ops/s at 20+ CPUs. At 144 CPUs it is slightly faster again at ~80K. Good to know, thanks! With its race conditions, it's basically a "best case" for that particular design, which tells me that more significant changes

Re: Updated scalable urandom patchkit

2015-10-13 Thread Andi Kleen
I tested the two proposed patches from earlier this thread on a 4S system. This was just with my (worst case) micro. Unfortunately both patches scale much worse than the duplicated pools, and can be even worse than the baseline (not sure why). The base line peaks at slightly above 200K ops/s wit

Re: Updated scalable urandom patchkit

2015-10-13 Thread George Spelvin
> This might be stupid, but could something asynchronous work? Perhaps > have the entropy generators dump their entropy into a central pool via > a cycbuf, and have a background kthread manage the per-cpu or > per-process entropy pools? No for two reasons: (Minor): One of the functions of the

Re: Updated scalable urandom patchkit

2015-10-12 Thread George Spelvin
> We're really getting into low-level implementations here, and I think > it's best to worry about these sorts of things when we have a patch to > review. > it's probably safer to simply not exit it. > I'm personally more inclined to keep it with the task struct, so that > different threads w

Re: Updated scalable urandom patchkit

2015-10-12 Thread Raymond Jennings
On Mon, Oct 12, 2015 at 7:46 PM, Theodore Ts'o wrote: On Mon, Oct 12, 2015 at 04:30:59PM -0400, George Spelvin wrote: > Segregating abusers solves both problems. If we do this then we don't > need to drop the locks from the nonblocking pool, which solves the > security problem. Er, sor

Re: Updated scalable urandom patchkit

2015-10-12 Thread Theodore Ts'o
On Mon, Oct 12, 2015 at 04:30:59PM -0400, George Spelvin wrote: > > Segregating abusers solves both problems. If we do this then we don't > > need to drop the locks from the nonblocking pool, which solves the > > security problem. > > Er, sort of. I still think my points were valid, but they're

Re: Updated scalable urandom patchkit

2015-10-12 Thread George Spelvin
(BTW, my previous e-mail went out early due to a mis-click. It was almost done, but please excuse any unfinished sentences.) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.o

Re: Updated scalable urandom patchkit

2015-10-12 Thread George Spelvin
Theodore Ts'o wrote: > On Mon, Oct 12, 2015 at 03:49:08AM -0400, George Spelvin wrote: >> Words 5..9 never got mixed at all. > Hmm, good point. It *shouldn't* matter if the hash is secure, but the > potentially uneven mixing would be a concern. I don't quite follow the "shouldn't matter" part.

Re: Updated scalable urandom patchkit

2015-10-12 Thread Theodore Ts'o
On Mon, Oct 12, 2015 at 03:49:08AM -0400, George Spelvin wrote: > > Suppose CPU 0 has an offset of 0, and CPU 1 has an offset of 12. > (I'll also use a count-up index even though I know it's actually > count-down in the code.) > > CPU0 writes 5 words at 0..4, leaving add_ptr = 5. > Then CPU 1 wri

Re: Updated scalable urandom patchkit

2015-10-12 Thread George Spelvin
>> I'm not thrilled with incrementing the pointer from i to len, but mixing >> at positions i+k to i+k+len. The whole LFSR scheme relies on a regular >> pass structure. > That part I'm not worried about. We still have a regular pass > structure --- since for each CPU, we are still iterating over

Re: Updated scalable urandom patchkit

2015-10-11 Thread Theodore Ts'o
On Sun, Oct 11, 2015 at 08:16:01PM -0400, George Spelvin wrote: > > I'm not thrilled with incrementing the pointer from i to len, but mixing > at positions i+k to i+k+len. The whole LFSR scheme relies on a regular > pass structure. That part I'm not worried about. We still have a regular pass s

Re: Updated scalable urandom patchkit

2015-10-11 Thread George Spelvin
TedTs'o wrote: > Yep, good catch; I need to subtract that off. I'm not thrilled with incrementing the pointer from i to len, but mixing at positions i+k to i+k+len. The whole LFSR scheme relies on a regular pass structure. How about this instead: drop the hashed offset, and instead let each writ

Re: Updated scalable urandom patchkit

2015-10-11 Thread Theodore Ts'o
On Sun, Oct 11, 2015 at 12:35:46AM -0400, George Spelvin wrote: > > One is the final write back of add_ptr on the last line of > _mix_pool_bytes. It actually writes i, which includes the per-cpu nonce, > and will have it jumping all over without the steady progression that > the mixing polynomial

Re: Updated scalable urandom patchkit

2015-10-10 Thread George Spelvin
Damn, I bow before the master. That is a much neater solution than mine; I had assumed a lock was required for writing. While it's good enough for benchmarking, there are a few leftover problems I mention so they don't get missed. One is the final write back of add_ptr on the last line of _mix_p

Re: Updated scalable urandom patchkit

2015-10-10 Thread Theodore Ts'o
On Sat, Oct 10, 2015 at 10:31:46PM -0400, Theodore Ts'o wrote: > To that end, Andi, can you try benchmarking the scalability of the > patch below? I really hope it will be good enough, since besides > using less memory, there are security advantages in not spreading the > entropy across N pools.

Re: Updated scalable urandom patchkit

2015-10-10 Thread Theodore Ts'o
On Sat, Oct 10, 2015 at 02:45:46PM -0400, George Spelvin wrote: > In general, fewer larger pools is better for entropy storage. The only > reason for the current three-pool design is to achieve strict isolation > of the /dev/random pool. You're absolutely right, and I would like to see if we can

Re: Updated scalable urandom patchkit

2015-10-10 Thread George Spelvin
I'm very very sorry to be so late to the party; I didn't see this thread until the week-delayed LWN article on the subject drew my attention to it. There's a fundamental design issue with this patch set that seems unnecessary to me: multiple nonblocking pools. I know, I know, that seems like the

Updated scalable urandom patchkit

2015-09-24 Thread Andi Kleen
Updated patchkit to make /dev/urandom scale on larger systesm. I addressed all the review comments. See ChangeLogs in the individual patches. -Andi -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info