- Forwarded message from Saso Kiselkov skiselkov...@gmail.com -
Date: Mon, 07 Oct 2013 23:18:21 +0100
From: Saso Kiselkov skiselkov...@gmail.com
To: z...@lists.illumos.org
CC: Zooko Wilcox-OHearn zo...@leastauthority.com
Subject: Re: [zfs] [Review] 4185 New hash algorithm support
Message-ID: 5253332d.6090...@gmail.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0)
Gecko/20130801 Thunderbird/17.0.8
Reply-To: z...@lists.illumos.org
On 10/7/13 10:17 PM, Zooko Wilcox-OHearn wrote:
Hi folks:
I just joined this list because I saw this thread. I'm one of the
architects of a distributed storage system named Tahoe-LAFS
(https://Tahoe-LAFS.org). It has quite a few things in common with
ZFS, architecturally, but it is also very different from ZFS, because
it's not so much a real *filesystem* as it is like a BitTorrent that
has an upload button as well as a download button. But it is like ZFS
inasmuch as they both involve a heck of a lot of hashing for
error-detection.
[..snip..]
Hi,
Thanks for joining, I'm glad we got somebody educated on the matters in
on the discussion! I studied BLAKE for inclusion instead of Skein, but
its performance was worse at the time. BLAKE2 is a different beast
(explained below).
So, before I go on with my pitch for why you should consider BLAKE2,
first please clarify for me whether ZFS really needs a
collision-resistant hash function, or whether it needs only a MAC. I
had thought until now that ZFS doesn't need a collision-resistant hash
unless dedup is turned on, and that if dedup is turned on it needs a
collision-resistant hash.
The reason is purely for dedup and pretty much nothing else. As such, we
only need a hash with a good pseudo-random output distribution and
collision resistance. We don't specifically need it to be super-secure.
The salted hashing support I added was simply to silence the endless
stream of wild hypotheticals on security.
But this thread seems to indicate that even when dedup is turned on,
it might be possible to use a MAC, by having a pool-wide secret to use
for the MAC key… If I understand correctly (which I probably don't),
that would make it impossible for anyone who doesn't know the secret
to cause collisions during dedup, but still possible for someone who
knows the secret (presumably root on that system, or someone who stole
the secret) to generate blocks that would collide during dedup. If you
used a collision-resistant hash for that purpose, then nobody would be
able to cause collisions.
Oh boy, I already regret having referred to the feature in the block
comments as a MAC... it's not and was never intended to be. It's really
just a way to mitigate fears of the potential exploitation of a
not-quite-secure hash function (Edon-R mostly). It is not meant to
authenticate data on the pool to the machine itself (how could it if we
store the key unencrypted on the pool anyway?), or protect us from
somebody who had physical access to the underlying pool. It's just to
prevent attacks on the dedup tables.
If you need a MAC, I suggest Poly1305-AES. It is very efficient, has a
nice proof that it is as secure as AES is, and it is part of a new
proposed cipher suite for TLS ².
² http://tools.ietf.org/html/draft-agl-tls-chacha20poly1305-01
Looked at it and looks fairly similar to the VMAC that Radio młodych
bandytów suggested. This is a MAC, whereas we need a hash (for dedup).
If you need a collision-resistant hash function, I suggest BLAKE2. It
is more efficient than SHA-256, Skein, or Keccak (see ³), and it has a
better reputation among cryptographers than Edon-R has. In fact,
BLAKE2's parent, BLAKE, was rated by NIST as being even more
well-studied than Keccak was — see my slides, linked above, for quotes
from NIST's final report on the SHA-3 contest.
[...snip...]
You can get the academic papers, source code (both simple reference
implementations and optimized implementations in various languages),
test vectors, and so on:
https://blake2.net
I tried BLAKE2 on my machine and it is true that it provides a slight
performance boost over Skein, though it doesn't even approach Edon-R.
For non-trivial reasons we can't use floating point registers in kernel
code without jumping through many hoops (so no SSE/AVX), plus we need to
be able to run on SPARC, but the pure 64-bit C implementation is a bit
faster than Skein, ~5.1 CPB for BLAKE2 vs ~5.9 CPB for Skein on my Core
i5 test rig. On an Athlon II 1.3 GHz it's actually a bit slower, at 7.75
CPB for BLAKE2 vs 7.45 CPB for Skein. Used the blake2b-ref.c
implementation, compiled GCC 4.4.4 with -fno-builtin -O2 -std=c99 to
mirror in-kernel build environment.
Overall, I'd say: I don't know. I've already sunk a fair amount of
effort to get Skein to work well and integrated with the KCF and ZFS and
I'm not exactly willing to jump in and expend more effort to get another
-0.1 to +0.7 CPB...
Cheers,
--
Saso
