subject:"\[RFC\] Preliminary BTRFS Encryption"

Re: [RFC] Preliminary BTRFS Encryption

2016-09-19 Thread Zygo Blaxell

On Mon, Sep 19, 2016 at 07:50:07PM +, Alex Elsayed wrote:
> > That would be true if the problem were not already long solved in btrfs.
> > The 32-bit CRC tree stores 4 bytes per block separately and efficiently.
> > With minor changes it can store a 32-byte HMAC for each block.
> 
> I disagree that this "solves" it - in particular, the fact that the fsck 
> tool support dropping/regenerating the extent tree is wildly unsafe in 
> the face of this.

Those fsck features should no longer work on the AEAD tree (or would
require the keys to work if there was enough filesystem left to salvage).

> For an AEAD that lacks nonce-misuse-resistance, it's "merely" downgrading 
> security from AEAD to simple encryption (GCM, for instance, becomes 
> exactly CTR). This would be almost okay (it's a fsck tool, after all), 
> but the fact that it's a fsck tool makes the next part worse.
> 
> In the case of nonce-misuse-resistant AEAD, it's much worse: Dropping the 
> checksum tree would permanently and irrevocably corrupt every single 
> extent, with no data recoverable at all. This is the _exact_ opposite of 
> _anything_ you would _ever_ want a fsck tool to do.

So...don't put those features in fsck?

In my experience, if you're dropping the checksum or especially the
extent tree, your filesystem is already so badly damaged you might as
well mkfs+restore the filesystem.  It'll take longer to reverify the
data at the application level or compare with the last backup.

An AEAD tree would just be like that, except there's no point in even
offering the option.  It would just be "rebuilding the AEAD tree will
erase all your encrypted data, leaving only plaintext data on the
filesystem if you had any, are you very sure about this y/N"

> This is, fundamentally, the problem with treating an "auth tag" as a 
> separate thing: It's only separate at all in weaker systems, and the act 
> of separating the data induces incredibly nasty failure modes.
> 
> It gets even worse if you consider _why_ that option exists for the fsck 
> tool: Because of the possibility that the _structure_ of the checksum 
> tree becomes corrupted. As a result, two bit-flips (one for each 
> duplicate of the metadata) would be entirely capable of irrevocably 
> destroying _all encrypted data on the FS_.

That event already destroys a btrfs filesystem, even without encryption.
btrfs already includes much of the verification process of a Merkle tree,
with weak checksums and no auth.  Currently, if you lose both copies of an
interior tree node, it is only possible to recover the filesystem offline
by brute-force search of the metadata.  It's one of the reasons why it's
so important to have duplicate metadata even on a single disk.

The only difference with encryption is that recovery would be
theoretically impossible instead of just practically infeasible.

> Separating the "auth tag" - simply considering an "auth tag" a separate 
> thing from the overall ciphertext - is a dangerous thing to do.
> 
> >> If you're _not_ using a nonce-misuse-resistant AEAD, it's even worse:
> >> keeping the tag out-of-band makes it far too easy to fail to verify it,
> >> or verify it only after decrypting the ciphertext to plaintext.
> >> Bluntly: that is an immediate security vulnerability.
> >> 
> >> tl;dr: Don't encrypt pages, encrypt extents. They grow a little for the
> >> auth tag, and that's fine.
> >> 
> >> Btrfs already handles needing to read the full extent in order to get a
> >> page out of it with compression, anyway.
> > 
> > It does, but compressed extents are limited to 128K.  Uncompressed
> > extents come in sizes up to 128M, far too large to read in their
> > entirety for many applications.
> 
> Er, yes, and? Just as compressed extents have a different cap for reasons 
> of practicality, so too can encrypted extents.

...which very inefficient space usage for short extents.

signature.asc
Description: Digital signature

Re: [RFC] Preliminary BTRFS Encryption

2016-09-19 Thread Alex Elsayed

On Mon, 19 Sep 2016 14:08:06 -0400, Zygo Blaxell wrote:

> On Sat, Sep 17, 2016 at 06:37:16AM +, Alex Elsayed wrote:
>> > Encryption in ext4 is a per-directory-tree affair. One starts by
>> > setting an encryption policy (using an ioctl() call) for a given
>> > directory, which must be empty at the time; that policy includes a
>> > master key used for all files and directories stored below the target
>> > directory. Each individual file is encrypted with its own key, which
>> > is derived from the master key and a per-file random nonce value
>> > (which is stored in an extended attribute attached to the file's
>> > inode). File names and symbolic links are also encrypted.
> 
> Probably the simplest way to map this to btrfs is to move the nonce from
> the inode to the extent.

I agree. Mostly, I was making a point about how the ext4/VFS code (which 
_does_ put it on the inode) can't just be transported over to btrfs 
unchanged, which is what I read Dave Chinner as advocating.

> Inodes aren't unique within a btrfs filesystem, extents can be shared by
> multiple inodes, and a single extent can appear multiple times in the
> same inode at different offsets.  Attaching the nonce to the inode would
> not be sufficient to read the extent in all but the special case of a
> single reference at the original offset where it was written, and it
> also leads to the replay problems with duplicate inodes you pointed out.

Yup.

> Extents in a btrfs filesystem are unique and carry their own attributes
> (e.g. compression format, checksums) and reference count.  They can
> easily carry a reference to an encryption policy object and a nonce
> attribute.

Definitely agreed.

> Nonces within metadata are more complicated.  btrfs doesn't have
> directory files like ext4 does, so it doesn't get directory filename
> encryption for free with file encryption.  Encryption could be done
> per-item in the metadata trees, but in the special case of directories
> that happen to the the roots of subvols, it would be possible to encrypt
> entire pages of metadata at a time (with the caveat that a snapshot
> would require shared encryption policy between the origin and snapshot
> subvols).

Encrypting tree values per-item is actually one of the best arguments in 
_favor_ of nonce-misuse-resistant AEAD. Its security notion is very, very 
strong:

If a (key, nonce, associated data, message) tuple is repeated, the only 
data an attacker can discover is the fact that the two ciphertexts have 
the same value (a one-bit leak).

In other words, if you encrypt each value in the b-tree with some key, 
some nonce, use the b-tree key as the associated data, and use the value 
as the message, you get a _very_ secure system against a _very_ wide 
variety of attacks - essentially for free. And all _without_ sacrificing 
flexibility, as one could use distinct (crypto) keys for distinct (b-
tree) keys.

(You still need something for protecting the _structure_ of the B-tree, 
but that's a different issue).

> This is what makes keys at the subvol root level so attractive.

Pretty much.

>> So there isn't quite a "subvol key" in the VFS approach - each
>> directory has a key, and there are derived keys for the entries below
>> it. (I'll note that this framing does not address shared extents _at
>> all_, and would love to have clarification on that).
> 
> Files are modified by creating new extents (using parameters inherited
> from the inode to fill in the extent attributes) and updating the inode
> to refer to the new extent instead of the old one at the modified
> offset. Cloned extents are references to existing extents associated
> with a different inode or at a different place within the same inode (if
> the extent is not compatible with the destination inode, clone fails
> with an error).  A snapshot is an efficient way to clone an entire
> subvol tree at once, including all inodes and attributes.

There is the caveat of chattr +C, which would need hard-disabled for 
extent-level encryption (vs block level).

> Inode attributes and extent attributes can sometimes conflict,
> especially during a clone operation.  Encryption attributes could become
> one of these cases (i.e. to prevent an extent from one encryption policy
> from being cloned to an inode under a different encryption policy).

That is a good approach.

>> > I don't see how snapshots could work, writable or otherwise, without
>> > separating the key identity from the subvol identity and having a
>> > many-to-one relationship between subvols and keys.  The extents in
>> > each subvol would be shared, and they'd be encrypted with a single
>> > secret, so there's not really another way to do this.
>> 
>> That's not the issue. The issue is that, assuming the key stays the
>> same,
>> then a user could quite possibly create a snapshot, write into both the
>> original and the snapshot, causing encryption to occur twice with the
>> same key, same nonce, and different data.
> 
> If the

Re: [RFC] Preliminary BTRFS Encryption

2016-09-19 Thread Alex Elsayed

On Mon, 19 Sep 2016 14:57:33 -0400, Zygo Blaxell wrote:

> On Sat, Sep 17, 2016 at 07:13:45AM +, Alex Elsayed wrote:
>> IMO, this is already a flawed framing - in particular, if encrypting at
>> the extent level, one _should not_ be encrypting (or authenticating)
>> individual pages. The meaningful unit is the extent, and encrypting at
>> page granularity puts you right back where dmcrypt is: dealing with
>> fixed-
>> size space, and needing to find somewhere else to put the auth tag.
>> 
>> This is not a good place to be, and I strongly suspect it motivated
>> choosing XTS in the first place - something I feel is an _error_ in the
>> long run, and a dangerous one. (IMO, anything _but_ AEAD should be
>> forbidden in FS-level encryption.)
>> 
>> In a nonce-misuse-resistent AEAD, there _is_ no auth tag: There's some
>> amount of inherent ciphertext expansion, and the ciphertext _cannot be
>> decrypted at all_ unless all of it is present. In essence, a built-in
>> all-
>> or-nothing transform.
>> 
>> You could, potentially, chop off part of that and store it elsewhere,
>> but now you're dealing with significant added complexity, for
>> absolutely zero gain.
> 
> That would be true if the problem were not already long solved in btrfs.
> The 32-bit CRC tree stores 4 bytes per block separately and efficiently.
> With minor changes it can store a 32-byte HMAC for each block.

I disagree that this "solves" it - in particular, the fact that the fsck 
tool support dropping/regenerating the extent tree is wildly unsafe in 
the face of this.

For an AEAD that lacks nonce-misuse-resistance, it's "merely" downgrading 
security from AEAD to simple encryption (GCM, for instance, becomes 
exactly CTR). This would be almost okay (it's a fsck tool, after all), 
but the fact that it's a fsck tool makes the next part worse.

In the case of nonce-misuse-resistant AEAD, it's much worse: Dropping the 
checksum tree would permanently and irrevocably corrupt every single 
extent, with no data recoverable at all. This is the _exact_ opposite of 
_anything_ you would _ever_ want a fsck tool to do.

This is, fundamentally, the problem with treating an "auth tag" as a 
separate thing: It's only separate at all in weaker systems, and the act 
of separating the data induces incredibly nasty failure modes.

It gets even worse if you consider _why_ that option exists for the fsck 
tool: Because of the possibility that the _structure_ of the checksum 
tree becomes corrupted. As a result, two bit-flips (one for each 
duplicate of the metadata) would be entirely capable of irrevocably 
destroying _all encrypted data on the FS_.

Separating the "auth tag" - simply considering an "auth tag" a separate 
thing from the overall ciphertext - is a dangerous thing to do.

>> If you're _not_ using a nonce-misuse-resistant AEAD, it's even worse:
>> keeping the tag out-of-band makes it far too easy to fail to verify it,
>> or verify it only after decrypting the ciphertext to plaintext.
>> Bluntly: that is an immediate security vulnerability.
>> 
>> tl;dr: Don't encrypt pages, encrypt extents. They grow a little for the
>> auth tag, and that's fine.
>> 
>> Btrfs already handles needing to read the full extent in order to get a
>> page out of it with compression, anyway.
> 
> It does, but compressed extents are limited to 128K.  Uncompressed
> extents come in sizes up to 128M, far too large to read in their
> entirety for many applications.

Er, yes, and? Just as compressed extents have a different cap for reasons 
of practicality, so too can encrypted extents.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-19 Thread Zygo Blaxell

On Sat, Sep 17, 2016 at 07:13:45AM +, Alex Elsayed wrote:
> IMO, this is already a flawed framing - in particular, if encrypting at 
> the extent level, one _should not_ be encrypting (or authenticating) 
> individual pages. The meaningful unit is the extent, and encrypting at 
> page granularity puts you right back where dmcrypt is: dealing with fixed-
> size space, and needing to find somewhere else to put the auth tag.
> 
> This is not a good place to be, and I strongly suspect it motivated 
> choosing XTS in the first place - something I feel is an _error_ in the 
> long run, and a dangerous one. (IMO, anything _but_ AEAD should be 
> forbidden in FS-level encryption.)
> 
> In a nonce-misuse-resistent AEAD, there _is_ no auth tag: There's some 
> amount of inherent ciphertext expansion, and the ciphertext _cannot be 
> decrypted at all_ unless all of it is present. In essence, a built-in all-
> or-nothing transform.
> 
> You could, potentially, chop off part of that and store it elsewhere, but 
> now you're dealing with significant added complexity, for absolutely zero 
> gain.

That would be true if the problem were not already long solved in btrfs.
The 32-bit CRC tree stores 4 bytes per block separately and efficiently.
With minor changes it can store a 32-byte HMAC for each block.

> If you're _not_ using a nonce-misuse-resistant AEAD, it's even worse: 
> keeping the tag out-of-band makes it far too easy to fail to verify it, 
> or verify it only after decrypting the ciphertext to plaintext. Bluntly: 
> that is an immediate security vulnerability.
> 
> tl;dr: Don't encrypt pages, encrypt extents. They grow a little for the 
> auth tag, and that's fine.
> 
> Btrfs already handles needing to read the full extent in order to get a 
> page out of it with compression, anyway.

It does, but compressed extents are limited to 128K.  Uncompressed extents
come in sizes up to 128M, far too large to read in their entirety for
many applications.



signature.asc
Description: Digital signature

Re: [RFC] Preliminary BTRFS Encryption

2016-09-19 Thread Zygo Blaxell

On Sat, Sep 17, 2016 at 06:37:16AM +, Alex Elsayed wrote:
> > Encryption in ext4 is a per-directory-tree affair. One starts by
> > setting an encryption policy (using an ioctl() call) for a given
> > directory, which must be empty at the time; that policy includes a
> > master key used for all files and directories stored below the target
> > directory. Each individual file is encrypted with its own key, which is
> > derived from the master key and a per-file random nonce value (which is
> > stored in an extended attribute attached to the file's inode). File
> > names and symbolic links are also encrypted.

Probably the simplest way to map this to btrfs is to move the nonce from
the inode to the extent.

Inodes aren't unique within a btrfs filesystem, extents can be shared
by multiple inodes, and a single extent can appear multiple times in the
same inode at different offsets.  Attaching the nonce to the inode would
not be sufficient to read the extent in all but the special case of a
single reference at the original offset where it was written, and it
also leads to the replay problems with duplicate inodes you pointed out.

Extents in a btrfs filesystem are unique and carry their own attributes
(e.g. compression format, checksums) and reference count.  They can easily
carry a reference to an encryption policy object and a nonce attribute.

Nonces within metadata are more complicated.  btrfs doesn't have directory
files like ext4 does, so it doesn't get directory filename encryption
for free with file encryption.  Encryption could be done per-item in the
metadata trees, but in the special case of directories that happen to
the the roots of subvols, it would be possible to encrypt entire pages
of metadata at a time (with the caveat that a snapshot would require
shared encryption policy between the origin and snapshot subvols).
This is what makes keys at the subvol root level so attractive.

> So there isn't quite a "subvol key" in the VFS approach - each directory 
> has a key, and there are derived keys for the entries below it. (I'll 
> note that this framing does not address shared extents _at all_, and 
> would love to have clarification on that).

Files are modified by creating new extents (using parameters inherited
from the inode to fill in the extent attributes) and updating the inode to
refer to the new extent instead of the old one at the modified offset.
Cloned extents are references to existing extents associated with a
different inode or at a different place within the same inode (if the
extent is not compatible with the destination inode, clone fails with
an error).  A snapshot is an efficient way to clone an entire subvol
tree at once, including all inodes and attributes.

Inode attributes and extent attributes can sometimes conflict, especially
during a clone operation.  Encryption attributes could become one of
these cases (i.e. to prevent an extent from one encryption policy from
being cloned to an inode under a different encryption policy).

> > I don't see how snapshots could work, writable or otherwise, without
> > separating the key identity from the subvol identity and having a
> > many-to-one relationship between subvols and keys.  The extents in each
> > subvol would be shared, and they'd be encrypted with a single secret,
> > so there's not really another way to do this.
> 
> That's not the issue. The issue is that, assuming the key stays the same, 
> then a user could quite possibly create a snapshot, write into both the 
> original and the snapshot, causing encryption to occur twice with the 
> same key, same nonce, and different data.

If the extents have nonces (and inodes do not) then this doesn't happen.
A write to either snapshot necessarily creates new extents in all cases
(the nodatacow feature, the only way to modify a data extent in-place,
is disabled when the extent is shared).

signature.asc
Description: Digital signature

Re: [RFC] Preliminary BTRFS Encryption

2016-09-18 Thread Anand Jain




On 09/18/2016 04:35 AM, David Sterba wrote:

On Fri, Sep 16, 2016 at 07:56:02PM +0800, Anand Jain wrote:




however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Also would like to mention that a review from the security experts is due,
which is important and I believe those review comments can be accommodated
without major changes from here.


I disagree. Others commented on the crypto stuff, I see enough points to
address that would lead to major changes.


Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.


Implementing the recent vfs encryption in btrfs is ok, it's just feature
parity using an existing API.



  As mentioned 'inline with vfs layer' I mean to say to use
  fs/crypto KPIs. Which I haven't seen what parts of the code
  from ext4 was made as generic KPIs. If that's getting stuff
  correct in the encryption related, I think it would here as well.


So you were not talking about the 'fs/crypto' that was merged in 4.6?


 Looks like I am out of sync here, looks like I miss understood,
  'Implementing the recent vfs encryption in btrfs is ok'
 I was ref to fs/crypto


  Internal to btrfs - I had challenges to get the extents encoding
  done properly without bailout, and the test plan. Which I think
  is addressed here in this code. as mentioned.


Sorry, I don't understand what you mean.


   basically making sure all the extents are really encoded, does not
   matter which crypto (unless like in compress where extents may not
   be encoded,in some situation) and having a test plan, now the test
   plan is same as
 mount option -o 'compress=ctr(aes)' with dummykey or dummy encrypt.
   for encryption.


 Thanks for integrating  most of the patches in the ML.


-Anand
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-17 Thread David Sterba

On Fri, Sep 16, 2016 at 07:56:02PM +0800, Anand Jain wrote:
> 
> 
> >> however here below is the quick example
> >> on the cli usage. Please try out, let me know if I have missed something.
> >>
> >> Also would like to mention that a review from the security experts is due,
> >> which is important and I believe those review comments can be accommodated
> >> without major changes from here.
> >
> > I disagree. Others commented on the crypto stuff, I see enough points to
> > address that would lead to major changes.
> >
> >> Also yes, thanks for the emails, I hear, per file encryption and inline
> >> with vfs layer is also important, which is wip among other things in the
> >> list.
> >
> > Implementing the recent vfs encryption in btrfs is ok, it's just feature
> > parity using an existing API.
> 
> 
>   As mentioned 'inline with vfs layer' I mean to say to use
>   fs/crypto KPIs. Which I haven't seen what parts of the code
>   from ext4 was made as generic KPIs. If that's getting stuff
>   correct in the encryption related, I think it would here as well.

So you were not talking about the 'fs/crypto' that was merged in 4.6?

>   Internal to btrfs - I had challenges to get the extents encoding
>   done properly without bailout, and the test plan. Which I think
>   is addressed here in this code. as mentioned.

Sorry, I don't understand what you mean.

> > And a note from me with maintainer's hat on, there are enough pending
> > patches and patchsets that need review, and bugs to fix, I'm not going
> > to spend time on something that we don't need at the moment if there are
> > alternatives.
> 
>   Honestly I agree. I even suggested but I had no choice.
> 
> 
> PS:
>   Pls, feel free to flame on the (raid) patches if its not correct,
>   because its rather more productive than no reply.

If it's the hot-spare and auto-replace feature, I've expressed my stance
in http://marc.info/?l=linux-btrfs=146252575330106, there hasn't
been any change. IMO the hot-spare feature makes most sense with the
raid56, which is stuck where it is, so we need to get it working first.

Lack of reply usually means lack of time (which I would not spend on
flaming but evaluation and review).
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-17 Thread Chris Murphy

On Sat, Sep 17, 2016 at 10:12 AM, Anand Jain  wrote:

>   btrfs keeps it only in-memory and key hash goes to the disk.
>   Further in the long we need an integration with key management
>   system as well.

Maybe LUKS2 is usable for this part, and still adaptable since it's
not finished yet? It looks to me like essentially unlimited keyslots
compared to the current 8. You don't really care about the dm-crypt
part of it, but the key management part of it, perhaps.

Both the original and new subvolumes initially share one DEK that go
with the shared encrypted extents, but upon snapshot happening the new
extents in each subvolume need their own DEK. Policy wise these DEKs
can be wrapped in the same or separate passphrases or KEKs, as there
could be hundreds or thousands of DEKs that apply to the many possible
shared encrypted extents in a subvolume. If that's true, then it's an
explosive number of keys per subvolume potentially. It doesn't depend
on space as much as it depends on fs lifetime

Otherwise I don't see how this is different than using a single DEK
across all company hard drives. Compromise one, you've compromised
them all.

-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-17 Thread David Sterba

On Sat, Sep 17, 2016 at 12:38:30AM -0400, Zygo Blaxell wrote:
> There's also a nasty problem with the extent tree--there's only one per
> filesystem, it's shared between all subvols and block groups, and every
> extent in that tree has back references to the (possibly encrypted) subvol
> trees.  I'll leave that problem as an exercise for other readers.  ;)

A design point that I'm not mentioning for the first time: there would
be per-subvolume group extent trees, ie. a set of subvolumes with
attached extent tree where similar to what we have now. So, encrypted
and unencrypted extent metadata will never be mixed.
(the crypto key questions are not addressed here)

This hasn't been implemented but I'm making sure this will be possible
when somebody mentions changes to the extent tree or blockgroup reworks
(to actually solve other problems).
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-17 Thread Anand Jain




Hi Eric,

 Thanks for the constructive feedback, pls see inline below.


On 09/17/2016 02:58 PM, Eric Biggers wrote:

On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:


This patchset adds btrfs encryption support.



Hi Anand,

I'm part of a team that will be maintaining and improving ext4 encryption.
Because f2fs now shares much of the code, it will benefit from the ext4
encryption work too.  It would be really nice if btrfs could be in the same
boat, since that would avoid redundant work and help prevent bugs and design
flaws.  So I strongly suggest that you explore how btrfs encryption might reuse
the existing code (and maybe extend it if there is very good reason to).


In fact my first attempt was using f2fs/ext4, found its too complicated,
further couldn't stable it, so re-wrote completely to a version where I
won't worry too much on the cipher mode _at the moment_, so this version
came with a caveat as mentioned.
Now looking to integrate with fs/crypto, however have the following 
concerns,


fs/crypto:
.
  fscrypt_context:master_key_descriptor[FS_KEY_DESCRIPTOR_SIZE];
  should it go to the disk ? its just a descriptor which might
  change at the user end and still may contain the right key
  in the payload.

  btrfs keeps it only in-memory and key hash goes to the disk.
  Further in the long we need an integration with key management
  system as well.

.
  ext4/f2fs allows per file keys, when we are talking about large
  data center FS for cloud services, it would need key services
  to scale along with the FS. And there will be a lot of resources
  which is allocated but not used.

.
 system keyring already defines struct user_key_payload
 probably we should have used it instead of

 71 struct fscrypt_key {
 72 u32 mode;
 73 u8 raw[FS_MAX_KEY_SIZE];
 74 u32 size;
 75 } __packed;


.
 Some of key derivation functions should have been part of
 crypto library rather.

.
 page->index based IV won't suite btrfs, so it uses a random, but yes
 it needs crypto hardened. I am kind of opinion that, for a real need
 of retrievable random number we are replacing it with sector(truecrypt)
 /page-offset number, I think we aren't addressing problem in a right
 way ? If that that's the best solution we could achieve, yet I am not
 sure how to solve the need of FS independent decrypt ? I am yet to
 look at gpg.

.
 Yes needs AEAD. But at the same time we need
   - MAC to be separated from the ciphertext and ciphertext-size
 == plaintext-size.
   To make sure for sync,dio we do create extents and IO which matches
   with the application IO. So that performance tuning will be things
   as usual.



There also needs to be a proper design document for btrfs encryption.  This is
especially true if for some (very, very good) reason you can't reuse the
infrastructure from ext4 and f2fs.  There also could be unique challenges for
btrfs such as encryption keys and/or IVs being reused in reflinked extents.

You will also not get a proper review without a proper design document which
details things like the threat model and the security properties provided.  But
I did take a short look at the code anyway because I was interested.


 Thank You !!  I should have sent the code when doc is ready rather.
 Sorry about that.



 The
results were not pretty.  As far as I can see the current proposal is fatally
flawed as it does not provide confidentiality of file contents against a basic
attack.

The main two flaws are:

1. Use of CTR mode of operation
2. Reuse of same (key, IV) pair for all pages of a given inode

CTR mode is well known to fall over completely when used with repeated (key, IV)
pairs.  This makes the encryption nearly worthless.  In more detail: suppose I
am an attacker who has access to a file's ciphertext.  By the definition of CTR
mode, each ciphertext block is given by: C = E(ctr) XOR P, where C and P denote
the ciphertext and plaintext blocks respectively, E denotes encryption with the
block cipher using the secret key, and 'ctr' denotes the counter value.  Due to
flaw (2) the ctr values repeat every page.  Consequently, if I can correctly
guess the plaintext P1 of *any* page in the file and I want to know the
plaintext P2 of some other page, I can trivially compute P2 = P1 XOR C1 XOR C2.
No secret key needed.

Essentially: if there is any part of a file which is easily guessable, such as
a header or even a zeroed region, then the whole file is revealed.


 Yes this will be fixed. No TFM is claimed to be btrfs default
 as of now.


The solution is to use a less brittle mode of operation such as XTS in
combination with per-page IVs (or "tweaks") and derived per-file keys.  This is
already done in ext4 and f2fs, where the per-page IV is just the page offset.
Note that per-file keys were needed to prevent the same (key, IV) pair from
being used in multiple places.  So if you could reuse the fs/crypto
infrastructure, you could take advantage of the fact that this problem

Re: [RFC] Preliminary BTRFS Encryption

2016-09-17 Thread Alex Elsayed

On Fri, 16 Sep 2016 23:58:31 -0700, Eric Biggers wrote:

> On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:
>>
>> This patchset adds btrfs encryption support.
>>
>>
> Hi Anand,

> Note: even better would be an authenticated encryption mode.  That isn't
> yet done by ext4 or f2fs --- I think because there wasn't a good place
> to store a per-page authentication tag.  It would be interesting to know
> whether this would be possible for btrfs.

IMO, this is already a flawed framing - in particular, if encrypting at 
the extent level, one _should not_ be encrypting (or authenticating) 
individual pages. The meaningful unit is the extent, and encrypting at 
page granularity puts you right back where dmcrypt is: dealing with fixed-
size space, and needing to find somewhere else to put the auth tag.

This is not a good place to be, and I strongly suspect it motivated 
choosing XTS in the first place - something I feel is an _error_ in the 
long run, and a dangerous one. (IMO, anything _but_ AEAD should be 
forbidden in FS-level encryption.)

In a nonce-misuse-resistent AEAD, there _is_ no auth tag: There's some 
amount of inherent ciphertext expansion, and the ciphertext _cannot be 
decrypted at all_ unless all of it is present. In essence, a built-in all-
or-nothing transform.

You could, potentially, chop off part of that and store it elsewhere, but 
now you're dealing with significant added complexity, for absolutely zero 
gain.

If you're _not_ using a nonce-misuse-resistant AEAD, it's even worse: 
keeping the tag out-of-band makes it far too easy to fail to verify it, 
or verify it only after decrypting the ciphertext to plaintext. Bluntly: 
that is an immediate security vulnerability.

tl;dr: Don't encrypt pages, encrypt extents. They grow a little for the 
auth tag, and that's fine.

Btrfs already handles needing to read the full extent in order to get a 
page out of it with compression, anyway.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-17 Thread Eric Biggers

On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:
>
> This patchset adds btrfs encryption support.
>

Hi Anand,

I'm part of a team that will be maintaining and improving ext4 encryption.
Because f2fs now shares much of the code, it will benefit from the ext4
encryption work too.  It would be really nice if btrfs could be in the same
boat, since that would avoid redundant work and help prevent bugs and design
flaws.  So I strongly suggest that you explore how btrfs encryption might reuse
the existing code (and maybe extend it if there is very good reason to).

There also needs to be a proper design document for btrfs encryption.  This is
especially true if for some (very, very good) reason you can't reuse the
infrastructure from ext4 and f2fs.  There also could be unique challenges for
btrfs such as encryption keys and/or IVs being reused in reflinked extents.

You will also not get a proper review without a proper design document which
details things like the threat model and the security properties provided.  But
I did take a short look at the code anyway because I was interested.  The
results were not pretty.  As far as I can see the current proposal is fatally
flawed as it does not provide confidentiality of file contents against a basic
attack.

The main two flaws are:

1. Use of CTR mode of operation
2. Reuse of same (key, IV) pair for all pages of a given inode

CTR mode is well known to fall over completely when used with repeated (key, IV)
pairs.  This makes the encryption nearly worthless.  In more detail: suppose I
am an attacker who has access to a file's ciphertext.  By the definition of CTR
mode, each ciphertext block is given by: C = E(ctr) XOR P, where C and P denote
the ciphertext and plaintext blocks respectively, E denotes encryption with the
block cipher using the secret key, and 'ctr' denotes the counter value.  Due to
flaw (2) the ctr values repeat every page.  Consequently, if I can correctly
guess the plaintext P1 of *any* page in the file and I want to know the
plaintext P2 of some other page, I can trivially compute P2 = P1 XOR C1 XOR C2.
No secret key needed.

Essentially: if there is any part of a file which is easily guessable, such as
a header or even a zeroed region, then the whole file is revealed.

The solution is to use a less brittle mode of operation such as XTS in
combination with per-page IVs (or "tweaks") and derived per-file keys.  This is
already done in ext4 and f2fs, where the per-page IV is just the page offset.
Note that per-file keys were needed to prevent the same (key, IV) pair from
being used in multiple places.  So if you could reuse the fs/crypto
infrastructure, you could take advantage of the fact that this problem was
already solved.

Note: even better would be an authenticated encryption mode.  That isn't yet
done by ext4 or f2fs --- I think because there wasn't a good place to store a
per-page authentication tag.  It would be interesting to know whether this would
be possible for btrfs.

I also noticed that unlike ext4 and f2fs, filenames and symlinks are not being
encrypted in btrfs.  I know it may seem somewhat ad-hoc that filenames are
encrypted but not other metadata, but apparently filenames were considered
quite important and a lot of work went into making it possible to encrypt them
in ext4.

(Apologies if I misunderstood anything.  The proposal would be easier to review
with a design document, as mentioned.)

Hope this is helpful,

Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-17 Thread Alex Elsayed

On Sat, 17 Sep 2016 00:38:30 -0400, Zygo Blaxell wrote:

> On Fri, Sep 16, 2016 at 06:49:53AM +, Alex Elsayed wrote:
>> The main issue I see is that subvolumes as btrfs has them _do_
>> introduce novel concerns - in particular, how should snapshots interact
>> with keying (and nonces)? None of the AEADs currently in the kernel are
>> nonce-misuse resistant, which means that if different data is encrypted
>> under the same key and nonce, things go _very_ badly wrong. With
>> writable snapshots, I'd consider that a nontrivial risk.
> 
> Snapshots should copy subvolume keys (or key UUIDs, since the keys
> aren't stored in the filesystem), i.e. an ioctl could say "create a new
> subvol 'foo' with the same key as existing subvol 'bar'".  This could
> also handle nested subvols (child copies key of parent) if the nested
> subvols weren't created with their own separate keys.  For snapshots,
> we wouldn't even ask--the snapshot and its origin subvol would share a
> key unconditionally. (*)

I'll quote the LWN article on the way EXT4 (and VFS) encryption works 
(https://lwn.net/Articles/639427/):

> Encryption in ext4 is a per-directory-tree affair. One starts by
> setting an encryption policy (using an ioctl() call) for a given
> directory, which must be empty at the time; that policy includes a
> master key used for all files and directories stored below the target
> directory. Each individual file is encrypted with its own key, which is
> derived from the master key and a per-file random nonce value (which is
> stored in an extended attribute attached to the file's inode). File
> names and symbolic links are also encrypted.

So there isn't quite a "subvol key" in the VFS approach - each directory 
has a key, and there are derived keys for the entries below it. (I'll 
note that this framing does not address shared extents _at all_, and 
would love to have clarification on that).

> I don't see how snapshots could work, writable or otherwise, without
> separating the key identity from the subvol identity and having a
> many-to-one relationship between subvols and keys.  The extents in each
> subvol would be shared, and they'd be encrypted with a single secret,
> so there's not really another way to do this.

That's not the issue. The issue is that, assuming the key stays the same, 
then a user could quite possibly create a snapshot, write into both the 
original and the snapshot, causing encryption to occur twice with the 
same key, same nonce, and different data.

This invalidates both the integrity and confidentialiyy of AES-GCM (and 
any other AEAD that is not nonce-misuse resistant), allowing them to 
effectively mount offline decryption attacks against things they could 
not ordinarily read, or replace files without being caught.

> If the key is immutable (which it probably is, given that it's used to
> encrypt at the extent level, and extents are (mostly) immutable) then
> just giving each subvol a copy of the key ID is sufficient.

Sufficient for reading data, yes. Sufficient for really nasty nonce-reuse 
attacks, also yes.

> (*) OK, we could ask, but if the answer was "no, please do not use the
> origin subvol's key", then btrfs would return EINVAL and not create the
> snapshot, since there would be no way to read any data contained within
> it without the key.

It _might_ be possible to get away with only allowing RO snapshots for 
encrypted subvols, but this really requires much more careful thought 
than it's getting.

>> > Indeed, with the generic file encryption, btrfs may not even need the
>> > special subvolume encryption pixies. i.e. you can effectively
>> > implement subvolume encryption via configuration of a multi-user
>> > encryption key for each subvolume and apply it to the subvolume tree
>> > root at creation time. Then only users with permission to unlock the
>> > subvolume key can access it.
> 
> Life is pretty easy when we're only encrypting data extents.

Agreed.

> Encrypted subvol trees cause quite a few problems for btrfs when it
> needs to relocate extents (e.g. to shrink a filesystem or change RAID
> profile) or validate data integrity.  Ideally it would still be able to
> do these operations without decrypting the data; otherwise, there are
> bad cases, e.g. if a disk fails, all of the subvolumes would have to be
> unlocked in order to replace a disk.

Sure; there are certainly caveats. At very least, the free space map 
would need to be either unencrypted, or encrypted under a "global" key 
(no worse in security properties than dmcrypt, though, with the added 
boon of AEAD). That would at least make operation with some subvolumes 
locked safe.

One could also just encrypt all of the subvolume trees under one global 
key. Encrypting _all_ the metadata in _some fashion_ is _required_ if we 
ever want dmcrypt to be unnecessary, but if that's the goal, per-
subvolume keys for that aren't needed (though they would be nice).

(If making dmcrypt unnecessary is a non-goal, then

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread Zygo Blaxell

On Fri, Sep 16, 2016 at 06:49:53AM +, Alex Elsayed wrote:
> The main issue I see is that subvolumes as btrfs has them _do_ introduce 
> novel concerns - in particular, how should snapshots interact with keying 
> (and nonces)? None of the AEADs currently in the kernel are nonce-misuse 
> resistant, which means that if different data is encrypted under the same 
> key and nonce, things go _very_ badly wrong. With writable snapshots, I'd 
> consider that a nontrivial risk.

Snapshots should copy subvolume keys (or key UUIDs, since the keys aren't
stored in the filesystem), i.e. an ioctl could say "create a new subvol
'foo' with the same key as existing subvol 'bar'".  This could also
handle nested subvols (child copies key of parent) if the nested
subvols weren't created with their own separate keys.  For snapshots,
we wouldn't even ask--the snapshot and its origin subvol would share a
key unconditionally. (*)

I don't see how snapshots could work, writable or otherwise, without
separating the key identity from the subvol identity and having a
many-to-one relationship between subvols and keys.  The extents in each
subvol would be shared, and they'd be encrypted with a single secret,
so there's not really another way to do this.

If the key is immutable (which it probably is, given that it's used to
encrypt at the extent level, and extents are (mostly) immutable) then just
giving each subvol a copy of the key ID is sufficient.

(*) OK, we could ask, but if the answer was "no, please do not use the
origin subvol's key", then btrfs would return EINVAL and not create
the snapshot, since there would be no way to read any data contained
within it without the key.

> > Indeed, with the generic file encryption, btrfs may not even need the
> > special subvolume encryption pixies. i.e. you can effectively implement
> > subvolume encryption via configuration of a multi-user encryption key
> > for each subvolume and apply it to the subvolume tree root at creation
> > time. Then only users with permission to unlock the subvolume key can
> > access it.

Life is pretty easy when we're only encrypting data extents.

Encrypted subvol trees cause quite a few problems for btrfs when it needs
to relocate extents (e.g. to shrink a filesystem or change RAID profile)
or validate data integrity.  Ideally it would still be able to do these
operations without decrypting the data; otherwise, there are bad cases,
e.g. if a disk fails, all of the subvolumes would have to be unlocked
in order to replace a disk.

Still, there could be a half way point here.  If btrfs could tie
block groups to subvol encryption keys, it could arrange for all of
the extents in a metadata block group to use the same encryption key.
Then it would be possible to relocate the entire metadata block group
without decrypting its contents.  It would only be necessary to copy
the block group's encrypted data, then update the virtual-to-physical
address mappings in the chunk tree.  Something would have to be done
about checksums during the copy but that's a larger question (are there
two sets of checksums, one authenticated for the encrypted data, and
the crc32 check for device-level data corruption?).

There's also a nasty problem with the extent tree--there's only one per
filesystem, it's shared between all subvols and block groups, and every
extent in that tree has back references to the (possibly encrypted) subvol
trees.  I'll leave that problem as an exercise for other readers.  ;)

signature.asc
Description: Digital signature

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread Zygo Blaxell

On Thu, Sep 15, 2016 at 10:24:02AM -0400, Austin S. Hemmelgarn wrote:
> On 2016-09-15 10:06, Anand Jain wrote:
> >>How does this handle cloning of extents?  Can extents be cloned across
> >>subvolume boundaries when one of the subvolumes is encrypted?
> >
> > Yes only if both the subvol keys match.
> OK, that makes sense.
> >
> >>Can they
> >>be cloned within an encrypted subvolume?
> >
> > Yes. That's things as usual.
> Glad to see that that still works.  Most people I know who do batch
> deduplication do so within subvolumes but not across them, so that still
> working with encrypted subvolumes is a good thing.

I do continual filesystem-wide deduplication across subvolumes, but I
don't think this is a problem.

There are already a number of conditions when IOC_FILE_EXTENT_SAME might
fail and deduplicators must tolerate those failures.  Cross-subvol dedup
has to loop over all duplicate block references (including those in
other subvols) until all references to one of the blocks are eliminated.
So dedup should still work by sheer brute force, banging extents together
until they stick, but it would be noisy and slower if it was not aware
of encrypted subvols.

If there's a way to look at the subvolume properties and figure out
whether the extents are clonable (e.g. equal key IDs == clonable) then
it should be easy to avoid submitting FILE_EXTENT_SAME extent pairs
belonging to incompatibly encrypted subvols.  They can also be stored in
separate DDT entries (e.g. by extending the hash field) so that blocks
from incompatibly encrypted subvols won't have matching extended hashes.

signature.asc
Description: Digital signature

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread Anand Jain





however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Also would like to mention that a review from the security experts is due,
which is important and I believe those review comments can be accommodated
without major changes from here.


I disagree. Others commented on the crypto stuff, I see enough points to
address that would lead to major changes.


Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.


Implementing the recent vfs encryption in btrfs is ok, it's just feature
parity using an existing API.



 As mentioned 'inline with vfs layer' I mean to say to use
 fs/crypto KPIs. Which I haven't seen what parts of the code
 from ext4 was made as generic KPIs. If that's getting stuff
 correct in the encryption related, I think it would here as well.

 Internal to btrfs - I had challenges to get the extents encoding
 done properly without bailout, and the test plan. Which I think
 is addressed here in this code. as mentioned.




And a note from me with maintainer's hat on, there are enough pending
patches and patchsets that need review, and bugs to fix, I'm not going
to spend time on something that we don't need at the moment if there are
alternatives.


 Honestly I agree. I even suggested but I had no choice.


PS:
 Pls, feel free to flame on the (raid) patches if its not correct,
 because its rather more productive than no reply.


Thanks, Anand
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread Anand Jain

On 09/16/2016 09:12 AM, Dave Chinner wrote:

On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:

This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Yup, that best practices say "do not roll your own encryption
infrastructure".

This is just my 2c worth - take it or leave it, don't other flaming.
Keep in mind that I'm not picking on btrfs here - I asked similar
hard questions about the proposed f2fs encryption implementation.
That was a "copy and snowflake" version of the ext4 encryption code -
they made changes and now we have generic code and common
functionality between ext4 and f2fs.

Also would like to mention that a review from the security experts is due,
which is important and I believe those review comments can be accommodated
without major changes from here.

That's a fairly significant red flag to me - security reviews need
to be done at the design phase against specific threat models -
security review is not a code/implementation review...

The ext4 developers got this right by publishing threat models and
design docs, which got quite a lot of review and feedback before
code was published for review.

https://docs.google.com/document/d/1ft26lUQyuSpiu6VleP70_npaWdRfXFoNnB8JYnykNTg/edit#heading=h.qmnirp22ipew

As mentioned 'inline with vfs layer' I mean to say to use
fs/crypto KPIs. Which I haven't seen what parts of the code
from ext4 was made as generic KPIs. If that's getting stuff
correct in the encryption related, I think it would here as well.

Internal to btrfs - I had challenges to get the extents encoding
done properly without bailout, and the test plan. Which I think
is addressed here in this code.

Thanks, Anand

[small reorder of comments]

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs, which is
easier for data center solution-ing, seamlessly persistent and easy to
manage.

We've got dmcrypt for this sort of transparent "device level"
encryption. Do we really need another btrfs layer that re-implements
generic, robust, widely deployed, stable functionality?

What concerns me the most here is that it seems like that nothing
has been learnt from the btrfs RAID5/6 debacle. i.e. the btrfs
reimplementation of existing robust, stable, widely deployed
infrastructure was fatally flawed and despite regular corruption
reports they were ignored for, what, 2 years? And then a /user/
spent the time to isolate the problem, and now several months later
it still hasn't been fixed. I haven't seen any developer interest in
fixing it, either.

This meets the definition of unmaintained software, and it sets a
poor example for how complex new btrfs features might be maintained
in the long term. Encryption simply cannot be treated like this - it
has to be right, and it has to be well maintained.

So what is being done differently ito the RAID5/6 review process
this time that will make the new btrfs-specific encryption
implementation solid and have minimal risk of zero-day fatal flaws?
And how are you going to guarantee that it will be adequately
maintained several years down the track?

Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.

The generic file encryption code is solid, reviewed, tested and
already widely deployed via two separate filesystems. There is a
much wider pool of developers who will maintain it, reveiw changes
and know all the traps that a new implementation might fall into.
There's a much bigger safety net here, which significantly lowers
the risk of zero-day fatal flaws in a new implementation and of
flaws in future modifications and enhancements.

Hence, IMO, the first thing to do is implement and make the generic
file encryption support solid and robust, not tack it on as an
afterthought for the magic btrfs encryption pixies to take care of.

Indeed, with the generic file encryption, btrfs may not even need
the special subvolume encryption pixies. i.e. you can effectively
implement subvolume encryption via configuration of a multi-user
encryption key for each subvolume and apply it to the subvolume tree
root at creation time. Then only users with permission to unlock the
subvolume key can access it.

Once the generic file encryption is solid and fulfils the needs of
most users, then you can look to solving the less common threat
models that neither dmcrypt or per-file encryption address. Only if
the generic code cannot be expanded to address specific threat
models should you then implement something that is unique to
btrfs

Cheers,

Dave.

--
To unsubscribe from this list: send the

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread Anand Jain




On 09/15/2016 07:47 PM, Alex Elsayed wrote:

On Thu, 15 Sep 2016 19:33:48 +0800, Anand Jain wrote:


Thanks for commenting. pls see inline below.

On 09/15/2016 12:53 PM, Alex Elsayed wrote:

On Tue, 13 Sep 2016 21:39:46 +0800, Anand Jain wrote:


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick
example on the cli usage. Please try out, let me know if I have missed
something.

Also would like to mention that a review from the security experts is
due,
which is important and I believe those review comments can be
accommodated without major changes from here.

Also yes, thanks for the emails, I hear, per file encryption and
inline with vfs layer is also important, which is wip among other
things in the list.

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs, which
is easier for data center solution-ing, seamlessly persistent and easy
to manage.


Steps:
-

Make sure following kernel TFMs are compiled in.
# cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
name : ctr(aes)
name : cbc(aes)


First problem: These are purely encryption algorithms, rather than AE
(Authenticated Encryption) or AEAD (Authenticated Encryption with
Associated Data). As a result, they are necessarily vulnerable to
adaptive chosen-ciphertext attacks, and CBC has historically had other
issues. I highly recommend using a well-reviewed AE or AEAD mode, such
as AES-GCM (as ecryptfs does), as long as the code can handle the
ciphertext being longer than the plaintext.

If it _cannot_ handle the ciphertext being longer than the plaintext,
please consider that a very serious red flag: It means that you cannot
provide better security than block-level encryption, which greatly
reduces the benefit of filesystem-integrated encryption. Being at the
extent level _should_ permit using AEAD - if it does not, something is
wrong.

If at all possible, I'd suggest _only_ permitting AEAD cipher modes to
be used.

Anyway, even for block-level encryption, CTR and CBC have been
considered obsolete and potentially dangerous to use in disk encryption
for quite a while - current recommendations for block-level encryption
are to use either a narrow-block tweakable cipher mode (such as XTS),
or a wide- block one (such as EME or CMC), with the latter providing
slightly better security, but worse performance.


   Yes. CTR should be changed, so I have kept it as a cli option. And
   with the current internal design, hope we can plugin more algorithms
   as suggested/if-its-outdated and yes code can handle (or with a
   little tweak) bigger ciphertext (than plaintext) as well.

   encryption + keyhash (as below) + Btrfs-data-checksum provides
   similar to AE,  right ?


No, it does not provide anything remotely similar to AE. AE requires
_cryptographic_ authentication of the data. Not only is a CRC (as Btrfs
uses for the data checksum) not enough, a _cryptographic hash_ (such as
SHA256) isn't even enough. A MAC (message authentication code) is
necessary.

Moreover, combining an encryption algorithm and a MAC is very easy to get
wrong, in ways that absolutely ruin security - as an example, see the
Vaudenay/Lucky13 padding oracle attacks on TLS.

In order for this to be secure, you need to use a secure encryption
system that also authenticates the data in a cryptographically secure
manner. Certain schemes are well-studied and believed to be secure - AES-
GCM and ChaCha20-Poly1305 are common and well-regarded, and there's a
generic security reduction for Encrypt-then-MAC constructions (using CTR
together with HMAC in such a construction is generally acceptable).

The Btrfs data checksum is wholly inadequate, and the keyhash is a non-
sequitur - it prevents accidentally opening the subvolume with the wrong
key, but neither it (nor the btrfs data checksum, which is a CRC rather
than a cryptographic MAC) protect adequately against malicious corruption
of the ciphertext.

I'd suggest pulling in Herbert Xu, as he'd likely be able to tell you
what of the Crypto API is actually sane to use for this.



 As mentioned 'inline with vfs layer' I mean to say to use
 fs/crypto KPIs. Which I haven't seen what parts of the code
 was made as generic KPIs from ext4. If that's solving the
 problem, then it would here as well.



Create encrypted subvolume.
# btrfs su create -e 'ctr(aes)' /btrfs/e1 Create subvolume '/btrfs/e1'
Passphrase:
Again passphrase:


I presume the command first creates a key, then creates a subvolume
referencing that key? If so, that seems sensible.


  Hmm I didn't get the why part, any help ? (this doesn't encrypt
  metadata part).


Basically, if your tool merely sets up an entry in the kernel keyring,
then calls the subvolume creation interface (passing in

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread Brendan Hide

For the most part, I agree with you, especially about the strategy being
backward - and file encryption being a viable more-easily-implementable
direction.

However, you are doing yourself a disservice to compare btrfs' features
as a "re-implementation" of existing tools. The existing tools cannot do
what btrfs' devs want to implement. See below inline.

On 09/16/2016 03:12 AM, Dave Chinner wrote:

On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:

This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Yup, that best practices say "do not roll your own encryption
infrastructure".

100% agreed

Also would like to mention that a review from the security experts is due,
which is important and I believe those review comments can be accommodated
without major changes from here.

That's a fairly significant red flag to me - security reviews need
to be done at the design phase against specific threat models -
security review is not a code/implementation review...

Also agreed. This is a bit backward.

The ext4 developers got this right by publishing threat models and
design docs, which got quite a lot of review and feedback before
code was published for review.

https://docs.google.com/document/d/1ft26lUQyuSpiu6VleP70_npaWdRfXFoNnB8JYnykNTg/edit#heading=h.qmnirp22ipew

[small reorder of comments]

We've got dmcrypt for this sort of transparent "device level"
encryption. Do we really need another btrfs layer that re-implements ...

[snip]
Woah, woah. This is partly addressed by Roman's reply - but ...

Subvolumes:
Subvolumes are not comparable to block devices. This thinking is flawed
at best; cancerous at worst.

As a user I tend to think of subvolumes simply as directly-mountable
folders.

As a sysadmin I also think of them as snapshottable/send-receiveable
folders.

And as a dev I know they're actually not that different from regular
folders. They have some extra metadata so aren't as lightweight - but of
course they expose very useful flexibility not available in a regular
folder.

MD/raid comparison:
In much the same way, comparing btrfs' raid features to md directly is
also flawed. Btrfs even re-uses code in md to implement raid-type
features in ways that md cannot.

I can't answer for the current raid5/6 stability issues - but I am
confident that the overall design is good, and that it will be fixed.

Hence, IMO, the first thing to do is implement and make the generic
file encryption support solid and robust, not tack it on as an
afterthought for the magic btrfs encryption pixies to take care of.

Agreed, this sounds like a far safer and achievable implementation process.

Cheers,

Dave.

--
__
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread David Sterba

On Thu, Sep 15, 2016 at 10:24:02AM -0400, Austin S. Hemmelgarn wrote:
> >> What happens when you try to
> >> clone them in either case if it isn't supported?
> >
> >  Gets -EOPNOTSUPP.
> That actually makes more sense than what my first thought for a return 
> code was (-EINVAL).

Should be -EXDEV, as we do already.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread David Sterba

On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:
> This patchset adds btrfs encryption support.
> 
> The main objective of this series is to have bugs fixed and stability.
> I have verified with fstests to confirm that there is no regression.
> 
> A design write-up is coming next,

You're approaching it from the wrong side. The detailed specification
must come first. Don't bother to send the code again.

> however here below is the quick example
> on the cli usage. Please try out, let me know if I have missed something.
> 
> Also would like to mention that a review from the security experts is due,
> which is important and I believe those review comments can be accommodated
> without major changes from here.

I disagree. Others commented on the crypto stuff, I see enough points to
address that would lead to major changes.

> Also yes, thanks for the emails, I hear, per file encryption and inline
> with vfs layer is also important, which is wip among other things in the
> list.

Implementing the recent vfs encryption in btrfs is ok, it's just feature
parity using an existing API.

And a note from me with maintainer's hat on, there are enough pending
patches and patchsets that need review, and bugs to fix, I'm not going
to spend time on something that we don't need at the moment if there are
alternatives.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-16 Thread Alex Elsayed

On Fri, 16 Sep 2016 11:12:13 +1000, Dave Chinner wrote:

> On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:
>> 
>> This patchset adds btrfs encryption support.
>> 
>> The main objective of this series is to have bugs fixed and stability.
>> I have verified with fstests to confirm that there is no regression.
>> 
>> A design write-up is coming next, however here below is the quick
>> example on the cli usage. Please try out, let me know if I have missed
>> something.
> 
> Yup, that best practices say "do not roll your own encryption
> infrastructure".

IMO, (some of) this _is_ substantively justified by subvolumes being a 
meaningful unit of isolation/separation. However, yes, other parts really 
should be using Things That Have Already Been Figured Out, such as AEAD.

> This is just my 2c worth - take it or leave it, don't other flaming.
> Keep in mind that I'm not picking on btrfs here - I asked similar hard
> questions about the proposed f2fs encryption implementation.
> That was a "copy and snowflake" version of the ext4 encryption code -
> they made changes and now we have generic code and common functionality
> between ext4 and f2fs.
> 
>> Also would like to mention that a review from the security experts is
>> due,
>> which is important and I believe those review comments can be
>> accommodated without major changes from here.
> 
> That's a fairly significant red flag to me - security reviews need to be
> done at the design phase against specific threat models -
> security review is not a code/implementation review...
> 
> The ext4 developers got this right by publishing threat models and
> design docs, which got quite a lot of review and feedback before code
> was published for review.
> 
> https://docs.google.com/document/
d/1ft26lUQyuSpiu6VleP70_npaWdRfXFoNnB8JYnykNTg/edit#heading=h.qmnirp22ipew
> 
> [small reorder of comments]
> 
>> As of now these patch set supports encryption on per subvolume, as
>> managing properties on per subvolume is a kind of core to btrfs, which
>> is easier for data center solution-ing, seamlessly persistent and easy
>> to manage.
> 
> We've got dmcrypt for this sort of transparent "device level"
> encryption. Do we really need another btrfs layer that re-implements
> generic, robust, widely deployed, stable functionality?

The reason we do, in four words: dmcrypt cannot use AEAD. Because it 
operates on blocks rather than extents, it is _incapable_ of providing 
the security advantages of AEAD, as those intrinsically cause ciphertext 
expansion.

> What concerns me the most here is that it seems like that nothing has
> been learnt from the btrfs RAID5/6 debacle. i.e. the btrfs
> reimplementation of existing robust, stable, widely deployed
> infrastructure was fatally flawed and despite regular corruption reports
> they were ignored for, what, 2 years? And then a /user/
> spent the time to isolate the problem, and now several months later it
> still hasn't been fixed. I haven't seen any developer interest in fixing
> it, either.

This is, fundamentally, not comparable to dmcrypt - this is not a 
reimplementation of the same tool, but a substantively different tool 
despite a similar goal in the _specific_ domain of "composability".

Because dm-crypt cannot use AEAD, it is incapable (as in, there's a 
nonexistence proof) of meeting the IND-CCA2 security notion. By operating 
on extents, this can.

> This meets the definition of unmaintained software, and it sets a poor
> example for how complex new btrfs features might be maintained in the
> long term. Encryption simply cannot be treated like this - it has to be
> right, and it has to be well maintained.

Entirely agreed - but dmcrypt does not do the job this aims to do, so the 
conversation needs to be reframed. This is, honestly, more like 
integrating a vastly more efficient ecryptfs, keyed on a per-subvolume 
basis, than dmcrypt - and needs to be evaluated as such.

> So what is being done differently ito the RAID5/6 review process this
> time that will make the new btrfs-specific encryption implementation
> solid and have minimal risk of zero-day fatal flaws?
> And how are you going to guarantee that it will be adequately maintained
> several years down the track?
> 
>> Also yes, thanks for the emails, I hear, per file encryption and inline
>> with vfs layer is also important, which is wip among other things in
>> the list.
> 
> The generic file encryption code is solid, reviewed, tested and already
> widely deployed via two separate filesystems. There is a much wider pool
> of developers who will maintain it, reveiw changes and know all the
> traps that a new implementation might fall into.
> There's a much bigger safety net here, which significantly lowers the
> risk of zero-day fatal flaws in a new implementation and of flaws in
> future modifications and enhancements.

This, I do agree with - I think it would be a good idea to start from the 
generic file encryption code. However it's fallacious to

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Roman Mamedov

On Fri, 16 Sep 2016 11:12:13 +1000
Dave Chinner  wrote:

> > As of now these patch set supports encryption on per subvolume, as
> > managing properties on per subvolume is a kind of core to btrfs, which is
> > easier for data center solution-ing, seamlessly persistent and easy to
> > manage.
> 
> We've got dmcrypt for this sort of transparent "device level"
> encryption. Do we really need another btrfs layer that re-implements
> generic, robust, widely deployed, stable functionality?

"Btrfs subvolume-level" is far from "device-level", subvolumes are so
lightweight and dynamic that they are akin to regular directories for most
intents and purposes, not devices or partitions.

And yes I'd say (effectively) a directory-level encryption in an FS can be
useful; for example encrypting /home, but not the rest of the filesystem, or
any other scenarios where only some of the stored data needs to be encrypted,
and it's not known in advance what proportion, so it's not convenient to have
any static partition or LVM based bounds.

Currently this can be achieved with tools like encfs or ecryptfs -- so it's
those you'd want to measure Btrfs encryption against, not dmcrypt.

-- 
With respect,
Roman

pgpDncVUCrA04.pgp
Description: OpenPGP digital signature

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Dave Chinner

On Tue, Sep 13, 2016 at 09:39:46PM +0800, Anand Jain wrote:
> 
> This patchset adds btrfs encryption support.
> 
> The main objective of this series is to have bugs fixed and stability.
> I have verified with fstests to confirm that there is no regression.
> 
> A design write-up is coming next, however here below is the quick example
> on the cli usage. Please try out, let me know if I have missed something.

Yup, that best practices say "do not roll your own encryption
infrastructure".

This is just my 2c worth - take it or leave it, don't other flaming.
Keep in mind that I'm not picking on btrfs here - I asked similar
hard questions about the proposed f2fs encryption implementation.
That was a "copy and snowflake" version of the ext4 encryption code -
they made changes and now we have generic code and common
functionality between ext4 and f2fs.

> Also would like to mention that a review from the security experts is due,
> which is important and I believe those review comments can be accommodated
> without major changes from here.

That's a fairly significant red flag to me - security reviews need
to be done at the design phase against specific threat models -
security review is not a code/implementation review...

The ext4 developers got this right by publishing threat models and
design docs, which got quite a lot of review and feedback before
code was published for review.

https://docs.google.com/document/d/1ft26lUQyuSpiu6VleP70_npaWdRfXFoNnB8JYnykNTg/edit#heading=h.qmnirp22ipew

[small reorder of comments]

> As of now these patch set supports encryption on per subvolume, as
> managing properties on per subvolume is a kind of core to btrfs, which is
> easier for data center solution-ing, seamlessly persistent and easy to
> manage.

We've got dmcrypt for this sort of transparent "device level"
encryption. Do we really need another btrfs layer that re-implements
generic, robust, widely deployed, stable functionality?

What concerns me the most here is that it seems like that nothing
has been learnt from the btrfs RAID5/6 debacle. i.e. the btrfs
reimplementation of existing robust, stable, widely deployed
infrastructure was fatally flawed and despite regular corruption
reports they were ignored for, what, 2 years? And then a /user/
spent the time to isolate the problem, and now several months later
it still hasn't been fixed. I haven't seen any developer interest in
fixing it, either.

This meets the definition of unmaintained software, and it sets a
poor example for how complex new btrfs features might be maintained
in the long term. Encryption simply cannot be treated like this - it
has to be right, and it has to be well maintained.

So what is being done differently ito the RAID5/6 review process
this time that will make the new btrfs-specific encryption
implementation solid and have minimal risk of zero-day fatal flaws?
And how are you going to guarantee that it will be adequately
maintained several years down the track?

> Also yes, thanks for the emails, I hear, per file encryption and inline
> with vfs layer is also important, which is wip among other things in the
> list.

The generic file encryption code is solid, reviewed, tested and
already widely deployed via two separate filesystems. There is a
much wider pool of developers who will maintain it, reveiw changes
and know all the traps that a new implementation might fall into.
There's a much bigger safety net here, which significantly lowers
the risk of zero-day fatal flaws in a new implementation and of
flaws in future modifications and enhancements.

Hence, IMO, the first thing to do is implement and make the generic
file encryption support solid and robust, not tack it on as an
afterthought for the magic btrfs encryption pixies to take care of.

Indeed, with the generic file encryption, btrfs may not even need
the special subvolume encryption pixies. i.e. you can effectively
implement subvolume encryption via configuration of a multi-user
encryption key for each subvolume and apply it to the subvolume tree
root at creation time. Then only users with permission to unlock the
subvolume key can access it.

Once the generic file encryption is solid and fulfils the needs of
most users, then you can look to solving the less common threat
models that neither dmcrypt or per-file encryption address. Only if
the generic code cannot be expanded to address specific threat
models should you then implement something that is unique to
btrfs

Cheers,

Dave.
-- 
Dave Chinner
da...@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Austin S. Hemmelgarn


On 2016-09-15 10:06, Anand Jain wrote:


Thanks for comments.
Pls see inline as below.

On 09/15/2016 07:37 PM, Austin S. Hemmelgarn wrote:

On 2016-09-13 09:39, Anand Jain wrote:


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick
example
on the cli usage. Please try out, let me know if I have missed
something.

Also would like to mention that a review from the security experts is
due,
which is important and I believe those review comments can be
accommodated
without major changes from here.

Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs,
which is
easier for data center solution-ing, seamlessly persistent and easy to
manage.


Steps:
-

Make sure following kernel TFMs are compiled in.
# cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
name : ctr(aes)
name : cbc(aes)

Create encrypted subvolume.
# btrfs su create -e 'ctr(aes)' /btrfs/e1
Create subvolume '/btrfs/e1'
Passphrase:
Again passphrase:

A key is created and its hash is updated into the subvolume item,
and then added to the system keyctl.
# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (594790215)

# keyctl show 594790215
Keyring
 594790215 --alsw-v  0 0  logon: btrfs:75197c8e


Now any file data extents under the subvol /btrfs/e1 will be
encrypted.

You may revoke key using keyctl or btrfs(8) as below.
# btrfs su encrypt -k out /btrfs/e1

# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (Required key not
available)

# keyctl show 594790215
Keyring
Unable to dump key: Key has been revoked

As the key hash is updated, If you provide wrong passphrase in the next
key in, it won't add key to the system. So we have key verification
from the day1.

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
ERROR: failed to set attribute 'btrfs.encrypt' to
'ctr(aes)@btrfs:75197c8e' : Key was rejected by service

ERROR: key set failed: Key was rejected by service

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'

Now if you revoke the key the read / write fails with key error.

# md5sum /btrfs/e1/2k-test-file
8c9fbc69125ebe84569a5c1ca088cb14  /btrfs/e1/2k-test-file

# btrfs su encrypt -k out /btrfs/e1

# md5sum /btrfs/e1/2k-test-file
md5sum: /btrfs/e1/2k-test-file: Key has been revoked

# cp /tfs/1k-test-file /btrfs/e1/
cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been
revoked

Plain text memory scratches for security reason is pending. As there
are some
key revoke notification challenges to coincide with encryption context
switch,
which I do believe should be fixed in the due course, but is not a
roadblock
at this stage.






Before I make any other comments, I should state that I asbolutely agree
with Alex Elsayed about the issues with using CBC or CTR mode, and not
supporting AE or AEAD modes.


  Alex comments was quite detailed, I did reply to it.
  Looks like you missed my reply to Alex's comments ?
I've been having issues with GMail delaying random e-mails for excessive 
amounts of time (hours sometimes), so I didn't see your reply before 
sending this.  Even so, I do want it on the record that I agree with him 
completely.



How does this handle cloning of extents?  Can extents be cloned across
subvolume boundaries when one of the subvolumes is encrypted?


 Yes only if both the subvol keys match.

OK, that makes sense.



Can they
be cloned within an encrypted subvolume?


 Yes. That's things as usual.
Glad to see that that still works.  Most people I know who do batch 
deduplication do so within subvolumes but not across them, so that still 
working with encrypted subvolumes is a good thing.



What happens when you try to
clone them in either case if it isn't supported?


 Gets -EOPNOTSUPP.
That actually makes more sense than what my first thought for a return 
code was (-EINVAL).

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Anand Jain



Thanks for comments.
Pls see inline as below.

On 09/15/2016 07:37 PM, Austin S. Hemmelgarn wrote:

On 2016-09-13 09:39, Anand Jain wrote:


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Also would like to mention that a review from the security experts is
due,
which is important and I believe those review comments can be
accommodated
without major changes from here.

Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs, which is
easier for data center solution-ing, seamlessly persistent and easy to
manage.


Steps:
-

Make sure following kernel TFMs are compiled in.
# cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
name : ctr(aes)
name : cbc(aes)

Create encrypted subvolume.
# btrfs su create -e 'ctr(aes)' /btrfs/e1
Create subvolume '/btrfs/e1'
Passphrase:
Again passphrase:

A key is created and its hash is updated into the subvolume item,
and then added to the system keyctl.
# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (594790215)

# keyctl show 594790215
Keyring
 594790215 --alsw-v  0 0  logon: btrfs:75197c8e


Now any file data extents under the subvol /btrfs/e1 will be
encrypted.

You may revoke key using keyctl or btrfs(8) as below.
# btrfs su encrypt -k out /btrfs/e1

# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (Required key not
available)

# keyctl show 594790215
Keyring
Unable to dump key: Key has been revoked

As the key hash is updated, If you provide wrong passphrase in the next
key in, it won't add key to the system. So we have key verification
from the day1.

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
ERROR: failed to set attribute 'btrfs.encrypt' to
'ctr(aes)@btrfs:75197c8e' : Key was rejected by service

ERROR: key set failed: Key was rejected by service

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'

Now if you revoke the key the read / write fails with key error.

# md5sum /btrfs/e1/2k-test-file
8c9fbc69125ebe84569a5c1ca088cb14  /btrfs/e1/2k-test-file

# btrfs su encrypt -k out /btrfs/e1

# md5sum /btrfs/e1/2k-test-file
md5sum: /btrfs/e1/2k-test-file: Key has been revoked

# cp /tfs/1k-test-file /btrfs/e1/
cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been
revoked

Plain text memory scratches for security reason is pending. As there
are some
key revoke notification challenges to coincide with encryption context
switch,
which I do believe should be fixed in the due course, but is not a
roadblock
at this stage.






Before I make any other comments, I should state that I asbolutely agree
with Alex Elsayed about the issues with using CBC or CTR mode, and not
supporting AE or AEAD modes.


  Alex comments was quite detailed, I did reply to it.
  Looks like you missed my reply to Alex's comments ?


How does this handle cloning of extents?  Can extents be cloned across
subvolume boundaries when one of the subvolumes is encrypted?


 Yes only if both the subvol keys match.


Can they
be cloned within an encrypted subvolume?


 Yes. That's things as usual.


What happens when you try to
clone them in either case if it isn't supported?


 Gets -EOPNOTSUPP.

Thanks, Anand

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Alex Elsayed

On Thu, 15 Sep 2016 19:33:48 +0800, Anand Jain wrote:

> Thanks for commenting. pls see inline below.
> 
> On 09/15/2016 12:53 PM, Alex Elsayed wrote:
>> On Tue, 13 Sep 2016 21:39:46 +0800, Anand Jain wrote:
>>
>>> This patchset adds btrfs encryption support.
>>>
>>> The main objective of this series is to have bugs fixed and stability.
>>> I have verified with fstests to confirm that there is no regression.
>>>
>>> A design write-up is coming next, however here below is the quick
>>> example on the cli usage. Please try out, let me know if I have missed
>>> something.
>>>
>>> Also would like to mention that a review from the security experts is
>>> due,
>>> which is important and I believe those review comments can be
>>> accommodated without major changes from here.
>>>
>>> Also yes, thanks for the emails, I hear, per file encryption and
>>> inline with vfs layer is also important, which is wip among other
>>> things in the list.
>>>
>>> As of now these patch set supports encryption on per subvolume, as
>>> managing properties on per subvolume is a kind of core to btrfs, which
>>> is easier for data center solution-ing, seamlessly persistent and easy
>>> to manage.
>>>
>>>
>>> Steps:
>>> -
>>>
>>> Make sure following kernel TFMs are compiled in.
>>> # cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
>>> name : ctr(aes)
>>> name : cbc(aes)
>>
>> First problem: These are purely encryption algorithms, rather than AE
>> (Authenticated Encryption) or AEAD (Authenticated Encryption with
>> Associated Data). As a result, they are necessarily vulnerable to
>> adaptive chosen-ciphertext attacks, and CBC has historically had other
>> issues. I highly recommend using a well-reviewed AE or AEAD mode, such
>> as AES-GCM (as ecryptfs does), as long as the code can handle the
>> ciphertext being longer than the plaintext.
>>
>> If it _cannot_ handle the ciphertext being longer than the plaintext,
>> please consider that a very serious red flag: It means that you cannot
>> provide better security than block-level encryption, which greatly
>> reduces the benefit of filesystem-integrated encryption. Being at the
>> extent level _should_ permit using AEAD - if it does not, something is
>> wrong.
>>
>> If at all possible, I'd suggest _only_ permitting AEAD cipher modes to
>> be used.
>>
>> Anyway, even for block-level encryption, CTR and CBC have been
>> considered obsolete and potentially dangerous to use in disk encryption
>> for quite a while - current recommendations for block-level encryption
>> are to use either a narrow-block tweakable cipher mode (such as XTS),
>> or a wide- block one (such as EME or CMC), with the latter providing
>> slightly better security, but worse performance.
> 
>Yes. CTR should be changed, so I have kept it as a cli option. And
>with the current internal design, hope we can plugin more algorithms
>as suggested/if-its-outdated and yes code can handle (or with a
>little tweak) bigger ciphertext (than plaintext) as well.
> 
>encryption + keyhash (as below) + Btrfs-data-checksum provides
>similar to AE,  right ?

No, it does not provide anything remotely similar to AE. AE requires 
_cryptographic_ authentication of the data. Not only is a CRC (as Btrfs 
uses for the data checksum) not enough, a _cryptographic hash_ (such as 
SHA256) isn't even enough. A MAC (message authentication code) is 
necessary.

Moreover, combining an encryption algorithm and a MAC is very easy to get 
wrong, in ways that absolutely ruin security - as an example, see the 
Vaudenay/Lucky13 padding oracle attacks on TLS.

In order for this to be secure, you need to use a secure encryption 
system that also authenticates the data in a cryptographically secure 
manner. Certain schemes are well-studied and believed to be secure - AES-
GCM and ChaCha20-Poly1305 are common and well-regarded, and there's a 
generic security reduction for Encrypt-then-MAC constructions (using CTR 
together with HMAC in such a construction is generally acceptable).

The Btrfs data checksum is wholly inadequate, and the keyhash is a non-
sequitur - it prevents accidentally opening the subvolume with the wrong 
key, but neither it (nor the btrfs data checksum, which is a CRC rather 
than a cryptographic MAC) protect adequately against malicious corruption 
of the ciphertext.

I'd suggest pulling in Herbert Xu, as he'd likely be able to tell you 
what of the Crypto API is actually sane to use for this.

>>> Create encrypted subvolume.
>>> # btrfs su create -e 'ctr(aes)' /btrfs/e1 Create subvolume '/btrfs/e1'
>>> Passphrase:
>>> Again passphrase:
>>
>> I presume the command first creates a key, then creates a subvolume
>> referencing that key? If so, that seems sensible.
> 
>   Hmm I didn't get the why part, any help ? (this doesn't encrypt
>   metadata part).

Basically, if your tool merely sets up an entry in the kernel keyring, 
then calls the subvolume creation interface (passing in the key ID), then 
it

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Austin S. Hemmelgarn


On 2016-09-13 09:39, Anand Jain wrote:


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Also would like to mention that a review from the security experts is due,
which is important and I believe those review comments can be accommodated
without major changes from here.

Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs, which is
easier for data center solution-ing, seamlessly persistent and easy to
manage.


Steps:
-

Make sure following kernel TFMs are compiled in.
# cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
name : ctr(aes)
name : cbc(aes)

Create encrypted subvolume.
# btrfs su create -e 'ctr(aes)' /btrfs/e1
Create subvolume '/btrfs/e1'
Passphrase:
Again passphrase:

A key is created and its hash is updated into the subvolume item,
and then added to the system keyctl.
# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (594790215)

# keyctl show 594790215
Keyring
 594790215 --alsw-v  0 0  logon: btrfs:75197c8e


Now any file data extents under the subvol /btrfs/e1 will be
encrypted.

You may revoke key using keyctl or btrfs(8) as below.
# btrfs su encrypt -k out /btrfs/e1

# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (Required key not 
available)

# keyctl show 594790215
Keyring
Unable to dump key: Key has been revoked

As the key hash is updated, If you provide wrong passphrase in the next
key in, it won't add key to the system. So we have key verification
from the day1.

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
ERROR: failed to set attribute 'btrfs.encrypt' to 'ctr(aes)@btrfs:75197c8e' : 
Key was rejected by service

ERROR: key set failed: Key was rejected by service

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'

Now if you revoke the key the read / write fails with key error.

# md5sum /btrfs/e1/2k-test-file
8c9fbc69125ebe84569a5c1ca088cb14  /btrfs/e1/2k-test-file

# btrfs su encrypt -k out /btrfs/e1

# md5sum /btrfs/e1/2k-test-file
md5sum: /btrfs/e1/2k-test-file: Key has been revoked

# cp /tfs/1k-test-file /btrfs/e1/
cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been revoked

Plain text memory scratches for security reason is pending. As there are some
key revoke notification challenges to coincide with encryption context switch,
which I do believe should be fixed in the due course, but is not a roadblock
at this stage.

Before I make any other comments, I should state that I asbolutely agree 
with Alex Elsayed about the issues with using CBC or CTR mode, and not 
supporting AE or AEAD modes.  If that's going to be the case, then 
there's essentially no point in merging this as is, as it has worse 
security than other filesystem level encryption options in the kernel by 
a pretty significant margin.  This absolutely _needs_ to be done right 
the first time, otherwise the reputation of BTRFS will suffer further, 
and nobody sane is going to use subvolume encryption for years after 
it's 'fixed' to be properly secure.


Now, the other thing I wanted to comment about:
How does this handle cloning of extents?  Can extents be cloned across 
subvolume boundaries when one of the subvolumes is encrypted?  Can they 
be cloned within an encrypted subvolume?  What happens when you try to 
clone them in either case if it isn't supported?


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Anand Jain



Thanks for commenting. pls see inline below.

On 09/15/2016 12:53 PM, Alex Elsayed wrote:

On Tue, 13 Sep 2016 21:39:46 +0800, Anand Jain wrote:


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick
example on the cli usage. Please try out, let me know if I have missed
something.

Also would like to mention that a review from the security experts is
due,
which is important and I believe those review comments can be
accommodated without major changes from here.

Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs, which
is easier for data center solution-ing, seamlessly persistent and easy
to manage.


Steps:
-

Make sure following kernel TFMs are compiled in.
# cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
name : ctr(aes)
name : cbc(aes)


First problem: These are purely encryption algorithms, rather than AE
(Authenticated Encryption) or AEAD (Authenticated Encryption with
Associated Data). As a result, they are necessarily vulnerable to
adaptive chosen-ciphertext attacks, and CBC has historically had other
issues. I highly recommend using a well-reviewed AE or AEAD mode, such as
AES-GCM (as ecryptfs does), as long as the code can handle the ciphertext
being longer than the plaintext.

If it _cannot_ handle the ciphertext being longer than the plaintext,
please consider that a very serious red flag: It means that you cannot
provide better security than block-level encryption, which greatly
reduces the benefit of filesystem-integrated encryption. Being at the
extent level _should_ permit using AEAD - if it does not, something is
wrong.

If at all possible, I'd suggest _only_ permitting AEAD cipher modes to be
used.

Anyway, even for block-level encryption, CTR and CBC have been considered
obsolete and potentially dangerous to use in disk encryption for quite a
while - current recommendations for block-level encryption are to use
either a narrow-block tweakable cipher mode (such as XTS), or a wide-
block one (such as EME or CMC), with the latter providing slightly better
security, but worse performance.


  Yes. CTR should be changed, so I have kept it as a cli option. And
  with the current internal design, hope we can plugin more algorithms
  as suggested/if-its-outdated and yes code can handle (or with a little
  tweak) bigger ciphertext (than plaintext) as well.

  encryption + keyhash (as below) + Btrfs-data-checksum provides
  similar to AE,  right ?



Create encrypted subvolume.
# btrfs su create -e 'ctr(aes)' /btrfs/e1 Create subvolume '/btrfs/e1'
Passphrase:
Again passphrase:


I presume the command first creates a key, then creates a subvolume
referencing that key? If so, that seems sensible.


 Hmm I didn't get the why part, any help ? (this doesn't encrypt
 metadata part).


A key is created and its hash is updated into the subvolume item,
and then added to the system keyctl.
# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (594790215)

# keyctl show 594790215 Keyring
 594790215 --alsw-v  0 0  logon: btrfs:75197c8e


That's entirely reasonable, though you may want to support "trusted and
encrypted keys" (Documentation/security/keys-trusted-encrypted.txt)


  Yes. that's in the list.


Now any file data extents under the subvol /btrfs/e1 will be encrypted.

You may revoke key using keyctl or btrfs(8) as below.
# btrfs su encrypt -k out /btrfs/e1

# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (Required key not

available)


# keyctl show 594790215 Keyring Unable to dump key: Key has been revoked

As the key hash is updated, If you provide wrong passphrase in the next
key in, it won't add key to the system. So we have key verification from
the day1.


This is good.


  Thanks.

Thanks, Anand



# btrfs su encrypt -k in /btrfs/e1 Passphrase:
Again passphrase:
ERROR: failed to set attribute 'btrfs.encrypt' to
'ctr(aes)@btrfs:75197c8e' : Key was rejected by service

ERROR: key set failed: Key was rejected by service

# btrfs su encrypt -k in /btrfs/e1 Passphrase:
Again passphrase:
key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'

Now if you revoke the key the read / write fails with key error.

# md5sum /btrfs/e1/2k-test-file 8c9fbc69125ebe84569a5c1ca088cb14
/btrfs/e1/2k-test-file

# btrfs su encrypt -k out /btrfs/e1

# md5sum /btrfs/e1/2k-test-file md5sum: /btrfs/e1/2k-test-file: Key has
been revoked

# cp /tfs/1k-test-file /btrfs/e1/
cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been
revoked

Plain text

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Anand Jain



Thanks for the comments. Pls see inline below..

On 09/15/2016 01:38 PM, Chris Murphy wrote:

On Tue, Sep 13, 2016 at 7:39 AM, Anand Jain  wrote:


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.


What's the behavior with nested subvolumes having different keys?

subvolume A (encrypted with key A)
|
- subvolume B (encrypted with key B)

Without encryption I can discover either A or B whether top-level, A,
or B are mounted.

With encryption, must A be opened [1] for B to be discovered? Must A
be opened before B can be opened? Or is the subvolume metadata always
non-encrypted, and it's just file extents that are encrypted? Are
filenames in those subvolumes discoverable (e.g. btrfs-debug-tree,
btrfs-image) if the subvolume is not opened? And reflink handling
between subvolumes behaves how?


  nested encrypting subvolume isn't supported, its just that it wasn't
  in my mind or the use case analysis review which I did, didn't tell
  me that. However I did a bit of code changes, its not that tough to
  get that in the current setup though. Yes only extent encrypted.

Thanks, Anand



[1] open in the cryptsetup open/luksOpen sense



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-14 Thread Chris Murphy

On Tue, Sep 13, 2016 at 7:39 AM, Anand Jain  wrote:
>
> This patchset adds btrfs encryption support.
>
> The main objective of this series is to have bugs fixed and stability.
> I have verified with fstests to confirm that there is no regression.
>
> A design write-up is coming next, however here below is the quick example
> on the cli usage. Please try out, let me know if I have missed something.

What's the behavior with nested subvolumes having different keys?

subvolume A (encrypted with key A)
|
- subvolume B (encrypted with key B)

Without encryption I can discover either A or B whether top-level, A,
or B are mounted.

With encryption, must A be opened [1] for B to be discovered? Must A
be opened before B can be opened? Or is the subvolume metadata always
non-encrypted, and it's just file extents that are encrypted? Are
filenames in those subvolumes discoverable (e.g. btrfs-debug-tree,
btrfs-image) if the subvolume is not opened? And reflink handling
between subvolumes behaves how?

[1] open in the cryptsetup open/luksOpen sense

-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Preliminary BTRFS Encryption

2016-09-14 Thread Alex Elsayed

On Tue, 13 Sep 2016 21:39:46 +0800, Anand Jain wrote:

> This patchset adds btrfs encryption support.
> 
> The main objective of this series is to have bugs fixed and stability.
> I have verified with fstests to confirm that there is no regression.
> 
> A design write-up is coming next, however here below is the quick
> example on the cli usage. Please try out, let me know if I have missed
> something.
> 
> Also would like to mention that a review from the security experts is
> due,
> which is important and I believe those review comments can be
> accommodated without major changes from here.
> 
> Also yes, thanks for the emails, I hear, per file encryption and inline
> with vfs layer is also important, which is wip among other things in the
> list.
> 
> As of now these patch set supports encryption on per subvolume, as
> managing properties on per subvolume is a kind of core to btrfs, which
> is easier for data center solution-ing, seamlessly persistent and easy
> to manage.
> 
> 
> Steps:
> -
> 
> Make sure following kernel TFMs are compiled in.
> # cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
> name : ctr(aes)
> name : cbc(aes)

First problem: These are purely encryption algorithms, rather than AE 
(Authenticated Encryption) or AEAD (Authenticated Encryption with 
Associated Data). As a result, they are necessarily vulnerable to 
adaptive chosen-ciphertext attacks, and CBC has historically had other 
issues. I highly recommend using a well-reviewed AE or AEAD mode, such as 
AES-GCM (as ecryptfs does), as long as the code can handle the ciphertext 
being longer than the plaintext.

If it _cannot_ handle the ciphertext being longer than the plaintext, 
please consider that a very serious red flag: It means that you cannot 
provide better security than block-level encryption, which greatly 
reduces the benefit of filesystem-integrated encryption. Being at the 
extent level _should_ permit using AEAD - if it does not, something is 
wrong.

If at all possible, I'd suggest _only_ permitting AEAD cipher modes to be 
used.

Anyway, even for block-level encryption, CTR and CBC have been considered 
obsolete and potentially dangerous to use in disk encryption for quite a 
while - current recommendations for block-level encryption are to use 
either a narrow-block tweakable cipher mode (such as XTS), or a wide-
block one (such as EME or CMC), with the latter providing slightly better 
security, but worse performance.

> Create encrypted subvolume.
> # btrfs su create -e 'ctr(aes)' /btrfs/e1 Create subvolume '/btrfs/e1'
> Passphrase:
> Again passphrase:

I presume the command first creates a key, then creates a subvolume 
referencing that key? If so, that seems sensible.

> A key is created and its hash is updated into the subvolume item,
> and then added to the system keyctl.
> # btrfs su show /btrfs/e1 | egrep -i encrypt
>   Encryption: ctr(aes)@btrfs:75197c8e (594790215)
> 
> # keyctl show 594790215 Keyring
>  594790215 --alsw-v  0 0  logon: btrfs:75197c8e

That's entirely reasonable, though you may want to support "trusted and 
encrypted keys" (Documentation/security/keys-trusted-encrypted.txt)

> Now any file data extents under the subvol /btrfs/e1 will be encrypted.
> 
> You may revoke key using keyctl or btrfs(8) as below.
> # btrfs su encrypt -k out /btrfs/e1
> 
> # btrfs su show /btrfs/e1 | egrep -i encrypt
>   Encryption: ctr(aes)@btrfs:75197c8e (Required key not 
available)
> 
> # keyctl show 594790215 Keyring Unable to dump key: Key has been revoked
> 
> As the key hash is updated, If you provide wrong passphrase in the next
> key in, it won't add key to the system. So we have key verification from
> the day1.

This is good.

> # btrfs su encrypt -k in /btrfs/e1 Passphrase:
> Again passphrase:
> ERROR: failed to set attribute 'btrfs.encrypt' to
> 'ctr(aes)@btrfs:75197c8e' : Key was rejected by service
> 
> ERROR: key set failed: Key was rejected by service
> 
> # btrfs su encrypt -k in /btrfs/e1 Passphrase:
> Again passphrase:
> key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'
> 
> Now if you revoke the key the read / write fails with key error.
> 
> # md5sum /btrfs/e1/2k-test-file 8c9fbc69125ebe84569a5c1ca088cb14 
> /btrfs/e1/2k-test-file
> 
> # btrfs su encrypt -k out /btrfs/e1
> 
> # md5sum /btrfs/e1/2k-test-file md5sum: /btrfs/e1/2k-test-file: Key has
> been revoked
> 
> # cp /tfs/1k-test-file /btrfs/e1/
> cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been
> revoked
> 
> Plain text memory scratches for security reason is pending. As there are
> some key revoke notification challenges to coincide with encryption
> context switch,
> which I do believe should be fixed in the due course, but is not a
> roadblock at this stage.
> 
> Thanks, Anand

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at

Re: [RFC] Preliminary BTRFS Encryption

2016-09-14 Thread Wilson Meier



> Am 14.09.2016 um 09:02 schrieb Anand Jain :
> 
> 
> 
> Wilson,
> 
> Thanks for commenting. Pls see inline below..
> 
>> On 09/14/2016 12:42 AM, Wilson Meier wrote:
>> Hi Anand,
>> 
>> these are great news! Thanks for yor work. I'm looking forward to use the 
>> encryption.
>> 
>> I would like to ask a few question regarding the feature set.
>> 
>> 1. is encryption of an existing, filled and unencrypted subvolume without 
>> manually moving the data possible?
> 
>  Encrypt contexts are set only on newly created files. However you can
>  create empty encrypted subvol and move files and dir into it. In short
>  you can't set encrypt property on non-empty subvolume as of now.

Ok, so manually moving to an new encrypted subvolume is the only possibility. 
Maybe there will be a possibility in the feature. ;)

> 
>> 2. What about encrypting the root and boot subvolume? Will it work with 
>> grub2?
> 
>  Keys are only in-memory, which does not persist or prompt
>  for it across boot. I had keyctl code written to prompt
>  for key but it isn't successful yet. Probably once we support
>  keyfile, then root/boot support is possible.
> 

Currently i'm using dm-crypt and btrfs to achieve a fully encrypted system. I'm 
looking forward to switch to a pure btrfs solution. Hopefully this will be 
possible soon.

>> 3. How does btrfs rescue handle the encrypted subvolume to recover data in 
>> case of an emergency?
> 
>  btrfs rescue / btrfsck works as usual. btrfs restore which
>  needs decrypt isn't supported.
> 

Don't get me wrong, but not being able to use btrfs restore is a showstopper as 
i already had a case where i could only rescue my data using the restore 
command. In my opinion in the current state of btrfs such recover options are 
key.

>> 4. Is it possible to unlock a subvolume using a keyfile?
> 
>  keyfile support is on top of the list to be supported, it helps
>  testing as well.
> 

This goes hand in hand with my question about boot/root unlocking. 

> Thanks, Anand

Thanks for your feedback. I really appreciate.
Wilson

> 
>> Thanks in advance,
>> 
>> Wilson
>> 
>> 
>>> Am 13.09.2016 um 15:39 schrieb Anand Jain :
>>> 
>>> 
>>> This patchset adds btrfs encryption support.
>>> 
>>> The main objective of this series is to have bugs fixed and stability.
>>> I have verified with fstests to confirm that there is no regression.
>>> 
>>> A design write-up is coming next, however here below is the quick example
>>> on the cli usage. Please try out, let me know if I have missed something.
>>> 
>>> Also would like to mention that a review from the security experts is due,
>>> which is important and I believe those review comments can be accommodated
>>> without major changes from here.
>>> 
>>> Also yes, thanks for the emails, I hear, per file encryption and inline
>>> with vfs layer is also important, which is wip among other things in the
>>> list.
>>> 
>>> As of now these patch set supports encryption on per subvolume, as
>>> managing properties on per subvolume is a kind of core to btrfs, which is
>>> easier for data center solution-ing, seamlessly persistent and easy to
>>> manage.
>>> 
>>> 
>>> Steps:
>>> -
>>> 
>>> Make sure following kernel TFMs are compiled in.
>>> # cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
>>> name : ctr(aes)
>>> name : cbc(aes)
>>> 
>>> Create encrypted subvolume.
>>> # btrfs su create -e 'ctr(aes)' /btrfs/e1
>>> Create subvolume '/btrfs/e1'
>>> Passphrase:
>>> Again passphrase:
>>> 
>>> A key is created and its hash is updated into the subvolume item,
>>> and then added to the system keyctl.
>>> # btrfs su show /btrfs/e1 | egrep -i encrypt
>>>   Encryption:ctr(aes)@btrfs:75197c8e (594790215)
>>> 
>>> # keyctl show 594790215
>>> Keyring
>>> 594790215 --alsw-v  0 0  logon: btrfs:75197c8e
>>> 
>>> 
>>> Now any file data extents under the subvol /btrfs/e1 will be
>>> encrypted.
>>> 
>>> You may revoke key using keyctl or btrfs(8) as below.
>>> # btrfs su encrypt -k out /btrfs/e1
>>> 
>>> # btrfs su show /btrfs/e1 | egrep -i encrypt
>>>   Encryption:ctr(aes)@btrfs:75197c8e (Required key not available)
>>> 
>>> # keyctl show 594790215
>>> Keyring
>>> Unable to dump key: Key has been revoked
>>> 
>>> As the key hash is updated, If you provide wrong passphrase in the next
>>> key in, it won't add key to the system. So we have key verification
>>> from the day1.
>>> 
>>> # btrfs su encrypt -k in /btrfs/e1
>>> Passphrase:
>>> Again passphrase:
>>> ERROR: failed to set attribute 'btrfs.encrypt' to 'ctr(aes)@btrfs:75197c8e' 
>>> : Key was rejected by service
>>> 
>>> ERROR: key set failed: Key was rejected by service
>>> 
>>> # btrfs su encrypt -k in /btrfs/e1
>>> Passphrase:
>>> Again passphrase:
>>> key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'
>>> 
>>> Now if you revoke the key the read / write fails with key error.
>>> 
>>> # md5sum /btrfs/e1/2k-test-file
>>>

Re: [RFC] Preliminary BTRFS Encryption

2016-09-14 Thread Anand Jain




Wilson,

Thanks for commenting. Pls see inline below..

On 09/14/2016 12:42 AM, Wilson Meier wrote:

Hi Anand,

these are great news! Thanks for yor work. I'm looking forward to use the 
encryption.

I would like to ask a few question regarding the feature set.

1. is encryption of an existing, filled and unencrypted subvolume without 
manually moving the data possible?


  Encrypt contexts are set only on newly created files. However you can
  create empty encrypted subvol and move files and dir into it. In short
  you can't set encrypt property on non-empty subvolume as of now.


2. What about encrypting the root and boot subvolume? Will it work with grub2?


  Keys are only in-memory, which does not persist or prompt
  for it across boot. I had keyctl code written to prompt
  for key but it isn't successful yet. Probably once we support
  keyfile, then root/boot support is possible.


3. How does btrfs rescue handle the encrypted subvolume to recover data in case 
of an emergency?


  btrfs rescue / btrfsck works as usual. btrfs restore which
  needs decrypt isn't supported.


4. Is it possible to unlock a subvolume using a keyfile?


  keyfile support is on top of the list to be supported, it helps
  testing as well.

Thanks, Anand



Thanks in advance,

Wilson



Am 13.09.2016 um 15:39 schrieb Anand Jain :


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Also would like to mention that a review from the security experts is due,
which is important and I believe those review comments can be accommodated
without major changes from here.

Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs, which is
easier for data center solution-ing, seamlessly persistent and easy to
manage.


Steps:
-

Make sure following kernel TFMs are compiled in.
# cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
name : ctr(aes)
name : cbc(aes)

Create encrypted subvolume.
# btrfs su create -e 'ctr(aes)' /btrfs/e1
Create subvolume '/btrfs/e1'
Passphrase:
Again passphrase:

A key is created and its hash is updated into the subvolume item,
and then added to the system keyctl.
# btrfs su show /btrfs/e1 | egrep -i encrypt
   Encryption:ctr(aes)@btrfs:75197c8e (594790215)

# keyctl show 594790215
Keyring
594790215 --alsw-v  0 0  logon: btrfs:75197c8e


Now any file data extents under the subvol /btrfs/e1 will be
encrypted.

You may revoke key using keyctl or btrfs(8) as below.
# btrfs su encrypt -k out /btrfs/e1

# btrfs su show /btrfs/e1 | egrep -i encrypt
   Encryption:ctr(aes)@btrfs:75197c8e (Required key not available)

# keyctl show 594790215
Keyring
Unable to dump key: Key has been revoked

As the key hash is updated, If you provide wrong passphrase in the next
key in, it won't add key to the system. So we have key verification
from the day1.

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
ERROR: failed to set attribute 'btrfs.encrypt' to 'ctr(aes)@btrfs:75197c8e' : 
Key was rejected by service

ERROR: key set failed: Key was rejected by service

# btrfs su encrypt -k in /btrfs/e1
Passphrase:
Again passphrase:
key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'

Now if you revoke the key the read / write fails with key error.

# md5sum /btrfs/e1/2k-test-file
8c9fbc69125ebe84569a5c1ca088cb14  /btrfs/e1/2k-test-file

# btrfs su encrypt -k out /btrfs/e1

# md5sum /btrfs/e1/2k-test-file
md5sum: /btrfs/e1/2k-test-file: Key has been revoked

# cp /tfs/1k-test-file /btrfs/e1/
cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been revoked

Plain text memory scratches for security reason is pending. As there are some
key revoke notification challenges to coincide with encryption context switch,
which I do believe should be fixed in the due course, but is not a roadblock
at this stage.

Thanks, Anand


Anand Jain (1):
 btrfs: Encryption: Add btrfs encryption support

fs/btrfs/Makefile   |   4 +-
fs/btrfs/btrfs_inode.h  |   6 +
fs/btrfs/compression.c  |  30 +-
fs/btrfs/compression.h  |  10 +-
fs/btrfs/ctree.h|   4 +
fs/btrfs/disk-io.c  |   3 +
fs/btrfs/encrypt.c  | 807 
fs/btrfs/encrypt.h  |  94 +
fs/btrfs/inode.c| 255 -
fs/btrfs/ioctl.c|  67 
fs/btrfs/lzo.c  |   2 +-
fs/btrfs/props.c| 331 +++-

Re: [RFC] Preliminary BTRFS Encryption

2016-09-13 Thread Wilson Meier

Hi Anand,

these are great news! Thanks for yor work. I'm looking forward to use the 
encryption.

I would like to ask a few question regarding the feature set.

1. is encryption of an existing, filled and unencrypted subvolume without 
manually moving the data possible?

2. What about encrypting the root and boot subvolume? Will it work with grub2?

3. How does btrfs rescue handle the encrypted subvolume to recover data in case 
of an emergency? 

4. Is it possible to unlock a subvolume using a keyfile?

Thanks in advance,

Wilson


> Am 13.09.2016 um 15:39 schrieb Anand Jain :
> 
> 
> This patchset adds btrfs encryption support.
> 
> The main objective of this series is to have bugs fixed and stability.
> I have verified with fstests to confirm that there is no regression.
> 
> A design write-up is coming next, however here below is the quick example
> on the cli usage. Please try out, let me know if I have missed something.
> 
> Also would like to mention that a review from the security experts is due,
> which is important and I believe those review comments can be accommodated
> without major changes from here.
> 
> Also yes, thanks for the emails, I hear, per file encryption and inline
> with vfs layer is also important, which is wip among other things in the
> list.
> 
> As of now these patch set supports encryption on per subvolume, as
> managing properties on per subvolume is a kind of core to btrfs, which is
> easier for data center solution-ing, seamlessly persistent and easy to
> manage.
> 
> 
> Steps:
> -
> 
> Make sure following kernel TFMs are compiled in.
> # cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
> name : ctr(aes)
> name : cbc(aes)
> 
> Create encrypted subvolume.
> # btrfs su create -e 'ctr(aes)' /btrfs/e1
> Create subvolume '/btrfs/e1'
> Passphrase: 
> Again passphrase: 
> 
> A key is created and its hash is updated into the subvolume item,
> and then added to the system keyctl.
> # btrfs su show /btrfs/e1 | egrep -i encrypt
>Encryption:ctr(aes)@btrfs:75197c8e (594790215)
> 
> # keyctl show 594790215
> Keyring
> 594790215 --alsw-v  0 0  logon: btrfs:75197c8e
> 
> 
> Now any file data extents under the subvol /btrfs/e1 will be
> encrypted.
> 
> You may revoke key using keyctl or btrfs(8) as below.
> # btrfs su encrypt -k out /btrfs/e1
> 
> # btrfs su show /btrfs/e1 | egrep -i encrypt
>Encryption:ctr(aes)@btrfs:75197c8e (Required key not available)
> 
> # keyctl show 594790215
> Keyring
> Unable to dump key: Key has been revoked
> 
> As the key hash is updated, If you provide wrong passphrase in the next
> key in, it won't add key to the system. So we have key verification
> from the day1.
> 
> # btrfs su encrypt -k in /btrfs/e1
> Passphrase: 
> Again passphrase: 
> ERROR: failed to set attribute 'btrfs.encrypt' to 'ctr(aes)@btrfs:75197c8e' : 
> Key was rejected by service
> 
> ERROR: key set failed: Key was rejected by service
> 
> # btrfs su encrypt -k in /btrfs/e1
> Passphrase: 
> Again passphrase: 
> key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'
> 
> Now if you revoke the key the read / write fails with key error.
> 
> # md5sum /btrfs/e1/2k-test-file 
> 8c9fbc69125ebe84569a5c1ca088cb14  /btrfs/e1/2k-test-file
> 
> # btrfs su encrypt -k out /btrfs/e1
> 
> # md5sum /btrfs/e1/2k-test-file 
> md5sum: /btrfs/e1/2k-test-file: Key has been revoked
> 
> # cp /tfs/1k-test-file /btrfs/e1/
> cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been revoked
> 
> Plain text memory scratches for security reason is pending. As there are some
> key revoke notification challenges to coincide with encryption context switch,
> which I do believe should be fixed in the due course, but is not a roadblock
> at this stage.
> 
> Thanks, Anand
> 
> 
> Anand Jain (1):
>  btrfs: Encryption: Add btrfs encryption support
> 
> fs/btrfs/Makefile   |   4 +-
> fs/btrfs/btrfs_inode.h  |   6 +
> fs/btrfs/compression.c  |  30 +-
> fs/btrfs/compression.h  |  10 +-
> fs/btrfs/ctree.h|   4 +
> fs/btrfs/disk-io.c  |   3 +
> fs/btrfs/encrypt.c  | 807 
> fs/btrfs/encrypt.h  |  94 +
> fs/btrfs/inode.c| 255 -
> fs/btrfs/ioctl.c|  67 
> fs/btrfs/lzo.c  |   2 +-
> fs/btrfs/props.c| 331 +++-
> fs/btrfs/super.c|  27 +-
> fs/btrfs/tests/crypto-tests.c   | 376 +++
> fs/btrfs/tests/crypto-tests.h   |  38 ++
> fs/btrfs/zlib.c |   2 +-
> include/uapi/linux/btrfs_tree.h |   6 +-
> 17 files changed, 2027 insertions(+), 35 deletions(-)
> create mode 100644 fs/btrfs/encrypt.c
> create mode 100644 fs/btrfs/encrypt.h
> create mode 100755 fs/btrfs/tests/crypto-tests.c
> create mode 100755 fs/btrfs/tests/crypto-tests.h
> 
> Anand Jain (2):
>  btrfs-progs: make

[RFC] Preliminary BTRFS Encryption

2016-09-13 Thread Anand Jain


This patchset adds btrfs encryption support.

The main objective of this series is to have bugs fixed and stability.
I have verified with fstests to confirm that there is no regression.

A design write-up is coming next, however here below is the quick example
on the cli usage. Please try out, let me know if I have missed something.

Also would like to mention that a review from the security experts is due,
which is important and I believe those review comments can be accommodated
without major changes from here.

Also yes, thanks for the emails, I hear, per file encryption and inline
with vfs layer is also important, which is wip among other things in the
list.

As of now these patch set supports encryption on per subvolume, as
managing properties on per subvolume is a kind of core to btrfs, which is
easier for data center solution-ing, seamlessly persistent and easy to
manage.


Steps:
-

Make sure following kernel TFMs are compiled in.
# cat /proc/crypto | egrep 'cbc\(aes\)|ctr\(aes\)'
name : ctr(aes)
name : cbc(aes)

Create encrypted subvolume.
# btrfs su create -e 'ctr(aes)' /btrfs/e1
Create subvolume '/btrfs/e1'
Passphrase: 
Again passphrase: 

A key is created and its hash is updated into the subvolume item,
and then added to the system keyctl.
# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (594790215)

# keyctl show 594790215
Keyring
 594790215 --alsw-v  0 0  logon: btrfs:75197c8e


Now any file data extents under the subvol /btrfs/e1 will be
encrypted.

You may revoke key using keyctl or btrfs(8) as below.
# btrfs su encrypt -k out /btrfs/e1

# btrfs su show /btrfs/e1 | egrep -i encrypt
Encryption: ctr(aes)@btrfs:75197c8e (Required key not 
available)

# keyctl show 594790215
Keyring
Unable to dump key: Key has been revoked

As the key hash is updated, If you provide wrong passphrase in the next
key in, it won't add key to the system. So we have key verification
from the day1.

# btrfs su encrypt -k in /btrfs/e1
Passphrase: 
Again passphrase: 
ERROR: failed to set attribute 'btrfs.encrypt' to 'ctr(aes)@btrfs:75197c8e' : 
Key was rejected by service

ERROR: key set failed: Key was rejected by service

# btrfs su encrypt -k in /btrfs/e1
Passphrase: 
Again passphrase: 
key for '/btrfs/e1' has  logged in with keytag 'btrfs:75197c8e'

Now if you revoke the key the read / write fails with key error.

# md5sum /btrfs/e1/2k-test-file 
8c9fbc69125ebe84569a5c1ca088cb14  /btrfs/e1/2k-test-file

# btrfs su encrypt -k out /btrfs/e1

# md5sum /btrfs/e1/2k-test-file 
md5sum: /btrfs/e1/2k-test-file: Key has been revoked

# cp /tfs/1k-test-file /btrfs/e1/
cp: cannot create regular file ‘/btrfs/e1/1k-test-file’: Key has been revoked

Plain text memory scratches for security reason is pending. As there are some
key revoke notification challenges to coincide with encryption context switch,
which I do believe should be fixed in the due course, but is not a roadblock
at this stage.

Thanks, Anand


Anand Jain (1):
  btrfs: Encryption: Add btrfs encryption support

 fs/btrfs/Makefile   |   4 +-
 fs/btrfs/btrfs_inode.h  |   6 +
 fs/btrfs/compression.c  |  30 +-
 fs/btrfs/compression.h  |  10 +-
 fs/btrfs/ctree.h|   4 +
 fs/btrfs/disk-io.c  |   3 +
 fs/btrfs/encrypt.c  | 807 
 fs/btrfs/encrypt.h  |  94 +
 fs/btrfs/inode.c| 255 -
 fs/btrfs/ioctl.c|  67 
 fs/btrfs/lzo.c  |   2 +-
 fs/btrfs/props.c| 331 +++-
 fs/btrfs/super.c|  27 +-
 fs/btrfs/tests/crypto-tests.c   | 376 +++
 fs/btrfs/tests/crypto-tests.h   |  38 ++
 fs/btrfs/zlib.c |   2 +-
 include/uapi/linux/btrfs_tree.h |   6 +-
 17 files changed, 2027 insertions(+), 35 deletions(-)
 create mode 100644 fs/btrfs/encrypt.c
 create mode 100644 fs/btrfs/encrypt.h
 create mode 100755 fs/btrfs/tests/crypto-tests.c
 create mode 100755 fs/btrfs/tests/crypto-tests.h

Anand Jain (2):
  btrfs-progs: make wait_for_commit non static
  btrfs-progs: add encryption support

 Makefile.in   |   5 +-
 btrfs-list.c  |  33 
 cmds-filesystem.c |   4 +-
 cmds-restore.c|  16 ++
 cmds-subvolume.c  | 112 --
 commands.h|   1 +
 ctree.h   |   5 +-
 encrypt.c | 455 ++
 encrypt.h |  46 ++
 props.c   |   4 +
 utils.h   |   2 +
 11 files changed, 665 insertions(+), 18 deletions(-)
 create mode 100644 encrypt.c
 create mode 100644 encrypt.h

Anand Jain (1):
  fstests: btrfs: support encryption

 common/filter.btrfs |   2 +-
 common/rc   |   2 +-
 tests/btrfs/041 |   2 +
 tests/btrfs/041.out |  13 
 tests/btrfs/052 |  12 +++
 tests/btrfs/052.out | 214

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

Re: [RFC] Preliminary BTRFS Encryption

[RFC] Preliminary BTRFS Encryption

36 matches

Site Navigation

Mail list logo

Footer information