Re: XTS cipher mode for cluster file encryption

2022-02-02 Thread Sasasu
On 2022/2/2 01:50, Bruce Momjian wrote:
> Well, I sent an email a week ago asking if people want to advance this
> feature forward, and so far you are the only person to reply, which I
> think means there isn't enough interest in this feature to advance it.

I am still focus on this thread.

and I have a small patch to solve the current buffile problem.
https://www.postgresql.org/message-id/a859a753-70f2-bb17-6830-19dbcad11c17%40sasa.su


OpenPGP_signature
Description: OpenPGP digital signature


Re: [PATCH] buffile: ensure start offset is aligned with BLCKSZ

2021-11-29 Thread Sasasu
On 2021/11/29 18:05, Antonin Houska wrote:
> Does this test really pass regression tests? In BufFileRead(), I would
> understand if you did
> 
> + file->pos = offsetInBlock;
> + file->curOffset -= offsetInBlock;
> 
> rather than
> 
> + file->pos += offsetInBlock;
> + file->curOffset -= offsetInBlock;

It pass all regression tests. this patch is compatible with
BufFileSeek().

to generate a correct alignment, we need to make sure
  pos_new + offset_new = pos_old + offset_old 
offset_new = offset_old - offset_old % BLCKSZ
it means
  pos_new = pos_old + offset_old % BLCKSZ
  = pos_old + "offsetInBlock"

with your code, backend will read a wrong buffile at the end of buffile
reading. for example: physical file size = 20 and pos = 10, off = 10,
read start at 20. after the '=' code: pos = 10, off = 0, read start at
10, which is wrong.

> Anyway, BufFileDumpBuffer() does not seem to enforce curOffset to end up at
> block boundary, not to mention BufFileSeek().
> 
> When I was implementing this for our fork [1], I concluded that the encryption
> code path is too specific, so I left the existing code for the unecrypted data
> and added separate functions for the encrypted data.
> 
> One specific thing is that if you encrypt and write n bytes, but only need
> part of it later, you need to read and decrypt exactly those n bytes anyway,
> otherwise the decryption won't work. So I decided not only to keep curOffset
> at BLCKSZ boundary, but also to read / write BLCKSZ bytes at a time. This also
> makes sense if the scope of the initialization vector (IV) is BLCKSZ bytes.
> 
> Another problem is that you might want to store the IV somewhere in between
> the data. In short, the encryption makes the buffered IO rather different and
> the specific code should be kept aside, although the same API is used to
> invoke it.
> 
but I want to make less change on existed code. with this path. the only
code added to critical code path is this:

diff --git a/src/backend/storage/file/buffile.c 
b/src/backend/storage/file/buffile.c
index 3be08eb723..ceae85584b 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -512,6 +512,9 @@ BufFileDumpBuffer(BufFile *file)
/* and the buffer is aligned with BLCKSZ */
Assert(file->curOffset % BLCKSZ == 0);
 
+   /* encrypt before write */
+   TBD_ENC(file->buffer.data + wpos /* buffer */, bytestowrite /* 
size */, file->curOffset /* context to find IV */);
+
thisfile = file->files[file->curFile];
bytestowrite = FileWrite(thisfile,
 
file->buffer.data + wpos,
@@ -582,6 +585,9 @@ BufFileRead(BufFile *file, void *ptr, size_t size)
BufFileLoadBuffer(file);
if (file->nbytes <= 0 || (file->nbytes == file->pos && 
file->nbytes != BLCKSZ))
break;  /* no more data 
available */
+
+   /* decrypt after read */
+   TBD_DEC(file->buffer /* buffer */, file->nbytes /* size 
*/, file->curOffset /* context to find IV */);
}
 
nthistime = file->nbytes - file->pos;

those change will allow TDE to use any encryption algorithm (read offset
and write offset are matched) and implement on-the-fly IV generation.


OpenPGP_signature
Description: OpenPGP digital signature


[PATCH] buffile: ensure start offset is aligned with BLCKSZ

2021-11-28 Thread Sasasu
Hi hackers,

there are a very long discuss about TDE, and we agreed on that if the
temporary file I/O can be aligned to some fixed size, it will be easier
to use some kind of encryption algorithm.

discuss:
https://www.postgresql.org/message-id/20211025155814.GD20998%40tamriel.snowman.net

This patch adjust file->curOffset and file->pos before the real IO to
ensure the start offset is aligned.From e04c2595711998d2c3eb4546e98a38d340200949 Mon Sep 17 00:00:00 2001
From: Sasasu 
Date: Fri, 26 Nov 2021 00:12:14 +0800
Subject: [PATCH] buffile: ensure start offset is aligned with BLCKSZ.

If we can ensure that all IO is aligned with BLCKSZ, some other work (TDE
recently and maybe AIO in future) will like it.

this commit ensure all IO on buffile is aligned with BLCKSZ by careful
adjusting file->curOffset and file->pos.

Signed-off-by: Sasasu 
---
 src/backend/storage/file/buffile.c | 20 
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index b673150dbe..3be08eb723 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -430,6 +430,9 @@ BufFileLoadBuffer(BufFile *file)
 {
 	File		thisfile;
 
+	/* the offset of the buffile is aligned with BLCKSZ */
+	Assert(file->curOffset % BLCKSZ == 0);
+
 	/*
 	 * Advance to next component file if necessary and possible.
 	 */
@@ -437,7 +440,7 @@ BufFileLoadBuffer(BufFile *file)
 		file->curFile + 1 < file->numFiles)
 	{
 		file->curFile++;
-		file->curOffset = 0L;
+		file->curOffset -= MAX_PHYSICAL_FILESIZE;
 	}
 
 	/*
@@ -506,6 +509,9 @@ BufFileDumpBuffer(BufFile *file)
 		if ((off_t) bytestowrite > availbytes)
 			bytestowrite = (int) availbytes;
 
+		/* and the buffer is aligned with BLCKSZ */
+		Assert(file->curOffset % BLCKSZ == 0);
+
 		thisfile = file->files[file->curFile];
 		bytestowrite = FileWrite(thisfile,
  file->buffer.data + wpos,
@@ -564,11 +570,17 @@ BufFileRead(BufFile *file, void *ptr, size_t size)
 		if (file->pos >= file->nbytes)
 		{
 			/* Try to load more data into buffer. */
-			file->curOffset += file->pos;
-			file->pos = 0;
+			int offsetInBlock = file->curOffset % BLCKSZ;
 			file->nbytes = 0;
+			file->pos += offsetInBlock;
+			file->curOffset -= offsetInBlock;
+
+			/* pos out of current block, move to next block */
+			if (file->pos >= BLCKSZ)
+file->pos -= BLCKSZ, file->curOffset += BLCKSZ;
+
 			BufFileLoadBuffer(file);
-			if (file->nbytes <= 0)
+			if (file->nbytes <= 0 || (file->nbytes == file->pos && file->nbytes != BLCKSZ))
 break;			/* no more data available */
 		}
 
-- 
2.34.1



OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-11-02 Thread Sasasu

On 2021/11/2 02:24, Stephen Frost wrote:

I can understand the general idea that we should be sure to engineer
this in a way that multiple methods can be used, as surely one day folks
will say that AES128 isn't acceptable any more.

Cheers!


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-26 Thread Sasasu

On 2021/10/26 17:33, Yura Sokolov wrote:

Yes, AES with AES-NI is fastest. But not so much.

And, AES-CTR could be easily used instead of ChaCha12 in Adiantum.
Adiantum uses ChaCha12 as a stream cipher, and any other stream cipher will
be ok as well with minor modifications to algorithm.


not so much ~= half speed.

I prefer to use AES on all devices because AES is faster and more power 
efficient. using chacha only on low-end arm devices running complex 
program which most people do not have this device.


I reserve my opinion on this point, but I agree with you on the rest.
And I also agree and think it should add more algorithms. The current 
implementation does not have any reserved fields, which makes any 
upgrade like adding a new algorithm unfeasible.


On 2021/10/26 17:33, Yura Sokolov wrote:
> That is argument. But, again, openssl could be used for primitives:
> AES + AES-CTR + Poly/GCM. And Adiantum like construction could be
> composed from them quite easily.

implement Adiantum is a small problem (which doesn't look good, lack 
security audits). the real problem is there are too many code path can 
trigger disk IO.
TDE need modify them. each code path has different behavior (align or 
unaligned, once or more than once). and front-end tools even not use VF 
layer, they use pread with offset. TDE need fix them all. at the same 
time, keep the patch small enough.


I still think there need a unified IO API, not only modify 
BufFileDumpBuffer() and BufFileLoadBuffer(). the front-end tools also 
use this API will be great. with that API, TDE can focus on block IO. 
then give out a small patch.


Other works can also benefit from this API, like PostgreSQL with AIO, 
PostgreSQL on S3 (BLKSZ=4M), PostgreSQL on PMEM(no FS) and many.


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-25 Thread Sasasu

On 2021/10/26 04:32, Yura Sokolov wrote:

And among others Adiantum looks best: it is fast even without hardware
acceleration,


No, AES is fast on modern high-end hardware.

on X86 AMD 3700X
type  1024 bytes  8192 bytes   16384 bytes
aes-128-ctr   8963982.50k 11124613.88k 11509149.42k
aes-128-gcm   3978860.44k 4669417.10k  4732070.64k
aes-128-xts   7776628.39k 9073664.63k  9264617.74k
chacha20-poly1305 2043729.73k 2131296.36k  2141002.10k

on ARM RK3399, A53 middle-end with AES-NI
type  1024 bytes   8192 bytes   16384 bytes
aes-128-ctr   1663857.66k  1860930.22k  1872991.57k
aes-128-xts   685086.38k   712906.07k   716073.64k
aes-128-gcm   985578.84k   1054818.30k  1056768.00k
chacha20-poly1305 309012.82k   318889.98k   319711.91k

I think the baseline is the speed when using read(2) syscall on 
/dev/zero (which is 3.6GiB/s, on ARM is 980MiB/s)
chacha is fast on the low-end arm, but I haven't seen any HTTPS sites 
using chacha, including Cloudflare and Google.


On 2021/10/26 04:32, Yura Sokolov wrote:
>> That sounds like a great thing to think about adding ... after we get
>> something in that's based on XTS.
> Why? I see no points to do it after. Why not XTS after Adiantum?
>
> Ok, I see one: XTS is standartized.
:>
PostgreSQL even not discuss single-table key rotation or remote KMS.
I think it's too hard to use an encryption algorithm which openssl 
doesn't implement.


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-21 Thread Sasasu

On 2021/10/22 01:28, Stephen Frost wrote:

None of these are actually working with or changing the data though,
they're just copying it.  I don't think we'd actually want these to
decrypt and reencrypt the data.


OK, but why ALTER TABLE SET TABLESPACE use smgrread() and smgrextend() 
instead of copy_file().

TDE needs to modify these code paths, and make the patch bigger.

On 2021/10/22 01:28, Stephen Frost wrote:
> No, the CTR approach isn't great because, as has been discussed quite a
> bit already, using the LSN as the IV means that different data might be
> re-encrypted with the same LSN and that's not an acceptable thing to
> have happen with CTR.
On 2021/10/22 01:28, Stephen Frost wrote:
> Thankfully, for WAL
> (unlike heap and index blocks) we don't really have that issue- we
> hopefully aren't going to write different WAL records at the same LSN
> and so using the LSN there should be alright.
On 2021/10/22 01:28, Stephen Frost wrote:
> We've discussed at length how using CTR for heap isn't a good idea even
> if we're using the LSN for the IV, while if we use XTS then we don't
> have the issues that CTR has with IV re-use and using the LSN (plus
> block number and perhaps other things).  Nothing in what has been
> discussed here has really changed anything there that I can see and so
> it's unclear to me why we continue to go round and round with it.

I am not re-discuss using CTR for heap table. I mean use some CTR-like 
algorithm *only* for WAL encryption. My idea is exactly the same when 
you are typing "we hopefully aren't going to write different WAL records 
at the same LSN and so using the LSN there should be alright."


The point of disagreement between you and me is only on the block-based API.

On 2021/10/22 01:28, Stephen Frost wrote:
> it's unclear to me why we continue to go round and round with it.

same to me. I am monitoring this thread about 9 months, watching yours 
discuss key management/CBC/CRT/GCM round and round.


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-20 Thread Sasasu

On 2021/10/20 20:24, Stephen Frost wrote:
> PG does have a block-based IO API, it's just not exposed as hooks.  In
> particular, take a look at md.c, though perhaps you'd be more interested
> in the higher level bufmgr.c routines.  For the specific places where
> encryption may hook in, looking at the DataChecksumsEnabled() call sites
> may be informative (there aren't that many of them).

md.c is great, easy to understand. but PG does not have a unified API. 
There has many unexpected pread()/pwrite() in many corners. md.c only 
for heap table, bufmgr.c only for a buffered heap table.


eg: XLogWrite() looks like a block API, but is a range write. equivalent 
to the append(2)
eg: ALTER DATABASE SET TABLESPACE , the movedb() call. use copy_file() 
on heap table. which is just pread() pwrite() with 8*BLCKSZ.
eg: all front-end tools use pread() to read heap table. in particular, 
pg_rewind write heap table by offset.

eg: in contrib, pg_standby use system("cp") to copy WAL.

On 2021/10/20 20:24, Stephen Frost wrote:
> Breaking our ondisk format explicitly means that pg_upgrade won't work
> any longer and folks won't be able to do in-place upgrades. That's a
> pretty huge deal and it's something we've not done in over a decade.
> I doubt that's going to fly.

I completely agree.

On 2021/10/20 20:24, Stephen Frost wrote:
> Yes, using another fork for this is something that's been considered but
> it's not without its own drawbacks, in particular having to do another
> write and later fsync when a page changes.
>
> Further, none of this is necessary for XTS, but only for GCM. This is
> why it was put forward that GCM involves a lot more changes to the
> system and means that we won't be able to do things like binary
> replication to switch from an unencrypted to encrypted cluster. Those
> are good reasons to consider an XTS implementation first and then later,
> perhaps, implement GCM.

same as Robert Haas. I wish PG can do some infrastructure first. add 
more abstract layers like md.c (maybe a block-based API with ondisk 
format version field). so people can dive in without understanding the 
things which isolated by the abstract layer.


On 2021/10/20 20:24, Stephen Frost wrote:
> What's the point of using GCM if we aren't going to actually verify the
> tag? Also, the Cybertec patch didn't add an extra reserved field to the
> page format, and it used CTR anyway..

Oh, I am wrong, Cybertec patch can not use XTS, because WAL may not be 
aligned to 16bytes. for WAL need a stream cipher. The CTR implement is 
still correct.


CTR with hash(offset) as IV is basically equal to XTS. if use another 
AES key to encrypt the hash(offset), and block size is 16bytes it is XTS.
The point is that can not save random IV for WAL without adding a 
reserved field, no matter use GCM or CTR.


Because WAL only does append to the end, using CTR for WAL is safer than 
using XTS for heap table. If you want more security for WAL encryption, 
add HKDF[1].


[1]: https://en.wikipedia.org/wiki/HKDF


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-20 Thread Sasasu

On 2021/10/20 02:54, Stephen Frost wrote:
> I agree with Robert- using hooks for this really isn't realistic.

OK, I agree use hooks is too invasive. Just a whim, never mind.

But If PG has a clear block-based IO API, TDE is much easier to 
understand. security people may lack database knowledge but they can 
understand block IO.

This will allow more people to join PG community.

On 2021/10/20 02:54, Stephen Frost wrote:
> Where would you store the tag for GCM without changes in core?

If can add 32bit reserve field (in CGM is 28bits) will be best.
data file size will increase 0.048% (if BLCKSZ = 8KiB), I think it is 
acceptable even for the user who does not use TDE. but need ondisk 
format change.
If without of modify anything in core and doing GCM, the under-layer can 
write out a key fork, fsync(2) key fork with the same strategy for main 
fork. this is crash-safe. The consistency is ensured by WAL. (means 
wal_log_hints need set to on)
Or the underlayer can re-struct the IO request. insert one IV block per 
2730(=BLKSZ/IV_SIZE) data blocks. this method like the _mdfd_getseg() in 
md.c which split file by 1GiB. No perception in the upper layers.

Both of them can use cache to reduce performance downgrade.

for WAL encryption, the CybertecDB implement is correct. we can not 
write any extra data without adding a reserved field in core. because 
can not guarantee consistency. If use GCM for WAL encryption must 
disable HMAC verification.


* only shows the possibility, not mean anyone should implement TDE in 
that way.

* blahblah


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-19 Thread Sasasu

On 2021/10/19 00:37, Robert Haas wrote:

I think what we ought to be doing at
this point is combining our efforts to try to get some things
committed which make future work in this area committed - like that
patch to preserve relfilenode and database OID, or maybe some patches
to drive all of our I/O through a smaller number of code paths instead
of having every different type of temporary file we write reinvent the
wheel.


A unified block-based I/O API sounds great. Has anyone tried to do this 
before? It would be nice if the front-end tools could also use these API.


As there are so many threat models, I propose to do the TDE feature by a 
set of hooks. those hooks are on the critical path of IO operation, add 
the ability to let extension replace the IO API. and also load extension 
when initdb, single-mode, and in front-end tools.
This sounds Like using $LD_PRELOAD to replace pread(2) and pwrite(2), 
which widely used in folder based encryption. but the hook will pass 
more context (filenode, tableoid, blocksize, and many) to the under 
layer, this hook API will look like object_access_hook.
then implement the simplest AES-XTS. and put it to contrib. provide a 
tool to deactivate AES-XTS to make PostgreSQL upgradeable.


I think this is the most peaceful method. GCM people will not reject 
this just because XTS. and XTS people will satisfied(maybe?) with the 
complexity. for performance, just one more long-jump compare with 
current AES-XTS code.


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-17 Thread Sasasu

On 2021/10/16 04:57, Tomas Vondra wrote:
>
> Seems reasonable, on the assumption the threat models are the same.

On 2021/10/16 03:22, Stephen Frost wrote:

plain64: the initial vector is the 64-bit little-endian version of the
sector number, padded with zeros if necessary

That is, the default for LUKS is AES, XTS, with a simple IV.  That
strikes me as a pretty ringing endorsement

On 2021/10/18 05:23, Tomas Vondra wrote:
>
> AFAICS the threat model the patch aims to address is an attacker who can
> observe the data (e.g. a low-privileged OS user), but can't modify the
> files. Which seems like a reasonable model for shared environments.

I agree this threat model.

And if PostgreSQL is using XTS, there is no different with dm-encrypt.
The user can use dm-encrypt directly.


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: XTS cipher mode for cluster file encryption

2021-10-17 Thread Sasasu
Just a mention. the HMAC (or AE/AD) can be disabled in AES-GCM. HMAC in 
AES-GCM is an encrypt-then-hash MAC.


CRC-32 is not a crypto-safe hash (technically CRC-32 is not a hash 
function). Cryptographers may unhappy with CRC-32.


I think CRC or SHA is not such important. If IV can be stored, I believe 
there should have enough space to store HMAC.


On 2021/10/18 05:23, Tomas Vondra wrote:


I've argued for storing the nonce, but I don't quite see why would we 
need integrity guarantees?


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: storing an explicit nonce

2021-10-11 Thread Sasasu

On 2021/10/6 23:01, Robert Haas wrote:
> This seems wrong to me. CTR requires that you not reuse the IV. If you
> re-encrypt the page with a different IV, torn pages are a problem. If
> you re-encrypt it with the same IV, then it's not secure any more.

for CBC if the IV is predictable will case "dictionary attack".
and for CBC and GCM reuse IV will case "known plaintext attack".
XTS works like CBC but adds a tweak step. the tweak step does not add 
randomness. It means XTS still has "known plaintext attack", due to the 
same reason from CBC.

many mails before this mail do a clear explanation, I just repeat. :>

On 2021/10/7 22:28, Robert Haas wrote:
> I'm a little concerned by the email from "Sasasu" saying that even in
> XTS reusing the IV is not cryptographically weak. I don't know enough
> about these different encryption modes to know if he's right, but if
> he is then perhaps we need to consider his suggestion of using
> AES-GCM. Or, uh, something else.

a cryptography algorithm always lists some attack method (the scope), if 
the algorithm can defend against this attack, the algorithm is good. If 
software uses this algorithm but is attacked by the method not on that 
list. It is the software using this algorithm incorrectly, or should not 
use this algorithm.


On 2021/10/8 03:38, Stephen Frost wrote:
>   I strongly suspect we'll have one of two
> reactions- either we'll be more-or-less ignored and it'll be crickets
> from the security folks, or we're going to get beat up by them for
> $reasons, almost regardless of what we actually do.  Best bet to
> limit the risk (;)  ) of the latter happening would be to try our best
> to do what existing solutions already do- such as by using XTS.

If using an existing wonderful algorithm but out of the design scope, 
cryptographers will laugh at you.


On 2021/10/9 02:34, Stephen Frost wrote:

Greetings,

* Antonin Houska (a...@cybertec.at) wrote:

Stephen Frost  wrote:

* Robert Haas (robertmh...@gmail.com) wrote:

On Thu, Oct 7, 2021 at 3:38 PM Stephen Frost  wrote:

While I certainly also appreciate that we want to get this as right as
we possibly can from the start, I strongly suspect we'll have one of two
reactions- either we'll be more-or-less ignored and it'll be crickets
from the security folks, or we're going to get beat up by them for
$reasons, almost regardless of what we actually do.  Best bet to
limit the risk ( ;) ) of the latter happening would be to try our best
to do what existing solutions already do- such as by using XTS.
There's things we can do to limit the risk of known-plaintext attacks,
like simply not encrypting empty pages, or about possible known-IV
risks, like using the LSN as part of the IV/tweak.  Will we get
everything?  Probably not, but I don't think that we're going to really
go wrong by using XTS as it's quite popularly used today and it's
explicitly used for cases where you haven't got a place to store the
extra nonce that you would need for AEAD encryption schemes.


I agree that using a popular approach is a good way to go. If we do
what other people do, then hopefully our stuff won't be significantly
more broken than their stuff, and whatever is can be fixed.


Right.


As long as we're clear that this initial version of TDE is with XTS then
I really don't think we'll end up with anyone showing up and saying we
screwed up by not generating a per-page nonce to store with it- the point
of XTS is that you don't need that.


I agree that we shouldn't really catch flack for any weaknesses of the
underlying algorithm: if XTS turns out to be secure even when used
properly, and we use it properly, the resulting weakness is somebody
else's fault. On the other hand, if we use it improperly, that's our
fault, so we need to be really sure that we understand what guarantees
we need to provide from our end, and that we are providing them. Like
if we pick an encryption mode that requires nonces to be unique, we
will be at fault if they aren't; if it requires nonces to be
unpredictable, we will be at fault if they aren't; and so on.


Sure, I get that.  Would be awesome if all these things were clearly
documented somewhere but I've never been able to find it quite as
explicitly laid out as one would like.


So that's what is making me nervous here ... it doesn't seem likely we
have complete unanimity about whether XTS is the right thing, though
that does seem to be the majority position certainly, and it is not
really clear to me that any of us can speak with authority about what
the requirements are around the nonces in particular.


The authority to look at, in my view anyway, are NIST publications.
Following a bit more digging, I came across something which makes sense
to me as intuitive but explains it in a way that might help everyone
understand a bit better what's going on here:

https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.

Re: storing an explicit nonce

2021-09-05 Thread Sasasu



在 2021/9/5 下午10:51, Sasasu 写道:


For AES-GCM, a predictable IV is fine. I think we can decrypt and 
re-encrypt the user data in pg_upgrade. this will allows us to use 
relfile oid + block number as nonce.


relfile oid + block number + some counter for heap table IV. I mean.


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: storing an explicit nonce

2021-09-05 Thread Sasasu

Hi, community,

It looks like we are still considering AES-CBC, AES-XTS, and 
AES-GCM(-SIV). I want to say something that we don't think about.


For AES-CBC, the IV should be not predictable. I think LSN or HASH(LSN, 
block number or something) is predictable. There are many CVE related to 
AES-CBC with a predictable IV.


https://cwe.mitre.org/data/definitions/329.html

For AES-XTS, use block number or any fixed variable as tweak still has 
weaknesses similar to IV reuse (in CBC not GCM). the attacker can 
decrypt one block if he knows a kind of plaintext of this block.
In Luks/BitLocker/HardwareBasedSolution, the physical location is not 
available to the user. filesystem running in kernel space. and they not 
do encrypt when filesystem allocating a data block.
But in PostgreSQL, the attacker can capture an encrypted 'ALL-ZERO' page 
in `mdextend`, with this, the attacker can decode the ciphertext of all 
following data in this block.


For AES-GCM, a predictable IV is fine. I think we can decrypt and 
re-encrypt the user data in pg_upgrade. this will allows us to use 
relfile oid + block number as nonce.


OpenPGP_0x4E72AF09097DAE2E.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature