Re: Moving forward with TDE [PATCH v3]

2024-02-01 Thread vignesh C
On Mon, 22 Jan 2024 at 11:47, Peter Smith wrote: > > 2024-01 Commitfest. > > Hi, This patch has a CF status of "Needs Review" [1], but it seems > there were CFbot test failures last time it was run [2]. Please have a > look and post an updated version if necessary. The patch which you submitted

Re: Moving forward with TDE [PATCH v3]

2024-01-21 Thread Peter Smith
2024-01 Commitfest. Hi, This patch has a CF status of "Needs Review" [1], but it seems there were CFbot test failures last time it was run [2]. Please have a look and post an updated version if necessary. == [1] https://commitfest.postgresql.org/46/3985/ [2]

Re: Moving forward with TDE

2023-12-26 Thread Bruce Momjian
On Sun, Dec 17, 2023 at 06:30:50AM +, Chris Travers wrote: > Hi, > > I was re-reading the patches here and there was one thing I didn't > understand. > > There are provisions for a separation of data encryption keys for primary and > replica I see, and these share a single WAL key. > >

Re: Moving forward with TDE

2023-12-16 Thread Chris Travers
Hi, I was re-reading the patches here and there was one thing I didn't understand. There are provisions for a separation of data encryption keys for primary and replica I see, and these share a single WAL key. But if I am setting up a replica from the primary, and the primary is already

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 5:49 PM Andres Freund wrote: > Hi, > > On 2023-11-06 09:56:37 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > I still am quite quite unconvinced that using the LSN as a nonce is a > good > > > design decision. > > > > This is a really

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 6:47 PM Andres Freund wrote: > Hi, > > On 2023-11-06 11:26:44 +0100, Matthias van de Meent wrote: > > On Sat, 4 Nov 2023 at 03:38, Andres Freund wrote: > > > On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > > > > I'm quite surprised at the significant number

Re: Moving forward with TDE [PATCH v3]

2023-11-07 Thread Andres Freund
Hi, On 2023-11-06 11:26:44 +0100, Matthias van de Meent wrote: > On Sat, 4 Nov 2023 at 03:38, Andres Freund wrote: > > On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > > > I'm quite surprised at the significant number of changes being made > > > outside the core storage manager

Re: Moving forward with TDE [PATCH v3]

2023-11-07 Thread Andres Freund
Hi, On 2023-11-06 09:56:37 -0500, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > I still am quite quite unconvinced that using the LSN as a nonce is a good > > design decision. > > This is a really important part of the overall path to moving this > forward, so I wanted

Re: Moving forward with TDE [PATCH v3]

2023-11-07 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Nov 6, 2023 at 09:56:37AM -0500, Stephen Frost wrote: > > The gist is, without a suggestion of things to try, we're left > > to our own devices to try and figure out things which might be > > successful, only to have those turned

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread David Christensen
On Fri, Nov 3, 2023 at 9:53 PM Andres Freund wrote: > On 2023-11-02 19:32:28 -0700, Andres Freund wrote: > > > From 327e86d52be1df8de9c3a324cb06b85ba5db9604 Mon Sep 17 00:00:00 2001 > > > From: David Christensen > > > Date: Fri, 29 Sep 2023 15:16:00 -0400 > > > Subject: [PATCH v3 5/5] Add

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread David Christensen
Hi, thanks for the detailed feedback here. I do think it's worth addressing the question Stephen raised as far as what we use for the IV[1]; whether LSN or something else entirely, and if so what. The choice of LSN here is fairly fundamental to the existing implementation, so if we decide to do

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread Bruce Momjian
On Mon, Nov 6, 2023 at 09:56:37AM -0500, Stephen Frost wrote: > The gist is, without a suggestion of things to try, we're left > to our own devices to try and figure out things which might be > successful, only to have those turned down too when we come back with > them, see [1] for what feels

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread Bruce Momjian
On Thu, Nov 2, 2023 at 07:32:28PM -0700, Andres Freund wrote: > On 2023-10-31 16:23:17 -0500, David Christensen wrote: > > +Implementation > > +-- > > + > > +To enable cluster file encryption, the initdb option > > +--cluster-key-command must be used, which specifies a command to > >

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread Stephen Frost
Greetings, Thanks for your feedback on this. * Andres Freund (and...@anarazel.de) wrote: > I still am quite quite unconvinced that using the LSN as a nonce is a good > design decision. This is a really important part of the overall path to moving this forward, so I wanted to jump to it and have

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread Matthias van de Meent
On Sat, 4 Nov 2023 at 03:38, Andres Freund wrote: > > Hi, > > On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > > I'm quite surprised at the significant number of changes being made > > outside the core storage manager files. I thought that changing out > > mdsmgr with an encrypted

Re: Moving forward with TDE [PATCH v3]

2023-11-03 Thread Andres Freund
On 2023-11-02 19:32:28 -0700, Andres Freund wrote: > > From 327e86d52be1df8de9c3a324cb06b85ba5db9604 Mon Sep 17 00:00:00 2001 > > From: David Christensen > > Date: Fri, 29 Sep 2023 15:16:00 -0400 > > Subject: [PATCH v3 5/5] Add encrypted/authenticated WAL > > > > When using an encrypted cluster,

Re: Moving forward with TDE [PATCH v3]

2023-11-03 Thread Andres Freund
Hi, On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > I'm quite surprised at the significant number of changes being made > outside the core storage manager files. I thought that changing out > mdsmgr with an encrypted smgr (that could wrap mdsmgr if so desired) > would be the most

Re: Moving forward with TDE [PATCH v3]

2023-11-02 Thread Andres Freund
Hi, On 2023-10-31 16:23:17 -0500, David Christensen wrote: > The patches are as follows: > > 0001 - doc updates > 0002 - Basic key management and cipher support > 0003 - Backend-related changes to support heap encryption > 0004 - modifications to bin tools and programs to manage key rotation and

Re: Moving forward with TDE [PATCH v3]

2023-11-02 Thread Matthias van de Meent
On Tue, 31 Oct 2023 at 22:23, David Christensen wrote: > > Greetings, > > I am including an updated version of this patch series; it has been rebased > onto 6ec62b7799 and reworked somewhat. > > The patches are as follows: > > 0001 - doc updates > 0002 - Basic key management and cipher support >

Re: Moving forward with TDE [PATCH v3]

2023-10-31 Thread Bruce Momjian
On Tue, Oct 31, 2023 at 04:32:38PM -0500, David Christensen wrote: > On Tue, Oct 31, 2023 at 4:30 PM Bruce Momjian wrote: > Temporary /files/ are handled in a different patch set and are not included > here (not sure of the status of integrating at this point). I  believe that > this patch should

Re: Moving forward with TDE [PATCH v3]

2023-10-31 Thread David Christensen
On Tue, Oct 31, 2023 at 4:30 PM Bruce Momjian wrote: > On Tue, Oct 31, 2023 at 04:23:17PM -0500, David Christensen wrote: > > Greetings, > > > > I am including an updated version of this patch series; it has been > rebased > > onto 6ec62b7799 and reworked somewhat. > > > > The patches are as

Re: Moving forward with TDE [PATCH v3]

2023-10-31 Thread Bruce Momjian
On Tue, Oct 31, 2023 at 04:23:17PM -0500, David Christensen wrote: > Greetings, > > I am including an updated version of this patch series; it has been rebased > onto 6ec62b7799 and reworked somewhat. > > The patches are as follows: > > 0001 - doc updates > 0002 - Basic key management and

Re: Moving forward with TDE

2023-03-28 Thread Chris Travers
On Tue, Mar 28, 2023 at 8:35 AM Bruce Momjian wrote: > On Tue, Mar 28, 2023 at 02:03:50AM +0200, Stephen Frost wrote: > > The remote storage is certainly an independent system. Multi-mount LUNs > are > > entirely possible in a SAN (and absolutely with NFS, or just the NFS > server > > itself is

Re: Moving forward with TDE

2023-03-28 Thread Chris Travers
On Tue, Mar 28, 2023 at 5:02 AM Stephen Frost wrote: > > > There's clearly user demand for it as there's a number of organizations >> > who have forks which are providing it in one shape or another. This >> > kind of splintering of the community is actually an actively bad thing >> > for the

Re: Moving forward with TDE

2023-03-27 Thread Stephen Frost
Greetings, On Mon, Mar 27, 2023 at 21:35 Bruce Momjian wrote: > On Tue, Mar 28, 2023 at 02:03:50AM +0200, Stephen Frost wrote: > > The remote storage is certainly an independent system. Multi-mount LUNs > are > > entirely possible in a SAN (and absolutely with NFS, or just the NFS > server > >

Re: Moving forward with TDE

2023-03-27 Thread Bruce Momjian
On Tue, Mar 28, 2023 at 02:03:50AM +0200, Stephen Frost wrote: > The remote storage is certainly an independent system. Multi-mount LUNs are > entirely possible in a SAN (and absolutely with NFS, or just the NFS server > itself is compromised..), so while the attacker may not have any access to

Re: Moving forward with TDE

2023-03-27 Thread Stephen Frost
Greetings, On Mon, Mar 27, 2023 at 19:19 Bruce Momjian wrote: > On Tue, Mar 28, 2023 at 12:57:42AM +0200, Stephen Frost wrote: > > I consider the operating system and its processes as much more of a > > single entity than TLS over a network. > > > > This may be the case sometimes but

Re: Moving forward with TDE

2023-03-27 Thread Bruce Momjian
On Tue, Mar 28, 2023 at 12:57:42AM +0200, Stephen Frost wrote: > I consider the operating system and its processes as much more of a > single entity than TLS over a network. > > This may be the case sometimes but there’s absolutely no shortage of other > cases and it’s almost more the

Re: Moving forward with TDE

2023-03-27 Thread Stephen Frost
Greetings, On Mon, Mar 27, 2023 at 18:17 Bruce Momjian wrote: > On Tue, Mar 28, 2023 at 12:01:56AM +0200, Stephen Frost wrote: > > Greetings, > > > > On Mon, Mar 27, 2023 at 12:38 Bruce Momjian wrote: > > > > On Wed, Mar 8, 2023 at 04:25:04PM -0500, Stephen Frost wrote: > > > Agreed,

Re: Moving forward with TDE

2023-03-27 Thread Bruce Momjian
On Tue, Mar 28, 2023 at 12:01:56AM +0200, Stephen Frost wrote: > Greetings, > > On Mon, Mar 27, 2023 at 12:38 Bruce Momjian wrote: > > On Wed, Mar  8, 2023 at 04:25:04PM -0500, Stephen Frost wrote: > > Agreed, though the latest efforts include an option for *authenticated* > >

Re: Moving forward with TDE

2023-03-27 Thread Stephen Frost
Greetings, On Mon, Mar 27, 2023 at 12:38 Bruce Momjian wrote: > On Wed, Mar 8, 2023 at 04:25:04PM -0500, Stephen Frost wrote: > > Agreed, though the latest efforts include an option for *authenticated* > > encryption as well as unauthenticated. That makes it much more > > difficult to make

Re: Moving forward with TDE

2023-03-27 Thread Bruce Momjian
On Wed, Mar 8, 2023 at 04:25:04PM -0500, Stephen Frost wrote: > Agreed, though the latest efforts include an option for *authenticated* > encryption as well as unauthenticated. That makes it much more > difficult to make undetected changes to the data that's protected by > the authenticated

Re: Moving forward with TDE

2023-03-08 Thread Stephen Frost
Greetings, * Chris Travers (chris.trav...@gmail.com) wrote: > From the documentation, the primary threat model of TDE is to prevent > decryption of data from archived wal segments (and data files), for example > on a backup system. While there are other methods around this problem to > date,

Re: Moving forward with TDE

2023-03-06 Thread Chris Travers
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested I have decided to write a review here in terms of whether we want this

Re: Moving forward with TDE

2023-01-05 Thread vignesh C
On Fri, 4 Nov 2022 at 03:36, David Christensen wrote: > > > Unless somebody in the community remembers open questions/issues with > > TDE that were never addressed I suggest simply iterating with our > > usual testing/reviewing process. For now I'm going to change the > > status of the CF entry

Re: Moving forward with TDE

2022-11-17 Thread David Christensen
Hi Dilip, Thanks for the feedback here. I will review the docs changes and add to my tree. Best, David

Re: Moving forward with TDE

2022-11-17 Thread David Christensen
Hi Jacob, Thanks, I've added this patch in my tree [1]. (For now, just adding fixes and the like atop the original separate patches, but will eventually get things winnowed down into probably the same 12 parts the originals were reviewed in. Best, David [1]

Re: Moving forward with TDE

2022-11-15 Thread Jacob Champion
On Tue, Nov 15, 2022 at 11:39 AM David Christensen wrote: > Good to know about the next steps, thanks. You're welcome! > This was just a refresh of the old patches on the wiki to work as written on > HEAD. If there are known TODOs here this then that work is still needing to > be done. > > I

Re: Moving forward with TDE

2022-11-15 Thread David Christensen
> On Nov 15, 2022, at 1:08 PM, Jacob Champion wrote: > > On Mon, Oct 24, 2022 at 9:29 AM David Christensen > wrote: >> I would love to open a discussion about how to move forward and get >> some of these features built out. The historical threads here are >> quite long and complicated; is

Re: Moving forward with TDE

2022-11-15 Thread Jacob Champion
On Mon, Oct 24, 2022 at 9:29 AM David Christensen wrote: > I would love to open a discussion about how to move forward and get > some of these features built out. The historical threads here are > quite long and complicated; is there a "current state" other than the > wiki that reflects the

Re: Moving forward with TDE

2022-11-04 Thread Dilip Kumar
On Fri, Nov 4, 2022 at 3:36 AM David Christensen wrote: > > > Unless somebody in the community remembers open questions/issues with > > TDE that were never addressed I suggest simply iterating with our > > usual testing/reviewing process. For now I'm going to change the > > status of the CF entry

Re: Moving forward with TDE

2022-11-03 Thread Aleksander Alekseev
Hi David, > Working with Stephen, I am attempting to pick up some of the work that > was left off with TDE and the key management infrastructure. I have > rebased Bruce's KMS/TDE patches as they existed on the > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption wiki > page, which are