On Thu, Apr 18, 2024 at 1:49 PM Jelte Fennema-Nio wrote:
> I think this is an interesting idea. I can indeed see use cases for
> e.g. inserting a new row based on another row (where the secret is the
> same).
>
> IMHO that means that we should also bump the protocol version for this
> change,
On Thu, 18 Apr 2024 at 18:46, Robert Haas wrote:
> With regard to the Bind message, I suggest that we regard the protocol
> change as reserving a currently-unused bit in the message to indicate
> whether the value is pre-encrypted, without reference to the protocol
> extension. It could be legal
On Wed, Apr 10, 2024 at 6:13 AM Peter Eisentraut wrote:
> Obviously, it's early days, so there will be plenty of time to have
> discussions on various other aspects of this patch. I'm keeping a keen
> eye on the discussion of protocol extensions, for example.
I think the way that you handled
On Thu, 18 Apr 2024 at 13:25, Peter Eisentraut wrote:
> Hopefully, the reason for key rotation is mainly that policies require
> key rotation, not that keys get compromised all the time.
These key rotation policies are generally in place to reduce the
impact of a key compromise by limiting the
On 10.04.24 16:14, Jelte Fennema-Nio wrote:
(The CEK can't be rotated easily, since
that would require reading out all the data from a table/column and
reencrypting it. We could/should add some custom tooling for that,
but it wouldn't be a routine operation.)
This seems like something that
On Wed, 10 Apr 2024 at 12:13, Peter Eisentraut wrote:
>
> To kick some things off for PG18, here is an updated version of the
> patch for automatic client-side column-level encryption.
I only read the docs and none of the code, but here is my feedback on
the current design:
> (The CEK can't be
On 30.03.23 20:35, Stephen Frost wrote:
I do feel that column encryption is a useful capability and there's
large parts of this approach that I agree with, but I dislike the idea
of having our clients be able to depend on what gets returned for
non-encrypted columns while not being able to trust
On 30.03.23 17:55, Andres Freund wrote:
I find it very hard to belief that details of the catalog representation like
this will matter to users. How would would it conceivably affect users that we
store (key, encryption method) in pg_attribute vs storing an oid that's
effectively a foreign key
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2023-03-30 16:01:46 +0200, Peter Eisentraut wrote:
> > On 30.03.23 03:29, Andres Freund wrote:
> > > > One might think that, but the precedent in other equivalent systems is
> > > > that
> > > > you reference the key and the algorithm
Hi,
On 2023-03-30 16:01:46 +0200, Peter Eisentraut wrote:
> On 30.03.23 03:29, Andres Freund wrote:
> > > One might think that, but the precedent in other equivalent systems is
> > > that
> > > you reference the key and the algorithm separately. There is some
> > > (admittedly not very
On 30.03.23 03:29, Andres Freund wrote:
One might think that, but the precedent in other equivalent systems is that
you reference the key and the algorithm separately. There is some
(admittedly not very conclusive) discussion about this near [0].
[0]:
Hi,
On 2023-03-29 19:08:25 +0200, Peter Eisentraut wrote:
> On 29.03.23 18:24, Andres Freund wrote:
> > On 2023-03-29 18:08:29 +0200, Peter Eisentraut wrote:
> > > On 24.03.23 19:12, Andres Freund wrote:
> > > > > I thought about this some more. I think we could get rid of
> > > > >
On 29.03.23 18:24, Andres Freund wrote:
Hi,
On 2023-03-29 18:08:29 +0200, Peter Eisentraut wrote:
On 24.03.23 19:12, Andres Freund wrote:
I thought about this some more. I think we could get rid of attusertypmod
and just hardcode it as -1. The idea would be that if you ask for an
encrypted
Hi,
On 2023-03-29 18:08:29 +0200, Peter Eisentraut wrote:
> On 24.03.23 19:12, Andres Freund wrote:
> > > I thought about this some more. I think we could get rid of attusertypmod
> > > and just hardcode it as -1. The idea would be that if you ask for an
> > > encrypted column of type, say,
On 24.03.23 19:12, Andres Freund wrote:
I thought about this some more. I think we could get rid of attusertypmod
and just hardcode it as -1. The idea would be that if you ask for an
encrypted column of type, say, varchar(500), the server isn't able to
enforce that anyway, so we could just
Hi,
On 2023-03-23 14:54:48 +0100, Peter Eisentraut wrote:
> On 22.03.23 10:00, Peter Eisentraut wrote:
> > > I get that for the type, but why do we need the typmod duplicated as
> > > well?
> >
> > Earlier patch versions didn't do that, but that got really confusing
> > about which type the
On 23.03.23 16:55, Robert Haas wrote:
On Thu, Mar 23, 2023 at 9:55 AM Peter Eisentraut
wrote:
I thought about this some more. I think we could get rid of
attusertypmod and just hardcode it as -1. The idea would be that if you
ask for an encrypted column of type, say, varchar(500), the server
On Thu, Mar 23, 2023 at 9:55 AM Peter Eisentraut
wrote:
> I thought about this some more. I think we could get rid of
> attusertypmod and just hardcode it as -1. The idea would be that if you
> ask for an encrypted column of type, say, varchar(500), the server isn't
> able to enforce that
On 22.03.23 10:00, Peter Eisentraut wrote:
I get that for the type, but why do we need the typmod duplicated as
well?
Earlier patch versions didn't do that, but that got really confusing
about which type the typmod really belonged to, since code currently
assumes that typid+typmod makes
On 21.03.23 18:47, Andres Freund wrote:
On 2023-03-21 18:05:15 +0100, Peter Eisentraut wrote:
On 16.03.23 17:36, Andres Freund wrote:
Maybe a daft question, but why do we need a separate type and typmod for
encrypted columns? Why isn't the fact that the column is encrypted exactly one
new
Hi,
On 2023-03-21 18:05:15 +0100, Peter Eisentraut wrote:
> On 16.03.23 17:36, Andres Freund wrote:
> > Maybe a daft question, but why do we need a separate type and typmod for
> > encrypted columns? Why isn't the fact that the column is encrypted exactly
> > one
> > new field, and we use the
On 16.03.23 17:36, Andres Freund wrote:
Maybe a daft question, but why do we need a separate type and typmod for
encrypted columns? Why isn't the fact that the column is encrypted exactly one
new field, and we use the existing type/typmod fields?
The way this is implemented is that for an
Hi,
On 2023-03-16 11:26:46 +0100, Peter Eisentraut wrote:
> On 13.03.23 22:11, Andres Freund wrote:
> > > It adds branches, and it makes tupledescs wider. In tight spots, such as
> > > printtup, that can hurt, even if the branches aren't ever entered.
> > In fact, I do see a noticable, but not
On 13.03.23 22:11, Andres Freund wrote:
It adds branches, and it makes tupledescs wider. In tight spots, such as
printtup, that can hurt, even if the branches aren't ever entered.
In fact, I do see a noticable, but not huge, regression:
I tried to reproduce your measurements, but I don't have
Hi,
On 2023-03-13 13:41:19 -0700, Andres Freund wrote:
> On 2023-03-13 21:22:29 +0100, Peter Eisentraut wrote:
> > On 12.03.23 01:11, Andres Freund wrote:
> > > Have you done benchmarks of some simple workloads to verify this doesn't
> > > cause
> > > slowdowns (when not using encryption,
Hi,
On 2023-03-13 21:22:29 +0100, Peter Eisentraut wrote:
> On 12.03.23 01:11, Andres Freund wrote:
> > Have you done benchmarks of some simple workloads to verify this doesn't
> > cause
> > slowdowns (when not using encryption, obviously)? printtup.c is a
> > performance
> > sensitive portion
On 12.03.23 01:11, Andres Freund wrote:
Have you done benchmarks of some simple workloads to verify this doesn't cause
slowdowns (when not using encryption, obviously)? printtup.c is a performance
sensitive portion for simple queries, particularly when they return multiple
columns.
The
Hi,
On 2023-03-10 08:18:34 +0100, Peter Eisentraut wrote:
> Here is an updated patch. I have done some cosmetic polishing and fixed a
> minor Windows-related bug.
>
> In my mind, the patch is complete.
>
> If someone wants to do some in-depth code review, I suggest focusing on the
> following
> On Mar 9, 2023, at 11:18 PM, Peter Eisentraut
> wrote:
>
> Here is an updated patch.
Thanks, Peter. The patch appears to be in pretty good shape, but I have a few
comments and concerns.
CEKIsVisible() and CMKIsVisible() are obviously copied from
TSParserIsVisible(), and the code
On 12.02.23 15:48, Mark Dilger wrote:
I should mention, src/sgml/html/libpq-exec.html needs clarification:
paramFormats[]Specifies whether parameters are text (put a zero in the array
entry for the corresponding parameter) or binary (put a one in the array entry
for the corresponding
uot;.
Did you intend the use of "transparent data encryption" (rather than "transparent
column encryption") in datatype.sgml? If so, what's the difference?
There are all addressed in the v16 patch I just posted.
Is this feature intended to be available from ecpg? If so, can w
> On Feb 11, 2023, at 1:54 PM, Mark Dilger wrote:
>
> Here are some observations
I should mention, src/sgml/html/libpq-exec.html needs clarification:
> paramFormats[]Specifies whether parameters are text (put a zero in the array
> entry for the corresponding parameter) or binary (put a
ion of certain parameters in the
client library (see its documentation)" should link to
linkend="libpq-connect-column-encryption".
Did you intend the use of "transparent data encryption" (rather than
"transparent column encryption") in datatype.sgml? If so,
On Tue, Jan 31, 2023 at 5:26 AM Peter Eisentraut
wrote:
> See here for example:
> https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/always-encrypted-database-engine?view=sql-server-ver15
Hm. I notice they haven't implemented more than one algorithm, so I
wonder if
On 30.01.23 23:30, Jacob Champion wrote:
The column encryption algorithm is set per-column -- but isn't it
tightly coupled to the CEK, since the key length has to match? From a
layperson perspective, using the same key to encrypt the same plaintext
under two different algorithms (if they happen
On Wed, Jan 25, 2023 at 11:00 AM Peter Eisentraut
wrote:
> > When writing the paragraph on RSA-OAEP I was reminded that we didn't
> > really dig into the asymmetric/symmetric discussion. Assuming that most
> > first-time users will pick the builtin CMK encryption method, do we
> > still want to
On 19.01.23 21:48, Jacob Champion wrote:
I like the existing "caveats" documentation, and I've attached a sample
patch with some more caveats documented, based on some of the upthread
conversation:
- text format makes fixed-length columns leak length information too
- you only get partial
On 12.01.23 17:32, Peter Eisentraut wrote:
Can we do anything about the attack vector wherein a malicious DBA
simply copies the encrypted datum from one row to another?
We discussed this earlier [0]. This patch is not that feature. We
could get there eventually, but it would appear to be an
On 07.01.23 01:34, Justin Pryzby wrote:
"ON (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END =
t.oid)\n"
This breaks interoperability with older servers:
ERROR: column a.attrealtypid does not exist
Same in describe.c
Find attached some typos and bad indentation. I'm
On 12/31/22 06:17, Peter Eisentraut wrote:
> On 21.12.22 06:46, Peter Eisentraut wrote:
>> And another update. The main changes are that I added an 'unspecified'
>> CMK algorithm, which indicates that the external KMS knows what it is
>> but the database system doesn't. This was discussed a
On 10.01.23 18:26, Mark Dilger wrote:
I wonder if logical replication could be made to work more easily with this
feature. Specifically, subscribers of encrypted columns will need the
encrypted column encryption key (CEK) and the name of the column master key
(CMD) as exists on the
On Sat, 31 Dec 2022 at 19:47, Peter Eisentraut
wrote:
>
> On 21.12.22 06:46, Peter Eisentraut wrote:
> > And another update. The main changes are that I added an 'unspecified'
> > CMK algorithm, which indicates that the external KMS knows what it is
> > but the database system doesn't. This was
> On Jan 10, 2023, at 9:26 AM, Mark Dilger wrote:
>
>-- Cryptographically connected to the encrypted record
>patient_id BIGINT NOT NULL,
>patient_ssn CHAR(11),
>
>-- The encrypted record
>patient_record TEXT ENCRYPTED WITH (column_encryption_key = cek1,
>
> On Dec 31, 2022, at 6:17 AM, Peter Eisentraut
> wrote:
>
> Another update, with some merge conflicts resolved.
Hi Peter, thanks for the patch!
I wonder if logical replication could be made to work more easily with this
feature. Specifically, subscribers of encrypted columns will need
as a column type.
@@ -5383,7 +5383,7 @@ WHERE ...
Clients that don't support transparent column encryption or have disabled
it will see the encrypted values in this format. Clients that support
transparent data encryption will not see these types in result sets, as
-
On Wed, 23 Nov 2022 19:45:06 +0100
Peter Eisentraut wrote:
> On 28.10.22 12:16, Jehan-Guillaume de Rorthais wrote:
[...]
> >* I wonder if encryption related fields in ParameterDescription and
> > RowDescription could be optional somehow? The former might be quite
> > large when
ot support column decryption», currently buried in pg_dump page
* «When transparent column encryption is enabled, the client encoding must
match the server encoding», currently buried in the protocol description
page.
* In the libpq doc of PQexecPrepared2, "paramForceColumnEnc
Hi,
Here are a few more things I noticed :
If a CEK is encrypted with cmk1 and cmk2, but cmk1 isn't found on the
client,the following error is printed twice for the very first SELECT
statement:
could not open file "/path/to/cmk1.pem": No such file or directory
...and nothing is returned.
arly shown to user somewhere in
chapter 5.5 instead of being buried deep in documentation:
* «COPY does not support column decryption», currently buried in pg_dump page
* «When transparent column encryption is enabled, the client encoding must
match the server encoding», curren
If memory serves me correctly, if you statically link openssl this will
work. If you are using ssl in a DLL, I believe that the DLL has its own
"c-library" and its own heap.
On Thu, Oct 13, 2022 at 9:43 AM Peter Eisentraut <
peter.eisentr...@enterprisedb.com> wrote:
> On 06.10.22 17:19, Andres
On 06.10.22 17:19, Andres Freund wrote:
psql error: stderr: 'OPENSSL_Uplink(7FFC165CBD50,08): no OPENSSL_Applink'
What in the world is that about? What scant information I could find
suggests that it has something to do with building a "release" build against
an "debug" build of the
Hi,
On 2022-10-06 16:25:51 +0200, Peter Eisentraut wrote:
> On 02.10.22 09:16, Andres Freund wrote:
> > On 2022-09-27 15:51:25 +0200, Peter Eisentraut wrote:
> > > Updated version with meson build system support added (for added files and
> > > new tests).
> >
> > This fails on windows:
On 02.10.22 09:16, Andres Freund wrote:
On 2022-09-27 15:51:25 +0200, Peter Eisentraut wrote:
Updated version with meson build system support added (for added files and
new tests).
This fails on windows: https://cirrus-ci.com/task/6151847080624128
Hi,
On 2022-09-27 15:51:25 +0200, Peter Eisentraut wrote:
> Updated version with meson build system support added (for added files and
> new tests).
This fails on windows: https://cirrus-ci.com/task/6151847080624128
On Tue, Aug 30, 2022 at 4:53 AM Peter Eisentraut
wrote:
> I would be interested in learning more about such padding systems. I
> have done a lot of reading for this development project, and I have
> never come across a cryptographic approach to hide length differences by
> padding. Of course,
On 27.07.22 01:19, Jacob Champion wrote:
Now, if we don't have a padding system
built into the feature, then that does put even more on the user; it's
hard to argue with that.
Right. If they can even fix it at all. Having a well-documented padding
feature would not only help mitigate that, it
On 20.07.22 08:12, Masahiko Sawada wrote:
---
Regarding the documentation, I'd like to have a page that describes
the generic information of the transparent column encryption for users
such as what this feature actually does, what can be achieved by this
feature, CMK rotation, and its known
On 7/26/22 13:25, Robert Haas wrote:
> I certainly have no objection to being clear about such details in the
> documentation.
Cool.
> I fear the phenomenon where
> doing anything about a problem makes you responsible for the whole
> problem. If we disclaim the ability to hide the length of
On Tue, Jul 26, 2022 at 2:27 PM Jacob Champion wrote:
> Right. My point is, if you have a column that has exactly one
> important value that is 17 bytes long when converted to text, you're
> going to want to know that block size exactly, because the encryption
> will be effectively useless for
On Tue, Jul 26, 2022 at 10:52 AM Robert Haas wrote:
> On Thu, Jul 21, 2022 at 2:30 PM Jacob Champion
> wrote:
> > A minimum padding option would fix the leak here, right? If every
> > entry is the same length then there's no information to be gained, at
> > least in an offline analysis.
>
>
On Thu, Jul 21, 2022 at 2:30 PM Jacob Champion wrote:
> On Mon, Jul 18, 2022 at 9:07 AM Robert Haas wrote:
> > Even there, what can be accomplished with a feature that only encrypts
> > individual column values is by nature somewhat limited. If you have a
> > text column that, for one row,
On Mon, Jul 18, 2022 at 12:53:23PM +0200, Peter Eisentraut wrote:
> Asymmetric keys gives you some more options for how you set up the keys at
> the beginning. For example, you create the asymmetric key pair on the host
> where your client program that wants access to the encrypted data will run.
On Mon, Jul 18, 2022 at 9:07 AM Robert Haas wrote:
> Even there, what can be accomplished with a feature that only encrypts
> individual column values is by nature somewhat limited. If you have a
> text column that, for one row, stores the value 'a', and for some
> other row, stores the entire
On Mon, Jul 18, 2022 at 3:53 AM Peter Eisentraut
wrote:
> Some other products make use of secure enclaves to do computations on
> (otherwise) encrypted values on the server. I don't fully know how that
> works, but I suspect that asymmetric keys can play a role in that. (I
> don't have any
the documentation and tests and done some tests.
Before looking at the code in depth, I'd like to share my
comments/questions.
---
Regarding the documentation, I'd like to have a page that describes
the generic information of the transparent column encryption for users
such as what this feature actua
On Mon, Jul 18, 2022 at 6:53 AM Peter Eisentraut
wrote:
> I think a way to look at this is that this column encryption feature
> isn't suitable for disguising the existence or absence of data, it can
> only disguise the particular data that you know exists.
+1.
Even there, what can be
ommands need to do also depends on
what pg_dump needs them to do.
+
+ Null values are not encrypted by transparent column encryption; null values
+ sent by the client are visible as null values in the database. If the fact
+ that a value is null needs to be hidden from the server,
unencrypted columns could be provided
directly.
Another idle thought I had was that it'd be nice to have some syntax for
providing a null value to \gencr (assuming I didn't overlook it in the
patch). But that brings me to...
> +
> + Null values are not encrypted by transparent column encryption; n
On 17.12.21 01:41, Jacob Champion wrote:
(And I think the client should be able to enforce encryption in
the first place, before I distract you too much with other stuff.)
Yes, this is a useful point that I have added to my notes.
On Thu, 2021-12-09 at 11:04 +0100, Tomas Vondra wrote:
> On 12/9/21 01:12, Jacob Champion wrote:
> >
> > The rabbit hole I led you down is one where we use the rest of the row
> > as AD, to try to freeze pieces of it in place. That might(?) have some
> > useful security properties (if the client
On 16.12.21 05:47, Greg Stark wrote:
In the server, the encrypted datums are stored in types called
encryptedr and encryptedd (for randomized and deterministic
encryption). These are essentially cousins of bytea.
Does that mean someone could go in with psql and select out the data
without any
> In the server, the encrypted datums are stored in types called
> encryptedr and encryptedd (for randomized and deterministic
> encryption). These are essentially cousins of bytea.
Does that mean someone could go in with psql and select out the data
without any keys and just get a raw
On 12/9/21 01:12, Jacob Champion wrote:
> On Wed, 2021-12-08 at 02:58 +0100, Tomas Vondra wrote:
>>
>> On 12/8/21 00:26, Jacob Champion wrote:
>>> On Tue, 2021-12-07 at 22:21 +0100, Tomas Vondra wrote:
IMO it's impossible to solve this attack within TCE, because it requires
ensuring
On Wed, 2021-12-08 at 02:58 +0100, Tomas Vondra wrote:
>
> On 12/8/21 00:26, Jacob Champion wrote:
> > On Tue, 2021-12-07 at 22:21 +0100, Tomas Vondra wrote:
> > > IMO it's impossible to solve this attack within TCE, because it requires
> > > ensuring consistency at the row level, but TCE
On 12/8/21 00:26, Jacob Champion wrote:
> On Tue, 2021-12-07 at 22:21 +0100, Tomas Vondra wrote:
>> IMO it's impossible to solve this attack within TCE, because it requires
>> ensuring consistency at the row level, but TCE obviously works at column
>> level only.
>
> I was under the
On Tue, 2021-12-07 at 22:21 +0100, Tomas Vondra wrote:
> IMO it's impossible to solve this attack within TCE, because it requires
> ensuring consistency at the row level, but TCE obviously works at column
> level only.
I was under the impression that clients already had to be modified to
figure
On 12/7/21 19:02, Jacob Champion wrote:
On Tue, 2021-12-07 at 16:39 +0100, Peter Eisentraut wrote:
On 06.12.21 21:44, Jacob Champion wrote:
I think reusing a zero IV will potentially leak more information than
just equality, depending on the cipher in use. You may be interested in
synthetic
On Tue, 2021-12-07 at 16:39 +0100, Peter Eisentraut wrote:
> On 06.12.21 21:44, Jacob Champion wrote:
> > I think reusing a zero IV will potentially leak more information than
> > just equality, depending on the cipher in use. You may be interested in
> > synthetic IVs and nonce-misuse resistance
On 06.12.21 21:44, Jacob Champion wrote:
I think reusing a zero IV will potentially leak more information than
just equality, depending on the cipher in use. You may be interested in
synthetic IVs and nonce-misuse resistance (e.g. [1]), since they seem
like they would match this use case
On 06.12.21 19:28, Robert Haas wrote:
Speaking of parameter descriptions, the trickiest part of this whole
thing appears to be how to get transparently encrypted data into the
database (as opposed to reading it out). It is required to use
protocol-level prepared statements (i.e., extended
On Fri, 2021-12-03 at 22:32 +0100, Peter Eisentraut wrote:
> This feature does support deterministic
> encryption as an alternative to the default randomized encryption, so
> in that mode you can do equality lookups, at the cost of some
> security.
> + if (enc_det)
> +
On Fri, Dec 3, 2021 at 4:32 PM Peter Eisentraut
wrote:
> But it's missing the remaining 90% of the work,
> including additional DDL support, error handling, robust memory
> management, protocol versioning, forward and backward compatibility,
> pg_dump support, psql \d support, refinement of the
82 matches
Mail list logo
