On Fri, Jul 31, 2020 at 04:06:38PM +0900, Masahiko Sawada wrote: > > Given that the purpose of the key manager is to help TDE, discussing > > the SQL interface part (i.g., the second patch) deviates from the > > original purpose. I think we should discuss the design and > > implementation of the key manager first and then other additional > > interfaces. So I’ve attached a new version patch and removed the > > second patch part so that we can focus on only the key manager part. > > > > Since the previous patch sets conflicts with the current HEAD, I've > attached the rebased patch set.
I have updated the attached patch and am hoping to move this feature forward. The changes I made are: * handle merge conflicts * changed ssl initialization to match other places in our code * changed StrNCpy() to strlcpy * update the docs The first three were needed to get it to compile. I then ran some tests using the attached shell script as my password script. First, I found that initdb called the script twice. The first call worked fine, but the second call would accept a password that didn't match the first call. This is because there are no keys defined, so there is nothing for kmgr_verify_passphrase() to check for passkey verification, so it just succeeds. In fact, I can't figure out how to create any keys with the patch, and pg_encrypt() is documented, but not defined anywhere. Second, in testing starting/stopping the server, pg_ctl doesn't allow the cluster_passphrase_command to read from /dev/tty, which I think is a requirement because the command could likely require a user-supplied unlock key, even if that is not the actual passphrase, just like ssl keys. This is because pg_ctl calls setsid() just before calling execl() to start the server, and setsid() disassociates itself from the controlling terminal. I think the fix is to remove setsid() from pg_ctl and add a postmaster flag to call setsid() after it has potentially called cluster_passphrase_command, and pg_ctl would use that flag. -- Bruce Momjian <br...@momjian.us> https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
new file mode 100644
index 2768c85..44e0c1e
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
*************** COPY postgres_log FROM '/full/path/to/lo
*** 7793,7798 ****
--- 7793,7833 ----
</variablelist>
</sect1>
+ <sect1 id="runtime-config-encryption">
+ <title>Encryption Key Management</title>
+
+ <variablelist>
+ <varlistentry id="guc-cluster-passphrase-command" xreflabel="cluster_passphrase_command">
+ <term><varname>cluster_passphrase_command</varname> (<type>string</type>)
+ <indexterm>
+ <primary><varname>cluster_passphrase_command</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This option specifies an external command to be invoked when a
+ passphrase for key management system needs to be obtained.
+ </para>
+ <para>
+ The command must print the passphrase to the standard
+ output and have a zero exit code. In the parameter value,
+ <literal>%p</literal> is replaced by a prompt string. (Write
+ <literal>%%</literal> for a literal <literal>%</literal>.)
+ Note that the prompt string will probably contain whitespace,
+ so be sure to quote its use adequately. A single newline is
+ stripped from the end of the output if present. The passphrase
+ must be at least 64 bytes.
+ </para>
+ <para>
+ This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server
+ command line.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect1>
+
<sect1 id="runtime-config-client">
<title>Client Connection Defaults</title>
*************** dynamic_library_path = 'C:\tools\postgre
*** 9636,9641 ****
--- 9671,9692 ----
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="guc-key-management-enabled" xreflabel="key_management_enabled">
+ <term><varname>key_management_enabled</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary>Key management configuration parameter parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Reports whether encryption key management
+ is enabled for this cluster. See <xref
+ linkend="app-initdb-cluster-passphrase-command"/> for more
+ information.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry id="guc-data-directory-mode" xreflabel="data_directory_mode">
<term><varname>data_directory_mode</varname> (<type>integer</type>)
diff --git a/doc/src/sgml/database-encryption.sgml b/doc/src/sgml/database-encryption.sgml
new file mode 100644
index ...db84b0d
*** a/doc/src/sgml/database-encryption.sgml
--- b/doc/src/sgml/database-encryption.sgml
***************
*** 0 ****
--- 1,292 ----
+ <!-- doc/src/sgml/database-encryption.sgml -->
+
+ <chapter id="database-encryption">
+ <title>Database Encryption</title>
+
+ <indexterm zone="database-encryption">
+ <primary>Server Side Encryption</primary>
+ </indexterm>
+
+ <para>
+ The purpose of database encryption is to protect the confidential data
+ stored in a database from being revealed.
+ </para>
+
+ <sect1 id="encryption-key-management">
+ <title>Encryption Key Management</title>
+
+ <para>
+ <productname>PostgreSQL</productname> supports internal
+ <firstterm>Encryption Key Management System</firstterm>, which is designed
+ to manage the life cycles of cryptographic keys within the
+ <productname>PostgreSQL</productname>. This includes dealing with their
+ generation, storage, usage and rotation.
+ </para>
+
+ <para>
+ Encryption key management system is enabled when
+ <productname>PostgreSQL</productname> is built with
+ <literal>--with-openssl</literal> and
+ <xref linkend="app-initdb-cluster-passphrase-command"/> is specified during
+ <command>initdb</command>. The cluster passphrase provided by the
+ <option>--cluster-passphrase-command</option> option during
+ <command>initdb</command> and the one generated by
+ <xref linkend="guc-cluster-passphrase-command"/> in the
+ <filename>postgresql.conf</filename> must match, otherwise, the database
+ cluster will not start up. Note that the cluster passphrase command
+ passed to <command>initdb</command> must return a passphrase of at least
+ 64 bytes and less than 1024 bytes. For example.
+ <programlisting>
+ initdb -D dbname --cluster-passphrase-command="cat /path/to/passphrase-file"
+ </programlisting>
+ </para>
+
+ <para>
+ Encryption keys managed by <productname>PostgreSQL</productname>'s
+ key management system are only used for the internal usage such as
+ transparent data encryption in a future release. These encryption keys
+ can never be taken out of database in plaintext form. Also, the
+ encryption key management system creates some internal encryption keys.
+ There is no interface to add and remove these keys.
+ </para>
+
+ <sect2 id="key-encryption-key">
+ <title>Key Encryption Key(<acronym>KEK</acronym>)</title>
+
+ <para>
+ During the <command>initdb</command> process, the cluster passphrase
+ provided by <option>--cluster-passphrase-command</option>
+ is derived into a <firstterm>Key Encryption Key
+ (<acronym>KEK</acronym>)</firstterm>. <acronym>KEK</acronym>
+ encapsulates cryptographic keys managed inside
+ <productname>PostgreSQL</productname> described in <xref
+ linkend="key-derivations"/> using by a way of authenticated encryption
+ described in <xref linkend="key-wrapping"/> before storing the keys
+ to a persistent storage. <acronym>KEK</acronym> must be stored in
+ a trusted key store, such as key vault software or services, or a
+ hardware security module.
+ </para>
+
+ <para>
+ When a <productname>PostgreSQL</productname> server
+ with enabled encryption key management is started, the
+ <varname>cluster_passphrase_command</varname> parameter in
+ <filename>postgresql.conf</filename> will be evaluated and the cluster
+ passphrase will be derived into <acronym>KEK</acronym> in similar
+ ways as initdb.
+ </para>
+
+ <para>
+ After that, the cryptographic keys will be retrieved from
+ <filename>pg_cryptokeys</filename> directory to be restored and
+ integrity-checked by the key management system using the <acronym>KEK</acronym>.
+ If this process fails, it is likely that the cluster passphrase supplied
+ to the cluster is not the same as that supplied to the
+ <command>initdb</command> process. The cluster will refuse to start in this
+ case and user has to manually correct the cluster passphrase.
+ </para>
+
+ <para>
+ <acronym>KEK</acronym> is not stored physically within the
+ <productname>PostgreSQL</productname> server as they are designed
+ to be derived from the correctly configured cluster passphrase.
+ </para>
+ </sect2>
+
+ <sect2 id="key-derivations">
+ <title>Key Derivations</title>
+
+ <para>
+ Encryption key management systems can manage multiple
+ cryptographic keys that have different purposes and usages within
+ <productname>PostgreSQL</productname>. Currently, the Postgres
+ encryption key management system manages no cryptographic keys.
+ </para>
+ </sect2>
+
+ <sect2 id="key-wrapping">
+ <title>Key Protection</title>
+
+ <para>
+ Key management system persists cryptographic keys to the disk after wrapping
+ them by <acronym>KEK</acronym>. This section describes how key maangement
+ system wrap and unwrap key.
+ </para>
+
+ <para>
+ The key management system uses Encryption with Associated Data
+ (<acronym>AEAD</acronym>) to wrap cryptographic keys, which is
+ a form of encryption. In addition to providing a way to protect
+ confidential data from being revealed, it provides a way to check
+ the integrity and authenticity of some associated data. It uses
+ the Encrypt-Then-MAC approach, based on the Advanced Encryption
+ Standard (<acronym>AES</acronym>) in Cipher Block Chaining
+ (<acronym>CBC</acronym>) mode. It uses a random initialization
+ vector (<acronym>IV</acronym>) and a <literal>HMAC-SHA</literal>
+ message authentication code (<acronym>MAC</acronym>).
+ </para>
+
+ <para>
+ Key management system uses two kinds of cryptographic keys for key wrapping:
+ </para>
+
+ <para>
+ <variablelist>
+ <varlistentry>
+ <term><literal>Encryption Key</literal></term>
+ <listitem>
+ <para>
+ Encryption key is 256 bits long randomly generate key. It is primarily used
+ as a key for encapsulate or restore data with <acronym>AES256</acronym>
+ algorithm.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>MAC Key</literal></term>
+ <listitem>
+ <para>
+ <acronym>MAC</acronym> key is a 512-bit randomly generated key.
+ <acronym>SHA512</acronym> is the algorithm used along with the
+ <acronym>MAC</acronym> key to compute a cryptographic hash for integrity
+ purposes.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Key management systems's key wrapping algorithm is as follows:
+
+ <orderedlist>
+ <listitem>
+ <simpara>Generate random <acronym>IV</acronym>.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Add padding to the plaintext following PKCS#7 described in
+ <ulink url="https://tools.ietf.org/html/rfc2315";>RFC2315</ulink>.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Encrypt padded plain text with the <acronym>IV</acronym>
+ using <acronym>AES256</acronym> in <acronym>CBC</acronym>
+ mode.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Compute <acronym>HMAC</acronym> over the encrypted data.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Concatenate <acronym>HMAC</acronym>, <acronym>IV</acronym>
+ and encrypted ciphertext as the result of ciphertext.</simpara>
+ </listitem>
+ </orderedlist>
+ </para>
+
+ <para>
+ The length of the result ciphertext can be inferred from that of the plaintext
+ by following formula:
+ <programlisting>
+ Ciphertext Length = 64 + 16 + 16 * (floor(input_size / 16) + 1)
+ </programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="key-management-rotation">
+ <title>Key Rotation Process</title>
+
+ <para>
+ Encryption keys in general are not interminable —
+ the longer the same key is in use, the greater the chance
+ of it being breached. Performing key rotation at regular
+ intervals helps meet standardized security practices such as <ulink
+ url="https://www.pcisecuritystandards.org/";>PCI-DSS</ulink> and it is
+ a good practice in security to limit the number of encrypted bytes
+ available for a specific key version. The key lifetimes are based
+ on key length, key strength, algorithm and total number of bytes
+ enciphered. The key management system provides a efficient method to
+ perform key rotation.
+ </para>
+
+ <para>
+ Please be aware that the phrase <literal>"key rotation"</literal> here
+ only refers to the rotation of <acronym>KEK</acronym>. The cryptographic
+ keys managed by encryption key management system are not rotated; they
+ will in fact be the same before and after a <literal>"key rotation"</literal>.
+ This can be justified because the actual keys are never stored anywhere
+ physically, presented to user or captured in logging. What is being
+ rotated here is the <acronym>KEK</acronym> who is responsible for
+ encapsulating and restoring cryptographic keys.
+ </para>
+
+ <para>
+ Since <acronym>KEK</acronym> is derived from a cluster passphrase, the
+ <literal>"key rotation"</literal> ultimately refers to the rotation of
+ cluster passphrase and deriving a new <acronym>KEK</acronym> from the
+ new cluster passphrase. The new <acronym>KEK</acronym> can then be used
+ to encapsulate all encryptions keys and store the new results in
+ <filename>pg_cryptokeys</filename> directory.
+ </para>
+
+ <para>
+ To complete the cluster passphrase rotation, user needs to follow the
+ steps below:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Ensure the <productname>PostgreSQL</productname> server is running
+ correctly with KMS enabled. Passphrase rotation cannot be completed
+ with the server shut down.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Update <xref linkend="guc-cluster-passphrase-command"/> parameter and
+ load such that the new command will return a new cluster passphrase.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In a session, execute <function>pg_rotate_cluster_passphrase()</function>
+ SQL function to initiate the rotation. The function returns true upon
+ successful key rotation and false if otherwise.
+ <programlisting>
+ =# SELECT pg_rotate_cluster_passphrase();
+ pg_rotate_cluster_passphrase
+ ------------------------------
+ t
+ (1 row)
+ </programlisting>
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ Upon successful cluster passphrase rotation, all managed cryptographic
+ keys will be re-encapsulated by the new <acronym>KEK</acronym>
+ derived from the new cluster passphrase. The new encapsulated keys
+ will be stored in <filename>pg_cryptokeys</filename> directory.
+ Please note that the cryptographic keys are the same as before; the
+ rotation process only changes the <acronym>KEK</acronym> that is used
+ to encapsulate and verify the actual cryptographic keys. This way,
+ there is no need to decrypt all the encrypted data with the old keys
+ and re-encrypt them with the new.
+ </para>
+
+ <para>
+ In case of a crash during the cluster passphrase rotation
+ process, the key management system is able to recover to the
+ previous sets of cryptographic keys the next time server starts
+ up. This is possible because the key rotation and encapsulation
+ process are done on a separate temporary key directory called
+ <filename>pg_cryptokeys_tmp</filename> and it will replace
+ <filename>pg_cryptokeys</filename> and be deleted only when everything
+ is successfully finished. If the server starts with pg_cryptokeys_tmp
+ folder present, it would indicate that previous attempt of cluster
+ passphrase rotation was not completed. In this case, the server will
+ discard <filename>pg_cryptokeys_tmp</filename> folder and load the
+ keys in <filename>pg_cryptokeys</filename> as usual.
+ </para>
+ </sect2>
+ </sect1>
+ </chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
new file mode 100644
index 38e8aa0..b96f4ac
*** a/doc/src/sgml/filelist.sgml
--- b/doc/src/sgml/filelist.sgml
***************
*** 49,54 ****
--- 49,55 ----
<!ENTITY wal SYSTEM "wal.sgml">
<!ENTITY logical-replication SYSTEM "logical-replication.sgml">
<!ENTITY jit SYSTEM "jit.sgml">
+ <!ENTITY database-encryption SYSTEM "database-encryption.sgml">
<!-- programmer's guide -->
<!ENTITY bgworker SYSTEM "bgworker.sgml">
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
new file mode 100644
index e7cff98..e537e98
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
*************** SELECT m.* FROM pg_statistic_ext join pg
*** 26770,26773 ****
--- 26770,26872 ----
</sect1>
+ <sect1 id="functions-encryption">
+ <title>Encryption Functions</title>
+
+ <sect2 id="functions-data-encryption">
+ <title>Data Encryption Function</title>
+ <para>
+ The functions shown in
+ <xref linkend="functions-encryption-table"/> are for encrypting
+ and decrypting data with the <literal>SQL key</literal> described in
+ <xref linkend="encryption-key-management"/>.
+ </para>
+
+ <table id="functions-encryption-table">
+ <title>Encryption <acronym>SQL</acronym> Functions</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Function</entry>
+ <entry>Return Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+
+ <row>
+ <entry>
+ <indexterm>
+ <primary>pg_encrypt</primary>
+ </indexterm>
+ <literal><function>pg_encrypt(<parameter>data</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry>
+ <type>bytea</type>
+ </entry>
+ <entry>
+ Encrypt the given data with the internal SQL key
+ </entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
+ <primary>pg_unwrap</primary>
+ </indexterm>
+ <literal><function>pg_decrypt(<parameter>data</parameter> <type>bytea</type>)</function></literal>
+ </entry>
+ <entry>
+ <type>text</type>
+ </entry>
+ <entry>
+ Decrypt the given data with the internal SQL key
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="functions-key-management">
+ <title>Key Management Functions</title>
+ <para>
+ The function shown in
+ <xref linkend="functions-key-management-table"/> are for encryption
+ key management described in <xref linkend="encryption-key-management"/>.
+ </para>
+
+ <table id="functions-key-management-table">
+ <title>Encryption Key Management <acronym>SQL</acronym> Functions</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Function</entry>
+ <entry>Return Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <indexterm>
+ <primary>pg_rotate_cluster_passphrase</primary>
+ </indexterm>
+ <literal><function>pg_rotate_cluster_passphrase()</function></literal>
+ </entry>
+ <entry>
+ <type>boolean</type>
+ </entry>
+ <entry>
+ Rotate the cluster passphrase. See
+ <xref linkend="key-management-rotation"/> for details.
+ </entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ </sect1>
+
</chapter>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
new file mode 100644
index 3ac588d..a6e55ba
*** a/doc/src/sgml/installation.sgml
--- b/doc/src/sgml/installation.sgml
*************** build-postgresql:
*** 979,986 ****
<listitem>
<para>
Build with support for <acronym>SSL</acronym> (encrypted)
! connections. This requires the <productname>OpenSSL</productname>
! package to be installed. <filename>configure</filename> will check
for the required header files and libraries to make sure that
your <productname>OpenSSL</productname> installation is sufficient
before proceeding.
--- 979,987 ----
<listitem>
<para>
Build with support for <acronym>SSL</acronym> (encrypted)
! connections and key management. This requires the
! <productname>OpenSSL</productname> package to be installed.
! <filename>configure</filename> will check
for the required header files and libraries to make sure that
your <productname>OpenSSL</productname> installation is sufficient
before proceeding.
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
new file mode 100644
index 730d5fd..0ea7da6
*** a/doc/src/sgml/postgres.sgml
--- b/doc/src/sgml/postgres.sgml
*************** break is not needed in a wider output re
*** 171,176 ****
--- 171,177 ----
&wal;
&logical-replication;
&jit;
+ &database-encryption;
®ress;
</part>
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
new file mode 100644
index 385ac25..1d0678a
*** a/doc/src/sgml/ref/initdb.sgml
--- b/doc/src/sgml/ref/initdb.sgml
*************** PostgreSQL documentation
*** 163,168 ****
--- 163,187 ----
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-cluster-passphrase-command" xreflabel="cluster passphrase command">
+ <term><option>--cluster-passphrase-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ This option specifies an external command to be invoked when a passphrase
+ for key management system needs to be obtained.
+ </para>
+ <para>
+ The command must print the passphrase to the standard output and exit
+ with code 0. In the parameter value, <literal>%p</literal> is
+ replaced by a prompt string. (Write <literal>%%</literal> for a
+ literal <literal>%</literal>.) Note that the prompt string will
+ probably contain whitespace, so be sure to quote adequately. A single
+ newline is stripped from the end of the output if present. The passphrase
+ must be at least 64 bytes.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
<term><option>--pgdata=<replaceable class="parameter">directory</replaceable></option></term>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
new file mode 100644
index b59c569..29b0015
*** a/doc/src/sgml/ref/pgupgrade.sgml
--- b/doc/src/sgml/ref/pgupgrade.sgml
*************** psql --username=postgres --file=script.s
*** 823,828 ****
--- 823,835 ----
is down.
</para>
+ <para>
+ During the upgrade <command>pg_upgrade</command> copies the all internal keys
+ to the new cluster. If you want to upgrade from the old cluster that enables
+ the key management to the new cluster that also enables, you must use the same
+ <varname>cluster_passphrase_command</varname> to both clusters. Otherwise
+ <command>pg_upgrade</command> fails due to mismatching the cluster passphrase.
+ </para>
</refsect1>
<refsect1>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
new file mode 100644
index 3234adb..05e9b26
*** a/doc/src/sgml/storage.sgml
--- b/doc/src/sgml/storage.sgml
*************** Item
*** 78,83 ****
--- 78,88 ----
</row>
<row>
+ <entry><filename>pg_cryptokeys</filename></entry>
+ <entry>Subdirectory containing cryptographic keys</entry>
+ </row>
+
+ <row>
<entry><filename>pg_dynshmem</filename></entry>
<entry>Subdirectory containing files used by the dynamic shared memory
subsystem</entry>
diff --git a/src/backend/Makefile b/src/backend/Makefile
new file mode 100644
index 9706a95..4ace302
*** a/src/backend/Makefile
--- b/src/backend/Makefile
*************** SUBDIRS = access bootstrap catalog parse
*** 21,27 ****
main nodes optimizer partitioning port postmaster \
regex replication rewrite \
statistics storage tcop tsearch utils $(top_builddir)/src/timezone \
! jit
include $(srcdir)/common.mk
--- 21,27 ----
main nodes optimizer partitioning port postmaster \
regex replication rewrite \
statistics storage tcop tsearch utils $(top_builddir)/src/timezone \
! jit crypto
include $(srcdir)/common.mk
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
new file mode 100644
index 52a67b1..d392911
*** a/src/backend/access/transam/xlog.c
--- b/src/backend/access/transam/xlog.c
***************
*** 44,49 ****
--- 44,50 ----
#include "commands/tablespace.h"
#include "common/controldata_utils.h"
#include "executor/instrument.h"
+ #include "crypto/kmgr.h"
#include "miscadmin.h"
#include "pg_trace.h"
#include "pgstat.h"
***************
*** 81,86 ****
--- 82,88 ----
#include "utils/timestamp.h"
extern uint32 bootstrap_data_checksum_version;
+ extern uint32 bootstrap_key_management_version;
/* Unsupported old recovery command file names (relative to $PGDATA) */
#define RECOVERY_COMMAND_FILE "recovery.conf"
*************** InitControlFile(uint64 sysidentifier)
*** 4607,4612 ****
--- 4609,4615 ----
ControlFile->wal_log_hints = wal_log_hints;
ControlFile->track_commit_timestamp = track_commit_timestamp;
ControlFile->data_checksum_version = bootstrap_data_checksum_version;
+ ControlFile->key_management_version = bootstrap_key_management_version;
}
static void
*************** ReadControlFile(void)
*** 4894,4899 ****
--- 4897,4905 ----
/* Make the initdb settings visible as GUC variables, too */
SetConfigOption("data_checksums", DataChecksumsEnabled() ? "yes" : "no",
PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ SetConfigOption("key_management", KeyManagementEnabled() ? "yes" : "no",
+ PGC_INTERNAL, PGC_S_OVERRIDE);
}
/*
*************** DataChecksumsEnabled(void)
*** 4937,4942 ****
--- 4943,4958 ----
}
/*
+ * Are key management enabled?
+ */
+ bool
+ KeyManagementEnabled(void)
+ {
+ Assert(ControlFile != NULL);
+ return (ControlFile->key_management_version > 0);
+ }
+
+ /*
* Returns a fake LSN for unlogged relations.
*
* Each call generates an LSN that is greater than any previous value
*************** BootStrapXLOG(void)
*** 5343,5348 ****
--- 5359,5368 ----
/* some additional ControlFile fields are set in WriteControlFile() */
WriteControlFile();
+ /* Enable key manager if required */
+ if (ControlFile->key_management_version > 0)
+ BootStrapKmgr();
+
/* Bootstrap the commit log, too */
BootStrapCLOG();
BootStrapCommitTs();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
new file mode 100644
index 76b2f50..8c46b20
*** a/src/backend/bootstrap/bootstrap.c
--- b/src/backend/bootstrap/bootstrap.c
***************
*** 28,33 ****
--- 28,34 ----
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/link-canary.h"
+ #include "crypto/kmgr.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
***************
*** 51,56 ****
--- 52,58 ----
#include "utils/relmapper.h"
uint32 bootstrap_data_checksum_version = 0; /* No checksum */
+ uint32 bootstrap_key_management_version = 0; /* disabled */
static void CheckerModeMain(void);
*************** AuxiliaryProcessMain(int argc, char *arg
*** 224,230 ****
/* If no -x argument, we are a CheckerProcess */
MyAuxProcType = CheckerProcess;
! while ((flag = getopt(argc, argv, "B:c:d:D:Fkr:x:X:-:")) != -1)
{
switch (flag)
{
--- 226,232 ----
/* If no -x argument, we are a CheckerProcess */
MyAuxProcType = CheckerProcess;
! while ((flag = getopt(argc, argv, "B:c:d:D:eFkr:x:X:-:")) != -1)
{
switch (flag)
{
*************** AuxiliaryProcessMain(int argc, char *arg
*** 247,252 ****
--- 249,257 ----
pfree(debugstr);
}
break;
+ case 'e':
+ bootstrap_key_management_version = KMGR_VERSION;
+ break;
case 'F':
SetConfigOption("fsync", "false", PGC_POSTMASTER, PGC_S_ARGV);
break;
diff --git a/src/backend/crypto/Makefile b/src/backend/crypto/Makefile
new file mode 100644
index ...c273620
*** a/src/backend/crypto/Makefile
--- b/src/backend/crypto/Makefile
***************
*** 0 ****
--- 1,18 ----
+ #-------------------------------------------------------------------------
+ #
+ # Makefile
+ # Makefile for src/backend/crypto
+ #
+ # IDENTIFICATION
+ # src/backend/crypto/Makefile
+ #
+ #-------------------------------------------------------------------------
+
+ subdir = src/backend/crypto
+ top_builddir = ../../..
+ include $(top_builddir)/src/Makefile.global
+
+ OBJS = \
+ kmgr.o
+
+ include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/crypto/kmgr.c b/src/backend/crypto/kmgr.c
new file mode 100644
index ...2a37c38
*** a/src/backend/crypto/kmgr.c
--- b/src/backend/crypto/kmgr.c
***************
*** 0 ****
--- 1,457 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr.c
+ * Key manager routines
+ *
+ * Key manager is enabled if user requests during initdb. During bootstrap,
+ * we generate internal keys, wrap them with KEK which is derived from the
+ * user-provided passphrase, and store them into each file located at KMGR_DIR.
+ * Once generated, these are not changed. During startup, we decrypt all
+ * internal keys and load them to the shared memory space. Internal keys on
+ * the shared memory are read-only. All wrapping and unwrapping key routines
+ * depends on openssl library for now.
+ *
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/crypto/kmgr.c
+ *-------------------------------------------------------------------------
+ */
+
+ #include "postgres.h"
+
+ #include <sys/stat.h>
+ #include <unistd.h>
+
+ #include "funcapi.h"
+ #include "miscadmin.h"
+ #include "pgstat.h"
+
+ #include "common/sha2.h"
+ #include "common/kmgr_utils.h"
+ #include "crypto/kmgr.h"
+ #include "storage/fd.h"
+ #include "storage/ipc.h"
+ #include "storage/shmem.h"
+ #include "utils/builtins.h"
+ #include "utils/guc.h"
+ #include "utils/memutils.h"
+
+ /* Struct stores internal keys in plaintext format */
+ typedef struct KmgrShmemData
+ {
+ /*
+ * Internal cryptographic keys. Keys are stored at its ID'th.
+ */
+ CryptoKey intlKeys[KMGR_MAX_INTERNAL_KEYS];
+ } KmgrShmemData;
+ static KmgrShmemData *KmgrShmem;
+
+ /*
+ * Key lengths in bytes of internal keys.
+ *
+ * No key supported for now.
+ */
+ static int internalKeyLengths[KMGR_MAX_INTERNAL_KEYS];
+
+ /* GUC variables */
+ bool key_management_enabled = false;;
+ char *cluster_passphrase_command = NULL;
+
+ static void KmgrSaveCryptoKeys(const char *dir, CryptoKey *keys);
+ static CryptoKey *generate_crypto_key(int len);
+ static void recoverIncompleteRotation(void);
+
+ /*
+ * This function must be called ONCE on system install.
+ */
+ void
+ BootStrapKmgr(void)
+ {
+ PgKeyWrapCtx *ctx;
+ CryptoKey keys_wrap[KMGR_MAX_INTERNAL_KEYS];
+ char passphrase[KMGR_MAX_PASSPHRASE_LEN];
+ uint8 kekenc[KMGR_ENC_KEY_LEN];
+ uint8 kekhmac[KMGR_MAC_KEY_LEN];
+ int passlen;
+
+ /*
+ * Requirement check. We need openssl library to enable key management
+ * because all encryption and decryption calls happen via openssl function
+ * calls.
+ */
+ #ifndef USE_OPENSSL
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ (errmsg("cluster encryption is not supported because OpenSSL is not supported by this build"),
+ errhint("Compile with --with-openssl to use cluster encryption."))));
+ #endif
+
+ memset(keys_wrap, 0, sizeof(keys_wrap));
+
+ /* Get key encryption key from the passphrase command */
+ passlen = kmgr_run_cluster_passphrase_command(cluster_passphrase_command,
+ passphrase, KMGR_MAX_PASSPHRASE_LEN);
+ if (passlen < KMGR_MIN_PASSPHRASE_LEN)
+ ereport(ERROR,
+ (errmsg("passphrase must be more than %d bytes",
+ KMGR_MIN_PASSPHRASE_LEN)));
+
+ /* Get key encryption key and HMAC key from passphrase */
+ kmgr_derive_keys(passphrase, passlen, kekenc, kekhmac);
+
+ /* Create temporarily key wrap context */
+ ctx = pg_create_keywrap_ctx(kekenc, kekhmac);
+ if (!ctx)
+ elog(ERROR, "could not initialize encryption contect");
+
+ /* Wrap all internal keys by key encryption key */
+ for (int id = 0; id < KMGR_MAX_INTERNAL_KEYS; id++)
+ {
+ CryptoKey *key;
+
+ /* generate an internal key */
+ key = generate_crypto_key(internalKeyLengths[id]);
+
+ if (!kmgr_wrap_key(ctx, key, &(keys_wrap[id])))
+ {
+ pg_free_keywrap_ctx(ctx);
+ elog(ERROR, "failed to wrap cluster encryption key");
+ }
+ }
+
+ /* Save internal keys to the disk */
+ KmgrSaveCryptoKeys(KMGR_DIR, keys_wrap);
+
+ pg_free_keywrap_ctx(ctx);
+ }
+
+ /* Report shared-memory space needed by KmgrShmem */
+ Size
+ KmgrShmemSize(void)
+ {
+ if (!key_management_enabled)
+ return 0;
+
+ return MAXALIGN(sizeof(KmgrShmemData));
+ }
+
+ /* Allocate and initialize key manager memory */
+ void
+ KmgrShmemInit(void)
+ {
+ bool found;
+
+ if (!key_management_enabled)
+ return;
+
+ KmgrShmem = (KmgrShmemData *) ShmemInitStruct("Key manager",
+ KmgrShmemSize(), &found);
+
+ if (!found)
+ memset(KmgrShmem, 0, KmgrShmemSize());
+ }
+
+ /*
+ * Get encryption key passphrase and verify it, then get the internal keys.
+ * This function is called by postmaster at startup time.
+ */
+ void
+ InitializeKmgr(void)
+ {
+ CryptoKey *keys_wrap;
+ char passphrase[KMGR_MAX_PASSPHRASE_LEN];
+ int passlen;
+ int nkeys;
+
+ if (!key_management_enabled)
+ return;
+
+ elog(DEBUG1, "starting up key management system");
+
+ /* Recover the failure of the last passphrase rotation if necessary */
+ recoverIncompleteRotation();
+
+ /* Get the crypto keys from the file */
+ keys_wrap = kmgr_get_cryptokeys(KMGR_DIR, &nkeys);
+ Assert(nkeys == KMGR_MAX_INTERNAL_KEYS);
+
+ /* Get cluster passphrase */
+ passlen = kmgr_run_cluster_passphrase_command(cluster_passphrase_command,
+ passphrase, KMGR_MAX_PASSPHRASE_LEN);
+
+ /*
+ * Verify passphrase and prepare an internal key in plaintext on shared memory.
+ *
+ * XXX: do we need to prevent internal keys from being swapped out using
+ * mlock?
+ */
+ if (!kmgr_verify_passphrase(passphrase, passlen, keys_wrap, KmgrShmem->intlKeys,
+ KMGR_MAX_INTERNAL_KEYS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cluster passphrase does not match expected passphrase")));
+ }
+
+ const CryptoKey *
+ KmgrGetKey(int id)
+ {
+ Assert(id < KMGR_MAX_INTERNAL_KEYS);
+
+ return (const CryptoKey *) &(KmgrShmem->intlKeys[id]);
+ }
+
+ /* Generate an empty CryptoKey */
+ static CryptoKey *
+ generate_crypto_key(int len)
+ {
+ CryptoKey *newkey;
+
+ Assert(len < KMGR_MAX_KEY_LEN);
+ newkey = (CryptoKey *) palloc0(sizeof(CryptoKey));
+
+ if (!pg_strong_random(newkey->key, len))
+ elog(ERROR, "failed to generate new crypto key");
+
+ newkey->klen = len;
+
+ return newkey;
+ }
+
+ /*
+ * Save the given crypto keys to the disk. We don't need CRC check for crypto
+ * keys because these keys have HMAC which is used for integrity check
+ * during unwrapping.
+ */
+ static void
+ KmgrSaveCryptoKeys(const char *dir, CryptoKey *keys)
+ {
+ elog(DEBUG2, "saving all cryptographic keys");
+
+ for (int i = 0; i < KMGR_MAX_INTERNAL_KEYS; i++)
+ {
+ int fd;
+ char path[MAXPGPATH];
+
+ CryptoKeyFilePath(path, dir, i);
+
+ if ((fd = BasicOpenFile(path, O_RDWR | O_CREAT | O_EXCL | PG_BINARY)) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m",
+ path)));
+
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_WRITE);
+ if (write(fd, &(keys[i]), sizeof(CryptoKey)) != sizeof(CryptoKey))
+ {
+ /* if write didn't set errno, assume problem is no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ path)));
+ }
+ pgstat_report_wait_end();
+
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_SYNC);
+ if (pg_fsync(fd) != 0)
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not fsync file \"%s\": %m",
+ path)));
+ pgstat_report_wait_end();
+
+ if (close(fd) != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m",
+ path)));
+ }
+ }
+
+
+ /*
+ * Check the last passphrase rotation was completed. If not, we decide which wrapped
+ * keys will be used according to the status of temporary directory and its wrapped
+ * keys.
+ */
+ static void
+ recoverIncompleteRotation(void)
+ {
+ struct stat st;
+ struct stat st_tmp;
+ CryptoKey *keys;
+ int nkeys_tmp;
+
+ /* The cluster passphrase rotation was completed, nothing to do */
+ if (stat(KMGR_TMP_DIR, &st_tmp) != 0)
+ return;
+
+ /*
+ * If there is only temporary directory, it means that the previous
+ * rotation failed after wrapping the all internal keys by the new
+ * passphrase. Therefore we use the new cluster passphrase.
+ */
+ if (stat(KMGR_DIR, &st) != 0)
+ {
+ ereport(DEBUG1,
+ (errmsg("there is only temporary directory, use the newly wrapped keys")));
+
+ if (rename(KMGR_TMP_DIR, KMGR_DIR) != 0)
+ ereport(ERROR,
+ errmsg("could not rename directory \"%s\" to \"%s\": %m",
+ KMGR_TMP_DIR, KMGR_DIR));
+ ereport(LOG,
+ errmsg("cryptographic keys wrapped by new passphrase command are chosen"),
+ errdetail("last cluster passphrase rotation failed in the middle"));
+ return;
+ }
+
+ /*
+ * In case where both the original directory and temporary directory
+ * exist, there are two possibilities: (a) the all internal keys are
+ * wrapped by the new passphrase but rotation failed before removing the
+ * original directory, or (b) the rotation failed during wrapping internal
+ * keys by the new passphrase. In case of (a) we need to use the wrapped
+ * keys in the temporary directory as rotation is essentially completed,
+ * but in case of (b) we use the wrapped keys in the original directory.
+ *
+ * To check the possibility of (b) we validate the wrapped keys in the
+ * temporary directory by checking the number of wrapped keys. Since the
+ * wrapped key length is smaller than one disk sector, which is 512 bytes
+ * on common hardware, saving wrapped key is atomic write. So we can
+ * ensure that the all wrapped keys are valid if the number of wrapped
+ * keys in the temporary directory is KMGR_MAX_INTERNAL_KEYS.
+ */
+ keys = kmgr_get_cryptokeys(KMGR_TMP_DIR, &nkeys_tmp);
+
+ if (nkeys_tmp == KMGR_MAX_INTERNAL_KEYS)
+ {
+ /*
+ * This is case (a), the all wrapped keys in temporary directory are
+ * valid. Remove the original directory and rename.
+ */
+ ereport(DEBUG1,
+ (errmsg("last passphrase rotation failed before renaming direcotry name, use the newly wrapped keys")));
+
+ if (!rmtree(KMGR_DIR, true))
+ ereport(ERROR,
+ (errmsg("could not remove directory \"%s\"",
+ KMGR_DIR)));
+ if (rename(KMGR_TMP_DIR, KMGR_DIR) != 0)
+ ereport(ERROR,
+ errmsg("could not rename directory \"%s\" to \"%s\": %m",
+ KMGR_TMP_DIR, KMGR_DIR));
+
+ ereport(LOG,
+ errmsg("cryptographic keys wrapped by new passphrase command are chosen"),
+ errdetail("last cluster passphrase rotation failed in the middle"));
+ }
+ else
+ {
+ /*
+ * This is case (b), the last passphrase rotation failed during
+ * wrapping keys. Remove the keys in the temporary directory and use
+ * keys in the original keys.
+ */
+ ereport(DEBUG1,
+ (errmsg("last passphrase rotation failed during wrapping keys, use the old wrapped keys")));
+
+ if (!rmtree(KMGR_TMP_DIR, true))
+ ereport(ERROR,
+ (errmsg("could not remove directory \"%s\"",
+ KMGR_DIR)));
+ ereport(LOG,
+ errmsg("cryptographic keys wrapped by old passphrase command are chosen"),
+ errdetail("last cluster passphrase rotation failed in the middle"));
+ }
+
+ pfree(keys);
+ }
+
+ /*
+ * SQL function to rotate the cluster passphrase. This function assumes that
+ * the cluster_passphrase_command is already reloaded to the new value.
+ * All internal keys are wrapped by the new passphrase and saved to the disk.
+ * To update all crypto keys atomically we save the newly wrapped keys to the
+ * temporary directory, pg_cryptokeys_tmp, and remove the original directory,
+ * pg_cryptokeys, and rename it. These operation is performed without the help
+ * of WAL. In the case of failure during rotationpg_cryptokeys directory and
+ * pg_cryptokeys_tmp directory can be left in incomplete status. We recover
+ * the incomplete situation by checkIncompleteRotation.
+ */
+ Datum
+ pg_rotate_cluster_passphrase(PG_FUNCTION_ARGS)
+ {
+ PgKeyWrapCtx *ctx;
+ CryptoKey newkeys[KMGR_MAX_INTERNAL_KEYS];
+ char passphrase[KMGR_MAX_PASSPHRASE_LEN];
+ uint8 new_kekenc[KMGR_ENC_KEY_LEN];
+ uint8 new_kekhmac[KMGR_MAC_KEY_LEN];
+ int passlen;
+
+ if (!key_management_enabled)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("could not rotate cluster passphrase because key management is not supported")));
+
+ memset(newkeys, 0, sizeof(newkeys));
+
+ /* Recover the failure of the last passphrase rotation if necessary */
+ recoverIncompleteRotation();
+
+ passlen = kmgr_run_cluster_passphrase_command(cluster_passphrase_command,
+ passphrase,
+ KMGR_MAX_PASSPHRASE_LEN);
+ if (passlen < KMGR_MIN_PASSPHRASE_LEN)
+ ereport(ERROR,
+ (errmsg("passphrase must be more than %d bytes",
+ KMGR_MIN_PASSPHRASE_LEN)));
+
+ /* Get new key encryption key and encryption context */
+ kmgr_derive_keys(passphrase, passlen, new_kekenc, new_kekhmac);
+ ctx = pg_create_keywrap_ctx(new_kekenc, new_kekhmac);
+ if (!ctx)
+ elog(ERROR, "could not initialize encryption contect");
+
+ for (int id = 0; id < KMGR_MAX_INTERNAL_KEYS; id++)
+ {
+ if (!kmgr_wrap_key(ctx, &(KmgrShmem->intlKeys[id]), &(newkeys[id])))
+ elog(ERROR, "failed to wrap key");
+ }
+
+ /* Create temporary directory */
+ if (MakePGDirectory(KMGR_TMP_DIR) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create temporary directory \"%s\": %m",
+ KMGR_TMP_DIR)));
+ fsync_fname(KMGR_TMP_DIR, true);
+
+ /* Prevent concurrent key rotation */
+ LWLockAcquire(KmgrFileLock, LW_EXCLUSIVE);
+
+ /* Save the key wrapped by the new passphrase to the temporary directory */
+ KmgrSaveCryptoKeys(KMGR_TMP_DIR, newkeys);
+
+ /* Remove the original directory */
+ if (!rmtree(KMGR_DIR, true))
+ ereport(ERROR,
+ (errmsg("could not remove directory \"%s\"",
+ KMGR_DIR)));
+
+ /* Rename to the original directory */
+ if (rename(KMGR_TMP_DIR, KMGR_DIR) != 0)
+ ereport(ERROR,
+ (errmsg("could not rename directory \"%s\" to \"%s\": %m",
+ KMGR_TMP_DIR, KMGR_DIR)));
+ fsync_fname(KMGR_DIR, true);
+
+ LWLockRelease(KmgrFileLock);
+
+ pg_free_keywrap_ctx(ctx);
+ PG_RETURN_BOOL(true);
+ }
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
new file mode 100644
index 8b21ff4..744fa96
*** a/src/backend/libpq/be-secure-openssl.c
--- b/src/backend/libpq/be-secure-openssl.c
***************
*** 30,35 ****
--- 30,36 ----
#endif
#include <openssl/ssl.h>
+ #include <openssl/conf.h>
#include <openssl/dh.h>
#include <openssl/conf.h>
#ifndef OPENSSL_NO_ECDH
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
new file mode 100644
index 822f0eb..a1faea7
*** a/src/backend/postmaster/pgstat.c
--- b/src/backend/postmaster/pgstat.c
*************** pgstat_get_wait_io(WaitEventIO w)
*** 4134,4139 ****
--- 4134,4148 ----
case WAIT_EVENT_DSM_FILL_ZERO_WRITE:
event_name = "DSMFillZeroWrite";
break;
+ case WAIT_EVENT_KEY_FILE_READ:
+ event_name = "KeyFileRead";
+ break;
+ case WAIT_EVENT_KEY_FILE_WRITE:
+ event_name = "KeyFileWrite";
+ break;
+ case WAIT_EVENT_KEY_FILE_SYNC:
+ event_name = "KeyFileSync";
+ break;
case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ:
event_name = "LockFileAddToDataDirRead";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
new file mode 100644
index 959e3b8..6ebab83
*** a/src/backend/postmaster/postmaster.c
--- b/src/backend/postmaster/postmaster.c
***************
*** 100,105 ****
--- 100,106 ----
#include "common/file_perm.h"
#include "common/ip.h"
#include "common/string.h"
+ #include "crypto/kmgr.h"
#include "lib/ilist.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
*************** PostmasterMain(int argc, char *argv[])
*** 1351,1356 ****
--- 1352,1362 ----
autovac_init();
/*
+ * Initialize key manager.
+ */
+ InitializeKmgr();
+
+ /*
* Load configuration files for client authentication.
*/
if (!load_hba())
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
new file mode 100644
index b89df01..6ad051d
*** a/src/backend/replication/basebackup.c
--- b/src/backend/replication/basebackup.c
***************
*** 18,23 ****
--- 18,24 ----
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+ #include "common/kmgr_utils.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
*************** struct exclude_list_item
*** 152,157 ****
--- 153,161 ----
*/
static const char *const excludeDirContents[] =
{
+ /* Skip temporary crypto key files */
+ KMGR_TMP_DIR,
+
/*
* Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
* when stats_temp_directory is set because PGSS_TEXT_FILE is always
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
new file mode 100644
index 96c2aaa..64fe10c
*** a/src/backend/storage/ipc/ipci.c
--- b/src/backend/storage/ipc/ipci.c
***************
*** 23,28 ****
--- 23,29 ----
#include "access/syncscan.h"
#include "access/twophase.h"
#include "commands/async.h"
+ #include "crypto/kmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
*************** CreateSharedMemoryAndSemaphores(void)
*** 149,154 ****
--- 150,156 ----
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, KmgrShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
*************** CreateSharedMemoryAndSemaphores(void)
*** 267,272 ****
--- 269,275 ----
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ KmgrShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
new file mode 100644
index 774292f..a44805d
*** a/src/backend/storage/lmgr/lwlocknames.txt
--- b/src/backend/storage/lmgr/lwlocknames.txt
*************** XactTruncationLock 44
*** 53,55 ****
--- 53,56 ----
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+ KmgrFileLock 48
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
new file mode 100644
index 411cfad..7572f18
*** a/src/backend/tcop/postgres.c
--- b/src/backend/tcop/postgres.c
***************
*** 42,47 ****
--- 42,48 ----
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/prepare.h"
+ #include "crypto/kmgr.h"
#include "executor/spi.h"
#include "jit/jit.h"
#include "libpq/libpq.h"
*************** PostgresMain(int argc, char *argv[],
*** 3902,3907 ****
--- 3903,3915 ----
BaseInit();
/*
+ * Initialize kmgr for cluster encryption. Since kmgr needs to attach to
+ * shared memory the initialization must be called after BaseInit().
+ */
+ if (!IsUnderPostmaster)
+ InitializeKmgr();
+
+ /*
* Create a per-backend PGPROC struct in shared memory, except in the
* EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
* this before we can use LWLocks (and in the EXEC_BACKEND case we already
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index a62d64e..411f068
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 47,52 ****
--- 47,53 ----
#include "commands/vacuum.h"
#include "commands/variable.h"
#include "common/string.h"
+ #include "crypto/kmgr.h"
#include "funcapi.h"
#include "jit/jit.h"
#include "libpq/auth.h"
*************** const char *const config_group_names[] =
*** 745,750 ****
--- 746,753 ----
gettext_noop("Statistics / Monitoring"),
/* STATS_COLLECTOR */
gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* ENCRYPTION */
+ gettext_noop("Encryption"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
*************** static struct config_bool ConfigureNames
*** 2036,2041 ****
--- 2039,2055 ----
NULL, NULL, NULL
},
+ {
+ {"key_management", PGC_INTERNAL, PRESET_OPTIONS,
+ gettext_noop("Show whether key management is enabled for this cluster."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ },
+ &key_management_enabled,
+ false,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
*************** static struct config_string ConfigureNam
*** 4392,4397 ****
--- 4406,4421 ----
"",
NULL, NULL, NULL
},
+
+ {
+ {"cluster_passphrase_command", PGC_SIGHUP, ENCRYPTION,
+ gettext_noop("Command to obtain passphrase for database encryption."),
+ NULL
+ },
+ &cluster_passphrase_command,
+ "",
+ NULL, NULL, NULL
+ },
{
{"application_name", PGC_USERSET, LOGGING_WHAT,
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
new file mode 100644
index 9cb571f..fc30641
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 633,638 ****
--- 633,643 ----
# autovacuum, -1 means use
# vacuum_cost_limit
+ #------------------------------------------------------------------------------
+ # ENCRYPTION
+ #------------------------------------------------------------------------------
+
+ #cluster_passphrase_command = ''
#------------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
new file mode 100644
index ee3bfa8..f088151
*** a/src/bin/initdb/initdb.c
--- b/src/bin/initdb/initdb.c
*************** static bool data_checksums = false;
*** 146,151 ****
--- 146,152 ----
static char *xlog_dir = NULL;
static char *str_wal_segment_size_mb = NULL;
static int wal_segment_size_mb;
+ static char *cluster_passphrase = NULL;
/* internal vars */
*************** static const char *const subdirs[] = {
*** 203,208 ****
--- 204,210 ----
"global",
"pg_wal/archive_status",
"pg_commit_ts",
+ "pg_cryptokeys",
"pg_dynshmem",
"pg_notify",
"pg_serial",
*************** setup_config(void)
*** 1185,1190 ****
--- 1187,1199 ----
"password_encryption = md5");
}
+ if (cluster_passphrase)
+ {
+ snprintf(repltok, sizeof(repltok), "cluster_passphrase_command = '%s'",
+ escape_quotes(cluster_passphrase));
+ conflines = replace_token(conflines, "#cluster_passphrase_command = ''", repltok);
+ }
+
/*
* If group access has been enabled for the cluster then it makes sense to
* ensure that the log files also allow group access. Otherwise a backup
*************** bootstrap_template1(void)
*** 1395,1408 ****
unsetenv("PGCLIENTENCODING");
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x1 -X %u %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
boot_options,
debug ? "-d 5" : "");
-
PG_CMD_OPEN;
for (line = bki_lines; *line != NULL; line++)
--- 1404,1417 ----
unsetenv("PGCLIENTENCODING");
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x1 -X %u %s %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
+ cluster_passphrase ? "-e" : "",
boot_options,
debug ? "-d 5" : "");
PG_CMD_OPEN;
for (line = bki_lines; *line != NULL; line++)
*************** usage(const char *progname)
*** 2290,2295 ****
--- 2299,2306 ----
printf(_(" --wal-segsize=SIZE size of WAL segments, in megabytes\n"));
printf(_("\nLess commonly used options:\n"));
printf(_(" -d, --debug generate lots of debugging output\n"));
+ printf(_(" -c --cluster-passphrase-command=COMMAND\n"
+ " set command to obtain passphrase for key management\n"));
printf(_(" -k, --data-checksums use data page checksums\n"));
printf(_(" -L DIRECTORY where to find the input files\n"));
printf(_(" -n, --no-clean do not clean up after errors\n"));
*************** main(int argc, char *argv[])
*** 2958,2963 ****
--- 2969,2975 ----
{"wal-segsize", required_argument, NULL, 12},
{"data-checksums", no_argument, NULL, 'k'},
{"allow-group-access", no_argument, NULL, 'g'},
+ {"cluster-passphrase-command", required_argument, NULL, 'c'},
{NULL, 0, NULL, 0}
};
*************** main(int argc, char *argv[])
*** 2999,3005 ****
/* process command-line options */
! while ((c = getopt_long(argc, argv, "dD:E:kL:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
{
switch (c)
{
--- 3011,3017 ----
/* process command-line options */
! while ((c = getopt_long(argc, argv, "c:dD:E:kL:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
{
switch (c)
{
*************** main(int argc, char *argv[])
*** 3081,3086 ****
--- 3093,3101 ----
case 9:
pwfilename = pg_strdup(optarg);
break;
+ case 'c':
+ cluster_passphrase = pg_strdup(optarg);
+ break;
case 's':
show_setting = true;
break;
*************** main(int argc, char *argv[])
*** 3151,3156 ****
--- 3166,3179 ----
exit(1);
}
+ #ifndef USE_OPENSSL
+ if (cluster_passphrase)
+ {
+ pg_log_error("cluster encryption is not supported because OpenSSL is not supported by this build");
+ exit(1);
+ }
+ #endif
+
check_authmethod_unspecified(&authmethodlocal);
check_authmethod_unspecified(&authmethodhost);
*************** main(int argc, char *argv[])
*** 3218,3223 ****
--- 3241,3251 ----
else
printf(_("Data page checksums are disabled.\n"));
+ if (cluster_passphrase)
+ printf(_("Key management system is enabled.\n"));
+ else
+ printf(_("Key management system is disabled.\n"));
+
if (pwprompt || pwfilename)
get_su_pwd();
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
new file mode 100644
index 3e00ac0..39ba7f0
*** a/src/bin/pg_controldata/pg_controldata.c
--- b/src/bin/pg_controldata/pg_controldata.c
***************
*** 25,30 ****
--- 25,31 ----
#include "access/xlog_internal.h"
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
+ #include "common/kmgr_utils.h"
#include "common/logging.h"
#include "getopt_long.h"
#include "pg_getopt.h"
*************** main(int argc, char *argv[])
*** 334,338 ****
--- 335,341 ----
ControlFile->data_checksum_version);
printf(_("Mock authentication nonce: %s\n"),
mock_auth_nonce_str);
+ printf(_("Key management version: %u\n"),
+ ControlFile->key_management_version);
return 0;
}
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
new file mode 100644
index cb6ef19..a9d200e
*** a/src/bin/pg_resetwal/pg_resetwal.c
--- b/src/bin/pg_resetwal/pg_resetwal.c
*************** PrintControlValues(bool guessed)
*** 804,809 ****
--- 804,811 ----
(ControlFile.float8ByVal ? _("by value") : _("by reference")));
printf(_("Data page checksum version: %u\n"),
ControlFile.data_checksum_version);
+ printf(_("Key management version: %u\n"),
+ ControlFile.key_management_version);
}
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
new file mode 100644
index 1abc257..3084ae7
*** a/src/bin/pg_rewind/filemap.c
--- b/src/bin/pg_rewind/filemap.c
*************** static const char *excludeDirContents[]
*** 74,79 ****
--- 74,87 ----
"pg_notify",
/*
+ * Skip cryptographic keys. It's generally not good idea to copy the
+ * cryptographic keys from source database because these might use
+ * different cluster passphrase.
+ */
+ "pg_cryptokeys", /* defined as KMGR_DIR */
+ "pg_cryptokeys_tmp", /* defined as KMGR_TMP_DIR */
+
+ /*
* Old contents are loaded for possible debugging but are not required for
* normal operation, see SerialInit().
*/
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
new file mode 100644
index 39bcaa8..e5794ff
*** a/src/bin/pg_upgrade/controldata.c
--- b/src/bin/pg_upgrade/controldata.c
***************
*** 9,18 ****
--- 9,24 ----
#include "postgres_fe.h"
+ #include <dirent.h>
#include <ctype.h>
#include "pg_upgrade.h"
+ #include "access/xlog_internal.h"
+ #include "common/controldata_utils.h"
+ #include "common/file_utils.h"
+ #include "common/kmgr_utils.h"
+
/*
* get_control_data()
*
*************** get_control_data(ClusterInfo *cluster, b
*** 59,64 ****
--- 65,71 ----
bool got_date_is_int = false;
bool got_data_checksum_version = false;
bool got_cluster_state = false;
+ bool got_key_management_enabled = false;
char *lc_collate = NULL;
char *lc_ctype = NULL;
char *lc_monetary = NULL;
*************** get_control_data(ClusterInfo *cluster, b
*** 202,207 ****
--- 209,221 ----
got_data_checksum_version = true;
}
+ /* Only in <= 14 */
+ if (GET_MAJOR_VERSION(cluster->major_version) <= 1400)
+ {
+ cluster->controldata.key_management_enabled = false;
+ got_key_management_enabled = true;
+ }
+
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
*************** get_control_data(ClusterInfo *cluster, b
*** 485,490 ****
--- 499,516 ----
cluster->controldata.data_checksum_version = str2uint(p);
got_data_checksum_version = true;
}
+ else if ((p = strstr(bufin, "Key management:")) != NULL)
+ {
+ p = strchr(p, ':');
+
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ p++; /* remove ':' char */
+ /* used later for contrib check */
+ cluster->controldata.key_management_enabled = strstr(p, "on") != NULL;
+ got_key_management_enabled = true;
+ }
}
pclose(output);
*************** get_control_data(ClusterInfo *cluster, b
*** 539,545 ****
!got_index || !got_toast ||
(!got_large_object &&
cluster->controldata.ctrl_ver >= LARGE_OBJECT_SIZE_PG_CONTROL_VER) ||
! !got_date_is_int || !got_data_checksum_version)
{
if (cluster == &old_cluster)
pg_log(PG_REPORT,
--- 565,572 ----
!got_index || !got_toast ||
(!got_large_object &&
cluster->controldata.ctrl_ver >= LARGE_OBJECT_SIZE_PG_CONTROL_VER) ||
! !got_date_is_int || !got_data_checksum_version ||
! !got_key_management_enabled)
{
if (cluster == &old_cluster)
pg_log(PG_REPORT,
*************** get_control_data(ClusterInfo *cluster, b
*** 605,610 ****
--- 632,641 ----
if (!got_data_checksum_version)
pg_log(PG_REPORT, " data checksum version\n");
+ /* value added in Postgres 12 */
+ if (!got_key_management_enabled)
+ pg_log(PG_REPORT, " key management enabled\n");
+
pg_fatal("Cannot continue without required control information, terminating\n");
}
}
*************** check_control_data(ControlData *oldctrl,
*** 669,674 ****
--- 700,713 ----
pg_fatal("old cluster uses data checksums but the new one does not\n");
else if (oldctrl->data_checksum_version != newctrl->data_checksum_version)
pg_fatal("old and new cluster pg_controldata checksum versions do not match\n");
+
+ /*
+ * We cannot upgrade if the old cluster enables the key management but
+ * the new one doesn't support because the old one might already have
+ * data encrypted by the master encryption key.
+ */
+ if (oldctrl->key_management_enabled && !newctrl->key_management_enabled)
+ pg_fatal("old cluster uses key management but the new one does not\n");
}
diff --git a/src/bin/pg_upgrade/file.c b/src/bin/pg_upgrade/file.c
new file mode 100644
index cc8a675..282359f
*** a/src/bin/pg_upgrade/file.c
--- b/src/bin/pg_upgrade/file.c
***************
*** 11,16 ****
--- 11,17 ----
#include <sys/stat.h>
#include <fcntl.h>
+ #include <dirent.h>
#ifdef HAVE_COPYFILE_H
#include <copyfile.h>
#endif
***************
*** 21,26 ****
--- 22,28 ----
#include "access/visibilitymap.h"
#include "common/file_perm.h"
+ #include "common/file_utils.h"
#include "pg_upgrade.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
*************** check_hard_link(void)
*** 372,374 ****
--- 374,451 ----
unlink(new_link_file);
}
+
+ /*
+ * Copy cryptographic keys from the old cluster to the new cluster.
+ */
+ void
+ copy_master_encryption_key(ClusterInfo *old_cluster, ClusterInfo * new_cluster)
+ {
+ DIR *dir;
+ struct dirent *de;
+ char path[MAXPGPATH];
+
+ /* We copy the crypto keys only if both clusters enable the key management */
+ if (!old_cluster->controldata.key_management_enabled ||
+ !new_cluster->controldata.key_management_enabled)
+ return;
+
+ prep_status("Copying master encryption key");
+
+ snprintf(path, MAXPGPATH, "%s/%s", old_cluster->pgdata, KMGR_DIR);
+
+ if ((dir = opendir(path)) == NULL)
+ pg_fatal("could not open directory \"%s\": %m", path);
+
+ while ((de = readdir(dir)) != NULL)
+ {
+ if (strlen(de->d_name) == 4 &&
+ strspn(de->d_name, "0123456789ABCDEF") == 4)
+ {
+ CryptoKey key;
+ char src_path[MAXPGPATH];
+ char dst_path[MAXPGPATH];
+ uint32 id;
+ int src_fd;
+ int dst_fd;
+ int len;
+
+ id = strtoul(de->d_name, NULL, 16);
+
+ snprintf(src_path, MAXPGPATH, "%s/%s/%04X",
+ old_cluster->pgdata, KMGR_DIR, id);
+ snprintf(dst_path, MAXPGPATH, "%s/%s/%04X",
+ new_cluster->pgdata, KMGR_DIR, id);
+
+ if ((src_fd = open(src_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ pg_fatal("could not open file \"%s\": %m", src_path);
+
+ if ((dst_fd = open(dst_path, O_RDWR | O_CREAT | O_TRUNC | PG_BINARY,
+ pg_file_create_mode)) < 0)
+ pg_fatal("could not open file \"%s\": %m", dst_path);
+
+ /* Read the source key */
+ len = read(src_fd, &key, sizeof(CryptoKey));
+ if (len != sizeof(CryptoKey))
+ {
+ if (len < 0)
+ pg_fatal("could not read file \"%s\": %m", src_path);
+ else
+ pg_fatal("could not read file \"%s\": read %d of %zu",
+ src_path, len, sizeof(CryptoKey));
+ }
+
+ /* Write to the dest key */
+ len = write(dst_fd, &key, sizeof(CryptoKey));
+ if (len != sizeof(CryptoKey))
+ pg_fatal("could not write fie \"%s\"", dst_path);
+
+ close(src_fd);
+ close(dst_fd);
+ }
+ }
+
+ closedir(dir);
+
+ check_ok();
+ }
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
new file mode 100644
index 1bc86e4..d1bacdf
*** a/src/bin/pg_upgrade/pg_upgrade.c
--- b/src/bin/pg_upgrade/pg_upgrade.c
*************** main(int argc, char **argv)
*** 158,163 ****
--- 158,170 ----
old_cluster.pgdata, new_cluster.pgdata);
/*
+ * Copy the internal encryption keys from the old cluster to the new one.
+ * This is necessary because the data in the old cluster might be
+ * encrypted with the old master encryption key.
+ */
+ copy_master_encryption_key(&old_cluster, &new_cluster);
+
+ /*
* Assuming OIDs are only used in system tables, there is no need to
* restore the OID counter because we have not transferred any OIDs from
* the old system, but we do it anyway just in case. We do it late here
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
new file mode 100644
index 8b90cef..32ab236
*** a/src/bin/pg_upgrade/pg_upgrade.h
--- b/src/bin/pg_upgrade/pg_upgrade.h
***************
*** 11,16 ****
--- 11,17 ----
#include <sys/time.h>
#include "libpq-fe.h"
+ #include "common/kmgr_utils.h"
/* Use port in the private/dynamic port number range */
#define DEF_PGUPORT 50432
*************** typedef struct
*** 219,224 ****
--- 220,226 ----
bool date_is_int;
bool float8_pass_by_value;
bool data_checksum_version;
+ bool key_management_enabled;
} ControlData;
/*
*************** void rewriteVisibilityMap(const char *f
*** 375,380 ****
--- 377,384 ----
const char *schemaName, const char *relName);
void check_file_clone(void);
void check_hard_link(void);
+ void copy_master_encryption_key(ClusterInfo *old_cluster,
+ ClusterInfo * new_cluster);
/* fopen_priv() is no longer different from fopen() */
#define fopen_priv(path, mode) fopen(path, mode)
diff --git a/src/common/Makefile b/src/common/Makefile
new file mode 100644
index 25c55bd..cfb34fd
*** a/src/common/Makefile
--- b/src/common/Makefile
*************** OBJS_COMMON = \
*** 49,54 ****
--- 49,55 ----
archive.o \
base64.o \
checksum_helper.o \
+ cipher.o \
config_info.o \
controldata_utils.o \
d2s.o \
*************** OBJS_COMMON = \
*** 61,66 ****
--- 62,68 ----
ip.o \
jsonapi.o \
keywords.o \
+ kmgr_utils.o \
kwlookup.o \
link-canary.o \
md5.o \
*************** OBJS_COMMON = \
*** 81,86 ****
--- 83,89 ----
ifeq ($(with_openssl),yes)
OBJS_COMMON += \
+ cipher_openssl.o \
protocol_openssl.o \
sha2_openssl.o
else
diff --git a/src/common/cipher.c b/src/common/cipher.c
new file mode 100644
index ...908afe8
*** a/src/common/cipher.c
--- b/src/common/cipher.c
***************
*** 0 ****
--- 1,87 ----
+ /*-------------------------------------------------------------------------
+ *
+ * cipher.c
+ * Shared frontend/backend for cryptographic functions
+ *
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/cipher.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include "common/cipher.h"
+ #ifdef USE_OPENSSL
+ #include "common/cipher_openssl.h"
+ #endif
+
+ /*
+ * Return a newly created cipher context. 'cipher' specifies cipher algorithm
+ * by identifer like PG_CIPHER_XXX.
+ */
+ PgCipherCtx *
+ pg_cipher_ctx_create(int cipher, uint8 *key, int klen)
+ {
+ PgCipherCtx *ctx = NULL;
+
+ if (cipher >= PG_MAX_CIPHER_ID)
+ return NULL;
+
+ #ifdef USE_OPENSSL
+ ctx = (PgCipherCtx *) palloc0(sizeof(PgCipherCtx));
+
+ ctx->encctx = ossl_cipher_ctx_create(cipher, key, klen, true);
+ ctx->decctx = ossl_cipher_ctx_create(cipher, key, klen, false);
+ #endif
+
+ return ctx;
+ }
+
+ void
+ pg_cipher_ctx_free(PgCipherCtx *ctx)
+ {
+ #ifdef USE_OPENSSL
+ ossl_cipher_ctx_free(ctx->encctx);
+ ossl_cipher_ctx_free(ctx->decctx);
+ #endif
+ }
+
+ bool
+ pg_cipher_encrypt(PgCipherCtx *ctx, const uint8 *in, int inlen,
+ uint8 *out, int *outlen, const uint8 *iv)
+ {
+ bool r = false;
+ #ifdef USE_OPENSSL
+ r = ossl_cipher_encrypt(ctx->encctx, in, inlen, out, outlen, iv);
+ #endif
+ return r;
+ }
+
+ bool
+ pg_cipher_decrypt(PgCipherCtx *ctx, const uint8 *in, int inlen,
+ uint8 *out, int *outlen, const uint8 *iv)
+ {
+ bool r = false;
+ #ifdef USE_OPENSSL
+ r = ossl_cipher_decrypt(ctx->decctx, in, inlen, out, outlen, iv);
+ #endif
+ return r;
+ }
+
+ bool
+ pg_HMAC_SHA512(const uint8 *key, const uint8 *in, int inlen,
+ uint8 *out)
+ {
+ bool r = false;
+ #ifdef USE_OPENSSL
+ r = ossl_HMAC_SHA512(key, in, inlen, out);
+ #endif
+ return r;
+ }
diff --git a/src/common/cipher_openssl.c b/src/common/cipher_openssl.c
new file mode 100644
index ...e8bc241
*** a/src/common/cipher_openssl.c
--- b/src/common/cipher_openssl.c
***************
*** 0 ****
--- 1,181 ----
+ /*-------------------------------------------------------------------------
+ * cipher_openssl.c
+ * Cryptographic function using OpenSSL
+ *
+ * This contains the common low-level functions needed in both frontend and
+ * backend, for implement the database encryption.
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/cipher_openssl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include "common/sha2.h"
+ #include "common/cipher_openssl.h"
+
+ #include <openssl/ssl.h>
+ #include <openssl/conf.h>
+ #include <openssl/evp.h>
+ #include <openssl/err.h>
+ #include <openssl/hmac.h>
+
+ /*
+ * prototype for the EVP functions that return an algorithm, e.g.
+ * EVP_aes_128_cbc().
+ */
+ typedef const EVP_CIPHER *(*ossl_EVP_cipher_func) (void);
+
+ static bool ossl_initialized = false;
+
+ static ossl_EVP_cipher_func get_evp_aes_cbc(int klen);
+
+ static ossl_EVP_cipher_func
+ get_evp_aes_cbc(int klen)
+ {
+ switch (klen)
+ {
+ case PG_AES128_KEY_LEN:
+ return EVP_aes_128_cbc;
+ case PG_AES192_KEY_LEN:
+ return EVP_aes_192_cbc;
+ case PG_AES256_KEY_LEN:
+ return EVP_aes_256_cbc;
+ default:
+ return NULL;
+ }
+ }
+
+ /*
+ * Initialize and return an EVP_CIPHER_CTX. Return NULL if the given
+ * cipher algorithm is not supported or on failure..
+ */
+ EVP_CIPHER_CTX *
+ ossl_cipher_ctx_create(int cipher, uint8 *key, int klen, bool enc)
+ {
+ EVP_CIPHER_CTX *ctx;
+ ossl_EVP_cipher_func func;
+ int ret;
+
+ if (!ossl_initialized)
+ {
+ #ifdef HAVE_OPENSSL_INIT_SSL
+ OPENSSL_init_ssl(OPENSSL_INIT_LOAD_CONFIG, NULL);
+ #else
+ OPENSSL_config(NULL);
+ SSL_library_init();
+ SSL_load_error_strings();
+ #endif
+ ossl_initialized = true;
+ }
+
+ ctx = EVP_CIPHER_CTX_new();
+
+ switch (cipher)
+ {
+ case PG_CIPHER_AES_CBC:
+ func = get_evp_aes_cbc(klen);
+ if (!func)
+ goto failed;
+ break;
+ default:
+ goto failed;
+ }
+
+
+ if (enc)
+ ret = EVP_EncryptInit_ex(ctx, (const EVP_CIPHER *) func(), NULL, key, NULL);
+ else
+ ret = EVP_DecryptInit_ex(ctx, (const EVP_CIPHER *) func(), NULL, key, NULL);
+
+ if (!ret)
+ goto failed;
+
+ if (!EVP_CIPHER_CTX_set_key_length(ctx, PG_AES256_KEY_LEN))
+ goto failed;
+
+ /*
+ * Always enable padding. We don't need to check the return value as
+ * EVP_CIPHER_CTX_set_padding always returns 1.
+ */
+ EVP_CIPHER_CTX_set_padding(ctx, 1);
+
+ return ctx;
+
+ failed:
+ EVP_CIPHER_CTX_free(ctx);
+ return NULL;
+ }
+
+ void
+ ossl_cipher_ctx_free(EVP_CIPHER_CTX *ctx)
+ {
+ return EVP_CIPHER_CTX_free(ctx);
+ }
+
+ bool
+ ossl_cipher_encrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv)
+ {
+ int len;
+ int enclen;
+
+ if (!EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv))
+ return false;
+
+ if (!EVP_EncryptUpdate(ctx, out, &len, in, inlen))
+ return false;
+
+ enclen = len;
+
+ if (!EVP_EncryptFinal_ex(ctx, (uint8 *) ((char *) out + enclen),
+ &len))
+ return false;
+
+ *outlen = enclen + len;
+
+ return true;
+ }
+
+ bool
+ ossl_cipher_decrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv)
+ {
+ int declen;
+ int len;
+
+ if (!EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv))
+ return false;
+
+ if (!EVP_DecryptUpdate(ctx, out, &len, in, inlen))
+ return false;
+
+ declen = len;
+
+ if (!EVP_DecryptFinal_ex(ctx, (uint8 *) ((char *) out + declen),
+ &len))
+ return false;
+
+ *outlen = declen + len;
+
+ return true;
+ }
+
+ bool
+ ossl_HMAC_SHA512(const uint8 *key, const uint8 *in, int inlen,
+ uint8 *out)
+ {
+ return HMAC(EVP_sha512(), key, PG_SHA512_DIGEST_LENGTH,
+ in, (uint32) inlen, out, NULL);
+ }
diff --git a/src/common/kmgr_utils.c b/src/common/kmgr_utils.c
new file mode 100644
index ...42ab462
*** a/src/common/kmgr_utils.c
--- b/src/common/kmgr_utils.c
***************
*** 0 ****
--- 1,523 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr_utils.c
+ * Shared frontend/backend for cryptographic key management
+ *
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/kmgr_utils.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include <unistd.h>
+ #include <sys/stat.h>
+
+ #ifdef FRONTEND
+ #include "common/logging.h"
+ #endif
+ #include "common/file_perm.h"
+ #include "common/kmgr_utils.h"
+ #include "common/sha2.h"
+ #include "crypto/kmgr.h"
+ #include "utils/elog.h"
+ #include "storage/fd.h"
+
+ #ifndef FRONTEND
+ #include "pgstat.h"
+ #include "storage/fd.h"
+ #endif
+
+ #define KMGR_PROMPT_MSG "Enter database encryption pass phrase: "
+
+ #ifdef FRONTEND
+ static FILE *open_pipe_stream(const char *command);
+ static int close_pipe_stream(FILE *file);
+ #endif
+
+ static void read_one_keyfile(const char *dataDir, uint32 id, CryptoKey *key_p);
+
+ /* Return a key wrap context initialized with the given keys */
+ PgKeyWrapCtx *
+ pg_create_keywrap_ctx(uint8 key[KMGR_ENC_KEY_LEN], uint8 mackey[KMGR_MAC_KEY_LEN])
+ {
+ PgKeyWrapCtx *ctx;
+
+ ctx = (PgKeyWrapCtx *) palloc0(sizeof(PgKeyWrapCtx));
+
+ /* Create and initialize a cipher context */
+ ctx->cipherctx = pg_cipher_ctx_create(PG_CIPHER_AES_CBC, key, KMGR_ENC_KEY_LEN);
+ if (ctx->cipherctx == NULL)
+ return NULL;
+
+ /* Set encryption key and MAC key */
+ memcpy(ctx->key, key, KMGR_ENC_KEY_LEN);
+ memcpy(ctx->mackey, mackey, KMGR_MAC_KEY_LEN);
+
+ return ctx;
+ }
+
+ /* Free the key wrap context */
+ void
+ pg_free_keywrap_ctx(PgKeyWrapCtx *ctx)
+ {
+ if (!ctx)
+ return;
+
+ Assert(ctx->cipherctx);
+
+ pg_cipher_ctx_free(ctx->cipherctx);
+
+ #ifndef FRONTEND
+ pfree(ctx);
+ #else
+ pg_free(ctx);
+ #endif
+ }
+
+ /*
+ * Encrypt the given data. Return true and set encrypted data to 'out' if
+ * success. Otherwise return false. The caller must allocate sufficient space
+ * for cipher data calculated by using KmgrSizeOfCipherText(). Please note that
+ * this function modifies 'out' data even on failure case.
+ */
+ bool
+ kmgr_wrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out)
+ {
+ uint8 *hmac;
+ uint8 *iv;
+ uint8 *enc;
+ int enclen;
+
+ Assert(ctx && in && out);
+
+ hmac = out->key;
+ iv = hmac + KMGR_HMAC_LEN;
+ enc = iv + PG_AES_IV_SIZE;
+
+ /* Generate IV */
+ if (!pg_strong_random(iv, PG_AES_IV_SIZE))
+ return false;
+
+ if (!pg_cipher_encrypt(ctx->cipherctx, in->key, in->klen, enc, &enclen, iv))
+ return false;
+
+ if (!pg_HMAC_SHA512(ctx->mackey, enc, enclen, hmac))
+ return false;
+
+ out->klen = KmgrSizeOfCipherText(in->klen);;
+ Assert(out->klen == KMGR_HMAC_LEN + PG_AES_IV_SIZE + enclen);
+
+ return true;
+ }
+
+ /*
+ * Decrypt the given Data. Return true and set plain text data to `out` if
+ * success. Otherwise return false. The caller must allocate sufficient space
+ * for cipher data calculated by using KmgrSizeOfPlainText(). Please note that
+ * this function modifies 'out' data even on failure case.
+ */
+ bool
+ kmgr_unwrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out)
+ {
+ uint8 hmac[KMGR_HMAC_LEN];
+ uint8 *expected_hmac;
+ uint8 *iv;
+ uint8 *enc;
+ int enclen;
+
+ Assert(ctx && in && out);
+
+ expected_hmac = in->key;
+ iv = expected_hmac + KMGR_HMAC_LEN;
+ enc = iv + PG_AES_IV_SIZE;
+ enclen = in->klen - (enc - in->key);
+
+ /* Verify the correctness of HMAC */
+ if (!pg_HMAC_SHA512(ctx->mackey, enc, enclen, hmac))
+ return false;
+
+ if (memcmp(hmac, expected_hmac, KMGR_HMAC_LEN) != 0)
+ return false;
+
+ /* Decrypt encrypted data */
+ if (!pg_cipher_decrypt(ctx->cipherctx, enc, enclen, out->key, &(out->klen), iv))
+ return false;
+
+ return true;
+ }
+
+ /*
+ * Verify the correctness of the given passphrase by unwrapping the given keys.
+ * If the given passphrase is correct we set unwrapped keys to keys_out and return
+ * true. Otherwise return false. Please note that this function changes the
+ * contents of keys_out even on failure. Both keys_in and keys_out must be the
+ * same length, nkey.
+ */
+ bool
+ kmgr_verify_passphrase(char *passphrase, int passlen,
+ CryptoKey *keys_in, CryptoKey *keys_out, int nkeys)
+ {
+ PgKeyWrapCtx *tmpctx;
+ uint8 user_enckey[KMGR_ENC_KEY_LEN];
+ uint8 user_hmackey[KMGR_MAC_KEY_LEN];
+
+ /*
+ * Create temporary wrap context with encryption key and HMAC key extracted
+ * from the passphrase.
+ */
+ kmgr_derive_keys(passphrase, passlen, user_enckey, user_hmackey);
+ tmpctx = pg_create_keywrap_ctx(user_enckey, user_hmackey);
+
+ for (int i = 0; i < nkeys; i++)
+ {
+
+ if (!kmgr_unwrap_key(tmpctx, &(keys_in[i]), &(keys_out[i])))
+ {
+ /* The passphrase is not correct */
+ pg_free_keywrap_ctx(tmpctx);
+ return false;
+ }
+ }
+
+ /* The passphrase is correct, free the cipher context */
+ pg_free_keywrap_ctx(tmpctx);
+
+ return true;
+ }
+
+ /* Generate encryption key and mac key from given passphrase */
+ void
+ kmgr_derive_keys(char *passphrase, Size passlen,
+ uint8 enckey[KMGR_ENC_KEY_LEN],
+ uint8 mackey[KMGR_MAC_KEY_LEN])
+ {
+ pg_sha256_ctx ctx1;
+ pg_sha512_ctx ctx2;
+
+ StaticAssertStmt(KMGR_ENC_KEY_LEN == PG_AES256_KEY_LEN,
+ "derived encryption key size does not match AES256 key size");
+ StaticAssertStmt(KMGR_MAC_KEY_LEN == PG_HMAC_SHA512_KEY_LEN,
+ "derived mac key size does not match HMAC-SHA512 key size");
+
+ /* Generate encryption key from passphrase */
+ pg_sha256_init(&ctx1);
+ pg_sha256_update(&ctx1, (const uint8 *) passphrase, passlen);
+ pg_sha256_final(&ctx1, enckey);
+
+ /* Generate mac key from passphrase */
+ pg_sha512_init(&ctx2);
+ pg_sha512_update(&ctx2, (const uint8 *) passphrase, passlen);
+ pg_sha512_final(&ctx2, mackey);
+ }
+
+ /*
+ * Run cluster passphrase command.
+ *
+ * prompt will be substituted for %p.
+ *
+ * The result will be put in buffer buf, which is of size size.
+ * The return value is the length of the actual result.
+ */
+ int
+ kmgr_run_cluster_passphrase_command(char *passphrase_command, char *buf,
+ int size)
+ {
+ char command[MAXPGPATH];
+ char *p;
+ char *dp;
+ char *endp;
+ FILE *fh;
+ int pclose_rc;
+ size_t len = 0;
+
+ Assert(size > 0);
+ buf[0] = '\0';
+
+ dp = command;
+ endp = command + MAXPGPATH - 1;
+ *endp = '\0';
+
+ for (p = passphrase_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'p':
+ strlcpy(dp, KMGR_PROMPT_MSG, strlen(KMGR_PROMPT_MSG)+1);
+ dp += strlen(KMGR_PROMPT_MSG);
+ p++;
+ break;
+ case '%':
+ p++;
+ if (dp < endp)
+ *dp++ = *p;
+ break;
+ default:
+ if (dp < endp)
+ *dp++ = *p;
+ break;
+ }
+ }
+ else
+ {
+ if (dp < endp)
+ *dp++ = *p;
+ }
+ }
+ *dp = '\0';
+
+ #ifdef FRONTEND
+ fh = open_pipe_stream(command);
+ if (fh == NULL)
+ {
+ pg_log_fatal("could not execute command \"%s\": %m",
+ command);
+ exit(EXIT_FAILURE);
+ }
+ #else
+ fh = OpenPipeStream(command, "r");
+ if (fh == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not execute command \"%s\": %m",
+ command)));
+ #endif
+
+ if ((len = fread(buf, sizeof(char), size, fh)) < size)
+ {
+ if (ferror(fh))
+ {
+ #ifdef FRONTEND
+ pg_log_fatal("could not read from command \"%s\": %m",
+ command);
+ exit(EXIT_FAILURE);
+ #else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command)));
+ #endif
+ }
+ }
+
+ #ifdef FRONTEND
+ pclose_rc = close_pipe_stream(fh);
+ #else
+ pclose_rc = ClosePipeStream(fh);
+ #endif
+
+ if (pclose_rc == -1)
+ {
+ #ifdef FRONTEND
+ pg_log_fatal("could not close pipe to external command: %m");
+ exit(EXIT_FAILURE);
+ #else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to external command: %m")));
+ #endif
+ }
+ else if (pclose_rc != 0)
+ {
+ #ifdef FRONTEND
+ pg_log_fatal("command \"%s\" failed", command);
+ exit(EXIT_FAILURE);
+ #else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("command \"%s\" failed",
+ command),
+ errdetail_internal("%s", wait_result_to_str(pclose_rc))));
+ #endif
+ }
+
+ return len;
+ }
+
+ #ifdef FRONTEND
+ static FILE *
+ open_pipe_stream(const char *command)
+ {
+ FILE *res;
+
+ #ifdef WIN32
+ size_t cmdlen = strlen(command);
+ char *buf;
+ int save_errno;
+
+ buf = malloc(cmdlen + 2 + 1);
+ if (buf == NULL)
+ {
+ errno = ENOMEM;
+ return NULL;
+ }
+ buf[0] = '"';
+ mempcy(&buf[1], command, cmdlen);
+ buf[cmdlen + 1] = '"';
+ buf[cmdlen + 2] = '\0';
+
+ res = _popen(buf, "r");
+
+ save_errno = errno;
+ free(buf);
+ errno = save_errno;
+ #else
+ res = popen(command, "r");
+ #endif /* WIN32 */
+ return res;
+ }
+
+ static int
+ close_pipe_stream(FILE *file)
+ {
+ #ifdef WIN32
+ return _pclose(file);
+ #else
+ return pclose(file);
+ #endif /* WIN32 */
+ }
+ #endif /* FRONTEND */
+
+ CryptoKey *
+ kmgr_get_cryptokeys(const char *path, int *nkeys)
+ {
+ struct dirent *de;
+ DIR *dir;
+ CryptoKey *keys;
+
+ #ifndef FRONTEND
+ if ((dir = AllocateDir(path)) == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open directory \"%s\": %m",
+ path)));
+ #else
+ if ((dir = opendir(path)) == NULL)
+ pg_log_fatal("could not open directory \"%s\": %m", path);
+ #endif
+
+ keys = (CryptoKey *) palloc0(sizeof(CryptoKey) * KMGR_MAX_INTERNAL_KEYS);
+ *nkeys = 0;
+
+ #ifndef FRONTEND
+ while ((de = ReadDir(dir, KMGR_DIR)) != NULL)
+ #else
+ while ((de = readdir(dir)) != NULL)
+ #endif
+ {
+ if (strlen(de->d_name) == 4 &&
+ strspn(de->d_name, "0123456789ABCDEF") == 4)
+ {
+ uint32 id;
+
+ id = strtoul(de->d_name, NULL, 16);
+
+ if (id < 0 || id >= KMGR_MAX_INTERNAL_KEYS)
+ {
+ #ifndef FRONTEND
+ elog(ERROR, "invalid cryptographic key identifier %u", id);
+ #else
+ pg_log_fatal("invalid cryptographic key identifier %u", id);
+ #endif
+ }
+
+ if (*nkeys >= KMGR_MAX_INTERNAL_KEYS)
+ {
+ #ifndef FRONTEND
+ elog(ERROR, "too many cryptographic kes");
+ #else
+ pg_log_fatal("too many cryptographic keys");
+ #endif
+ }
+
+ read_one_keyfile(path, id, &(keys[id]));
+ (*nkeys)++;
+ }
+ }
+
+ #ifndef FRONTEND
+ FreeDir(dir);
+ #else
+ closedir(dir);
+ #endif
+
+ return keys;
+ }
+
+ static void
+ read_one_keyfile(const char *cryptoKeyDir, uint32 id, CryptoKey *key_p)
+ {
+ char path[MAXPGPATH];
+ int fd;
+ int r;
+
+ CryptoKeyFilePath(path, cryptoKeyDir, id);
+
+ #ifndef FRONTEND
+ if ((fd = OpenTransientFile(path, O_RDONLY | PG_BINARY)) == -1)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\" for reading: %m",
+ path)));
+ #else
+ if ((fd = open(path, O_RDONLY | PG_BINARY, 0)) == -1)
+ pg_log_fatal("could not open file \"%s\" for reading: %m",
+ path);
+ #endif
+
+ #ifndef FRONTEND
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_READ);
+ #endif
+
+ /* Get key bytes */
+ r = read(fd, key_p, sizeof(CryptoKey));
+ if (r != sizeof(CryptoKey))
+ {
+ if (r < 0)
+ {
+ #ifndef FRONTEND
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read file \"%s\": %m", path)));
+ #else
+ pg_log_fatal("could not read file \"%s\": %m", path);
+ #endif
+ }
+ else
+ {
+ #ifndef FRONTEND
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("could not read file \"%s\": read %d of %zu",
+ path, r, sizeof(CryptoKey))));
+ #else
+ pg_log_fatal("could not read file \"%s\": read %d of %zu",
+ path, r, sizeof(CryptoKey));
+ #endif
+ }
+ }
+
+ #ifndef FRONTEND
+ pgstat_report_wait_end();
+ #endif
+
+ #ifndef FRONTEND
+ if (CloseTransientFile(fd) != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m",
+ path)));
+ #else
+ if (close(fd) != 0)
+ pg_log_fatal("could not close file \"%s\": %m", path);
+ #endif
+ }
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
new file mode 100644
index 221af87..33f28c7
*** a/src/include/access/xlog.h
--- b/src/include/access/xlog.h
*************** extern void UpdateControlFile(void);
*** 319,324 ****
--- 319,325 ----
extern uint64 GetSystemIdentifier(void);
extern char *GetMockAuthenticationNonce(void);
extern bool DataChecksumsEnabled(void);
+ extern bool KeyManagementEnabled(void);
extern XLogRecPtr GetFakeLSNForUnloggedRel(void);
extern Size XLOGShmemSize(void);
extern void XLOGShmemInit(void);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
new file mode 100644
index 06bed90..c7df6d6
*** a/src/include/catalog/pg_control.h
--- b/src/include/catalog/pg_control.h
*************** typedef struct ControlFileData
*** 226,231 ****
--- 226,234 ----
*/
char mock_authentication_nonce[MOCK_AUTH_NONCE_LEN];
+ /* Key management cipher. Zero if no version */
+ uint32 key_management_version;
+
/* CRC of all above ... MUST BE LAST! */
pg_crc32c crc;
} ControlFileData;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
new file mode 100644
index 22340ba..b1523e1
*** a/src/include/catalog/pg_proc.dat
--- b/src/include/catalog/pg_proc.dat
***************
*** 10992,10995 ****
--- 10992,11001 ----
proname => 'is_normalized', prorettype => 'bool', proargtypes => 'text text',
prosrc => 'unicode_is_normalized' },
+ # function for key managements
+ { oid => '8200', descr => 'rotate cluter passphrase',
+ proname => 'pg_rotate_cluster_passphrase',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_rotate_cluster_passphrase' },
+
]
diff --git a/src/include/common/cipher.h b/src/include/common/cipher.h
new file mode 100644
index ...f782791
*** a/src/include/common/cipher.h
--- b/src/include/common/cipher.h
***************
*** 0 ****
--- 1,78 ----
+ /*-------------------------------------------------------------------------
+ *
+ * cipher.h
+ * Declarations for cryptographic functions
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/common/cipher.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef CIPHER_H
+ #define CIPHER_H
+
+ #ifdef USE_OPENSSL
+ #include <openssl/evp.h>
+ #include <openssl/conf.h>
+ #include <openssl/err.h>
+ #endif
+
+ /*
+ * Supported symmetric encryption algorithm. These identifiers are passed
+ * to pg_cipher_ctx_create() function, and then actual encryption
+ * implementations need to initialize their context of the given encryption
+ * algorithm.
+ */
+ #define PG_CIPHER_AES_CBC 0
+ #define PG_MAX_CIPHER_ID 1
+
+ /* AES128/192/256 various length definitions */
+ #define PG_AES128_KEY_LEN (128 / 8)
+ #define PG_AES192_KEY_LEN (192 / 8)
+ #define PG_AES256_KEY_LEN (256 / 8)
+
+ /*
+ * The encrypted data is a series of blocks of size. Initialization
+ * vector(IV) is the same size of cipher block.
+ */
+ #define PG_AES_BLOCK_SIZE 16
+ #define PG_AES_IV_SIZE (PG_AES_BLOCK_SIZE)
+
+ /* HMAC key and HMAC length. We use HMAC-SHA256 */
+ #define PG_HMAC_SHA512_KEY_LEN 64
+ #define PG_HMAC_SHA512_LEN 64
+
+ #ifdef USE_OPENSSL
+ typedef EVP_CIPHER_CTX cipher_private_ctx;
+ #else
+ typedef void cipher_private_ctx;
+ #endif
+
+ /*
+ * This struct has two implementation-private context for
+ * encryption and decryption. The caller must create the encryption
+ * context using by pg_cipher_ctx_create() and pass the context to
+ * pg_cipher_encrypt() or pg_cipher_decrypt().
+ */
+ typedef struct PgCipherCtx
+ {
+ cipher_private_ctx *encctx;
+ cipher_private_ctx *decctx;
+ } PgCipherCtx;
+
+ extern PgCipherCtx *pg_cipher_ctx_create(int cipher, uint8 *key, int klen);
+ extern void pg_cipher_ctx_free(PgCipherCtx *ctx);
+ extern bool pg_cipher_encrypt(PgCipherCtx *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool pg_cipher_decrypt(PgCipherCtx *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool pg_HMAC_SHA512(const uint8 *key,
+ const uint8 *in, int inlen,
+ uint8 *out);
+
+ #endif /* CIPHER_H */
diff --git a/src/include/common/cipher_openssl.h b/src/include/common/cipher_openssl.h
new file mode 100644
index ...0fd1308
*** a/src/include/common/cipher_openssl.h
--- b/src/include/common/cipher_openssl.h
***************
*** 0 ****
--- 1,37 ----
+ /*-------------------------------------------------------------------------
+ *
+ * cipher_openssl.h
+ * Declarations for helper functions using OpenSSL
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/common/cipher_openssl.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef CIPHER_OPENSSL_H
+ #define CIPHER_OPENSSL_H
+
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include "common/cipher.h"
+
+ extern EVP_CIPHER_CTX *ossl_cipher_ctx_create(int cipher, uint8 *key, int klen,
+ bool enc);
+ extern void ossl_cipher_ctx_free(EVP_CIPHER_CTX *ctx);
+ extern bool ossl_cipher_encrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool ossl_cipher_decrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool ossl_HMAC_SHA512(const uint8 *key,
+ const uint8 *in, int inlen,
+ uint8 *out);
+ #endif
diff --git a/src/include/common/kmgr_utils.h b/src/include/common/kmgr_utils.h
new file mode 100644
index ...1dc8f43
*** a/src/include/common/kmgr_utils.h
--- b/src/include/common/kmgr_utils.h
***************
*** 0 ****
--- 1,105 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr_utils.h
+ * Declarations for utility function for key management
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/common/kmgr_utils.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef KMGR_UTILS_H
+ #define KMGR_UTILS_H
+
+ #include "common/cipher.h"
+
+ /* Current version number */
+ #define KMGR_VERSION 1
+
+ /*
+ * Directory where cryptographic keys reside within PGDATA. KMGR_DIR_TMP
+ * is used during cluster passphrase rotation.
+ */
+ #define KMGR_DIR "pg_cryptokeys"
+ #define KMGR_TMP_DIR "pg_cryptokeys_tmp"
+
+ /*
+ * Identifiers of internal keys. When adding a new internal key, we
+ * also need to add its key length to internalKeyLengths.
+ */
+ /* #define KMGR_XXX_KEY_ID 0 */
+ #define KMGR_MAX_INTERNAL_KEYS 0
+
+ /* Encryption key and MAC key used for key wrapping */
+ #define KMGR_ENC_KEY_LEN PG_AES256_KEY_LEN
+ #define KMGR_MAC_KEY_LEN PG_HMAC_SHA512_KEY_LEN
+ #define KMGR_HMAC_LEN PG_HMAC_SHA512_LEN
+
+ /* Key wrapping key consists of encryption key and mac key */
+ #define KMGR_KEY_LEN (PG_AEAD_ENC_KEY_LEN + PG_AEAD_MAC_KEY_LEN)
+
+ /* Allowed length of cluster passphrase */
+ #define KMGR_MIN_PASSPHRASE_LEN 64
+ #define KMGR_MAX_PASSPHRASE_LEN 1024
+
+ /* Maximum length of key the key manager can store */
+ #define KMGR_MAX_KEY_LEN 128
+ #define KMGR_MAX_WRAPPED_KEY_LEN KmgrSizeOfCipherText(KMGR_MAX_KEY_LEN)
+
+ /*
+ * Size of encrypted key size with padding. We use PKCS#7 padding,
+ * described in RFC 5652.
+ */
+ #define SizeOfDataWithPadding(klen) \
+ ((int)(klen) + (PG_AES_BLOCK_SIZE - ((int)(klen) % PG_AES_BLOCK_SIZE)))
+
+ /* Macros to compute the size of cipher text and plain text */
+ #define KmgrSizeOfCipherText(len) \
+ (KMGR_MAC_KEY_LEN + PG_AES_IV_SIZE + SizeOfDataWithPadding((int)(len)))
+ #define KmgrSizeOfPlainText(klen) \
+ ((int)(klen) - (KMGR_MAC_KEY_LEN + PG_AES_IV_SIZE))
+
+ /* CryptoKey file name is keys id */
+ #define CryptoKeyFilePath(path, dir, id) \
+ snprintf((path), MAXPGPATH, "%s/%04X", (dir), (id))
+
+ /*
+ * Cryptographic key data structure. This structure is used for
+ * both on-disk (raw key) and on-memory (wrapped key).
+ */
+ typedef struct CryptoKey
+ {
+ int klen;
+ uint8 key[KMGR_MAX_WRAPPED_KEY_LEN];
+ } CryptoKey;
+
+ /* Key wrapping cipher context */
+ typedef struct PgKeyWrapCtx
+ {
+ uint8 key[KMGR_ENC_KEY_LEN];
+ uint8 mackey[KMGR_MAC_KEY_LEN];
+ PgCipherCtx *cipherctx;
+ } PgKeyWrapCtx;
+
+ extern PgKeyWrapCtx *pg_create_keywrap_ctx(uint8 key[KMGR_ENC_KEY_LEN],
+ uint8 mackey[KMGR_MAC_KEY_LEN]);
+ extern void pg_free_keywrap_ctx(PgKeyWrapCtx *ctx);
+ extern bool kmgr_wrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+ extern bool kmgr_unwrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+
+
+
+ extern void kmgr_derive_keys(char *passphrase, Size passlen,
+ uint8 enckey[KMGR_ENC_KEY_LEN],
+ uint8 mackey[KMGR_MAC_KEY_LEN]);
+ extern bool kmgr_verify_passphrase(char *passphrase, int passlen,
+ CryptoKey *keys_in, CryptoKey *keys_out,
+ int nkey);
+ extern bool kmgr_wrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+ extern bool kmgr_unwrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+ extern int kmgr_run_cluster_passphrase_command(char *passphrase_command,
+ char *buf, int size);
+ extern CryptoKey *kmgr_get_cryptokeys(const char *path, int *nkeys);
+
+ #endif /* KMGR_UTILS_H */
diff --git a/src/include/crypto/kmgr.h b/src/include/crypto/kmgr.h
new file mode 100644
index ...783f06d
*** a/src/include/crypto/kmgr.h
--- b/src/include/crypto/kmgr.h
***************
*** 0 ****
--- 1,29 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr.h
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/crypto/kmgr.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef KMGR_H
+ #define KMGR_H
+
+ #include "common/cipher.h"
+ #include "common/kmgr_utils.h"
+ #include "storage/relfilenode.h"
+ #include "storage/bufpage.h"
+
+ /* GUC parameters */
+ extern bool key_management_enabled;
+ extern char *cluster_passphrase_command;
+
+ extern Size KmgrShmemSize(void);
+ extern void KmgrShmemInit(void);
+ extern void BootStrapKmgr(void);
+ extern void InitializeKmgr(void);
+ extern const CryptoKey *KmgrGetKey(int id);
+
+ #endif /* KMGR_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
new file mode 100644
index fb270df..d50a7c9
*** a/src/include/pg_config.h.in
--- b/src/include/pg_config.h.in
***************
*** 385,390 ****
--- 385,393 ----
/* Define to 1 if you have the `OPENSSL_init_ssl' function. */
#undef HAVE_OPENSSL_INIT_SSL
+ /* Define to 1 if you have the `OPENSSL_init_crypto' function. */
+ #undef HAVE_OPENSSL_INIT_CRYPTO
+
/* Define to 1 if you have the <ossp/uuid.h> header file. */
#undef HAVE_OSSP_UUID_H
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
new file mode 100644
index a821ff4..6549260
*** a/src/include/pgstat.h
--- b/src/include/pgstat.h
*************** typedef enum
*** 998,1003 ****
--- 998,1006 ----
WAIT_EVENT_DATA_FILE_TRUNCATE,
WAIT_EVENT_DATA_FILE_WRITE,
WAIT_EVENT_DSM_FILL_ZERO_WRITE,
+ WAIT_EVENT_KEY_FILE_READ,
+ WAIT_EVENT_KEY_FILE_WRITE,
+ WAIT_EVENT_KEY_FILE_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE,
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
new file mode 100644
index 04431d0..a0e582a
*** a/src/include/utils/guc_tables.h
--- b/src/include/utils/guc_tables.h
*************** enum config_group
*** 89,94 ****
--- 89,95 ----
STATS,
STATS_MONITORING,
STATS_COLLECTOR,
+ ENCRYPTION,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/test/Makefile b/src/test/Makefile
new file mode 100644
index efb206a..5276c41
*** a/src/test/Makefile
--- b/src/test/Makefile
*************** endif
*** 29,35 ****
endif
ifeq ($(with_openssl),yes)
ifneq (,$(filter ssl,$(PG_TEST_EXTRA)))
! SUBDIRS += ssl
endif
endif
--- 29,35 ----
endif
ifeq ($(with_openssl),yes)
ifneq (,$(filter ssl,$(PG_TEST_EXTRA)))
! SUBDIRS += ssl crypto
endif
endif
pass.sh
Description: Bourne shell script
