This is an automated email from the ASF dual-hosted git repository.
okumin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new af44ec916b2 HIVE-29970: Document mask_hash and aes_encrypt limitations
(#6896)
af44ec916b2 is described below
commit af44ec916b2a11bab27be487248f6cdf232dc428
Author: Ruben Quesada Lopez <[email protected]>
AuthorDate: Thu Oct 22 15:08:43 2026 +0100
HIVE-29970: Document mask_hash and aes_encrypt limitations (#6896)
* HIVE-29970: Document mask_hash and aes_encrypt limitations
* Update THREAT_MODEL.md
Co-authored-by: Shohei Okumiya <[email protected]>
* Update THREAT_MODEL.md
Co-authored-by: Shohei Okumiya <[email protected]>
---------
Co-authored-by: Shohei Okumiya <[email protected]>
---
THREAT_MODEL.md | 13 +++++++++++++
.../hadoop/hive/ql/udf/generic/GenericUDFAesBase.java | 4 +++-
.../hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java | 2 ++
.../hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java | 4 ++++
.../hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java | 7 ++++++-
.../test/results/clientpositive/llap/udf_aes_decrypt.q.out | 1 +
.../test/results/clientpositive/llap/udf_aes_encrypt.q.out | 1 +
ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out | 5 +++++
8 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/THREAT_MODEL.md b/THREAT_MODEL.md
index a6350cd24bc..30c9502292b 100644
--- a/THREAT_MODEL.md
+++ b/THREAT_MODEL.md
@@ -212,6 +212,15 @@ controls. *(maintainer — okumin)*
(e.g. `hive.query.max.length`) and YARN resource pools, and — where stronger
isolation than HS2 + YARN can provide is required — separate HS2 instances or
separate Hadoop/YARN clusters. *(maintainer — okumin, §14 Q11)*
+- **Cryptographic strength of built-in masking / encryption UDFs.**
+ `mask_hash` is a deterministic, unsalted, unkeyed SHA-256/512 digest (so
+ masked values remain joinable); `aes_encrypt`/`aes_decrypt` currently use
+ AES/ECB/PKCS5Padding (no IV, no authentication) by default, for
+ compatibility with existing ciphertexts and MySQL-style `AES_ENCRYPT`. These
+ functions do not provide semantic security or resistance to
+ dictionary/enumeration attacks, and Hive does not claim they do. Choosing a
+ masking type appropriate to the data (e.g. nullify/redact for low-entropy
+ columns) is the policy author's responsibility. *(documented:
GenericUDFMaskHash, GenericUDFAesEncrypt, GenericUDFAesDecrypt)*.
## §8 — Key configuration levers (load-bearing)
@@ -303,6 +312,10 @@ These are the dispositions the PMC has recorded; confirm
and extend
the operator has not configured the blacklist → `OUT-OF-MODEL:
operator responsibility`, unless the report shows a
*bypass* of a configured blacklist. *(maintainer — okumin.)*
+- **"`mask_hash` is invertible / `aes_encrypt` uses ECB / a built-in hash UDF
+ is weak."** By design — see §7. Valid only if the report shows a bypass of a
+ configured masking policy (e.g. reading the unmasked column), not the
+ inherent properties of the function. `BY-DESIGN`. *(documented:
GenericUDFMaskHash, GenericUDFAesEncrypt, GenericUDFAesDecrypt))*.
- **Dependency-tail CVEs** (Hadoop, Log4j, a transitive JAR) surfaced by an
SCA scanner against Hive's build — triage upstream unless Hive's own code
reaches the vulnerable path with untrusted input.
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java
index 41d5c8c5b6f..d3d63f5d028 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java
@@ -106,7 +106,9 @@ public ObjectInspector initialize(ObjectInspector[]
arguments) throws UDFArgumen
}
try {
- cipher = Cipher.getInstance("AES");
+ // Explicit form of the provider default for "AES". ECB is kept for
compatibility with existing
+ // ciphertexts and MySQL-style AES_ENCRYPT; see the
aes_encrypt/aes_decrypt descriptions.
+ cipher = Cipher.getInstance("AES/ECB/PKCS5Padding");
} catch (NoSuchPaddingException | NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java
index d5c3d456e02..028279d725e 100644
---
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java
@@ -31,6 +31,8 @@
+ "Key lengths of 128, 192 or 256 bits can be used. 192 and 256 bits keys
can be used if "
+ "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction
Policy Files are installed. "
+ "If either argument is NULL or the key length is not one of the
permitted values, the return value is NULL.\n"
+ + "Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding,
MySQL-style) with no IV and no "
+ + "authentication; see the aes_encrypt description for its limitations.\n"
+ "Example: > SELECT _FUNC_(unbase64('y6Ss+zCYObpCbgfWfyNWTw=='),
'1234567890123456');\n 'ABC'")
public class GenericUDFAesDecrypt extends GenericUDFAesBase {
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java
index b5be296b548..eab265941dd 100644
---
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java
@@ -31,6 +31,10 @@
+ "Key lengths of 128, 192 or 256 bits can be used. 192 and 256 bits keys
can be used if "
+ "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction
Policy Files are installed. "
+ "If either argument is NULL or the key length is not one of the
permitted values, the return value is NULL.\n"
+ + "Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding,
MySQL-style) with no IV and no "
+ + "authentication. Encryption is deterministic: equal inputs encrypted
with the same key produce equal "
+ + "outputs, so anyone who can read the ciphertext can detect duplicates
and infer value frequencies, and "
+ + "ciphertexts are malleable. It is not suitable for protecting columns
with repetitive or low-entropy values.\n"
+ "Example: > SELECT base64(_FUNC_('ABC', '1234567890123456'));\n
'y6Ss+zCYObpCbgfWfyNWTw=='")
public class GenericUDFAesEncrypt extends GenericUDFAesBase {
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
index 0f216ea11c1..1900420779c 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
@@ -34,7 +34,12 @@
extended = "Examples:\n "
+ " mask_hash(value)\n "
+ "Arguments:\n "
- + " value - value to mask. Supported types: STRING,
VARCHAR, CHAR"
+ + " value - value to mask. Supported types: STRING,
VARCHAR, CHAR\n "
+ + "Note:\n "
+ + " The result is an unsalted, unkeyed SHA-256 digest
(SHA-512 if hive.masking.algo=sha512).\n "
+ + " It is deterministic, so values from small or
predictable domains (e.g. phone numbers,\n "
+ + " national IDs, dates) can be recovered by hashing
every candidate value. For such\n "
+ + " columns prefer a nullifying or redacting mask."
)
public class GenericUDFMaskHash extends BaseMaskUDF {
public static final String UDF_NAME = "mask_hash";
diff --git a/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out
b/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out
index c4d983c82eb..c6f7c367129 100644
--- a/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out
+++ b/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out
@@ -9,6 +9,7 @@ POSTHOOK: query: DESC FUNCTION EXTENDED aes_decrypt
POSTHOOK: type: DESCFUNCTION
aes_decrypt(input binary, key string/binary) - Decrypt input using AES.
AES (Advanced Encryption Standard) algorithm. Key lengths of 128, 192 or 256
bits can be used. 192 and 256 bits keys can be used if Java Cryptography
Extension (JCE) Unlimited Strength Jurisdiction Policy Files are installed. If
either argument is NULL or the key length is not one of the permitted values,
the return value is NULL.
+Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding, MySQL-style)
with no IV and no authentication; see the aes_encrypt description for its
limitations.
Example: > SELECT aes_decrypt(unbase64('y6Ss+zCYObpCbgfWfyNWTw=='),
'1234567890123456');
'ABC'
Function class:org.apache.hadoop.hive.ql.udf.generic.GenericUDFAesDecrypt
diff --git a/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out
b/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out
index 4879bf7009b..1afd2ee4755 100644
--- a/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out
+++ b/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out
@@ -9,6 +9,7 @@ POSTHOOK: query: DESC FUNCTION EXTENDED aes_encrypt
POSTHOOK: type: DESCFUNCTION
aes_encrypt(input string/binary, key string/binary) - Encrypt input using AES.
AES (Advanced Encryption Standard) algorithm. Key lengths of 128, 192 or 256
bits can be used. 192 and 256 bits keys can be used if Java Cryptography
Extension (JCE) Unlimited Strength Jurisdiction Policy Files are installed. If
either argument is NULL or the key length is not one of the permitted values,
the return value is NULL.
+Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding, MySQL-style)
with no IV and no authentication. Encryption is deterministic: equal inputs
encrypted with the same key produce equal outputs, so anyone who can read the
ciphertext can detect duplicates and infer value frequencies, and ciphertexts
are malleable. It is not suitable for protecting columns with repetitive or
low-entropy values.
Example: > SELECT base64(aes_encrypt('ABC', '1234567890123456'));
'y6Ss+zCYObpCbgfWfyNWTw=='
Function class:org.apache.hadoop.hive.ql.udf.generic.GenericUDFAesEncrypt
diff --git a/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out
b/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out
index e823d829a8f..893336058cf 100644
--- a/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out
+++ b/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out
@@ -12,6 +12,11 @@ Examples:
mask_hash(value)
Arguments:
value - value to mask. Supported types: STRING, VARCHAR, CHAR
+ Note:
+ The result is an unsalted, unkeyed SHA-256 digest (SHA-512 if
hive.masking.algo=sha512).
+ It is deterministic, so values from small or predictable domains (e.g.
phone numbers,
+ national IDs, dates) can be recovered by hashing every candidate value. For
such
+ columns prefer a nullifying or redacting mask.
Function class:org.apache.hadoop.hive.ql.udf.generic.GenericUDFMaskHash
Function type:BUILTIN
PREHOOK: query: explain select mask_hash('TestString-123')