Saurabh Rai created MAPREDUCE-7440:
--------------------------------------
Summary: Enhancing Security in Hadoop Delegation Tokens: Phasing
out DIGEST-MD5 Auth mechanism
Key: MAPREDUCE-7440
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7440
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: security
Reporter: Saurabh Rai
SASL secured connections are commonly configured to negotiate confidential
(encrypted) connections, known as the "auth-conf" quality of protection. This
ensures both authentication and data encryption, enhancing the security of wire
communication. The use of AES encryption, negotiated on "auth-conf" connections
with Kerberos/GSSAPI, meets the requirements of modern commercial and
governmental cryptographic regulations and policies.
However, when deploying a YARN job that incorporates a network client expecting
to negotiate the same level of security ({color:#1d1c1d}for example an HBase
client, but any code that integrates Hadoop's UGI and related and the JRE's
SASLClient will be affected{color}). The problem arises from the fact that
delegation tokens, the only hard-coded option available for tasks, rely on the
Digest-MD5 SASL mechanism. Unfortunately, the Digest-MD5 negotiation standard
supports only five outdated and slow ciphers for SASL confidentiality: RC4 (40
bits key length), RC4 (56 bits key length), RC4 (128 bits key length), DES, and
Triple DES. Notably, the use of RC4 has been prohibited by the IETF since 2015,
and DES was compromised in 1999 and subsequently withdrawn as a standard by
NIST.
The limitations of the Digest-MD5 mechanism have significant implications for
compliance with modern cryptographic regulations and policies that mandate wire
encryption. As a result, YARN applications utilizing Digest-MD5 for
confidentiality negotiation cannot adhere to these requirements. It is worth
noting that this issue is not documented in the Hadoop documentation or logs,
potentially leading developers and operators to remain unaware of the problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
