[ https://issues.apache.org/jira/browse/NIFI-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431558#comment-15431558 ]
ASF GitHub Bot commented on NIFI-1831: -------------------------------------- Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/834#discussion_r75752491 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/AESSensitivePropertyProvider.java --- @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.properties; + +import java.nio.charset.StandardCharsets; +import java.security.InvalidAlgorithmParameterException; +import java.security.InvalidKeyException; +import java.security.NoSuchAlgorithmException; +import java.security.NoSuchProviderException; +import java.security.SecureRandom; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.stream.Collectors; +import javax.crypto.BadPaddingException; +import javax.crypto.Cipher; +import javax.crypto.IllegalBlockSizeException; +import javax.crypto.NoSuchPaddingException; +import javax.crypto.SecretKey; +import javax.crypto.spec.IvParameterSpec; +import javax.crypto.spec.SecretKeySpec; +import org.apache.commons.lang3.StringUtils; +import org.bouncycastle.util.encoders.Base64; +import org.bouncycastle.util.encoders.DecoderException; +import org.bouncycastle.util.encoders.EncoderException; +import org.bouncycastle.util.encoders.Hex; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class AESSensitivePropertyProvider implements SensitivePropertyProvider { + + private static final Logger logger = LoggerFactory.getLogger(AESSensitivePropertyProvider.class); + + private static final String IMPLEMENTATION_NAME = "AES Sensitive Property Provider"; + private static final String IMPLEMENTATION_KEY = "aes/gcm/"; + private static final String ALGORITHM = "AES/GCM/NoPadding"; + private static final String PROVIDER = "BC"; + private static final String DELIMITER = "||"; // "|" is not a valid Base64 character, so ensured not to be present in cipher text + private static final int IV_LENGTH = 12; + private static final int MIN_CIPHER_TEXT_LENGTH = IV_LENGTH * 4 / 3 + DELIMITER.length() + 1; + + private Cipher cipher; + private SecretKey key; + + public AESSensitivePropertyProvider() { --- End diff -- There was a use case for this originally, but now the only access is from the constructor, so I will make this fix. > Allow encrypted passwords in configuration files > ------------------------------------------------ > > Key: NIFI-1831 > URL: https://issues.apache.org/jira/browse/NIFI-1831 > Project: Apache NiFi > Issue Type: New Feature > Components: Configuration, Core Framework > Affects Versions: 0.6.1 > Reporter: Andy LoPresto > Assignee: Andy LoPresto > Priority: Critical > Labels: configuration, encryption, password, security > Fix For: 1.0.0 > > Original Estimate: 504h > Remaining Estimate: 504h > > Storing passwords in plaintext in configuration files is not a security best > practice. While file access can be restricted through OS permissions, these > configuration files can be accidentally checked into source control, shared > or deployed to multiple instances, etc. > NiFi should allow a deployer to provide an encrypted password in the > configuration file to minimize exposure of the passwords. On application > start-up, NiFi should decrypt the passwords in memory. NiFi should also > include a utility to encrypt the raw passwords (and optionally populate the > configuration files and provide additional metadata in the configuration > files). > I am aware this simply shifts the responsibility/delegation of trust from the > passwords in the properties file to a new location on the same system, but > mitigating the visibility of the raw passwords in the properties file can be > one step in a defense in depth approach and is often mandated by security > policies within organizations using NiFi. > The key used for encryption should not be hard-coded into the application > source code, nor should it be universally consistent. The key could be > determined by reading static information from the deployed system and feeding > it to a key derivation function based on a cryptographically-secure hash > function, such as PBKDF2, bcrypt, or scrypt. However, this does introduce > upgrade, system migration, and portability issues. These challenges will have > to be kept in consideration when determining the key derivation process. > Manual key entry is a possibility, and then the master key would only be > present in memory, but this prevents automatic reboot on loss of power or > other recovery scenario. > This must be backward-compatible to allow systems with plaintext passwords to > continue operating. Options for achieving this are to only attempt to decrypt > passwords when a sibling property is present, or to match a specific format. > For these examples, I have used the following default values: > {code} > password: thisIsABadPassword > key: 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 > iv: 0123456789ABCDEFFEDCBA9876543210 > algorithm: AES/CBC 256-bit > {code} > **Note: These values should not be used in production systems -- the key and > IV are common test values, and an AEAD cipher is preferable to provide cipher > text integrity assurances, however OpenSSL does not support the use of AEAD > ciphers for command-line encryption at this time** > Example 1: *here the sibling property indicates the password is encrypted and > with which implementation; the absence of the property would default to a raw > password* > {code} > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:25:56 $ echo "thisIsABadPassword" > password.txt > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:26:47 $ ossl aes-256-cbc -e -nosalt -p -K > 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 -iv > 0123456789ABCDEFFEDCBA9876543210 -a -in password.txt -out password.enc > key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 > iv =0123456789ABCDEFFEDCBA9876543210 > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:27:09 $ xxd password.enc > 0000000: 5643 5856 6146 6250 4158 364f 5743 7646 VCXVaFbPAX6OWCvF > 0000010: 6963 6b76 4a63 7744 3854 6b67 3731 4c76 ickvJcwD8Tkg71Lv > 0000020: 4d38 6d32 7952 4776 5739 413d 0a M8m2yRGvW9A=. > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:27:16 $ more password.enc > VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A= > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:27:55 $ > {code} > In {{nifi.properties}}: > {code} > nifi.security.keystorePasswd=VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A= > nifi.security.keystorePasswd.encrypted=AES-CBC-256 > {code} > Example 2: *here the encrypted password has a header tag indicating both that > it is encrypted and the algorithm used* > {code:java} > @Test > public void testShouldDecryptPassword() throws Exception { > // Arrange > KeyedCipherProvider cipherProvider = new AESKeyedCipherProvider() > final String PLAINTEXT = "thisIsABadPassword" > logger.info("Expected: ${Hex.encodeHexString(PLAINTEXT.bytes)}") > final byte[] IV = Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" as > char[]) > final byte[] LOCAL_KEY = > Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" * 2 as char[]) > // Generated via openssl enc -a > final String CIPHER_TEXT = > "VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=" > byte[] cipherBytes = Base64.decoder.decode(CIPHER_TEXT) > SecretKey localKey = new SecretKeySpec(LOCAL_KEY, "AES") > EncryptionMethod encryptionMethod = EncryptionMethod.AES_CBC > logger.info("Using algorithm: ${encryptionMethod.getAlgorithm()}") > logger.info("Cipher text: \$nifipw\$${CIPHER_TEXT} > ${cipherBytes.length + 8}") > // Act > Cipher cipher = cipherProvider.getCipher(encryptionMethod, localKey, > IV, false) > byte[] recoveredBytes = cipher.doFinal(cipherBytes) > > // OpenSSL adds a newline character during encryption > String recovered = new String(recoveredBytes, "UTF-8").trim() > logger.info("Recovered: ${recovered} > ${Hex.encodeHexString(recoveredBytes)}") > // Assert > assert PLAINTEXT.equals(recovered) > } > {code} > In {{nifi.properties}}: > {code} > nifi.security.keystorePasswd=$nifipw$VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A= > {code} > Ideally, NiFi would use a pluggable implementation architecture to allow > users to integrate with a variety of secret management services. There are > both commercial and open source solutions, including CyberArk Enterprise > Password Vault [1], Hashicorp Vault [2], and Square Keywhiz [3]. In the > future, this could also be extended to Hardware Security Modules (HSM) like > SafeNet Luna [4] and Amazon CloudHSM [5]. > [1] > http://www.cyberark.com/products/privileged-account-security-solution/enterprise-password-vault/ > [2] https://www.vaultproject.io/ > [3] https://square.github.io/keywhiz/ > [4] > http://www.safenet-inc.com/data-encryption/hardware-security-modules-hsms/luna-hsms-key-management/luna-sa-network-hsm/ > [5] https://aws.amazon.com/cloudhsm/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)