[ https://issues.apache.org/jira/browse/NIFI-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398556#comment-15398556 ]
Andy LoPresto edited comment on NIFI-1831 at 7/29/16 1:34 AM: -------------------------------------------------------------- I have completed the architecture and the first implementation (AES/GCM 128/256-bit encryption) which provides transparent encryption/decryption of the sensitive {{nifi.properties}} values to the rest of the application. I had to pause to review [NIFI-2193] and will now resume writing the command-line utility to allow deployers to write these encrypted values into the properties files. Current progress can be seen [https://github.com/apache/nifi/compare/master...alopresto:NIFI-1831?expand=1](here). was (Author: alopresto): I have completed the architecture and the first implementation (AES/GCM 128/256-bit encryption) which provides transparent encryption/decryption of the sensitive {{nifi.properties}} values to the rest of the application. I had to pause to review [NIFI-2193] and will now resume writing the command-line utility to allow deployers to write these encrypted values into the properties files. > Allow encrypted passwords in configuration files > ------------------------------------------------ > > Key: NIFI-1831 > URL: https://issues.apache.org/jira/browse/NIFI-1831 > Project: Apache NiFi > Issue Type: New Feature > Components: Configuration, Core Framework > Affects Versions: 0.6.1 > Reporter: Andy LoPresto > Assignee: Andy LoPresto > Priority: Critical > Labels: configuration, encryption, password, security > Fix For: 1.0.0 > > Original Estimate: 504h > Remaining Estimate: 504h > > Storing passwords in plaintext in configuration files is not a security best > practice. While file access can be restricted through OS permissions, these > configuration files can be accidentally checked into source control, shared > or deployed to multiple instances, etc. > NiFi should allow a deployer to provide an encrypted password in the > configuration file to minimize exposure of the passwords. On application > start-up, NiFi should decrypt the passwords in memory. NiFi should also > include a utility to encrypt the raw passwords (and optionally populate the > configuration files and provide additional metadata in the configuration > files). > I am aware this simply shifts the responsibility/delegation of trust from the > passwords in the properties file to a new location on the same system, but > mitigating the visibility of the raw passwords in the properties file can be > one step in a defense in depth approach and is often mandated by security > policies within organizations using NiFi. > The key used for encryption should not be hard-coded into the application > source code, nor should it be universally consistent. The key could be > determined by reading static information from the deployed system and feeding > it to a key derivation function based on a cryptographically-secure hash > function, such as PBKDF2, bcrypt, or scrypt. However, this does introduce > upgrade, system migration, and portability issues. These challenges will have > to be kept in consideration when determining the key derivation process. > Manual key entry is a possibility, and then the master key would only be > present in memory, but this prevents automatic reboot on loss of power or > other recovery scenario. > This must be backward-compatible to allow systems with plaintext passwords to > continue operating. Options for achieving this are to only attempt to decrypt > passwords when a sibling property is present, or to match a specific format. > For these examples, I have used the following default values: > {code} > password: thisIsABadPassword > key: 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 > iv: 0123456789ABCDEFFEDCBA9876543210 > algorithm: AES/CBC 256-bit > {code} > **Note: These values should not be used in production systems -- the key and > IV are common test values, and an AEAD cipher is preferable to provide cipher > text integrity assurances, however OpenSSL does not support the use of AEAD > ciphers for command-line encryption at this time** > Example 1: *here the sibling property indicates the password is encrypted and > with which implementation; the absence of the property would default to a raw > password* > {code} > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:25:56 $ echo "thisIsABadPassword" > password.txt > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:26:47 $ ossl aes-256-cbc -e -nosalt -p -K > 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 -iv > 0123456789ABCDEFFEDCBA9876543210 -a -in password.txt -out password.enc > key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 > iv =0123456789ABCDEFFEDCBA9876543210 > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:27:09 $ xxd password.enc > 0000000: 5643 5856 6146 6250 4158 364f 5743 7646 VCXVaFbPAX6OWCvF > 0000010: 6963 6b76 4a63 7744 3854 6b67 3731 4c76 ickvJcwD8Tkg71Lv > 0000020: 4d38 6d32 7952 4776 5739 413d 0a M8m2yRGvW9A=. > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:27:16 $ more password.enc > VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A= > hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) > alopresto > 🔓 0s @ 16:27:55 $ > {code} > In {{nifi.properties}}: > {code} > nifi.security.keystorePasswd=VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A= > nifi.security.keystorePasswd.encrypted=AES-CBC-256 > {code} > Example 2: *here the encrypted password has a header tag indicating both that > it is encrypted and the algorithm used* > {code:java} > @Test > public void testShouldDecryptPassword() throws Exception { > // Arrange > KeyedCipherProvider cipherProvider = new AESKeyedCipherProvider() > final String PLAINTEXT = "thisIsABadPassword" > logger.info("Expected: ${Hex.encodeHexString(PLAINTEXT.bytes)}") > final byte[] IV = Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" as > char[]) > final byte[] LOCAL_KEY = > Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" * 2 as char[]) > // Generated via openssl enc -a > final String CIPHER_TEXT = > "VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=" > byte[] cipherBytes = Base64.decoder.decode(CIPHER_TEXT) > SecretKey localKey = new SecretKeySpec(LOCAL_KEY, "AES") > EncryptionMethod encryptionMethod = EncryptionMethod.AES_CBC > logger.info("Using algorithm: ${encryptionMethod.getAlgorithm()}") > logger.info("Cipher text: \$nifipw\$${CIPHER_TEXT} > ${cipherBytes.length + 8}") > // Act > Cipher cipher = cipherProvider.getCipher(encryptionMethod, localKey, > IV, false) > byte[] recoveredBytes = cipher.doFinal(cipherBytes) > > // OpenSSL adds a newline character during encryption > String recovered = new String(recoveredBytes, "UTF-8").trim() > logger.info("Recovered: ${recovered} > ${Hex.encodeHexString(recoveredBytes)}") > // Assert > assert PLAINTEXT.equals(recovered) > } > {code} > In {{nifi.properties}}: > {code} > nifi.security.keystorePasswd=$nifipw$VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A= > {code} > Ideally, NiFi would use a pluggable implementation architecture to allow > users to integrate with a variety of secret management services. There are > both commercial and open source solutions, including CyberArk Enterprise > Password Vault [1], Hashicorp Vault [2], and Square Keywhiz [3]. In the > future, this could also be extended to Hardware Security Modules (HSM) like > SafeNet Luna [4] and Amazon CloudHSM [5]. > [1] > http://www.cyberark.com/products/privileged-account-security-solution/enterprise-password-vault/ > [2] https://www.vaultproject.io/ > [3] https://square.github.io/keywhiz/ > [4] > http://www.safenet-inc.com/data-encryption/hardware-security-modules-hsms/luna-hsms-key-management/luna-sa-network-hsm/ > [5] https://aws.amazon.com/cloudhsm/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)