[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[ https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295137#comment-16295137 ] ASF GitHub Bot commented on NIFI-4701: -- GitHub user kevdoran opened a pull request: https://github.com/apache/nifi/pull/2350 NIFI-4701 Support encrypted authorizers.xml Enable properties in authorizers.xml to be encrypted by the master key. Adds authorizers.xml to the files understood by the encrypt-config tool in the NiFi Toolkit. If passed to the tool using the CLI, then the sensitive properties for LdapUserGroupProvider will be encrypted. Also fixes a bug wherein encrypt-config replaces multiple XML nodes in login-indentity-providers.xml when LdapProvider is not the first provider listed in the file. --- Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kevdoran/nifi NIFI-4701 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2350.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2350 commit 35d69e06cd878ae22e0de142803d9d0d9112c7a0 Author: Kevin Doran Date: 2017-12-17T16:40:06Z NIFI-4701 Support encrypted authorizers.xml Enable properties in authorizers.xml to be encrypted by the master key. commit a9c496e97a8e3f4fbcee3b7c0ab260e88ecc9b19 Author: Kevin Doran Date: 2017-12-18T04:46:39Z NIFI-4701 Add authorizers.xml support to toolkit Adds authorizers.xml to the files understood by the encrypt-config tool in the NiFi Toolkit. If passed to the tool using the CLI, then the sensitive properties for LdapUserGroupProvider will be encrypted. Also fixes a bug wherein encrypt-config replaces multiple XML nodes in login-indentity-providers.xml when LdapProvider is not the first provider listed in the file. > Support encrypted properties in authorizers.xml > --- > > Key: NIFI-4701 > URL: https://issues.apache.org/jira/browse/NIFI-4701 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 1.5.0 > > > Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0, > authorizers.xml can now contain properties for LDAP Server credentials. > This ticket is to enable properties in authorizers.xml to be encrypted, so > that the LDAP Server Manager credentials can be protected similar to > LdapProvider which is configured via login-identity-providers.xml. > The main changes are in nifi-authorizers are: > * authorizers.xsd to add an encryption attribute to Property > * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt > the property value if necessary when creating the the configuration context > Additionally, support for creating an encrypted authorizers.xml, protected by > t
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[ https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300250#comment-16300250 ] ASF GitHub Bot commented on NIFI-4701: -- Github user kevdoran commented on a diff in the pull request: https://github.com/apache/nifi/pull/2350#discussion_r158322844 --- Diff: nifi-docs/src/main/asciidoc/administration-guide.adoc --- @@ -1455,25 +1455,27 @@ The default encryption algorithm utilized is AES/GCM 128/256-bit. 128-bit is use You can use the following command line options with the `encrypt-config` tool: --- End diff -- The ordering is changed here (to match to ordering of the usage output when running the tool), so it looks like a larger change. There are only two new options: * `-a`,`--authorizers ` The authorizers.xml file containing unprotected config values (will be overwritten) * `-u`,`--outputAuthorizers ` The destination authorizers.xml file containing protected config values (will not modify input authorizers.xml) > Support encrypted properties in authorizers.xml > --- > > Key: NIFI-4701 > URL: https://issues.apache.org/jira/browse/NIFI-4701 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 1.5.0 > > > Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0, > authorizers.xml can now contain properties for LDAP Server credentials. > This ticket is to enable properties in authorizers.xml to be encrypted, so > that the LDAP Server Manager credentials can be protected similar to > LdapProvider which is configured via login-identity-providers.xml. > The main changes are in nifi-authorizers are: > * authorizers.xsd to add an encryption attribute to Property > * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt > the property value if necessary when creating the the configuration context > Additionally, support for creating an encrypted authorizers.xml, protected by > the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306522#comment-16306522
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159101417
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
---
@@ -473,6 +536,34 @@ class ConfigEncryptionTool {
}
}
+/**
+ * Loads the authorizers configuration from the provided file path.
+ *
+ * @param existingKeyHex the key used to encrypt the configs (defaults
to the current key)
+ *
+ * @return the file content
+ * @throw IOException if the authorizers.xml file cannot be read
+ */
+private String loadAuthorizers(String existingKeyHex = keyHex) throws
IOException {
+File authorizersFile
+if (authorizersPath && (authorizersFile = new
File(authorizersPath)).exists()) {
+try {
+String xmlContent = authorizersFile.text
+List lines = authorizersFile.readLines()
+logger.info("Loaded Authroizers content (${lines.size()}
lines)")
--- End diff --
I think this was copied from the LIP section and should be fixed there too
-- this is redundant. In order to capture the number of lines and get all the
contents as a single string, we should use the `readLines()` method and then
`join` the `List`.
> Support encrypted properties in authorizers.xml
> ---
>
> Key: NIFI-4701
> URL: https://issues.apache.org/jira/browse/NIFI-4701
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 1.5.0
>
>
> Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0,
> authorizers.xml can now contain properties for LDAP Server credentials.
> This ticket is to enable properties in authorizers.xml to be encrypted, so
> that the LDAP Server Manager credentials can be protected similar to
> LdapProvider which is configured via login-identity-providers.xml.
> The main changes are in nifi-authorizers are:
> * authorizers.xsd to add an encryption attribute to Property
> * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt
> the property value if necessary when creating the the configuration context
> Additionally, support for creating an encrypted authorizers.xml, protected by
> the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306525#comment-16306525
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159101938
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
---
@@ -730,6 +821,42 @@ class ConfigEncryptionTool {
}
}
+String decryptAuthorizers(String encryptedXml, String existingKeyHex =
keyHex) {
+AESSensitivePropertyProvider sensitivePropertyProvider = new
AESSensitivePropertyProvider(existingKeyHex)
+
+try {
+def doc = new XmlSlurper().parseText(encryptedXml)
+// Find the provider element by class even if it has been
renamed
+def passwords = doc.userGroupProvider.find { it.'class' as
String == LDAP_USER_GROUP_PROVIDER_CLASS }.property.findAll {
+it.@name =~ "Password" && it.@encryption =~
"aes/gcm/\\d{3}"
+}
+
+if (passwords.isEmpty()) {
+if (isVerbose) {
+logger.info("No encrypted password property elements
found in authorizers.xml")
+}
+return encryptedXml
+}
+
+passwords.each { password ->
+if (isVerbose) {
--- End diff --
Informational note: in the event the file is in an unsupported state
(perhaps manually decrypted but the `encryption` attribute is still present),
this will log the plaintext password to the console output before attempting to
decrypt. This is not necessarily a vulnerability of the tool, as the incoming
data is not in the expected format. It would take additional effort to capture
the "raw" value, compare the attempted decryption and the original value, and
output the raw value if the contents are different. This would still allow the
tool to print the attempted decryption input value if the attempt throws an
exception, but this level of effort is unnecessary for this edge case. Just a
note for the future.
> Support encrypted properties in authorizers.xml
> ---
>
> Key: NIFI-4701
> URL: https://issues.apache.org/jira/browse/NIFI-4701
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 1.5.0
>
>
> Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0,
> authorizers.xml can now contain properties for LDAP Server credentials.
> This ticket is to enable properties in authorizers.xml to be encrypted, so
> that the LDAP Server Manager credentials can be protected similar to
> LdapProvider which is configured via login-identity-providers.xml.
> The main changes are in nifi-authorizers are:
> * authorizers.xsd to add an encryption attribute to Property
> * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt
> the property value if necessary when creating the the configuration context
> Additionally, support for creating an encrypted authorizers.xml, protected by
> the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306526#comment-16306526
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159102020
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
---
@@ -772,6 +899,48 @@ class ConfigEncryptionTool {
}
}
+String encryptAuthorizers(String plainXml, String newKeyHex = keyHex) {
+AESSensitivePropertyProvider sensitivePropertyProvider = new
AESSensitivePropertyProvider(newKeyHex)
+
+// TODO: Switch to XmlParser & XmlNodePrinter to maintain "empty"
element structure
+try {
+def doc = new XmlSlurper().parseText(plainXml)
+// Find the provider element by class even if it has been
renamed
+def passwords = doc.userGroupProvider.find { it.'class' as
String == LDAP_USER_GROUP_PROVIDER_CLASS }
+.property.findAll {
+// Only operate on un-encrypted passwords
+it.@name =~ "Password" && (it.@encryption == "none" ||
it.@encryption == "") && it.text()
+}
+
+if (passwords.isEmpty()) {
+if (isVerbose) {
+logger.info("No unencrypted password property elements
found in login-identity-providers.xml")
--- End diff --
The message should be updated to `authorizers.xml`.
> Support encrypted properties in authorizers.xml
> ---
>
> Key: NIFI-4701
> URL: https://issues.apache.org/jira/browse/NIFI-4701
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 1.5.0
>
>
> Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0,
> authorizers.xml can now contain properties for LDAP Server credentials.
> This ticket is to enable properties in authorizers.xml to be encrypted, so
> that the LDAP Server Manager credentials can be protected similar to
> LdapProvider which is configured via login-identity-providers.xml.
> The main changes are in nifi-authorizers are:
> * authorizers.xsd to add an encryption attribute to Property
> * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt
> the property value if necessary when creating the the configuration context
> Additionally, support for creating an encrypted authorizers.xml, protected by
> the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306528#comment-16306528
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159102075
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
---
@@ -772,6 +899,48 @@ class ConfigEncryptionTool {
}
}
+String encryptAuthorizers(String plainXml, String newKeyHex = keyHex) {
+AESSensitivePropertyProvider sensitivePropertyProvider = new
AESSensitivePropertyProvider(newKeyHex)
+
+// TODO: Switch to XmlParser & XmlNodePrinter to maintain "empty"
element structure
+try {
+def doc = new XmlSlurper().parseText(plainXml)
+// Find the provider element by class even if it has been
renamed
+def passwords = doc.userGroupProvider.find { it.'class' as
String == LDAP_USER_GROUP_PROVIDER_CLASS }
+.property.findAll {
+// Only operate on un-encrypted passwords
+it.@name =~ "Password" && (it.@encryption == "none" ||
it.@encryption == "") && it.text()
+}
+
+if (passwords.isEmpty()) {
+if (isVerbose) {
+logger.info("No unencrypted password property elements
found in login-identity-providers.xml")
+}
+return plainXml
+}
+
+passwords.each { password ->
+if (isVerbose) {
+logger.info("Attempting to encrypt ${password.name()}")
+}
+String encryptedValue =
sensitivePropertyProvider.protect(password.text().trim())
+password.replaceNode {
+property(name: password.@name, encryption:
sensitivePropertyProvider.identifierKey, encryptedValue)
+}
+}
+
+// Does not preserve whitespace formatting or comments
+String updatedXml = XmlUtil.serialize(doc)
+logger.info("Updated XML content: ${updatedXml}")
+updatedXml
+} catch (Exception e) {
+if (isVerbose) {
+logger.error("Encountered exception", e)
+}
+printUsageAndThrow("Cannot encrypt login identity providers
XML content", ExitCode.SERVICE_ERROR)
--- End diff --
This message should also be updated to `authorizers.xml`.
> Support encrypted properties in authorizers.xml
> ---
>
> Key: NIFI-4701
> URL: https://issues.apache.org/jira/browse/NIFI-4701
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 1.5.0
>
>
> Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0,
> authorizers.xml can now contain properties for LDAP Server credentials.
> This ticket is to enable properties in authorizers.xml to be encrypted, so
> that the LDAP Server Manager credentials can be protected similar to
> LdapProvider which is configured via login-identity-providers.xml.
> The main changes are in nifi-authorizers are:
> * authorizers.xsd to add an encryption attribute to Property
> * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt
> the property value if necessary when creating the the configuration context
> Additionally, support for creating an encrypted authorizers.xml, protected by
> the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306531#comment-16306531
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159102294
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
---
@@ -921,6 +1090,39 @@ class ConfigEncryptionTool {
}
}
+/**
+ * Writes the contents of the authorizers configuration file with
encrypted values to the output {@code authorizers.xml} file.
+ *
+ * @throw IOException if there is a problem reading or writing the
authorizers.xml file
+ */
+private void writeAuthorizers() throws IOException {
+if (!outputAuthorizersPath) {
+throw new IllegalArgumentException("Cannot write encrypted
properties to empty authorizers.xml path")
+}
+
+File outputAuthorizersFile = new File(outputAuthorizersPath)
+
+if (isSafeToWrite(outputAuthorizersFile)) {
+try {
+String updatedXmlContent
+File authorizersFile = new File(authorizersPath)
+if (authorizersFile.exists() && authorizersFile.canRead())
{
+// Instead of just writing the XML content to a file,
this method attempts to maintain the structure of the original file and
preserves comments
+updatedXmlContent =
serializeAuthorizersAndPreserveFormat(authorizers, authorizersFile).join("\n")
+}
--- End diff --
Due to a possible race condition (`authorizersFile` exists and can be read
when the tool execution starts, but has been deleted/made unreadable by an
external process before `writeAuthorizers` executes), the value of
`updatedXmlContent` will be empty, and it will overwrite `authorizers.xml`.
There should be an `else` branch here which simply serializes `authorizers` to
XML without the preserved whitespace and comments in order to maintain the
content.
This should probably also be done for the LDAP section.
> Support encrypted properties in authorizers.xml
> ---
>
> Key: NIFI-4701
> URL: https://issues.apache.org/jira/browse/NIFI-4701
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 1.5.0
>
>
> Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0,
> authorizers.xml can now contain properties for LDAP Server credentials.
> This ticket is to enable properties in authorizers.xml to be encrypted, so
> that the LDAP Server Manager credentials can be protected similar to
> LdapProvider which is configured via login-identity-providers.xml.
> The main changes are in nifi-authorizers are:
> * authorizers.xsd to add an encryption attribute to Property
> * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt
> the property value if necessary when creating the the configuration context
> Additionally, support for creating an encrypted authorizers.xml, protected by
> the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306534#comment-16306534
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159102652
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
---
@@ -319,6 +320,59 @@ class ConfigEncryptionToolTest extends GroovyTestCase {
}
}
+@Test
+void testShouldParseAuthorizersArgument() {
+// Arrange
+def flags = ["-a", "--authorizers"]
+String authorizersPath = "src/test/resources/authorizers.xml"
+ConfigEncryptionTool tool = new ConfigEncryptionTool()
+
+// Act
+flags.each { String arg ->
+tool.parse([arg, authorizersPath] as String[])
+logger.info("Parsed authorizers.xml location:
${tool.authorizersPath}")
+
+// Assert
+assert tool.authorizersPath == authorizersPath
+assert tool.handlingAuthorizers
+}
+}
+
+@Test
+void testShouldParseOutputAuthorizersArgument() {
+// Arrange
+def flags = ["-u", "--outputAuthorizers"]
+String authorizersPath = "src/test/resources/authorizers.xml"
+ConfigEncryptionTool tool = new ConfigEncryptionTool()
+
+// Act
+flags.each { String arg ->
+tool.parse([arg, authorizersPath, "-a", authorizersPath] as
String[])
--- End diff --
Change so the `outputAuthorizersPath` is different from `authorizersPath`
(just call `authorizersPath.reverse()`; it doesn't have to be a valid file) to
ensure from the equality check at the end that the correct value is being read
here.
> Support encrypted properties in authorizers.xml
> ---
>
> Key: NIFI-4701
> URL: https://issues.apache.org/jira/browse/NIFI-4701
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 1.5.0
>
>
> Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0,
> authorizers.xml can now contain properties for LDAP Server credentials.
> This ticket is to enable properties in authorizers.xml to be encrypted, so
> that the LDAP Server Manager credentials can be protected similar to
> LdapProvider which is configured via login-identity-providers.xml.
> The main changes are in nifi-authorizers are:
> * authorizers.xsd to add an encryption attribute to Property
> * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt
> the property value if necessary when creating the the configuration context
> Additionally, support for creating an encrypted authorizers.xml, protected by
> the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306969#comment-16306969
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159130193
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
---
@@ -2506,92 +2604,789 @@ class ConfigEncryptionToolTest extends
GroovyTestCase {
}
@Test
-void
testShouldPerformFullOperationForNiFiPropertiesAndLoginIdentityProviders() {
+void testShouldDecryptAuthorizers() {
// Arrange
-exit.expectSystemExitWithStatus(0)
+String authorizersPath =
"src/test/resources/authorizers-populated-encrypted.xml"
+File authorizersFile = new File(authorizersPath)
-File tmpDir = setupTmpDir()
+setupTmpDir()
-File emptyKeyFile = new
File("src/test/resources/bootstrap_with_empty_master_key.conf")
-File bootstrapFile = new File("target/tmp/tmp_bootstrap.conf")
-bootstrapFile.delete()
+File workingFile = new File("target/tmp/tmp-authorizers.xml")
+workingFile.delete()
+Files.copy(authorizersFile.toPath(), workingFile.toPath())
+ConfigEncryptionTool tool = new ConfigEncryptionTool()
-Files.copy(emptyKeyFile.toPath(), bootstrapFile.toPath())
-final List originalBootstrapLines =
bootstrapFile.readLines()
-String originalKeyLine = originalBootstrapLines.find {
-it.startsWith(ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX)
-}
-logger.info("Original key line from bootstrap.conf:
${originalKeyLine}")
-assert originalKeyLine == ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX
+// Sanity check for decryption
+String cipherText =
"q4r7WIgN0MaxdAKM||SGgdCTPGSFEcuH4RraMYEdeyVbOx93abdWTVSWvh1w+klA"
+String EXPECTED_PASSWORD = "thisIsABadPassword"
+AESSensitivePropertyProvider spp = new
AESSensitivePropertyProvider(KEY_HEX_128)
+assert spp.unprotect(cipherText) == EXPECTED_PASSWORD
-final String EXPECTED_KEY_LINE =
ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX + KEY_HEX
+tool.keyHex = KEY_HEX_128
-// Set up the NFP file
-File inputPropertiesFile = new
File("src/test/resources/nifi_with_sensitive_properties_unprotected.properties")
-File outputPropertiesFile = new
File("target/tmp/tmp_nifi.properties")
-outputPropertiesFile.delete()
+def lines = workingFile.readLines()
+logger.info("Read lines: \n${lines.join("\n")}")
-NiFiProperties inputProperties = new
NiFiPropertiesLoader().load(inputPropertiesFile)
-logger.info("Loaded ${inputProperties.size()} properties from
input file")
-ProtectedNiFiProperties protectedInputProperties = new
ProtectedNiFiProperties(inputProperties)
-def originalSensitiveValues =
protectedInputProperties.getSensitivePropertyKeys().collectEntries { String key
-> [(key): protectedInputProperties.getProperty(key)] }
-logger.info("Original sensitive values:
${originalSensitiveValues}")
+// Act
+def decryptedLines =
tool.decryptAuthorizers(lines.join("\n")).split("\n")
+logger.info("Decrypted lines: \n${decryptedLines.join("\n")}")
-// Set up the LIP file
-File inputLIPFile = new
File("src/test/resources/login-identity-providers-populated.xml")
-File outputLIPFile = new File("target/tmp/tmp-lip.xml")
-outputLIPFile.delete()
+// Assert
+def passwordLines = decryptedLines.findAll { it =~
PASSWORD_PROP_REGEX }
+assert passwordLines.size() == AUTHORIZERS_PASSWORD_LINE_COUNT
+assert passwordLines.every { it =~ ">thisIsABadPassword<" }
+// Some lines were not encrypted originally so the encryption
attribute would not have been updated
+assert passwordLines.any { it =~ "encryption=\"none\"" }
+}
-String originalXmlContent = inputLIPFile.text
-logger.info("Original XML content: ${originalXmlContent}")
+@Test
+void testShouldDecryptAuthorizersWithMultilineElements() {
+// Arrange
+String authorizersPath =
"src/test/resources/authorizers-populated-encrypted-multiline.xml"
+File authorizersFile = new File(authorizersPath)
+
+setupTmpDir()
+
+File workingFile = new File("target/tmp/tmp-authorizers.xml")
+workingFile.delete()
+Files.copy(author
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306974#comment-16306974
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159130239
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
---
@@ -2506,92 +2604,789 @@ class ConfigEncryptionToolTest extends
GroovyTestCase {
}
@Test
-void
testShouldPerformFullOperationForNiFiPropertiesAndLoginIdentityProviders() {
+void testShouldDecryptAuthorizers() {
// Arrange
-exit.expectSystemExitWithStatus(0)
+String authorizersPath =
"src/test/resources/authorizers-populated-encrypted.xml"
+File authorizersFile = new File(authorizersPath)
-File tmpDir = setupTmpDir()
+setupTmpDir()
-File emptyKeyFile = new
File("src/test/resources/bootstrap_with_empty_master_key.conf")
-File bootstrapFile = new File("target/tmp/tmp_bootstrap.conf")
-bootstrapFile.delete()
+File workingFile = new File("target/tmp/tmp-authorizers.xml")
+workingFile.delete()
+Files.copy(authorizersFile.toPath(), workingFile.toPath())
+ConfigEncryptionTool tool = new ConfigEncryptionTool()
-Files.copy(emptyKeyFile.toPath(), bootstrapFile.toPath())
-final List originalBootstrapLines =
bootstrapFile.readLines()
-String originalKeyLine = originalBootstrapLines.find {
-it.startsWith(ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX)
-}
-logger.info("Original key line from bootstrap.conf:
${originalKeyLine}")
-assert originalKeyLine == ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX
+// Sanity check for decryption
+String cipherText =
"q4r7WIgN0MaxdAKM||SGgdCTPGSFEcuH4RraMYEdeyVbOx93abdWTVSWvh1w+klA"
+String EXPECTED_PASSWORD = "thisIsABadPassword"
+AESSensitivePropertyProvider spp = new
AESSensitivePropertyProvider(KEY_HEX_128)
+assert spp.unprotect(cipherText) == EXPECTED_PASSWORD
-final String EXPECTED_KEY_LINE =
ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX + KEY_HEX
+tool.keyHex = KEY_HEX_128
-// Set up the NFP file
-File inputPropertiesFile = new
File("src/test/resources/nifi_with_sensitive_properties_unprotected.properties")
-File outputPropertiesFile = new
File("target/tmp/tmp_nifi.properties")
-outputPropertiesFile.delete()
+def lines = workingFile.readLines()
+logger.info("Read lines: \n${lines.join("\n")}")
-NiFiProperties inputProperties = new
NiFiPropertiesLoader().load(inputPropertiesFile)
-logger.info("Loaded ${inputProperties.size()} properties from
input file")
-ProtectedNiFiProperties protectedInputProperties = new
ProtectedNiFiProperties(inputProperties)
-def originalSensitiveValues =
protectedInputProperties.getSensitivePropertyKeys().collectEntries { String key
-> [(key): protectedInputProperties.getProperty(key)] }
-logger.info("Original sensitive values:
${originalSensitiveValues}")
+// Act
+def decryptedLines =
tool.decryptAuthorizers(lines.join("\n")).split("\n")
+logger.info("Decrypted lines: \n${decryptedLines.join("\n")}")
-// Set up the LIP file
-File inputLIPFile = new
File("src/test/resources/login-identity-providers-populated.xml")
-File outputLIPFile = new File("target/tmp/tmp-lip.xml")
-outputLIPFile.delete()
+// Assert
+def passwordLines = decryptedLines.findAll { it =~
PASSWORD_PROP_REGEX }
+assert passwordLines.size() == AUTHORIZERS_PASSWORD_LINE_COUNT
+assert passwordLines.every { it =~ ">thisIsABadPassword<" }
+// Some lines were not encrypted originally so the encryption
attribute would not have been updated
+assert passwordLines.any { it =~ "encryption=\"none\"" }
+}
-String originalXmlContent = inputLIPFile.text
-logger.info("Original XML content: ${originalXmlContent}")
+@Test
+void testShouldDecryptAuthorizersWithMultilineElements() {
+// Arrange
+String authorizersPath =
"src/test/resources/authorizers-populated-encrypted-multiline.xml"
+File authorizersFile = new File(authorizersPath)
+
+setupTmpDir()
+
+File workingFile = new File("target/tmp/tmp-authorizers.xml")
+workingFile.delete()
+Files.copy(author
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[
https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306975#comment-16306975
]
ASF GitHub Bot commented on NIFI-4701:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2350#discussion_r159130240
--- Diff:
nifi-toolkit/nifi-toolkit-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
---
@@ -2506,92 +2604,789 @@ class ConfigEncryptionToolTest extends
GroovyTestCase {
}
@Test
-void
testShouldPerformFullOperationForNiFiPropertiesAndLoginIdentityProviders() {
+void testShouldDecryptAuthorizers() {
// Arrange
-exit.expectSystemExitWithStatus(0)
+String authorizersPath =
"src/test/resources/authorizers-populated-encrypted.xml"
+File authorizersFile = new File(authorizersPath)
-File tmpDir = setupTmpDir()
+setupTmpDir()
-File emptyKeyFile = new
File("src/test/resources/bootstrap_with_empty_master_key.conf")
-File bootstrapFile = new File("target/tmp/tmp_bootstrap.conf")
-bootstrapFile.delete()
+File workingFile = new File("target/tmp/tmp-authorizers.xml")
+workingFile.delete()
+Files.copy(authorizersFile.toPath(), workingFile.toPath())
+ConfigEncryptionTool tool = new ConfigEncryptionTool()
-Files.copy(emptyKeyFile.toPath(), bootstrapFile.toPath())
-final List originalBootstrapLines =
bootstrapFile.readLines()
-String originalKeyLine = originalBootstrapLines.find {
-it.startsWith(ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX)
-}
-logger.info("Original key line from bootstrap.conf:
${originalKeyLine}")
-assert originalKeyLine == ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX
+// Sanity check for decryption
+String cipherText =
"q4r7WIgN0MaxdAKM||SGgdCTPGSFEcuH4RraMYEdeyVbOx93abdWTVSWvh1w+klA"
+String EXPECTED_PASSWORD = "thisIsABadPassword"
+AESSensitivePropertyProvider spp = new
AESSensitivePropertyProvider(KEY_HEX_128)
+assert spp.unprotect(cipherText) == EXPECTED_PASSWORD
-final String EXPECTED_KEY_LINE =
ConfigEncryptionTool.BOOTSTRAP_KEY_PREFIX + KEY_HEX
+tool.keyHex = KEY_HEX_128
-// Set up the NFP file
-File inputPropertiesFile = new
File("src/test/resources/nifi_with_sensitive_properties_unprotected.properties")
-File outputPropertiesFile = new
File("target/tmp/tmp_nifi.properties")
-outputPropertiesFile.delete()
+def lines = workingFile.readLines()
+logger.info("Read lines: \n${lines.join("\n")}")
-NiFiProperties inputProperties = new
NiFiPropertiesLoader().load(inputPropertiesFile)
-logger.info("Loaded ${inputProperties.size()} properties from
input file")
-ProtectedNiFiProperties protectedInputProperties = new
ProtectedNiFiProperties(inputProperties)
-def originalSensitiveValues =
protectedInputProperties.getSensitivePropertyKeys().collectEntries { String key
-> [(key): protectedInputProperties.getProperty(key)] }
-logger.info("Original sensitive values:
${originalSensitiveValues}")
+// Act
+def decryptedLines =
tool.decryptAuthorizers(lines.join("\n")).split("\n")
+logger.info("Decrypted lines: \n${decryptedLines.join("\n")}")
-// Set up the LIP file
-File inputLIPFile = new
File("src/test/resources/login-identity-providers-populated.xml")
-File outputLIPFile = new File("target/tmp/tmp-lip.xml")
-outputLIPFile.delete()
+// Assert
+def passwordLines = decryptedLines.findAll { it =~
PASSWORD_PROP_REGEX }
+assert passwordLines.size() == AUTHORIZERS_PASSWORD_LINE_COUNT
+assert passwordLines.every { it =~ ">thisIsABadPassword<" }
+// Some lines were not encrypted originally so the encryption
attribute would not have been updated
+assert passwordLines.any { it =~ "encryption=\"none\"" }
+}
-String originalXmlContent = inputLIPFile.text
-logger.info("Original XML content: ${originalXmlContent}")
+@Test
+void testShouldDecryptAuthorizersWithMultilineElements() {
+// Arrange
+String authorizersPath =
"src/test/resources/authorizers-populated-encrypted-multiline.xml"
+File authorizersFile = new File(authorizersPath)
+
+setupTmpDir()
+
+File workingFile = new File("target/tmp/tmp-authorizers.xml")
+workingFile.delete()
+Files.copy(author
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[ https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307323#comment-16307323 ] ASF GitHub Bot commented on NIFI-4701: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2350 > Support encrypted properties in authorizers.xml > --- > > Key: NIFI-4701 > URL: https://issues.apache.org/jira/browse/NIFI-4701 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 1.5.0 > > > Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0, > authorizers.xml can now contain properties for LDAP Server credentials. > This ticket is to enable properties in authorizers.xml to be encrypted, so > that the LDAP Server Manager credentials can be protected similar to > LdapProvider which is configured via login-identity-providers.xml. > The main changes are in nifi-authorizers are: > * authorizers.xsd to add an encryption attribute to Property > * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt > the property value if necessary when creating the the configuration context > Additionally, support for creating an encrypted authorizers.xml, protected by > the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[ https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307322#comment-16307322 ] ASF subversion and git services commented on NIFI-4701: --- Commit 482f371958a96b33791c3f6cc4c26efd47c43169 in nifi's branch refs/heads/master from [~kdoran] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=482f371 ] NIFI-4701 Add authorizers.xml support to toolkit. Adds authorizers.xml to the files understood by the encrypt-config tool in the NiFi Toolkit. If enabled, then the sensitive properties for LdapUserGroupProvider in authorizers.xml will be encrypted. Also fixes a bug wherein encrypt-config replaces multiple XML nodes in login-indentity-providers.xml when LdapProvider is not the first provider listed in the file. Enable properties in authorizers.xml to be encrypted by the master key. This closes #2350. Signed-off-by: Andy LoPresto > Support encrypted properties in authorizers.xml > --- > > Key: NIFI-4701 > URL: https://issues.apache.org/jira/browse/NIFI-4701 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 1.5.0 > > > Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0, > authorizers.xml can now contain properties for LDAP Server credentials. > This ticket is to enable properties in authorizers.xml to be encrypted, so > that the LDAP Server Manager credentials can be protected similar to > LdapProvider which is configured via login-identity-providers.xml. > The main changes are in nifi-authorizers are: > * authorizers.xsd to add an encryption attribute to Property > * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt > the property value if necessary when creating the the configuration context > Additionally, support for creating an encrypted authorizers.xml, protected by > the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[ https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307325#comment-16307325 ] ASF GitHub Bot commented on NIFI-4701: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2350 Made all changes noted in the review comments. Ran sample executions with invalid and valid input files / scenarios to ensure the tooling changes worked. Ran `contrib-check` and all tests pass. +1, merging. > Support encrypted properties in authorizers.xml > --- > > Key: NIFI-4701 > URL: https://issues.apache.org/jira/browse/NIFI-4701 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 1.5.0 > > > Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0, > authorizers.xml can now contain properties for LDAP Server credentials. > This ticket is to enable properties in authorizers.xml to be encrypted, so > that the LDAP Server Manager credentials can be protected similar to > LdapProvider which is configured via login-identity-providers.xml. > The main changes are in nifi-authorizers are: > * authorizers.xsd to add an encryption attribute to Property > * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt > the property value if necessary when creating the the configuration context > Additionally, support for creating an encrypted authorizers.xml, protected by > the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
