Modified: nifi/site/trunk/docs/nifi-docs/html/administration-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/administration-guide.html?rev=1758318&r1=1758317&r2=1758318&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/administration-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/administration-guide.html Tue Aug 30 03:18:37 2016 @@ -442,14 +442,24 @@ body.book #toc,body.book #preamble,body. <li><a href="#system-requirements">System Requirements</a></li> <li><a href="#how-to-install-and-start-nifi">How to install and start NiFi</a></li> <li><a href="#configuration-best-practices">Configuration Best Practices</a></li> -<li><a href="#security-configuration">Security Configuration</a></li> +<li><a href="#security-configuration">Security Configuration</a> +<ul class="sectlevel2"> +<li><a href="#tls-generation-toolkit">TLS Generation Toolkit</a></li> +</ul> +</li> <li><a href="#user-authentication">User Authentication</a> <ul class="sectlevel2"> <li><a href="#lightweight-directory-access-protocol-ldap">Lightweight Directory Access Protocol (LDAP)</a></li> <li><a href="#kerberos_login_identity_provider">Kerberos</a></li> </ul> </li> -<li><a href="#controlling-levels-of-access">Controlling Levels of Access</a></li> +<li><a href="#multi-tenant-authorization">Multi-Tenant Authorization</a> +<ul class="sectlevel2"> +<li><a href="#authorizer-configuration">Authorizer Configuration</a></li> +<li><a href="#authorizers-setup">Authorizers.xml Setup</a></li> +<li><a href="#config-users-access-policies">Configuring Users & Access Policies</a></li> +</ul> +</li> <li><a href="#encryption">Encryption Configuration</a> <ul class="sectlevel2"> <li><a href="#key-derivation-functions">Key Derivation Functions</a></li> @@ -458,6 +468,13 @@ body.book #toc,body.book #preamble,body. <li><a href="#allow-insecure-cryptographic-modes">Allow Insecure Cryptographic Modes</a></li> </ul> </li> +<li><a href="#encrypted-passwords-in-configuration-files">Encrypted Passwords in Configuration Files</a> +<ul class="sectlevel2"> +<li><a href="#encrypt-config_tool">Encrypt-Config Tool</a></li> +<li><a href="#encrypt-config_password">Password Key Derivation</a></li> +<li><a href="#encrypt-config_secure_prompt">Secure Prompt</a></li> +</ul> +</li> <li><a href="#clustering">Clustering Configuration</a></li> <li><a href="#state_management">State Management</a> <ul class="sectlevel2"> @@ -474,7 +491,32 @@ body.book #toc,body.book #preamble,body. <li><a href="#kerberos_service_notes">Notes</a></li> </ul> </li> -<li><a href="#system_properties">System Properties</a></li> +<li><a href="#system_properties">System Properties</a> +<ul class="sectlevel2"> +<li><a href="#core-properties-br">Core Properties<br></a></li> +<li><a href="#state-management-br">State Management<br></a></li> +<li><a href="#h2-settings">H2 Settings</a></li> +<li><a href="#flowfile-repository">FlowFile Repository</a></li> +<li><a href="#swap-management">Swap Management</a></li> +<li><a href="#content-repository">Content Repository</a></li> +<li><a href="#file-system-content-repository-properties">File System Content Repository Properties</a></li> +<li><a href="#volatile-content-repository-properties">Volatile Content Repository Properties</a></li> +<li><a href="#provenance-repository">Provenance Repository</a></li> +<li><a href="#persistent-provenance-repository-properties">Persistent Provenance Repository Properties</a></li> +<li><a href="#volatile-provenance-repository-properties">Volatile Provenance Repository Properties</a></li> +<li><a href="#component-status-repository">Component Status Repository</a></li> +<li><a href="#site_to_site_properties">Site to Site Properties</a></li> +<li><a href="#web-properties">Web Properties</a></li> +<li><a href="#security-properties">Security Properties</a></li> +<li><a href="#identity-mapping-properties">Identity Mapping Properties</a></li> +<li><a href="#cluster-common-properties">Cluster Common Properties</a></li> +<li><a href="#cluster-node-properties">Cluster Node Properties</a></li> +<li><a href="#claim_management">Claim Management</a></li> +<li><a href="#zookeeper-properties">ZooKeeper Properties</a></li> +<li><a href="#kerberos_properties">Kerberos Properties</a></li> +<li><a href="#custom_properties">Custom Properties</a></li> +</ul> +</li> </ul> </div> </div> @@ -488,7 +530,7 @@ body.book #toc,body.book #preamble,body. <div class="ulist"> <ul> <li> -<p>Requires Java 7 or newer</p> +<p>Requires Java 8 or newer</p> </li> <li> <p>Supported Operating Systems:</p> @@ -514,16 +556,16 @@ body.book #toc,body.book #preamble,body. <div class="ulist"> <ul> <li> -<p>Internet Explorer 9+ (see note below)</p> +<p>Microsoft Edge: Current & (Current - 1)</p> </li> <li> -<p>Mozilla FireFox 24+</p> +<p>Mozilla FireFox: Current & (Current - 1)</p> </li> <li> -<p>Google Chrome 36+</p> +<p>Google Chrome: Current & (Current - 1)</p> </li> <li> -<p>Safari 8</p> +<p>Safari: Current & (Current - 1)</p> </li> </ul> </div> @@ -531,12 +573,6 @@ body.book #toc,body.book #preamble,body. </ul> </div> <div class="paragraph"> -<p><strong>Note</strong> There is a known issue in Internet Explorer (IE) 10 and 11 that can cause problems when moving items on the NiFi graph. If you encounter this problem, we suggest using a browser other than IE. This known issue is described here: <a href="https://connect.microsoft.com/IE/Feedback/Details/1050422"; class="bare">https://connect.microsoft.com/IE/Feedback/Details/1050422</a>.</p> -</div> -<div class="paragraph"> -<p><strong>Note</strong> Java 7 default perm gen sizing can result in <em>out of memory errors</em> due to the amount of classes loaded by NiFi. See the <a href="#bootstrap_properties">Bootstrap Properties</a> section for more information.</p> -</div> -<div class="paragraph"> <p><strong>Note</strong> Under sustained and extremely high throughput the CodeCache settings may need to be tuned to avoid sudden performance loss. See the <a href="#bootstrap_properties">Bootstrap Properties</a> section for more information.</p> </div> </div> @@ -796,10 +832,11 @@ and for the partition(s) of interest add </thead> <tfoot> <tr> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.anonymous.authorities</code></p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the roles that should be granted to users that connect over HTTPS anonymously. All users can make - use of anonymous access, however if they have been granted a particular level of access by an administrator - it will take precedence if they access NiFi using a client certificate or once they have logged in.</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.needClientAuth</code></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether or not connecting clients must authenticate themselves. Specifically this property is used + by the NiFi cluster protocol. If the Truststore properties are not set, this must be <code>false</code>. Otherwise, a value + of <code>true</code> indicates that nodes in the cluster will be authenticated and must have certificates that are trusted + by the Truststores.</p></td> </tr> </tfoot> <tbody> @@ -832,13 +869,6 @@ and for the partition(s) of interest add <td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.truststorePasswd</code></p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Truststore.</p></td> </tr> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.needClientAuth</code></p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether or not connecting clients must authenticate themselves. Specifically this property is used - by the NiFi cluster protocol. If the Truststore properties are not set, this must be <code>false</code>. Otherwise, a value - of <code>true</code> indicates that nodes in the cluster will be authenticated and must have certificates that are trusted - by the Truststores.</p></td> -</tr> </tbody> </table> <div class="paragraph"> @@ -861,15 +891,164 @@ It is important when enabling HTTPS that </div> <div class="paragraph"> <p>Similar to <code>nifi.security.needClientAuth</code>, the web server can be configured to require certificate based client authentication for users accessing -the User Interface. In order to do this it must be configured to not support username/password authentication (see below) and not grant access to -anonymous users (see <code>nifi.security.anonymous.authorities</code> above). Either of these options will configure the web server to WANT certificate based client -authentication. This will allow it to support users with certificates and those without that may be logging in with their credentials or those accessing -anonymously. If username/password authentication and anonymous access are not configured, the web server will REQUIRE certificate based client authentication.</p> +the User Interface. In order to do this it must be configured to not support username/password authentication (see below). Either of these options +will configure the web server to WANT certificate based client authentication. This will allow it to support users with certificates and those without +that may be logging in with their credentials or those accessing anonymously. If username/password authentication and anonymous access are not configured, +the web server will REQUIRE certificate based client authentication.</p> </div> <div class="paragraph"> <p>Now that the User Interface has been secured, we can easily secure Site-to-Site connections and inner-cluster communications, as well. This is accomplished by setting the <code>nifi.remote.input.secure</code> and <code>nifi.cluster.protocol.is.secure</code> properties, respectively, to <code>true</code>.</p> </div> +<div class="sect2"> +<h3 id="tls-generation-toolkit"><a class="anchor" href="#tls-generation-toolkit"></a>TLS Generation Toolkit</h3> +<div class="paragraph"> +<p>In order to facilitate the secure setup of NiFi, you can use the <code>tls-toolkit</code> command line utility to automatically generate the required keystores, truststore, and relevant configuration files. This is especially useful for securing multiple NiFi nodes, which can be a tedious and error-prone process.</p> +</div> +<div class="paragraph"> +<p>The <code>tls-toolkit</code> command line tool has two primary modes of operation:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Standalone — generates the certificate authority, keystores, truststores, and nifi.properties files in one command.</p> +</li> +<li> +<p>Client/Server mode — uses a Certificate Authority Server that accepts Certificate Signing Requests from clients, signs them, and sends the resulting certificates back. Both client and server validate the otherâs identity through a shared secret.</p> +</li> +</ol> +</div> +<div class="sect3"> +<h4 id="standalone"><a class="anchor" href="#standalone"></a>Standalone</h4> +<div class="paragraph"> +<p>Standalone mode is invoked by running <code>./bin/tls-toolkit.sh standalone -h</code> which prints the usage information along with descriptions of options that can be specified.</p> +</div> +<div class="paragraph"> +<p>The most common options to specify are:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-n,--hostnames</code> The comma-separated list of hostnames that youâd like to generate certificates for. It can be specified multiple times. Range and instance patterns are supported. See below for details.</p> +</li> +<li> +<p><code>-C,--clientCertDn</code> The DN that you’d like to generate a client certificate for. It can be specified multiple times.</p> +</li> +<li> +<p><code>-f,--nifiPropertiesFile</code> The base <em>nifi.properties</em> file that the tool will update for each host.</p> +</li> +<li> +<p><code>-o,--outputDirectory</code> The directory to use for the resulting Certificate Authority files and NiFi configurations. A subdirectory will be made for each host.</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Hostname Patterns:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Square brackets can be used in order to easily specify a range of hostnames. Example: [01-20]</p> +</li> +<li> +<p>Parentheses can be used in order to specify that more than one NiFi instance will run on the given host(s). Example: (5)</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Examples:</p> +</div> +<div class="paragraph"> +<p>Create 4 sets of keystore, truststore, nifi.properties for localhost along with a client certificate with the given DN:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>bin/tls-toolkit.sh standalone -n 'localhost(4)' -C 'CN=username,OU=NIFI'</pre> +</div> +</div> +<div class="paragraph"> +<p>Create keystore, truststore, nifi.properties for 10 NiFi hostnames in each of 4 subdomains:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>bin/tls-toolkit.sh standalone -n 'nifi[01-10].subdomain[1-4].domain'</pre> +</div> +</div> +<div class="paragraph"> +<p>Create 2 sets of keystore, truststore, nifi.properties for 10 NiFi hostnames in each of 4 subdomains along with a client certificate with the given DN:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>bin/tls-toolkit.sh standalone -n 'nifi[01-10].subdomain[1-4].domain(2)' -C 'CN=username,OU=NIFI'</pre> +</div> +</div> +</div> +<div class="sect3"> +<h4 id="client-server"><a class="anchor" href="#client-server"></a>Client/Server</h4> +<div class="paragraph"> +<p>Client/Server mode relies on a long-running Certificate Authority (CA) to issue certificates. The CA can be stopped when youâre not bringing nodes online.</p> +</div> +<div class="sect4"> +<h5 id="server"><a class="anchor" href="#server"></a>Server</h5> +<div class="paragraph"> +<p>The CA server is invoked by running <code>./bin/tls-toolkit server -h</code> prints the usage information along with descriptions of options that can be specified.</p> +</div> +<div class="paragraph"> +<p>The most common options to specify are:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-f,--configJson</code> The location of the json config (written after first run)</p> +</li> +<li> +<p><code>-F,--useConfigJson</code> Loads all relevant configuration from the config json (configJson is the only other argument necessary)</p> +</li> +<li> +<p><code>-t,--token</code> The token used to prevent man in the middle attacks (this should be a long, random value and needs to be known when invoking the client)</p> +</li> +<li> +<p><code>-D,--dn</code> The DN for the CA</p> +</li> +</ul> +</div> +</div> +<div class="sect4"> +<h5 id="client"><a class="anchor" href="#client"></a>Client</h5> +<div class="paragraph"> +<p>The client can be used to request new Certificates from the CA. The client utility generates a keypair and Certificate Signing Request (CSR) and sends the CSR to the Certificate Authority. The client is invoked by running <code>./bin/tls-toolkit.sh client -h</code> which prints the usage information along with descriptions of options that can be specified.</p> +</div> +<div class="paragraph"> +<p>The most common options to specify are:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-f,--configJson</code> The json config file</p> +</li> +<li> +<p><code>-c,--certificateAuthorityHostname</code> The hostname of the CA</p> +</li> +<li> +<p><code>-D,--DN</code> The DN for the CSR (and Certificate)</p> +</li> +<li> +<p><code>-t,--token</code> The token used to prevent man in the middle attacks (this should be a long, random value and needs to be the same one used to start the CA server)</p> +</li> +<li> +<p><code>-T,--keyStoreType</code> The type of keystore to create (leave default for NiFi nodes, specify PKCS12 to create client cert)</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>After running the client you will have the CAâs certificate, a keystore, a truststore, and a config.json with information about them as well as their passwords.</p> +</div> +<div class="paragraph"> +<p>For a client certificate that can be easily imported into the browser, specify: <code>-T PKCS12</code></p> +</div> +</div> +</div> +</div> </div> </div> <div class="sect1"> @@ -1087,220 +1266,866 @@ explicity enabled.</p> </div> </div> <div class="sect1"> -<h2 id="controlling-levels-of-access"><a class="anchor" href="#controlling-levels-of-access"></a>Controlling Levels of Access</h2> +<h2 id="multi-tenant-authorization"><a class="anchor" href="#multi-tenant-authorization"></a>Multi-Tenant Authorization</h2> <div class="sectionbody"> <div class="paragraph"> -<p>Once NiFi is configured to run securely and an authentication mechanism is configured, it is necessary -to configure who will have access to the system and what types of access those people will have. -NiFi controls this through the user of an <em>Authority Provider.</em> The Authority Provider is a pluggable -mechanism for providing authorizations to different users. Which Authority Provider to use is configured -using two properties in the <em>nifi.properties</em> file.</p> +<p>After you have configured NiFi to run securely and with an authentication mechanism, you must configure who has access to the system, and the level of their access. +You can do this using <em>multi-tenant authorization</em>. Multi-tenant authorization enables multiple groups of users (tenants) to command, control, and observe different +parts of the dataflow, with varying levels of authorization. When an authenticated user attempts to view or modify a NiFi resource, the system checks whether the +user has privileges to perform that action. These privileges are defined by policies that you can apply system-wide or to individual components.</p> </div> +<div class="sect2"> +<h3 id="authorizer-configuration"><a class="anchor" href="#authorizer-configuration"></a>Authorizer Configuration</h3> <div class="paragraph"> -<p>The <code>nifi.authority.provider.configuration.file</code> property specifies the configuration file for Authority Providers. -The <code>nifi.security.user.authority.provider</code> property indicates which of the configured Authority Providers should be -used.</p> +<p>An <em>authorizer</em> grants users the privileges to manage users and policies by creating preliminary authorizations at startup.</p> </div> <div class="paragraph"> -<p>By default, the <code>file-provider</code> Authority Provider is selected and is configured to use the permissions granted in -the <em>authorized-users.xml</em> file. This is typically sufficient for instances of NiFi that are run in "standalone" mode. -If the NiFi instance is configured to run in a cluster, the node will typically use the <code>cluster-node-provider</code> -Provider and the Cluster Manager will typically use the <code>cluster-ncm-provider</code> Provider. Both of these Providers -have a default configuration in the <em>authority-providers.xml</em> file but are commented out.</p> +<p>Authorizers are configured using two properties in the <em>nifi.properties</em> file:</p> </div> -<div class="paragraph"> -<p>When using the <code>cluster-node-provider</code> Provider, all of the authorization is provided by the Cluster Manager. In this -way, the configuration only has to be maintained in one place and will be consistent across the entire cluster.</p> +<div class="ulist"> +<ul> +<li> +<p>The <code>nifi.authorizer.configuration.file</code> property specifies the configuration file where authorizers are defined. By default, the <em>authorizers.xml</em> file located in the root installation conf directory is selected.</p> +</li> +<li> +<p>The <code>nifi.security.user.authorizer</code> property indicates which of the configured authorizers in the <em>authorizers.xml</em> file to use.</p> +</li> +</ul> </div> +</div> +<div class="sect2"> +<h3 id="authorizers-setup"><a class="anchor" href="#authorizers-setup"></a>Authorizers.xml Setup</h3> <div class="paragraph"> -<p>When configuring the Cluster Manager or a standalone node, it is necessary to manually designate an ADMIN user -in the <em>authorized-users.xml</em> file, which is located in the root installation’s conf directory. -After this ADMIN user has been added, s/he may grant access -to other users, systems, and other instances of NiFi, through the User Interface (UI) without having to manually edit the <em>authorized-users.xml</em> -file. If you are the administrator, you would add yourself as the ADMIN user in this file.</p> +<p>The <em>authorizers.xml</em> file is used to define and configure available authorizers. The default authorizer is the FileAuthorizer, however, you can develop additional authorizers as extensions. The FileAuthorizer has the following properties:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Authorizations File - The file where the FileAuthorizer stores policies. By default, the <em>authorizations.xml</em> in the <em>conf</em> directory is chosen.</p> +</li> +<li> +<p>Users File - The file where the FileAuthorizer stores users and groups. By default, the <em>users.xml</em> in the <em>conf</em> directory is chosen.</p> +</li> +<li> +<p>Initial Admin Identity - The identity of an initial admin user that is granted access to the UI and given the ability to create additional users, groups, and policies. This property is only used when there are no other users, groups, and policies defined.</p> +</li> +<li> +<p>Legacy Authorized Users File - The full path to an existing authorized-users.xml that is automatically converted to the multi-tenant authorization model. This property is only used when there are no other users, groups, and policies defined.</p> +</li> +<li> +<p>Node Identity - The identity of a NiFi cluster node. When clustered, a property for each node should be defined, so that every node knows about every other node. If not clustered, these properties can be ignored.</p> +</li> +</ul> </div> +<div class="sect3"> +<h4 id="initial-admin-identity"><a class="anchor" href="#initial-admin-identity"></a>Initial Admin Identity (New NiFi Instance)</h4> <div class="paragraph"> -<p>Open the <em>authorized-users.xml</em> file in a text editor. You will notice that it includes a template -to guide you, with example entries that are commented out.</p> +<p>If you are setting up a secured NiFi instance for the first time, you must manually designate an âInitial Admin Identityâ in the <em>authorizers.xml</em> file. This initial admin user is granted access to the UI and given the ability to create additional users, groups, and policies. The value of this property could be a DN (when using certificates or LDAP) or a Kerberos principal. If you are the NiFi administrator, add yourself as the âInitial Admin Identityâ.</p> </div> <div class="paragraph"> -<p>It is only necessary to manually add one user, the ADMIN user, to this file. -So, at a minimum, the following example entry should be included and contain the user Distinguished Name (DN) -in place of "user dn - read only and admin":</p> +<p>Here is an example LDAP entry using the name John Smith:</p> </div> <div class="listingblock"> <div class="content"> -<pre><users> - <user dn="[user dn - read only and admin]"> - <role name="ROLE_ADMIN"/> - </user> -</users></pre> +<pre><authorizer> + <identifier>file-provider</identifier> + <class>org.apache.nifi.authorization.FileAuthorizer</class> + <property name="Authorizations File">./conf/authorizations.xml</property> + <property name="Users File">./conf/users.xml</property> + <property name="Initial Admin Identity">cn=John Smith,ou=people,dc=example,dc=com</property> + <property name="Legacy Authorized Users File"></property> + <!-- + <property name="Node Identity 1"></property> + <property name="Node Identity 2"></property> + --> + </authorizer> +</authorizers></pre> </div> </div> <div class="paragraph"> -<p>Here is an LDAP example entry using the name John Smith:</p> +<p>Here is a example Kerberos entry using the name John Smith and realm <code>NIFI.APACHE.ORG</code>:</p> </div> <div class="listingblock"> <div class="content"> -<pre><users> - <user dn="cn=John Smith,ou=people,dc=example,dc=com"> - <role name="ROLE_ADMIN"/> - </user> -</users></pre> +<pre><authorizer> + <identifier>file-provider</identifier> + <class>org.apache.nifi.authorization.FileAuthorizer</class> + <property name="Authorizations File">./conf/authorizations.xml</property> + <property name="Users File">./conf/users.xml</property> + <property name="Initial Admin Identity">johnsm...@nifi.apache.org</property> + <property name="Legacy Authorized Users File"></property> + <!-- + <property name="Node Identity 1"></property> + <property name="Node Identity 2"></property> + --> + </authorizer> +</authorizers></pre> </div> </div> <div class="paragraph"> -<p>Here is a Kerberos example entry using the name John Smith and realm <code>NIFI.APACHE.ORG</code>:</p> +<p>After you have edited and saved the <em>authorizers.xml</em> file, restart NiFi. The âInitial Admin Identityâ user and administrative policies are added to the <em>authorizations.xml</em> file during restart. Once NiFi starts, the âInitial Admin Identityâ user is able to access the UI and begin managing users, groups, and policies.</p> </div> -<div class="listingblock"> -<div class="content"> -<pre><users> - <user dn="johnsm...@nifi.apache.org"> - <role name="ROLE_ADMIN"/> - </user> -</users></pre> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +For a brand new secure flow, providing the "Initial Admin Identity" gives that user access to get into the UI and to manage users, groups and policies. But if that user wants to start modifying the flow, they need to grant themselves policies for the root process group. The system is unable to do this automatically because in a new flow the UUID of the root process group is not permanent until the flow.xml.gz is generated. If the NiFi instance is an upgrade from an existing flow.xml.gz or a 1.x instance going from unsecure to secure, then the "Initial Admin Identity" user is automatically given the privileges to modify the flow. +</td> +</tr> +</table> </div> </div> +<div class="sect3"> +<h4 id="legacy-authorized-users"><a class="anchor" href="#legacy-authorized-users"></a>Legacy Authorized Users (NiFi Instance Upgrade)</h4> <div class="paragraph"> -<p>After the <em>authorized-users.xml</em> file has been edited and saved, restart NiFi. -Once the application starts, the ADMIN user is -able to access the UI at the HTTPS URL that is configured in the <em>nifi.properties</em> file.</p> +<p>If you are upgrading from a 0.x NiFi instance, you can convert your previously configured users and roles to the multi-tenant authorization model. In the <em>authorizers.xml</em> file, specify the location of your existing <em>authorized-users.xml</em> file in the âLegacy Authorized Users Fileâ property.</p> </div> <div class="paragraph"> -<p>From the UI, click on the Users icon ( <span class="image"><img src="./images/iconUsers.png" alt="Users" width="32"></span> ) in the -Management Toolbar (upper-right corner of the UI), and the User Management Page opens.</p> +<p>Here is an example entry:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre><authorizers> + <authorizer> + <identifier>file-provider</identifier> + <class>org.apache.nifi.authorization.FileAuthorizer</class> + <property name="Authorizations File">./conf/authorizations.xml</property> + <property name="Users File">./conf/users.xml</property> + <property name="Initial Admin Identity"></property> + <property name="Legacy Authorized Users File">/Users/johnsmith/config_files/authorized-users.xml</property> + </authorizer> +</authorizers></pre> +</div> </div> <div class="paragraph"> -<p>The ADMIN user should be listed. Click on the pencil icon to see this user’s role(s). You may edit the -roles by selecting the appropriate checkboxes.</p> +<p>After you have edited and saved the <em>authorizers.xml</em> file, restart NiFi. Users and roles from the <em>authorized-users.xml</em> file are converted and added as identities and policies in the <em>authorizations.xml</em> file. Once the application starts, users who previously had a legacy Administrator role can access the UI and begin managing users, groups, and policies.</p> </div> <div class="paragraph"> -<p>The following roles are available in NiFi:</p> +<p>Here is a summary of policies assigned to each legacy role if the NiFi instance has an existing flow.xml.gz:</p> </div> <table class="tableblock frame-all grid-all spread"> <colgroup> -<col style="width: 50%;"> -<col style="width: 50%;"> +<col style="width: 14%;"> +<col style="width: 14%;"> +<col style="width: 14%;"> +<col style="width: 14%;"> +<col style="width: 14%;"> +<col style="width: 14%;"> +<col style="width: 14%;"> </colgroup> <thead> <tr> -<th class="tableblock halign-left valign-top">Role Name</th> -<th class="tableblock halign-left valign-top">Description</th> +<th class="tableblock halign-right valign-top"></th> +<th class="tableblock halign-center valign-top">Admin</th> +<th class="tableblock halign-center valign-top">DFM</th> +<th class="tableblock halign-center valign-top">Monitor</th> +<th class="tableblock halign-center valign-top">Provenance</th> +<th class="tableblock halign-center valign-top">NiFi</th> +<th class="tableblock halign-center valign-top">Proxy</th> </tr> </thead> -<tfoot> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Proxy</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The Proxy Role is assigned to a system in order to grant that system permission to make requests - on behalf of a user. For instance, if an HTTP proxy service is used to gain access to the system, - the certificate being used by that service can be given the Proxy Role.</p></td> -</tr> -</tfoot> <tbody> <tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Administrator</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Administrator is able to configure thread pool sizes and user accounts as well as - purge the dataflow change history.</p></td> -</tr> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Data Flow Manager</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Data Flow Manager is given the ability to manipulate the dataflow. S/he is able to - add, remove, and manipulate components on the graph; add, remove, and manipulate - Controller Services and Reporting Tasks; create and manage templates; - view statistics; and view the bulletin board.</p></td> -</tr> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Read Only</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Users with Read Only access are able to view the dataflow but are unable to change anything.</p></td> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the UI</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the controller</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>modify the controller</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view system diagnostics</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the dataflow</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>modify the dataflow</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the users/groups</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>modify the users/groups</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view policies</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>modify policies</strong></p></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>query provenance</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the data</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>modify the data</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>retrieve site-to-site details</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +</tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>send proxy user requests</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> </tr> +</tbody> +</table> +<div class="paragraph"> +<p>For details on the policies in the table, see <a href="#access-policies">Access Policies</a>.</p> +</div> +<div class="admonitionblock note"> +<table> <tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Provenance</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Users with Provenance access are able to query the Data Provenance repository and view - the lineage of data. Additionally, this role provides the ability to view or download - the content of a FlowFile from a Provenance event (assuming that the content is still - available in the Content Repository and that the Authority Provider also grants access). - This access is not provided to users with Read Only - (unless the user has both Read Only and Provenance roles) because the information provided - to users with this role can potentially be very sensitive in nature, as all FlowFile attributes - and data are exposed. In order to Replay a Provenance event, a user is required to have both - the Provenance role as well as the Data Flow Manager role.</p></td> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +NiFi fails to restart if values exist for both the âInitial Admin Identityâ and âLegacy Authorized Users Fileâ properties. You can specify only one of these values to initialize authorizations. +</td> </tr> +</table> +</div> +<div class="admonitionblock note"> +<table> <tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">NiFi</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The NiFi Role is intended to be assigned to machines that will interact with an instance of NiFi - via Site-to-Site. This role provides the ability to send data to or retrieve data from Root - Group Ports (but only those that they are given permissions to interact with - see the User Guide - for more information on providing access to specific Ports) as well as obtain information about - which Ports exist. Note that this role allows the client to know only about the Ports that it - has permissions to interact with.</p></td> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +Do not manually edit the <em>authorizations.xml</em> file. Create authorizations only during initial setup and afterwards using the NiFi UI. +</td> </tr> -</tbody> </table> -<div class="paragraph"> -<p>When users want access to the NiFi UI, they navigate to the configured URL and are -prompted to request access. When someone has requested access, the ADMIN user sees a star -on the Users icon in the Management Toolbar, alerting the ADMIN to the fact that a request is -pending. Upon opening the User Management Page, the pending request is visible, and the ADMIN -can grant access and click on the pencil icon to set the user’s roles appropriately.</p> </div> +</div> +<div class="sect3"> +<h4 id="cluster-node-identities"><a class="anchor" href="#cluster-node-identities"></a>Cluster Node Identities</h4> <div class="paragraph"> -<p>The ADMIN may also select multiple users and add them to a "Group". Hold down the Shift key and select -multiple users, then click the <code>Group</code> button in the upper-right corner of the User Management Page. -Then, provide a name for the group.</p> +<p>If you are running NiFi in a clustered environment, you must specify the identities for each node. The authorization policies required for the nodes to communicate are created during startup.</p> </div> <div class="paragraph"> -<p>The group feature is especially useful when a remote NiFi cluster is connecting to this NiFi using -a Remote Process Group. In that scenario, all the nodes -in the remote cluster can be included in the same group. When the ADMIN wants to grant port access to the remote -cluster, s/he can grant it to the group and avoid having to grant it individually to each node in the cluster.</p> +<p>For example, if you are setting up a 2 node cluster with the following DNs for each node:</p> </div> +<div class="listingblock"> +<div class="content"> +<pre>cn=nifi-1,ou=people,dc=example,dc=com +cn=nifi-2,ou=people,dc=example,dc=com</pre> </div> </div> -<div class="sect1"> -<h2 id="encryption"><a class="anchor" href="#encryption"></a>Encryption Configuration</h2> -<div class="sectionbody"> -<div class="paragraph"> -<p>This section provides an overview of the capabilities of NiFi to encrypt and decrypt data.</p> +<div class="listingblock"> +<div class="content"> +<pre><authorizer> + <identifier>file-provider</identifier> + <class>org.apache.nifi.authorization.FileAuthorizer</class> + <property name="Authorizations File">./conf/authorizations.xml</property> + <property name="Users File">./conf/users.xml</property> + <property name="Initial Admin Identity">johnsm...@nifi.apache.org</property> + <property name="Legacy Authorized Users File"></property> + <property name="Node Identity 1">cn=nifi-1,ou=people,dc=example,dc=com</property> + <property name="Node Identity 2">cn=nifi-2,ou=people,dc=example,dc=com</property> + </authorizer> +</authorizers></pre> +</div> +</div> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +In a cluster, all nodes must have the same <em>authorizations.xml</em>. If a node has a different <em>authorizations.xml</em>, it cannot join the cluster. The only exception is if a node has an empty <em>authorizations.xml</em>. In this scenario, the node inherits the <em>authorizations.xml</em> from the cluster. +</td> +</tr> +</table> </div> <div class="paragraph"> -<p>The <code>EncryptContent</code> processor allows for the encryption and decryption of data, both internal to NiFi and integrated with external systems, such as <code>openssl</code> and other data sources and consumers.</p> +<p>Now that initial authorizations have been created, additional users, groups and authorizations can be created and managed in the NiFi UI.</p> +</div> +</div> </div> <div class="sect2"> -<h3 id="key-derivation-functions"><a class="anchor" href="#key-derivation-functions"></a>Key Derivation Functions</h3> +<h3 id="config-users-access-policies"><a class="anchor" href="#config-users-access-policies"></a>Configuring Users & Access Policies</h3> <div class="paragraph"> -<p>Key Derivation Functions (KDF) are mechanisms by which human-readable information, usually a password or other secret information, is translated into a cryptographic key suitable for data protection. For further information, read <a href="https://en.wikipedia.org/wiki/Key_derivation_function";>the Wikipedia entry on Key Derivation Functions</a>. -Currently, KDFs are ingested by <code>CipherProvider</code> implementations and return a fully-initialized <code>Cipher</code> object to be used for encryption or decryption. Due to the use of a <code>CipherProviderFactory</code>, the KDFs are not customizable at this time. Future enhancements will include the ability to provide custom cost parameters to the KDF at initialization time. As a work-around, <code>CipherProvider</code> instances can be initialized with custom cost parameters in the constructor but this is not currently supported by the <code>CipherProviderFactory</code>. -Here are the KDFs currently supported by NiFi (primarily in the <code>EncryptContent</code> processor for password-based encryption (PBE)) and relevant notes:</p> +<p>This section describes:</p> </div> <div class="ulist"> <ul> <li> -<p>NiFi Legacy KDF</p> -<div class="ulist"> -<ul> -<li> -<p>The original KDF used by NiFi for internal key derivation for PBE, this is 1000 iterations of the MD5 digest over the concatenation of the password and 8 or 16 bytes of random salt (the salt length depends on the selected cipher block size).</p> -</li> -<li> -<p>This KDF is <strong>deprecated as of NiFi 0.5.0</strong> and should only be used for backwards compatibility to decrypt data that was previously encrypted by a legacy version of NiFi.</p> -</li> -</ul> -</div> +<p>How to create users and groups</p> </li> <li> -<p>OpenSSL PKCS#5 v1.5 EVP_BytesToKey</p> -<div class="ulist"> -<ul> -<li> -<p>This KDF was added in v0.4.0.</p> +<p>How access policies are used to define authorizations</p> </li> <li> -<p>This KDF is provided for compatibility with data encrypted using OpenSSL’s default PBE, known as <code>EVP_BytesToKey</code>. This is a single iteration of MD5 over the concatenation of the password and 8 bytes of random ASCII salt. OpenSSL recommends using <code>PBKDF2</code> for key derivation but does not expose the library method necessary to the command-line tool, so this KDF is still the de facto default for command-line encryption.</p> +<p>How to configure access policies by walking through specific examples</p> </li> </ul> </div> -</li> -<li> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +Instructions requiring interaction with the UI assume the application is being accessed by User1, a user with administrator privileges, such as the âInitial Admin Identityâ user or a converted legacy admin user (see <a href="#authorizers-setup">Authorizers.xml Setup</a>). +</td> +</tr> +</table> +</div> +<div class="sect3"> +<h4 id="creating-users-groups"><a class="anchor" href="#creating-users-groups"></a>Creating Users and Groups</h4> +<div class="paragraph"> +<p>From the UI, select âUsersâ from the Global Menu. This opens a dialog to create and manage users and groups.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/nifi-users-dialog.png" alt="NiFi Users Dialog"></span></p> +</div> +<div class="paragraph"> +<p>Click the Add icon (<span class="image"><img src="./images/iconAddUser.png" alt="Add User Icon"></span>). To create a user, enter the <em>Identity</em> information relevant to the authentication method chosen to secure your NiFi instance. Click OK.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user-creation-dialog.png" alt="User Creation Dialog"></span></p> +</div> +<div class="paragraph"> +<p>To create a group, select the âGroupâ radio button, enter the name of the group and select the users to be included in the group. Click OK.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/group-creation-dialog.png" alt="Group Creation Dialog"></span></p> +</div> +</div> +<div class="sect3"> +<h4 id="access-policies"><a class="anchor" href="#access-policies"></a>Access Policies</h4> +<div class="paragraph"> +<p>You can manage the ability for users and groups to view or modify NiFi resources using <em>access policies</em>. There are two types of access policies that can be applied to a resource:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>View —  If a view policy is created for a resource, only the users or groups that are added to that policy are able to see the details of that resource.</p> +</li> +<li> +<p>Modify — If a resource has a modify policy, only the users or groups that are added to that policy can change the configuration of that resource.</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>You can create and apply access policies on both global and component levels.</p> +</div> +<div class="sect4"> +<h5 id="global-access-policies"><a class="anchor" href="#global-access-policies"></a>Global Access Policies</h5> +<div class="paragraph"> +<p>Global access policies govern the following system level authorizations:</p> +</div> +<table class="tableblock frame-all grid-all spread"> +<colgroup> +<col style="width: 33%;"> +<col style="width: 33%;"> +<col style="width: 33%;"> +</colgroup> +<thead> +<tr> +<th class="tableblock halign-left valign-top">Policy</th> +<th class="tableblock halign-left valign-top">Privilege</th> +<th class="tableblock halign-left valign-top">Global Menu Selection</th> +</tr> +</thead> +<tbody> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">view the UI</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allow users to view the UI</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">access the controller</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the controller including Reporting Tasks, Controller Services, and Nodes in the Cluster</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Controller Settings</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">query provenance</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to submit a Provenance Search and request Event Lineage</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Data Provenance</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">access all policies</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the policies for all components</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Policies</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">access users/user groups</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the users and user groups</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Users</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">retrieve site-to-site details</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows other NiFi instances to retrieve Site-To-Site details</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">view system diagnostics</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view System Diagnostics</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Summary</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">proxy user requests</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows proxy machines to send requests on the behalf of others</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">access counters</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify Counters</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Counters</p></td> +</tr> +</tbody> +</table> +</div> +<div class="sect4"> +<h5 id="component-level-access-policies"><a class="anchor" href="#component-level-access-policies"></a>Component Level Access Policies</h5> +<div class="paragraph"> +<p>Component level access policies govern the following component level authorizations:</p> +</div> +<table class="tableblock frame-all grid-all spread"> +<colgroup> +<col style="width: 50%;"> +<col style="width: 50%;"> +</colgroup> +<thead> +<tr> +<th class="tableblock halign-left valign-top">Policy</th> +<th class="tableblock halign-left valign-top">Privilege</th> +</tr> +</thead> +<tbody> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">view the component</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view component configuration details</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">modify the component</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to modify component configuration details</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">view the data</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows user to view metadata and content for this component through provenance data and flowfile queues in outbound connections</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">modify the data</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows user to empty flowfile queues in outbound connections and submit replays</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">view the policies</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view the list of users who can view/modify a component</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">modify the policies</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to modify the list of users who can view/modify a component</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">retrieve data via site-to-site</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows a port to receive data from NiFi instances</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">send data via site-to-site</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows a port to send data from NiFi instances</p></td> +</tr> +</tbody> +</table> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +You can apply access policies to all component types except connections. Connection authorizations are inferred by the individual access policies on the source and destination components of the connection, as well as the access policy of the process group containing the components. This is discussed in more detail in the <a href="#creating-a-connection">Creating a Connection</a> and <a href="#editing-a-connection">Editing a Connection</a> examples below. +</td> +</tr> +</table> +</div> +</div> +<div class="sect4"> +<h5 id="access-policy-inheritance"><a class="anchor" href="#access-policy-inheritance"></a>Access Policy Inheritance</h5> +<div class="paragraph"> +<p>An administrator does not need to manually create policies for every component in the dataflow. To reduce the amount of time admins spend on authorization management, policies are inherited from parent resource to child resource. For example, if a user is given access to view and modify a process group, that user can also view and modify the components in the process group. Policy inheritance enables an administrator to assign policies at one time and have the policies apply throughout the entire dataflow.</p> +</div> +<div class="paragraph"> +<p>You can override an inherited policy (as described in the <a href="#moving-a-processor">Moving a Processor</a> example below). Overriding a policy removes the inherited policy, breaking the chain of inheritance from parent to child, and creates a replacement policy to add users as desired. Inherited policies and their users can be restored by deleting the replacement policy.</p> +</div> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +âView the policiesâ and âmodify the policiesâ component-level access policies are an exception to this inherited behavior.âWhen a user is added to either policy, they are added to the current list of administrators.âThey do not override higher level administrators.âFor this reason, only component specific administrators are displayed for the âview the policiesâ and âmodify the policies" access policies. +</td> +</tr> +</table> +</div> +</div> +</div> +<div class="sect3"> +<h4 id="access-policy-config-examples"><a class="anchor" href="#access-policy-config-examples"></a>Access Policy Configuration Examples</h4> +<div class="paragraph"> +<p>The most effective way to understand how to create and apply access policies is to walk through some common examples. The following scenarios assume User1 is an administrator and User2 is a newly added user that has only been given access to the UI.</p> +</div> +<div class="paragraph"> +<p>Letâs begin with two processors on the canvas as our starting point: GenerateFlowFile and LogAttribute.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/access-policy-config-start.png" alt="Access Policy Config Start"></span></p> +</div> +<div class="paragraph"> +<p>User1 can add components to the dataflow and is able to move, edit and connect all processors. The details and properties of the root process group and processors are visible to User1.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user1-full-access.png" alt="User1 Full Access"></span></p> +</div> +<div class="paragraph"> +<p>User1 wants to maintain their current privileges to the dataflow and its components.</p> +</div> +<div class="paragraph"> +<p>User2 is unable to add components to the dataflow or move, edit, or connect components. The details and properties of the root process group and processors are hidden from User2.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-restricted-access.png" alt="User2 Restricted Access"></span></p> +</div> +<div class="sect4"> +<h5 id="moving-a-processor"><a class="anchor" href="#moving-a-processor"></a>Moving a Processor</h5> +<div class="paragraph"> +<p>To allow User2 to move the GenerateFlowFile processor in the dataflow and only that processor, User1 performs the following steps:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Select the GenerateFlowFile processor so that it is highlighted.</p> +</li> +<li> +<p>Select the Access Policies icon (<span class="image"><img src="./images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette and the Access Policies dialog opens.</p> +</li> +<li> +<p>Select âmodify the componentâ from the policy drop-down. +<span class="image"><img src="./images/processor-modify-policy.png" alt="Processor Modify Policy"></span> +The âmodify the componentâ policy that currently exists on the processor (child) is the âmodify the componentâ policy inherited from the root process group (parent) on which User1 has privileges.</p> +</li> +<li> +<p>Select the Override link in the policy inheritance message to create a replacement policy.</p> +</li> +<li> +<p>On the replacement policy that is created, select the Add User icon (<span class="image"><img src="./images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User1 in the User Identity field and select OK. Select the Add User icon again, find or enter User2 and select OK.</p> +</li> +</ol> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/processor-replacement-modify-policy.png" alt="Processor Replacement Modify Policy"></span></p> +</div> +<div class="paragraph"> +<p>With these changes, User1 maintains the ability to move both processors on the canvas. User2 can now move the GenerateFlowFile processor but cannot move the LogAttribute processor.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-moved-processor.png" alt="User2 Moved Processor"></span></p> +</div> +</div> +<div class="sect4"> +<h5 id="editing-a-processor"><a class="anchor" href="#editing-a-processor"></a>Editing a Processor</h5> +<div class="paragraph"> +<p>In the âMoving a Processorâ example above, User2 was added to the âmodify the componentâ policy for GenerateFlowFile. Without the ability to view the processor properties, User2 is unable to modify the processorâs configuration. In order to edit a component, a user must be on both the âview the componentâ and âmodify the componentâ policies. To implement this, User1 performs the following steps:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Select the GenerateFlowFile processor.</p> +</li> +<li> +<p>Select the Access Policies icon (<span class="image"><img src="./images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette and the Access Policies dialog opens.</p> +</li> +<li> +<p>Select "view the componentâ from the policy drop-down. +<span class="image"><img src="./images/processor-view-policy.png" alt="Processor View Policy"></span> +The view the componentâ policy that currently exists on the processor (child) is the "view the componentâ policy inherited from the root process group (parent) on which User1 has privileges.</p> +</li> +<li> +<p>Select the Override link in the policy inheritance message to create a replacement policy.</p> +</li> +<li> +<p>On the replacement policy that is created, select the Add User icon (<span class="image"><img src="./images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User1 in the User Identity field and select OK. Select the Add User icon again, find or enter User2 and select OK.</p> +</li> +</ol> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/processor-replacement-view-policy.png" alt="Processor Replacement View Policy"></span></p> +</div> +<div class="paragraph"> +<p>With these changes, User1 maintains the ability to view and edit the processors on the canvas. User2 can now view and edit the GenerateFlowFile processor.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-edit-processor.png" alt="User2 Edit Processor"></span></p> +</div> +</div> +<div class="sect4"> +<h5 id="creating-a-connection"><a class="anchor" href="#creating-a-connection"></a>Creating a Connection</h5> +<div class="paragraph"> +<p>With the access policies configured as discussed in the previous two examples, User1 is able to connect GenerateFlowFile to LogAttribute:</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user1-create-connection.png" alt="User1 Create Connection"></span></p> +</div> +<div class="paragraph"> +<p>User2 cannot make the connection:</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-no-connection.png" alt="User2 No Connection"></span></p> +</div> +<div class="paragraph"> +<p>This is because:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>User2 does not have modify access on the process group and is therefore not able to create a connection.</p> +</li> +<li> +<p>Even though User2 has view and modify access to the source component (GenerateFlowFile), User2 does not have any access policy on the destination component (LogAttribute).</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>To allow User2 to connect GenerateFlowFile to LogAttribute, as User1:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Select the root process group. The Operate palette is updated with details for the root process group.</p> +</li> +<li> +<p>Select the Access Policies icon (<span class="image"><img src="./images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette and the Access Policies dialog opens.</p> +</li> +<li> +<p>Select "modify the componentâ from the policy drop-down. +<span class="image"><img src="./images/process-group-modify-policy.png" alt="Process Group Modify Policy"></span> +[start=4]</p> +</li> +<li> +<p>Select the Add User icon (<span class="image"><img src="./images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User2 and select OK.</p> +</li> +</ol> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/process-group-modify-policy-add-user2.png" alt="Process Group Modify Policy Add User2"></span></p> +</div> +<div class="paragraph"> +<p>By adding User2 to the âmodify the componentâ policy on the process group, User2 is added to the âmodify the componentâ policy on the LogAttribute processor by policy inheritance. To confirm this, highlight the LogAttribute processor and select the Access Policies icon (<span class="image"><img src="./images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette:</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/processor-inherited-modify-policy.png" alt="User2 Inherited Edit Processor"></span></p> +</div> +<div class="paragraph"> +<p>With these changes, User2 can now connect the GenerateFlowFile processor to the LogAttribute processor.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-can-connect.png" alt="User2 Can Connect"></span></p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-connected-processors.png" alt="User2 Connected Processors"></span></p> +</div> +</div> +<div class="sect4"> +<h5 id="editing-a-connection"><a class="anchor" href="#editing-a-connection"></a>Editing a Connection</h5> +<div class="paragraph"> +<p>Assume User1 or User2 adds a ReplaceText processor to the root process group:</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/replacetext-processor-added.png" alt="ReplaceText Processor Added"></span></p> +</div> +<div class="paragraph"> +<p>User1 can select and change the existing connection (between GenerateFlowFile to LogAttribute) to now connect GenerateFlowFile to ReplaceText:</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user1-edit-connection.png" alt="User1 Edit Connection"></span></p> +</div> +<div class="paragraph"> +<p>User 2 is unable to perform this action.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-no-edit-connection.png" alt="User2 No Edit Connection"></span></p> +</div> +<div class="paragraph"> +<p>To allow User2 to connect GenerateFlowFile to ReplaceText, as User1:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Select the root process group. The Operate palette is updated with details for the root process group.</p> +</li> +<li> +<p>Select the Access Policies icon (<span class="image"><img src="./images/iconAccessPolicies.png" alt="Access Policies Icon"></span>).</p> +</li> +<li> +<p>Select "view the componentâ from the policy drop-down. +<span class="image"><img src="./images/process-group-view-policy.png" alt="Process Group View Policy"></span> +[start=4]</p> +</li> +<li> +<p>Select the Add User icon (<span class="image"><img src="./images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User2 and select OK.</p> +</li> +</ol> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/process-group-view-policy-add-user2.png" alt="Process Group View Policy Add User2"></span></p> +</div> +<div class="paragraph"> +<p>Being added to both the view and modify policies for the process group, User2 can now connect the GenerateFlowFile processor to the ReplaceText processor.</p> +</div> +<div class="paragraph"> +<p><span class="image"><img src="./images/user2-edit-connection.png" alt="User2 Edit Connection"></span></p> +</div> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="encryption"><a class="anchor" href="#encryption"></a>Encryption Configuration</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>This section provides an overview of the capabilities of NiFi to encrypt and decrypt data.</p> +</div> +<div class="paragraph"> +<p>The <code>EncryptContent</code> processor allows for the encryption and decryption of data, both internal to NiFi and integrated with external systems, such as <code>openssl</code> and other data sources and consumers.</p> +</div> +<div class="sect2"> +<h3 id="key-derivation-functions"><a class="anchor" href="#key-derivation-functions"></a>Key Derivation Functions</h3> +<div class="paragraph"> +<p>Key Derivation Functions (KDF) are mechanisms by which human-readable information, usually a password or other secret information, is translated into a cryptographic key suitable for data protection. For further information, read <a href="https://en.wikipedia.org/wiki/Key_derivation_function";>the Wikipedia entry on Key Derivation Functions</a>. +Currently, KDFs are ingested by <code>CipherProvider</code> implementations and return a fully-initialized <code>Cipher</code> object to be used for encryption or decryption. Due to the use of a <code>CipherProviderFactory</code>, the KDFs are not customizable at this time. Future enhancements will include the ability to provide custom cost parameters to the KDF at initialization time. As a work-around, <code>CipherProvider</code> instances can be initialized with custom cost parameters in the constructor but this is not currently supported by the <code>CipherProviderFactory</code>. +Here are the KDFs currently supported by NiFi (primarily in the <code>EncryptContent</code> processor for password-based encryption (PBE)) and relevant notes:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>NiFi Legacy KDF</p> +<div class="ulist"> +<ul> +<li> +<p>The original KDF used by NiFi for internal key derivation for PBE, this is 1000 iterations of the MD5 digest over the concatenation of the password and 8 or 16 bytes of random salt (the salt length depends on the selected cipher block size).</p> +</li> +<li> +<p>This KDF is <strong>deprecated as of NiFi 0.5.0</strong> and should only be used for backwards compatibility to decrypt data that was previously encrypted by a legacy version of NiFi.</p> +</li> +</ul> +</div> +</li> +<li> +<p>OpenSSL PKCS#5 v1.5 EVP_BytesToKey</p> +<div class="ulist"> +<ul> +<li> +<p>This KDF was added in v0.4.0.</p> +</li> +<li> +<p>This KDF is provided for compatibility with data encrypted using OpenSSL’s default PBE, known as <code>EVP_BytesToKey</code>. This is a single iteration of MD5 over the concatenation of the password and 8 bytes of random ASCII salt. OpenSSL recommends using <code>PBKDF2</code> for key derivation but does not expose the library method necessary to the command-line tool, so this KDF is still the de facto default for command-line encryption.</p> +</li> +</ul> +</div> +</li> +<li> <p>Bcrypt</p> <div class="ulist"> <ul> @@ -1618,33 +2443,202 @@ Here are the KDFs currently supported by <div class="ulist"> <ul> <li> -<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html";>JCE Unlimited Strength Jurisdiction Policy files for Java 7</a></p> +<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html";>JCE Unlimited Strength Jurisdiction Policy files for Java 8</a></p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>If on a system where the unlimited strength policies cannot be installed, it is recommended to switch to an algorithm that supports longer passwords (see table above).</p> +</div> +<div class="admonitionblock warning"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-warning" title="Warning"></i> +</td> +<td class="content"> +<div class="title">Allowing Weak Crypto</div> +<div class="paragraph"> +<p>If it is not possible to install the unlimited strength jurisdiction policies, the <code>Allow Weak Crypto</code> setting can be changed to <code>allowed</code>, but <strong>this is <em>not</em> recommended</strong>. Changing this setting explicitly acknowledges the inherent risk in using weak cryptographic configurations.</p> +</div> +</td> +</tr> +</table> +</div> +<div class="paragraph"> +<p>It is preferable to request upstream/downstream systems to switch to <a href="https://cwiki.apache.org/confluence/display/NIFI/Encryption+Information";>keyed encryption</a> or use a "strong" <a href="https://cwiki.apache.org/confluence/display/NIFI/Key+Derivation+Function+Explanations";>Key Derivation Function (KDF) supported by NiFi</a>.</p> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="encrypted-passwords-in-configuration-files"><a class="anchor" href="#encrypted-passwords-in-configuration-files"></a>Encrypted Passwords in Configuration Files</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>In order to facilitate the secure setup of NiFi, you can use the <code>encrypt-config</code> command line utility to encrypt raw configuration values that NiFi decrypts in memory on startup. This extensible protection scheme transparently allows NiFi to use raw values in operation, while protecting them at rest. In the future, hardware security modules (HSM) and external secure storage mechanisms will be integrated, but for now, an AES encryption provider is the default implementation.</p> +</div> +<div class="paragraph"> +<p>This is a change in behavior; prior to 1.0, all configuration values were stored in plaintext on the file system. POSIX file permissions were recommended to limit unauthorized access to these files</p> +</div> +<div class="paragraph"> +<p>If no administrator action is taken, the configuration values remain unencrypted.</p> +</div> +<div class="sect2"> +<h3 id="encrypt-config_tool"><a class="anchor" href="#encrypt-config_tool"></a>Encrypt-Config Tool</h3> +<div class="paragraph"> +<p>The <code>encrypt-config</code> command line tool (invoked as <code>./bin/encrypt-config.sh</code> or <code>bin\encrypt-config.bat</code>) reads from a <em>nifi.properties</em> file with plaintext sensitive configuration values, prompts for a master password or raw hexadecimal key, and encrypts each value. It replaces the plain values with the protected value in the same file, or writes to a new <em>nifi.properties</em> file if specified.</p> +</div> +<div class="paragraph"> +<p>The default encryption algorithm utilized is AES/GCM 128/256-bit. 128-bit is used if the JCE Unlimited Strength Cryptographic Jurisdiction Policy files are not installed, and 256-bit is used if they are installed.</p> +</div> +<div class="paragraph"> +<p>You can use the following command line options with the <code>encrypt-config</code> tool:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-b,--bootstrapConf <arg></code> The bootstrap.conf file to persist master key</p> +</li> +<li> +<p><code>-h,--help</code> Prints this usage message</p> +</li> +<li> +<p><code>-k,--key <arg></code> The raw hexadecimal key to use to encrypt the sensitive properties (the key can be entered with spaces or <em>-</em> delimiters to assist manual entry — these will be ignored)</p> +</li> +<li> +<p><code>-n,--niFiProperties <arg></code> The <em>nifi.properties</em> file containing unprotected config values (will be overwritten by default unless <code>-o</code> is provided)</p> +</li> +<li> +<p><code>-o,--outputNiFiProperties <arg></code> The destination <em>nifi.properties</em> file containing protected config values (will not modify input <em>nifi.properties</em>)</p> +</li> +<li> +<p><code>-p,--password <arg></code> The password from which to derive the key to use to encrypt the sensitive properties</p> +</li> +<li> +<p><code>-r,--useRawKey</code> If provided, the secure console will prompt for the raw key value in hexadecimal form</p> +</li> +<li> +<p><code>-v,--verbose</code> Sets verbose mode (default false)</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>As an example of how the tool works, assume that you have installed the tool on a machine supporting 256-bit encryption and with the following existing values in the <em>nifi.properties</em> file:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre># security properties # +nifi.sensitive.props.key=thisIsABadSensitiveKeyPassword +nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL +nifi.sensitive.props.provider=BC +nifi.sensitive.props.additional.keys= + +nifi.security.keystore=/path/to/keystore.jks +nifi.security.keystoreType=JKS +nifi.security.keystorePasswd=thisIsABadKeystorePassword +nifi.security.keyPasswd=thisIsABadKeyPassword +nifi.security.truststore= +nifi.security.truststoreType= +nifi.security.truststorePasswd=</pre> +</div> +</div> +<div class="paragraph"> +<p>Enter the following arguments when using the tool:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>encrypt-config.sh +-b bootstrap.conf +-k 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 +-n nifi.properties</pre> +</div> +</div> +<div class="paragraph"> +<p>As a result, the <em>nifi.properties</em> file is overwritten with protected properties and sibling encryption identifiers (<code>aes/gcm/256</code>, the currently supported algorithm):</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre># security properties # +nifi.sensitive.props.key=n2z+tTTbHuZ4V4V2||uWhdasyDXD4ZG2lMAes/vqh6u4vaz4xgL4aEbF4Y/dXevqk3ulRcOwf1vc4RDQ== +nifi.sensitive.props.key.protected=aes/gcm/256 +nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL +nifi.sensitive.props.provider=BC +nifi.sensitive.props.additional.keys= + +nifi.security.keystore=/path/to/keystore.jks +nifi.security.keystoreType=JKS +nifi.security.keystorePasswd=oBjT92hIGRElIGOh||MZ6uYuWNBrOA6usq/Jt3DaD2e4otNirZDytac/w/KFe0HOkrJR03vcbo +nifi.security.keystorePasswd.protected=aes/gcm/256 +nifi.security.keyPasswd=ac/BaE35SL/esLiJ||+ULRvRLYdIDA2VqpE0eQXDEMjaLBMG2kbKOdOwBk/hGebDKlVg== +nifi.security.keyPasswd.protected=aes/gcm/256 +nifi.security.truststore= +nifi.security.truststoreType= +nifi.security.truststorePasswd=</pre> +</div> +</div> +<div class="paragraph"> +<p>Additionally, the <em>bootstrap.conf</em> file is updated with the encryption key as follows:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre># Master key in hexadecimal format for encrypted sensitive configuration values +nifi.bootstrap.sensitive.key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210</pre> +</div> +</div> +<div class="paragraph"> +<p>Sensitive configuration values are encrypted by the tool by default, however you can encrypt any additional properties, if desired. To encrypt additional properties, specify them as comma-separated values in the <code>nifi.sensitive.props.additional.keys</code> property.</p> +</div> +<div class="paragraph"> +<p>If the <em>nifi.properties</em> file already has valid protected values, those property values are not modified by the tool.</p> +</div> +</div> +<div class="sect2"> +<h3 id="encrypt-config_password"><a class="anchor" href="#encrypt-config_password"></a>Password Key Derivation</h3> +<div class="paragraph"> +<p>Instead of providing a 32 or 64 character raw hexadecimal key, you can provide a password from which the key will be derived. As of 1.0.0, the password must be at least 12 characters, and the key will be derived using <code>SCrypt</code> with the parameters:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>pw</code> — the password bytes in <code>UTF-8</code></p> +</li> +<li> +<p><code>salt</code> — the fixed salt value (<code>NIFI_SCRYPT_SALT</code>) bytes in <code>UTF-8</code></p> +</li> +<li> +<p><code>N</code> — 2<sup>16</sup></p> +</li> +<li> +<p><code>r</code> — 8</p> </li> <li> -<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html";>JCE Unlimited Strength Jurisdiction Policy files for Java 8</a></p> +<p><code>p</code> — 1</p> +</li> +<li> +<p><code>dkLen</code> — determined by the JCE policies available</p> </li> </ul> </div> <div class="paragraph"> -<p>If on a system where the unlimited strength policies cannot be installed, it is recommended to switch to an algorithm that supports longer passwords (see table above).</p> +<p>As of August 2016, these values are determined to be strong for this threat model but may change in future versions.</p> </div> -<div class="admonitionblock warning"> +<div class="admonitionblock note"> <table> <tr> <td class="icon"> -<i class="fa icon-warning" title="Warning"></i> +<i class="fa icon-note" title="Note"></i> </td> <td class="content"> -<div class="title">Allowing Weak Crypto</div> -<div class="paragraph"> -<p>If it is not possible to install the unlimited strength jurisdiction policies, the <code>Allow Weak Crypto</code> setting can be changed to <code>allowed</code>, but <strong>this is <em>not</em> recommended</strong>. Changing this setting explicitly acknowledges the inherent risk in using weak cryptographic configurations.</p> -</div> +While fixed salts are counter to best practices, a static salt is necessary for deterministic key derivation without additional storage of the salt value. </td> </tr> </table> </div> +</div> +<div class="sect2"> +<h3 id="encrypt-config_secure_prompt"><a class="anchor" href="#encrypt-config_secure_prompt"></a>Secure Prompt</h3> <div class="paragraph"> -<p>It is preferable to request upstream/downstream systems to switch to <a href="https://cwiki.apache.org/confluence/display/NIFI/Encryption+Information";>keyed encryption</a> or use a "strong" <a href="https://cwiki.apache.org/confluence/display/NIFI/Key+Derivation+Function+Explanations";>Key Derivation Function (KDF) supported by NiFi</a>.</p> +<p>If you prefer not to provide the password or raw key in the command-line invocation of the tool, leaving these arguments absent will prompt a secure console read of the password (by default) or raw key (if the <code>-r</code> flag is provided at invocation).</p> </div> </div> </div> @@ -1653,20 +2647,37 @@ Here are the KDFs currently supported by <h2 id="clustering"><a class="anchor" href="#clustering"></a>Clustering Configuration</h2> <div class="sectionbody"> <div class="paragraph"> -<p>This section provides a quick overview of NiFi Clustering and instructions on how to set up a basic cluster. In the future, we hope to provide supplemental documentation that covers the NiFi Cluster Architecture in depth.</p> +<p>This section provides a quick overview of NiFi Clustering and instructions on how to set up a basic cluster. +In the future, we hope to provide supplemental documentation that covers the NiFi Cluster Architecture in depth.</p> +</div> +<div class="imageblock"> +<div class="content"> +<img src="./images/zero-master-cluster-http-access.png" alt="NiFi Cluster HTTP Access"> +</div> </div> <div class="paragraph"> -<p>The design of NiFi clustering is a simple master/slave model where there is a master and one or more slaves. -While the model is that of master and slave, if the master dies, the slaves are all instructed to continue operating -as they were to ensure the dataflow remains live. The absence of the master simply means new slaves cannot join the -cluster and cluster flow changes cannot occur until the master is restored. In NiFi clustering, we call the master -the NiFi Cluster Manager (NCM), and the slaves are called Nodes. See a full description of each in the Terminology section below.</p> +<p>NiFi employs a Zero-Master Clustering paradigm. Each node in the cluster performs the same tasks on +the data, but each operates on a different set of data. One of the nodes is automatically elected (via Apache +ZooKeeper) as the Cluster Coordinator. All nodes in the cluster will then send heartbeat/status information +to this node, and this node is responsible for disconnecting nodes that do not report any heartbeat status +for some amount of time. Additionally, when a new node elects to join the cluster, the new node must first +connect to the currently-elected Cluster Coordinator in order to obtain the most up-to-date flow. If the Cluster +Coordinator determines that the node is allowed to join (based on its configured Firewall file), the current +flow is provided to that node, and that node is able to join the cluster, assuming that the node’s copy of the +flow matches the copy provided by the Cluster Coordinator. If the node’s version of the flow configuration differs +from that of the Cluster Coordinator’s, the node will not join the cluster.</p> </div> <div class="paragraph"> <p><strong>Why Cluster?</strong><br></p> </div> <div class="paragraph">
[... 978 lines stripped ...]
