Author: danbress Date: Fri Aug 14 01:50:54 2015 New Revision: 1695810 URL: http://svn.apache.org/r1695810 Log: NIFI-847 - Generate AsciiDoc for 0.2.1 and push to website
Added: nifi/site/trunk/docs/nifi-docs/html/images/controller-services-edit-buttons2.png (with props) nifi/site/trunk/docs/nifi-docs/html/images/iconConnection.png (with props) nifi/site/trunk/docs/nifi-docs/html/images/nifi_first_launch_screenshot.png (with props) nifi/site/trunk/docs/nifi-docs/html/images/provenance-table.png (with props) nifi/site/trunk/docs/nifi-docs/html/images/reporting-tasks-edit-buttons.png (with props) Modified: nifi/site/trunk/docs/nifi-docs/html/administration-guide.html nifi/site/trunk/docs/nifi-docs/html/developer-guide.html nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html nifi/site/trunk/docs/nifi-docs/html/overview.html nifi/site/trunk/docs/nifi-docs/html/user-guide.html Modified: nifi/site/trunk/docs/nifi-docs/html/administration-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/administration-guide.html?rev=1695810&r1=1695809&r2=1695810&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/administration-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/administration-guide.html Fri Aug 14 01:50:54 2015 @@ -450,27 +450,27 @@ body.book #toc,body.book #preamble,body. <h1>NiFi System Administrator’s Guide</h1> <div class="details"> <span id="author" class="author">Apache NiFi Team</span><br> -<span id="email" class="email"><a href="mailto:d...@nifi.incubator.apache.org">d...@nifi.incubator.apache.org</a></span><br> +<span id="email" class="email"><a href="mailto:d...@nifi.apache.org">d...@nifi.apache.org</a></span><br> </div> <div id="toc" class="toc"> <div id="toctitle">Table of Contents</div> <ul class="sectlevel1"> -<li><a href="administration-guide.html#system-requirements">System Requirements</a></li> -<li><a href="administration-guide.html#how-to-install-and-start-nifi">How to install and start NiFi</a></li> -<li><a href="administration-guide.html#configuration-best-practices">Configuration Best Practices</a></li> -<li><a href="administration-guide.html#security-configuration">Security Configuration</a></li> -<li><a href="administration-guide.html#controlling-levels-of-access">Controlling Levels of Access</a></li> -<li><a href="administration-guide.html#clustering">Clustering Configuration</a></li> -<li><a href="administration-guide.html#system_properties">System Properties</a></li> +<li><a href="#system-requirements">System Requirements</a></li> +<li><a href="#how-to-install-and-start-nifi">How to install and start NiFi</a></li> +<li><a href="#configuration-best-practices">Configuration Best Practices</a></li> +<li><a href="#security-configuration">Security Configuration</a></li> +<li><a href="#controlling-levels-of-access">Controlling Levels of Access</a></li> +<li><a href="#clustering">Clustering Configuration</a></li> +<li><a href="#system_properties">System Properties</a></li> </ul> </div> </div> <div id="content"> <div class="sect1"> -<h2 id="system-requirements"><a class="anchor" href="administration-guide.html#system-requirements"></a>System Requirements</h2> +<h2 id="system-requirements"><a class="anchor" href="#system-requirements"></a>System Requirements</h2> <div class="sectionbody"> <div class="paragraph"> -<p>Apache NiFi can run on something as simple as a laptop, but it can also be clustered across many enterprise-class servers. Therefore, the amount of hardware and memory needed will depend on the size and nature of the dataflow involved. The data is stored on disk while NiFi is processing it. So NiFi needs to have sufficient disk space allocated for its various repositories, particularly the content repository, flowfile repository, and provenance repository (see the <a href="administration-guide.html#system_properties">System Properties</a> section for more information about these repositories). NiFi has the following minimum system requirements:</p> +<p>Apache NiFi can run on something as simple as a laptop, but it can also be clustered across many enterprise-class servers. Therefore, the amount of hardware and memory needed will depend on the size and nature of the dataflow involved. The data is stored on disk while NiFi is processing it. So NiFi needs to have sufficient disk space allocated for its various repositories, particularly the content repository, flowfile repository, and provenance repository (see the <a href="#system_properties">System Properties</a> section for more information about these repositories). NiFi has the following minimum system requirements:</p> </div> <div class="ulist"> <ul> @@ -523,7 +523,7 @@ body.book #toc,body.book #preamble,body. </div> </div> <div class="sect1"> -<h2 id="how-to-install-and-start-nifi"><a class="anchor" href="administration-guide.html#how-to-install-and-start-nifi"></a>How to install and start NiFi</h2> +<h2 id="how-to-install-and-start-nifi"><a class="anchor" href="#how-to-install-and-start-nifi"></a>How to install and start NiFi</h2> <div class="sectionbody"> <div class="ulist"> <ul> @@ -539,7 +539,7 @@ body.book #toc,body.book #preamble,body. <div class="ulist"> <ul> <li> -<p>At a minimum, we recommend editing the <em>nifi.properties</em> file and entering a password for the nifi.sensitive.props.key (see <a href="administration-guide.html#system_properties">System Properties</a> below)</p> +<p>At a minimum, we recommend editing the <em>nifi.properties</em> file and entering a password for the nifi.sensitive.props.key (see <a href="#system_properties">System Properties</a> below)</p> </li> </ul> </div> @@ -594,7 +594,7 @@ body.book #toc,body.book #preamble,body. <div class="ulist"> <ul> <li> -<p>At a minimum, we recommend editing the <em>nifi.properties</em> file and entering a password for the nifi.sensitive.props.key (see <a href="administration-guide.html#system_properties">System Properties</a> below)</p> +<p>At a minimum, we recommend editing the <em>nifi.properties</em> file and entering a password for the nifi.sensitive.props.key (see <a href="#system_properties">System Properties</a> below)</p> </li> </ul> </div> @@ -648,12 +648,12 @@ body.book #toc,body.book #preamble,body. </ul> </div> <div class="paragraph"> -<p>See the <a href="administration-guide.html#system_properties">System Properties</a> section of this guide for more information about configuring NiFi repositories and configuration files.</p> +<p>See the <a href="#system_properties">System Properties</a> section of this guide for more information about configuring NiFi repositories and configuration files.</p> </div> </div> </div> <div class="sect1"> -<h2 id="configuration-best-practices"><a class="anchor" href="administration-guide.html#configuration-best-practices"></a>Configuration Best Practices</h2> +<h2 id="configuration-best-practices"><a class="anchor" href="#configuration-best-practices"></a>Configuration Best Practices</h2> <div class="sectionbody"> <div class="admonitionblock note"> <table> @@ -758,7 +758,7 @@ and for the partition(s) of interest add </div> </div> <div class="sect1"> -<h2 id="security-configuration"><a class="anchor" href="administration-guide.html#security-configuration"></a>Security Configuration</h2> +<h2 id="security-configuration"><a class="anchor" href="#security-configuration"></a>Security Configuration</h2> <div class="sectionbody"> <div class="paragraph"> <p>NiFi provides several different configuration options for security purposes. The most important properties are those under the @@ -841,7 +841,7 @@ accomplished by setting the <code>nifi.r </div> </div> <div class="sect1"> -<h2 id="controlling-levels-of-access"><a class="anchor" href="administration-guide.html#controlling-levels-of-access"></a>Controlling Levels of Access</h2> +<h2 id="controlling-levels-of-access"><a class="anchor" href="#controlling-levels-of-access"></a>Controlling Levels of Access</h2> <div class="sectionbody"> <div class="paragraph"> <p>Once NiFi is configured to run securely as discussed in the previous section, it is necessary @@ -909,7 +909,7 @@ Once the application starts, the ADMIN u able to access the UI at the HTTPS URL that is configured in the <em>nifi.properties</em> file.</p> </div> <div class="paragraph"> -<p>From the UI, click on the Users icon ( <span class="image"><img src="images/iconUsers.png" alt="Users" width="32"></span> ) in the +<p>From the UI, click on the Users icon ( <span class="image"><img src="./images/iconUsers.png" alt="Users" width="32"></span> ) in the Management Toolbar (upper-right corner of the UI), and the User Management Page opens.</p> </div> <div class="paragraph"> @@ -999,7 +999,7 @@ cluster, s/he can grant it to the group </div> </div> <div class="sect1"> -<h2 id="clustering"><a class="anchor" href="administration-guide.html#clustering"></a>Clustering Configuration</h2> +<h2 id="clustering"><a class="anchor" href="#clustering"></a>Clustering Configuration</h2> <div class="sectionbody"> <div class="paragraph"> <p>This section provides a quick overview of NiFi Clustering and instructions on how to set up a basic cluster. In the future, we hope to provide supplemental documentation that covers the NiFi Cluster Architecture in depth.</p> @@ -1030,19 +1030,19 @@ the NiFi Cluster Manager (NCM), and the <p><strong>Nodes</strong>: Each cluster is made up of the NCM and one or more nodes. The nodes do the actual data processing. (The NCM does not process any data; all data runs through the nodes.) While nodes are connected to a cluster, the DFM may not access the User Interface for any of the individual nodes. The User Interface of a node may only be accessed if the node is manually removed from the cluster.</p> </div> <div class="paragraph"> -<p><strong>Primary Node</strong>: Every cluster has one Primary Node. On this node, it is possible to run "Isolated Processors" (see below). By default, the NCM will elect the first node that connects to the cluster as the Primary Node; however, the DFM may select a new node as the Primary Node in the Cluster Management page of the User Interface if desired. If the cluster restarts, the NCM will "remember" which node was the Primary Node and wait for that node to re-connect before allowing the DFM to make any changes to the dataflow. The ADMIN may adjust how long the NCM waits for the Primary Node to reconnect by adjusting the property <em>nifi.cluster.manager.safemode.duration</em> in the <em>nifi.properties</em> file, which is discussed in the <a href="administration-guide.html#system_properties">System Properties</a> section of this document.</p> +<p><strong>Primary Node</strong>: Every cluster has one Primary Node. On this node, it is possible to run "Isolated Processors" (see below). By default, the NCM will elect the first node that connects to the cluster as the Primary Node; however, the DFM may select a new node as the Primary Node in the Cluster Management page of the User Interface if desired. If the cluster restarts, the NCM will "remember" which node was the Primary Node and wait for that node to re-connect before allowing the DFM to make any changes to the dataflow. The ADMIN may adjust how long the NCM waits for the Primary Node to reconnect by adjusting the property <em>nifi.cluster.manager.safemode.duration</em> in the <em>nifi.properties</em> file, which is discussed in the <a href="#system_properties">System Properties</a> section of this document.</p> </div> <div class="paragraph"> <p><strong>Isolated Processors</strong>: In a NiFi cluster, the same dataflow runs on all the nodes. As a result, every component in the flow runs on every node. However, there may be cases when the DFM would not want every processor to run on every node. The most common case is when using a processor that communicates with an external service using a protocol that does not scale well. For example, the GetSFTP processor pulls from a remote directory, and if the GetSFTP on every node in the cluster tries simultaneously to pull from the same remote directory, there could be race conditions. Therefore, the DFM could configure the GetSFTP on the Primary Node to run in isolation, meaning that it only runs on that node. It could pull in data and -with the proper dataflow configuration- load-balance it across the rest of the nodes in the cluster. Note that while this feature exists, it is also very common to simply use a standalone NiFi instance to pull data and feed it to the cluster. It just depends on the resources available and how the Administrator decides to configure the cluster.</p> </div> <div class="paragraph"> -<p><strong>Heartbeats</strong>: The nodes communicate their health and status to the NCM via "heartbeats", which let the NCM know they are still connected to the cluster and working properly. By default, the nodes emit heartbeats to the NCM every 5 seconds, and if the NCM does not receive a heartbeat from a node within 45 seconds, it disconnects the node due to "lack of heartbeat". (The 5-second and 45-second settings are configurable in the <em>nifi.properties</em> file. See the <a href="administration-guide.html#system_properties">System Properties</a> section of this document for more information.) The reason that the NCM disconnects the node is because the NCM needs to ensure that every node in the cluster is in sync, and if a node is not heard from regularly, the NCM cannot be sure it is still in sync with the rest of the cluster. If, after 45 seconds, the node does send a new heartbeat, the NCM will automatically reconnect the node to the cluster. Both the disconnection due to lack of heartbeat and the reconnection once a heartbeat is received are reported to the DFM in the NCM’s User Interface.</p> +<p><strong>Heartbeats</strong>: The nodes communicate their health and status to the NCM via "heartbeats", which let the NCM know they are still connected to the cluster and working properly. By default, the nodes emit heartbeats to the NCM every 5 seconds, and if the NCM does not receive a heartbeat from a node within 45 seconds, it disconnects the node due to "lack of heartbeat". (The 5-second and 45-second settings are configurable in the <em>nifi.properties</em> file. See the <a href="#system_properties">System Properties</a> section of this document for more information.) The reason that the NCM disconnects the node is because the NCM needs to ensure that every node in the cluster is in sync, and if a node is not heard from regularly, the NCM cannot be sure it is still in sync with the rest of the cluster. If, after 45 seconds, the node does send a new heartbeat, the NCM will automatically reconnect the node to the cluster. Both the disconnection due to lack of heartbeat and th e reconnection once a heartbeat is received are reported to the DFM in the NCM’s User Interface.</p> </div> <div class="paragraph"> <p><strong>Communication within the Cluster</strong><br></p> </div> <div class="paragraph"> -<p>As noted, the nodes communicate with the NCM via heartbeats. The communication that allows the nodes to find the NCM may be set up as multicast or unicast; this is configured in the <em>nifi.properties</em> file (See <a href="administration-guide.html#system_properties">System Properties</a> ). By default, unicast is used. It is important to note that the nodes in a NiFi cluster are not aware of each other. They only communicate with the NCM. Therefore, if one of the nodes goes down, the other nodes in the cluster will not automatically pick up the load of the missing node. It is possible for the DFM to configure the dataflow for failover contingencies; however, this is dependent on the dataflow design and does not happen automatically.</p> +<p>As noted, the nodes communicate with the NCM via heartbeats. The communication that allows the nodes to find the NCM may be set up as multicast or unicast; this is configured in the <em>nifi.properties</em> file (See <a href="#system_properties">System Properties</a> ). By default, unicast is used. It is important to note that the nodes in a NiFi cluster are not aware of each other. They only communicate with the NCM. Therefore, if one of the nodes goes down, the other nodes in the cluster will not automatically pick up the load of the missing node. It is possible for the DFM to configure the dataflow for failover contingencies; however, this is dependent on the dataflow design and does not happen automatically.</p> </div> <div class="paragraph"> <p>When the DFM makes changes to the dataflow, the NCM communicates those changes to the nodes and waits for each node to respond, indicating that it has made the change on its local flow. If the DFM wants to make another change, the NCM will only allow this to happen once all the nodes have acknowledged that they’ve implemented the last change. This is a safeguard to ensure that all the nodes in the cluster have the correct and up-to-date flow.</p> @@ -1076,7 +1076,7 @@ the NiFi Cluster Manager (NCM), and the <p>Administrators may install each instance on a separate server; however, it is also perfectly fine to install the NCM and one of the nodes on the same server, as the NCM is very lightweight. Just keep in mind that the ports assigned to each instance must not collide if the NCM and one of the nodes share the same server.</p> </div> <div class="paragraph"> -<p>For each instance, certain properties in the <em>nifi.properties</em> file will need to be updated. In particular, the Web and Clustering properties should be evaluated for your situation and adjusted accordingly. All the properties are described in the <a href="administration-guide.html#system_properties">System Properties</a> section of this guide; however, in this section, we will focus on the minimum properties that must be set for a simple cluster.</p> +<p>For each instance, certain properties in the <em>nifi.properties</em> file will need to be updated. In particular, the Web and Clustering properties should be evaluated for your situation and adjusted accordingly. All the properties are described in the <a href="#system_properties">System Properties</a> section of this guide; however, in this section, we will focus on the minimum properties that must be set for a simple cluster.</p> </div> <div class="paragraph"> <p>For all three instances, the Cluster Common Properties can be left with the default settings. Note, however, that if you change these settings, they must be set the same on every instance in the cluster (NCM and nodes).</p> @@ -1172,7 +1172,7 @@ the NiFi Cluster Manager (NCM), and the <p>Now, it is possible to start up the cluster. Technically, it does not matter which instance starts up first. However, you could start the NCM first, then Node 1 and then Node 2. Since the first node that connects is automatically elected as the Primary Node, this sequence should create a cluster where Node 1 is the Primary Node. Navigate to the URL for the NCM in your web browser, and the User Interface should look similar to the following:</p> </div> <div class="paragraph"> -<p><span class="image"><img src="images/ncm.png" alt="NCM User Interface" width="940"></span></p> +<p><span class="image"><img src="./images/ncm.png" alt="NCM User Interface" width="940"></span></p> </div> <div class="paragraph"> <p><strong>Troubleshooting</strong></p> @@ -1191,7 +1191,7 @@ additivity="false"> </div> </div> <div class="sect1"> -<h2 id="system_properties"><a class="anchor" href="administration-guide.html#system_properties"></a>System Properties</h2> +<h2 id="system_properties"><a class="anchor" href="#system_properties"></a>System Properties</h2> <div class="sectionbody"> <div class="paragraph"> <p>The <em>nifi.properties</em> file in the conf directory is the main configuration file for controlling how NiFi runs. This section provides an overview of the properties in this file and includes some notes on how to configure it in a way that will make upgrading easier. <strong>After making changes to this file, restart NiFi in order @@ -1955,7 +1955,7 @@ If multicast is used, the following nifi </div> <div id="footer"> <div id="footer-text"> -Last updated 2015-06-22 14:41:44 EDT +Last updated 2015-07-23 13:20:05 PDT </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/developer-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/developer-guide.html?rev=1695810&r1=1695809&r2=1695810&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/developer-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/developer-guide.html Fri Aug 14 01:50:54 2015 @@ -450,91 +450,91 @@ body.book #toc,body.book #preamble,body. <h1>NiFi Developer’s Guide</h1> <div class="details"> <span id="author" class="author">Apache NiFi Team</span><br> -<span id="email" class="email"><a href="mailto:d...@nifi.incubator.apache.org">d...@nifi.incubator.apache.org</a></span><br> +<span id="email" class="email"><a href="mailto:d...@nifi.apache.org">d...@nifi.apache.org</a></span><br> </div> <div id="toc" class="toc"> <div id="toctitle">Table of Contents</div> <ul class="sectlevel1"> -<li><a href="developer-guide.html#introduction">Introduction</a></li> -<li><a href="developer-guide.html#components">NiFi Components</a></li> -<li><a href="developer-guide.html#processor_api">Processor API</a> +<li><a href="#introduction">Introduction</a></li> +<li><a href="#components">NiFi Components</a></li> +<li><a href="#processor_api">Processor API</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#supporting_api">Supporting API</a></li> -<li><a href="developer-guide.html#AbstractProcessor">AbstractProcessor API</a></li> -<li><a href="developer-guide.html#component-lifecycle">Component Lifecycle</a></li> -<li><a href="developer-guide.html#reporting-processor-activity">Reporting Processor Activity</a></li> +<li><a href="#supporting_api">Supporting API</a></li> +<li><a href="#AbstractProcessor">AbstractProcessor API</a></li> +<li><a href="#component-lifecycle">Component Lifecycle</a></li> +<li><a href="#reporting-processor-activity">Reporting Processor Activity</a></li> </ul> </li> -<li><a href="developer-guide.html#documenting-a-component">Documenting a Component</a> +<li><a href="#documenting-a-component">Documenting a Component</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#documenting-properties">Documenting Properties</a></li> -<li><a href="developer-guide.html#documenting-relationships">Documenting Relationships</a></li> -<li><a href="developer-guide.html#documenting-capability-and-keywords">Documenting Capability and Keywords</a></li> -<li><a href="developer-guide.html#documenting-flowfile-attribute-interaction">Documenting FlowFile Attribute Interaction</a></li> -<li><a href="developer-guide.html#documenting-related-components">Documenting Related Components</a></li> -<li><a href="developer-guide.html#advanced-documentation">Advanced Documentation</a></li> +<li><a href="#documenting-properties">Documenting Properties</a></li> +<li><a href="#documenting-relationships">Documenting Relationships</a></li> +<li><a href="#documenting-capability-and-keywords">Documenting Capability and Keywords</a></li> +<li><a href="#documenting-flowfile-attribute-interaction">Documenting FlowFile Attribute Interaction</a></li> +<li><a href="#documenting-related-components">Documenting Related Components</a></li> +<li><a href="#advanced-documentation">Advanced Documentation</a></li> </ul> </li> -<li><a href="developer-guide.html#common-processor-patterns">Common Processor Patterns</a> +<li><a href="#common-processor-patterns">Common Processor Patterns</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#ingress">Data Ingress</a></li> -<li><a href="developer-guide.html#data-egress">Data Egress</a></li> -<li><a href="developer-guide.html#route-based-on-content-one-to-one">Route Based on Content (One-to-One)</a></li> -<li><a href="developer-guide.html#route-based-on-content-one-to-many">Route Based on Content (One-to-Many)</a></li> -<li><a href="developer-guide.html#route-streams-based-on-content-one-to-many">Route Streams Based on Content (One-to-Many)</a></li> -<li><a href="developer-guide.html#route-based-on-attributes">Route Based on Attributes</a></li> -<li><a href="developer-guide.html#split-content-one-to-many">Split Content (One-to-Many)</a></li> -<li><a href="developer-guide.html#update-attributes-based-on-content">Update Attributes Based on Content</a></li> -<li><a href="developer-guide.html#enrich-modify-content">Enrich/Modify Content</a></li> +<li><a href="#ingress">Data Ingress</a></li> +<li><a href="#data-egress">Data Egress</a></li> +<li><a href="#route-based-on-content-one-to-one">Route Based on Content (One-to-One)</a></li> +<li><a href="#route-based-on-content-one-to-many">Route Based on Content (One-to-Many)</a></li> +<li><a href="#route-streams-based-on-content-one-to-many">Route Streams Based on Content (One-to-Many)</a></li> +<li><a href="#route-based-on-attributes">Route Based on Attributes</a></li> +<li><a href="#split-content-one-to-many">Split Content (One-to-Many)</a></li> +<li><a href="#update-attributes-based-on-content">Update Attributes Based on Content</a></li> +<li><a href="#enrich-modify-content">Enrich/Modify Content</a></li> </ul> </li> -<li><a href="developer-guide.html#error-handling">Error Handling</a> +<li><a href="#error-handling">Error Handling</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#exceptions-within-the-processor">Exceptions within the Processor</a></li> -<li><a href="developer-guide.html#exceptions-within-a-callback-ioexception-runtimeexception">Exceptions within a callback: IOException, RuntimeException</a></li> -<li><a href="developer-guide.html#penalization-vs-yielding">Penalization vs. Yielding</a></li> -<li><a href="developer-guide.html#session-rollback">Session Rollback</a></li> +<li><a href="#exceptions-within-the-processor">Exceptions within the Processor</a></li> +<li><a href="#exceptions-within-a-callback-ioexception-runtimeexception">Exceptions within a callback: IOException, RuntimeException</a></li> +<li><a href="#penalization-vs-yielding">Penalization vs. Yielding</a></li> +<li><a href="#session-rollback">Session Rollback</a></li> </ul> </li> -<li><a href="developer-guide.html#general-design-considerations">General Design Considerations</a> +<li><a href="#general-design-considerations">General Design Considerations</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#consider-the-user">Consider the User</a></li> -<li><a href="developer-guide.html#cohesion-and-reusability">Cohesion and Reusability</a></li> -<li><a href="developer-guide.html#naming-convensions">Naming Conventions</a></li> -<li><a href="developer-guide.html#processor-behavior-annotations">Processor Behavior Annotations</a></li> -<li><a href="developer-guide.html#data-buffering">Data Buffering</a></li> +<li><a href="#consider-the-user">Consider the User</a></li> +<li><a href="#cohesion-and-reusability">Cohesion and Reusability</a></li> +<li><a href="#naming-convensions">Naming Conventions</a></li> +<li><a href="#processor-behavior-annotations">Processor Behavior Annotations</a></li> +<li><a href="#data-buffering">Data Buffering</a></li> </ul> </li> -<li><a href="developer-guide.html#controller-services">Controller Services</a> +<li><a href="#controller-services">Controller Services</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#developing-controller-service">Developing a ControllerService</a></li> -<li><a href="developer-guide.html#interacting-with-controller-service">Interacting with a ControllerService</a></li> +<li><a href="#developing-controller-service">Developing a ControllerService</a></li> +<li><a href="#interacting-with-controller-service">Interacting with a ControllerService</a></li> </ul> </li> -<li><a href="developer-guide.html#reporting-tasks">Reporting Tasks</a> +<li><a href="#reporting-tasks">Reporting Tasks</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#developing-a-reporting-task">Developing a Reporting Task</a></li> +<li><a href="#developing-a-reporting-task">Developing a Reporting Task</a></li> </ul> </li> -<li><a href="developer-guide.html#testing">Testing</a> +<li><a href="#testing">Testing</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#instantiate-testrunner">Instantiate TestRunner</a></li> -<li><a href="developer-guide.html#add-controllerservices">Add ControllerServices</a></li> -<li><a href="developer-guide.html#set-property-values">Set Property Values</a></li> -<li><a href="developer-guide.html#enqueue-flowfiles">Enqueue FlowFiles</a></li> -<li><a href="developer-guide.html#run-the-processor">Run the Processor</a></li> -<li><a href="developer-guide.html#validate-output">Validate Output</a></li> -<li><a href="developer-guide.html#mocking-external-resources">Mocking External Resources</a></li> -<li><a href="developer-guide.html#additional-testing-capabilities">Additional Testing Capabilities</a></li> +<li><a href="#instantiate-testrunner">Instantiate TestRunner</a></li> +<li><a href="#add-controllerservices">Add ControllerServices</a></li> +<li><a href="#set-property-values">Set Property Values</a></li> +<li><a href="#enqueue-flowfiles">Enqueue FlowFiles</a></li> +<li><a href="#run-the-processor">Run the Processor</a></li> +<li><a href="#validate-output">Validate Output</a></li> +<li><a href="#mocking-external-resources">Mocking External Resources</a></li> +<li><a href="#additional-testing-capabilities">Additional Testing Capabilities</a></li> </ul> </li> -<li><a href="developer-guide.html#nars">NiFi Archives (NARs)</a></li> -<li><a href="developer-guide.html#how-to-contribute-to-apache-nifi">How to contribute to Apache NiFi</a> +<li><a href="#nars">NiFi Archives (NARs)</a></li> +<li><a href="#how-to-contribute-to-apache-nifi">How to contribute to Apache NiFi</a> <ul class="sectlevel2"> -<li><a href="developer-guide.html#technologies">Technologies</a></li> -<li><a href="developer-guide.html#where-to-start">Where to Start?</a></li> -<li><a href="developer-guide.html#supplying-a-contribution">Supplying a contribution</a></li> -<li><a href="developer-guide.html#contact-us">Contact Us</a></li> +<li><a href="#technologies">Technologies</a></li> +<li><a href="#where-to-start">Where to Start?</a></li> +<li><a href="#supplying-a-contribution">Supplying a contribution</a></li> +<li><a href="#contact-us">Contact Us</a></li> </ul> </li> </ul> @@ -542,10 +542,10 @@ body.book #toc,body.book #preamble,body. </div> <div id="content"> <div class="sect1"> -<h2 id="introduction"><a class="anchor" href="developer-guide.html#introduction"></a>Introduction</h2> +<h2 id="introduction"><a class="anchor" href="#introduction"></a>Introduction</h2> <div class="sectionbody"> <div class="paragraph"> -<p>The intent of this Developer Guide is to provide the reader with the information needed to understand how Apache NiFi (incubating) +<p>The intent of this Developer Guide is to provide the reader with the information needed to understand how Apache NiFi extensions are developed and help to explain the thought process behind developing the components. It provides an introduction to and explanation of the API that is used to develop extensions. It does not, however, go into great detail about each of the methods in the API, as this guide is intended to supplement the JavaDocs of the API rather than replace them. @@ -559,7 +559,7 @@ and the <a href="user-guide.html">NiFi U </div> </div> <div class="sect1"> -<h2 id="components"><a class="anchor" href="developer-guide.html#components"></a>NiFi Components</h2> +<h2 id="components"><a class="anchor" href="#components"></a>NiFi Components</h2> <div class="sectionbody"> <div class="paragraph"> <p>NiFi provides several extension points to provide developers the @@ -574,7 +574,7 @@ high-level description of the most commo <ul> <li> <p>The Processor interface is the mechanism through which NiFi exposes access to -<a href="developer-guide.html#flowfile">FlowFile</a>s, their attributes, and their content. The Processor is the basic building +<a href="#flowfile">FlowFile</a>s, their attributes, and their content. The Processor is the basic building block used to comprise a NiFi dataflow. This interface is used to accomplish all of the following tasks:</p> <div class="ulist"> @@ -646,7 +646,7 @@ many different Processors to load the da <div class="ulist"> <ul> <li> -<p>The FlowFilePrioritizer interface provides a mechanism by which <a href="developer-guide.html#flowfile">FlowFile</a>s +<p>The FlowFilePrioritizer interface provides a mechanism by which <a href="#flowfile">FlowFile</a>s in a queue can be prioritized, or sorted, so that the FlowFiles can be processed in an order that is most effective for a particular use case.</p> </li> @@ -669,7 +669,7 @@ a given user should be granted.</p> </div> </div> <div class="sect1"> -<h2 id="processor_api"><a class="anchor" href="developer-guide.html#processor_api"></a>Processor API</h2> +<h2 id="processor_api"><a class="anchor" href="#processor_api"></a>Processor API</h2> <div class="sectionbody"> <div class="paragraph"> <p>The Processor is the most widely used Component available in NiFi. @@ -710,13 +710,13 @@ must be thread-safe. If unfamiliar with recommended that you familiarize yourself with the principles of Java concurrency.</p> </div> <div class="sect2"> -<h3 id="supporting_api"><a class="anchor" href="developer-guide.html#supporting_api"></a>Supporting API</h3> +<h3 id="supporting_api"><a class="anchor" href="#supporting_api"></a>Supporting API</h3> <div class="paragraph"> <p>In order to understand the Processor API, we must first understand - at least at a high level - several supporting classes and interfaces, which are discussed below.</p> </div> <div class="sect3"> -<h4 id="flowfile"><a class="anchor" href="developer-guide.html#flowfile"></a>FlowFile</h4> +<h4 id="flowfile"><a class="anchor" href="#flowfile"></a>FlowFile</h4> <div class="paragraph"> <p>A FlowFile is a logical notion that correlates a piece of data with a set of Attributes about that data. @@ -728,7 +728,7 @@ immutable. Modifications to a FlowFile a </div> </div> <div class="sect3"> -<h4 id="process_session"><a class="anchor" href="developer-guide.html#process_session"></a>ProcessSession</h4> +<h4 id="process_session"><a class="anchor" href="#process_session"></a>ProcessSession</h4> <div class="paragraph"> <p>The ProcessSession, often referred to as simply a "session," provides a mechanism by which FlowFiles can be created, destroyed, examined, cloned, and transferred to other @@ -740,7 +740,7 @@ ProcessSession can be either committed o </div> </div> <div class="sect3"> -<h4 id="process_context"><a class="anchor" href="developer-guide.html#process_context"></a>ProcessContext</h4> +<h4 id="process_context"><a class="anchor" href="#process_context"></a>ProcessContext</h4> <div class="paragraph"> <p>The ProcessContext provides a bridge between a Processor and the framework. It provides information about how the Processor is currently configured and allows the Processor to perform @@ -749,7 +749,7 @@ Processors to run without consuming reso </div> </div> <div class="sect3"> -<h4 id="property_descriptor"><a class="anchor" href="developer-guide.html#property_descriptor"></a>PropertyDescriptor</h4> +<h4 id="property_descriptor"><a class="anchor" href="#property_descriptor"></a>PropertyDescriptor</h4> <div class="paragraph"> <p>PropertyDescriptor defines a property that is to be used by a Processor, ReportingTask, or ControllerService. @@ -765,7 +765,7 @@ the <code>build</code> method.</p> </div> </div> <div class="sect3"> -<h4 id="validator"><a class="anchor" href="developer-guide.html#validator"></a>Validator</h4> +<h4 id="validator"><a class="anchor" href="#validator"></a>Validator</h4> <div class="paragraph"> <p>A PropertyDescriptor may specify one or more Validators that can be used to ensure that the user-entered value @@ -775,7 +775,7 @@ able to be run or used until the propert </div> </div> <div class="sect3"> -<h4 id="validation_context"><a class="anchor" href="developer-guide.html#validation_context"></a>ValidationContext</h4> +<h4 id="validation_context"><a class="anchor" href="#validation_context"></a>ValidationContext</h4> <div class="paragraph"> <p>When validating property values, a ValidationContext can be used to obtain ControllerServices, @@ -784,7 +784,7 @@ using the Expression Language.</p> </div> </div> <div class="sect3"> -<h4 id="property_value"><a class="anchor" href="developer-guide.html#property_value"></a>PropertyValue</h4> +<h4 id="property_value"><a class="anchor" href="#property_value"></a>PropertyValue</h4> <div class="paragraph"> <p>All property values returned to a Processor are returned in the form of a PropertyValue object. This @@ -795,7 +795,7 @@ Expression Language.</p> </div> </div> <div class="sect3"> -<h4 id="relationship"><a class="anchor" href="developer-guide.html#relationship"></a>Relationship</h4> +<h4 id="relationship"><a class="anchor" href="#relationship"></a>Relationship</h4> <div class="paragraph"> <p>Relationships define the routes to which a FlowFile may be transfered from a Processor. Relationships @@ -806,7 +806,7 @@ to fill in the details of the Relationsh </div> </div> <div class="sect3"> -<h4 id="processor_initialization_context"><a class="anchor" href="developer-guide.html#processor_initialization_context"></a>ProcessorInitializationContext</h4> +<h4 id="processor_initialization_context"><a class="anchor" href="#processor_initialization_context"></a>ProcessorInitializationContext</h4> <div class="paragraph"> <p>After a Processor is created, its <code>initialize</code> method will be called with an <code>InitializationContext</code> object. @@ -816,7 +816,7 @@ such as the unique identifier of the Pro </div> </div> <div class="sect3"> -<h4 id="ProcessorLog"><a class="anchor" href="developer-guide.html#ProcessorLog"></a>ProcessorLog</h4> +<h4 id="ProcessorLog"><a class="anchor" href="#ProcessorLog"></a>ProcessorLog</h4> <div class="paragraph"> <p>Processors are encouraged to perform their logging via the <code>ProcessorLog</code> interface, rather than obtaining @@ -833,7 +833,7 @@ identifier in log messages.</p> </div> </div> <div class="sect2"> -<h3 id="AbstractProcessor"><a class="anchor" href="developer-guide.html#AbstractProcessor"></a>AbstractProcessor API</h3> +<h3 id="AbstractProcessor"><a class="anchor" href="#AbstractProcessor"></a>AbstractProcessor API</h3> <div class="paragraph"> <p>Since the vast majority of Processors will be created by extending the AbstractProcessor, it is the @@ -842,7 +842,7 @@ AbstractProcessor provides several metho will be of interest to Processor developers.</p> </div> <div class="sect3"> -<h4 id="processor-initialization"><a class="anchor" href="developer-guide.html#processor-initialization"></a>Processor Initialization</h4> +<h4 id="processor-initialization"><a class="anchor" href="#processor-initialization"></a>Processor Initialization</h4> <div class="paragraph"> <p>When a Processor is created, before any other methods are invoked, the <code>init</code> method of the @@ -859,7 +859,7 @@ subclasses via the <code>getLogger</code </div> </div> <div class="sect3"> -<h4 id="exposing-processor-s-relationships"><a class="anchor" href="developer-guide.html#exposing-processor-s-relationships"></a>Exposing Processor’s Relationships</h4> +<h4 id="exposing-processor-s-relationships"><a class="anchor" href="#exposing-processor-s-relationships"></a>Exposing Processor’s Relationships</h4> <div class="paragraph"> <p>In order for a Processor to transfer a FlowFile to a new destination for follow-on processing, the @@ -885,7 +885,7 @@ pattern lends itself to cleaner code and </div> </div> <div class="sect3"> -<h4 id="exposing-processor-properties"><a class="anchor" href="developer-guide.html#exposing-processor-properties"></a>Exposing Processor Properties</h4> +<h4 id="exposing-processor-properties"><a class="anchor" href="#exposing-processor-properties"></a>Exposing Processor Properties</h4> <div class="paragraph"> <p>Most Processors will require some amount of user configuration before they are able to be used. The properties @@ -920,7 +920,7 @@ properties.</p> </div> </div> <div class="sect3"> -<h4 id="validating-processor-properties"><a class="anchor" href="developer-guide.html#validating-processor-properties"></a>Validating Processor Properties</h4> +<h4 id="validating-processor-properties"><a class="anchor" href="#validating-processor-properties"></a>Validating Processor Properties</h4> <div class="paragraph"> <p>A Processor is not able to be started if its configuration is not valid. Validation of a Processor property can @@ -946,7 +946,7 @@ validation of a Processor’s config </div> </div> <div class="sect3"> -<h4 id="responding-to-changes-in-configuration"><a class="anchor" href="developer-guide.html#responding-to-changes-in-configuration"></a>Responding to Changes in Configuration</h4> +<h4 id="responding-to-changes-in-configuration"><a class="anchor" href="#responding-to-changes-in-configuration"></a>Responding to Changes in Configuration</h4> <div class="paragraph"> <p>It is sometimes desirable to have a Processor eagerly react when its properties are changed. The <code>onPropertyModified</code> @@ -969,7 +969,7 @@ creates its own threads.</p> </div> </div> <div class="sect3"> -<h4 id="performing-the-work"><a class="anchor" href="developer-guide.html#performing-the-work"></a>Performing the Work</h4> +<h4 id="performing-the-work"><a class="anchor" href="#performing-the-work"></a>Performing the Work</h4> <div class="paragraph"> <p>When a Processor has work to do, it is scheduled to do so by having its <code>onTrigger</code> method called by the framework. @@ -985,7 +985,7 @@ Relationships.</p> </div> </div> <div class="sect3"> -<h4 id="when-processors-are-triggered"><a class="anchor" href="developer-guide.html#when-processors-are-triggered"></a>When Processors are Triggered</h4> +<h4 id="when-processors-are-triggered"><a class="anchor" href="#when-processors-are-triggered"></a>When Processors are Triggered</h4> <div class="paragraph"> <p>A Processor’s <code>onTrigger</code> method will be called only when it is scheduled to run and when work exists for the Processor. @@ -1035,7 +1035,7 @@ be taken to ensure that the Processor is </div> </div> <div class="sect2"> -<h3 id="component-lifecycle"><a class="anchor" href="developer-guide.html#component-lifecycle"></a>Component Lifecycle</h3> +<h3 id="component-lifecycle"><a class="anchor" href="#component-lifecycle"></a>Component Lifecycle</h3> <div class="paragraph"> <p>The NiFi API provides lifecycle support through use of Java Annotations. The <code>org.apache.nifi.annotations.lifecycle</code> package @@ -1048,7 +1048,7 @@ Component Lifecycle, we will define a Ni Processor, ControllerServices, or ReportingTask.</p> </div> <div class="sect3"> -<h4 id="onadded"><a class="anchor" href="developer-guide.html#onadded"></a>@OnAdded</h4> +<h4 id="onadded"><a class="anchor" href="#onadded"></a>@OnAdded</h4> <div class="paragraph"> <p>The <code>@OnAdded</code> annotation causes a method to be invoked as soon as a component is created. The @@ -1065,7 +1065,7 @@ Methods with this Annotation must take z </div> </div> <div class="sect3"> -<h4 id="onremoved"><a class="anchor" href="developer-guide.html#onremoved"></a>@OnRemoved</h4> +<h4 id="onremoved"><a class="anchor" href="#onremoved"></a>@OnRemoved</h4> <div class="paragraph"> <p>The <code>@OnRemoved</code> annotation causes a method to be invoked before a component is removed from the flow. @@ -1076,7 +1076,7 @@ will still be removed.</p> </div> </div> <div class="sect3"> -<h4 id="onscheduled"><a class="anchor" href="developer-guide.html#onscheduled"></a>@OnScheduled</h4> +<h4 id="onscheduled"><a class="anchor" href="#onscheduled"></a>@OnScheduled</h4> <div class="paragraph"> <p>This annotation indicates that a method should be called every time the component is scheduled to run. Because ControllerServices @@ -1102,7 +1102,7 @@ is a ReportingTask.</p> </div> </div> <div class="sect3"> -<h4 id="onunscheduled"><a class="anchor" href="developer-guide.html#onunscheduled"></a>@OnUnscheduled</h4> +<h4 id="onunscheduled"><a class="anchor" href="#onunscheduled"></a>@OnUnscheduled</h4> <div class="paragraph"> <p>Methods with this annotation will be called whenever a Processor or ReportingTask is no longer scheduled to run. At that time, many threads @@ -1119,20 +1119,20 @@ component is a ReportingTask.</p> </div> </div> <div class="sect3"> -<h4 id="onstopped"><a class="anchor" href="developer-guide.html#onstopped"></a>@OnStopped</h4> +<h4 id="onstopped"><a class="anchor" href="#onstopped"></a>@OnStopped</h4> <div class="paragraph"> <p>Methods with this annotation will be called when a Processor or ReportingTask is no longer scheduled to run and all threads have returned from the <code>onTrigger</code> method. If such a method throws an Exception, -a lot message will be generated, and the Exception will otherwise be +a log message will be generated, and the Exception will otherwise be ignored; other methods with this annotation will still be invoked. Methods with this annotation must take zero arguments.</p> </div> </div> <div class="sect3"> -<h4 id="onshutdown"><a class="anchor" href="developer-guide.html#onshutdown"></a>@OnShutdown</h4> +<h4 id="onshutdown"><a class="anchor" href="#onshutdown"></a>@OnShutdown</h4> <div class="paragraph"> <p>Any method that is annotated with the <code>@OnShutdown</code> annotation will be called when NiFi is successfully @@ -1153,7 +1153,7 @@ relied upon for critical business logic. </div> </div> <div class="sect2"> -<h3 id="reporting-processor-activity"><a class="anchor" href="developer-guide.html#reporting-processor-activity"></a>Reporting Processor Activity</h3> +<h3 id="reporting-processor-activity"><a class="anchor" href="#reporting-processor-activity"></a>Reporting Processor Activity</h3> <div class="paragraph"> <p>Processors are responsible for reporting their activity so that users are able to understand what happens @@ -1202,7 +1202,7 @@ perspective of the Processor.</p> </div> </div> <div class="sect1"> -<h2 id="documenting-a-component"><a class="anchor" href="developer-guide.html#documenting-a-component"></a>Documenting a Component</h2> +<h2 id="documenting-a-component"><a class="anchor" href="#documenting-a-component"></a>Documenting a Component</h2> <div class="sectionbody"> <div class="paragraph"> <p>NiFi attempts to make the user experience as simple and convenient as @@ -1214,7 +1214,7 @@ exposes a few different mechanisms for s the framework.</p> </div> <div class="sect2"> -<h3 id="documenting-properties"><a class="anchor" href="developer-guide.html#documenting-properties"></a>Documenting Properties</h3> +<h3 id="documenting-properties"><a class="anchor" href="#documenting-properties"></a>Documenting Properties</h3> <div class="paragraph"> <p>Individual properties can be documented by calling the <code>description</code> method of a PropertyDescriptor’s builder as such:</p> @@ -1253,7 +1253,7 @@ public static final PropertyDescriptor L </div> </div> <div class="sect2"> -<h3 id="documenting-relationships"><a class="anchor" href="developer-guide.html#documenting-relationships"></a>Documenting Relationships</h3> +<h3 id="documenting-relationships"><a class="anchor" href="#documenting-relationships"></a>Documenting Relationships</h3> <div class="paragraph"> <p>Processor Relationships are documented in much the same way that properties are - by calling the <code>description</code> method of a @@ -1269,7 +1269,7 @@ Relationship’s builder:</p> </div> </div> <div class="sect2"> -<h3 id="documenting-capability-and-keywords"><a class="anchor" href="developer-guide.html#documenting-capability-and-keywords"></a>Documenting Capability and Keywords</h3> +<h3 id="documenting-capability-and-keywords"><a class="anchor" href="#documenting-capability-and-keywords"></a>Documenting Capability and Keywords</h3> <div class="paragraph"> <p>The <code>org.apache.nifi.annotations.documentation</code> package provides Java annotations that can be used to document components. The @@ -1302,7 +1302,7 @@ public static final ExampleProcessor ext </div> </div> <div class="sect2"> -<h3 id="documenting-flowfile-attribute-interaction"><a class="anchor" href="developer-guide.html#documenting-flowfile-attribute-interaction"></a>Documenting FlowFile Attribute Interaction</h3> +<h3 id="documenting-flowfile-attribute-interaction"><a class="anchor" href="#documenting-flowfile-attribute-interaction"></a>Documenting FlowFile Attribute Interaction</h3> <div class="paragraph"> <p>Many times a processor will expect certain FlowFile attributes be set on in-bound FlowFiles in order for the processor to function properly. In other cases a processor may update or @@ -1328,7 +1328,7 @@ public final class InvokeHTTP extends Ab </div> </div> <div class="sect2"> -<h3 id="documenting-related-components"><a class="anchor" href="developer-guide.html#documenting-related-components"></a>Documenting Related Components</h3> +<h3 id="documenting-related-components"><a class="anchor" href="#documenting-related-components"></a>Documenting Related Components</h3> <div class="paragraph"> <p>Often Processors and ControllerServices are related to one another. Sometimes its a put/get relation as in <code>PutFile</code> and <code>GetFile</code>. Sometimes a Processor uses a ControllerService like <code>InvokeHTTP</code> and <code>StandardSSLContextService</code>. Sometimes one ControllerService uses another @@ -1344,7 +1344,7 @@ public class PutFile extends AbstractPro </div> </div> <div class="sect2"> -<h3 id="advanced-documentation"><a class="anchor" href="developer-guide.html#advanced-documentation"></a>Advanced Documentation</h3> +<h3 id="advanced-documentation"><a class="anchor" href="#advanced-documentation"></a>Advanced Documentation</h3> <div class="paragraph"> <p>When the documentation methods above are not sufficient, NiFi provides the ability to expose more advanced documentation to the user via the @@ -1372,7 +1372,7 @@ documentation for Processors, Controller </div> </div> <div class="sect1"> -<h2 id="common-processor-patterns"><a class="anchor" href="developer-guide.html#common-processor-patterns"></a>Common Processor Patterns</h2> +<h2 id="common-processor-patterns"><a class="anchor" href="#common-processor-patterns"></a>Common Processor Patterns</h2> <div class="sectionbody"> <div class="paragraph"> <p>While there are many different Processors available to NiFi users, the @@ -1385,7 +1385,7 @@ and recommendations discussed below are hardened rules.</p> </div> <div class="sect2"> -<h3 id="ingress"><a class="anchor" href="developer-guide.html#ingress"></a>Data Ingress</h3> +<h3 id="ingress"><a class="anchor" href="#ingress"></a>Data Ingress</h3> <div class="paragraph"> <p>A Processor that ingests data into NiFi has a single Relationship names <code>success</code>. This Processor generates @@ -1473,7 +1473,7 @@ annotated with the <code>@OnStopped</cod </div> </div> <div class="sect2"> -<h3 id="data-egress"><a class="anchor" href="developer-guide.html#data-egress"></a>Data Egress</h3> +<h3 id="data-egress"><a class="anchor" href="#data-egress"></a>Data Egress</h3> <div class="paragraph"> <p>A Processor that publishes data to an external source has two Relationships: <code>success</code> and <code>failure</code>. The @@ -1534,7 +1534,7 @@ error depends on a few considerations. I network condition, the FlowFile is generally routed to <code>failure</code>. The FlowFile is not penalized because there is not necessary a problem with the data. Unlike the -case of the <a href="developer-guide.html#ingress">Data Ingress</a> Processor, we typically do not call <code>yield</code> on +case of the <a href="#ingress">Data Ingress</a> Processor, we typically do not call <code>yield</code> on the ProcessContext. This is because in the case of ingest, the FlowFile does not exist until the Processor is able to perform its function. However, in the case of a Put Processor, @@ -1570,7 +1570,7 @@ annotated with <code>@OnStopped</code> s </div> </div> <div class="sect2"> -<h3 id="route-based-on-content-one-to-one"><a class="anchor" href="developer-guide.html#route-based-on-content-one-to-one"></a>Route Based on Content (One-to-One)</h3> +<h3 id="route-based-on-content-one-to-one"><a class="anchor" href="#route-based-on-content-one-to-one"></a>Route Based on Content (One-to-One)</h3> <div class="paragraph"> <p>A Processor that routes data based on its content will take one of two forms: Route an incoming FlowFile to exactly @@ -1611,7 +1611,7 @@ package.</p> </div> </div> <div class="sect2"> -<h3 id="route-based-on-content-one-to-many"><a class="anchor" href="developer-guide.html#route-based-on-content-one-to-many"></a>Route Based on Content (One-to-Many)</h3> +<h3 id="route-based-on-content-one-to-many"><a class="anchor" href="#route-based-on-content-one-to-many"></a>Route Based on Content (One-to-Many)</h3> <div class="paragraph"> <p>If a Processor will route a single FlowFile to potentially many relationships, this Processor will be slightly different than @@ -1696,7 +1696,7 @@ package.</p> </div> </div> <div class="sect2"> -<h3 id="route-streams-based-on-content-one-to-many"><a class="anchor" href="developer-guide.html#route-streams-based-on-content-one-to-many"></a>Route Streams Based on Content (One-to-Many)</h3> +<h3 id="route-streams-based-on-content-one-to-many"><a class="anchor" href="#route-streams-based-on-content-one-to-many"></a>Route Streams Based on Content (One-to-Many)</h3> <div class="paragraph"> <p>The previous description of Route Based on Content (One-to-Many) provides an abstraction @@ -1848,7 +1848,7 @@ Capturing Group).</p> </div> </div> <div class="sect2"> -<h3 id="route-based-on-attributes"><a class="anchor" href="developer-guide.html#route-based-on-attributes"></a>Route Based on Attributes</h3> +<h3 id="route-based-on-attributes"><a class="anchor" href="#route-based-on-attributes"></a>Route Based on Attributes</h3> <div class="paragraph"> <p>This Processor is almost identical to the Route Data Based on Content Processors described above. It takes two different forms: One-to-One @@ -1862,7 +1862,7 @@ in this case.</p> </div> </div> <div class="sect2"> -<h3 id="split-content-one-to-many"><a class="anchor" href="developer-guide.html#split-content-one-to-many"></a>Split Content (One-to-Many)</h3> +<h3 id="split-content-one-to-many"><a class="anchor" href="#split-content-one-to-many"></a>Split Content (One-to-Many)</h3> <div class="paragraph"> <p>This Processor generally requires no user configuration, with the exception of the size of each Split to create. The <code>onTrigger</code> method @@ -2017,7 +2017,7 @@ logged; and the method returns.</p> </div> </div> <div class="sect2"> -<h3 id="update-attributes-based-on-content"><a class="anchor" href="developer-guide.html#update-attributes-based-on-content"></a>Update Attributes Based on Content</h3> +<h3 id="update-attributes-based-on-content"><a class="anchor" href="#update-attributes-based-on-content"></a>Update Attributes Based on Content</h3> <div class="paragraph"> <p>This Processor is very similar to the Route Based on Content Processors discussed above. Rather than @@ -2054,7 +2054,7 @@ when appropriate.</p> </div> </div> <div class="sect2"> -<h3 id="enrich-modify-content"><a class="anchor" href="developer-guide.html#enrich-modify-content"></a>Enrich/Modify Content</h3> +<h3 id="enrich-modify-content"><a class="anchor" href="#enrich-modify-content"></a>Enrich/Modify Content</h3> <div class="paragraph"> <p>The Enrich/Modify Content pattern is very common and very generic. This pattern is responsible for any @@ -2085,7 +2085,7 @@ and routes the FlowFile to failure.</p> </div> </div> <div class="sect1"> -<h2 id="error-handling"><a class="anchor" href="developer-guide.html#error-handling"></a>Error Handling</h2> +<h2 id="error-handling"><a class="anchor" href="#error-handling"></a>Error Handling</h2> <div class="sectionbody"> <div class="paragraph"> <p>When writing a Processor, there are several different unexpected cases that can occur. @@ -2095,7 +2095,7 @@ what error handling is expected of Proce handle unexpected errors during the course of their work.</p> </div> <div class="sect2"> -<h3 id="exceptions-within-the-processor"><a class="anchor" href="developer-guide.html#exceptions-within-the-processor"></a>Exceptions within the Processor</h3> +<h3 id="exceptions-within-the-processor"><a class="anchor" href="#exceptions-within-the-processor"></a>Exceptions within the Processor</h3> <div class="paragraph"> <p>During the execution of the <code>onTrigger</code> method of a Processor, many things can potentially go awry. Common failure conditions include:</p> @@ -2140,7 +2140,7 @@ in the <code>nifi.properties</code> file </div> </div> <div class="sect2"> -<h3 id="exceptions-within-a-callback-ioexception-runtimeexception"><a class="anchor" href="developer-guide.html#exceptions-within-a-callback-ioexception-runtimeexception"></a>Exceptions within a callback: IOException, RuntimeException</h3> +<h3 id="exceptions-within-a-callback-ioexception-runtimeexception"><a class="anchor" href="#exceptions-within-a-callback-ioexception-runtimeexception"></a>Exceptions within a callback: IOException, RuntimeException</h3> <div class="paragraph"> <p>More often than not, when an Exception occurs in a Processor, it occurs from within a callback (I.e., <code>InputStreamCallback</code>, <code>OutputStreamCallback</code>, or <code>StreamCallback</code>). That is, during the processing of a @@ -2170,7 +2170,7 @@ condition appropriately. Catching the ge </div> </div> <div class="sect2"> -<h3 id="penalization-vs-yielding"><a class="anchor" href="developer-guide.html#penalization-vs-yielding"></a>Penalization vs. Yielding</h3> +<h3 id="penalization-vs-yielding"><a class="anchor" href="#penalization-vs-yielding"></a>Penalization vs. Yielding</h3> <div class="paragraph"> <p>When an issue occurs during processing, the framework exposes two methods to allow Processor developers to avoid performing unnecessary work: "penalization" and "yielding." These two concepts can become confusing for developers new to the NiFi API. @@ -2195,7 +2195,7 @@ there’s nothing that it can do - i </div> </div> <div class="sect2"> -<h3 id="session-rollback"><a class="anchor" href="developer-guide.html#session-rollback"></a>Session Rollback</h3> +<h3 id="session-rollback"><a class="anchor" href="#session-rollback"></a>Session Rollback</h3> <div class="paragraph"> <p>Thus far, when we have discussed the <code>ProcessSession</code>, we have typically referred to it simply as a mechanism for accessing FlowFiles. However, it provides another very important capability, which is transactionality. All methods that are called @@ -2233,14 +2233,14 @@ These "batched" commits are not rolled b </div> </div> <div class="sect1"> -<h2 id="general-design-considerations"><a class="anchor" href="developer-guide.html#general-design-considerations"></a>General Design Considerations</h2> +<h2 id="general-design-considerations"><a class="anchor" href="#general-design-considerations"></a>General Design Considerations</h2> <div class="sectionbody"> <div class="paragraph"> <p>When designing a Processor, there are a few important design considering to keep in mind. This section of the Developer Guide brings to the forefront some of the ideas that a developer should be thinking about when creating a Processor.</p> </div> <div class="sect2"> -<h3 id="consider-the-user"><a class="anchor" href="developer-guide.html#consider-the-user"></a>Consider the User</h3> +<h3 id="consider-the-user"><a class="anchor" href="#consider-the-user"></a>Consider the User</h3> <div class="paragraph"> <p>One of the most important concepts to keep in mind when developing a Processor (or any other component) is the user experience that you are creating. It’s important to remember that as the developer of such a component, you may have @@ -2249,7 +2249,7 @@ less familiar with the process are able </div> <div class="paragraph"> <p>When thinking about the user experience, it is also important to note that consistency is very important. It is best -to stick with the standard <a href="developer-guide.html#naming-convensions">Naming Conventions</a>. This is true for Processor names, Property names and value, Relationship +to stick with the standard <a href="#naming-convensions">Naming Conventions</a>. This is true for Processor names, Property names and value, Relationship names, and any other aspect that the user will experience.</p> </div> <div class="paragraph"> @@ -2259,7 +2259,7 @@ tell users to just leave the default val </div> </div> <div class="sect2"> -<h3 id="cohesion-and-reusability"><a class="anchor" href="developer-guide.html#cohesion-and-reusability"></a>Cohesion and Reusability</h3> +<h3 id="cohesion-and-reusability"><a class="anchor" href="#cohesion-and-reusability"></a>Cohesion and Reusability</h3> <div class="paragraph"> <p>For the sake of making a single, cohesive unit, developers are sometimes tempted to combine several functions into a single Processor. This is very true for the case when a Processor expects input data to be in format X so that the Processor can convert the data into @@ -2296,7 +2296,7 @@ Processor to send data to the remote res </div> </div> <div class="sect2"> -<h3 id="naming-convensions"><a class="anchor" href="developer-guide.html#naming-convensions"></a>Naming Conventions</h3> +<h3 id="naming-convensions"><a class="anchor" href="#naming-convensions"></a>Naming Conventions</h3> <div class="paragraph"> <p>In order to deliver a consistent look and feel to users, it is advisable that Processors keep with standard naming conventions. The following is a list of standard conventions that are used:</p> @@ -2320,7 +2320,7 @@ sources over a known Protocol (such as G </div> </div> <div class="sect2"> -<h3 id="processor-behavior-annotations"><a class="anchor" href="developer-guide.html#processor-behavior-annotations"></a>Processor Behavior Annotations</h3> +<h3 id="processor-behavior-annotations"><a class="anchor" href="#processor-behavior-annotations"></a>Processor Behavior Annotations</h3> <div class="paragraph"> <p>When creating a Processor, the developer is able to provide hints to the framework about how to utilize the Processor most effectively. This is done by applying annotations to the Processor’s class. The annotations that can be applied to a @@ -2383,7 +2383,7 @@ periodically to time out a network conne </div> </div> <div class="sect2"> -<h3 id="data-buffering"><a class="anchor" href="developer-guide.html#data-buffering"></a>Data Buffering</h3> +<h3 id="data-buffering"><a class="anchor" href="#data-buffering"></a>Data Buffering</h3> <div class="paragraph"> <p>An important point to keep in mind is that NiFi provides a generic data processing capability. Data can be in any format. Processors are generally scheduled with several threads. A common mistake that developers new to NiFi make is to buffer all the contents of a @@ -2403,7 +2403,7 @@ amount of data, as appropriate.</p> </div> </div> <div class="sect1"> -<h2 id="controller-services"><a class="anchor" href="developer-guide.html#controller-services"></a>Controller Services</h2> +<h2 id="controller-services"><a class="anchor" href="#controller-services"></a>Controller Services</h2> <div class="sectionbody"> <div class="paragraph"> <p>The <code>ControllerService</code> interface allows developers to share @@ -2417,7 +2417,7 @@ integrated into the flow directly. Rathe they are used Processors, Reporting Tasks, and other Controller Services.</p> </div> <div class="sect2"> -<h3 id="developing-controller-service"><a class="anchor" href="developer-guide.html#developing-controller-service"></a>Developing a ControllerService</h3> +<h3 id="developing-controller-service"><a class="anchor" href="#developing-controller-service"></a>Developing a ControllerService</h3> <div class="paragraph"> <p>Just like with the Processor interface, the ControllerService interface exposes methods for configuration, @@ -2450,11 +2450,11 @@ in order to make this work, the Processo implementation must share the same definition of the Controller Service interface. Therefore, both of these NARs must depend on the NAR that houses the -Controller Service’s interface. See <a href="developer-guide.html#nars">NiFi Archives (NARs)</a> for more information.</p> +Controller Service’s interface. See <a href="#nars">NiFi Archives (NARs)</a> for more information.</p> </div> </div> <div class="sect2"> -<h3 id="interacting-with-controller-service"><a class="anchor" href="developer-guide.html#interacting-with-controller-service"></a>Interacting with a ControllerService</h3> +<h3 id="interacting-with-controller-service"><a class="anchor" href="#interacting-with-controller-service"></a>Interacting with a ControllerService</h3> <div class="paragraph"> <p>ControllerServices may be obtained by a Processor, another ControllerService, or a ReportingTask @@ -2512,7 +2512,7 @@ detail.</p> </div> </div> <div class="sect1"> -<h2 id="reporting-tasks"><a class="anchor" href="developer-guide.html#reporting-tasks"></a>Reporting Tasks</h2> +<h2 id="reporting-tasks"><a class="anchor" href="#reporting-tasks"></a>Reporting Tasks</h2> <div class="sectionbody"> <div class="paragraph"> <p>So far, we have mentioned little about how to convey to the outside @@ -2531,7 +2531,7 @@ interface. ReportingTasks are given acce determine how the system is performing.</p> </div> <div class="sect2"> -<h3 id="developing-a-reporting-task"><a class="anchor" href="developer-guide.html#developing-a-reporting-task"></a>Developing a Reporting Task</h3> +<h3 id="developing-a-reporting-task"><a class="anchor" href="#developing-a-reporting-task"></a>Developing a Reporting Task</h3> <div class="paragraph"> <p>Just like with the Processor and ControllerService interfaces, the ReportingTask interface exposes methods for @@ -2557,7 +2557,7 @@ configured. However, this method of obta not the preferred method. Rather, the preferred method for obtaining a Controller Service is to reference the Controller Service in a PropertyDescriptor, -as is discussed in the <a href="developer-guide.html#interacting-with-controller-service">Interacting with a ControllerService</a> section.</p> +as is discussed in the <a href="#interacting-with-controller-service">Interacting with a ControllerService</a> section.</p> </div> <div class="paragraph"> <p>The <code>EventAccess</code> object that is exposed via the ReportingContext @@ -2589,7 +2589,7 @@ needed for any number of operational con </div> </div> <div class="sect1"> -<h2 id="testing"><a class="anchor" href="developer-guide.html#testing"></a>Testing</h2> +<h2 id="testing"><a class="anchor" href="#testing"></a>Testing</h2> <div class="sectionbody"> <div class="paragraph"> <p>Testing the components that will be used within a larger framework can often be very cumbersome @@ -2612,7 +2612,7 @@ as well as invoking the necessary lifecy same way in the unit tests as it does in production.</p> </div> <div class="sect2"> -<h3 id="instantiate-testrunner"><a class="anchor" href="developer-guide.html#instantiate-testrunner"></a>Instantiate TestRunner</h3> +<h3 id="instantiate-testrunner"><a class="anchor" href="#instantiate-testrunner"></a>Instantiate TestRunner</h3> <div class="paragraph"> <p>Most unit tests for a Processor or a Controller Service start by creating an instance of the <code>TestRunner</code> class. In order to add the necessary classes to your Processor, @@ -2634,7 +2634,7 @@ either be the class of the Processor to </div> </div> <div class="sect2"> -<h3 id="add-controllerservices"><a class="anchor" href="developer-guide.html#add-controllerservices"></a>Add ControllerServices</h3> +<h3 id="add-controllerservices"><a class="anchor" href="#add-controllerservices"></a>Add ControllerServices</h3> <div class="paragraph"> <p>After creating a new Test Runner, we can add any Controller Services to the Test Runner that our Processor will need in order to perform its job. We do this by calling the <code>addControllerService</code> method and supply @@ -2659,7 +2659,7 @@ will throw an IllegalStateException. Oth </div> </div> <div class="sect2"> -<h3 id="set-property-values"><a class="anchor" href="developer-guide.html#set-property-values"></a>Set Property Values</h3> +<h3 id="set-property-values"><a class="anchor" href="#set-property-values"></a>Set Property Values</h3> <div class="paragraph"> <p>After configuring any necessary Controller Services, we need to configure our Processor. We can do this by calling the same methods as we do for Controller Services, without specifying any Controller Service. I.e., @@ -2672,7 +2672,7 @@ Processor is valid or not, according to </div> </div> <div class="sect2"> -<h3 id="enqueue-flowfiles"><a class="anchor" href="developer-guide.html#enqueue-flowfiles"></a>Enqueue FlowFiles</h3> +<h3 id="enqueue-flowfiles"><a class="anchor" href="#enqueue-flowfiles"></a>Enqueue FlowFiles</h3> <div class="paragraph"> <p>Before triggering a Processor to run, it is usually necessary to enqueue FlowFiles for the Processor to process. This can be achieved by using the <code>enqueue</code> methods of the <code>TestRunner</code> class. The <code>enqueue</code> method has several @@ -2685,7 +2685,7 @@ to obtain the output of a Processor and </div> </div> <div class="sect2"> -<h3 id="run-the-processor"><a class="anchor" href="developer-guide.html#run-the-processor"></a>Run the Processor</h3> +<h3 id="run-the-processor"><a class="anchor" href="#run-the-processor"></a>Run the Processor</h3> <div class="paragraph"> <p>After configuring the Controller Services and enqueuing the necessary FlowFiles, the Processor can be triggered to run by calling the <code>run</code> method of <code>TestRunner</code>. If this method is called without any arguments, it will @@ -2714,7 +2714,7 @@ the number of times that the Processor s </div> </div> <div class="sect2"> -<h3 id="validate-output"><a class="anchor" href="developer-guide.html#validate-output"></a>Validate Output</h3> +<h3 id="validate-output"><a class="anchor" href="#validate-output"></a>Validate Output</h3> <div class="paragraph"> <p>After a Processor has finished running, a unit test will generally want to validate that the FlowFiles went where they were expected to go. This can be achieved using the <code>TestRunners</code> <code>assertAllFlowFilesTransferred</code> and @@ -2738,7 +2738,7 @@ as expected.</p> </div> </div> <div class="sect2"> -<h3 id="mocking-external-resources"><a class="anchor" href="developer-guide.html#mocking-external-resources"></a>Mocking External Resources</h3> +<h3 id="mocking-external-resources"><a class="anchor" href="#mocking-external-resources"></a>Mocking External Resources</h3> <div class="paragraph"> <p>One of the biggest problems when testing a NiFi processor that connects to a remote resource is that we don’t want to actually connect to some remote resource from a unit test. We can stand up a simple server ourselves in the unit test @@ -2781,7 +2781,7 @@ to the client.</p> </div> </div> <div class="sect2"> -<h3 id="additional-testing-capabilities"><a class="anchor" href="developer-guide.html#additional-testing-capabilities"></a>Additional Testing Capabilities</h3> +<h3 id="additional-testing-capabilities"><a class="anchor" href="#additional-testing-capabilities"></a>Additional Testing Capabilities</h3> <div class="paragraph"> <p>In addition to the above-mentioned capabilities provided by the testing framework, the TestRunner provides several @@ -2801,7 +2801,7 @@ be set via the <code>setThreadCount(int) </div> </div> <div class="sect1"> -<h2 id="nars"><a class="anchor" href="developer-guide.html#nars"></a>NiFi Archives (NARs)</h2> +<h2 id="nars"><a class="anchor" href="#nars"></a>NiFi Archives (NARs)</h2> <div class="sectionbody"> <div class="paragraph"> <p>When software from many different organizations is all hosted within @@ -2863,7 +2863,7 @@ we refer to NAR B as the <em>Parent</em> <div class="paragraph"> <p>This linkage of Parent ClassLoaders is the mechanism that NiFi uses in order to enable Controller Services to be shared -across all NARs. As mentioned in the <a href="developer-guide.html#developing-controller-service">Developing a ControllerService</a> +across all NARs. As mentioned in the <a href="#developing-controller-service">Developing a ControllerService</a> section, A Controller Service must be separated into an interface that extends <code>ControllerService</code> and an implementation that implements that interface. Controller Services @@ -2967,7 +2967,7 @@ API artifacts into the same NAR is often </div> </div> <div class="sect1"> -<h2 id="how-to-contribute-to-apache-nifi"><a class="anchor" href="developer-guide.html#how-to-contribute-to-apache-nifi"></a>How to contribute to Apache NiFi</h2> +<h2 id="how-to-contribute-to-apache-nifi"><a class="anchor" href="#how-to-contribute-to-apache-nifi"></a>How to contribute to Apache NiFi</h2> <div class="sectionbody"> <div class="paragraph"> <p>We are always excited to have contributions from the community - especially from new contributors! @@ -2975,7 +2975,7 @@ We are interested in accepting contribut can be applied as icons or styling to the application.</p> </div> <div class="sect2"> -<h3 id="technologies"><a class="anchor" href="developer-guide.html#technologies"></a>Technologies</h3> +<h3 id="technologies"><a class="anchor" href="#technologies"></a>Technologies</h3> <div class="paragraph"> <p>The back end of Apache NiFi is written in Java. The web tier makes use of JAX-RS and JavaScript is extensively used to provide a user interface. We depend on several third-party JavaScript libraries, including D3 and JQuery, @@ -2986,7 +2986,7 @@ among others. We make use of Apache Mave </div> </div> <div class="sect2"> -<h3 id="where-to-start"><a class="anchor" href="developer-guide.html#where-to-start"></a>Where to Start?</h3> +<h3 id="where-to-start"><a class="anchor" href="#where-to-start"></a>Where to Start?</h3> <div class="paragraph"> <p><a href="http://issues.apache.org/jira/browse/NIFI">NiFi’s JIRA page</a> can be used to find tickets that are tagged as "beginner", or you can dig into any of the tickets for creating Processors. Processors should be self-contained and not rely on other @@ -2999,7 +2999,7 @@ Tools available to facilitate documentat </div> </div> <div class="sect2"> -<h3 id="supplying-a-contribution"><a class="anchor" href="developer-guide.html#supplying-a-contribution"></a>Supplying a contribution</h3> +<h3 id="supplying-a-contribution"><a class="anchor" href="#supplying-a-contribution"></a>Supplying a contribution</h3> <div class="paragraph"> <p>Contributions can be provided either by creating a patch:</p> </div> @@ -3011,9 +3011,9 @@ Tools available to facilitate documentat </div> </div> <div class="sect2"> -<h3 id="contact-us"><a class="anchor" href="developer-guide.html#contact-us"></a>Contact Us</h3> +<h3 id="contact-us"><a class="anchor" href="#contact-us"></a>Contact Us</h3> <div class="paragraph"> -<p>The developer mailing list (<a href="mailto:d...@nifi.incubator.apache.org">d...@nifi.incubator.apache.org</a>) is monitored pretty closely, and we tend to respond pretty +<p>The developer mailing list (<a href="mailto:d...@nifi.apache.org">d...@nifi.apache.org</a>) is monitored pretty closely, and we tend to respond pretty quickly. If you have a question, don’t hesitate to shoot us an e-mail - we’re here to help! Unfortunately, though, e-mails can get lost in the shuffle, so if you do send an e-mail and don’t get a response within a day or two, it’s our fault - don’t worry about bothering us. Just ping the mailing list again.</p> @@ -3024,7 +3024,7 @@ worry about bothering us. Just ping the </div> <div id="footer"> <div id="footer-text"> -Last updated 2015-05-11 15:23:17 EDT +Last updated 2015-07-23 13:20:05 PDT </div> </div> </body>