[jira] [Created] (NIFI-104) Provenance ROUTE Events should not be sent to Prov Repo if FlowFile has recently been routed to the same connection multiple times

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-104:


 Summary: Provenance ROUTE Events should not be sent to Prov Repo 
if FlowFile has recently been routed to the same connection multiple times
 Key: NIFI-104
 URL: https://issues.apache.org/jira/browse/NIFI-104
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


If we keep track of the Connections in which a FlowFile has recently been 
enqueued, we can choose to ignore Provenance ROUTE events if a FlowFile has 
been routed to a given Connection multiple times. This is advantageous because 
it allows us to do things like emit ROUTE events to indicate that a FlowFile 
was routed to failure. Currently, we very much frown upon this behavior because 
DFM's often route failure back to self and this would result in thousands (or 
more) route events being generated for a FlowFile.

However, if we determine that a FlowFile has already been routed to the failure 
relationship (for instance) of a given processor recently (perhaps 3+ times?) 
then we can decide not to emit that event, thus preventing us from generating 
lots of useless events.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-105) StandardRemoteGroupPort should support event driven, but perhaps not allow an unbounded thread count

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-105:


 Summary: StandardRemoteGroupPort should support event driven, but 
perhaps not allow an unbounded thread count
 Key: NIFI-105
 URL: https://issues.apache.org/jira/browse/NIFI-105
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


Currently, this Port is very CPU-intensive if there's no data to send because 
it's continually checking for data.

It's important that we consider both SEND and RECEIVE ports...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-106) Processor Counters should be included in the Status Reports

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-106:


 Summary: Processor Counters should be included in the Status 
Reports
 Key: NIFI-106
 URL: https://issues.apache.org/jira/browse/NIFI-106
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


This would allow a Processor's Status HIstory to show counters that were 
maintained over time periods instead of having only a single count since system 
start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-107) Provenance Reporter should allow developers to call route(FlowFile, Relationship, Throwable cause) or route(FlowFlow, Relationship, String details, Throwable cause)

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-107:


 Summary: Provenance Reporter should allow developers to call 
route(FlowFile, Relationship, Throwable cause) or route(FlowFlow, Relationship, 
String details, Throwable cause)
 Key: NIFI-107
 URL: https://issues.apache.org/jira/browse/NIFI-107
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


This would handle filling in the details with the toString() and stack trace 
from the Throwable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-108) DFM should be allowed to inspect FlowFiles on a Connection

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-108:


 Summary: DFM should be allowed to inspect FlowFiles on a Connection
 Key: NIFI-108
 URL: https://issues.apache.org/jira/browse/NIFI-108
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework, Core UI
Reporter: Matt Gilman


User should be able to see the attributes, as well as download the content and 
possibly remove an item from the queue (destroy the FlowFile entirely).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-109) When viewing Provenance Event in UI, user should be able to click Next / Previous to see the next/previous event for the same FlowFile

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-109:


 Summary: When viewing Provenance Event in UI, user should be able 
to click Next / Previous to see the next/previous event for the same FlowFile
 Key: NIFI-109
 URL: https://issues.apache.org/jira/browse/NIFI-109
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework, Core UI
Reporter: Matt Gilman
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-111) Investigate adding support to adjust z-index of all components

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-111:


 Summary: Investigate adding support to adjust z-index of all 
components
 Key: NIFI-111
 URL: https://issues.apache.org/jira/browse/NIFI-111
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Matt Gilman
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-112) Search Results should show Run Status for anything for which it is applicable

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-112:


 Summary: Search Results should show Run Status for anything for 
which it is applicable
 Key: NIFI-112
 URL: https://issues.apache.org/jira/browse/NIFI-112
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Matt Gilman
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-114) Consider evaluating cubism.js...

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-114:


 Summary: Consider evaluating cubism.js...
 Key: NIFI-114
 URL: https://issues.apache.org/jira/browse/NIFI-114
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Matt Gilman
Priority: Minor


... for visualing time series. Used to build a realtime dashboard.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-119) Populate initial NCM flow with existing flow.xml.gz

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-119:


 Summary: Populate initial NCM flow with existing flow.xml.gz
 Key: NIFI-119
 URL: https://issues.apache.org/jira/browse/NIFI-119
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


Consider the ramifications of populating the flow.tar (cluster) with an 
existing flow.xml.gz if no existing flow.tar is found.

This would only be appropriate if no flow.tar or any of its counterparts (such 
as flow.tar.stale or flow.tar.unknown) exists.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-127) Status History for Process Groups should show average latency

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-127:


 Summary: Status History for Process Groups should show average 
latency
 Key: NIFI-127
 URL: https://issues.apache.org/jira/browse/NIFI-127
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


Calculated by time that FlowFile was dropped or transferred out of group minus 
time that FlowFile entered group.

We will not persist the time at which the FlowFile entered a ProcessGroupl, so 
on restart the FlowFiles that are restored will not be counted towards this 
latency calculation.

Should work across swapping!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-130) Site-to-Site's SocketClientProtocol and SocketFlowFileServerProtocol need to be refactored

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-130:


 Summary: Site-to-Site's SocketClientProtocol and 
SocketFlowFileServerProtocol need to be refactored
 Key: NIFI-130
 URL: https://issues.apache.org/jira/browse/NIFI-130
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


Currently they have the same code repeated for transferFlowFiles, 
receiveFlowFiles. These methods should be refactored into a util class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-134) Mock Framework should expose mechanism for testing that Processors behave nicely

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-134:


 Summary: Mock Framework should expose mechanism for testing that 
Processors behave nicely
 Key: NIFI-134
 URL: https://issues.apache.org/jira/browse/NIFI-134
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman


Should perform tasks as such:

- throw FlowFileAccessException from callback and ensuring that the session is 
rolled back, rather than the Processor catching the exception and dealing with 
it itself.
- Ensure that @CapabilityDescription is on Processor
- Ensure that @Tags are used
- Ensure that Processor is in the META-INF/services file
- Ensure that there is documentation html for Processor

Should allow configuration object to configure which of these actions is 
performed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-135) If unable to expire data from Persistent Prov Repo, should keep retrying until successful

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-135:


 Summary: If unable to expire data from Persistent Prov Repo, 
should keep retrying until successful
 Key: NIFI-135
 URL: https://issues.apache.org/jira/browse/NIFI-135
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-136) When site-to-site refers to self, can get authorization errors that should not occur

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-136:


 Summary: When site-to-site refers to self, can get authorization 
errors that should not occur
 Key: NIFI-136
 URL: https://issues.apache.org/jira/browse/NIFI-136
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


In the StandardRemoteProcessGroup, we periodically refresh the peers from the 
remote instance. In order to do this, we choose arbitrary Input Port as the 
target, simply because we will get the same results regardless of the Port.

However, this makes the assumption that we have access to communicate with all 
Ports that we know about. When referring to your own cluster, though, we don't 
ask the NCM for the list of Ports; we simply use the Ports that we know about. 
As a result, we could choose a Port for which we have not given ourselves 
access. This results in a Bulletin showing up indicating that we don't have 
access when we refresh peers.

This is harmless but annoying and scary when the bulletin is seen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-137) Cluster should generate bulletin for any Node State change

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-137:


 Summary: Cluster should generate bulletin for any Node State change
 Key: NIFI-137
 URL: https://issues.apache.org/jira/browse/NIFI-137
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


Currently, if node restarts quickly, it will connect before its last heartbeat 
times out, so nodes goes from Connected to Connecting. UI shows '4/5' for 
example, because the node is now in Connecting. But there's no bulletin to 
indicate what happened because it never went to disconnected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-138) For Status History, should fetch the appropriate Stats Fields for each Processor, rather than for each class of Component

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-138:


 Summary: For Status History, should fetch the appropriate Stats 
Fields for each Processor, rather than for each class of Component
 Key: NIFI-138
 URL: https://issues.apache.org/jira/browse/NIFI-138
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-139) Allow user to see the distribution of FlowFiles per Node on a Connection

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-139:


 Summary: Allow user to see the distribution of FlowFiles per Node 
on a Connection
 Key: NIFI-139
 URL: https://issues.apache.org/jira/browse/NIFI-139
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor


When the user is looking at a Connection on the graph and it has a queue, it 
would be very nice to have the ability to somehow see which nodes have how much 
of that queue. Perhaps this should be part of a more general View Queue 
capability.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-108) DFM should be allowed to inspect FlowFiles on a Connection

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-108:
-
Issue Type: New Feature  (was: Improvement)

 DFM should be allowed to inspect FlowFiles on a Connection
 --

 Key: NIFI-108
 URL: https://issues.apache.org/jira/browse/NIFI-108
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Core Framework, Core UI
Reporter: Matt Gilman

 User should be able to see the attributes, as well as download the content 
 and possibly remove an item from the queue (destroy the FlowFile entirely).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-111) Investigate adding support to adjust z-index of all components

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237938#comment-14237938
 ] 

Joseph Witt commented on NIFI-111:
--

Is the purpose of this to allow the user to choose the depth of a component?  
I'm not sure this would result in a good user experience.  When would they want 
to change that and that this would be a good use?  Before tackling this we 
should document a good use case I think.

 Investigate adding support to adjust z-index of all components
 --

 Key: NIFI-111
 URL: https://issues.apache.org/jira/browse/NIFI-111
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Matt Gilman
Priority: Minor





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-106) Processor Counters should be included in the Status Reports

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237941#comment-14237941
 ] 

Joseph Witt commented on NIFI-106:
--

We should document a strong use case here.  I am suspect of whether counters 
are still a strong play beyond purely transient/developer-centric use cases 
especially given provenance the intent to add support for generating complex 
reports on such data.  Given that I think counters as-is are done well.  But if 
we have use cases to consider then of course...

 Processor Counters should be included in the Status Reports
 ---

 Key: NIFI-106
 URL: https://issues.apache.org/jira/browse/NIFI-106
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor

 This would allow a Processor's Status HIstory to show counters that were 
 maintained over time periods instead of having only a single count since 
 system start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-105) StandardRemoteGroupPort should support event driven, but perhaps not allow an unbounded thread count

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-105:
-
Priority: Major  (was: Minor)

 StandardRemoteGroupPort should support event driven, but perhaps not allow an 
 unbounded thread count
 

 Key: NIFI-105
 URL: https://issues.apache.org/jira/browse/NIFI-105
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman

 Currently, this Port is very CPU-intensive if there's no data to send because 
 it's continually checking for data.
 It's important that we consider both SEND and RECEIVE ports...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-105) StandardRemoteGroupPort should support event driven, but perhaps not allow an unbounded thread count

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237946#comment-14237946
 ] 

Joseph Witt commented on NIFI-105:
--

Excessive CPU usage will be important to tackle.  If there is no data flowing 
or anything useful going on needless consumption of resources can create issues 
for those wanting to use NiFi in resource contrained or metered environments.  
Bumping to major.

 StandardRemoteGroupPort should support event driven, but perhaps not allow an 
 unbounded thread count
 

 Key: NIFI-105
 URL: https://issues.apache.org/jira/browse/NIFI-105
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman

 Currently, this Port is very CPU-intensive if there's no data to send because 
 it's continually checking for data.
 It's important that we consider both SEND and RECEIVE ports...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-131) Require Terms of Use acknowledgement

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-131:
-
Issue Type: New Feature  (was: Improvement)

 Require Terms of Use acknowledgement
 

 Key: NIFI-131
 URL: https://issues.apache.org/jira/browse/NIFI-131
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Core UI
Reporter: Matt Gilman
Priority: Minor





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r1643898 - /incubator/nifi/site/trunk/lib/path.pm

[tkurc]

Author: tkurc
Date: Mon Dec  8 19:43:12 2014
New Revision: 1643898

URL: http://svn.apache.org/r1643898
Log:
comment change to try to force cms to rebuild

Modified:
incubator/nifi/site/trunk/lib/path.pm

Modified: incubator/nifi/site/trunk/lib/path.pm
URL: 
http://svn.apache.org/viewvc/incubator/nifi/site/trunk/lib/path.pm?rev=1643898r1=1643897r2=1643898view=diff
==
--- incubator/nifi/site/trunk/lib/path.pm (original)
+++ incubator/nifi/site/trunk/lib/path.pm Mon Dec  8 19:43:12 2014
@@ -1,6 +1,5 @@
 package path;
 
-#
 # taken from django's url.py
 
 our @patterns = (

[30/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.js
--
diff --git 
a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.js
 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.js
new file mode 100644
index 000..f5d3ecc
--- /dev/null
+++ 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.js
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+with (Scripting) {
+
+var instance = new ReaderScript({
+route: function (input) {
+str = IOUtils.toString(input);
+if (str.match(/sed do/i)) {
+return Script.FAIL_RELATIONSHIP;
+} else {
+return Script.SUCCESS_RELATIONSHIP;
+}
+}
+});
+logger.debug(Got a logger and properties + properties);
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.py
--
diff --git 
a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.py
 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.py
new file mode 100644
index 000..91e6ca7
--- /dev/null
+++ 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.py
@@ -0,0 +1,32 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the License); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an AS IS BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os, re, subprocess
+
+class SimpleJythonReader(ReaderScript):
+def route(self, input):
+logger.info(In route)
+returnid = os.system(c:\\cygwin\\bin\\echo GOOD)
+fname = self.getAttribute(filename)
+counter = self.getAttribute(counter)
+fname = fname + '.' + counter
+self.setAttribute(filename, fname)
+for line in FileUtil.wrap(input):
+if re.match(^sed,line): return self.FAIL_RELATIONSHIP
+
+return self.SUCCESS_RELATIONSHIP
+
+
+
+instance = SimpleJythonReader()

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.rb
--
diff --git 
a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.rb
 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.rb
new file mode 100644
index 000..c10765d
--- /dev/null
+++ 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/readTest.rb
@@ -0,0 +1,30 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the License); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an AS IS BASIS,
+# WITHOUT 

[28/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/UserService.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/UserService.java
 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/UserService.java
new file mode 100644
index 000..76e54d6
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/UserService.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.admin.service;
+
+import java.util.Collection;
+import java.util.Set;
+import org.apache.nifi.authorization.Authority;
+import org.apache.nifi.user.NiFiUser;
+import org.apache.nifi.user.NiFiUserGroup;
+
+/**
+ * Manages NiFi user accounts.
+ */
+public interface UserService {
+
+/**
+ * Creates a new user account using the specified dn and justification.
+ *
+ * @param dn
+ * @param justification
+ * @return
+ */
+NiFiUser createPendingUserAccount(String dn, String justification);
+
+/**
+ * Determines if there are any PENDING user accounts present.
+ *
+ * @return
+ */
+Boolean hasPendingUserAccount();
+
+/**
+ * Updates a user group using the specified group comprised of the 
specified
+ * users. Returns all the users that are currently in the specified group.
+ *
+ * @param group
+ * @param userIds
+ * @param authorities
+ * @return
+ */
+NiFiUserGroup updateGroup(String group, SetString userIds, 
SetAuthority authorities);
+
+/**
+ * Authorizes the user specified.
+ *
+ * @param dn
+ * @return
+ */
+NiFiUser checkAuthorization(String dn);
+
+/**
+ * Deletes the user with the specified id.
+ *
+ * @param id
+ */
+void deleteUser(String id);
+
+/**
+ * Disables the specified users account.
+ *
+ * @param id
+ * @return
+ */
+NiFiUser disable(String id);
+
+/**
+ * Disables the specified user group.
+ *
+ * @param group
+ * @return
+ */
+NiFiUserGroup disableGroup(String group);
+
+/**
+ * Updates the specified user with the specified authorities.
+ *
+ * @param id
+ * @param authorities
+ * @return
+ */
+NiFiUser update(String id, SetAuthority authorities);
+
+/**
+ * Invalidates the specified user account.
+ *
+ * @param id
+ */
+void invalidateUserAccount(String id);
+
+/**
+ * Invalidates the user accounts associated with the specified user group.
+ *
+ * @param group
+ */
+void invalidateUserGroupAccount(String group);
+
+/**
+ * Ungroups the specified group.
+ *
+ * @param group
+ */
+void ungroup(String group);
+
+/**
+ * Ungroups the specified user.
+ *
+ * @param id
+ */
+void ungroupUser(String id);
+
+/**
+ * Returns a collection of all NiFiUsers.
+ *
+ * @return
+ */
+CollectionNiFiUser getUsers();
+
+/**
+ * Finds the specified user by id.
+ *
+ * @param id
+ * @return
+ */
+NiFiUser getUserById(String id);
+
+/**
+ * Finds the specified user by dn.
+ *
+ * @param dn
+ * @return
+ * @throws AdministrationException
+ */
+NiFiUser getUserByDn(String dn);
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/AbstractUserAction.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/AbstractUserAction.java
 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/AbstractUserAction.java
new file mode 100644
index 000..41c97fe
--- /dev/null
+++ 

[03/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/FlowFromDOMFactory.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/FlowFromDOMFactory.java
 
b/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/FlowFromDOMFactory.java
new file mode 100644
index 000..c67181a
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/FlowFromDOMFactory.java
@@ -0,0 +1,418 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.connectable.Size;
+import org.apache.nifi.encrypt.StringEncryptor;
+import org.apache.nifi.groups.RemoteProcessGroupPortDescriptor;
+import org.apache.nifi.remote.StandardRemoteProcessGroupPortDescriptor;
+import org.apache.nifi.scheduling.SchedulingStrategy;
+import org.apache.nifi.util.DomUtils;
+import org.apache.nifi.web.api.dto.ConnectableDTO;
+import org.apache.nifi.web.api.dto.ConnectionDTO;
+import org.apache.nifi.web.api.dto.FlowSnippetDTO;
+import org.apache.nifi.web.api.dto.FunnelDTO;
+import org.apache.nifi.web.api.dto.LabelDTO;
+import org.apache.nifi.web.api.dto.PortDTO;
+import org.apache.nifi.web.api.dto.PositionDTO;
+import org.apache.nifi.web.api.dto.ProcessGroupDTO;
+import org.apache.nifi.web.api.dto.ProcessorConfigDTO;
+import org.apache.nifi.web.api.dto.ProcessorDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupDTO;
+
+import org.w3c.dom.Element;
+import org.w3c.dom.NodeList;
+
+public class FlowFromDOMFactory {
+
+public static PositionDTO getPosition(final Element positionElement) {
+if (positionElement == null) {
+throw new IllegalArgumentException(Invalid Flow: Found no 
'position' element);
+}
+return new 
PositionDTO(Double.parseDouble(positionElement.getAttribute(x)), 
Double.parseDouble(positionElement.getAttribute(y)));
+}
+
+public static Size getSize(final Element sizeElement) {
+if (sizeElement == null) {
+throw new IllegalArgumentException(Invalid Flow: Found no 'size' 
element);
+}
+
+return new Size(Double.parseDouble(sizeElement.getAttribute(width)), 
Double.parseDouble(sizeElement.getAttribute(height)));
+}
+
+public static MapString, String getStyle(final Element stylesElement) {
+final MapString, String styles = new HashMap();
+if (stylesElement == null) {
+return styles;
+}
+
+for (final Element styleElement : getChildrenByTagName(stylesElement, 
style)) {
+final String styleName = styleElement.getAttribute(name);
+final String styleValue = styleElement.getTextContent();
+styles.put(styleName, styleValue);
+}
+
+return styles;
+}
+
+public static ProcessGroupDTO getProcessGroup(final String parentId, final 
Element element, final StringEncryptor encryptor) {
+final ProcessGroupDTO dto = new ProcessGroupDTO();
+final String groupId = getString(element, id);
+dto.setId(groupId);
+dto.setParentGroupId(parentId);
+dto.setName(getString(element, name));
+dto.setPosition(getPosition(DomUtils.getChild(element, position)));
+dto.setComments(getString(element, comment));
+
+final SetProcessorDTO processors = new HashSet();
+final SetConnectionDTO connections = new HashSet();
+final SetFunnelDTO funnels = new HashSet();
+final SetPortDTO inputPorts = new HashSet();
+final SetPortDTO outputPorts = new HashSet();
+final SetLabelDTO labels = new HashSet();
+final SetProcessGroupDTO processGroups = new HashSet();
+final SetRemoteProcessGroupDTO remoteProcessGroups = new HashSet();
+
+final FlowSnippetDTO 

[31/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html
--
diff --git 
a/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html
 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html
new file mode 100644
index 000..acb47c5
--- /dev/null
+++ 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html
@@ -0,0 +1,264 @@
+!DOCTYPE html
+html lang=en
+!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the License); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+  http://www.apache.org/licenses/LICENSE-2.0
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an AS IS BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+--
+head
+meta charset=utf-8 /
+titleExecuteScript/title
+
+link rel=stylesheet href=../../css/component-usage.css 
type=text/css /
+/head
+
+body
+!-- Processor Documentation 
== --
+h2Description:/h2
+p
+This processor provides the capability to execute scripts in 
various scripting languages, and passes into the scripts 
+the input stream and output stream(s) representing an incoming 
flow file and any created flow files. The processor is designed to be
+thread safe, so multiple concurrent tasks may execute against a 
single script. The processor provides a framework which enables 
+script writers to implement 3 different types of scripts:
+ul
+ReaderScript - which enables stream-based reading of a FlowFile's 
content/br
+WriterScript - which enables stream-based reading and 
writing/modifying of a FlowFile's content/br
+ConverterScript - which enables stream-based reading a FlowFile's 
content and stream-based writing to newly created FlowFiles/br
+/ul
+Presently, the processor supports 3 scripting languages: Ruby, Python, 
and JavaScript. The processor is built on the 
+javax.script API which enables ScriptEngine discovery, thread 
management, and encapsulates much of the low level bridging-code that 
+enables Java to Script language integration. Thus, it is designed to 
be easily extended to other scripting languages. /br
+The attributes of a FlowFile and properties of the Processor are 
exposed to the script by either a variable in the base class or 
+a getter method. A script may declare new Processor Properties and 
different Relationships via overriding the getPropertyDescriptors 
+and getRelationships methods, respectively. 
+/p
+The processor provides some boilerplate script to aid in the creation of 
the three different types of scripts. For example,
+the processor provides import statements for classes commonly used within 
a processor.
+pre
+'org.apache.nifi.components.PropertyDescriptor'
+'org.apache.nifi.components.Validator'
+'org.apache.nifi.processor.util.StandardValidators'
+'org.apache.nifi.processor.Relationship'
+'org.apache.nifi.logging.ProcessorLog'
+'org.apache.nifi.scripting.ReaderScript'
+'org.apache.nifi.scripting.WriterScript'
+'org.apache.nifi.scripting.ConverterScript'
+/pre
+The processor appends to the script's execution path the parent directory 
of the specified script file and a sub-directory
+called 'lib', which may be useful for supporting scripts. /p
+p
+strongShared Variables/strong
+/p
+The following variables are provided as shared variables for the scripts:
+ul
+lilogger 
+ul
+li The processor's logger /li
+li Scope is GLOBAL, thus in Ruby the syntax is $logger/li
+/ul
+/li
+liproperties
+ul
+li A Map of the processor's configuration properties; key and 
value are strings/li
+li Scope is INSTANCE, thus in Ruby the 

[13/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/DisconnectedNodeMutableRequestException.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/DisconnectedNodeMutableRequestException.java
 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/DisconnectedNodeMutableRequestException.java
new file mode 100644
index 000..412a555
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/DisconnectedNodeMutableRequestException.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.manager.exception;
+
+/**
+ * Represents the exceptional case when a HTTP request that may change a node's
+ * dataflow is to be replicated while one or more nodes are disconnected.
+ *
+ * @author unattributed
+ */
+public class DisconnectedNodeMutableRequestException extends 
MutableRequestException {
+
+public DisconnectedNodeMutableRequestException() {
+}
+
+public DisconnectedNodeMutableRequestException(String msg) {
+super(msg);
+}
+
+public DisconnectedNodeMutableRequestException(Throwable cause) {
+super(cause);
+}
+
+public DisconnectedNodeMutableRequestException(String msg, Throwable 
cause) {
+super(msg, cause);
+}
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/IllegalClusterStateException.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/IllegalClusterStateException.java
 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/IllegalClusterStateException.java
new file mode 100644
index 000..6c4e670
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/IllegalClusterStateException.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.manager.exception;
+
+/**
+ * Signals that an operation to be performed on a cluster has been invoked at 
an
+ * illegal or inappropriate time.
+ *
+ * @author unattributed
+ */
+public class IllegalClusterStateException extends ClusterException {
+
+public IllegalClusterStateException() {
+}
+
+public IllegalClusterStateException(String msg) {
+super(msg);
+}
+
+public IllegalClusterStateException(Throwable cause) {
+super(cause);
+}
+
+public IllegalClusterStateException(String msg, Throwable cause) {
+super(msg, cause);
+}
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/exception/IllegalNodeDeletionException.java
--
diff --git 

[44/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/NumberLiteralEvaluator.java
--
diff --git 
a/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/NumberLiteralEvaluator.java
 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/NumberLiteralEvaluator.java
new file mode 100644
index 000..d7569e0
--- /dev/null
+++ 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/NumberLiteralEvaluator.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.literals;
+
+import java.util.Map;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
+import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+
+public class NumberLiteralEvaluator extends NumberEvaluator {
+
+private final long literal;
+
+public NumberLiteralEvaluator(final String value) {
+this.literal = Long.parseLong(value);
+}
+
+@Override
+public QueryResultLong evaluate(final MapString, String attributes) {
+return new NumberQueryResult(literal);
+}
+
+@Override
+public Evaluator? getSubjectEvaluator() {
+return null;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/StringLiteralEvaluator.java
--
diff --git 
a/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/StringLiteralEvaluator.java
 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/StringLiteralEvaluator.java
new file mode 100644
index 000..d739ac7
--- /dev/null
+++ 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/literals/StringLiteralEvaluator.java
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.literals;
+
+import java.util.Map;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import 
org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+
+public class StringLiteralEvaluator extends StringEvaluator {
+
+private final String value;
+
+public StringLiteralEvaluator(final String value) {
+// need to escape characters after backslashes
+final StringBuilder sb = new StringBuilder();
+boolean lastCharIsBackslash = false;
+for (int i = 0; i  value.length(); i++) {
+final char c = value.charAt(i);
+
+if (lastCharIsBackslash) {
+ 

[37/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/remote-communications-utils/src/main/java/org/apache/nifi/remote/io/CompressionOutputStream.java
--
diff --git 
a/commons/remote-communications-utils/src/main/java/org/apache/nifi/remote/io/CompressionOutputStream.java
 
b/commons/remote-communications-utils/src/main/java/org/apache/nifi/remote/io/CompressionOutputStream.java
new file mode 100644
index 000..bc46b0f
--- /dev/null
+++ 
b/commons/remote-communications-utils/src/main/java/org/apache/nifi/remote/io/CompressionOutputStream.java
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.remote.io;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.zip.Deflater;
+
+public class CompressionOutputStream extends OutputStream {
+
+public static final byte[] SYNC_BYTES = new byte[]{'S', 'Y', 'N', 'C'};
+
+public static final int DEFAULT_COMPRESSION_LEVEL = 1;
+public static final int DEFAULT_BUFFER_SIZE = 64  10;
+public static final int MIN_BUFFER_SIZE = 8  10;
+
+private final OutputStream out;
+private final Deflater deflater;
+
+private final byte[] buffer;
+private final byte[] compressed;
+
+private int bufferIndex = 0;
+private boolean dataWritten = false;
+
+public CompressionOutputStream(final OutputStream outStream) {
+this(outStream, DEFAULT_BUFFER_SIZE);
+}
+
+public CompressionOutputStream(final OutputStream outStream, final int 
bufferSize) {
+this(outStream, bufferSize, DEFAULT_COMPRESSION_LEVEL, 
Deflater.DEFAULT_STRATEGY);
+}
+
+public CompressionOutputStream(final OutputStream outStream, final int 
bufferSize, final int level, final int strategy) {
+if (bufferSize  MIN_BUFFER_SIZE) {
+throw new IllegalArgumentException(Buffer size must be at least  
+ MIN_BUFFER_SIZE);
+}
+
+this.out = outStream;
+this.deflater = new Deflater(level);
+this.deflater.setStrategy(strategy);
+buffer = new byte[bufferSize];
+compressed = new byte[bufferSize + 64];
+}
+
+/**
+ * Compresses the currently buffered chunk of data and sends it to the
+ * output stream
+ *
+ * @throws IOException
+ */
+protected void compressAndWrite() throws IOException {
+if (bufferIndex = 0) {
+return;
+}
+
+deflater.setInput(buffer, 0, bufferIndex);
+deflater.finish();
+final int compressedBytes = deflater.deflate(compressed);
+
+writeChunkHeader(compressedBytes);
+out.write(compressed, 0, compressedBytes);
+
+bufferIndex = 0;
+deflater.reset();
+}
+
+private void writeChunkHeader(final int compressedBytes) throws 
IOException {
+// If we have already written data, write out a '1' to indicate that 
we have more data; when we close
+// the stream, we instead write a '0' to indicate that we are finished 
sending data.
+if (dataWritten) {
+out.write(1);
+}
+out.write(SYNC_BYTES);
+dataWritten = true;
+
+writeInt(out, bufferIndex);
+writeInt(out, compressedBytes);
+}
+
+private void writeInt(final OutputStream out, final int val) throws 
IOException {
+out.write(val  24);
+out.write(val  16);
+out.write(val  8);
+out.write(val);
+}
+
+protected boolean bufferFull() {
+return bufferIndex = buffer.length;
+}
+
+@Override
+public void write(final int b) throws IOException {
+buffer[bufferIndex++] = (byte) (b  0xFF);
+if (bufferFull()) {
+compressAndWrite();
+}
+}
+
+@Override
+public void write(final byte[] b) throws IOException {
+write(b, 0, b.length);
+}
+
+@Override
+public void write(final byte[] b, final int off, final int len) throws 
IOException {
+int bytesLeft = len;
+while (bytesLeft  0) {
+final int free = buffer.length - bufferIndex;
+final int bytesThisIteration = Math.min(bytesLeft, free);
+

[47/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/Query.java
--
diff --git 
a/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/Query.java
 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/Query.java
new file mode 100644
index 000..6d9cc91
--- /dev/null
+++ 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/Query.java
@@ -0,0 +1,1183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language;
+
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ALL_ATTRIBUTES;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ALL_DELINEATED_VALUES;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ALL_MATCHING_ATTRIBUTES;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.AND;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ANY_ATTRIBUTE;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ANY_DELINEATED_VALUE;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ANY_MATCHING_ATTRIBUTE;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.APPEND;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ATTRIBUTE_REFERENCE;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ATTR_NAME;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.CONTAINS;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.DIVIDE;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.ENDS_WITH;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.EQUALS;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.EQUALS_IGNORE_CASE;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.EXPRESSION;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.FALSE;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.FIND;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.FORMAT;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.GREATER_THAN;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.GREATER_THAN_OR_EQUAL;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.HOSTNAME;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.INDEX_OF;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.IP;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.IS_NULL;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.LAST_INDEX_OF;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.LENGTH;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.LESS_THAN;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.LESS_THAN_OR_EQUAL;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.MATCHES;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.MINUS;
+import static 
org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser.MOD;
+import static 

[45/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ModEvaluator.java
--
diff --git 
a/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ModEvaluator.java
 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ModEvaluator.java
new file mode 100644
index 000..6b7b838
--- /dev/null
+++ 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ModEvaluator.java
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import java.util.Map;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
+import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+
+public class ModEvaluator extends NumberEvaluator {
+
+private final NumberEvaluator subject;
+private final NumberEvaluator modValue;
+
+public ModEvaluator(final NumberEvaluator subject, final NumberEvaluator 
modValue) {
+this.subject = subject;
+this.modValue = modValue;
+}
+
+@Override
+public QueryResultLong evaluate(final MapString, String attributes) {
+final Long subjectValue = subject.evaluate(attributes).getValue();
+if (subjectValue == null) {
+return new NumberQueryResult(null);
+}
+
+final Long mod = modValue.evaluate(attributes).getValue();
+if (mod == null) {
+return new NumberQueryResult(null);
+}
+
+final long result = subjectValue % mod;
+return new NumberQueryResult(result);
+}
+
+@Override
+public Evaluator? getSubjectEvaluator() {
+return subject;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/MultiplyEvaluator.java
--
diff --git 
a/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/MultiplyEvaluator.java
 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/MultiplyEvaluator.java
new file mode 100644
index 000..a3b03ed
--- /dev/null
+++ 
b/commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/MultiplyEvaluator.java
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import java.util.Map;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
+import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+
+public class 

[02/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowService.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowService.java
 
b/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowService.java
new file mode 100644
index 000..1443737
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowService.java
@@ -0,0 +1,875 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.InetSocketAddress;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.nio.file.StandardOpenOption;
+import java.util.Calendar;
+import java.util.Date;
+import java.util.UUID;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.zip.GZIPInputStream;
+import java.util.zip.GZIPOutputStream;
+
+import org.apache.nifi.cluster.ConnectionException;
+import org.apache.nifi.cluster.protocol.ConnectionRequest;
+import org.apache.nifi.cluster.protocol.ConnectionResponse;
+import org.apache.nifi.cluster.protocol.DataFlow;
+import org.apache.nifi.cluster.protocol.NodeIdentifier;
+import org.apache.nifi.cluster.protocol.ProtocolException;
+import org.apache.nifi.cluster.protocol.ProtocolHandler;
+import org.apache.nifi.cluster.protocol.StandardDataFlow;
+import org.apache.nifi.cluster.protocol.UnknownServiceAddressException;
+import org.apache.nifi.cluster.protocol.impl.NodeProtocolSenderListener;
+import org.apache.nifi.cluster.protocol.message.ConnectionRequestMessage;
+import 
org.apache.nifi.cluster.protocol.message.ControllerStartupFailureMessage;
+import org.apache.nifi.cluster.protocol.message.DisconnectMessage;
+import org.apache.nifi.cluster.protocol.message.FlowRequestMessage;
+import org.apache.nifi.cluster.protocol.message.FlowResponseMessage;
+import org.apache.nifi.cluster.protocol.message.PrimaryRoleAssignmentMessage;
+import org.apache.nifi.cluster.protocol.message.ProtocolMessage;
+import org.apache.nifi.cluster.protocol.message.ReconnectionFailureMessage;
+import org.apache.nifi.cluster.protocol.message.ReconnectionRequestMessage;
+import org.apache.nifi.cluster.protocol.message.ReconnectionResponseMessage;
+import org.apache.nifi.engine.FlowEngine;
+import org.apache.nifi.events.BulletinFactory;
+import org.apache.nifi.file.FileUtils;
+import org.apache.nifi.groups.ProcessGroup;
+import org.apache.nifi.lifecycle.LifeCycleStartException;
+import org.apache.nifi.logging.LogLevel;
+import org.apache.nifi.persistence.FlowConfigurationDAO;
+import org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO;
+import org.apache.nifi.reporting.Bulletin;
+import org.apache.nifi.services.FlowService;
+import org.apache.nifi.util.FormatUtils;
+import org.apache.nifi.util.NiFiProperties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.encrypt.StringEncryptor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class StandardFlowService implements FlowService, ProtocolHandler {
+
+private static final String EVENT_CATEGORY = Controller;
+
+private final FlowController controller;
+private final Path flowXml;
+private final Path taskConfigXml;
+private final Path serviceConfigXml;
+private final FlowConfigurationDAO dao;
+private final int gracefulShutdownSeconds;
+private final boolean autoResumeState;
+private final int connectionRetryMillis;
+private final StringEncryptor encryptor;
+
+// Lock is used to protect the flow.xml file.
+private final ReentrantReadWriteLock rwLock 

[01/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

Repository: incubator-nifi
Updated Branches:
  refs/heads/master [created] 4d998c12c


http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowSynchronizer.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowSynchronizer.java
 
b/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowSynchronizer.java
new file mode 100644
index 000..17a1702
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core/src/main/java/org/apache/nifi/controller/StandardFlowSynchronizer.java
@@ -0,0 +1,1026 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardOpenOption;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.zip.GZIPInputStream;
+
+import javax.xml.XMLConstants;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.validation.Schema;
+import javax.xml.validation.SchemaFactory;
+
+import org.apache.nifi.cluster.protocol.DataFlow;
+import org.apache.nifi.cluster.protocol.StandardDataFlow;
+import org.apache.nifi.connectable.Connectable;
+import org.apache.nifi.connectable.ConnectableType;
+import org.apache.nifi.connectable.Connection;
+import org.apache.nifi.connectable.Funnel;
+import org.apache.nifi.connectable.Port;
+import org.apache.nifi.connectable.Position;
+import org.apache.nifi.connectable.Size;
+import org.apache.nifi.controller.exception.ProcessorInstantiationException;
+import org.apache.nifi.controller.label.Label;
+import org.apache.nifi.events.BulletinFactory;
+import org.apache.nifi.file.FileUtils;
+import org.apache.nifi.fingerprint.FingerprintException;
+import org.apache.nifi.fingerprint.FingerprintFactory;
+import org.apache.nifi.flowfile.FlowFilePrioritizer;
+import org.apache.nifi.groups.ProcessGroup;
+import org.apache.nifi.groups.RemoteProcessGroup;
+import org.apache.nifi.groups.RemoteProcessGroupPortDescriptor;
+import org.apache.nifi.logging.LogLevel;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.remote.RemoteGroupPort;
+import org.apache.nifi.remote.RootGroupPort;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.scheduling.SchedulingStrategy;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.web.api.dto.ConnectableDTO;
+import org.apache.nifi.web.api.dto.ConnectionDTO;
+import org.apache.nifi.web.api.dto.FlowSnippetDTO;
+import org.apache.nifi.web.api.dto.FunnelDTO;
+import org.apache.nifi.web.api.dto.LabelDTO;
+import org.apache.nifi.web.api.dto.PortDTO;
+import org.apache.nifi.web.api.dto.PositionDTO;
+import org.apache.nifi.web.api.dto.ProcessGroupDTO;
+import org.apache.nifi.web.api.dto.ProcessorConfigDTO;
+import org.apache.nifi.web.api.dto.ProcessorDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupDTO;
+
+import org.apache.commons.collections4.CollectionUtils;
+import org.apache.nifi.encrypt.StringEncryptor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+import org.w3c.dom.NodeList;
+import org.xml.sax.SAXException;
+
+/**
+ * @author unattributed
+ */
+public class StandardFlowSynchronizer implements FlowSynchronizer {
+
+private static final Logger logger = 
LoggerFactory.getLogger(StandardFlowSynchronizer.class);
+public static final URL FLOW_XSD_RESOURCE = 
StandardFlowSynchronizer.class.getResource(/FlowConfiguration.xsd);
+private final StringEncryptor encryptor;
+
+public 

[48/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackagerV2.java
--
diff --git 
a/commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackagerV2.java
 
b/commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackagerV2.java
new file mode 100644
index 000..500015f
--- /dev/null
+++ 
b/commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackagerV2.java
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.util;
+
+import java.io.EOFException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FlowFileUnpackagerV2 implements FlowFileUnpackager {
+
+private final byte readBuffer[] = new byte[8192];
+private MapString, String nextAttributes = null;
+private boolean haveReadSomething = false;
+
+@Override
+public boolean hasMoreData() throws IOException {
+return nextAttributes != null || !haveReadSomething;
+}
+
+protected MapString, String readAttributes(final InputStream in) throws 
IOException {
+final MapString, String attributes = new HashMap();
+final Integer numAttributes = readFieldLength(in); //read number of 
attributes
+if (numAttributes == null) {
+return null;
+}
+if (numAttributes == 0) {
+throw new IOException(flow files cannot have zero attributes);
+}
+for (int i = 0; i  numAttributes; i++) { //read each attribute 
key/value pair
+final String key = readString(in);
+final String value = readString(in);
+attributes.put(key, value);
+}
+
+return attributes;
+}
+
+@Override
+public MapString, String unpackageFlowFile(final InputStream in, final 
OutputStream out) throws IOException {
+final MapString, String attributes;
+if (nextAttributes != null) {
+attributes = nextAttributes;
+} else {
+attributes = readAttributes(in);
+}
+
+final long expectedNumBytes = readLong(in); // read length of payload
+copy(in, out, expectedNumBytes); // read payload
+
+nextAttributes = readAttributes(in);
+haveReadSomething = true;
+
+return attributes;
+}
+
+protected String readString(final InputStream in) throws IOException {
+final Integer numBytes = readFieldLength(in);
+if (numBytes == null) {
+throw new EOFException();
+}
+final byte[] bytes = new byte[numBytes];
+fillBuffer(in, bytes, numBytes);
+return new String(bytes, UTF-8);
+}
+
+private void fillBuffer(final InputStream in, final byte[] buffer, final 
int length) throws IOException {
+int bytesRead;
+int totalBytesRead = 0;
+while ((bytesRead = in.read(buffer, totalBytesRead, length - 
totalBytesRead))  0) {
+totalBytesRead += bytesRead;
+}
+if (totalBytesRead != length) {
+throw new EOFException();
+}
+}
+
+protected long copy(final InputStream in, final OutputStream out, final 
long numBytes) throws IOException {
+int bytesRead;
+long totalBytesRead = 0L;
+while ((bytesRead = in.read(readBuffer, 0, (int) 
Math.min(readBuffer.length, numBytes - totalBytesRead)))  0) {
+out.write(readBuffer, 0, bytesRead);
+totalBytesRead += bytesRead;
+}
+
+if (totalBytesRead  numBytes) {
+throw new EOFException(Expected  + numBytes +  but received  + 
totalBytesRead);
+}
+
+return totalBytesRead;
+}
+
+protected long readLong(final InputStream in) throws IOException {
+fillBuffer(in, readBuffer, 8);
+return (((long) readBuffer[0]  56)
++ ((long) (readBuffer[1]  255)  48)
++ ((long) (readBuffer[2]  255)  40)
++ ((long) (readBuffer[3]  255)  32)
++ ((long) 

[38/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-utils/src/test/java/org/apache/nifi/util/timebuffer/TestTimedBuffer.java
--
diff --git 
a/commons/nifi-utils/src/test/java/org/apache/nifi/util/timebuffer/TestTimedBuffer.java
 
b/commons/nifi-utils/src/test/java/org/apache/nifi/util/timebuffer/TestTimedBuffer.java
new file mode 100644
index 000..39ca330
--- /dev/null
+++ 
b/commons/nifi-utils/src/test/java/org/apache/nifi/util/timebuffer/TestTimedBuffer.java
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.util.timebuffer;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNull;
+
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+
+public class TestTimedBuffer {
+
+@Test
+public void testAgesOff() throws InterruptedException {
+final LongEntityAccess access = new LongEntityAccess();
+final TimedBufferTimestampedLong buffer = new 
TimedBuffer(TimeUnit.SECONDS, 2, access);
+
+buffer.add(new TimestampedLong(100L));
+TimestampedLong aggregate = 
buffer.getAggregateValue(System.currentTimeMillis() - 3L);
+assertEquals(100L, aggregate.getValue().longValue());
+Thread.sleep(1000L);
+aggregate = buffer.getAggregateValue(System.currentTimeMillis() - 
3L);
+assertEquals(100L, aggregate.getValue().longValue());
+Thread.sleep(1500L);
+aggregate = buffer.getAggregateValue(System.currentTimeMillis() - 
3L);
+assertNull(aggregate);
+}
+
+@Test
+public void testAggregation() throws InterruptedException {
+final LongEntityAccess access = new LongEntityAccess();
+final TimedBufferTimestampedLong buffer = new 
TimedBuffer(TimeUnit.SECONDS, 2, access);
+
+buffer.add(new TimestampedLong(100L));
+buffer.add(new TimestampedLong(100L));
+buffer.add(new TimestampedLong(25000L));
+
+TimestampedLong aggregate = 
buffer.getAggregateValue(System.currentTimeMillis() - 3L);
+assertEquals(2025000L, aggregate.getValue().longValue());
+Thread.sleep(1000L);
+aggregate = buffer.getAggregateValue(System.currentTimeMillis() - 
3L);
+assertEquals(2025000L, aggregate.getValue().longValue());
+Thread.sleep(1500L);
+aggregate = buffer.getAggregateValue(System.currentTimeMillis() - 
3L);
+assertNull(aggregate);
+}
+
+private static class TimestampedLong {
+
+private final Long value;
+private final long timestamp = System.currentTimeMillis();
+
+public TimestampedLong(final Long value) {
+this.value = value;
+}
+
+public Long getValue() {
+return value;
+}
+
+public long getTimestamp() {
+return timestamp;
+}
+}
+
+private static class LongEntityAccess implements 
EntityAccessTimestampedLong {
+
+@Override
+public TimestampedLong aggregate(TimestampedLong oldValue, 
TimestampedLong toAdd) {
+if (oldValue == null  toAdd == null) {
+return new TimestampedLong(0L);
+} else if (oldValue == null) {
+return toAdd;
+} else if (toAdd == null) {
+return oldValue;
+}
+
+return new TimestampedLong(oldValue.getValue().longValue() + 
toAdd.getValue().longValue());
+}
+
+@Override
+public TimestampedLong createNew() {
+return new TimestampedLong(0L);
+}
+
+@Override
+public long getTimestamp(TimestampedLong entity) {
+return entity == null ? 0L : entity.getTimestamp();
+}
+}
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-web-utils/pom.xml
--
diff --git a/commons/nifi-web-utils/pom.xml b/commons/nifi-web-utils/pom.xml
new file mode 100644
index 000..434e1a3
--- /dev/null
+++ b/commons/nifi-web-utils/pom.xml
@@ -0,0 +1,56 @@
+?xml version=1.0 

[09/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/connectable/Connectable.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/connectable/Connectable.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/connectable/Connectable.java
new file mode 100644
index 000..3d5c75d
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/connectable/Connectable.java
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.connectable;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.Triggerable;
+import org.apache.nifi.groups.ProcessGroup;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.scheduling.SchedulingStrategy;
+
+/**
+ * Represents a connectable component to which or from which data can flow.
+ */
+public interface Connectable extends Triggerable {
+
+/**
+ * @return the unique identifier for this codeConnectable/code
+ */
+String getIdentifier();
+
+/**
+ * @return a Collection of all relationships for this Connectable
+ */
+CollectionRelationship getRelationships();
+
+/**
+ * Returns the ProcessorRelationship whose name is given
+ *
+ * @param relationshipName
+ * @return a ProcessorRelationship whose name is given, or 
codenull/code
+ * if none exists
+ */
+Relationship getRelationship(String relationshipName);
+
+/**
+ * Adds the given connection to this Connectable.
+ *
+ * @param connection the connection to add
+ * @throws NullPointerException if the argument is null
+ * @throws IllegalArgumentException if the given Connection is not valid
+ */
+void addConnection(Connection connection) throws IllegalArgumentException;
+
+/**
+ * @return true if the Connectable is the destination of any other
+ * Connectable, false otherwise.
+ */
+boolean hasIncomingConnection();
+
+/**
+ *
+ * @param connection
+ * @throws IllegalStateException if the given Connection is not registered
+ * to codethis/code.
+ */
+void removeConnection(Connection connection) throws IllegalStateException;
+
+/**
+ * Updates any internal state that depends on the given connection. The
+ * given connection will share the same ID as the old connection.
+ *
+ * @param newConnection
+ * @throws IllegalStateException
+ */
+void updateConnection(Connection newConnection) throws 
IllegalStateException;
+
+/**
+ * @return a codeSet/code of all codeConnection/codes for which 
this
+ * codeConnectable/code is the destination
+ */
+ListConnection getIncomingConnections();
+
+/**
+ * @return a codeSet/code of all codeConnection/codes for which 
this
+ * codeConnectable/code is the source; if no connections exist, will
+ * return an empty Collection. Guaranteed not null.
+ */
+SetConnection getConnections();
+
+/**
+ * @param relationship
+ * @return a codeSet/code of all codeConnection/codes that contain
+ * the given relationship for which this codeConnectable/code is the
+ * source
+ */
+SetConnection getConnections(Relationship relationship);
+
+/**
+ * Returns the position on the graph where this Connectable is located
+ *
+ * @return
+ */
+Position getPosition();
+
+/**
+ * Updates this component's position on the graph
+ *
+ * @param position
+ */
+void setPosition(Position position);
+
+/**
+ * @return the name of this Connectable
+ */
+String getName();
+
+/**
+ * Sets the name of this Connectable so that its name will be visible on 
the
+ * UI
+ * @param name
+ */
+void setName(String name);
+
+/**
+ * 

[51/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

Initial code contribution


Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/4d998c12
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/4d998c12
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/4d998c12

Branch: refs/heads/master
Commit: 4d998c12c95a6e5ce3d66c0d861e75e33b5cf013
Parents: 
Author: joewitt joew...@apache.org
Authored: Mon Dec 8 15:22:14 2014 -0500
Committer: joewitt joew...@apache.org
Committed: Mon Dec 8 15:22:14 2014 -0500

--
 .gitignore  |9 +
 DISCLAIMER  |   15 +
 LICENSE |  202 +
 NOTICE  |7 +
 README.md   |   65 +
 assemblies/nifi/pom.xml |  485 +
 .../nifi/src/main/assembly/dependencies.xml |  123 +
 commons/core-flowfile-attributes/pom.xml|   32 +
 .../flowfile/attributes/CoreAttributes.java |   72 +
 .../attributes/FlowFileAttributeKey.java|   21 +
 commons/data-provenance-utils/.gitignore|2 +
 commons/data-provenance-utils/pom.xml   |   42 +
 .../nifi/provenance/AsyncLineageSubmission.java |   87 +
 .../nifi/provenance/AsyncQuerySubmission.java   |   81 +
 .../nifi/provenance/NamedSearchableField.java   |   95 +
 .../nifi/provenance/SearchableFieldParser.java  |   53 +
 .../nifi/provenance/SearchableFields.java   |   84 +
 .../nifi/provenance/StandardLineageResult.java  |  324 +
 .../StandardProvenanceEventRecord.java  |  752 ++
 .../nifi/provenance/StandardQueryResult.java|  168 +
 .../nifi/provenance/lineage/EdgeNode.java   |   74 +
 .../nifi/provenance/lineage/EventNode.java  |  109 +
 .../provenance/lineage/FlowFileLineage.java |   76 +
 .../nifi/provenance/lineage/FlowFileNode.java   |   83 +
 commons/flowfile-packager/pom.xml   |   43 +
 .../org/apache/nifi/util/FlowFilePackager.java  |   28 +
 .../apache/nifi/util/FlowFilePackagerV1.java|  104 +
 .../apache/nifi/util/FlowFilePackagerV2.java|  146 +
 .../apache/nifi/util/FlowFilePackagerV3.java|   93 +
 .../apache/nifi/util/FlowFileUnpackager.java|   30 +
 .../apache/nifi/util/FlowFileUnpackagerV1.java  |  155 +
 .../apache/nifi/util/FlowFileUnpackagerV2.java  |  143 +
 .../apache/nifi/util/FlowFileUnpackagerV3.java  |  161 +
 .../java/org/apache/nifi/util/Unpackage.java|  119 +
 .../nifi/util/TestPackageUnpackageV3.java   |   56 +
 commons/naive-search-ring-buffer/pom.xml|   30 +
 .../apache/nifi/util/NaiveSearchRingBuffer.java |  135 +
 .../nifi/util/TestNaiveSearchRingBuffer.java|   72 +
 commons/nifi-expression-language/pom.xml|   59 +
 .../language/antlr/AttributeExpressionLexer.g   |  207 +
 .../language/antlr/AttributeExpressionParser.g  |  139 +
 .../output/AttributeExpressionLexer.tokens  |   72 +
 .../expression/language/EmptyPreparedQuery.java |   62 +
 .../expression/language/PreparedQuery.java  |   39 +
 .../attribute/expression/language/Query.java| 1183 +++
 .../language/StandardAttributeExpression.java   |   65 +
 .../StandardExpressionLanguageCompiler.java |   58 +
 .../language/StandardPreparedQuery.java |   83 +
 .../language/evaluation/BooleanEvaluator.java   |   32 +
 .../language/evaluation/BooleanQueryResult.java |   43 +
 .../language/evaluation/DateEvaluator.java  |   34 +
 .../language/evaluation/DateQueryResult.java|   45 +
 .../language/evaluation/Evaluator.java  |   32 +
 .../language/evaluation/NumberEvaluator.java|   33 +
 .../language/evaluation/NumberQueryResult.java  |   43 +
 .../language/evaluation/QueryResult.java|   26 +
 .../language/evaluation/StringEvaluator.java|   32 +
 .../language/evaluation/StringQueryResult.java  |   43 +
 .../evaluation/cast/BooleanCastEvaluator.java   |   50 +
 .../evaluation/cast/DateCastEvaluator.java  |  115 +
 .../evaluation/cast/NumberCastEvaluator.java|   72 +
 .../evaluation/cast/StringCastEvaluator.java|   49 +
 .../evaluation/functions/AndEvaluator.java  |   60 +
 .../evaluation/functions/AppendEvaluator.java   |   50 +
 .../functions/AttributeEvaluator.java   |   45 +
 .../evaluation/functions/ContainsEvaluator.java |   53 +
 .../functions/DateToNumberEvaluator.java|   50 +
 .../evaluation/functions/DivideEvaluator.java   |   57 +
 .../evaluation/functions/EndsWithEvaluator.java |   53 +
 .../evaluation/functions/EqualsEvaluator.java   |   89 +
 .../functions/EqualsIgnoreCaseEvaluator.java|   60 +
 .../evaluation/functions/FindEvaluator.java |   72 +
 .../evaluation/functions/FormatEvaluator.java   |   60 +
 .../functions/GreaterThanEvaluator.java |   60 +
 .../functions/GreaterThanOrEqualEvaluator.java  |   60 +
 

[32/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
--
diff --git 
a/nar-bundles/execute-script-bundle/execute-script-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
new file mode 100644
index 000..9058cf4
--- /dev/null
+++ 
b/nar-bundles/execute-script-bundle/execute-script-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
@@ -0,0 +1,566 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.script;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import javax.script.ScriptException;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.io.BufferedInputStream;
+import org.apache.nifi.io.BufferedOutputStream;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.annotation.CapabilityDescription;
+import org.apache.nifi.processor.annotation.EventDriven;
+import org.apache.nifi.processor.annotation.Tags;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.scripting.ConverterScript;
+import org.apache.nifi.scripting.ReaderScript;
+import org.apache.nifi.scripting.Script;
+import org.apache.nifi.scripting.ScriptFactory;
+import org.apache.nifi.scripting.WriterScript;
+
+/**
+ * !-- Processor Documentation 
== --
+ * h2Description:/h2
+ * p
+ * This processor provides the capability to execute scripts in various
+ * scripting languages, and passes into the scripts the input stream and output
+ * stream(s) representing an incoming flow file and any created flow files. The
+ * processor is designed to be thread safe, so multiple concurrent tasks may
+ * execute against a single script. The processor provides a framework which
+ * enables script writers to implement 3 different types of scripts:
+ * ul
+ * ReaderScript - which enables stream-based reading of a FlowFile's
+ * content/br WriterScript - which enables stream-based reading and
+ * writing/modifying of a FlowFile's content/br ConverterScript - which
+ * enables stream-based reading a FlowFile's content and stream-based writing 
to
+ * newly created FlowFiles/br
+ * /ul
+ * Presently, the processor supports 3 scripting languages: Ruby, Python, and
+ * JavaScript. The processor is built on the javax.script API which enables
+ * ScriptEngine discovery, thread management, and encapsulates much of the low
+ * level bridging-code that enables Java to Script language integration. Thus,
+ * it is designed to be easily extended to other scripting languages. /br The
+ * attributes of a FlowFile and properties of the Processor are exposed to the
+ * script by either a variable in the base class or a getter method. A script
+ * may declare new Processor Properties and different Relationships via
+ * overriding the getPropertyDescriptors and getRelationships 

[49/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/StandardLineageResult.java
--
diff --git 
a/commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/StandardLineageResult.java
 
b/commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/StandardLineageResult.java
new file mode 100644
index 000..afb56e8
--- /dev/null
+++ 
b/commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/StandardLineageResult.java
@@ -0,0 +1,324 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.provenance;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.nifi.provenance.lineage.ComputeLineageResult;
+import org.apache.nifi.provenance.lineage.EdgeNode;
+import org.apache.nifi.provenance.lineage.EventNode;
+import org.apache.nifi.provenance.lineage.FlowFileNode;
+import org.apache.nifi.provenance.lineage.LineageEdge;
+import org.apache.nifi.provenance.lineage.LineageNode;
+
+/**
+ *
+ */
+public class StandardLineageResult implements ComputeLineageResult {
+
+public static final int TTL = (int) TimeUnit.MILLISECONDS.convert(30, 
TimeUnit.MINUTES);
+private static final Logger logger = 
LoggerFactory.getLogger(StandardLineageResult.class);
+
+private final CollectionString flowFileUuids;
+private final CollectionProvenanceEventRecord relevantRecords = new 
ArrayList();
+private final SetLineageNode nodes = new HashSet();
+private final SetLineageEdge edges = new HashSet();
+private final int numSteps;
+private final long creationNanos;
+private long computationNanos;
+
+private final ReadWriteLock rwLock = new ReentrantReadWriteLock();
+private final Lock readLock = rwLock.readLock();
+private final Lock writeLock = rwLock.writeLock();
+
+private Date expirationDate = null;
+private String error = null;
+private int numCompletedSteps = 0;
+
+private volatile boolean canceled = false;
+
+public StandardLineageResult(final int numSteps, final CollectionString 
flowFileUuids) {
+this.numSteps = numSteps;
+this.creationNanos = System.nanoTime();
+this.flowFileUuids = flowFileUuids;
+
+updateExpiration();
+}
+
+@Override
+public ListLineageNode getNodes() {
+readLock.lock();
+try {
+return new ArrayList(nodes);
+} finally {
+readLock.unlock();
+}
+}
+
+@Override
+public ListLineageEdge getEdges() {
+readLock.lock();
+try {
+return new ArrayList(edges);
+} finally {
+readLock.unlock();
+}
+}
+
+public int getNumberOfEdges() {
+readLock.lock();
+try {
+return edges.size();
+} finally {
+readLock.unlock();
+}
+}
+
+public int getNumberOfNodes() {
+readLock.lock();
+try {
+return nodes.size();
+} finally {
+readLock.unlock();
+}
+}
+
+public long getComputationTime(final TimeUnit timeUnit) {
+readLock.lock();
+try {
+return timeUnit.convert(computationNanos, TimeUnit.NANOSECONDS);
+} finally {
+readLock.unlock();
+}
+}
+
+@Override
+public Date getExpiration() {
+readLock.lock();
+try {
+return expirationDate;
+} finally {
+readLock.unlock();
+}
+}
+
+@Override
+public String getError() {
+readLock.lock();
+try {
+return error;
+} finally {
+

[07/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/repository/RepositoryStatusReport.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/repository/RepositoryStatusReport.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/repository/RepositoryStatusReport.java
new file mode 100644
index 000..e434905
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/repository/RepositoryStatusReport.java
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller.repository;
+
+import java.util.Map;
+
+public interface RepositoryStatusReport {
+
+void addReportEntry(FlowFileEvent entry);
+
+MapString, FlowFileEvent getReportEntries();
+
+FlowFileEvent getReportEntry(String componentId);
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceNode.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceNode.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceNode.java
new file mode 100644
index 000..6f9c237
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceNode.java
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller.service;
+
+import org.apache.nifi.controller.Availability;
+import org.apache.nifi.controller.ConfiguredComponent;
+import org.apache.nifi.controller.ControllerService;
+
+public interface ControllerServiceNode extends ConfiguredComponent {
+
+ControllerService getControllerService();
+
+Availability getAvailability();
+
+void setAvailability(Availability availability);
+
+boolean isDisabled();
+
+void setDisabled(boolean disabled);
+
+ControllerServiceReference getReferences();
+
+void addReference(ConfiguredComponent referringComponent);
+
+void removeReference(ConfiguredComponent referringComponent);
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceProvider.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceProvider.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceProvider.java
new file mode 100644
index 000..35a255d
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/service/ControllerServiceProvider.java
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for 

[25/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/test/java/org/apache/nifi/admin/service/action/SetUserAuthoritiesActionTest.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/test/java/org/apache/nifi/admin/service/action/SetUserAuthoritiesActionTest.java
 
b/nar-bundles/framework-bundle/framework/administration/src/test/java/org/apache/nifi/admin/service/action/SetUserAuthoritiesActionTest.java
new file mode 100644
index 000..dd3695c
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/administration/src/test/java/org/apache/nifi/admin/service/action/SetUserAuthoritiesActionTest.java
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.admin.service.action;
+
+import java.util.Collections;
+import java.util.EnumSet;
+import java.util.Set;
+import org.apache.nifi.admin.dao.AuthorityDAO;
+import org.apache.nifi.admin.dao.DAOFactory;
+import org.apache.nifi.admin.dao.UserDAO;
+import org.apache.nifi.admin.service.AccountNotFoundException;
+import org.apache.nifi.admin.service.AdministrationException;
+import org.apache.nifi.authorization.Authority;
+import org.apache.nifi.authorization.AuthorityProvider;
+import org.apache.nifi.authorization.exception.AuthorityAccessException;
+import org.apache.nifi.user.AccountStatus;
+import org.apache.nifi.user.NiFiUser;
+import org.apache.commons.lang3.StringUtils;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+/**
+ * Test case for SetUserAuthoritiesAction.
+ */
+public class SetUserAuthoritiesActionTest {
+
+private static final String USER_ID_1 = 1;
+private static final String USER_ID_2 = 2;
+private static final String USER_ID_3 = 3;
+
+private static final String USER_DN_2 = user dn 2;
+private static final String USER_DN_3 = user dn 3;
+
+private DAOFactory daoFactory;
+private UserDAO userDao;
+private AuthorityDAO authorityDao;
+private AuthorityProvider authorityProvider;
+
+@Before
+public void setup() throws Exception {
+// mock the user dao
+userDao = Mockito.mock(UserDAO.class);
+Mockito.doAnswer(new AnswerNiFiUser() {
+@Override
+public NiFiUser answer(InvocationOnMock invocation) throws 
Throwable {
+Object[] args = invocation.getArguments();
+String id = (String) args[0];
+
+NiFiUser user = null;
+if (USER_ID_1.equals(id)) {
+// leave user uninitialized
+} else if (USER_ID_2.equals(id)) {
+user = new NiFiUser();
+user.setId(USER_ID_2);
+user.setDn(USER_DN_2);
+} else if (USER_ID_3.equals(id)) {
+user = new NiFiUser();
+user.setId(USER_ID_3);
+user.setDn(USER_DN_3);
+
user.getAuthorities().addAll(EnumSet.of(Authority.ROLE_MONITOR));
+user.setStatus(AccountStatus.ACTIVE);
+}
+return user;
+}
+}).when(userDao).findUserById(Mockito.anyString());
+Mockito.doAnswer(new AnswerNiFiUser() {
+@Override
+public NiFiUser answer(InvocationOnMock invocation) throws 
Throwable {
+Object[] args = invocation.getArguments();
+String dn = (String) args[0];
+
+NiFiUser user = null;
+if (USER_DN_3.equals(dn)) {
+user = new NiFiUser();
+user.setId(USER_ID_3);
+user.setDn(USER_DN_3);
+
user.getAuthorities().addAll(EnumSet.of(Authority.ROLE_MONITOR));
+user.setStatus(AccountStatus.ACTIVE);
+}
+return user;
+}
+}).when(userDao).findUserByDn(Mockito.anyString());
+Mockito.doAnswer(new AnswerVoid() {

[17/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/impl/NodeProtocolSenderImpl.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/impl/NodeProtocolSenderImpl.java
 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/impl/NodeProtocolSenderImpl.java
new file mode 100644
index 000..dc86d24
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/impl/NodeProtocolSenderImpl.java
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.protocol.impl;
+
+import java.io.IOException;
+import java.net.Socket;
+
+import javax.net.ssl.SSLSocket;
+import javax.security.cert.X509Certificate;
+
+import org.apache.nifi.cluster.protocol.NodeProtocolSender;
+import org.apache.nifi.cluster.protocol.ProtocolContext;
+import org.apache.nifi.cluster.protocol.ProtocolException;
+import org.apache.nifi.cluster.protocol.ProtocolMessageMarshaller;
+import org.apache.nifi.cluster.protocol.ProtocolMessageUnmarshaller;
+import org.apache.nifi.cluster.protocol.UnknownServiceAddressException;
+import org.apache.nifi.cluster.protocol.message.ConnectionRequestMessage;
+import org.apache.nifi.cluster.protocol.message.ConnectionResponseMessage;
+import 
org.apache.nifi.cluster.protocol.message.ControllerStartupFailureMessage;
+import org.apache.nifi.cluster.protocol.message.HeartbeatMessage;
+import org.apache.nifi.cluster.protocol.message.NodeBulletinsMessage;
+import org.apache.nifi.cluster.protocol.message.ProtocolMessage;
+import org.apache.nifi.cluster.protocol.message.ProtocolMessage.MessageType;
+import org.apache.nifi.cluster.protocol.message.ReconnectionFailureMessage;
+import org.apache.nifi.io.socket.SocketConfiguration;
+import org.apache.nifi.io.socket.SocketUtils;
+import org.apache.nifi.io.socket.multicast.DiscoverableService;
+
+public class NodeProtocolSenderImpl implements NodeProtocolSender {
+private final SocketConfiguration socketConfiguration;
+private final ClusterServiceLocator clusterManagerProtocolServiceLocator;
+private final ProtocolContextProtocolMessage protocolContext;
+
+public NodeProtocolSenderImpl(final ClusterServiceLocator 
clusterManagerProtocolServiceLocator, 
+final SocketConfiguration socketConfiguration, final 
ProtocolContextProtocolMessage protocolContext) {
+if(clusterManagerProtocolServiceLocator == null) {
+throw new IllegalArgumentException(Protocol Service Locator may 
not be null.);
+} else if(socketConfiguration == null) {
+throw new IllegalArgumentException(Socket configuration may not 
be null.);
+} else if(protocolContext == null) {
+throw new IllegalArgumentException(Protocol Context may not be 
null.);
+}
+
+this.clusterManagerProtocolServiceLocator = 
clusterManagerProtocolServiceLocator;
+this.socketConfiguration = socketConfiguration;
+this.protocolContext = protocolContext;
+}
+
+
+@Override
+public ConnectionResponseMessage requestConnection(final 
ConnectionRequestMessage msg) throws ProtocolException, 
UnknownServiceAddressException {
+Socket socket = null;
+try {
+socket = createSocket();
+
+String ncmDn = null;
+if ( socket instanceof SSLSocket ) {
+final SSLSocket sslSocket = (SSLSocket) socket;
+try {
+final X509Certificate[] certChains = 
sslSocket.getSession().getPeerCertificateChain();
+if ( certChains != null  certChains.length  0 ) {
+ncmDn = certChains[0].getSubjectDN().getName();
+}
+} catch (final ProtocolException pe) {
+throw pe;
+} catch (final Exception e) {
+throw new ProtocolException(e);

[16/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/FlowResponseMessage.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/FlowResponseMessage.java
 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/FlowResponseMessage.java
new file mode 100644
index 000..0d34dae
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/FlowResponseMessage.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.protocol.message;
+
+import javax.xml.bind.annotation.XmlRootElement;
+
+import org.apache.nifi.cluster.protocol.StandardDataFlow;
+
+/**
+ * @author unattributed
+ */
+@XmlRootElement(name = flowResponseMessage)
+public class FlowResponseMessage extends ProtocolMessage {
+
+private StandardDataFlow dataFlow;
+
+@Override
+public MessageType getType() {
+return MessageType.FLOW_RESPONSE;
+}
+
+public StandardDataFlow getDataFlow() {
+return dataFlow;
+}
+
+public void setDataFlow(StandardDataFlow dataFlow) {
+this.dataFlow = dataFlow;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/HeartbeatMessage.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/HeartbeatMessage.java
 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/HeartbeatMessage.java
new file mode 100644
index 000..0064cb6
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/HeartbeatMessage.java
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.protocol.message;
+
+import org.apache.nifi.cluster.protocol.Heartbeat;
+import javax.xml.bind.annotation.XmlRootElement;
+
+/**
+ * @author unattributed
+ */
+@XmlRootElement(name = heartbeatMessage)
+public class HeartbeatMessage extends ProtocolMessage {
+
+private Heartbeat heartbeat;
+
+@Override
+public MessageType getType() {
+return MessageType.HEARTBEAT;
+}
+
+public Heartbeat getHeartbeat() {
+return heartbeat;
+}
+
+public void setHeartbeat(Heartbeat heartbeat) {
+this.heartbeat = heartbeat;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/MulticastProtocolMessage.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/message/MulticastProtocolMessage.java
 

[42/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java
--
diff --git 
a/commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java 
b/commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java
new file mode 100644
index 000..aa6f8f3
--- /dev/null
+++ 
b/commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.util;
+
+/**
+ * String Utils based on the Apache Commons Lang String Utils.
+ * These simple util methods here allow us to avoid a dependency in the core
+ */
+public class StringUtils {
+
+public static final String EMPTY = ;
+
+public static boolean isBlank(final String str) {
+if (str == null || str.isEmpty()) {
+return true;
+}
+for (int i = 0; i  str.length(); i++) {
+if (!Character.isWhitespace(str.charAt(i))) {
+return false;
+}
+}
+return true;
+}
+
+public static boolean isEmpty(final String str) {
+return str == null || str.isEmpty();
+}
+
+public static boolean startsWith(final String str, final String prefix) {
+if (str == null || prefix == null) {
+return (str == null  prefix == null);
+}
+if (prefix.length()  str.length()) {
+return false;
+}
+return str.regionMatches(false, 0, prefix, 0, prefix.length());
+}
+
+public static String substringAfter(final String str, final String 
separator) {
+if (isEmpty(str)) {
+return str;
+}
+if (separator == null) {
+return EMPTY;
+}
+int pos = str.indexOf(separator);
+if (pos == -1) {
+return EMPTY;
+}
+return str.substring(pos + separator.length());
+}
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-security-utils/pom.xml
--
diff --git a/commons/nifi-security-utils/pom.xml 
b/commons/nifi-security-utils/pom.xml
new file mode 100644
index 000..76e9ac1
--- /dev/null
+++ b/commons/nifi-security-utils/pom.xml
@@ -0,0 +1,40 @@
+project xmlns=http://maven.apache.org/POM/4.0.0; 
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation=http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd;
+!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the License); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+  http://www.apache.org/licenses/LICENSE-2.0
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an AS IS BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+--
+modelVersion4.0.0/modelVersion
+parent
+groupIdorg.apache.nifi/groupId
+artifactIdnifi-parent/artifactId
+version0.0.1-SNAPSHOT/version
+/parent
+
+artifactIdnifi-security-utils/artifactId
+version0.0.1-SNAPSHOT/version
+nameNiFi Security Utils/name
+descriptionContains security functionality./description
+
+dependencies
+dependency
+groupIdorg.slf4j/groupId
+artifactIdslf4j-api/artifactId
+/dependency
+dependency
+groupIdorg.apache.commons/groupId
+artifactIdcommons-lang3/artifactId
+version3.3.2/version
+/dependency
+/dependencies
+/project
+

[21/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/PortStatusDTO.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/PortStatusDTO.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/PortStatusDTO.java
new file mode 100644
index 000..c32efdb
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/PortStatusDTO.java
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.dto.status;
+
+import javax.xml.bind.annotation.XmlType;
+
+/**
+ * The status for a port in this NiFi.
+ */
+@XmlType(name = portStatus)
+public class PortStatusDTO extends StatusDTO {
+
+private String id;
+private String groupId;
+private String name;
+private Integer activeThreadCount;
+private String input;
+private String output;
+private Boolean transmitting;
+private String runStatus;
+
+/**
+ * Whether this port has incoming or outgoing connections to a remote NiFi.
+ *
+ * @return
+ */
+public Boolean isTransmitting() {
+return transmitting;
+}
+
+public void setTransmitting(Boolean transmitting) {
+this.transmitting = transmitting;
+}
+
+/**
+ * The active thread count for this port.
+ *
+ * @return
+ */
+public Integer getActiveThreadCount() {
+return activeThreadCount;
+}
+
+public void setActiveThreadCount(Integer activeThreadCount) {
+this.activeThreadCount = activeThreadCount;
+}
+
+/**
+ * The id of this port.
+ *
+ * @return
+ */
+public String getId() {
+return id;
+}
+
+public void setId(String id) {
+this.id = id;
+}
+
+/**
+ * The id of the group this port resides in.
+ *
+ * @return
+ */
+public String getGroupId() {
+return groupId;
+}
+
+public void setGroupId(String groupId) {
+this.groupId = groupId;
+}
+
+/**
+ * The name of this port.
+ *
+ * @return
+ */
+public String getName() {
+return name;
+}
+
+public void setName(String name) {
+this.name = name;
+}
+
+/**
+ * The run status of this port.
+ *
+ * @return
+ */
+public String getRunStatus() {
+return runStatus;
+}
+
+public void setRunStatus(String runStatus) {
+this.runStatus = runStatus;
+}
+
+/**
+ * The total count and size of flow files that have been accepted in the
+ * last five minutes.
+ *
+ * @return The total processed
+ */
+public String getInput() {
+return input;
+}
+
+public void setInput(String input) {
+this.input = input;
+}
+
+/**
+ * The total count and size of flow files that have been processed in the
+ * last five minutes.
+ *
+ * @return The total output
+ */
+public String getOutput() {
+return output;
+}
+
+public void setOutput(String output) {
+this.output = output;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/ProcessGroupStatusDTO.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/ProcessGroupStatusDTO.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/ProcessGroupStatusDTO.java
new file mode 100644
index 000..4fa2b64
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/status/ProcessGroupStatusDTO.java
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See 

[33/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/MapPutResult.java
--
diff --git 
a/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/MapPutResult.java
 
b/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/MapPutResult.java
new file mode 100644
index 000..29695eb
--- /dev/null
+++ 
b/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/MapPutResult.java
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.distributed.cache.server.map;
+
+import java.nio.ByteBuffer;
+
+public class MapPutResult {
+private final boolean successful;
+private final ByteBuffer key, value;
+private final ByteBuffer existingValue;
+private final ByteBuffer evictedKey, evictedValue;
+
+public MapPutResult(final boolean successful, final ByteBuffer key, final 
ByteBuffer value, final ByteBuffer existingValue, final ByteBuffer evictedKey, 
final ByteBuffer evictedValue) {
+this.successful = successful;
+this.key = key;
+this.value = value;
+this.existingValue = existingValue;
+this.evictedKey = evictedKey;
+this.evictedValue = evictedValue;
+}
+
+public boolean isSuccessful() {
+return successful;
+}
+
+public ByteBuffer getKey() {
+return key;
+}
+
+public ByteBuffer getValue() {
+return value;
+}
+
+public ByteBuffer getExistingValue() {
+return existingValue;
+}
+
+public ByteBuffer getEvictedKey() {
+return evictedKey;
+}
+
+public ByteBuffer getEvictedValue() {
+return evictedValue;
+}
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/PersistentMapCache.java
--
diff --git 
a/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/PersistentMapCache.java
 
b/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/PersistentMapCache.java
new file mode 100644
index 000..77fb77d
--- /dev/null
+++ 
b/nar-bundles/distributed-cache-services-bundle/distributed-cache-server/src/main/java/org/apache/nifi/distributed/cache/server/map/PersistentMapCache.java
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.distributed.cache.server.map;
+
+import java.io.DataInputStream;
+import java.io.EOFException;
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.wali.MinimalLockingWriteAheadLog;
+import org.wali.SerDe;
+import 

[40/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-stream-utils/src/main/java/org/apache/nifi/io/ByteArrayInputStream.java
--
diff --git 
a/commons/nifi-stream-utils/src/main/java/org/apache/nifi/io/ByteArrayInputStream.java
 
b/commons/nifi-stream-utils/src/main/java/org/apache/nifi/io/ByteArrayInputStream.java
new file mode 100644
index 000..9defab4
--- /dev/null
+++ 
b/commons/nifi-stream-utils/src/main/java/org/apache/nifi/io/ByteArrayInputStream.java
@@ -0,0 +1,250 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.io;
+
+import java.io.InputStream;
+
+/**
+ * This class performs the same function as java.io.ByteArrayInputStream but
+ * does not mark its methods as synchronized
+ */
+public class ByteArrayInputStream extends InputStream {
+
+/**
+ * An array of bytes that was provided by the creator of the stream.
+ * Elements codebuf[0]/code through codebuf[count-1]/code are the
+ * only bytes that can ever be read from the stream; element
+ * codebuf[pos]/code is the next byte to be read.
+ */
+protected byte buf[];
+
+/**
+ * The index of the next character to read from the input stream buffer.
+ * This value should always be nonnegative and not larger than the value of
+ * codecount/code. The next byte to be read from the input stream 
buffer
+ * will be codebuf[pos]/code.
+ */
+protected int pos;
+
+/**
+ * The currently marked position in the stream. ByteArrayInputStream 
objects
+ * are marked at position zero by default when constructed. They may be
+ * marked at another position within the buffer by the codemark()/code
+ * method. The current buffer position is set to this point by the
+ * codereset()/code method.
+ * p
+ * If no mark has been set, then the value of mark is the offset passed to
+ * the constructor (or 0 if the offset was not supplied).
+ *
+ * @since JDK1.1
+ */
+protected int mark = 0;
+
+/**
+ * The index one greater than the last valid character in the input stream
+ * buffer. This value should always be nonnegative and not larger than the
+ * length of codebuf/code. It is one greater than the position of the
+ * last byte within codebuf/code that can ever be read from the input
+ * stream buffer.
+ */
+protected int count;
+
+/**
+ * Creates a codeByteArrayInputStream/code so that it uses
+ * codebuf/code as its buffer array. The buffer array is not copied. 
The
+ * initial value of codepos/code is code0/code and the initial 
value
+ * of  codecount/code is the length of codebuf/code.
+ *
+ * @param buf the input buffer.
+ */
+public ByteArrayInputStream(byte buf[]) {
+this.buf = buf;
+this.pos = 0;
+this.count = buf.length;
+}
+
+/**
+ * Creates codeByteArrayInputStream/code that uses codebuf/code as
+ * its buffer array. The initial value of codepos/code is
+ * codeoffset/code and the initial value of codecount/code is the
+ * minimum of codeoffset+length/code and codebuf.length/code. The
+ * buffer array is not copied. The buffer's mark is set to the specified
+ * offset.
+ *
+ * @param buf the input buffer.
+ * @param offset the offset in the buffer of the first byte to read.
+ * @param length the maximum number of bytes to read from the buffer.
+ */
+public ByteArrayInputStream(byte buf[], int offset, int length) {
+this.buf = buf;
+this.pos = offset;
+this.count = Math.min(offset + length, buf.length);
+this.mark = offset;
+}
+
+/**
+ * Reads the next byte of data from this input stream. The value byte is
+ * returned as an codeint/code in the range code0/code to
+ * code255/code. If no byte is available because the end of the stream
+ * has been reached, the value code-1/code is returned.
+ * p
+ * This coderead/code method cannot block.
+ *
+ * @return the next byte of data, or code-1/code if the end of the
+ * stream has been reached.
+ */
+@Override
+  

[29/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/dao/impl/StandardActionDAO.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/dao/impl/StandardActionDAO.java
 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/dao/impl/StandardActionDAO.java
new file mode 100644
index 000..4b89655
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/dao/impl/StandardActionDAO.java
@@ -0,0 +1,1056 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.admin.dao.impl;
+
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.nifi.action.Action;
+import org.apache.nifi.action.Component;
+import org.apache.nifi.action.Operation;
+import org.apache.nifi.action.component.details.ComponentDetails;
+import org.apache.nifi.action.component.details.ProcessorDetails;
+import org.apache.nifi.action.component.details.RemoteProcessGroupDetails;
+import org.apache.nifi.action.details.ActionDetails;
+import org.apache.nifi.action.details.ConfigureDetails;
+import org.apache.nifi.action.details.ConnectDetails;
+import org.apache.nifi.action.details.MoveDetails;
+import org.apache.nifi.action.details.PurgeDetails;
+import org.apache.nifi.admin.RepositoryUtils;
+import org.apache.nifi.admin.dao.ActionDAO;
+import org.apache.nifi.admin.dao.DataAccessException;
+import org.apache.nifi.history.History;
+import org.apache.nifi.history.HistoryQuery;
+import org.apache.nifi.history.PreviousValue;
+import org.apache.commons.lang3.StringUtils;
+
+/**
+ *
+ */
+public class StandardActionDAO implements ActionDAO {
+
+// 
+// action table
+// 
+private static final String INSERT_ACTION = INSERT INTO ACTION (
++ USER_DN, USER_NAME, SOURCE_ID, SOURCE_NAME, SOURCE_TYPE, 
OPERATION, ACTION_TIMESTAMP
++ ) VALUES (
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ );
+
+// -
+// component details
+// -
+private static final String INSERT_PROCESSOR_DETAILS = INSERT INTO 
PROCESSOR_DETAILS (
++ ACTION_ID, TYPE
++ ) VALUES (
++ ?, 
++ ?
++ );
+
+private static final String INSERT_REMOTE_PROCESS_GROUP_DETAILS = INSERT 
INTO REMOTE_PROCESS_GROUP_DETAILS (
++ ACTION_ID, URI
++ ) VALUES (
++ ?, 
++ ?
++ );
+
+// --
+// action details
+// --
+private static final String INSERT_CONFIGURE_DETAILS = INSERT INTO 
CONFIGURE_DETAILS (
++ ACTION_ID, NAME, VALUE, PREVIOUS_VALUE
++ ) VALUES (
++ ?, 
++ ?, 
++ ?, 
++ ?
++ );
+
+private static final String INSERT_CONNECT_DETAILS = INSERT INTO 
CONNECT_DETAILS (
++ ACTION_ID, SOURCE_ID, SOURCE_NAME, SOURCE_TYPE, RELATIONSHIP, 
DESTINATION_ID, DESTINATION_NAME, DESTINATION_TYPE
++ ) VALUES (
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?
++ );
+
+private static final String INSERT_MOVE_DETAILS = INSERT INTO 
MOVE_DETAILS (
++ ACTION_ID, GROUP_ID, GROUP_NAME, PREVIOUS_GROUP_ID, 
PREVIOUS_GROUP_NAME
++ ) VALUES (
++ ?, 
++ ?, 
++ ?, 
++ ?, 
++ ?
++ );
+
+private static final String INSERT_PURGE_DETAILS = INSERT INTO 
PURGE_DETAILS 

[27/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserAuthoritiesCacheAction.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserAuthoritiesCacheAction.java
 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserAuthoritiesCacheAction.java
new file mode 100644
index 000..89661b2
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserAuthoritiesCacheAction.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.admin.service.action;
+
+import java.util.Set;
+import org.apache.nifi.admin.dao.AuthorityDAO;
+import org.apache.nifi.admin.dao.DAOFactory;
+import org.apache.nifi.admin.dao.DataAccessException;
+import org.apache.nifi.admin.dao.UserDAO;
+import org.apache.nifi.admin.service.AccountNotFoundException;
+import org.apache.nifi.authorization.Authority;
+import org.apache.nifi.authorization.AuthorityProvider;
+import org.apache.nifi.user.NiFiUser;
+import org.apache.commons.collections4.CollectionUtils;
+
+/**
+ * Updates a NiFiUser's authorities. Prior to invoking this action, the user's
+ * authorities should be set according to the business logic of the service in
+ * question. This should not be invoked directly when attempting to set user
+ * authorities as the authorityProvider is not called from this action.
+ */
+public class UpdateUserAuthoritiesCacheAction extends AbstractUserActionVoid 
{
+
+private final NiFiUser user;
+
+public UpdateUserAuthoritiesCacheAction(NiFiUser user) {
+this.user = user;
+}
+
+@Override
+public Void execute(DAOFactory daoFactory, AuthorityProvider 
authorityProvider) throws DataAccessException {
+UserDAO userDao = daoFactory.getUserDAO();
+AuthorityDAO authorityDao = daoFactory.getAuthorityDAO();
+
+// get the user
+NiFiUser currentUser = userDao.findUserById(user.getId());
+
+// ensure the user exists
+if (currentUser == null) {
+throw new AccountNotFoundException(String.format(Unable to find 
account with ID %s., user.getId()));
+}
+
+// determine what authorities need to be added/removed
+SetAuthority authorities = user.getAuthorities();
+SetAuthority authoritiesToAdd = 
determineAuthoritiesToAdd(currentUser, authorities);
+SetAuthority authoritiesToRemove = 
determineAuthoritiesToRemove(currentUser, authorities);
+
+// update the user authorities locally
+if (CollectionUtils.isNotEmpty(authoritiesToAdd)) {
+authorityDao.createAuthorities(authoritiesToAdd, user.getId());
+}
+if (CollectionUtils.isNotEmpty(authoritiesToRemove)) {
+authorityDao.deleteAuthorities(authoritiesToRemove, user.getId());
+}
+
+return null;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserCacheAction.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserCacheAction.java
 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserCacheAction.java
new file mode 100644
index 000..288e297
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/admin/service/action/UpdateUserCacheAction.java
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache 

[20/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CounterEntity.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CounterEntity.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CounterEntity.java
new file mode 100644
index 000..666dc75
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CounterEntity.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.entity;
+
+import javax.xml.bind.annotation.XmlRootElement;
+import org.apache.nifi.web.api.dto.CounterDTO;
+
+/**
+ * A serialized representation of this class can be placed in the entity body 
of
+ * a response from the API. This particular entity holds a reference to a
+ * CounterDTO.
+ */
+@XmlRootElement(name = counterEntity)
+public class CounterEntity extends Entity {
+
+private CounterDTO counter;
+
+/**
+ * Get the counter.
+ *
+ * @return
+ */
+public CounterDTO getCounter() {
+return counter;
+}
+
+public void setCounter(CounterDTO counter) {
+this.counter = counter;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CountersEntity.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CountersEntity.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CountersEntity.java
new file mode 100644
index 000..bee7669
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/CountersEntity.java
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.entity;
+
+import javax.xml.bind.annotation.XmlRootElement;
+import org.apache.nifi.web.api.dto.CountersDTO;
+
+/**
+ * A serialized representation of this class can be placed in the entity body 
of
+ * a response from the API. This particular entity holds a reference to a
+ * CountersDTO.
+ */
+@XmlRootElement(name = countersEntity)
+public class CountersEntity extends Entity {
+
+private CountersDTO counters;
+
+/**
+ * Get the counters which contains all the counter groups and a generation
+ * date.
+ *
+ * @return
+ */
+public CountersDTO getCounters() {
+return counters;
+}
+
+public void setCounters(CountersDTO counters) {
+this.counters = counters;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/Entity.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/Entity.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/Entity.java
new file mode 100644
index 000..6abcbf3
--- 

[26/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/History.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/History.java
 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/History.java
new file mode 100644
index 000..8536871
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/History.java
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.history;
+
+import java.util.Collection;
+import java.util.Date;
+import org.apache.nifi.action.Action;
+
+/**
+ * The result of running an action query.
+ */
+public class History {
+
+private Integer total;
+private Date lastRefreshed;
+private CollectionAction actions;
+
+public CollectionAction getActions() {
+return actions;
+}
+
+public void setActions(CollectionAction actions) {
+this.actions = actions;
+}
+
+public Integer getTotal() {
+return total;
+}
+
+public void setTotal(Integer totalRecordCount) {
+this.total = totalRecordCount;
+}
+
+public Date getLastRefreshed() {
+return lastRefreshed;
+}
+
+public void setLastRefreshed(Date lastRefreshed) {
+this.lastRefreshed = lastRefreshed;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/HistoryQuery.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/HistoryQuery.java
 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/HistoryQuery.java
new file mode 100644
index 000..53cc13c
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/administration/src/main/java/org/apache/nifi/history/HistoryQuery.java
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.history;
+
+import java.util.Date;
+
+/**
+ *
+ */
+public class HistoryQuery {
+
+private String userName;
+private String sourceId;
+private Date startDate;
+private Date endDate;
+private Integer offset;
+private Integer count;
+private String sortColumn;
+private String sortOrder;
+
+public Date getEndDate() {
+return endDate;
+}
+
+public void setEndDate(Date endDate) {
+this.endDate = endDate;
+}
+
+public String getSourceId() {
+return sourceId;
+}
+
+public void setSourceId(String sourceId) {
+this.sourceId = sourceId;
+}
+
+public Date getStartDate() {
+return startDate;
+}
+
+public void setStartDate(Date startDate) {
+this.startDate = startDate;
+}
+
+public String getUserName() {
+return userName;
+}
+
+public void setUserName(String userName) {
+this.userName = userName;
+}
+
+public Integer getOffset() {
+return offset;
+}
+
+public void setOffset(Integer offset) {
+this.offset = offset;
+}
+
+public Integer getCount() {
+

[23/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/SystemDiagnosticsDTO.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/SystemDiagnosticsDTO.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/SystemDiagnosticsDTO.java
new file mode 100644
index 000..0b20852
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/SystemDiagnosticsDTO.java
@@ -0,0 +1,461 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.dto;
+
+import java.util.Date;
+import java.util.Set;
+import javax.xml.bind.annotation.XmlType;
+import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
+import org.apache.nifi.web.api.dto.util.TimeAdapter;
+
+/**
+ * The diagnostics of the system this NiFi is running on.
+ */
+@XmlType(name = systemDiagnostics)
+public class SystemDiagnosticsDTO {
+
+private String totalNonHeap;
+private String usedNonHeap;
+private String freeNonHeap;
+private String maxNonHeap;
+private String nonHeapUtilization;
+
+private String totalHeap;
+private String usedHeap;
+private String freeHeap;
+private String maxHeap;
+private String heapUtilization;
+
+private Integer availableProcessors;
+private Double processorLoadAverage;
+
+private Integer totalThreads;
+private Integer daemonThreads;
+
+private StorageUsageDTO flowFileRepositoryStorageUsage;
+private SetStorageUsageDTO contentRepositoryStorageUsage;
+private SetGarbageCollectionDTO garbageCollection;
+
+private Date statsLastRefreshed;
+
+/* getters / setters */
+/**
+ * The number of available processors, if supported.
+ *
+ * @return
+ */
+public Integer getAvailableProcessors() {
+return availableProcessors;
+}
+
+public void setAvailableProcessors(Integer availableProcessors) {
+this.availableProcessors = availableProcessors;
+}
+
+/**
+ * The number of daemon threads.
+ *
+ * @return
+ */
+public Integer getDaemonThreads() {
+return daemonThreads;
+}
+
+public void setDaemonThreads(Integer daemonThreads) {
+this.daemonThreads = daemonThreads;
+}
+
+/**
+ * The amount of free heap.
+ *
+ * @return
+ */
+public String getFreeHeap() {
+return freeHeap;
+}
+
+public void setFreeHeap(String freeHeap) {
+this.freeHeap = freeHeap;
+}
+
+/**
+ * The amount of free non-heap.
+ *
+ * @return
+ */
+public String getFreeNonHeap() {
+return freeNonHeap;
+}
+
+public void setFreeNonHeap(String freeNonHeap) {
+this.freeNonHeap = freeNonHeap;
+}
+
+/**
+ * The max size of the heap.
+ *
+ * @return
+ */
+public String getMaxHeap() {
+return maxHeap;
+}
+
+public void setMaxHeap(String maxHeap) {
+this.maxHeap = maxHeap;
+}
+
+/**
+ * The max size of the non-heap.
+ *
+ * @return
+ */
+public String getMaxNonHeap() {
+return maxNonHeap;
+}
+
+public void setMaxNonHeap(String maxNonHeap) {
+this.maxNonHeap = maxNonHeap;
+}
+
+/**
+ * The processor load average, if supported.
+ *
+ * @return
+ */
+public Double getProcessorLoadAverage() {
+return processorLoadAverage;
+}
+
+public void setProcessorLoadAverage(Double processorLoadAverage) {
+this.processorLoadAverage = processorLoadAverage;
+}
+
+/**
+ * The total size of the heap.
+ *
+ * @return
+ */
+public String getTotalHeap() {
+return totalHeap;
+}
+
+public void setTotalHeap(String totalHeap) {
+this.totalHeap = totalHeap;
+}
+
+/**
+ * The total size of non-heap.
+ *
+ * @return
+ */
+public String getTotalNonHeap() {
+return totalNonHeap;
+}
+
+public void 

[34/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/distributed-cache-services-bundle/distributed-cache-client-service/src/main/java/org/apache/nifi/distributed/cache/client/DistributedSetCacheClientService.java
--
diff --git 
a/nar-bundles/distributed-cache-services-bundle/distributed-cache-client-service/src/main/java/org/apache/nifi/distributed/cache/client/DistributedSetCacheClientService.java
 
b/nar-bundles/distributed-cache-services-bundle/distributed-cache-client-service/src/main/java/org/apache/nifi/distributed/cache/client/DistributedSetCacheClientService.java
new file mode 100644
index 000..1d7c94c
--- /dev/null
+++ 
b/nar-bundles/distributed-cache-services-bundle/distributed-cache-client-service/src/main/java/org/apache/nifi/distributed/cache/client/DistributedSetCacheClientService.java
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.distributed.cache.client;
+
+import java.io.DataInputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.annotation.OnConfigured;
+import org.apache.nifi.distributed.cache.protocol.ProtocolHandshake;
+import org.apache.nifi.distributed.cache.protocol.exception.HandshakeException;
+import org.apache.nifi.io.ByteArrayOutputStream;
+import org.apache.nifi.io.DataOutputStream;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.StandardVersionNegotiator;
+import org.apache.nifi.remote.VersionNegotiator;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class DistributedSetCacheClientService extends 
AbstractControllerService implements DistributedSetCacheClient {
+
+private static final Logger logger = 
LoggerFactory.getLogger(DistributedMapCacheClientService.class);
+
+public static final PropertyDescriptor HOSTNAME = new 
PropertyDescriptor.Builder()
+.name(Server Hostname)
+.description(The name of the server that is running the 
DistributedSetCacheServer service)
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor PORT = new 
PropertyDescriptor.Builder()
+.name(Server Port)
+.description(The port on the remote server that is to be used 
when communicating with the DistributedSetCacheServer service)
+.required(true)
+.addValidator(StandardValidators.PORT_VALIDATOR)
+.defaultValue(4557)
+.build();
+public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new 
PropertyDescriptor.Builder()
+.name(SSL Context Service)
+.description(
+If specified, indicates the SSL Context Service that is 
used to communicate with the remote server. If not specified, communications 
will not be encrypted)
+.required(false)
+
.addValidator(StandardValidators.createControllerServiceExistsValidator(SSLContextService.class))
+.defaultValue(null)
+.build();
+public static final PropertyDescriptor COMMUNICATIONS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name(Communications Timeout)
+.description(
+Specifices how long to wait when communicating with the 
remote server before determining that there is a communications failure if data 
cannot be sent or received)
+.required(true)
+.addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+.defaultValue(30 secs)
+.build();
+
+private final BlockingQueueCommsSession queue = new 
LinkedBlockingQueue();
+

[08/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/StandardFlowFileQueue.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/StandardFlowFileQueue.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/StandardFlowFileQueue.java
new file mode 100644
index 000..3b880bb
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/controller/StandardFlowFileQueue.java
@@ -0,0 +1,1093 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.Iterator;
+import java.util.List;
+import java.util.PriorityQueue;
+import java.util.Queue;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+import org.apache.nifi.connectable.Connection;
+import org.apache.nifi.controller.repository.FlowFileRecord;
+import org.apache.nifi.controller.repository.claim.ContentClaim;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.FlowFilePrioritizer;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.FlowFileFilter;
+import org.apache.nifi.processor.FlowFileFilter.FlowFileFilterResult;
+import org.apache.nifi.processor.QueueSize;
+import org.apache.nifi.scheduling.SchedulingStrategy;
+import org.apache.nifi.util.FormatUtils;
+import org.apache.nifi.util.concurrency.TimedLock;
+import org.apache.nifi.util.timebuffer.LongEntityAccess;
+import org.apache.nifi.util.timebuffer.TimedBuffer;
+import org.apache.nifi.util.timebuffer.TimestampedLong;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A FlowFileQueue is used to queue FlowFile objects that are awaiting further
+ * processing. Must be thread safe.
+ *
+ * @author none
+ */
+public final class StandardFlowFileQueue implements FlowFileQueue {
+
+public static final int MAX_EXPIRED_RECORDS_PER_ITERATION = 10;
+public static final int SWAP_RECORD_POLL_SIZE = 1;
+
+// When we have very high contention on a FlowFile Queue, the writeLock 
quickly becomes the bottleneck. In order to avoid this,
+// we keep track of how often we are obtaining the write lock. If we 
exceed some threshold, we start performing a Pre-fetch so that
+// we can then poll many times without having to obtain the lock.
+// If lock obtained an average of more than PREFETCH_POLL_THRESHOLD times 
per second in order to poll from queue for last 5 seconds, do a pre-fetch.
+public static final int PREFETCH_POLL_THRESHOLD = 1000;
+public static final int PRIORITIZED_PREFETCH_SIZE = 10;
+public static final int UNPRIORITIZED_PREFETCH_SIZE = 1000;
+private volatile int prefetchSize = UNPRIORITIZED_PREFETCH_SIZE; // when 
we pre-fetch, how many should we pre-fetch?
+
+private static final Logger logger = 
LoggerFactory.getLogger(StandardFlowFileQueue.class);
+
+private PriorityQueueFlowFileRecord activeQueue = null;
+private long activeQueueContentSize = 0L;
+private ArrayListFlowFileRecord swapQueue = null;
+
+private int swappedRecordCount = 0;
+private long swappedContentSize = 0L;
+private String maximumQueueDataSize;
+private long maximumQueueByteCount;
+private boolean swapMode = false;
+private long maximumQueueObjectCount;
+
+private final AtomicLong flowFileExpirationMillis;
+private final Connection connection;
+private final AtomicReferenceString flowFileExpirationPeriod;
+private final ReentrantReadWriteLock lock = new 
ReentrantReadWriteLock(true);
+private final ListFlowFilePrioritizer priorities;
+

[18/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/NodeProtocolSender.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/NodeProtocolSender.java
 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/NodeProtocolSender.java
new file mode 100644
index 000..1edcb91
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/NodeProtocolSender.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.protocol;
+
+import org.apache.nifi.cluster.protocol.message.ConnectionRequestMessage;
+import org.apache.nifi.cluster.protocol.message.ConnectionResponseMessage;
+import 
org.apache.nifi.cluster.protocol.message.ControllerStartupFailureMessage;
+import org.apache.nifi.cluster.protocol.message.HeartbeatMessage;
+import org.apache.nifi.cluster.protocol.message.NodeBulletinsMessage;
+import org.apache.nifi.cluster.protocol.message.ReconnectionFailureMessage;
+
+/**
+ * An interface for sending protocol messages from a node to the cluster 
manager.
+ * @author unattributed
+ */
+public interface NodeProtocolSender {
+
+/**
+ * Sends a connection request message to the cluster manager.
+ * @param msg a message
+ * @return the response
+ * @throws UnknownServiceAddressException if the cluster manager's address 
is not known
+ * @throws ProtocolException if communication failed
+ */
+ConnectionResponseMessage requestConnection(ConnectionRequestMessage msg) 
throws ProtocolException, UnknownServiceAddressException;
+
+/**
+ * Sends a heartbeat message to the cluster manager.
+ * @param msg a message
+ * @throws UnknownServiceAddressException if the cluster manager's address 
is not known
+ * @throws ProtocolException if communication failed
+ */
+void heartbeat(HeartbeatMessage msg) throws ProtocolException, 
UnknownServiceAddressException;
+
+/**
+ * Sends a bulletins message to the cluster manager.
+ * @param msg
+ * @throws ProtocolException
+ * @throws UnknownServiceAddressException 
+ */
+void sendBulletins(NodeBulletinsMessage msg) throws ProtocolException, 
UnknownServiceAddressException;
+
+/**
+ * Sends a failure notification if the controller was unable start.
+ * @param msg a message
+ * @throws UnknownServiceAddressException if the cluster manager's address 
is not known
+ * @throws ProtocolException if communication failed
+ */
+void notifyControllerStartupFailure(ControllerStartupFailureMessage msg) 
throws ProtocolException, UnknownServiceAddressException;
+
+/**
+ * Sends a failure notification if the node was unable to reconnect to the 
cluster
+ * @param msg a message
+ * @throws UnknownServiceAddressException if the cluster manager's address 
is not known
+ * @throws ProtocolException if communication failed
+ */
+void notifyReconnectionFailure(ReconnectionFailureMessage msg) throws 
ProtocolException, UnknownServiceAddressException;
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/ProtocolContext.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/ProtocolContext.java
 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/ProtocolContext.java
new file mode 100644
index 000..b614e76
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/ProtocolContext.java
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license 

[06/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/TransferDirection.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/TransferDirection.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/TransferDirection.java
new file mode 100644
index 000..56432d5
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/TransferDirection.java
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.remote;
+
+public enum TransferDirection {
+
+SEND,
+RECEIVE;
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/VersionedRemoteResource.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/VersionedRemoteResource.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/VersionedRemoteResource.java
new file mode 100644
index 000..bfccd98
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/VersionedRemoteResource.java
@@ -0,0 +1,24 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.remote;
+
+public interface VersionedRemoteResource {
+
+VersionNegotiator getVersionNegotiator();
+
+String getResourceName();
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/codec/FlowFileCodec.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/codec/FlowFileCodec.java
 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/codec/FlowFileCodec.java
new file mode 100644
index 000..b4206b3
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/core-api/src/main/java/org/apache/nifi/remote/codec/FlowFileCodec.java
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.remote.codec;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.List;
+
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessSession;

[10/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponse.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponse.java
 
b/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponse.java
new file mode 100644
index 000..3aa2931
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponse.java
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.manager.testutils;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import javax.ws.rs.core.Response.Status;
+
+/**
+ * Encapsulates an HTTP response. The toString method returns the
+ * specification-compliant response.
+ *
+ * @author unattributed
+ */
+public class HttpResponse {
+
+private final Status status;
+private final String entity;
+private final MapString, String headers = new HashMap();
+
+public HttpResponse(final Status status, final String entity) {
+this.status = status;
+this.entity = entity;
+headers.put(content-length, 
String.valueOf(entity.getBytes().length));
+}
+
+public String getEntity() {
+return entity;
+}
+
+public Status getStatus() {
+return status;
+}
+
+public MapString, String getHeaders() {
+return Collections.unmodifiableMap(headers);
+}
+
+public void addHeader(final String key, final String value) {
+if (key.contains( )) {
+throw new IllegalArgumentException(Header key may not contain 
spaces.);
+} else if (content-length.equalsIgnoreCase(key)) {
+throw new IllegalArgumentException(Content-Length header is set 
automatically based on length of content.);
+}
+headers.put(key, value);
+}
+
+public void addHeaders(final MapString, String headers) {
+for (final Map.EntryString, String entry : headers.entrySet()) {
+addHeader(entry.getKey(), entry.getValue());
+}
+}
+
+@Override
+public String toString() {
+
+final StringBuilder strb = new StringBuilder();
+
+// response line
+strb.append(HTTP/1.1 )
+.append(status.getStatusCode())
+.append( )
+.append(status.getReasonPhrase())
+.append(\n);
+
+// headers
+for (final Map.EntryString, String entry : headers.entrySet()) {
+strb.append(entry.getKey()).append(: 
).append(entry.getValue()).append(\n);
+}
+
+strb.append(\n);
+
+// body
+strb.append(entity);
+
+return strb.toString();
+}
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponseAction.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponseAction.java
 
b/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponseAction.java
new file mode 100644
index 000..28615d0
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster/src/test/java/org/apache/nifi/cluster/manager/testutils/HttpResponseAction.java
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ 

[41/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/ServerSocketConfiguration.java
--
diff --git 
a/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/ServerSocketConfiguration.java
 
b/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/ServerSocketConfiguration.java
new file mode 100644
index 000..fc279fb
--- /dev/null
+++ 
b/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/ServerSocketConfiguration.java
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.io.socket;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.security.KeyManagementException;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.UnrecoverableKeyException;
+import java.security.cert.CertificateException;
+
+import javax.net.ssl.SSLContext;
+
+/**
+ * @author unattributed
+ */
+public final class ServerSocketConfiguration {
+
+private boolean needClientAuth;
+private Integer socketTimeout;
+private Boolean reuseAddress;
+private Integer receiveBufferSize;
+private SSLContextFactory sslContextFactory;
+
+public ServerSocketConfiguration() {
+}
+
+public SSLContext createSSLContext() throws KeyManagementException, 
NoSuchAlgorithmException, UnrecoverableKeyException, KeyStoreException, 
CertificateException, FileNotFoundException, IOException {
+return sslContextFactory == null ? null : 
sslContextFactory.createSslContext();
+}
+
+public void setSSLContextFactory(final SSLContextFactory 
sslContextFactory) {
+this.sslContextFactory = sslContextFactory;
+}
+
+public Integer getSocketTimeout() {
+return socketTimeout;
+}
+
+public void setSocketTimeout(Integer socketTimeout) {
+this.socketTimeout = socketTimeout;
+}
+
+public boolean getNeedClientAuth() {
+return needClientAuth;
+}
+
+public void setNeedClientAuth(boolean needClientAuth) {
+this.needClientAuth = needClientAuth;
+}
+
+public Boolean getReuseAddress() {
+return reuseAddress;
+}
+
+public void setReuseAddress(Boolean reuseAddress) {
+this.reuseAddress = reuseAddress;
+}
+
+public Integer getReceiveBufferSize() {
+return receiveBufferSize;
+}
+
+public void setReceiveBufferSize(Integer receiveBufferSize) {
+this.receiveBufferSize = receiveBufferSize;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/SocketConfiguration.java
--
diff --git 
a/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/SocketConfiguration.java
 
b/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/SocketConfiguration.java
new file mode 100644
index 000..c24b540
--- /dev/null
+++ 
b/commons/nifi-socket-utils/src/main/java/org/apache/nifi/io/socket/SocketConfiguration.java
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.io.socket;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.security.KeyManagementException;
+import 

[19/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserGroupEntity.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserGroupEntity.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserGroupEntity.java
new file mode 100644
index 000..a6542c8
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserGroupEntity.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.entity;
+
+import javax.xml.bind.annotation.XmlRootElement;
+import org.apache.nifi.web.api.dto.UserGroupDTO;
+
+/**
+ * A serialized representation of this class can be placed in the entity body 
of
+ * a request or response to or from the API. This particular entity holds a
+ * reference to a UserGroupDTO.
+ */
+@XmlRootElement(name = userGroupEntity)
+public class UserGroupEntity extends Entity {
+
+private UserGroupDTO userGroup;
+
+/**
+ * The UserGroupDTO that is being serialized.
+ *
+ * @return The UserGroupDTO object
+ */
+public UserGroupDTO getUserGroup() {
+return userGroup;
+}
+
+public void setUserGroup(UserGroupDTO userGroup) {
+this.userGroup = userGroup;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserSearchResultsEntity.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserSearchResultsEntity.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserSearchResultsEntity.java
new file mode 100644
index 000..baffe15
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/entity/UserSearchResultsEntity.java
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.entity;
+
+import java.util.List;
+import javax.xml.bind.annotation.XmlRootElement;
+import org.apache.nifi.web.api.dto.search.UserGroupSearchResultDTO;
+import org.apache.nifi.web.api.dto.search.UserSearchResultDTO;
+
+/**
+ * A serialized representation of this class can be placed in the entity body 
of
+ * a request or response to or from the API. This particular entity holds a
+ * reference to UserSearchResultDTOs and UserGroupSearchResultDTOs.
+ */
+@XmlRootElement(name = userSearchResultsEntity)
+public class UserSearchResultsEntity {
+
+private ListUserSearchResultDTO userResults;
+private ListUserGroupSearchResultDTO userGroupResults;
+
+/**
+ * The user search results.
+ *
+ * @return
+ */
+public ListUserSearchResultDTO getUserResults() {
+return userResults;
+}
+
+public void setUserResults(ListUserSearchResultDTO userResults) {
+this.userResults = userResults;
+}
+
+/**
+ * The user group search results.
+ *
+ * @return
+ */
+public ListUserGroupSearchResultDTO getUserGroupResults() {
+return 

[11/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/node/Node.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/node/Node.java
 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/node/Node.java
new file mode 100644
index 000..84565da
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/node/Node.java
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.node;
+
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.nifi.cluster.HeartbeatPayload;
+import org.apache.nifi.cluster.protocol.Heartbeat;
+import org.apache.nifi.cluster.protocol.NodeIdentifier;
+import org.apache.nifi.cluster.protocol.ProtocolException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Represents a connected flow controller. Nodes always have an immutable
+ * identifier and a status. The status may be changed, but never null.
+ *
+ * A Node may be cloned, but the cloning is a shallow copy of the instance.
+ *
+ * This class overrides hashCode and equals and considers two instances to be
+ * equal if they have the equal NodeIdentifiers.
+ *
+ * @author unattributed
+ */
+public class Node implements Cloneable, ComparableNode {
+
+private static final Logger lockLogger = 
LoggerFactory.getLogger(cluster.lock);
+
+/**
+ * The semantics of a Node status are as follows:
+ * ul
+ * liCONNECTED -- a flow controller that is connected to the cluster. A
+ * connecting node transitions to connected after the cluster receives the
+ * flow controller's first heartbeat. A connected node can transition to
+ * disconnecting./li
+ * liCONNECTING -- a flow controller has issued a connection request to
+ * the cluster, but has not yet sent a heartbeat. A connecting node can
+ * transition to disconnecting or connected. The cluster will not accept 
any
+ * external requests to change the flow while any node is connecting./li
+ * liDISCONNECTED -- a flow controller that is not connected to the
+ * cluster. A disconnected node can transition to connecting./li
+ * liDISCONNECTING -- a flow controller that is in the process of
+ * disconnecting from the cluster. A disconnecting node will always
+ * transition to disconnected./li
+ * /ul
+ */
+public static enum Status {
+
+CONNECTED,
+CONNECTING,
+DISCONNECTED,
+DISCONNECTING
+}
+
+/**
+ * the node's unique identifier
+ */
+private final NodeIdentifier nodeId;
+
+/**
+ * the node statue
+ */
+private Status status;
+
+/**
+ * the last heartbeat received by from the node
+ */
+private Heartbeat lastHeartbeat;
+
+/**
+ * the payload of the last heartbeat received from the node
+ */
+private HeartbeatPayload lastHeartbeatPayload;
+
+/**
+ * the last time the connection for this node was requested
+ */
+private AtomicLong connectionRequestedTimestamp = new AtomicLong(0L);
+
+/**
+ * a flag to indicate this node was disconnected because of a lack of
+ * heartbeat
+ */
+private boolean heartbeatDisconnection;
+
+public Node(final NodeIdentifier id, final Status status) {
+if (id == null) {
+throw new IllegalArgumentException(ID may not be null.);
+} else if (status == null) {
+throw new IllegalArgumentException(Status may not be null.);
+}
+this.nodeId = id;
+this.status = status;
+}
+
+public NodeIdentifier getNodeId() {
+return nodeId;
+}
+
+/**
+ * Returns the last received heartbeat or null if no heartbeat has been 
set.
+ *
+ * @return a heartbeat or null
+ */
+public Heartbeat getHeartbeat() {
+return lastHeartbeat;
+}
+
+public HeartbeatPayload getHeartbeatPayload() {
+return lastHeartbeatPayload;
+}
+
+

[22/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/provenance/ProvenanceResultsDTO.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/provenance/ProvenanceResultsDTO.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/provenance/ProvenanceResultsDTO.java
new file mode 100644
index 000..d6ea8a0
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/provenance/ProvenanceResultsDTO.java
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.dto.provenance;
+
+import org.apache.nifi.web.api.dto.util.DateTimeAdapter;
+import org.apache.nifi.web.api.dto.util.TimeAdapter;
+
+import javax.xml.bind.annotation.XmlType;
+import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
+import java.util.Date;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * Results of a provenance request.
+ */
+@XmlType(name = provenanceResults)
+public class ProvenanceResultsDTO {
+
+private ListProvenanceEventDTO provenanceEvents;
+private String total;
+private Long totalCount;
+private Date generated;
+private Date oldestEvent;
+private Integer timeOffset;
+
+private SetString errors;
+
+/**
+ * Any error messages.
+ *
+ * @return
+ */
+public SetString getErrors() {
+return errors;
+}
+
+public void setErrors(SetString errors) {
+this.errors = errors;
+}
+
+/**
+ * The provenance events that matched the search criteria.
+ *
+ * @return
+ */
+public ListProvenanceEventDTO getProvenanceEvents() {
+return provenanceEvents;
+}
+
+public void setProvenanceEvents(ListProvenanceEventDTO provenanceEvents) 
{
+this.provenanceEvents = provenanceEvents;
+}
+
+/**
+ * The total number of results formatted.
+ *
+ * @return
+ */
+public String getTotal() {
+return total;
+}
+
+public void setTotal(String total) {
+this.total = total;
+}
+
+/**
+ * The total number of results.
+ *
+ * @return
+ */
+public Long getTotalCount() {
+return totalCount;
+}
+
+public void setTotalCount(Long totalCount) {
+this.totalCount = totalCount;
+}
+
+/**
+ * When the search was performed.
+ *
+ * @return
+ */
+@XmlJavaTypeAdapter(TimeAdapter.class)
+public Date getGenerated() {
+return generated;
+}
+
+public void setGenerated(Date generated) {
+this.generated = generated;
+}
+
+/**
+ * The oldest event available in the provenance repository.
+ *
+ * @return
+ */
+@XmlJavaTypeAdapter(DateTimeAdapter.class)
+public Date getOldestEvent() {
+return oldestEvent;
+}
+
+public void setOldestEvent(Date oldestEvent) {
+this.oldestEvent = oldestEvent;
+}
+
+/**
+ * The time offset on the server thats used for event time.
+ *
+ * @return
+ */
+public Integer getTimeOffset() {
+return timeOffset;
+}
+
+public void setTimeOffset(Integer timeOffset) {
+this.timeOffset = timeOffset;
+}
+
+}

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/provenance/ProvenanceSearchableFieldDTO.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/provenance/ProvenanceSearchableFieldDTO.java
 
b/nar-bundles/framework-bundle/framework/client-dto/src/main/java/org/apache/nifi/web/api/dto/provenance/ProvenanceSearchableFieldDTO.java
new file mode 100644
index 000..af89f97
--- /dev/null
+++ 

[12/51] [partial] incubator-nifi git commit: Initial code contribution

[joewitt]

http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/impl/WebClusterManager.java
--
diff --git 
a/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/impl/WebClusterManager.java
 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/impl/WebClusterManager.java
new file mode 100644
index 000..3defea7
--- /dev/null
+++ 
b/nar-bundles/framework-bundle/framework/cluster/src/main/java/org/apache/nifi/cluster/manager/impl/WebClusterManager.java
@@ -0,0 +1,3616 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.manager.impl;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.io.Serializable;
+import java.net.URI;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.ListIterator;
+import java.util.Map;
+import java.util.Queue;
+import java.util.Set;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.TreeMap;
+import java.util.UUID;
+import java.util.concurrent.CompletionService;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ExecutorCompletionService;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.regex.Pattern;
+
+import javax.net.ssl.SSLContext;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.StreamingOutput;
+import javax.xml.XMLConstants;
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.validation.Schema;
+import javax.xml.validation.SchemaFactory;
+import javax.xml.validation.Validator;
+
+import org.apache.nifi.admin.service.AuditService;
+import org.apache.nifi.cluster.BulletinsPayload;
+import org.apache.nifi.cluster.ClusterNodeInformation;
+import org.apache.nifi.cluster.HeartbeatPayload;
+import org.apache.nifi.cluster.NodeInformation;
+import org.apache.nifi.cluster.context.ClusterContext;
+import org.apache.nifi.cluster.context.ClusterContextImpl;
+import org.apache.nifi.cluster.event.Event;
+import org.apache.nifi.cluster.event.EventManager;
+import org.apache.nifi.cluster.firewall.ClusterNodeFirewall;
+import org.apache.nifi.cluster.flow.ClusterDataFlow;
+import org.apache.nifi.cluster.flow.DaoException;
+import org.apache.nifi.cluster.flow.DataFlowManagementService;
+import org.apache.nifi.cluster.flow.PersistedFlowState;
+import org.apache.nifi.cluster.manager.HttpClusterManager;
+import org.apache.nifi.cluster.manager.HttpRequestReplicator;
+import org.apache.nifi.cluster.manager.HttpResponseMapper;
+import org.apache.nifi.cluster.manager.NodeResponse;
+import 
org.apache.nifi.cluster.manager.exception.ConnectingNodeMutableRequestException;
+import 
org.apache.nifi.cluster.manager.exception.DisconnectedNodeMutableRequestException;
+import org.apache.nifi.cluster.manager.exception.IllegalClusterStateException;
+import org.apache.nifi.cluster.manager.exception.IllegalNodeDeletionException;
+import 
org.apache.nifi.cluster.manager.exception.IllegalNodeDisconnectionException;
+import 
org.apache.nifi.cluster.manager.exception.IllegalNodeReconnectionException;
+import 
org.apache.nifi.cluster.manager.exception.IneligiblePrimaryNodeException;
+import org.apache.nifi.cluster.manager.exception.NoConnectedNodesException;
+import org.apache.nifi.cluster.manager.exception.NoResponseFromNodesException;
+import org.apache.nifi.cluster.manager.exception.NodeDisconnectionException;
+import 

[jira] [Created] (NIFI-142) Site to Site in non-secure mode failing due to classpath issue

[Joseph Witt (JIRA)]

Joseph Witt created NIFI-142:


 Summary: Site to Site in non-secure mode failing due to classpath 
issue
 Key: NIFI-142
 URL: https://issues.apache.org/jira/browse/NIFI-142
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Joseph Witt
Assignee: Joseph Witt
 Fix For: 0.0.1


2014-12-08 16:15:28,224 ERROR [Remote Process Group 
ddf2aa18-d75f-4df9-8e9a-c4d4720d84ed: http://192.168.0.108:8090 Thread-1] 
org.apache.nifi.engine.FlowEngine A flow contr
oller task execution stopped abnormally
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: 
javax/mail/internet/ParseException
at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.get(FutureTask.java:188) 
~[na:1.7.0_51]
at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:93) 
~[na:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) 
[na:1.7.0_51]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
Caused by: java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
at java.lang.Class.forName0(Native Method) ~[na:1.7.0_51]
at java.lang.Class.forName(Class.java:190) ~[na:1.7.0_51]
at 
com.sun.jersey.core.reflection.ReflectionHelper$3.run(ReflectionHelper.java:289)
 ~[na:na]
at 
com.sun.jersey.core.reflection.ReflectionHelper$3.run(ReflectionHelper.java:279)
 ~[na:na]
at java.security.AccessController.doPrivileged(Native Method) 
~[na:1.7.0_51]
at 
com.sun.jersey.spi.service.ServiceFinder$AbstractLazyIterator.hasNext(ServiceFinder.java:697)
 ~[na:na]
at 
com.sun.jersey.spi.service.ServiceFinder.toClassArray(ServiceFinder.java:549) 
~[na:na]
at 
com.sun.jersey.core.spi.component.ProviderServices.getServiceClasses(ProviderServices.java:345)
 ~[na:na]
at 
com.sun.jersey.core.spi.component.ProviderServices.getServiceClasses(ProviderServices.java:338)
 ~[na:na]
at 
com.sun.jersey.core.spi.component.ProviderServices.getServices(ProviderServices.java:162)
 ~[na:na]
at 
com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:176)
 ~[na:na]
at 
com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:162)
 ~[na:na]
at com.sun.jersey.api.client.Client.init(Client.java:343) ~[na:na]
at com.sun.jersey.api.client.Client.access$000(Client.java:119) ~[na:na]
at com.sun.jersey.api.client.Client$1.f(Client.java:192) ~[na:na]
at com.sun.jersey.api.client.Client$1.f(Client.java:188) ~[na:na]
at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193) 
~[na:na]
at com.sun.jersey.api.client.Client.init(Client.java:188) ~[na:na]
at com.sun.jersey.api.client.Client.init(Client.java:171) ~[na:na]
at com.sun.jersey.api.client.Client.create(Client.java:683) ~[na:na]
at 
org.apache.nifi.web.util.WebUtils.createClientHelper(WebUtils.java:139) ~[na:na]
at org.apache.nifi.web.util.WebUtils.createClient(WebUtils.java:75) 
~[na:na]
at 
org.apache.nifi.controller.util.RemoteProcessGroupUtils.getClient(RemoteProcessGroupUtils.java:121)
 ~[na:na]
at 
org.apache.nifi.controller.util.RemoteProcessGroupUtils.init(RemoteProcessGroupUtils.java:53)
 ~[na:na]
at 
org.apache.nifi.remote.StandardRemoteProcessGroup$InitializationTask.run(StandardRemoteProcessGroup.java:1304)
 ~[na:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
~[na:1.7.0_51]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
 ~[na:1.7.0_51]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 ~[na:1.7.0_51]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_51]
... 2 common frames omitted
Caused by: java.lang.ClassNotFoundException: javax.mail.internet.ParseException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_51]
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_51]
at java.security.AccessController.doPrivileged(Native Method) 
~[na:1.7.0_51]
at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
~[na:1.7.0_51]
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_51]
at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_51]
... 32 common frames omitted

[jira] [Created] (NIFI-143) After adding a process group when moving it things fail.

[Joseph Witt (JIRA)]

Joseph Witt created NIFI-143:


 Summary: After adding a process group when moving it things fail.
 Key: NIFI-143
 URL: https://issues.apache.org/jira/browse/NIFI-143
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Joseph Witt
Assignee: Joseph Witt


After adding a process group and trying to either move it or navigate into it 
this is the error that pops up:

Unable to locate group with id '91ea80f3-afbf-4d43-bac9-9fc26cb475e0'.

After that if you refresh the flow the group is gone.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-144) Test Failure: org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest

[Aldrin Piri (JIRA)]

Aldrin Piri created NIFI-144:


 Summary: Test Failure: 
org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
 Key: NIFI-144
 URL: https://issues.apache.org/jira/browse/NIFI-144
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 0.0.1
 Environment: OS X 10.9
Maven 3.2.3 (via homebrew)
JDK 1.7.0_72

Reporter: Aldrin Piri


Running org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Added 6 IP(s) to firewall.  Only requests originating from the configured IPs 
will be accepted.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- No IPs added to firewall.  Firewall will accept all requests.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Added 6 IP(s) to firewall.  Only requests originating from the configured IPs 
will be accepted.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- No IPs added to firewall.  Firewall will accept all requests.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Added 6 IP(s) to firewall.  Only requests originating from the configured IPs 
will be accepted.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- No IPs added to firewall.  Firewall will accept all requests.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Added 6 IP(s) to firewall.  Only requests originating from the configured IPs 
will be accepted.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- No IPs added to firewall.  Firewall will accept all requests.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Added 6 IP(s) to firewall.  Only requests originating from the configured IPs 
will be accepted.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- No IPs added to firewall.  Firewall will accept all requests.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Cluster firewall configuration loaded.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Loading cluster firewall configuration.
[main] INFO org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall 
- Added 6 IP(s) to firewall.  Only requests originating from the configured IPs 
will be accepted.
[main] INFO 

[jira] [Commented] (NIFI-144) Test Failure: org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest

[Joey Echeverria (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238744#comment-14238744
 ] 

Joey Echeverria commented on NIFI-144:
--

The only reason I can think for that test to fail is a weird DNS setup.

Can you post the output of {{host abc}}?

 Test Failure: 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
 

 Key: NIFI-144
 URL: https://issues.apache.org/jira/browse/NIFI-144
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 0.0.1
 Environment: OS X 10.9
 Maven 3.2.3 (via homebrew)
 JDK 1.7.0_72
Reporter: Aldrin Piri

 Running org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all 

[jira] [Commented] (NIFI-144) Test Failure: org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest

[Aldrin Piri (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238890#comment-14238890
 ] 

Aldrin Piri commented on NIFI-144:
--

Good hunch.  

I recently upgraded the firmware on my router and lost my configured DNS.
The result was the default FiOS DNS hijacking the request when the host could 
not be resolved.  Restoring the previously configured DNS resolved the issues.

Everything looks good now.

For posterity:

Before:
{noformat}
✔ ~/devel/code/incubator-nifi [master|✔]
21:28 $ host abc
abc has address 92.242.140.21
Host abc not found: 3(NXDOMAIN)
{noformat}

Restored Settings: 
{noformat}
✔ ~/devel/code/incubator-nifi [master|✔]
21:35 $ host abc
Host abc not found: 3(NXDOMAIN)
{noformat}


 Test Failure: 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
 

 Key: NIFI-144
 URL: https://issues.apache.org/jira/browse/NIFI-144
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 0.0.1
 Environment: OS X 10.9
 Maven 3.2.3 (via homebrew)
 JDK 1.7.0_72
Reporter: Aldrin Piri

 Running org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests 

[jira] [Commented] (NIFI-50) Initial NiFi Codebase Import

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238948#comment-14238948
 ] 

Joseph Witt commented on NIFI-50:
-

Code is imported.  Need to get verification from ASF Secretary and project 
champion/mentors that we're good to go or if anymore needs to be done here.

 Initial NiFi Codebase Import
 

 Key: NIFI-50
 URL: https://issues.apache.org/jira/browse/NIFI-50
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Joseph Witt
Priority: Blocker

 This is the initial code contribution for Apache NiFi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-3) Must handle exception better in WebClusterManager (esp. merging responses)

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-3:
---
Component/s: Core Framework

 Must handle exception better in WebClusterManager (esp. merging responses)
 --

 Key: NIFI-3
 URL: https://issues.apache.org/jira/browse/NIFI-3
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Joseph Witt
Priority: Minor

 in this case we received a socket timeout attempting to merge responses and 
 it wasn't being handled.
 Component: Core Framework



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2) fifo prioritizer does not work

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-2:
---
Component/s: Extensions

 fifo prioritizer does not work
 --

 Key: NIFI-2
 URL: https://issues.apache.org/jira/browse/NIFI-2
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Joseph Witt
Priority: Minor

 Should be checking a last enqueue data and sorting that.  Returning a default 
 of -1 is insufficient.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-6) Move annotations

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-6:
---
Component/s: Core Framework

 Move annotations
 

 Key: NIFI-6
 URL: https://issues.apache.org/jira/browse/NIFI-6
 Project: Apache NiFi
  Issue Type: Task
  Components: Core Framework
Reporter: Joseph Witt
Priority: Minor

 Currently, the annotations are in the nifi.processor.annotation package.  
 These are giong to be applied to reporting tasks and controller services as 
 well and should be moved to nifi.annotation. and possibly sub-packages 
 thereof such as nifi.annotation.documentation and nifi.annotation.lifecycle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-5) Add details to reason messages of standard validators

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-5:
---
Component/s: Core Framework

 Add details to reason messages of standard validators
 -

 Key: NIFI-5
 URL: https://issues.apache.org/jira/browse/NIFI-5
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Joseph Witt
Priority: Minor

 The reason messages for the standard validators are vague and make it 
 difficult for admins/DFMs to diagnose certain problems.  For intance, when 
 one of our processors has properties for two or three directories.  If only 
 one does not exist it just says 'directory does not exist' but doesn't 
 indicate which property that was for.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-8) Increase support for different JVMs.

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238951#comment-14238951
 ] 

Joseph Witt commented on NIFI-8:


Chad Zobrisky czobri...@gmail.com via nifi.incubator.apache.org 
10:55 PM (11 minutes ago)

to dev 
Hello,
In response to the OpenJDK comment, Nifi would not build for me with
openjdk 1.7.0_65 on ubuntu 14.04.  It gave the following error:

ERROR org.apache.nifi.provenance.PersistentProvenanceRepository -
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should
not throw IOException, we just don't want to rely on that in production
(undocumented).

 Increase support for different JVMs.
 

 Key: NIFI-8
 URL: https://issues.apache.org/jira/browse/NIFI-8
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Joseph Witt
Priority: Minor

 - Java 8
 - Non Oracle JVMs.
 Need to test on a variety of common JDK types and sort out any issues.  There 
 should be no issues running on other JDKs but we need to validate that any 
 provide support for them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-9) Default scheduling should be event-driven for processors that support it

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-9:
---
Component/s: Core Framework

 Default scheduling should be event-driven for processors that support it
 

 Key: NIFI-9
 URL: https://issues.apache.org/jira/browse/NIFI-9
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Joseph Witt
Priority: Minor

 Event-driven scheduling should offer better performance by default.  Should 
 make it the default for supporting processors.  Also should consider a small 
 bounded limit for max concurrent event driven threads or unbounded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-20) Allow Processors to indicate that they do not allow for incoming connections

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-20:

Component/s: Core Framework

 Allow Processors to indicate that they do not allow for incoming connections
 

 Key: NIFI-20
 URL: https://issues.apache.org/jira/browse/NIFI-20
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Mark Payne
Priority: Minor

 An Annotation should be created that indicates that incoming connections are 
 not allowed.
 This UI should prevent the user from making such a Processor the destination 
 of a Connection.
 If a Processor IS configured with incoming connections and does not support 
 them (which could be the result if a processor that previously did not have 
 the annotation adds the Annotation), the framework should mark the Processor 
 as invalid, just as it does when a Relationship is not connected to anything.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-14) Invoke HTTP Processor does not allow the sending certain NiFi paramaters as Http Headers

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-14:

Component/s: Extensions

 Invoke HTTP Processor does not allow the sending certain NiFi paramaters as 
 Http Headers
 

 Key: NIFI-14
 URL: https://issues.apache.org/jira/browse/NIFI-14
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Joseph Witt
Priority: Minor

 Invoke HTTP should allow greater configuration options for controlling what 
 does and does not get sent as HTTP Headers from flow file attributes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-15) When restoring Swap Files, should detect corrupt swap file (EOFException) and remove it

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-15?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-15:

Component/s: Core Framework

 When restoring Swap Files, should detect corrupt swap file (EOFException) and 
 remove it
 ---

 Key: NIFI-15
 URL: https://issues.apache.org/jira/browse/NIFI-15
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor

 Otherwise, we get errors in the logs and we keep the Content that is 
 referenced by the FlowFiles that we were able to read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (NIFI-144) Test Failure: org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt closed NIFI-144.

Resolution: Not a Problem

 Test Failure: 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
 

 Key: NIFI-144
 URL: https://issues.apache.org/jira/browse/NIFI-144
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 0.0.1
 Environment: OS X 10.9
 Maven 3.2.3 (via homebrew)
 JDK 1.7.0_72
Reporter: Aldrin Piri

 Running org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewallTest
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Added 6 
 IP(s) to firewall.  Only requests originating from the configured IPs will be 
 accepted.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Loading 
 cluster firewall configuration.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - No IPs 
 added to firewall.  Firewall will accept all requests.
 [main] INFO 
 org.apache.nifi.cluster.firewall.impl.FileBasedClusterNodeFirewall - Cluster 
 firewall configuration loaded.
 [main] INFO 
 

[jira] [Updated] (NIFI-90) Replace explicit penalization with automatic penalization/back-off

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-90:

Priority: Minor  (was: Major)

 Replace explicit penalization with automatic penalization/back-off
 --

 Key: NIFI-90
 URL: https://issues.apache.org/jira/browse/NIFI-90
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Joseph Witt
Priority: Minor

 Rather than having users configure explicit penalization periods and 
 requiring developers to implement it in their processors we can automate 
 this.  Perhaps keep a LinkedHashMapConnection ID, Counter of size 5 or so 
 in the FlowFileRecord construct.  When a FlowFile is routed to a Connection, 
 the counter is incremented.  If the counter exceeds 3 visits to the same 
 connection, the FlowFile will be automatically penalized.  This protects us 
 5 hops out so that if we have something like DistributeLoad - PostHTTP 
 with failure looping back to DistributeLoad, we will still penalize when 
 appropriate.
 In addition, we will remove the configuration option from the UI, setting the 
 penalization period to some default such as 5 seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-97) NCM code needs to be refactored

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-97?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-97:

Priority: Minor  (was: Major)

 NCM code needs to be refactored
 ---

 Key: NIFI-97
 URL: https://issues.apache.org/jira/browse/NIFI-97
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Joseph Witt
Priority: Minor

 WebClusterManager is too complex.
 Specific Refactorings include:
 applyRequests has 4 different overridden signatures.  Instead, have a single 
 applyRequest that takes a NodeRequest object perhaps with a builder.  This 
 will include either an 'object entity' or 'mapstring,string parameters' as 
 well ass an optional SetNodeIdentifier and should also include the URI and 
 the HTTP method and headers.
 MergeResponses.
 Need to rename property 'nifi.cluster.manager.flow.retrieval.delay'.  It is 
 no used any time that we request that a node retry later as well as when we 
 retry to issue reconnection request.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-94) When controller-level bulletins pop up, UI's title should change

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-94:

Priority: Minor  (was: Major)

 When controller-level bulletins pop up, UI's title should change
 

 Key: NIFI-94
 URL: https://issues.apache.org/jira/browse/NIFI-94
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core UI
Reporter: Joseph Witt
Priority: Minor
 Fix For: 0.0.1


 to indicate so that users that are browsing different tabs are notified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-65) Update AuthorityProvider with #authorizeDownload()

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-65?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-65:

Priority: Minor  (was: Major)

 Update AuthorityProvider with #authorizeDownload()
 --

 Key: NIFI-65
 URL: https://issues.apache.org/jira/browse/NIFI-65
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor
 Fix For: 0.0.1


 - Also include call to AuthorityProvider to verify access based on flowfile 
 attributes.
 - Replace calls to nf.Common.submit that make GET requests with 
 window.open(...)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-23) Remove DFM requirement for Provenance Download/View action

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-23:

Priority: Minor  (was: Major)

 Remove DFM requirement for Provenance Download/View action
 --

 Key: NIFI-23
 URL: https://issues.apache.org/jira/browse/NIFI-23
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Matt Gilman
Priority: Minor

 After adding #authorizeDownload() to AuthorityProvider, we support download 
 decisions based on the FlowFile in question. As a result, we can relax the 
 DFM constraint on these actions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-4) Revisit annotations with Reporting Tasks and Controller Services

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4:
---
Priority: Minor  (was: Major)

 Revisit annotations with Reporting Tasks and Controller Services
 

 Key: NIFI-4
 URL: https://issues.apache.org/jira/browse/NIFI-4
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework, Core UI
Reporter: Joseph Witt
Priority: Minor

 Annotations are geared towards processors and their lifecycle.  We may need a 
 different set of annotations for reporting tasks and controller services as 
 their lifecycle management moves to configurable at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)