from:"\"JPercivall\""

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50158533
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,19 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
--- End diff --

This error handling should be expanded a bit more. In the event there is no 
incoming flowfile then the processor should yield when an error is occurred. 
Also the log statements should be changed a bit to better reflect the state the 
processor is in. For example the else statement says that the error is caused 
by the flowfile but that isn't true when it's a source processor (the flowfile 
is created to hold the content, doesn't influence the query).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50158594
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,19 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
-if (incoming == null) {
+if (fileToProcess == null) {
 logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure", new Object[] {selectQuery, e});
 } else {
-logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, incoming, e});
-session.transfer(incoming, REL_FAILURE);
+logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, fileToProcess, e});
+session.transfer(fileToProcess, REL_FAILURE);
--- End diff --

The Failure relationship states "Incoming FlowFile will be penalized and 
routed to this relationship". This needs to be added


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50159234
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -125,14 +125,14 @@ public ExecuteSQL() {
 
 @Override
 public void onTrigger(final ProcessContext context, final 
ProcessSession session) throws ProcessException {
-FlowFile incoming = null;
+FlowFile fileToProcess = null;
--- End diff --

Was confused at first the change to only have one flowfile but this is 
definitely an improvement to change to only process one flowfile. Prior it 
would take a flowfile and then it would magically disappear. The user would 
have no idea where it went or what happened.
Good catch Matt!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50174170
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,22 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
-if (incoming == null) {
+if (fileToProcess == null) {
+// This can happen if any exceptions occur while setting 
up the connection, statement, etc.
 logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure", new Object[] {selectQuery, e});
+context.yield();
 } else {
-logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, incoming, e});
-session.transfer(incoming, REL_FAILURE);
+logger.error("Unable to execute SQL select query {} due to 
{}; routing to failure", new Object[] {selectQuery, e});
--- End diff --

Almost there, need to check if there is an incoming connection 
(context.hasIncomingConnection) to see if the flowfile was used to created 
query. If so it should be logged (the way the incoming flowfile was).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50213360
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,31 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
-if (incoming == null) {
-logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure", new Object[] {selectQuery, e});
+if (fileToProcess == null) {
+// This can happen if any exceptions occur while setting 
up the connection, statement, etc.
+logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure",
+new Object[] {selectQuery, e});
+context.yield();
 } else {
-logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, incoming, e});
-session.transfer(incoming, REL_FAILURE);
+if(context.hasIncomingConnection()) {
+logger.error("Unable to execute SQL select query {} 
for {} due to {}; routing to failure",
+new Object[] {selectQuery, fileToProcess, e});
+  session.transfer(fileToProcess, 
REL_FAILURE);
--- End diff --

I think this snuck in there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50214114
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,31 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
-if (incoming == null) {
-logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure", new Object[] {selectQuery, e});
+if (fileToProcess == null) {
+// This can happen if any exceptions occur while setting 
up the connection, statement, etc.
+logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure",
+new Object[] {selectQuery, e});
+context.yield();
 } else {
-logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, incoming, e});
-session.transfer(incoming, REL_FAILURE);
+if(context.hasIncomingConnection()) {
+logger.error("Unable to execute SQL select query {} 
for {} due to {}; routing to failure",
+new Object[] {selectQuery, fileToProcess, e});
+  session.transfer(fileToProcess, 
REL_FAILURE);
+}
+else {
--- End diff --

Having the "else" on the second causes contrib-check to fail.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1409: Fix missing transfer on error in Exe...

2016-01-25 Thread JPercivall

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/178#discussion_r50726859
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteSQL.java
 ---
@@ -164,17 +166,31 @@ public void process(final OutputStream out) throws 
IOException {
 });
 
 // set attribute how many rows were selected
-outgoing = session.putAttribute(outgoing, RESULT_ROW_COUNT, 
nrOfRows.get().toString());
+fileToProcess = session.putAttribute(fileToProcess, 
RESULT_ROW_COUNT, nrOfRows.get().toString());
 
-logger.info("{} contains {} Avro records; transferring to 
'success'", new Object[] {outgoing, nrOfRows.get()});
-session.getProvenanceReporter().modifyContent(outgoing, 
"Retrieved " + nrOfRows.get() + " rows", 
stopWatch.getElapsed(TimeUnit.MILLISECONDS));
-session.transfer(outgoing, REL_SUCCESS);
+logger.info("{} contains {} Avro records; transferring to 
'success'",
+new Object[] {fileToProcess, nrOfRows.get()});
+session.getProvenanceReporter().modifyContent(fileToProcess, 
"Retrieved " + nrOfRows.get() + " rows",
+stopWatch.getElapsed(TimeUnit.MILLISECONDS));
+session.transfer(fileToProcess, REL_SUCCESS);
 } catch (final ProcessException | SQLException e) {
-if (incoming == null) {
-logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure", new Object[] {selectQuery, e});
+if (fileToProcess == null) {
+// This can happen if any exceptions occur while setting 
up the connection, statement, etc.
+logger.error("Unable to execute SQL select query {} due to 
{}. No incoming flow file to route to failure",
+new Object[] {selectQuery, e});
+context.yield();
 } else {
-logger.error("Unable to execute SQL select query {} for {} 
due to {}; routing to failure", new Object[] {selectQuery, incoming, e});
-session.transfer(incoming, REL_FAILURE);
+if(context.hasIncomingConnection()) {
+logger.error("Unable to execute SQL select query {} 
for {} due to {}; routing to failure",
+new Object[] {selectQuery, fileToProcess, e});
+  session.transfer(fileToProcess, 
REL_FAILURE);
--- End diff --

Sorry, specifically the "session.transfer(fileToProcess, REL_FAILURE);" line


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/180#issuecomment-175086276
  
Could the unit tests be fleshed out a bit? The core method in 
AbstractElasticsearchProcessor (createClient) doesn't have any coverage and 
there are only two general tests run by default per processor. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/180#issuecomment-176433804
  
In the latest commit there is more code coverage but there are still only a 
couple tests. Ideally I would like to see tests that work with each of the 
configuration properties (verifying they work as expected) and some that work 
with an embedded version of ES.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/183#issuecomment-176533014
  
I believe flowfiles should be removed from the penalized list when 
MockProcessSession.remove() is called: 
https://github.com/apache/nifi/blob/fd35b8ffd7d4a66ef149c1bd1341dfee3ede80c7/nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java#L430


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/183#issuecomment-176540108
  
When MockProcessSession.rollback(boolean)[1] is called it is, the boolean 
is telling the session whether or not the flowfiles should be penalized. In the 
event the boolean is true if the session shouldn't it clear the penalized list 
then add all the flowfiles to the list?

[1] 
https://github.com/apache/nifi/blob/fd35b8ffd7d4a66ef149c1bd1341dfee3ede80c7/nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java#L522-L522


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/183#issuecomment-176544259
  
I know this isn't part of the ticket but, there are numerous lines in the 
unit tests that say " // getMyFlowFiles();". I believe they're relicts from 
before the refactor. Could you remove them in this commit?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/183#issuecomment-176546233
  
Did a maven contrib check build and verified the functionality using a test 
http server. Once the comments are addressed I can merge in.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1434 Prevent array index exception in PutE...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/187#issuecomment-176558911
  
There 100% needs to be better unit testing for the PutEmail processor and I 
agree that this ticket is outside the scope of that. I've created a new one 
here[1]. Stubbing things out is a valid approach since the unit test can't 
embed an smtp server. Feel free to comment there with more ideas!

[1] https://issues.apache.org/jira/browse/NIFI-1449


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1434 Prevent array index exception in PutE...

2016-01-29 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/187#issuecomment-176807151
  
@rpmiskin 
Following the RFC 822 guideline (that the MimeMessage class is based on) it 
says this about the "From" header: 
4.4.1. FROM / RESENT-FROM

   This field contains the identity of the person(s)  who  wished
   this  message to be sent.  The message-creation process should
   default this field  to  be  a  single,  authenticated  machine
   address,  indicating  the  AGENT  (person,  system or process)
   entering the message.  If this is not done, the "Sender" field
   MUST  be  present.  If the "From" field IS defaulted this way,
   the "Sender" field is  optional  and  is  redundant  with  the
   "From"  field.   In  all  cases, addresses in the "From" field
   must be machine-usable (addr-specs) and may not contain  named
   lists (groups).

MimeMessage doesn't recognize the "Sender" field as being different from 
"From" (in comments uses the terms interchangeably) so I'd say leave the "From" 
header as required.

I am on board with creating a custom validator for the internet addresses 
though, that would be great.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/180#issuecomment-178023848
  
If the "Identifier Attribute" is being left as the attribute containing the 
identifier then the "ATTRIBUTE_KEY_VALIDATOR" should be used instead of 
"NON_EMPTY_VALIDATOR".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/180#discussion_r51482142
  
--- Diff: 
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchProcessor.java
 ---
@@ -0,0 +1,227 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.elasticsearch;
+
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.util.StringUtils;
+import org.elasticsearch.client.Client;
+import org.elasticsearch.client.transport.TransportClient;
+import org.elasticsearch.common.settings.Settings;
+import org.elasticsearch.common.transport.InetSocketTransportAddress;
+
+import java.net.InetSocketAddress;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.concurrent.atomic.AtomicReference;
+
+
+public abstract class AbstractElasticsearchProcessor extends 
AbstractProcessor {
+
+protected static final PropertyDescriptor CLUSTER_NAME = new 
PropertyDescriptor.Builder()
+.name("Cluster Name")
+.description("Name of the ES cluster (for example, 
elasticsearch_brew). Defaults to 'elasticsearch'")
+.required(false)
+.addValidator(Validator.VALID)
+.defaultValue("elasticsearch")
+.build();
+protected static final PropertyDescriptor HOSTS = new 
PropertyDescriptor.Builder()
+.name("ElasticSearch Hosts")
+.description("ElasticSearch Hosts, which should be comma 
separated and colon for hostname/port "
++ "host1:port,host2:port,  For example 
testcluster:9200.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PROP_SSL_CONTEXT_SERVICE = new 
PropertyDescriptor.Builder()
+.name("SSL Context Service")
+.description("The SSL Context Service used to provide client 
certificate information for TLS/SSL "
++ "connections. This service only applies if the 
Shield plugin is available.")
+.required(false)
+.identifiesControllerService(SSLContextService.class)
+.build();
+
+public static final PropertyDescriptor PROP_SHIELD_LOCATION = new 
PropertyDescriptor.Builder()
+.name("Shield plugin URL")
+.description("Specifies the location of the JAR for the 
Elasticsearch Shield plugin."
++ " If the Elasticsearch cluster has been secured with 
the Shield plugin, then "
++ "an SSL Context Service must be defined, and the 
Shield plugin JAR must also "
++ "be available to this processor.")
+.required(false)
+.addValidator(StandardValidators.URL_VALIDATOR)
--- End diff --

I believe we should force this to be a file path instead of a URL so there 
isn't a network call out to grab the jar every time the client needs to restart.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/180#issuecomment-178198337
  
There needs to be support added for passing username/password (specifically 
for shield integration)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/180#discussion_r51483875
  
--- Diff: 
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchProcessor.java
 ---
@@ -0,0 +1,227 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.elasticsearch;
+
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.util.StringUtils;
+import org.elasticsearch.client.Client;
+import org.elasticsearch.client.transport.TransportClient;
+import org.elasticsearch.common.settings.Settings;
+import org.elasticsearch.common.transport.InetSocketTransportAddress;
+
+import java.net.InetSocketAddress;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.concurrent.atomic.AtomicReference;
+
+
+public abstract class AbstractElasticsearchProcessor extends 
AbstractProcessor {
+
+protected static final PropertyDescriptor CLUSTER_NAME = new 
PropertyDescriptor.Builder()
+.name("Cluster Name")
+.description("Name of the ES cluster (for example, 
elasticsearch_brew). Defaults to 'elasticsearch'")
+.required(false)
+.addValidator(Validator.VALID)
+.defaultValue("elasticsearch")
+.build();
+protected static final PropertyDescriptor HOSTS = new 
PropertyDescriptor.Builder()
+.name("ElasticSearch Hosts")
+.description("ElasticSearch Hosts, which should be comma 
separated and colon for hostname/port "
++ "host1:port,host2:port,  For example 
testcluster:9200.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PROP_SSL_CONTEXT_SERVICE = new 
PropertyDescriptor.Builder()
+.name("SSL Context Service")
+.description("The SSL Context Service used to provide client 
certificate information for TLS/SSL "
++ "connections. This service only applies if the 
Shield plugin is available.")
+.required(false)
+.identifiesControllerService(SSLContextService.class)
+.build();
+
+public static final PropertyDescriptor PROP_SHIELD_LOCATION = new 
PropertyDescriptor.Builder()
+.name("Shield plugin URL")
+.description("Specifies the location of the JAR for the 
Elasticsearch Shield plugin."
++ " If the Elasticsearch cluster has been secured with 
the Shield plugin, then "
--- End diff --

A bit nit picky but, the space at the beginning of this string should be at 
the end of the line before


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/180#discussion_r51490929
  
--- Diff: 
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchProcessor.java
 ---
@@ -0,0 +1,227 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.elasticsearch;
+
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.util.StringUtils;
+import org.elasticsearch.client.Client;
+import org.elasticsearch.client.transport.TransportClient;
+import org.elasticsearch.common.settings.Settings;
+import org.elasticsearch.common.transport.InetSocketTransportAddress;
+
+import java.net.InetSocketAddress;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.concurrent.atomic.AtomicReference;
+
+
+public abstract class AbstractElasticsearchProcessor extends 
AbstractProcessor {
+
+protected static final PropertyDescriptor CLUSTER_NAME = new 
PropertyDescriptor.Builder()
+.name("Cluster Name")
+.description("Name of the ES cluster (for example, 
elasticsearch_brew). Defaults to 'elasticsearch'")
+.required(false)
+.addValidator(Validator.VALID)
+.defaultValue("elasticsearch")
+.build();
+protected static final PropertyDescriptor HOSTS = new 
PropertyDescriptor.Builder()
+.name("ElasticSearch Hosts")
+.description("ElasticSearch Hosts, which should be comma 
separated and colon for hostname/port "
++ "host1:port,host2:port,  For example 
testcluster:9200.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PROP_SSL_CONTEXT_SERVICE = new 
PropertyDescriptor.Builder()
+.name("SSL Context Service")
+.description("The SSL Context Service used to provide client 
certificate information for TLS/SSL "
++ "connections. This service only applies if the 
Shield plugin is available.")
+.required(false)
+.identifiesControllerService(SSLContextService.class)
+.build();
+
+public static final PropertyDescriptor PROP_SHIELD_LOCATION = new 
PropertyDescriptor.Builder()
+.name("Shield plugin URL")
+.description("Specifies the location of the JAR for the 
Elasticsearch Shield plugin."
++ " If the Elasticsearch cluster has been secured with 
the Shield plugin, then "
++ "an SSL Context Service must be defined, and the 
Shield plugin JAR must also "
++ "be available to this processor.")
+.required(false)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+
+protected static final PropertyDescriptor PING_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("ElasticSearch Ping Timeout")
+.description("The ping timeout used to determine when a node 
is unreachable. " +

[GitHub] nifi pull request: NIFI-1420 Adding Splunk bundle

2016-03-01 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/233#issuecomment-190934511

I observed a couple of interesting interactions when testing the PutSplunk
processor. I'm not sure whether it is a problem with the processor, the Splunk
Java SDK, Splunk Enterprise or my Splunk configurations:
* I set up the PutSplunk processor to communicate via UDP and created a UDP
data input using the UI for Splunk Enterprise. The PutSplunk is transmitting
messages at ~1 per second. While it's transmitting no new events are created
yet in Splunk Enterprise (verified by viewing a real time view of the past 30
seconds). I stop the PutSplunk processor and then run a search in Splunk UI to
see if any events came in on that port. There is one event that is registered
at when I started the processor and contains effectively all (didn't count the
seconds I was sending) the data that was generated. I'm not sure why all the
UDP messages are grouped together. When I set PutSplunk to send messages every
20 seconds the UDP messages are treated as their own event.
* I set up PutSplunk to communicate via TCP using SSL. I looked for options
to create a Data Input in Splunk Enterprise using TCP and SSL but couldn't find
anything definitive (and all my searches turned up results for the Forwarder).
I enabled SSL in Splunk Web by going to the general settings and notice that
SSL properties are in the inputs.conf and server.conf files. I attempt to send
data using PutSplunk using SSL to the TCP data input. The event is received and
I see it in the Splunk UI in real time. The only problem being that it's still
encoded (see image). Typically when I get errors with SSL there is some obscure
error relating to truncation attacks or Cipher Suites but I didn't seem to get
any errors.
![screen shot 2016-03-01 at 5 21 54
pm](https://cloud.githubusercontent.com/assets/11302527/13444066/2357d1e2-dfd2-11e5-8150-9684b7299a6d.png)

---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1420 Adding Splunk bundle

2016-03-01 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/233#issuecomment-190970047
  
Whenever an SSL context is being created (done 4 times in multiple 
different classes) "SSLContextService.ClientAuth.REQUIRED" is used. Should we 
offer configuration options for this? For PutSplunk and GetSplunk I can 
understand it because we are requiring the server to provide authorization but 
maybe someone using ListenSplunkForwader doesn't need it to verify who's 
sending the data.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1420 Adding Splunk bundle

2016-03-04 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/233#issuecomment-192325813
  
Should have made this comment on the first commit but can the LogGenerator 
be put into a util package on the same path? It makes it more readable to only 
have the test classes in package "org.apache.nifi.processors.splunk".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1420 Adding Splunk bundle

2016-03-07 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/233#issuecomment-193459708
  
Just finished looking over the code and doing a round of tests. Everything 
looks good.

+1, Thanks for the contribution Bryan, looks like a very powerful new 
addition


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-194997331
  
Reviewing


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-195006367
  
Following the pattern for Controller Services and their API in 
nifi-standard-services, JMSConnectionFactoryProviderDefinition should be in 
it's own package. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1491 Throws exception when unable to delet...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/227#issuecomment-195028936
  
@trkurc what was the result of your review?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1518 InferAvroSchema note has an option to...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/235#issuecomment-195030029
  
@trkurc what was the result of your review?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-195413029
  
New commits fail contrib-check for these reasons:

[INFO] --- maven-checkstyle-plugin:2.15:check (check-style) @ 
nifi-jms-processors ---
[WARNING] 
src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java[63] (javadoc) 
NonEmptyAtclauseDescription: At-clause should have a non-empty description.
[WARNING] 
src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java[71] (javadoc) 
NonEmptyAtclauseDescription: At-clause should have a non-empty description.
[WARNING] 
src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java[72] (javadoc) 
NonEmptyAtclauseDescription: At-clause should have a non-empty description.
[WARNING] 
src/main/java/org/apache/nifi/jms/processors/MessageBodyToBytesConverter.java[35]
 (javadoc) NonEmptyAtclauseDescription: At-clause should have a non-empty 
description.
[WARNING] 
src/main/java/org/apache/nifi/jms/processors/MessageBodyToBytesConverter.java[36]
 (javadoc) NonEmptyAtclauseDescription: At-clause should have a non-empty 
description.
[WARNING] 
src/main/java/org/apache/nifi/jms/processors/MessageBodyToBytesConverter.java[48]
 (javadoc) NonEmptyAtclauseDescription: At-clause should have a non-empty 
description.
[WARNING] 
src/main/java/org/apache/nifi/jms/processors/MessageBodyToBytesConverter.java[49]
 (javadoc) NonEmptyAtclauseDescription: At-clause should have a non-empty 
description.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-195427093
  
What was the motivation for creating an additional details page for the 
Publish/Consume processors that is just a summary and an overview of the 
configuration? Both of those could be conveyed using the CapabilityDescription 
tag and the property descriptions. It also makes it harder to maintain. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-195428868
  
Why are the docs needed as place holders? It is not a breaking change to 
adding additional documentation pages in the future. Details regarding simple 
capability description and processor property descriptors should follow the 
same convention other components follow. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-195528105
  
Build is still failing contrib check after the latest patches due to:

[INFO] --- maven-checkstyle-plugin:2.15:check (check-style) @ 
nifi-jms-cf-service ---
[WARNING] 
src/main/java/org/apache/nifi/jms/cf/JMSConnectionFactoryProvider.java[58] 
(regexp) RegexpSinglelineJava: Line has trailing whitespace.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

2016-03-14 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-196527092
  
@mosermw shouldn't you just be able to create a typical SSL controller 
service and set the "SSL Context Service" property to point to it? This should 
automatically use the properties set in the SSL controller service.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56175269
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/PutCassandraQL.java
 ---
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.BoundStatement;
+import com.datastax.driver.core.DataType;
+import com.datastax.driver.core.PreparedStatement;
+import com.datastax.driver.core.ResultSetFuture;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.exceptions.AuthenticationException;
+import com.datastax.driver.core.exceptions.InvalidTypeException;
+import com.datastax.driver.core.exceptions.NoHostAvailableException;
+import com.datastax.driver.core.exceptions.QueryExecutionException;
+import com.datastax.driver.core.exceptions.QueryValidationException;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnShutdown;
+import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+@SupportsBatching
+@Tags({"cassandra", "cql", "put", "insert", "update", "set"})
+@EventDriven
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Execute provided Cassandra Query Language (CQL) 
statement. The content of an incoming FlowFile "
++ "is expected to be the CQL command to execute. The CQL command 
may use the ? to escape parameters. In this "
++ "case, the parameters to use must exist as FlowFile attributes 
with the naming convention cql.args.N.type "
++ "and cql.args.N.value, where N is a positive integer. The 
cql.args.N.type is expected to be "
++ "a lowercase string indicating the Cassandra type. The content 
of the FlowFile is expected to be in UTF-8 format.")
--- End diff --

"The content of the FlowFile is expected to be in UTF-8 format." After 
adding the charset property I don't believe this is the case.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56178725
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/PutCassandraQL.java
 ---
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.BoundStatement;
+import com.datastax.driver.core.DataType;
+import com.datastax.driver.core.PreparedStatement;
+import com.datastax.driver.core.ResultSetFuture;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.exceptions.AuthenticationException;
+import com.datastax.driver.core.exceptions.InvalidTypeException;
+import com.datastax.driver.core.exceptions.NoHostAvailableException;
+import com.datastax.driver.core.exceptions.QueryExecutionException;
+import com.datastax.driver.core.exceptions.QueryValidationException;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnShutdown;
+import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+@SupportsBatching
+@Tags({"cassandra", "cql", "put", "insert", "update", "set"})
+@EventDriven
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Execute provided Cassandra Query Language (CQL) 
statement. The content of an incoming FlowFile "
++ "is expected to be the CQL command to execute. The CQL command 
may use the ? to escape parameters. In this "
++ "case, the parameters to use must exist as FlowFile attributes 
with the naming convention cql.args.N.type "
++ "and cql.args.N.value, where N is a positive integer. The 
cql.args.N.type is expected to be "
++ "a lowercase string indicating the Cassandra type. The content 
of the FlowFile is expected to be in UTF-8 format.")
+@ReadsAttributes({
+@ReadsAttribute(attribute = "cql.args.N.type",
+description = "Incoming FlowFiles are expected to be 
parameterized CQL statements. The type of each "
++ "parameter is specified as a lowercase string 
corresponding to the Cassandra data type (text, "
+

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56187278
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/PutCassandraQL.java
 ---
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.BoundStatement;
+import com.datastax.driver.core.DataType;
+import com.datastax.driver.core.PreparedStatement;
+import com.datastax.driver.core.ResultSetFuture;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.exceptions.AuthenticationException;
+import com.datastax.driver.core.exceptions.InvalidTypeException;
+import com.datastax.driver.core.exceptions.NoHostAvailableException;
+import com.datastax.driver.core.exceptions.QueryExecutionException;
+import com.datastax.driver.core.exceptions.QueryValidationException;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnShutdown;
+import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+@SupportsBatching
+@Tags({"cassandra", "cql", "put", "insert", "update", "set"})
+@EventDriven
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Execute provided Cassandra Query Language (CQL) 
statement. The content of an incoming FlowFile "
++ "is expected to be the CQL command to execute. The CQL command 
may use the ? to escape parameters. In this "
++ "case, the parameters to use must exist as FlowFile attributes 
with the naming convention cql.args.N.type "
++ "and cql.args.N.value, where N is a positive integer. The 
cql.args.N.type is expected to be "
++ "a lowercase string indicating the Cassandra type. The content 
of the FlowFile is expected to be in UTF-8 format.")
+@ReadsAttributes({
+@ReadsAttribute(attribute = "cql.args.N.type",
+description = "Incoming FlowFiles are expected to be 
parameterized CQL statements. The type of each "
++ "parameter is specified as a lowercase string 
corresponding to the Cassandra data type (text, "
+

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56191239
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/PutCassandraQLTest.java
 ---
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.BoundStatement;
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.Configuration;
+import com.datastax.driver.core.ConsistencyLevel;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.PreparedStatement;
+import com.datastax.driver.core.ResultSet;
+import com.datastax.driver.core.ResultSetFuture;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.Statement;
+import com.datastax.driver.core.exceptions.InvalidQueryException;
+import com.datastax.driver.core.exceptions.NoHostAvailableException;
+import com.datastax.driver.core.exceptions.UnavailableException;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.Before;
+import org.junit.Test;
+
+import javax.net.ssl.SSLContext;
+import java.net.InetSocketAddress;
+import java.util.HashMap;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+import static org.junit.Assert.fail;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.anyLong;
+import static org.mockito.Matchers.anyString;
+import static org.mockito.Mockito.doReturn;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * Unit tests for the PutCassandraQL processor
+ */
+public class PutCassandraQLTest {
+
+private TestRunner testRunner;
+private MockPutCassandraQL processor;
+
+@Before
+public void setUp() throws Exception {
+processor = new MockPutCassandraQL();
+testRunner = TestRunners.newTestRunner(processor);
+}
+
+@Test
+public void testProcessor() {
+testRunner.setProperty(AbstractCassandraProcessor.CONTACT_POINTS, 
"localhost:9042");
+testRunner.assertValid();
+testRunner.setProperty(AbstractCassandraProcessor.PASSWORD, 
"password");
+testRunner.assertNotValid();
+testRunner.setProperty(AbstractCassandraProcessor.USERNAME, 
"username");
+
testRunner.setProperty(AbstractCassandraProcessor.CONSISTENCY_LEVEL, "ONE");
+testRunner.assertValid();
+
+testRunner.enqueue("INSERT INTO users (user_id, first_name, 
last_name, properties) VALUES ?, ?, ?, ?",
+new HashMap() {
+{
+put("cql.args.1.type", "int");
+put("cql.args.1.value", "1");
+put("cql.args.2.type", "text");
+put("cql.args.2.value", "Joe");
+put("cql.args.3.type", "text");
+// No value for arg 3 to test setNull
+put("cql.args.4.type", "map");
+put("cql.args.4.value", "{'a':'Hello', 
'b':'World'}");
+put("cql.args.5.type", "list");
+put("cql.args.5.value", "[true,false,true]");
+put("cql.args.6.type", "set");
+

[GitHub] nifi pull request: NIFI-614 Added initial support for new style JM...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/222#issuecomment-196988395
  
I am a +1 on merging this PR in (once commits have been squashed). 

I reviewed the code, did a contrib check build and did basic testing with 
ActiveMQ. With @digitalplummer having done more extensive testing with other 
JMS brokers, I am satisfied with the PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56235566
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/AbstractCassandraProcessor.java
 ---
@@ -0,0 +1,463 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.ConsistencyLevel;
+import com.datastax.driver.core.DataType;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.Row;
+import com.datastax.driver.core.SSLOptions;
+import com.datastax.driver.core.Session;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.authorization.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
+
+import javax.net.ssl.SSLContext;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+/**
+ * AbstractCassandraProcessor is a base class for Cassandra processors and 
contains logic and variables common to most
+ * processors integrating with Apache Cassandra.
+ */
+public abstract class AbstractCassandraProcessor extends AbstractProcessor 
{
+
+public static final int DEFAULT_CASSANDRA_PORT = 9042;
+
+private static final Validator HOSTNAME_PORT_VALIDATOR = new 
Validator() {
+@Override
+public ValidationResult validate(final String subject, final 
String input, final ValidationContext context) {
+final List esList = Arrays.asList(input.split(","));
+for (String hostnamePort : esList) {
+String[] addresses = hostnamePort.split(":");
+// Protect against invalid input like 
http://127.0.0.1:9042 (URL scheme should not be there)
+if (addresses.length != 2) {
+return new 
ValidationResult.Builder().subject(subject).input(input).explanation(
+"Each entry must be in hostname:port form (no 
scheme such as http://, and port must be specified)")
+.valid(false).build();
+}
+// Validate the port
+String port = addresses[1].trim();
+ValidationResult portValidatorResult = 
StandardValidators.PORT_VALIDATOR.validate(subject, port, context);
+if (!portValidatorResult.isValid()) {
+return portValidatorResult;
+}
+
+}
+return new 
ValidationResult.Builder().subject(subject).input(input).explanation(
+"Valid cluster definition").valid(true).build();
+}
+};
+
+// Common descriptors
+public static final PropertyDescriptor CONTACT_POINTS = new 
PropertyDescriptor.Builder()
+.name("Cassandra Contact Points")
+.description("Contact poi

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56236638
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/AbstractCassandraProcessor.java
 ---
@@ -0,0 +1,463 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.ConsistencyLevel;
+import com.datastax.driver.core.DataType;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.Row;
+import com.datastax.driver.core.SSLOptions;
+import com.datastax.driver.core.Session;
+import org.apache.avro.Schema;
+import org.apache.avro.SchemaBuilder;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.authorization.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.security.util.SslContextFactory;
+import org.apache.nifi.ssl.SSLContextService;
+
+import javax.net.ssl.SSLContext;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+/**
+ * AbstractCassandraProcessor is a base class for Cassandra processors and 
contains logic and variables common to most
+ * processors integrating with Apache Cassandra.
+ */
+public abstract class AbstractCassandraProcessor extends AbstractProcessor 
{
+
+public static final int DEFAULT_CASSANDRA_PORT = 9042;
+
+private static final Validator HOSTNAME_PORT_VALIDATOR = new 
Validator() {
+@Override
+public ValidationResult validate(final String subject, final 
String input, final ValidationContext context) {
+final List esList = Arrays.asList(input.split(","));
+for (String hostnamePort : esList) {
+String[] addresses = hostnamePort.split(":");
+// Protect against invalid input like 
http://127.0.0.1:9042 (URL scheme should not be there)
+if (addresses.length != 2) {
+return new 
ValidationResult.Builder().subject(subject).input(input).explanation(
+"Each entry must be in hostname:port form (no 
scheme such as http://, and port must be specified)")
+.valid(false).build();
+}
+// Validate the port
+String port = addresses[1].trim();
+ValidationResult portValidatorResult = 
StandardValidators.PORT_VALIDATOR.validate(subject, port, context);
+if (!portValidatorResult.isValid()) {
+return portValidatorResult;
+}
+
+}
+return new 
ValidationResult.Builder().subject(subject).input(input).explanation(
+"Valid cluster definition").valid(true).build();
+}
+};
+
+// Common descriptors
+public static final PropertyDescriptor CONTACT_POINTS = new 
PropertyDescriptor.Builder()
+.name("Cassandra Contact Points")
+.description("Contact poi

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56237303
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/PutCassandraQLTest.java
 ---
@@ -0,0 +1,219 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.BoundStatement;
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.Configuration;
+import com.datastax.driver.core.ConsistencyLevel;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.PreparedStatement;
+import com.datastax.driver.core.ResultSet;
+import com.datastax.driver.core.ResultSetFuture;
+import com.datastax.driver.core.Session;
+import com.datastax.driver.core.Statement;
+import com.datastax.driver.core.exceptions.InvalidQueryException;
+import com.datastax.driver.core.exceptions.NoHostAvailableException;
+import com.datastax.driver.core.exceptions.UnavailableException;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.Before;
+import org.junit.Test;
+
+import javax.net.ssl.SSLContext;
+import java.net.InetSocketAddress;
+import java.util.HashMap;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+import static org.junit.Assert.fail;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.anyLong;
+import static org.mockito.Matchers.anyString;
+import static org.mockito.Mockito.doReturn;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * Unit tests for the PutCassandraQL processor
+ */
+public class PutCassandraQLTest {
+
+private TestRunner testRunner;
+private MockPutCassandraQL processor;
+
+@Before
+public void setUp() throws Exception {
+processor = new MockPutCassandraQL();
+testRunner = TestRunners.newTestRunner(processor);
+}
+
+@Test
+public void testProcessorConfigValidity() {
+testRunner.setProperty(AbstractCassandraProcessor.CONTACT_POINTS, 
"localhost:9042");
+testRunner.assertValid();
+testRunner.setProperty(AbstractCassandraProcessor.PASSWORD, 
"password");
+testRunner.assertNotValid();
+testRunner.setProperty(AbstractCassandraProcessor.USERNAME, 
"username");
+
testRunner.setProperty(AbstractCassandraProcessor.CONSISTENCY_LEVEL, "ONE");
+testRunner.assertValid();
+}
+
+@Test
+public void testProcessorHappyPath() {
+setUpStandardTestConfig();
+
+testRunner.run(1, true, true);
+
testRunner.assertAllFlowFilesTransferred(PutCassandraQL.REL_SUCCESS, 1);
+testRunner.clearTransferState();
+}
+
+@Test
+public void testProcessorInvalidQueryException() {
+setUpStandardTestConfig();
+
+// Test exceptions
+processor.setExceptionToThrow(
+new InvalidQueryException(new 
InetSocketAddress("localhost", 9042), "invalid query"));
+testRunner.enqueue("UPDATE users SET cities = [ 'New York', 'Los 
Angeles' ] WHERE user_id = 'coast2coast';");
+testRunner.run(1, true, true);
+
testRunner.assertAllFlowFilesTransferred(PutCassandraQL.REL_FAILURE, 1);
+testRunner.clearTransferState();
+}
+
+@Test
+public void testProcessorUnavailableException() {
+setUpStandardTestConfig();
+
+processor.setExceptionToTh

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/237#discussion_r56237562
  
--- Diff: 
nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/AbstractCassandraProcessorTest.java
 ---
@@ -0,0 +1,284 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cassandra;
+
+import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.Configuration;
+import com.datastax.driver.core.DataType;
+import com.datastax.driver.core.Metadata;
+import com.datastax.driver.core.Row;
+import com.google.common.collect.Sets;
+import org.apache.nifi.authorization.exception.ProviderCreationException;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.Before;
+import org.junit.Test;
+
+import javax.net.ssl.SSLContext;
+import java.net.InetSocketAddress;
+import java.util.Arrays;
+import java.util.Calendar;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+
+/**
+ * Unit tests for the AbstractCassandraProcessor class
+ */
+public class AbstractCassandraProcessorTest {
+
+MockAbstractCassandraProcessor processor;
+private TestRunner testRunner;
+
+@Before
+public void setUp() throws Exception {
+processor = new MockAbstractCassandraProcessor();
+testRunner = TestRunners.newTestRunner(processor);
+}
+
+@Test
+public void testCustomValidate() throws Exception {
--- End diff --

Since you already have a testCustomValidate, might as well add check for 
testing a port that isn't in the valid range.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1274

2016-03-18 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/284#issuecomment-197582465
  
There are multiple "todo" comments. What is the plan for them? It appears 
that some are just comments on the code, some can be removed (already done), 
and a few could become follow-up tickets after committing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-901: Add QueryCassandra and PutCassandraQL...

2016-03-18 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/237#issuecomment-197461709
  
+1

Did a contrib check build, verified comments were addressed and ran through 
a Put and Query test with Cassandra 2.2.5 with no username/password, with just 
username/password and with SSL+username/password.

Good work!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1274

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/284#discussion_r56507392
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/JwtService.java
 ---
@@ -70,8 +70,6 @@ public String getAuthenticationFromToken(final String 
base64EncodedToken) throws
 
 // TODO: Validate issuer against active registry?
--- End diff --

Reason for keeping this TODO?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1274

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/284#issuecomment-197944422
  
+1 (with Gilman's last modifications)

Used Gilman's branch to do a contrib check build, reviewed that comments 
were addressed and ran through setting up a kerberos secure nifi instance. 
Looks good.

Thanks for contributing this awesome new feature!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1274

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/284#discussion_r56425560
  
--- Diff: 
nifi-nar-bundles/nifi-kerberos-iaa-providers-bundle/nifi-kerberos-iaa-providers/src/main/java/org/apache/nifi/kerberos/KerberosProvider.java
 ---
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.kerberos;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.authentication.AuthenticationResponse;
+import org.apache.nifi.authentication.LoginCredentials;
+import org.apache.nifi.authentication.LoginIdentityProvider;
+import 
org.apache.nifi.authentication.LoginIdentityProviderConfigurationContext;
+import 
org.apache.nifi.authentication.LoginIdentityProviderInitializationContext;
+import org.apache.nifi.authentication.exception.IdentityAccessException;
+import 
org.apache.nifi.authentication.exception.InvalidLoginCredentialsException;
+import org.apache.nifi.authorization.exception.ProviderCreationException;
+import 
org.apache.nifi.authorization.exception.ProviderDestructionException;
+import org.apache.nifi.util.FormatUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import 
org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
+import org.springframework.security.core.Authentication;
+import org.springframework.security.core.AuthenticationException;
+import 
org.springframework.security.kerberos.authentication.KerberosAuthenticationProvider;
+import 
org.springframework.security.kerberos.authentication.sun.SunJaasKerberosClient;
+
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Kerberos-based implementation of a login identity provider.
+ */
+public class KerberosProvider implements LoginIdentityProvider {
+
+private static final Logger logger = 
LoggerFactory.getLogger(KerberosProvider.class);
+
+private KerberosAuthenticationProvider provider;
+private String issuer;
+private long expiration;
+
+@Override
+public final void initialize(final 
LoginIdentityProviderInitializationContext initializationContext) throws 
ProviderCreationException {
+this.issuer = getClass().getSimpleName();
+}
+
+@Override
+public final void onConfigured(final 
LoginIdentityProviderConfigurationContext configurationContext) throws 
ProviderCreationException {
+final String rawExpiration = 
configurationContext.getProperty("Authentication Expiration");
+if (StringUtils.isBlank(rawExpiration)) {
+throw new ProviderCreationException("The Authentication 
Expiration must be specified.");
+}
+
+try {
+expiration = FormatUtils.getTimeDuration(rawExpiration, 
TimeUnit.MILLISECONDS);
+} catch (final IllegalArgumentException iae) {
+throw new ProviderCreationException(String.format("The 
Expiration Duration '%s' is not a valid time duration", rawExpiration));
+}
+
+provider = new KerberosAuthenticationProvider();
+SunJaasKerberosClient client = new SunJaasKerberosClient();
+client.setDebug(true);
+provider.setKerberosClient(client);
+provider.setUserDetailsService(new KerberosUserDetailsService());
+}
+
+@Override
+public final AuthenticationResponse authenticate(final 
LoginCredentials credentials) throws InvalidLoginCredentialsException, 
IdentityAccessException {
+if (provider == null) {
+throw new IdentityAccessException("The Kerberos authentication 
provider is not initialized.");
+}
+
+try {
+// TODO: Remove debug statements
+logger.info("[REMOVE] Attempting to authenticate Kerberos user 
{} with password {}", credentials.getUsername(), credentials.getPassword

[GitHub] nifi pull request: Nifi 1274

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/284#discussion_r56410434
  
--- Diff: 
nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-ui/src/main/java/org/apache/nifi/update/attributes/api/RuleResource.java
 ---
@@ -16,66 +16,63 @@
  */
 package org.apache.nifi.update.attributes.api;
 
-import java.text.Collator;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.Comparator;
-import java.util.List;
-import java.util.Locale;
-import java.util.Set;
-import java.util.UUID;
-
-import javax.servlet.ServletContext;
-import javax.servlet.http.HttpServletRequest;
-import javax.ws.rs.Consumes;
-import javax.ws.rs.DELETE;
-import javax.ws.rs.DefaultValue;
-import javax.ws.rs.GET;
-import javax.ws.rs.POST;
-import javax.ws.rs.PUT;
-import javax.ws.rs.Path;
-import javax.ws.rs.PathParam;
-import javax.ws.rs.Produces;
-import javax.ws.rs.QueryParam;
-import javax.ws.rs.WebApplicationException;
-import javax.ws.rs.core.CacheControl;
-import javax.ws.rs.core.Context;
-import javax.ws.rs.core.MediaType;
-import javax.ws.rs.core.Response;
-import javax.ws.rs.core.Response.ResponseBuilder;
-import javax.ws.rs.core.UriBuilder;
-import javax.ws.rs.core.UriInfo;
-
+import com.sun.jersey.api.NotFoundException;
+import org.apache.commons.lang3.StringUtils;
 import org.apache.nifi.update.attributes.Action;
 import org.apache.nifi.update.attributes.Condition;
 import org.apache.nifi.update.attributes.Criteria;
+import org.apache.nifi.update.attributes.FlowFilePolicy;
 import org.apache.nifi.update.attributes.Rule;
 import org.apache.nifi.update.attributes.UpdateAttributeModelFactory;
 import org.apache.nifi.update.attributes.dto.DtoFactory;
 import org.apache.nifi.update.attributes.dto.RuleDTO;
 import org.apache.nifi.update.attributes.entity.ActionEntity;
 import org.apache.nifi.update.attributes.entity.ConditionEntity;
+import org.apache.nifi.update.attributes.entity.EvaluationContextEntity;
 import org.apache.nifi.update.attributes.entity.RuleEntity;
 import org.apache.nifi.update.attributes.entity.RulesEntity;
 import org.apache.nifi.update.attributes.serde.CriteriaSerDe;
-import org.apache.nifi.web.InvalidRevisionException;
-import org.apache.nifi.web.Revision;
-import org.apache.commons.lang3.StringUtils;
-
-import com.sun.jersey.api.NotFoundException;
-
-import org.apache.nifi.update.attributes.FlowFilePolicy;
-import org.apache.nifi.update.attributes.entity.EvaluationContextEntity;
 import org.apache.nifi.web.ComponentDetails;
 import org.apache.nifi.web.HttpServletConfigurationRequestContext;
 import org.apache.nifi.web.HttpServletRequestContext;
+import org.apache.nifi.web.InvalidRevisionException;
 import org.apache.nifi.web.NiFiWebConfigurationContext;
 import org.apache.nifi.web.NiFiWebConfigurationRequestContext;
 import org.apache.nifi.web.NiFiWebRequestContext;
+import org.apache.nifi.web.Revision;
 import org.apache.nifi.web.UiExtensionType;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.servlet.ServletContext;
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.PUT;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.CacheControl;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import javax.ws.rs.core.Response.ResponseBuilder;
+import javax.ws.rs.core.UriBuilder;
+import javax.ws.rs.core.UriInfo;
+import java.text.Collator;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.UUID;
--- End diff --

From what I can tell, the only changes to this file are to the formatting 
of the imports. In general extraneous modifications should be limited.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1274

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/284#discussion_r56507643
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/JwtService.java
 ---
@@ -70,8 +70,6 @@ public String getAuthenticationFromToken(final String 
base64EncodedToken) throws
 
 // TODO: Validate issuer against active registry?
--- End diff --

NVM, it's not something that was modified in this patch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1274

2016-03-20 Thread JPercivall

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/284#discussion_r56423709
  
--- Diff: 
nifi-nar-bundles/nifi-kerberos-iaa-providers-bundle/nifi-kerberos-iaa-providers-nar/src/main/resources/META-INF/NOTICE
 ---
@@ -0,0 +1,37 @@
+nifi-kerberos-iaa-providers-nar
+Copyright 2014-2016 The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+**
+Apache Software License v2
+**
+
+The following binary components are provided under the Apache Software 
License v2
+
+  (ASLv2) Spring Framework
+The following NOTICE information applies:
+  Spring Framework 4.1.6.RELEASE
+  Copyright (c) 2002-2015 Pivotal, Inc.
--- End diff --

Spring Framework is offered under an ASLv2 with no NOTICE 
(https://github.com/spring-projects/spring-framework). Therefore we should not 
include it in our notice.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1274

2016-03-20 Thread JPercivall

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/284#discussion_r56509790
  
--- Diff: 
nifi-nar-bundles/nifi-kerberos-iaa-providers-bundle/nifi-kerberos-iaa-providers/src/test/resources/nifi.properties
 ---
@@ -0,0 +1,185 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Core Properties #
+nifi.version=0.6.0-SNAPSHOT
+nifi.flow.configuration.file=./conf/flow.xml.gz
+nifi.flow.configuration.archive.dir=./conf/archive/
+nifi.flowcontroller.autoResumeState=true
+nifi.flowcontroller.graceful.shutdown.period=10 sec
+nifi.flowservice.writedelay.interval=500 ms
+nifi.administrative.yield.duration=30 sec
+# If a component has no work to do (is "bored"), how long should we wait 
before checking again for work?
+nifi.bored.yield.duration=10 millis
+
+nifi.authority.provider.configuration.file=./conf/authority-providers.xml

+nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
+nifi.templates.directory=./conf/templates
+nifi.ui.banner.text=
+nifi.ui.autorefresh.interval=30 sec
+nifi.nar.library.directory=./lib
+nifi.nar.working.directory=./work/nar/
+nifi.documentation.working.directory=./work/docs/components
+
+
+# State Management #
+
+nifi.state.management.configuration.file=./conf/state-management.xml
+# The ID of the local state provider
+nifi.state.management.provider.local=local-provider
+# The ID of the cluster-wide state provider. This will be ignored if NiFi 
is not clustered but must be populated if running in a cluster.
+nifi.state.management.provider.cluster=zk-provider
+# Specifies whether or not this instance of NiFi should run an embedded 
ZooKeeper server
+nifi.state.management.embedded.zookeeper.start=false
+# Properties file that provides the ZooKeeper properties to use if 
 is set to true

+nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
+
+
+# H2 Settings
+nifi.database.directory=./database_repository
+nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
+
+# FlowFile Repository

+nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
+nifi.flowfile.repository.directory=./flowfile_repository
+nifi.flowfile.repository.partitions=256
+nifi.flowfile.repository.checkpoint.interval=2 mins
+nifi.flowfile.repository.always.sync=false
+

+nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
+nifi.queue.swap.threshold=2
+nifi.swap.in.period=5 sec
+nifi.swap.in.threads=1
+nifi.swap.out.period=5 sec
+nifi.swap.out.threads=4
+
+# Content Repository

+nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
+nifi.content.claim.max.appendable.size=10 MB
+nifi.content.claim.max.flow.files=100
+nifi.content.repository.directory.default=./content_repository
+nifi.content.repository.archive.max.retention.period=12 hours
+nifi.content.repository.archive.max.usage.percentage=50%
+nifi.content.repository.archive.enabled=true
+nifi.content.repository.always.sync=false
+nifi.content.viewer.url=/nifi-content-viewer/
+
+# Provenance Repository Properties

+nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository
+
+# Persistent Provenance Repository Properties
+nifi.provenance.repository.directory.default=./provenance_repository
+nifi.provenance.repository.max.storage.time=24 hours
+nifi.provenance.repository.max.storage.size=1 GB
+nifi.provenance.repository.rollover.time=30 secs
+nifi.provenance.repository.rollover.size=100 MB
+nifi.provenance.repository.query.threads=2
+nifi.provenance.repository.index.threads=1
+nifi.provenance.repository.compress.on.rollover=true
+nifi.provenance.repository.always.sync=false
+nifi.provenance.repository

[GitHub] nifi pull request: NIFI-1664 Preferring System.nanoTime to System....

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/297#issuecomment-199870316
  
I am testing on Windows 8.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1664 Preferring System.nanoTime to System....

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/297#issuecomment-199888509
  
A very weird error with contrib check I am only getting windows 8, an 
unapproved license for:

nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/TestEncryptContent/unsalted_128_raw.enc

It seems this should be excluded in the pom for standard processors but it 
isn't. What is more confusing is that neither Mac nor Travis fails to find this 
contrib check. Even more so, it looks like salted_128_raw.enc should fail as 
well but it doesn't on any tested machine.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1664 Preferring System.nanoTime to System....

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/297#issuecomment-199936658
  
With the added pom exclusions the build passes


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1664 Preferring System.nanoTime to System....

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/297#issuecomment-199939742
  
+1

Reviewed the code and did a contrib check build on Windows 8, looks good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1666: Fixed bug with EL evaluation in PutE...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/298#issuecomment-199960791
  
+1

Reviewed the code, did a contrib check build and verified the functionality 
in a NiFi instance against ES 2.1.1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 initial commit of UpdateAttribute wit...

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi/pull/309

NIFI-1582 initial commit of UpdateAttribute with state



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi NIFI-1582

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #309


commit 3a9035d9bd8fb75fcf82fd7fe328ad9cecffb78b
Author: jpercivall 
Date:   2016-03-28T21:52:09Z

NIFI-1582 initial commit of UpdateAttribute with state




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1662 Initial commit of adding Decimal and ...

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi/pull/310

NIFI-1662 Initial commit of adding Decimal and Math functions to EL



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi NIFI-1662

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/310.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #310


commit e56c055ac275376dc481c437cd3640f88b3ab704
Author: jpercivall 
Date:   2016-03-28T19:48:42Z

NIFI-1662 Initial commit of adding Decimal and Math functions to EL




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1662 Initial commit of adding Decimal and ...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/310#issuecomment-202964957
  
The reason I broke out the Decimal functions for math functions is to not 
force users to use a specific return value. 
For the logical math functions (greater than, etc.) I couldn't figure out 
how to parse the incoming values if they were passed in as strings. Since 
almost all attributes will be passed in as strings I couldn't make an 
assumption on whether to interpret them as longs or doubles.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 initial commit of UpdateAttribute wit...

Github user JPercivall closed the pull request at:

https://github.com/apache/nifi/pull/309


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 initial commit of UpdateAttribute wit...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/309#issuecomment-202969012
  
Little preemptive on the opening of the PR, forgot to add documentation for 
the new feature.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1662 Initial commit of adding Decimal and ...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/310#issuecomment-202971560
  
Forgot to fix contrib check when I made a final change and an experimental 
change snuck in.

This is not the PR that you are looking for.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1662 Initial commit of adding Decimal and ...

Github user JPercivall closed the pull request at:

https://github.com/apache/nifi/pull/310


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MIFNI-2 Created a Util class to transfor...

2016-03-30 Thread JPercivall

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi-minifi/pull/1

MIFNI-2 Created a Util class to transform prospective config.yml intoâ¦

â¦ flow.xml and nifi.properties

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi-minifi MINIFI-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/1.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1


commit 9e102dfc096ea76b71ef4d79fdfd13a16123178d
Author: Joseph Percivall 
Date:   2016-03-30T18:10:18Z

MIFNI-2 Created a Util class to transform prospective config.yml into 
flow.xml and nifi.properties




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-10 init commit of minifi-assembly

2016-03-30 Thread JPercivall

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi-minifi/pull/3

MINIFI-10 init commit of minifi-assembly

I created a minifi-assembly that compiles and creates the initial LICENSE, 
NOTICE, bin and conf directories in a distribution. When deps get added to the 
bootstrap I believe it will need to be added to minifi-assembly.

I re-used much of the same scripts, pom.xml and dependencies.xml from NIFI 
(reconfigured for minifi) but I assume that since MiNiFi is a sub project of 
NiFi we don't need to provide a separate notice for work taken directly from 
it. I did add NiFi in the NOTICE for the assembly.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi-minifi MINIFI-10

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/3.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3


commit 3e49921f4703aa5a94d30746204228f3b76572ee
Author: Joseph Percivall 
Date:   2016-03-30T21:17:03Z

MINIFI-10 init commit of minifi-assembly




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-10 init commit of minifi-assembly

2016-03-31 Thread JPercivall

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/3#discussion_r58056139
  
--- Diff: minifi-assembly/LICENSE ---
@@ -0,0 +1,1152 @@
+
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other 
modifications
+  represent, as a whole, an original work of authorship. For the 
purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces 
of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright 
owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control 
systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  copyright license to reproduce, prepare Derivative Works of,
+  publicly display, publicly perform, sublicense, and distribute the
+  Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  (except as stated in this section) patent license to make, have made,
+  use, offer to sell, sell, import, and other

[GitHub] nifi-minifi pull request: MINIFI-9 initial commit for boostrapping...

2016-03-31 Thread JPercivall

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi-minifi/pull/4

MINIFI-9 initial commit for boostrapping/init process

An initial commit for the minifi-boostrap. This is a repurposed 
nifi-bootstrap. It properly brings in the correct libs to start up but of 
course will relies on a MiNiFi.java class to start.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi-minifi MINIFI-9

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/4.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4


commit d492288bf8a800093367fe635e878925c17b36c9
Author: Joseph Percivall 
Date:   2016-03-31T21:31:12Z

MINIFI-9 initial commit for boostrapping/init process




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

2016-04-01 Thread JPercivall

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi/pull/319

NIFI-1582 added state to UpdateAttribute as well as updated a few parâ¦

â¦ts that hadn't be touched in years (referenced the 
'FlowFileMetadataEnhancer' processor'. Also added a 'NUMBER_VALIDATOR' to 
StandardValidators

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi NIFI-1582

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/319.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #319


commit 50f300cefdb98b60fd22dec1fdba7694064add96
Author: jpercivall 
Date:   2016-04-01T21:27:42Z

NIFI-1582 added state to UpdateAttribute as well as updated a few parts 
that hadn't be touched in years (referenced the 'FlowFileMetadataEnhancer' 
processor'. Also added a 'NUMBER_VALIDATOR' to StandardValidators




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1662 Initial commit of adding Decimal and ...

2016-04-01 Thread JPercivall

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi/pull/320

NIFI-1662 Initial commit of adding Decimal and Math functions to Exprâ¦

â¦ession Language

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi NIFI-1662

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/320.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #320


commit f7b9343182046802988a5ecaf8540fe984995e06
Author: jpercivall 
Date:   2016-04-01T22:36:30Z

NIFI-1662 Initial commit of adding Decimal and Math functions to Expression 
Language




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

2016-04-05 Thread JPercivall

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/319#issuecomment-206094554
  
Not sure why it failed on something unrelated on Travis it builds locally. 
Also I don't believe there is an easy way to rebuild the PR in Travis.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/319#issuecomment-206383057
  
@mattyb149 just tried pushing with no new commits and nothing happened. I 
believe I could push an empty commit but I don't think it's worth cluttering 
this PR with unnecessary commits. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58712548
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/MiNiFi.java
 ---
@@ -0,0 +1,236 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.Thread.UncaughtExceptionHandler;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.nifi.NiFiServer;
+import org.apache.nifi.documentation.DocGenerator;
+import org.apache.nifi.nar.ExtensionManager;
+import org.apache.nifi.nar.ExtensionMapping;
+import org.apache.nifi.nar.NarClassLoaders;
+import org.apache.nifi.nar.NarUnpacker;
+import org.apache.nifi.util.FileUtils;
+import org.apache.nifi.util.NiFiProperties;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.bridge.SLF4JBridgeHandler;
+
+public class MiNiFi {
--- End diff --

All of the log statements and exception messages refer to "NiFi" instead of 
"MiNiFi".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58728826
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework-nar/pom.xml ---
@@ -0,0 +1,67 @@
+
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+minifi-framework-bundle
+org.apache.nifi.minifi
+0.0.1-SNAPSHOT
+
+4.0.0
+
+org.apache.nifi.minifi
+minifi-framework-nar
--- End diff --

This package is effectively not used. Running a "find" for 
"minifi-framework-nar" on the project only turns up two places, here and and in 
minifi-framework-bundle pom declaring minifi-framwork-nar as a module.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58732157
  
--- Diff: minifi-assembly/pom.xml ---
@@ -96,160 +85,250 @@ limitations under the License.
 slf4j-api
 compile
 
+
+
 
-ch.qos.logback
-logback-classic
-compile
+org.apache.nifi.minifi
+minifi-resources
+resources
+runtime
+zip
+0.0.1-SNAPSHOT
+
+
+org.apache.nifi.minifi
+minifi-bootstrap
+0.0.1-SNAPSHOT
+
+
+org.apache.nifi.minifi
+minifi-runtime
+0.0.1-SNAPSHOT
+
+
+
+
+org.apache.nifi
+nifi-processor-utils
 
-
 
+
+
+org.apache.nifi
+nifi-api
+
+
+org.apache.nifi
+nifi-runtime
+
+
+org.apache.nifi
+nifi-bootstrap
--- End diff --

Why is nifi-bootstrap needed? Isn't minifi-bootstrap replacing that? 

Also nifi-bootstrap is explicitly included in the bootstrap lib in 
minifi-assembly's dependencies.xml.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58743593
  
--- Diff: pom.xml ---
@@ -158,12 +159,182 @@ limitations under the License.
 ${org.slf4j.version}
 provided
 
+
+
+
+org.apache.nifi
+nifi-api
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-utils
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-site-to-site-client
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-web-utils
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-expression-language
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-custom-ui-utilities
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-ui-extension
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-flowfile-packager
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-socket-utils
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-data-provenance-utils
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-runtime
+${org.apache.nifi.version}
+
+
+org.apache.nifi
+nifi-bootstrap
--- End diff --

This dependency is still in the pom after the second commit.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58744060
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/MiNiFi.java
 ---
@@ -0,0 +1,236 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.Thread.UncaughtExceptionHandler;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.nifi.NiFiServer;
+import org.apache.nifi.documentation.DocGenerator;
+import org.apache.nifi.nar.ExtensionManager;
+import org.apache.nifi.nar.ExtensionMapping;
+import org.apache.nifi.nar.NarClassLoaders;
+import org.apache.nifi.nar.NarUnpacker;
+import org.apache.nifi.util.FileUtils;
+import org.apache.nifi.util.NiFiProperties;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.bridge.SLF4JBridgeHandler;
+
+public class MiNiFi {
--- End diff --

Forgot to point out that the BootstrapListener's log and variable names 
also refer to nifi instead of minifi. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58744217
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework-nar/pom.xml ---
@@ -0,0 +1,67 @@
+
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+minifi-framework-bundle
+org.apache.nifi.minifi
+0.0.1-SNAPSHOT
+
+4.0.0
+
+org.apache.nifi.minifi
+minifi-framework-nar
--- End diff --

That works for me


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/319#issuecomment-206472004
  
@mattyb149 I did what you suggested, did a soft reset, re-committed (making 
a new hash like @apiri pointed out) and did a force push. It's rebuilding now


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-13 created a provenance reporting...

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi-minifi/pull/6

MINIFI-13 created a provenance reporting task to send provenance infoâ¦

â¦rmation via S2S

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi-minifi MINIFI-13

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/6.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #6


commit 6695855449bd076ddbdd76676c435b2cf9ccf2c1
Author: Joseph Percivall 
Date:   2016-04-08T17:17:33Z

MINIFI-13 created a provenance reporting task to send provenance 
information via S2S




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1028 initial commit of NiFi In Depth docum...

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi/pull/339

NIFI-1028 initial commit of NiFi In Depth documentation



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi NIFI-1028

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/339.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #339


commit 29adea0f7d4fa198e843e0c488e27c2a985eec3a
Author: jpercivall 
Date:   2016-04-08T18:00:21Z

NIFI-1028 initial commit of NiFi In Depth documentation




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-2 MINIFI-4 Configuration change n...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59085011
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java
 ---
@@ -0,0 +1,30 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.configuration;
+
+/**
+ * Interface for handling events detected and driven by an associated 
{@link ConfigurationChangeNotifier} to which the listener
+ * has registered via {@link 
ConfigurationChangeNotifier#registerListener(ConfigurationChangeListener)}.
+ */
+public interface ConfigurationChangeListener {
+
+/**
+ * Provides a mechanism for the implementation to interpret the 
specified
+ */
+void handleChange();
--- End diff --

How will the ConfigurationChangeNotifier let the listeners know what the 
configuration changed to?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-2 MINIFI-4 Configuration change n...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59085169
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/configuration/FileChangeNotifier.java
 ---
@@ -0,0 +1,139 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.configuration;
+
+import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.WatchEvent;
+import java.nio.file.WatchKey;
+import java.nio.file.WatchService;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * FileChangeNotifier provides a simple FileSystem monitor for detecting 
changes for a specified file as generated from its corresponding {@link Path}.  
Upon modifications to the associated file,
+ * associated listeners receive notification of a change allowing 
configuration logic to be reanalyzed.  The backing implementation is associated 
with a {@link ScheduledExecutorService} that
+ * ensures continuity of monitoring.
+ */
+public class FileChangeNotifier implements ConfigurationChangeNotifier, 
Closeable {
+
+private final ExecutorService executorService;
+private final Path configFile;
+private final WatchService configFileWatcher;
+private final Set 
configurationChangeListeners = new HashSet<>();
+
+private static final int DEFAULT_POLLING_PERIOD_INTERVAL = 15;
+private static final TimeUnit DEFAULT_POLLING_PERIOD_UNIT = 
TimeUnit.SECONDS;
+
+/**
+ * @param configFile to monitor for changes
+ * @throws IOException if there are any issues with accessing the 
specified config file or generating the associated {@link WatchService}.
+ */
+public FileChangeNotifier(Path configFile) throws IOException {
+final File file = configFile.toFile();
+if (!file.exists() || !file.canRead() || !file.isFile()) {
+throw new IllegalArgumentException(String.format("The 
specified path %s must be a readable file.", configFile));
--- End diff --

Should it be a valid state to start with no config file and wait for it to 
be put there?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-2 MINIFI-4 Configuration change n...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59087047
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/test/java/org/apache/nifi/minifi/configuration/TestFileChangeNotifier.java
 ---
@@ -0,0 +1,83 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.configuration;
+
+import java.nio.file.Paths;
+
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+public class TestFileChangeNotifier {
--- End diff --

Could a unit be added to test the watcher notifying when a config file gets 
changed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-2 Created a Util class to transfo...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/1#discussion_r59093641
  
--- Diff: 
minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java
 ---
@@ -0,0 +1,331 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.util.Map;
+import java.util.zip.GZIPOutputStream;
+
+public final class ConfigTransformer {
+// Underlying version NIFI POC will be using
+public static final String NIFI_VERSION = "0.6.0";
+
+public static final String NAME_KEY = "name";
+public static final String COMMENT_KEY = "comment";
+public static final String ALWAYS_SYNC_KEY = "always sync";
+public static final String YIELD_PERIOD_KEY = "yield period";
+public static final String MAX_CONCURRENT_TASKS_KEY = "max concurrent 
tasks";
+public static final String ID_KEY = "id";
+
+public static final String FLOW_CONTROLLER_PROPS_KEY = "Flow 
Controller";
+
+public static final String CORE_PROPS_KEY = "Core Properties";
+public static final String FLOW_CONTROLLER_SHUTDOWN_PERIOD_KEY = "flow 
controller graceful shutdown period";
+public static final String FLOW_SERVICE_WRITE_DELAY_INTERVAL_KEY = 
"flow service write delay interval";
+public static final String ADMINISTRATIVE_YIELD_DURATION_KEY = 
"administrative yield duration";
+public static final String BORED_YIELD_DURATION_KEY = "bored yield 
duration";
+
+public static final String FLOWFILE_REPO_KEY = "FlowFile Repository";
+public static final String PARTITIONS_KEY = "partitions";
+public static final String CHECKPOINT_INTERVAL_KEY = "checkpoint 
interval";
+public static final String THRESHOLD_KEY = "queue swap threshold";
+public static final String SWAP_PROPS_KEY = "Swap";
+public static final String IN_PERIOD_KEY = "in period";
+public static final String IN_THREADS_KEY = "in threads";
+public static final String OUT_PERIOD_KEY = "out period";
+public static final String OUT_THREADS_KEY = "out threads";
+
+
+public static final String CONTENT_REPO_KEY = "Content Repository";
+public static final String CONTENT_CLAIM_MAX_APPENDABLE_SIZE_KEY = 
"content claim max appendable size";
+public static final String CONTENT_CLAIM_MAX_FLOW_FILES_KEY = "content 
claim max flow files";
+
+public static final String COMPONENT_STATUS_REPO_KEY = "Component 
Status Repository";
+public static final String BUFFER_SIZE_KEY = "buffer size";
+public static final String SNAPSHOT_FREQUENCY_KEY = "snapshot 
frequency";
+
+public static final String SECURITY_PROPS_KEY = "Security Properties";
+public static final String KEYSTORE_KEY = "keystore";
+public static final String KEYSTORE_TYPE_KEY = "keystore type";
+public static final String KEYSTORE_PASSWORD_KEY = "keystore password";
+public static final String KEY_PASSWORD_KEY = "key password";
+public static final String TRUSTSTORE_KEY = "truststore";
+public static final String TRUSTSTORE_TYPE_KEY = "truststore type";
+public static final String TRUSTSTORE_PASSWORD_KEY = "truststore 
password";
+public s

[GitHub] nifi-minifi pull request: MINIFI-2 MINIFI-4 Configuration change n...

2016-04-10 Thread JPercivall

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59144065
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java
 ---
@@ -0,0 +1,30 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.configuration;
+
+/**
+ * Interface for handling events detected and driven by an associated 
{@link ConfigurationChangeNotifier} to which the listener
+ * has registered via {@link 
ConfigurationChangeNotifier#registerListener(ConfigurationChangeListener)}.
+ */
+public interface ConfigurationChangeListener {
+
+/**
+ * Provides a mechanism for the implementation to interpret the 
specified
+ */
+void handleChange();
--- End diff --

That works for me


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-2 MINIFI-4 Configuration change n...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59425771
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
 ---
@@ -21,6 +21,11 @@
 public interface ConfigurationChangeNotifier {
 
 /**
+ * Begins the associated notification service provided by the given 
implementation.  In most implementations, no action will occur until this 
method is invoked.
+ */
+void initialize();
--- End diff --

This should pass a Properties object so that change notifiers can be 
configured. The user would add properties to the bootstrap.conf (in the same 
way NiFi has notification services configured there) and RunMiNiFi would pass 
the bootstrap properties to each Change Notifier. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-2 MINIFI-4 Configuration change n...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59429946
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
 ---
@@ -21,6 +21,11 @@
 public interface ConfigurationChangeNotifier {
 
 /**
+ * Begins the associated notification service provided by the given 
implementation.  In most implementations, no action will occur until this 
method is invoked.
+ */
+void initialize();
--- End diff --

That works for me


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-13 created a provenance reporting...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59438406
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.AbstractReportingTask;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart of NiFi the task knows where it left off.")
+public class ProvenanceReportingTask extends AbstractReportingTask {
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to post the Provenance Events to.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+static final PropertyDescriptor PORT_NAME = new 
PropertyDescriptor.Builder()

[GitHub] nifi-minifi pull request: MINIFI-13 created a provenance reporting...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59438870
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.AbstractReportingTask;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart of NiFi the task knows where it left off.")
+public class ProvenanceReportingTask extends AbstractReportingTask {
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to post the Provenance Events to.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+static final PropertyDescriptor PORT_NAME = new 
PropertyDescriptor.Builder()

[GitHub] nifi-minifi pull request: MINIFI-13 created a provenance reporting...

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59445052
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/test/java/org/apache/nifi/minifi/provenance/reporting/TestProvenanceReportingTask.java
 ---
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.provenance.ProvenanceEventBuilder;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.provenance.ProvenanceEventType;
+import org.apache.nifi.provenance.StandardProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.EventAccess;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.ReportingInitializationContext;
+import org.apache.nifi.state.MockStateManager;
+import org.apache.nifi.util.MockPropertyValue;
+import org.junit.Assert;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+public class TestProvenanceReportingTask {
+
+@Test
+public void testSerializedForm() throws IOException, 
InitializationException {
+final String uuid = "1000----";
+final Map attributes = new HashMap<>();
+attributes.put("abc", "xyz");
+attributes.put("xyz", "abc");
+attributes.put("filename", "file-" + uuid);
+
+final Map prevAttrs = new HashMap<>();
+attributes.put("filename", "1234.xyz");
+
+final Set lineageIdentifiers = new HashSet<>();
+lineageIdentifiers.add("123");
+lineageIdentifiers.add("321");
+
+final ProvenanceEventBuilder builder = new 
StandardProvenanceEventRecord.Builder();
+builder.setEventTime(System.currentTimeMillis());
+builder.setEventType(ProvenanceEventType.RECEIVE);
+builder.setTransitUri("nifi://unit-test");
+attributes.put("uuid", uuid);
+builder.fromFlowFile(createFlowFile(3L, 3000L, attributes));
+builder.setAttributes(prevAttrs, attributes);
+builder.setComponentId("1234");
+builder.setComponentType("dummy processor");
+builder.setLineageIdentifiers(lineageIdentifiers);
+final ProvenanceEventRecord event = builder.build();
+
+final List dataSent = new ArrayList<>();
+final ProvenanceReportingTask task = new ProvenanceReportingTask() 
{
+@SuppressWarnings("unchecked")
+@Override
+protected SiteToSiteClient getClient() {
+final SiteToSiteClient client = 
Mockito.mock(SiteToSiteClient.class);
+final Transaction transaction = 
Mockito.mock(Transaction.class);
+
+try {
+Mockito.doAnswer(new Answer() {
+@Over

[GitHub] nifi-minifi pull request: MINIFI-2 MINIFI-4 Configuration change n...