date:20160412

2016-04-12 Thread ASF subversion and git services (JIRA)

Repository: nifi
Updated Branches:
  refs/heads/0.x a42fa991e -> e152dce07


NIFI-1180 Adding more IT tests.

This closes #336.

Signed-off-by: Andy LoPresto 


Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/e152dce0
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/e152dce0
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/e152dce0

Branch: refs/heads/0.x
Commit: e152dce07329576dbd894df938149a4a610082c3
Parents: a42fa99
Author: ijokarumawak 
Authored: Fri Apr 8 17:20:27 2016 +0900
Committer: Andy LoPresto 
Committed: Tue Apr 12 17:45:23 2016 -0700

--
 .../nifi/processors/aws/s3/ITFetchS3Object.java |  8 ++-
 .../nifi/processors/aws/s3/ITPutS3Object.java   | 61 
 2 files changed, 68 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/nifi/blob/e152dce0/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
--
diff --git 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
index db115bb..b318924 100644
--- 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
+++ 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
@@ -55,6 +55,10 @@ public class ITFetchS3Object extends AbstractS3IT {
 runner.run(1);
 
 runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1);
+final List ffs = 
runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS);
+MockFlowFile ff = ffs.get(0);
+ff.assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM);
+
ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME));
 }
 
 @Test
@@ -75,7 +79,9 @@ public class ITFetchS3Object extends AbstractS3IT {
 
 runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1);
 final List ffs = 
runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS);
-ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, 
ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
+MockFlowFile ff = ffs.get(0);
+ff.assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, 
ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
+
ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME));
 }
 
 @Test

http://git-wip-us.apache.org/repos/asf/nifi/blob/e152dce0/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
--
diff --git 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
index 5d7797e..2db6c74 100644
--- 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
+++ 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
@@ -119,6 +119,67 @@ public class ITPutS3Object extends AbstractS3IT {
 }
 }
 
+private void testPutThenFetch(String sseAlgorithm) throws IOException {
+
+// Put
+TestRunner runner = TestRunners.newTestRunner(new PutS3Object());
+
+runner.setProperty(PutS3Object.CREDENTIALS_FILE, CREDENTIALS_FILE);
+runner.setProperty(PutS3Object.REGION, REGION);
+runner.setProperty(PutS3Object.BUCKET, BUCKET_NAME);
+if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){
+runner.setProperty(PutS3Object.SERVER_SIDE_ENCRYPTION, 
sseAlgorithm);
+}
+
+final Map attrs = new HashMap<>();
+attrs.put("filename",  "filename-on-s3.txt");
+runner.enqueue(getResourcePath(SAMPLE_FILE_RESOURCE_NAME), attrs);
+runner.run();
+
+runner.assertAllFlowFilesTransferred(PutS3Object.REL_SUCCESS, 1);
+List ffs = 
runner.getFlowFilesForRelationship(PutS3Object.REL_SUCCESS);
+if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){
+ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, 
ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
+} else {
+ffs.get(0).assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM);
+}
+
+// Fetch
+runner = T

[jira] [Commented] (NIFI-1180) Modify PutS3Object to enable encryption


[ 
https://issues.apache.org/jira/browse/NIFI-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238335#comment-15238335
 ] 

ASF subversion and git services commented on NIFI-1180:
---

Commit dfa27263d22c6348873a751dc5cb4e5ba52bf860 in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=dfa2726 ]

NIFI-1180 Adding more IT tests.

This closes #336.

Signed-off-by: Andy LoPresto 


> Modify PutS3Object to enable encryption
> ---
>
> Key: NIFI-1180
> URL: https://issues.apache.org/jira/browse/NIFI-1180
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Elli Schwarz
>Priority: Minor
>
> Amazon allows encryption to be enforced on objects that are put to S3. In the 
> PutS3Object class, the objectMetadata object contains a method 
> setSSEAlgorithm which can set encryption for the objects. A Nifi property 
> should be added to the PutS3Object class so a user can enable/disable 
> encryption from the processor itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1180) Modify PutS3Object to enable encryption


[ 
https://issues.apache.org/jira/browse/NIFI-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238336#comment-15238336
 ] 

ASF GitHub Bot commented on NIFI-1180:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/336


> Modify PutS3Object to enable encryption
> ---
>
> Key: NIFI-1180
> URL: https://issues.apache.org/jira/browse/NIFI-1180
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Elli Schwarz
>Priority: Minor
>
> Amazon allows encryption to be enforced on objects that are put to S3. In the 
> PutS3Object class, the objectMetadata object contains a method 
> setSSEAlgorithm which can set encryption for the objects. A Nifi property 
> should be added to the PutS3Object class so a user can enable/disable 
> encryption from the processor itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

nifi git commit: NIFI-1180 Adding more IT tests.

Repository: nifi
Updated Branches:
  refs/heads/master acfc01213 -> dfa27263d


NIFI-1180 Adding more IT tests.

This closes #336.

Signed-off-by: Andy LoPresto 


Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/dfa27263
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/dfa27263
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/dfa27263

Branch: refs/heads/master
Commit: dfa27263d22c6348873a751dc5cb4e5ba52bf860
Parents: acfc012
Author: ijokarumawak 
Authored: Fri Apr 8 17:20:27 2016 +0900
Committer: Andy LoPresto 
Committed: Tue Apr 12 17:44:29 2016 -0700

--
 .../nifi/processors/aws/s3/ITFetchS3Object.java |  8 ++-
 .../nifi/processors/aws/s3/ITPutS3Object.java   | 61 
 2 files changed, 68 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/nifi/blob/dfa27263/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
--
diff --git 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
index db115bb..b318924 100644
--- 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
+++ 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java
@@ -55,6 +55,10 @@ public class ITFetchS3Object extends AbstractS3IT {
 runner.run(1);
 
 runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1);
+final List ffs = 
runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS);
+MockFlowFile ff = ffs.get(0);
+ff.assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM);
+
ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME));
 }
 
 @Test
@@ -75,7 +79,9 @@ public class ITFetchS3Object extends AbstractS3IT {
 
 runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1);
 final List ffs = 
runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS);
-ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, 
ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
+MockFlowFile ff = ffs.get(0);
+ff.assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, 
ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
+
ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME));
 }
 
 @Test

http://git-wip-us.apache.org/repos/asf/nifi/blob/dfa27263/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
--
diff --git 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
index 5d7797e..2db6c74 100644
--- 
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
+++ 
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java
@@ -119,6 +119,67 @@ public class ITPutS3Object extends AbstractS3IT {
 }
 }
 
+private void testPutThenFetch(String sseAlgorithm) throws IOException {
+
+// Put
+TestRunner runner = TestRunners.newTestRunner(new PutS3Object());
+
+runner.setProperty(PutS3Object.CREDENTIALS_FILE, CREDENTIALS_FILE);
+runner.setProperty(PutS3Object.REGION, REGION);
+runner.setProperty(PutS3Object.BUCKET, BUCKET_NAME);
+if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){
+runner.setProperty(PutS3Object.SERVER_SIDE_ENCRYPTION, 
sseAlgorithm);
+}
+
+final Map attrs = new HashMap<>();
+attrs.put("filename",  "filename-on-s3.txt");
+runner.enqueue(getResourcePath(SAMPLE_FILE_RESOURCE_NAME), attrs);
+runner.run();
+
+runner.assertAllFlowFilesTransferred(PutS3Object.REL_SUCCESS, 1);
+List ffs = 
runner.getFlowFilesForRelationship(PutS3Object.REL_SUCCESS);
+if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){
+ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, 
ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
+} else {
+ffs.get(0).assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM);
+}
+
+// Fetch
+runn

[jira] [Created] (NIFI-1759) Update to latest Commons Compress library

2016-04-12 Thread Koji Kawamura (JIRA)

Koji Kawamura created NIFI-1759:
---

 Summary: Update to latest Commons Compress library
 Key: NIFI-1759
 URL: https://issues.apache.org/jira/browse/NIFI-1759
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework, Extensions
Reporter: Koji Kawamura
Priority: Minor


The latest Commons Compress ver 1.11 was released on Apr 6, 2016.

This issue is used for tracking the changes to update our usage of Commons 
Compress throughout NiFi to use this new library and to ensure tests are still 
good and perhaps add some more to verify a few of following cases:

{quote}
Version 1.11 brings performance improvements to the 7ZIP code and
read-only support for ZIP archives using BZIP2 compression. In addition
it contains numerous bugfixes and enhancements, see the full list below.

The Apache Commons Compress Library defines a Java API for working with
ar, cpio, tar, zip, 7z, arj, dump, gzip, pack200, bzip2, lzma, snappy,
Z, xz and deflate files.
{quote}
quote from: [Release Announce | 
http://mail-archives.apache.org/mod_mbox/www-announce/201604.mbox/%3C87bn5m6ayl.fsf%40v45346.1blu.de%3E]

[Release Note | 
https://commons.apache.org/proper/commons-compress/changes-report.html#a1.11] 

As of Apr 2016, following NiFi implementations uses Commons Compress 1.9 
directly:

* nifi-flowfile-packager
** org.apache.nifi.util.FlowFilePackagerV1
** org.apache.nifi.util.FlowFileUnpackagerV1
* nifi-framework-cluster
** org.apache.nifi.cluster.flow.impl.DataFlowDaoImpl
* nifi-hdfs-processors
** org.apache.nifi.processors.hadoop.TarUnpackerSequenceFileWriter
* nifi-standard-processors
** org.apache.nifi.processors.standard.CompressContent
** org.apache.nifi.processors.standard.MergeContent
** org.apache.nifi.processors.standard.UnpackContent




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

nifi git commit: Update getting-started.adoc

Repository: nifi
Updated Branches:
  refs/heads/0.x 5e0476c36 -> a42fa991e


Update getting-started.adoc

Corrected button, menu item and icon inconsistencies/errors.  Fixed bulleted 
list formatting error in "Working with Templates" section.

This closes #329.

Signed-off-by: Andy LoPresto 


Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/a42fa991
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/a42fa991
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/a42fa991

Branch: refs/heads/0.x
Commit: a42fa991e1d7788e8cebc111663f509654936758
Parents: 5e0476c
Author: Andrew Lim 
Authored: Tue Apr 5 16:34:21 2016 -0400
Committer: Andy LoPresto 
Committed: Tue Apr 12 17:09:14 2016 -0700

--
 .../src/main/asciidoc/getting-started.adoc  | 48 ++--
 1 file changed, 23 insertions(+), 25 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/nifi/blob/a42fa991/nifi-docs/src/main/asciidoc/getting-started.adoc
--
diff --git a/nifi-docs/src/main/asciidoc/getting-started.adoc 
b/nifi-docs/src/main/asciidoc/getting-started.adoc
index 82ba1d8..7c51888 100644
--- a/nifi-docs/src/main/asciidoc/getting-started.adoc
+++ b/nifi-docs/src/main/asciidoc/getting-started.adoc
@@ -132,7 +132,7 @@ image:nifi-toolbar-components.png["Toolbar Components"]
 === Adding a Processor
 
 We can now begin creating our dataflow by adding a Processor to our canvas. To 
do this, drag the Processor icon
-image:iconProcessor.png["Processor"] from the top-left of the screen into the 
middle of the canvas (the graph paper-like
+(image:iconProcessor.png["Processor"]) from the top-left of the screen into 
the middle of the canvas (the graph paper-like
 background) and drop it there. This will give us a dialog that allows us to 
choose which Processor we want to add:
 
 image:add-processor.png["Add Processor"]
@@ -160,16 +160,16 @@ for each type. Properties that are in bold are required 
properties. The Processo
 properties have been configured. The most important property to configure for 
GetFile is the directory from which
 to pick up files. If we set the directory name to `./data-in`, this will cause 
the Processor to start picking up
 any data in the `data-in` subdirectory of the NiFi Home directory. We can 
choose to configure several different
-Properties for this Processor. If unsure what a particular Property does, we 
can hover over the help icon (
+Properties for this Processor. If unsure what a particular Property does, we 
can hover over the Help icon (
 image:iconInfo.png["Help"]
 )
 next to the Property Name with the mouse in order to read a description of the 
property. Additionally, the
-tooltip that is displayed when hovering over the help icon will provide the 
default value for that property,
+tooltip that is displayed when hovering over the Help icon will provide the 
default value for that property,
 if one exists, information about whether or not the property supports the 
Expression Language (see the
 <> section below), and previously configured values for 
that property.
 
 In order for this property to be valid, create a directory named `data-in` in 
the NiFi home directory and then
-click the `OK` button to close the dialog.
+click the `Ok` button to close the dialog.
 
 
 === Connecting Processors
@@ -239,9 +239,8 @@ Processor, and then hold the Shift key while selecting the 
other Processor in or
 right-click and choose the `Start` menu item. As an alternative to using the 
context menu, we can select the Processors and
 then click the Start icon in the toolbar at the top of the screen.
 
-Once started, the icon in the top-left corner of the Processors will change 
from a stopped icon to a Running icon. We can then
-stop the Processors in the same manner that we started them but using the Stop 
icon in the toolbar or the Stop menu item
-as opposed to the Start button.
+Once started, the icon in the top-left corner of the Processors will change 
from a stopped icon to a running icon. We can then
+stop the Processors by using the Stop icon in the toolbar or the `Stop` menu 
item.
 
 Once a Processor has started, we are not able to configure it anymore. 
Instead, when we right-click on the Processor, we are
 given the option to view its current configuration. In order to configure a 
Processor, we must first stop the Processor and
@@ -482,7 +481,7 @@ process the data.
 In addition to having Processors that are able to extract particular pieces of 
information from FlowFile content into Attributes,
 it is also common for users to want to add their own user-defined Attributes 
to each FlowFile at a particular place in the flow.
 The UpdateAttribute Processor is designed specifically for this purpose. Users 
are able to add a new prope

nifi git commit: Update getting-started.adoc

Repository: nifi
Updated Branches:
  refs/heads/master bd1f0a82a -> acfc01213


Update getting-started.adoc

Corrected button, menu item and icon inconsistencies/errors.  Fixed bulleted 
list formatting error in "Working with Templates" section.

This closes #329.

Signed-off-by: Andy LoPresto 


Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/acfc0121
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/acfc0121
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/acfc0121

Branch: refs/heads/master
Commit: acfc01213cf81bb920d2917bc89dcb8f4cd14f63
Parents: bd1f0a8
Author: Andrew Lim 
Authored: Tue Apr 5 16:34:21 2016 -0400
Committer: Andy LoPresto 
Committed: Tue Apr 12 17:08:15 2016 -0700

--
 .../src/main/asciidoc/getting-started.adoc  | 48 ++--
 1 file changed, 23 insertions(+), 25 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/nifi/blob/acfc0121/nifi-docs/src/main/asciidoc/getting-started.adoc
--
diff --git a/nifi-docs/src/main/asciidoc/getting-started.adoc 
b/nifi-docs/src/main/asciidoc/getting-started.adoc
index 82ba1d8..7c51888 100644
--- a/nifi-docs/src/main/asciidoc/getting-started.adoc
+++ b/nifi-docs/src/main/asciidoc/getting-started.adoc
@@ -132,7 +132,7 @@ image:nifi-toolbar-components.png["Toolbar Components"]
 === Adding a Processor
 
 We can now begin creating our dataflow by adding a Processor to our canvas. To 
do this, drag the Processor icon
-image:iconProcessor.png["Processor"] from the top-left of the screen into the 
middle of the canvas (the graph paper-like
+(image:iconProcessor.png["Processor"]) from the top-left of the screen into 
the middle of the canvas (the graph paper-like
 background) and drop it there. This will give us a dialog that allows us to 
choose which Processor we want to add:
 
 image:add-processor.png["Add Processor"]
@@ -160,16 +160,16 @@ for each type. Properties that are in bold are required 
properties. The Processo
 properties have been configured. The most important property to configure for 
GetFile is the directory from which
 to pick up files. If we set the directory name to `./data-in`, this will cause 
the Processor to start picking up
 any data in the `data-in` subdirectory of the NiFi Home directory. We can 
choose to configure several different
-Properties for this Processor. If unsure what a particular Property does, we 
can hover over the help icon (
+Properties for this Processor. If unsure what a particular Property does, we 
can hover over the Help icon (
 image:iconInfo.png["Help"]
 )
 next to the Property Name with the mouse in order to read a description of the 
property. Additionally, the
-tooltip that is displayed when hovering over the help icon will provide the 
default value for that property,
+tooltip that is displayed when hovering over the Help icon will provide the 
default value for that property,
 if one exists, information about whether or not the property supports the 
Expression Language (see the
 <> section below), and previously configured values for 
that property.
 
 In order for this property to be valid, create a directory named `data-in` in 
the NiFi home directory and then
-click the `OK` button to close the dialog.
+click the `Ok` button to close the dialog.
 
 
 === Connecting Processors
@@ -239,9 +239,8 @@ Processor, and then hold the Shift key while selecting the 
other Processor in or
 right-click and choose the `Start` menu item. As an alternative to using the 
context menu, we can select the Processors and
 then click the Start icon in the toolbar at the top of the screen.
 
-Once started, the icon in the top-left corner of the Processors will change 
from a stopped icon to a Running icon. We can then
-stop the Processors in the same manner that we started them but using the Stop 
icon in the toolbar or the Stop menu item
-as opposed to the Start button.
+Once started, the icon in the top-left corner of the Processors will change 
from a stopped icon to a running icon. We can then
+stop the Processors by using the Stop icon in the toolbar or the `Stop` menu 
item.
 
 Once a Processor has started, we are not able to configure it anymore. 
Instead, when we right-click on the Processor, we are
 given the option to view its current configuration. In order to configure a 
Processor, we must first stop the Processor and
@@ -482,7 +481,7 @@ process the data.
 In addition to having Processors that are able to extract particular pieces of 
information from FlowFile content into Attributes,
 it is also common for users to want to add their own user-defined Attributes 
to each FlowFile at a particular place in the flow.
 The UpdateAttribute Processor is designed specifically for this purpose. Users 
are able to add a new

[jira] [Commented] (NIFI-1478) Audit SSLContextFactory and SSLSocketFactory usage throughout application


[ 
https://issues.apache.org/jira/browse/NIFI-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238270#comment-15238270
 ] 

ASF GitHub Bot commented on NIFI-1478:
--

GitHub user alopresto opened a pull request:

https://github.com/apache/nifi/pull/346

Nifi 1753 Removed legacy X.509 certificate implementation references

Various logic throughout the application referenced 
`javax.security.cert.X509Certificate` which is a deprecated class and exists 
only for legacy compatibility with older JSSE implementations. As of Java SE 6, 
new development should use `java.security.cert.X509Certificate`. Most 
references to the legacy classes were in similar logic blocks to retrieve the 
distinguished name (DN) from the client certificate chain presented during TLS 
mutual authentication. 

I refactored this logic into a common utility method to deduplicate and 
provided utility methods for converting legacy `X509Certificate`s and the 
abstract `java.security.cert.Certificate` returned by the replacement method 
(`javax.net.ssl.SSLSession#getPeerCertificateChain()` is succeeded by 
`javax.net.ssl.SSLSession#getPeerCertificates()`) to the correct version of 
`X509Certificate`. 

The module `nifi-security-utils` was added as a dependency to `nifi-utils` 
but contains only two utility classes with static helper methods and four 
enums. This change may be reverted/expanded as part of the larger-scale work on 
NIFI-1478, NIFI-1480, etc., but that is 1.0.0 refactor work, while this was a 
surgical fix for both 0.7.0 and 1.0.0. 

This will be rebased & squashed before merging. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alopresto/nifi NIFI-1753

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/346.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #346


commit e2d1c359bb94148679a057b5521b3fb17f6a936a
Author: Andy LoPresto 
Date:   2016-04-12T03:11:45Z

NIFI-1753 Added convenience methods for converting legacy X.509 
certificates and abstract certificates to correct X.509 format.
Added unit tests for certificate manipulation.
Replaced logic retrieving legacy X.509 certificates with new logic in 
NodeProtocolSenderImpl.
Added bcpkix (Bouncy Castle PKI implementation) dependency to 
nifi-standard-processors pom.

commit b9868ef297228ef7398a30fbddb915abef98eda6
Author: Andy LoPresto 
Date:   2016-04-12T03:36:38Z

NIFI-1753 Added convenience method for extracting DN from peer certificate 
chain in SSL socket (canonical implementation to reduce code duplication and 
references to legacy certificate implementations).
Refactored logic retrieving legacy X.509 certificates with reference to 
convenience method in NodeProtocolSenderImpl.
Replaced logic retrieving legacy X.509 certificates with reference to 
convenience method in SocketProtocolListener.
Cleaned up exception handling in SocketProtocolListener.
Replaced legacy X.509 certificate declarations with new declarations in 
HandleHttpRequest (needs manual test).

commit 64208973bf9046d447230c8f096dde09d47d01bd
Author: Andy LoPresto 
Date:   2016-04-12T03:46:33Z

NIFI-1753 Replaced legacy X.509 certificate declarations with new 
declarations in PostHTTP.

commit cd35f9b36d1e373b99490a6f6df735cd16175383
Author: Andy LoPresto 
Date:   2016-04-12T04:16:57Z

NIFI-1753 Replaced legacy X.509 certificate declarations with new 
declarations in SSLSocketChannel and EndpointConnectionPool.
Temporary work-around of duplicate certificate conversion util method 
because nifi-utils cannot depend on nifi-security-utils.

commit 965b766f2a0be9f703d10e231ca595464bc57f07
Author: Andy LoPresto 
Date:   2016-04-12T22:19:14Z

NIFI-1753 Removed temporary work-around of duplicate certificate conversion 
util method and added nifi-security-utils as dependency of nifi-utils.

commit 613fe0b62e56898ebe99fed50a25f3aa515d727a
Author: Andy LoPresto 
Date:   2016-04-12T22:20:22Z

NIFI-1753 Fixed nifi-utils pom.xml comment about additional dependencies.




> Audit SSLContextFactory and SSLSocketFactory usage throughout application
> -
>
> Key: NIFI-1478
> URL: https://issues.apache.org/jira/browse/NIFI-1478
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 0.5.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Critical
>  Labels: certificate, security, tls
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> The internal use of {{SSLSocketFactory}} and {{SSLCont

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237999#comment-15237999
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/6


> Create a Reporting Task to Send Provenance data
> ---
>
> Key: MINIFI-13
> URL: https://issues.apache.org/jira/browse/MINIFI-13
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Data Format, Data Transmission
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
> Fix For: 0.0.1
>
>
> With initial effort to re-use as much of NiFi as possible it is not possible 
> to easily create a ProvenanceReporter to add provenance events as attributes 
> to FlowFiles as it would require changing the ProvenancenReporter interface. 
> This will require utilizing a different extension point to transmit the 
> provenance data back to a core NiFi instance. 
> Probably the most efficient way to do this is to create a ReportingTask which 
> reports the provenance events using the S2S protocol. 
> In the future this will probably be retired as a reporting task as MiNiFi 
> grows to rely less on NiFi but this Reporting task could also be contributed 
> back to NiFi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

nifi-minifi git commit: MINIFI-13 created a provenance reporting task to send provenance information via S2S

2016-04-12 Thread jpercivall

Repository: nifi-minifi
Updated Branches:
  refs/heads/master 59f2d4418 -> 61e3a925d


MINIFI-13 created a provenance reporting task to send provenance information 
via S2S

This closes #6


Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/61e3a925
Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/61e3a925
Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/61e3a925

Branch: refs/heads/master
Commit: 61e3a925d3af52da04a2866a298bb88264b20c5b
Parents: 59f2d44
Author: Joseph Percivall 
Authored: Fri Apr 8 13:17:33 2016 -0400
Committer: Joseph Percivall 
Committed: Tue Apr 12 17:03:59 2016 -0400

--
 minifi-assembly/pom.xml |   6 +
 .../minifi-framework-bundle/pom.xml |   2 +-
 .../minifi-provenance-reporting-nar/pom.xml |  41 ++
 .../src/main/resources/META-INF/NOTICE  |  15 +
 .../minifi-provenance-reporting-task/pom.xml|  82 
 .../reporting/ProvenanceReportingTask.java  | 457 +++
 .../org.apache.nifi.reporting.ReportingTask |  16 +
 .../reporting/TestProvenanceReportingTask.java  | 186 
 .../minifi-provenance-reporting-bundle/pom.xml  |  41 ++
 minifi-nar-bundles/pom.xml  |   3 +-
 pom.xml |  13 +-
 11 files changed, 854 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-assembly/pom.xml
--
diff --git a/minifi-assembly/pom.xml b/minifi-assembly/pom.xml
index b076459..8dd223a 100644
--- a/minifi-assembly/pom.xml
+++ b/minifi-assembly/pom.xml
@@ -105,6 +105,12 @@ limitations under the License.
 minifi-runtime
 0.0.1-SNAPSHOT
 
+
+org.apache.nifi.minifi
+minifi-provenance-reporting-nar
+0.0.1-SNAPSHOT
+nar
+
 
 
 

http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-nar-bundles/minifi-framework-bundle/pom.xml
--
diff --git a/minifi-nar-bundles/minifi-framework-bundle/pom.xml 
b/minifi-nar-bundles/minifi-framework-bundle/pom.xml
index 7fbd6b7..dca3a5c 100644
--- a/minifi-nar-bundles/minifi-framework-bundle/pom.xml
+++ b/minifi-nar-bundles/minifi-framework-bundle/pom.xml
@@ -20,7 +20,7 @@ limitations under the License.
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
 
 minifi-nar-bundles
-org.apache.nifi
+org.apache.nifi.minifi
 0.0.1-SNAPSHOT
 
 4.0.0

http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml
--
diff --git 
a/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml
 
b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml
new file mode 100644
index 000..be74790
--- /dev/null
+++ 
b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml
@@ -0,0 +1,41 @@
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd";>
+
+4.0.0
+
+org.apache.nifi.minifi
+minifi-provenance-reporting-bundle
+0.0.1-SNAPSHOT
+
+
+minifi-provenance-reporting-nar
+nar
+
+true
+true
+
+
+
+org.apache.nifi.minifi
+minifi-provenance-reporting-task
+0.0.1-SNAPSHOT
+
+
+org.apache.nifi
+nifi-standard-services-api-nar
+nar
+
+
+

http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE
--
diff --git 
a/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE
 
b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE
new file mode 100644
index 000..be55e59
--- /dev/null
+++ 
b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,15 @@
+minifi-provenance-reporting-task-nar
+Copyright 2015-2016 The Apache Software Foundation
+
+This

[jira] [Created] (NIFI-1758) Unable to scroll in the Comments field when configuring process group

2016-04-12 Thread Andrew Lim (JIRA)

Andrew Lim created NIFI-1758:


 Summary: Unable to scroll in the Comments field when configuring 
process group
 Key: NIFI-1758
 URL: https://issues.apache.org/jira/browse/NIFI-1758
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Affects Versions: 0.6.0
Reporter: Andrew Lim
Priority: Minor


When configuring a process group, the Comments field allows a large amount of 
text to be entered or pasted in.  The size of the comments field displays about 
6-7 rows (depending on what web browser is being used).  However, scrolling is 
not allowed for that field, so the user is unable to easily see or edit the 
values in the field.

A workaround is to use the keyboard arrow keys.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MINIFI-2) File based implementation


[ 
https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237955#comment-15237955
 ] 

ASF GitHub Bot commented on MINIFI-2:
-

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/7


> File based implementation
> -
>
> Key: MINIFI-2
> URL: https://issues.apache.org/jira/browse/MINIFI-2
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Processing Configuration
>Reporter: Aldrin Piri
> Fix For: 0.0.1
>
> Attachments: ConfigFileExplanation.xml, Properties_Guide.md, 
> config.yml
>
>
> A defined schema for processing configuration will need to be defined.  XML 
> or YAML seem like possible options for achieving this.
> Agent configuration will come from two places, a bootstrap properties file 
> and a flow configuration file. The bootstrap will make use of some of the 
> original nifi.properties in order to start up the agent. The flow 
> configuration file will essentially be a stripped down version of a flow.xml 
> (with potentially S2S properties from nifi.properties).
> The format for the bootstrap.properties should likely be similar to 
> nifi.properties. 
> Further, a way to incorporate this to leverage the libraries from core NiFi 
> will aid in achieving a proof of concept implementation more expediently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

nifi-minifi git commit: MINIFI-2 MINIFI-4 Establishing a base implementation of the configuration change listener and notification services as well as a File implementation.

2016-04-12 Thread aldrin

Repository: nifi-minifi
Updated Branches:
  refs/heads/master 5751e23d4 -> 59f2d4418


MINIFI-2 MINIFI-4 Establishing a base implementation of the configuration 
change listener and notification services as well as a File implementation.


Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/59f2d441
Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/59f2d441
Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/59f2d441

Branch: refs/heads/master
Commit: 59f2d441867eb349104795cbe98d490385eb415a
Parents: 5751e23
Author: Aldrin Piri 
Authored: Fri Apr 8 14:59:58 2016 -0400
Committer: Aldrin Piri 
Committed: Tue Apr 12 16:49:37 2016 -0400

--
 .../ConfigurationChangeListener.java|  34 +++
 .../ConfigurationChangeNotifier.java|  57 +
 .../minifi-framework/minifi-runtime/pom.xml |  28 ++-
 .../configuration/FileChangeNotifier.java   | 173 
 .../configuration/TestFileChangeNotifier.java   | 206 +++
 .../src/test/resources/config.yml   |   0
 pom.xml |  41 
 7 files changed, 533 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/59f2d441/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java
--
diff --git 
a/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java
 
b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java
new file mode 100644
index 000..0d09f72
--- /dev/null
+++ 
b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java
@@ -0,0 +1,34 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.configuration;
+
+import java.io.InputStream;
+
+/**
+ * Interface for handling events detected and driven by an associated {@link 
ConfigurationChangeNotifier} to which the listener
+ * has registered via {@link 
ConfigurationChangeNotifier#registerListener(ConfigurationChangeListener)}.
+ */
+public interface ConfigurationChangeListener {
+
+/**
+ * Provides a mechanism for the implementation to interpret the specified 
configuration change
+ *
+ * @param is stream of the detected content received from the change 
notifier
+ */
+void handleChange(InputStream is);
+
+}

http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/59f2d441/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
--
diff --git 
a/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
 
b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
new file mode 100644
index 000..30a98c2
--- /dev/null
+++ 
b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
@@ -0,0 +1,57 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.configuration;
+
+import java.util.Properties;
+import java.util.Set;
+
+public interfa

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237933#comment-15237933
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59449408
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -251,6 +254,17 @@ public void onTrigger(final ReportingContext context) {
 if (state.containsKey(LAST_EVENT_ID_KEY)) {
 firstEventId = 
Long.parseLong(state.get(LAST_EVENT_ID_KEY)) + 1;
 }
+
+if(currMaxId < firstEventId){
+getLogger().debug("Current provenance max id is {} which 
is less than what was stored in state as the last queried event, which was {}. 
This means the provenance restarted it's " +
--- End diff --

super minor: it's -> its


> Create a Reporting Task to Send Provenance data
> ---
>
> Key: MINIFI-13
> URL: https://issues.apache.org/jira/browse/MINIFI-13
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Data Format, Data Transmission
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
> Fix For: 0.0.1
>
>
> With initial effort to re-use as much of NiFi as possible it is not possible 
> to easily create a ProvenanceReporter to add provenance events as attributes 
> to FlowFiles as it would require changing the ProvenancenReporter interface. 
> This will require utilizing a different extension point to transmit the 
> provenance data back to a core NiFi instance. 
> Probably the most efficient way to do this is to create a ReportingTask which 
> reports the provenance events using the S2S protocol. 
> In the future this will probably be retired as a reporting task as MiNiFi 
> grows to rely less on NiFi but this Reporting task could also be contributed 
> back to NiFi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MINIFI-2) File based implementation


[ 
https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237924#comment-15237924
 ] 

ASF GitHub Bot commented on MINIFI-2:
-

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59448506
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/configuration/FileChangeNotifier.java
 ---
@@ -42,33 +43,18 @@
  */
 public class FileChangeNotifier implements Runnable, 
ConfigurationChangeNotifier, Closeable {
 
+private Path configFile;
+private WatchService watchService;
+private long pollingSeconds;
+
 private final ScheduledExecutorService executorService = 
Executors.newSingleThreadScheduledExecutor();
-private final Path configFile;
-private final WatchService watchService;
 private final Set 
configurationChangeListeners = new HashSet<>();
 
-private static final int DEFAULT_POLLING_PERIOD_INTERVAL = 15;
-private static final TimeUnit DEFAULT_POLLING_PERIOD_UNIT = 
TimeUnit.SECONDS;
-
-/**
- * @param configFile to monitor for changes
- * @throws IOException if there are any issues with accessing the 
specified config file or generating the associated {@link WatchService}.
- */
-public FileChangeNotifier(Path configFile) throws IOException {
-this(configFile, initializeWatcher(configFile));
-}
-
-
-public FileChangeNotifier(Path configFile, WatchService watchService) {
-final File file = configFile.toFile();
-if (!file.exists() || !file.canRead() || !file.isFile()) {
-throw new IllegalArgumentException(String.format("The 
specified path %s must be a readable file.", configFile));
-}
-
-this.configFile = configFile;
-this.watchService = watchService;
-}
+protected static final String CONFIG_FILE_PATH_KEY = 
"nifi.notifier.file.config.path";
+protected static final String POLLING_PERIOD_INTERVAL_KEY = 
"nifi.notifier.file.polling.period.seconds";
--- End diff --

Should include "minifi" either by replacing "nifi" with it or by adding it 
subsequently. 


> File based implementation
> -
>
> Key: MINIFI-2
> URL: https://issues.apache.org/jira/browse/MINIFI-2
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Processing Configuration
>Reporter: Aldrin Piri
> Fix For: 0.0.1
>
> Attachments: ConfigFileExplanation.xml, Properties_Guide.md, 
> config.yml
>
>
> A defined schema for processing configuration will need to be defined.  XML 
> or YAML seem like possible options for achieving this.
> Agent configuration will come from two places, a bootstrap properties file 
> and a flow configuration file. The bootstrap will make use of some of the 
> original nifi.properties in order to start up the agent. The flow 
> configuration file will essentially be a stripped down version of a flow.xml 
> (with potentially S2S properties from nifi.properties).
> The format for the bootstrap.properties should likely be similar to 
> nifi.properties. 
> Further, a way to incorporate this to leverage the libraries from core NiFi 
> will aid in achieving a proof of concept implementation more expediently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1323) UI visual design enhancement


[ 
https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237908#comment-15237908
 ] 

Rob Moran commented on NIFI-1323:
-

Updated these two to show examples of components that a user may be 
unauthorized to access.
*nifi-sample-flow*
*nifi-component-samples*

> UI visual design enhancement
> 
>
> Key: NIFI-1323
> URL: https://issues.apache.org/jira/browse/NIFI-1323
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Priority: Trivial
> Attachments: flowfont.zip, nifi-add-processor-dialog.png, 
> nifi-component-samples.png, 
> nifi-configure-processor-properties-set-value.png, 
> nifi-configure-processor-properties.png, 
> nifi-configure-processor-scheduling.png, 
> nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, 
> nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, 
> nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, 
> nifi-sample-flow.png, nifi-shell-samp...@800px.png
>
>
> (I will attach mockups and supporting files as they become available)
> I am starting to work on a design to modernize the look and feel of the NiFi 
> UI. The initial focus of the design is to freshen the UI (flat design, SVG 
> icons, etc.). Additionally, the new design will propose usability 
> improvements such as exposing more flow-related actions into collapsible 
> panes, improving hierarchy of information, etc.
> Going forward, the design plan is to help lay the foundation for other UI/UX 
> related issues such as those documented in NIFI-951.
> ---
> *flowfont.zip*
> Contains icon font and supporting files
> *nifi-add-processor-dialog.png*
> Dialog sample. This sample shows the 'Add Processor' dialog.
> *nifi-component-samples.png*
> To show styling for all components, as well as those components when a user 
> is unauthorized to access.
> *nifi-configure-processor-properties*
> *nifi-configure-processor-properties-set-value*
> *nifi-configure-processor-scheduling*
> *nifi-configure-processor-settings*
> Configure Processor dialog. See related Comments below (in Activity section).
> *nifi-dialog-sample-@800px*
> Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab 
> of a provenance event.
> *nifi-drop.svg*
> NiFi logo without 'nifi
> *nifi-global-menu*
> To show global menu
> *nifi-interaction-and-menu-samples.png*
> To demonstrate user interactions - hover states, tooltips, menus, etc.
> *nifi-lineage-graph-@800px*
> To show lineage graph with explicit action to get back to data provenance 
> event list.
> *nifi-sample-flow-@800px*
> Shows a very useable UI down to around 800px in width. The thinking here is 
> that at anything lower than this, the NiFi user experience will change to 
> more of a monitoring and/or administrative type workflow. Future mockups will 
> be created to illustrate this.
> *nifi-logo.svg*
> NiFi logo complete
> *nifi-sample-flow.png*
> Mockup of sample flow. Updated to show revised tool and status bars. 
> Management related actions will move to a menu via mouseover (see 
> _nifi-global-menu_). Added benefits here include reducing clutter and more 
> user-friendly menu with text labels to reduce time spent scanning only a 
> large set of icons. This also helps gain valuable viewport width in a browser 
> (see _nifi-sample-flow-@800px_)
> *nifi-shell-sample-@800px*
> Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1323) UI visual design enhancement

[
https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rob Moran updated NIFI-1323:

Description:
(I will attach mockups and supporting files as they become available)

I am starting to work on a design to modernize the look and feel of the NiFi
UI. The initial focus of the design is to freshen the UI (flat design, SVG
icons, etc.). Additionally, the new design will propose usability improvements
such as exposing more flow-related actions into collapsible panes, improving
hierarchy of information, etc.

Going forward, the design plan is to help lay the foundation for other UI/UX
related issues such as those documented in NIFI-951.

---
*flowfont.zip*
Contains icon font and supporting files

*nifi-add-processor-dialog.png*
Dialog sample. This sample shows the 'Add Processor' dialog.

*nifi-component-samples.png*
To show styling for all components, as well as those components when a user is
unauthorized to access.

*nifi-configure-processor-properties*
*nifi-configure-processor-properties-set-value*
*nifi-configure-processor-scheduling*
*nifi-configure-processor-settings*
Configure Processor dialog. See related Comments below (in Activity section).

*nifi-dialog-sample-@800px*
Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab
of a provenance event.

*nifi-drop.svg*
NiFi logo without 'nifi

*nifi-global-menu*
To show global menu

*nifi-interaction-and-menu-samples.png*
To demonstrate user interactions - hover states, tooltips, menus, etc.

*nifi-lineage-graph-@800px*
To show lineage graph with explicit action to get back to data provenance event
list.

*nifi-sample-flow-@800px*
Shows a very useable UI down to around 800px in width. The thinking here is
that at anything lower than this, the NiFi user experience will change to more
of a monitoring and/or administrative type workflow. Future mockups will be
created to illustrate this.

*nifi-logo.svg*
NiFi logo complete

*nifi-sample-flow.png*
Mockup of sample flow. Updated to show revised tool and status bars. Management
related actions will move to a menu via mouseover (see _nifi-global-menu_).
Added benefits here include reducing clutter and more user-friendly menu with
text labels to reduce time spent scanning only a large set of icons. This also
helps gain valuable viewport width in a browser (see _nifi-sample-flow-@800px_)

*nifi-shell-sample-@800px*
Shell sample in 800px wide viewport. This sample shows the 'Data Provenance'
table.

was:
(I will attach mockups and supporting files as they become available)

Going forward, the design plan is to help lay the foundation for other UI/UX
related issues such as those documented in NIFI-951.

---
*flowfont.zip*
Contains icon font and supporting files

*nifi-add-processor-dialog.png*
Dialog sample. This sample shows the 'Add Processor' dialog.

*nifi-component-samples.png*
To show styling for all components

*nifi-dialog-sample-@800px*
Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab
of a provenance event.

*nifi-drop.svg*
NiFi logo without 'nifi

*nifi-global-menu*
To show global menu

*nifi-interaction-and-menu-samples.png*
To demonstrate user interactions - hover states, tooltips, menus, etc.

*nifi-lineage-graph-@800px*
To show lineage graph with explicit action to get back to data provenance event
list.

*nifi-logo.svg*
NiFi logo complete

*nifi-shell-sample-@800px*
Shell sample in 800px wide viewport. This sample shows the 'Data Provenance'
table.

> UI visual design enhancement
>
>
> Key: NIFI-1323
>

[jira] [Updated] (NIFI-1323) UI visual design enhancement


 [ 
https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rob Moran updated NIFI-1323:

Attachment: (was: nifi-component-samples.png)

> UI visual design enhancement
> 
>
> Key: NIFI-1323
> URL: https://issues.apache.org/jira/browse/NIFI-1323
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Priority: Trivial
> Attachments: flowfont.zip, nifi-add-processor-dialog.png, 
> nifi-component-samples.png, 
> nifi-configure-processor-properties-set-value.png, 
> nifi-configure-processor-properties.png, 
> nifi-configure-processor-scheduling.png, 
> nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, 
> nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, 
> nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, 
> nifi-sample-flow.png, nifi-shell-samp...@800px.png
>
>
> (I will attach mockups and supporting files as they become available)
> I am starting to work on a design to modernize the look and feel of the NiFi 
> UI. The initial focus of the design is to freshen the UI (flat design, SVG 
> icons, etc.). Additionally, the new design will propose usability 
> improvements such as exposing more flow-related actions into collapsible 
> panes, improving hierarchy of information, etc.
> Going forward, the design plan is to help lay the foundation for other UI/UX 
> related issues such as those documented in NIFI-951.
> ---
> *flowfont.zip*
> Contains icon font and supporting files
> *nifi-add-processor-dialog.png*
> Dialog sample. This sample shows the 'Add Processor' dialog.
> *nifi-component-samples.png*
> To show styling for all components
> *nifi-configure-processor-properties*
> *nifi-configure-processor-properties-set-value*
> *nifi-configure-processor-scheduling*
> *nifi-configure-processor-settings*
> Configure Processor dialog. See related Comments below (in Activity section).
> *nifi-dialog-sample-@800px*
> Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab 
> of a provenance event.
> *nifi-drop.svg*
> NiFi logo without 'nifi
> *nifi-global-menu*
> To show global menu
> *nifi-interaction-and-menu-samples.png*
> To demonstrate user interactions - hover states, tooltips, menus, etc.
> *nifi-lineage-graph-@800px*
> To show lineage graph with explicit action to get back to data provenance 
> event list.
> *nifi-sample-flow-@800px*
> Shows a very useable UI down to around 800px in width. The thinking here is 
> that at anything lower than this, the NiFi user experience will change to 
> more of a monitoring and/or administrative type workflow. Future mockups will 
> be created to illustrate this.
> *nifi-logo.svg*
> NiFi logo complete
> *nifi-sample-flow.png*
> Mockup of sample flow. Updated to show revised tool and status bars. 
> Management related actions will move to a menu via mouseover (see 
> _nifi-global-menu_). Added benefits here include reducing clutter and more 
> user-friendly menu with text labels to reduce time spent scanning only a 
> large set of icons. This also helps gain valuable viewport width in a browser 
> (see _nifi-sample-flow-@800px_)
> *nifi-shell-sample-@800px*
> Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1323) UI visual design enhancement


 [ 
https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rob Moran updated NIFI-1323:

Attachment: nifi-sample-flow.png
nifi-component-samples.png

> UI visual design enhancement
> 
>
> Key: NIFI-1323
> URL: https://issues.apache.org/jira/browse/NIFI-1323
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Priority: Trivial
> Attachments: flowfont.zip, nifi-add-processor-dialog.png, 
> nifi-component-samples.png, 
> nifi-configure-processor-properties-set-value.png, 
> nifi-configure-processor-properties.png, 
> nifi-configure-processor-scheduling.png, 
> nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, 
> nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, 
> nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, 
> nifi-sample-flow.png, nifi-shell-samp...@800px.png
>
>
> (I will attach mockups and supporting files as they become available)
> I am starting to work on a design to modernize the look and feel of the NiFi 
> UI. The initial focus of the design is to freshen the UI (flat design, SVG 
> icons, etc.). Additionally, the new design will propose usability 
> improvements such as exposing more flow-related actions into collapsible 
> panes, improving hierarchy of information, etc.
> Going forward, the design plan is to help lay the foundation for other UI/UX 
> related issues such as those documented in NIFI-951.
> ---
> *flowfont.zip*
> Contains icon font and supporting files
> *nifi-add-processor-dialog.png*
> Dialog sample. This sample shows the 'Add Processor' dialog.
> *nifi-component-samples.png*
> To show styling for all components
> *nifi-configure-processor-properties*
> *nifi-configure-processor-properties-set-value*
> *nifi-configure-processor-scheduling*
> *nifi-configure-processor-settings*
> Configure Processor dialog. See related Comments below (in Activity section).
> *nifi-dialog-sample-@800px*
> Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab 
> of a provenance event.
> *nifi-drop.svg*
> NiFi logo without 'nifi
> *nifi-global-menu*
> To show global menu
> *nifi-interaction-and-menu-samples.png*
> To demonstrate user interactions - hover states, tooltips, menus, etc.
> *nifi-lineage-graph-@800px*
> To show lineage graph with explicit action to get back to data provenance 
> event list.
> *nifi-sample-flow-@800px*
> Shows a very useable UI down to around 800px in width. The thinking here is 
> that at anything lower than this, the NiFi user experience will change to 
> more of a monitoring and/or administrative type workflow. Future mockups will 
> be created to illustrate this.
> *nifi-logo.svg*
> NiFi logo complete
> *nifi-sample-flow.png*
> Mockup of sample flow. Updated to show revised tool and status bars. 
> Management related actions will move to a menu via mouseover (see 
> _nifi-global-menu_). Added benefits here include reducing clutter and more 
> user-friendly menu with text labels to reduce time spent scanning only a 
> large set of icons. This also helps gain valuable viewport width in a browser 
> (see _nifi-sample-flow-@800px_)
> *nifi-shell-sample-@800px*
> Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1323) UI visual design enhancement


 [ 
https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rob Moran updated NIFI-1323:

Attachment: (was: nifi-sample-flow.png)

> UI visual design enhancement
> 
>
> Key: NIFI-1323
> URL: https://issues.apache.org/jira/browse/NIFI-1323
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Priority: Trivial
> Attachments: flowfont.zip, nifi-add-processor-dialog.png, 
> nifi-component-samples.png, 
> nifi-configure-processor-properties-set-value.png, 
> nifi-configure-processor-properties.png, 
> nifi-configure-processor-scheduling.png, 
> nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, 
> nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, 
> nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, 
> nifi-sample-flow.png, nifi-shell-samp...@800px.png
>
>
> (I will attach mockups and supporting files as they become available)
> I am starting to work on a design to modernize the look and feel of the NiFi 
> UI. The initial focus of the design is to freshen the UI (flat design, SVG 
> icons, etc.). Additionally, the new design will propose usability 
> improvements such as exposing more flow-related actions into collapsible 
> panes, improving hierarchy of information, etc.
> Going forward, the design plan is to help lay the foundation for other UI/UX 
> related issues such as those documented in NIFI-951.
> ---
> *flowfont.zip*
> Contains icon font and supporting files
> *nifi-add-processor-dialog.png*
> Dialog sample. This sample shows the 'Add Processor' dialog.
> *nifi-component-samples.png*
> To show styling for all components
> *nifi-configure-processor-properties*
> *nifi-configure-processor-properties-set-value*
> *nifi-configure-processor-scheduling*
> *nifi-configure-processor-settings*
> Configure Processor dialog. See related Comments below (in Activity section).
> *nifi-dialog-sample-@800px*
> Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab 
> of a provenance event.
> *nifi-drop.svg*
> NiFi logo without 'nifi
> *nifi-global-menu*
> To show global menu
> *nifi-interaction-and-menu-samples.png*
> To demonstrate user interactions - hover states, tooltips, menus, etc.
> *nifi-lineage-graph-@800px*
> To show lineage graph with explicit action to get back to data provenance 
> event list.
> *nifi-sample-flow-@800px*
> Shows a very useable UI down to around 800px in width. The thinking here is 
> that at anything lower than this, the NiFi user experience will change to 
> more of a monitoring and/or administrative type workflow. Future mockups will 
> be created to illustrate this.
> *nifi-logo.svg*
> NiFi logo complete
> *nifi-sample-flow.png*
> Mockup of sample flow. Updated to show revised tool and status bars. 
> Management related actions will move to a menu via mouseover (see 
> _nifi-global-menu_). Added benefits here include reducing clutter and more 
> user-friendly menu with text labels to reduce time spent scanning only a 
> large set of icons. This also helps gain valuable viewport width in a browser 
> (see _nifi-sample-flow-@800px_)
> *nifi-shell-sample-@800px*
> Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (MINIFI-14) Add Listener/Notifier Logic to RunMiNiFi

2016-04-12 Thread Aldrin Piri (JIRA)


 [ 
https://issues.apache.org/jira/browse/MINIFI-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aldrin Piri reassigned MINIFI-14:
-

Assignee: Aldrin Piri

> Add Listener/Notifier Logic to RunMiNiFi
> 
>
> Key: MINIFI-14
> URL: https://issues.apache.org/jira/browse/MINIFI-14
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Processing Configuration
>Reporter: Joseph Percivall
>Assignee: Aldrin Piri
> Fix For: 0.0.1
>
>
> The interface for the ConfigurationChangeNotifier and 
> ConfigurationChangeListener is done as part of MINIFI-4 but configuration and 
> implementation in RunMiNiFi needs to be added. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237890#comment-15237890
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59445052
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/test/java/org/apache/nifi/minifi/provenance/reporting/TestProvenanceReportingTask.java
 ---
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.provenance.ProvenanceEventBuilder;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.provenance.ProvenanceEventType;
+import org.apache.nifi.provenance.StandardProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.EventAccess;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.ReportingInitializationContext;
+import org.apache.nifi.state.MockStateManager;
+import org.apache.nifi.util.MockPropertyValue;
+import org.junit.Assert;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+public class TestProvenanceReportingTask {
+
+@Test
+public void testSerializedForm() throws IOException, 
InitializationException {
+final String uuid = "1000----";
+final Map attributes = new HashMap<>();
+attributes.put("abc", "xyz");
+attributes.put("xyz", "abc");
+attributes.put("filename", "file-" + uuid);
+
+final Map prevAttrs = new HashMap<>();
+attributes.put("filename", "1234.xyz");
+
+final Set lineageIdentifiers = new HashSet<>();
+lineageIdentifiers.add("123");
+lineageIdentifiers.add("321");
+
+final ProvenanceEventBuilder builder = new 
StandardProvenanceEventRecord.Builder();
+builder.setEventTime(System.currentTimeMillis());
+builder.setEventType(ProvenanceEventType.RECEIVE);
+builder.setTransitUri("nifi://unit-test");
+attributes.put("uuid", uuid);
+builder.fromFlowFile(createFlowFile(3L, 3000L, attributes));
+builder.setAttributes(prevAttrs, attributes);
+builder.setComponentId("1234");
+builder.setComponentType("dummy processor");
+builder.setLineageIdentifiers(lineageIdentifiers);
+final ProvenanceEventRecord event = builder.build();
+
+final List dataSent = new ArrayList<>();
+final ProvenanceReportingTask task = new ProvenanceReportingTask() 
{
+@SuppressWarnings("unchecked")
+@Override
+protected SiteToSiteClient getClient() {
+final SiteToSiteClient client = 
Mockito.mock(SiteToSiteClient.class);
+final Transaction transaction = 
Mockito.mock(Transaction.class);
+
+try {
+Mockito.doAnswer(new

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237823#comment-15237823
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59438870
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.AbstractReportingTask;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart of NiFi the task knows where it left off.")
+public class ProvenanceReportingTask extends AbstractReportingTask {
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to post the Provenance Events to.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+static

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237818#comment-15237818
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59438406
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.AbstractReportingTask;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart of NiFi the task knows where it left off.")
+public class ProvenanceReportingTask extends AbstractReportingTask {
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to post the Provenance Events to.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+static

[jira] [Commented] (MINIFI-2) File based implementation


[ 
https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237735#comment-15237735
 ] 

ASF GitHub Bot commented on MINIFI-2:
-

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59429946
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
 ---
@@ -21,6 +21,11 @@
 public interface ConfigurationChangeNotifier {
 
 /**
+ * Begins the associated notification service provided by the given 
implementation.  In most implementations, no action will occur until this 
method is invoked.
+ */
+void initialize();
--- End diff --

That works for me


> File based implementation
> -
>
> Key: MINIFI-2
> URL: https://issues.apache.org/jira/browse/MINIFI-2
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Processing Configuration
>Reporter: Aldrin Piri
> Fix For: 0.0.1
>
> Attachments: ConfigFileExplanation.xml, Properties_Guide.md, 
> config.yml
>
>
> A defined schema for processing configuration will need to be defined.  XML 
> or YAML seem like possible options for achieving this.
> Agent configuration will come from two places, a bootstrap properties file 
> and a flow configuration file. The bootstrap will make use of some of the 
> original nifi.properties in order to start up the agent. The flow 
> configuration file will essentially be a stripped down version of a flow.xml 
> (with potentially S2S properties from nifi.properties).
> The format for the bootstrap.properties should likely be similar to 
> nifi.properties. 
> Further, a way to incorporate this to leverage the libraries from core NiFi 
> will aid in achieving a proof of concept implementation more expediently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-1757) If Bulletin Board is opened, the requests keep occurring in the background even after the Bulletin Board is closed

2016-04-12 Thread Mark Payne (JIRA)

Mark Payne created NIFI-1757:


 Summary: If Bulletin Board is opened, the requests keep occurring 
in the background even after the Bulletin Board is closed
 Key: NIFI-1757
 URL: https://issues.apache.org/jira/browse/NIFI-1757
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Reporter: Mark Payne


I clicked to open the Bulletin Board and then closed it. Opening Chrome's 
Developer Tools, I can see that the requests are still made to retrieve 
bulletins every few seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MINIFI-2) File based implementation


[ 
https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237689#comment-15237689
 ] 

ASF GitHub Bot commented on MINIFI-2:
-

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59427453
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
 ---
@@ -21,6 +21,11 @@
 public interface ConfigurationChangeNotifier {
 
 /**
+ * Begins the associated notification service provided by the given 
implementation.  In most implementations, no action will occur until this 
method is invoked.
+ */
+void initialize();
--- End diff --

Actually, given that this is a separate task, how would you feel about 
initialize in its current context being a #start method and then #initialize 
providing the properties as discussed? 


> File based implementation
> -
>
> Key: MINIFI-2
> URL: https://issues.apache.org/jira/browse/MINIFI-2
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Processing Configuration
>Reporter: Aldrin Piri
> Fix For: 0.0.1
>
> Attachments: ConfigFileExplanation.xml, Properties_Guide.md, 
> config.yml
>
>
> A defined schema for processing configuration will need to be defined.  XML 
> or YAML seem like possible options for achieving this.
> Agent configuration will come from two places, a bootstrap properties file 
> and a flow configuration file. The bootstrap will make use of some of the 
> original nifi.properties in order to start up the agent. The flow 
> configuration file will essentially be a stripped down version of a flow.xml 
> (with potentially S2S properties from nifi.properties).
> The format for the bootstrap.properties should likely be similar to 
> nifi.properties. 
> Further, a way to incorporate this to leverage the libraries from core NiFi 
> will aid in achieving a proof of concept implementation more expediently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MINIFI-14) Add Listener/Notifier Logic to RunMiNiFi

2016-04-12 Thread Joseph Percivall (JIRA)

Joseph Percivall created MINIFI-14:
--

 Summary: Add Listener/Notifier Logic to RunMiNiFi
 Key: MINIFI-14
 URL: https://issues.apache.org/jira/browse/MINIFI-14
 Project: Apache NiFi MiNiFi
  Issue Type: Sub-task
Reporter: Joseph Percivall


The interface for the ConfigurationChangeNotifier and 
ConfigurationChangeListener is done as part of MINIFI-4 but configuration and 
implementation in RunMiNiFi needs to be added. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MINIFI-2) File based implementation


[ 
https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237679#comment-15237679
 ] 

ASF GitHub Bot commented on MINIFI-2:
-

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59426270
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
 ---
@@ -21,6 +21,11 @@
 public interface ConfigurationChangeNotifier {
 
 /**
+ * Begins the associated notification service provided by the given 
implementation.  In most implementations, no action will occur until this 
method is invoked.
+ */
+void initialize();
--- End diff --

I can dig that.  Will add.


> File based implementation
> -
>
> Key: MINIFI-2
> URL: https://issues.apache.org/jira/browse/MINIFI-2
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Processing Configuration
>Reporter: Aldrin Piri
> Fix For: 0.0.1
>
> Attachments: ConfigFileExplanation.xml, Properties_Guide.md, 
> config.yml
>
>
> A defined schema for processing configuration will need to be defined.  XML 
> or YAML seem like possible options for achieving this.
> Agent configuration will come from two places, a bootstrap properties file 
> and a flow configuration file. The bootstrap will make use of some of the 
> original nifi.properties in order to start up the agent. The flow 
> configuration file will essentially be a stripped down version of a flow.xml 
> (with potentially S2S properties from nifi.properties).
> The format for the bootstrap.properties should likely be similar to 
> nifi.properties. 
> Further, a way to incorporate this to leverage the libraries from core NiFi 
> will aid in achieving a proof of concept implementation more expediently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MINIFI-2) File based implementation


[ 
https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237670#comment-15237670
 ] 

ASF GitHub Bot commented on MINIFI-2:
-

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/7#discussion_r59425771
  
--- Diff: 
minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java
 ---
@@ -21,6 +21,11 @@
 public interface ConfigurationChangeNotifier {
 
 /**
+ * Begins the associated notification service provided by the given 
implementation.  In most implementations, no action will occur until this 
method is invoked.
+ */
+void initialize();
--- End diff --

This should pass a Properties object so that change notifiers can be 
configured. The user would add properties to the bootstrap.conf (in the same 
way NiFi has notification services configured there) and RunMiNiFi would pass 
the bootstrap properties to each Change Notifier. 


> File based implementation
> -
>
> Key: MINIFI-2
> URL: https://issues.apache.org/jira/browse/MINIFI-2
> Project: Apache NiFi MiNiFi
>  Issue Type: Sub-task
>  Components: Processing Configuration
>Reporter: Aldrin Piri
> Fix For: 0.0.1
>
> Attachments: ConfigFileExplanation.xml, Properties_Guide.md, 
> config.yml
>
>
> A defined schema for processing configuration will need to be defined.  XML 
> or YAML seem like possible options for achieving this.
> Agent configuration will come from two places, a bootstrap properties file 
> and a flow configuration file. The bootstrap will make use of some of the 
> original nifi.properties in order to start up the agent. The flow 
> configuration file will essentially be a stripped down version of a flow.xml 
> (with potentially S2S properties from nifi.properties).
> The format for the bootstrap.properties should likely be similar to 
> nifi.properties. 
> Further, a way to incorporate this to leverage the libraries from core NiFi 
> will aid in achieving a proof of concept implementation more expediently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237593#comment-15237593
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59417825
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.AbstractReportingTask;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart of NiFi the task knows where it left off.")
+public class ProvenanceReportingTask extends AbstractReportingTask {
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to post the Provenance Events to.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+static final

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237592#comment-15237592
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59417575
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.AbstractReportingTask;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart of NiFi the task knows where it left off.")
+public class ProvenanceReportingTask extends AbstractReportingTask {
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to post the Provenance Events to.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+static final

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237588#comment-15237588
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59417236
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java
 ---
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.AbstractReportingTask;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.Severity;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.ssl.SSLContextService.ClientAuth;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart of NiFi the task knows where it left off.")
+public class ProvenanceReportingTask extends AbstractReportingTask {
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to post the Provenance Events to.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.URL_VALIDATOR)
+.build();
+static final

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237563#comment-15237563
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59414741
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/test/java/org/apache/nifi/minifi/provenance/reporting/TestProvenanceReportingTask.java
 ---
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.provenance.ProvenanceEventBuilder;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.provenance.ProvenanceEventType;
+import org.apache.nifi.provenance.StandardProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.EventAccess;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.ReportingInitializationContext;
+import org.apache.nifi.state.MockStateManager;
+import org.apache.nifi.util.MockPropertyValue;
+import org.junit.Assert;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+public class TestProvenanceReportingTask {
+
+@Test
+public void testSerializedForm() throws IOException, 
InitializationException {
+final String uuid = "1000----";
+final Map attributes = new HashMap<>();
+attributes.put("abc", "xyz");
+attributes.put("xyz", "abc");
+attributes.put("filename", "file-" + uuid);
+
+final Map prevAttrs = new HashMap<>();
+attributes.put("filename", "1234.xyz");
+
+final Set lineageIdentifiers = new HashSet<>();
+lineageIdentifiers.add("123");
+lineageIdentifiers.add("321");
+
+final ProvenanceEventBuilder builder = new 
StandardProvenanceEventRecord.Builder();
+builder.setEventTime(System.currentTimeMillis());
+builder.setEventType(ProvenanceEventType.RECEIVE);
+builder.setTransitUri("nifi://unit-test");
+attributes.put("uuid", uuid);
+builder.fromFlowFile(createFlowFile(3L, 3000L, attributes));
+builder.setAttributes(prevAttrs, attributes);
+builder.setComponentId("1234");
+builder.setComponentType("dummy processor");
+builder.setLineageIdentifiers(lineageIdentifiers);
+final ProvenanceEventRecord event = builder.build();
+
+final List dataSent = new ArrayList<>();
+final ProvenanceReportingTask task = new ProvenanceReportingTask() 
{
+@SuppressWarnings("unchecked")
+@Override
+protected SiteToSiteClient getClient() {
+final SiteToSiteClient client = 
Mockito.mock(SiteToSiteClient.class);
+final Transaction transaction = 
Mockito.mock(Transaction.class);
+
+try {
+Mockito.doAnswer(new Answ

[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data


[ 
https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237555#comment-15237555
 ] 

ASF GitHub Bot commented on MINIFI-13:
--

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/6#discussion_r59413942
  
--- Diff: 
minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/test/java/org/apache/nifi/minifi/provenance/reporting/TestProvenanceReportingTask.java
 ---
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.provenance.reporting;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.provenance.ProvenanceEventBuilder;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.provenance.ProvenanceEventType;
+import org.apache.nifi.provenance.StandardProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.reporting.EventAccess;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingContext;
+import org.apache.nifi.reporting.ReportingInitializationContext;
+import org.apache.nifi.state.MockStateManager;
+import org.apache.nifi.util.MockPropertyValue;
+import org.junit.Assert;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+public class TestProvenanceReportingTask {
+
+@Test
+public void testSerializedForm() throws IOException, 
InitializationException {
+final String uuid = "1000----";
+final Map attributes = new HashMap<>();
+attributes.put("abc", "xyz");
+attributes.put("xyz", "abc");
+attributes.put("filename", "file-" + uuid);
+
+final Map prevAttrs = new HashMap<>();
+attributes.put("filename", "1234.xyz");
+
+final Set lineageIdentifiers = new HashSet<>();
+lineageIdentifiers.add("123");
+lineageIdentifiers.add("321");
+
+final ProvenanceEventBuilder builder = new 
StandardProvenanceEventRecord.Builder();
+builder.setEventTime(System.currentTimeMillis());
+builder.setEventType(ProvenanceEventType.RECEIVE);
+builder.setTransitUri("nifi://unit-test");
+attributes.put("uuid", uuid);
+builder.fromFlowFile(createFlowFile(3L, 3000L, attributes));
+builder.setAttributes(prevAttrs, attributes);
+builder.setComponentId("1234");
+builder.setComponentType("dummy processor");
+builder.setLineageIdentifiers(lineageIdentifiers);
+final ProvenanceEventRecord event = builder.build();
+
+final List dataSent = new ArrayList<>();
+final ProvenanceReportingTask task = new ProvenanceReportingTask() 
{
+@SuppressWarnings("unchecked")
+@Override
+protected SiteToSiteClient getClient() {
+final SiteToSiteClient client = 
Mockito.mock(SiteToSiteClient.class);
+final Transaction transaction = 
Mockito.mock(Transaction.class);
+
+try {
+Mockito.doAnswer(new Answ

[jira] [Commented] (NIFI-1296) Add capabilities to Kafka NAR to use new Kafka API (0.9)


[ 
https://issues.apache.org/jira/browse/NIFI-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237463#comment-15237463
 ] 

ASF GitHub Bot commented on NIFI-1296:
--

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/345

NIFI-1296[REVIEW_ONLY] initial commit of Kafka processors that leverage 
Kafka 0.9 API

This PR is not ready to be merged and is only to initiate incremental 
review and initial testing of a new pair of Kafka processors that are build 
using Kafka 0.9 API

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-1296

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/345.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #345


commit 465f1e049bbc37aaed1864121c6e8bc18d05e8ee
Author: Oleg Zhurakousky 
Date:   2016-04-07T11:15:25Z

NIFI-1296 initial commit of Kafka processors that leverage Kafka 0.9 API




> Add capabilities to Kafka NAR to use new Kafka API (0.9)
> 
>
> Key: NIFI-1296
> URL: https://issues.apache.org/jira/browse/NIFI-1296
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 0.4.0
>Reporter: Oleg Zhurakousky
>Assignee: Oleg Zhurakousky
> Fix For: 0.7.0
>
>
> Not sure when can we address this, but the interesting comment in 
> https://github.com/apache/nifi/pull/143. The usage of new API may introduce 
> issues with running against older Kafka brokers (e.g., 0.8). Need to 
> investigate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-1756) RuntimeDelegate.class confliction.

2016-04-12 Thread Brandon Zachary (JIRA)

Brandon Zachary created NIFI-1756:
-

 Summary: RuntimeDelegate.class confliction.
 Key: NIFI-1756
 URL: https://issues.apache.org/jira/browse/NIFI-1756
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Brandon Zachary


Hello,
I'm creating a controller service that uses a Client object from the 
javax.ws.rs-api-2.0.1.jar. Whenever, I go to enable my controller service I get 
a ClassCastException where there is a jsr311-api-1.1.1.jar in the 
nifi-framework-nar  that has a javax.ws.rs.ext.RuntimeDelegate.class that is 
trying to cast to the RuntimeDelegate that's in my javax.jar. Clearly the two 
don't mesh as I feel the jsr311 version is out of date. I would prefer to have 
the client instantiated in the controller service but for now I'm creating it 
in the processor. Apparently the processor doesn't go through the same 
nifi-framework calls as controller services do. Can that jar be updated to a 
more recent implementation of javax.ws.rs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1323) UI visual design enhancement