[jira] [Commented] (NIFI-1752) EvaluateXPath invalid when starting processor
[ https://issues.apache.org/jira/browse/NIFI-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238626#comment-15238626 ] Joseph Witt commented on NIFI-1752: --- ahh interesting. We did fix several calls to ensure that they were done within a NarCloseable for 0.6.0. You are definitely on the right trail and in any case this is important to track down. Putting 0.7.0 on for fix version. > EvaluateXPath invalid when starting processor > - > > Key: NIFI-1752 > URL: https://issues.apache.org/jira/browse/NIFI-1752 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.0 >Reporter: Joseph Witt > Fix For: 0.7.0 > > > Guillaume Pool: > Hi, > > If anyone could help I would appreciate it: > > After upgrading to 0.6.0 all my EvaluateXMLPath processors report as invalid > with nothing specific reported as the error. > > I use flowfile attribute as the destination, xpath query similar to > /message/address > > Return type auto-detect / string > > Error: > > 2016-04-11 17:37:12,465 INFO [NiFi Web Server-64629] > org.apache.nifi.web.filter.RequestLogger Attempting request for (anonymous) > PUT > http://sv-hdpdf1.hdp.supergrp.net:8443/nifi-api/controller/process-groups/d02a0bbc-c142-37ba-adaa-ee191b98790d/processors/e49acf40-76d1-3b6e-a67d-dc40153605ad > (source ip: 172.27.0.97) > 2016-04-11 17:37:12,474 INFO [NiFi Web Server-64629] > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: > EvaluateXPath[id=e49acf40-76d1-3b6e-a67d-dc40153605ad] is not in a valid > state. Returning Conflict response. > > Reports this even if you drop a new processor in. > James Wing: > I am able to see the same EvaluateXPath issue Guillame Pool reported on NiFi > 0.6.0. I created a template gist at > https://gist.github.com/jvwing/7e7948d8eb5ad3643f38966b4ba3ce2e that > illustrates this for me. You don't even need to run the whole thing, just > try to start the EvaluateXPath processor. I see a pop-up error trying to > start it "EvaluateXPath[id=b481c570-686a-4b6e-859c-9ebe27398fe1] is not in a > valid state". > Trying to troubleshoot this, I attached a debugger and found an > IllegalStateException thrown by StandardProcessorNode.verifyCanStart() on > line 1141. The error text was: > 'address' validated against '/message/address' is invalid because Unable to > initialize XPath engine due to > javax.xml.xpath.XPathFactoryConfigurationException: No XPathFactory > implementation found for the object model: http://saxon.sf.net/jaxp/xpath/om > I haven't figured out what change is the root cause, but the trail seems to > lead into validating the dynamic XPath attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1752) EvaluateXPath invalid when starting processor
[ https://issues.apache.org/jira/browse/NIFI-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-1752: -- Fix Version/s: 0.7.0 > EvaluateXPath invalid when starting processor > - > > Key: NIFI-1752 > URL: https://issues.apache.org/jira/browse/NIFI-1752 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.0 >Reporter: Joseph Witt > Fix For: 0.7.0 > > > Guillaume Pool: > Hi, > > If anyone could help I would appreciate it: > > After upgrading to 0.6.0 all my EvaluateXMLPath processors report as invalid > with nothing specific reported as the error. > > I use flowfile attribute as the destination, xpath query similar to > /message/address > > Return type auto-detect / string > > Error: > > 2016-04-11 17:37:12,465 INFO [NiFi Web Server-64629] > org.apache.nifi.web.filter.RequestLogger Attempting request for (anonymous) > PUT > http://sv-hdpdf1.hdp.supergrp.net:8443/nifi-api/controller/process-groups/d02a0bbc-c142-37ba-adaa-ee191b98790d/processors/e49acf40-76d1-3b6e-a67d-dc40153605ad > (source ip: 172.27.0.97) > 2016-04-11 17:37:12,474 INFO [NiFi Web Server-64629] > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: > EvaluateXPath[id=e49acf40-76d1-3b6e-a67d-dc40153605ad] is not in a valid > state. Returning Conflict response. > > Reports this even if you drop a new processor in. > James Wing: > I am able to see the same EvaluateXPath issue Guillame Pool reported on NiFi > 0.6.0. I created a template gist at > https://gist.github.com/jvwing/7e7948d8eb5ad3643f38966b4ba3ce2e that > illustrates this for me. You don't even need to run the whole thing, just > try to start the EvaluateXPath processor. I see a pop-up error trying to > start it "EvaluateXPath[id=b481c570-686a-4b6e-859c-9ebe27398fe1] is not in a > valid state". > Trying to troubleshoot this, I attached a debugger and found an > IllegalStateException thrown by StandardProcessorNode.verifyCanStart() on > line 1141. The error text was: > 'address' validated against '/message/address' is invalid because Unable to > initialize XPath engine due to > javax.xml.xpath.XPathFactoryConfigurationException: No XPathFactory > implementation found for the object model: http://saxon.sf.net/jaxp/xpath/om > I haven't figured out what change is the root cause, but the trail seems to > lead into validating the dynamic XPath attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1752) EvaluateXPath invalid when starting processor
[ https://issues.apache.org/jira/browse/NIFI-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238625#comment-15238625 ] James Wing commented on NIFI-1752: -- I believe the problem is not concurrency, but thread-specific class loaders used by XPathFactory. Calls to XPathValidator.validate() on thread "Timer-Driven Process Thread-?" use a NarThreadContextClassLoader and succeed. Calls on thread "NiFi Web Server-?" used class loader WebAppClassLoader and failed loading the same class net.sf.saxon.xpath.XPathFactoryImpl. I'm not sure why this error seems intermittent and selective to particular installations. Maybe classpath? On one of my instances I was able to resolve the issue by dropping a copy of Saxon-HE-9.6.0-5.jar into the {{lib}} folder and restarting NiFi. This might be a workaround, but I certainly haven't tested it thoroughly. [~dionosys], would you be willing to try this? I also do not know if this is new in 0.6.0 or how it might have been introduced. I'm working on a fix that explicitly uses the XPathValidator's instance ClassLoader in the call to XPathFactory.newInstance(). > EvaluateXPath invalid when starting processor > - > > Key: NIFI-1752 > URL: https://issues.apache.org/jira/browse/NIFI-1752 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.0 >Reporter: Joseph Witt > > Guillaume Pool: > Hi, > > If anyone could help I would appreciate it: > > After upgrading to 0.6.0 all my EvaluateXMLPath processors report as invalid > with nothing specific reported as the error. > > I use flowfile attribute as the destination, xpath query similar to > /message/address > > Return type auto-detect / string > > Error: > > 2016-04-11 17:37:12,465 INFO [NiFi Web Server-64629] > org.apache.nifi.web.filter.RequestLogger Attempting request for (anonymous) > PUT > http://sv-hdpdf1.hdp.supergrp.net:8443/nifi-api/controller/process-groups/d02a0bbc-c142-37ba-adaa-ee191b98790d/processors/e49acf40-76d1-3b6e-a67d-dc40153605ad > (source ip: 172.27.0.97) > 2016-04-11 17:37:12,474 INFO [NiFi Web Server-64629] > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: > EvaluateXPath[id=e49acf40-76d1-3b6e-a67d-dc40153605ad] is not in a valid > state. Returning Conflict response. > > Reports this even if you drop a new processor in. > James Wing: > I am able to see the same EvaluateXPath issue Guillame Pool reported on NiFi > 0.6.0. I created a template gist at > https://gist.github.com/jvwing/7e7948d8eb5ad3643f38966b4ba3ce2e that > illustrates this for me. You don't even need to run the whole thing, just > try to start the EvaluateXPath processor. I see a pop-up error trying to > start it "EvaluateXPath[id=b481c570-686a-4b6e-859c-9ebe27398fe1] is not in a > valid state". > Trying to troubleshoot this, I attached a debugger and found an > IllegalStateException thrown by StandardProcessorNode.verifyCanStart() on > line 1141. The error text was: > 'address' validated against '/message/address' is invalid because Unable to > initialize XPath engine due to > javax.xml.xpath.XPathFactoryConfigurationException: No XPathFactory > implementation found for the object model: http://saxon.sf.net/jaxp/xpath/om > I haven't figured out what change is the root cause, but the trail seems to > lead into validating the dynamic XPath attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
svn commit: r13232 - /dev/nifi/nifi-0.6.1/
Author: joewitt Date: Wed Apr 13 02:29:38 2016 New Revision: 13232 Log: NIFI-1729 uploading RC2 artifacts Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.asc dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.md5 dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.sha1 dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.asc dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.md5 dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.sha1 dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip.asc dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip.md5 dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip.sha1 Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz == Binary files - no diff available. Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.asc == --- dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.asc (original) +++ dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.asc Wed Apr 13 02:29:38 2016 @@ -1,11 +1,11 @@ -BEGIN PGP SIGNATURE- Version: GnuPG v1 -iQEcBAABAgAGBQJXBWwiAAoJEJCTv4VPgRoa644H/2uuR/u1Cqd+5JX1NB5W5bmn -t3q97sa7MmDffI/wZ+FUx7eRPNcrdsT/RrwRfGBQLGLDWSNtaLYKVi/8EioIpiCr -Toy0Hfzwomz2jU0h+R9YeqUQ86gTYphYJnwZFRIwYlXUlNO/2lLmicD0UptU7JrW -x8cZ+gAL6i/mDLIARODKy1ygKIPetfS4D13lwDJL3sR0uC0ZD7EwVXZagiQ7SoFV -ZXUSJho94wyDHio9gM/Nn2jxzz7n3r2wBSIdvEUp6vd5QCsVsi7XrGM76A+yxHMc -YR2N/88aijeC3iWSYDUisLrHLTah2/7LXba8iwRF4qz9HGW0BexQwqPd70+hx88= -=q4Fp +iQEcBAABAgAGBQJXDauJAAoJEJCTv4VPgRoaxh4H/itNi4rEpqQJ/zmguXpnsaDI +MrGiZMqOiSjFGA4bZ3XcfVDgkqzOsDz33Z3cbK925Upt5yfPrK+GBjq80nsg3X5S +0vxYbt3evcNGf2YQkUVh5Pyb8iRjnST+pFKzep9KY48EQdWYjEa4K4aZbmcaLyVJ +/r4ui7v/aDyuKjCLuQWgYNOBFULz6YVxV2rKwbWRAAu7zinrg2DNpmpiDV7yNI2I +vC58mEJzC7yK0GP1hwTXxwHLJj95+TvXBQW9EsMGpMinAgOxS1cq2u9UMer5XnCw +xJgwkX8SWTlm9w9ZP3OVVJAKYY8fgQ+7TLzlsXS/1jinKqr5tdFNI0QWHZu/Jy8= +=CFrP -END PGP SIGNATURE- Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.md5 == --- dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.md5 (original) +++ dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.md5 Wed Apr 13 02:29:38 2016 @@ -1 +1 @@ -345492c9dea8e4c02cc76e31bc9e321b \ No newline at end of file +1d96ee62d58cd009d245ef34877f1121 \ No newline at end of file Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.sha1 == --- dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.sha1 (original) +++ dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.tar.gz.sha1 Wed Apr 13 02:29:38 2016 @@ -1 +1 @@ -728abd6b30f01f1d520f9a6a13114cda7ceb7db6 \ No newline at end of file +3d8d1a685212ba27b9eae7a0bc2684de1674553b \ No newline at end of file Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip == Binary files - no diff available. Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.asc == --- dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.asc (original) +++ dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.asc Wed Apr 13 02:29:38 2016 @@ -1,11 +1,11 @@ -BEGIN PGP SIGNATURE- Version: GnuPG v1 -iQEcBAABAgAGBQJXBWwWAAoJEJCTv4VPgRoas04H+gOCMGNyaRRRgiV52OiR5NhL -/lJ9DZjz90Cordwm4oUv1HuHWcFbnureDz8Mkfo9ZBwuWOpz0Ex8NdwQYauVZid4 -mM2QSFwWBw5h/JWVGcHWq7ZS9aqxEM4maeKypWyb1uOPObd5IAQcEPupbxcYyJYg -z9bSy2QOyeS8NFcCARFSpaYQnkjeOxbbONEM6GrHcsRcz1W6wiCvgxCBHD294Suz -c0eTWD/7a6b7TjltJocMRDzOLFJe73BxENpQBwv2WY7hUcyVc6UCgzU697fi7Fdv -hkW7meAzw2Ru9aHW7edsEVomRTNvKSaFUSoYiww8+rEXyppeFwbW5Mg4aYmLT+o= -=dnAw +iQEcBAABAgAGBQJXDavsAAoJEJCTv4VPgRoa7iMH/jA4+Q+269edTMgYpu4vcomp +Zw7+XvwcGFCmGXEgmdPZYxEA0SNtBD8yaG99/z6Mk+xOLbdRaNwL9GxBKyyWrRT+ +pTb2AH7BNtx5DHjELA8pmRIrIawFHUM5KFrhE6xdatE6eyInzCDhasID9QXSyx/l +0I7ftYwBLbuo1qJroS5CwqXbufYSC8zqjjk4sGAGIgi/dBgD49QVejxgf78KDBlo +i4JQFAiMWJrV2BHdrwTatD14nDGCeP3sxmKlzh4dJrqkeC95ojdr/gSInfuF0R6k +EVgBVId9V3Wfi5HfHSlHWm1l4aq/fiVckQyVyZ46Ikx9MP2uhT6sBcuqaoLUJJU= +=Mlsb -END PGP SIGNATURE- Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.md5 == --- dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.md5 (original) +++ dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.md5 Wed Apr 13 02:29:38 2016 @@ -1 +1 @@ -03cbb8a927c8029a61d4fabd5a4c7076 \ No newline at end of file +f0bab6c7ecec9b22c48a6f57e7f75cac \ No newline at end of file Modified: dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.sha1 == --- dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.sha1 (original) +++ dev/nifi/nifi-0.6.1/nifi-0.6.1-bin.zip.sha1 Wed Apr 13 02:29:38 2016 @@ -1 +1 @@ -590fd30137dce2ee9875174b4186fd43adc9b8a6 \ No newline at end of file +05bfcd4ad4be2
[nifi] Git Push Summary
Repository: nifi Updated Tags: refs/tags/nifi-0.6.1-RC2 [created] 8dfe90cd6
[jira] [Commented] (NIFI-1180) Modify PutS3Object to enable encryption
[ https://issues.apache.org/jira/browse/NIFI-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238338#comment-15238338 ] ASF subversion and git services commented on NIFI-1180: --- Commit e152dce07329576dbd894df938149a4a610082c3 in nifi's branch refs/heads/0.x from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=e152dce ] NIFI-1180 Adding more IT tests. This closes #336. Signed-off-by: Andy LoPresto > Modify PutS3Object to enable encryption > --- > > Key: NIFI-1180 > URL: https://issues.apache.org/jira/browse/NIFI-1180 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Elli Schwarz >Priority: Minor > > Amazon allows encryption to be enforced on objects that are put to S3. In the > PutS3Object class, the objectMetadata object contains a method > setSSEAlgorithm which can set encryption for the objects. A Nifi property > should be added to the PutS3Object class so a user can enable/disable > encryption from the processor itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-1180 Adding more IT tests.
Repository: nifi Updated Branches: refs/heads/0.x a42fa991e -> e152dce07 NIFI-1180 Adding more IT tests. This closes #336. Signed-off-by: Andy LoPresto Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/e152dce0 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/e152dce0 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/e152dce0 Branch: refs/heads/0.x Commit: e152dce07329576dbd894df938149a4a610082c3 Parents: a42fa99 Author: ijokarumawak Authored: Fri Apr 8 17:20:27 2016 +0900 Committer: Andy LoPresto Committed: Tue Apr 12 17:45:23 2016 -0700 -- .../nifi/processors/aws/s3/ITFetchS3Object.java | 8 ++- .../nifi/processors/aws/s3/ITPutS3Object.java | 61 2 files changed, 68 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/e152dce0/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java -- diff --git a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java index db115bb..b318924 100644 --- a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java +++ b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java @@ -55,6 +55,10 @@ public class ITFetchS3Object extends AbstractS3IT { runner.run(1); runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1); +final List ffs = runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS); +MockFlowFile ff = ffs.get(0); +ff.assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM); + ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME)); } @Test @@ -75,7 +79,9 @@ public class ITFetchS3Object extends AbstractS3IT { runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1); final List ffs = runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS); -ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION); +MockFlowFile ff = ffs.get(0); +ff.assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION); + ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME)); } @Test http://git-wip-us.apache.org/repos/asf/nifi/blob/e152dce0/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java -- diff --git a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java index 5d7797e..2db6c74 100644 --- a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java +++ b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java @@ -119,6 +119,67 @@ public class ITPutS3Object extends AbstractS3IT { } } +private void testPutThenFetch(String sseAlgorithm) throws IOException { + +// Put +TestRunner runner = TestRunners.newTestRunner(new PutS3Object()); + +runner.setProperty(PutS3Object.CREDENTIALS_FILE, CREDENTIALS_FILE); +runner.setProperty(PutS3Object.REGION, REGION); +runner.setProperty(PutS3Object.BUCKET, BUCKET_NAME); +if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){ +runner.setProperty(PutS3Object.SERVER_SIDE_ENCRYPTION, sseAlgorithm); +} + +final Map attrs = new HashMap<>(); +attrs.put("filename", "filename-on-s3.txt"); +runner.enqueue(getResourcePath(SAMPLE_FILE_RESOURCE_NAME), attrs); +runner.run(); + +runner.assertAllFlowFilesTransferred(PutS3Object.REL_SUCCESS, 1); +List ffs = runner.getFlowFilesForRelationship(PutS3Object.REL_SUCCESS); +if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){ +ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION); +} else { +ffs.get(0).assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM); +} + +// Fetch +runner = T
[jira] [Commented] (NIFI-1180) Modify PutS3Object to enable encryption
[ https://issues.apache.org/jira/browse/NIFI-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238335#comment-15238335 ] ASF subversion and git services commented on NIFI-1180: --- Commit dfa27263d22c6348873a751dc5cb4e5ba52bf860 in nifi's branch refs/heads/master from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=dfa2726 ] NIFI-1180 Adding more IT tests. This closes #336. Signed-off-by: Andy LoPresto > Modify PutS3Object to enable encryption > --- > > Key: NIFI-1180 > URL: https://issues.apache.org/jira/browse/NIFI-1180 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Elli Schwarz >Priority: Minor > > Amazon allows encryption to be enforced on objects that are put to S3. In the > PutS3Object class, the objectMetadata object contains a method > setSSEAlgorithm which can set encryption for the objects. A Nifi property > should be added to the PutS3Object class so a user can enable/disable > encryption from the processor itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1180) Modify PutS3Object to enable encryption
[ https://issues.apache.org/jira/browse/NIFI-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238336#comment-15238336 ] ASF GitHub Bot commented on NIFI-1180: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/336 > Modify PutS3Object to enable encryption > --- > > Key: NIFI-1180 > URL: https://issues.apache.org/jira/browse/NIFI-1180 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Elli Schwarz >Priority: Minor > > Amazon allows encryption to be enforced on objects that are put to S3. In the > PutS3Object class, the objectMetadata object contains a method > setSSEAlgorithm which can set encryption for the objects. A Nifi property > should be added to the PutS3Object class so a user can enable/disable > encryption from the processor itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-1180 Adding more IT tests.
Repository: nifi Updated Branches: refs/heads/master acfc01213 -> dfa27263d NIFI-1180 Adding more IT tests. This closes #336. Signed-off-by: Andy LoPresto Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/dfa27263 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/dfa27263 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/dfa27263 Branch: refs/heads/master Commit: dfa27263d22c6348873a751dc5cb4e5ba52bf860 Parents: acfc012 Author: ijokarumawak Authored: Fri Apr 8 17:20:27 2016 +0900 Committer: Andy LoPresto Committed: Tue Apr 12 17:44:29 2016 -0700 -- .../nifi/processors/aws/s3/ITFetchS3Object.java | 8 ++- .../nifi/processors/aws/s3/ITPutS3Object.java | 61 2 files changed, 68 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/dfa27263/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java -- diff --git a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java index db115bb..b318924 100644 --- a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java +++ b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITFetchS3Object.java @@ -55,6 +55,10 @@ public class ITFetchS3Object extends AbstractS3IT { runner.run(1); runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1); +final List ffs = runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS); +MockFlowFile ff = ffs.get(0); +ff.assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM); + ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME)); } @Test @@ -75,7 +79,9 @@ public class ITFetchS3Object extends AbstractS3IT { runner.assertAllFlowFilesTransferred(FetchS3Object.REL_SUCCESS, 1); final List ffs = runner.getFlowFilesForRelationship(FetchS3Object.REL_SUCCESS); -ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION); +MockFlowFile ff = ffs.get(0); +ff.assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION); + ff.assertContentEquals(getFileFromResourceName(SAMPLE_FILE_RESOURCE_NAME)); } @Test http://git-wip-us.apache.org/repos/asf/nifi/blob/dfa27263/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java -- diff --git a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java index 5d7797e..2db6c74 100644 --- a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java +++ b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/ITPutS3Object.java @@ -119,6 +119,67 @@ public class ITPutS3Object extends AbstractS3IT { } } +private void testPutThenFetch(String sseAlgorithm) throws IOException { + +// Put +TestRunner runner = TestRunners.newTestRunner(new PutS3Object()); + +runner.setProperty(PutS3Object.CREDENTIALS_FILE, CREDENTIALS_FILE); +runner.setProperty(PutS3Object.REGION, REGION); +runner.setProperty(PutS3Object.BUCKET, BUCKET_NAME); +if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){ +runner.setProperty(PutS3Object.SERVER_SIDE_ENCRYPTION, sseAlgorithm); +} + +final Map attrs = new HashMap<>(); +attrs.put("filename", "filename-on-s3.txt"); +runner.enqueue(getResourcePath(SAMPLE_FILE_RESOURCE_NAME), attrs); +runner.run(); + +runner.assertAllFlowFilesTransferred(PutS3Object.REL_SUCCESS, 1); +List ffs = runner.getFlowFilesForRelationship(PutS3Object.REL_SUCCESS); +if(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION.equals(sseAlgorithm)){ +ffs.get(0).assertAttributeEquals(PutS3Object.S3_SSE_ALGORITHM, ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION); +} else { +ffs.get(0).assertAttributeNotExists(PutS3Object.S3_SSE_ALGORITHM); +} + +// Fetch +runn
[jira] [Created] (NIFI-1759) Update to latest Commons Compress library
Koji Kawamura created NIFI-1759: --- Summary: Update to latest Commons Compress library Key: NIFI-1759 URL: https://issues.apache.org/jira/browse/NIFI-1759 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Koji Kawamura Priority: Minor The latest Commons Compress ver 1.11 was released on Apr 6, 2016. This issue is used for tracking the changes to update our usage of Commons Compress throughout NiFi to use this new library and to ensure tests are still good and perhaps add some more to verify a few of following cases: {quote} Version 1.11 brings performance improvements to the 7ZIP code and read-only support for ZIP archives using BZIP2 compression. In addition it contains numerous bugfixes and enhancements, see the full list below. The Apache Commons Compress Library defines a Java API for working with ar, cpio, tar, zip, 7z, arj, dump, gzip, pack200, bzip2, lzma, snappy, Z, xz and deflate files. {quote} quote from: [Release Announce | http://mail-archives.apache.org/mod_mbox/www-announce/201604.mbox/%3C87bn5m6ayl.fsf%40v45346.1blu.de%3E] [Release Note | https://commons.apache.org/proper/commons-compress/changes-report.html#a1.11] As of Apr 2016, following NiFi implementations uses Commons Compress 1.9 directly: * nifi-flowfile-packager ** org.apache.nifi.util.FlowFilePackagerV1 ** org.apache.nifi.util.FlowFileUnpackagerV1 * nifi-framework-cluster ** org.apache.nifi.cluster.flow.impl.DataFlowDaoImpl * nifi-hdfs-processors ** org.apache.nifi.processors.hadoop.TarUnpackerSequenceFileWriter * nifi-standard-processors ** org.apache.nifi.processors.standard.CompressContent ** org.apache.nifi.processors.standard.MergeContent ** org.apache.nifi.processors.standard.UnpackContent -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: Update getting-started.adoc
Repository: nifi Updated Branches: refs/heads/0.x 5e0476c36 -> a42fa991e Update getting-started.adoc Corrected button, menu item and icon inconsistencies/errors. Fixed bulleted list formatting error in "Working with Templates" section. This closes #329. Signed-off-by: Andy LoPresto Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/a42fa991 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/a42fa991 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/a42fa991 Branch: refs/heads/0.x Commit: a42fa991e1d7788e8cebc111663f509654936758 Parents: 5e0476c Author: Andrew Lim Authored: Tue Apr 5 16:34:21 2016 -0400 Committer: Andy LoPresto Committed: Tue Apr 12 17:09:14 2016 -0700 -- .../src/main/asciidoc/getting-started.adoc | 48 ++-- 1 file changed, 23 insertions(+), 25 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/a42fa991/nifi-docs/src/main/asciidoc/getting-started.adoc -- diff --git a/nifi-docs/src/main/asciidoc/getting-started.adoc b/nifi-docs/src/main/asciidoc/getting-started.adoc index 82ba1d8..7c51888 100644 --- a/nifi-docs/src/main/asciidoc/getting-started.adoc +++ b/nifi-docs/src/main/asciidoc/getting-started.adoc @@ -132,7 +132,7 @@ image:nifi-toolbar-components.png["Toolbar Components"] === Adding a Processor We can now begin creating our dataflow by adding a Processor to our canvas. To do this, drag the Processor icon -image:iconProcessor.png["Processor"] from the top-left of the screen into the middle of the canvas (the graph paper-like +(image:iconProcessor.png["Processor"]) from the top-left of the screen into the middle of the canvas (the graph paper-like background) and drop it there. This will give us a dialog that allows us to choose which Processor we want to add: image:add-processor.png["Add Processor"] @@ -160,16 +160,16 @@ for each type. Properties that are in bold are required properties. The Processo properties have been configured. The most important property to configure for GetFile is the directory from which to pick up files. If we set the directory name to `./data-in`, this will cause the Processor to start picking up any data in the `data-in` subdirectory of the NiFi Home directory. We can choose to configure several different -Properties for this Processor. If unsure what a particular Property does, we can hover over the help icon ( +Properties for this Processor. If unsure what a particular Property does, we can hover over the Help icon ( image:iconInfo.png["Help"] ) next to the Property Name with the mouse in order to read a description of the property. Additionally, the -tooltip that is displayed when hovering over the help icon will provide the default value for that property, +tooltip that is displayed when hovering over the Help icon will provide the default value for that property, if one exists, information about whether or not the property supports the Expression Language (see the <> section below), and previously configured values for that property. In order for this property to be valid, create a directory named `data-in` in the NiFi home directory and then -click the `OK` button to close the dialog. +click the `Ok` button to close the dialog. === Connecting Processors @@ -239,9 +239,8 @@ Processor, and then hold the Shift key while selecting the other Processor in or right-click and choose the `Start` menu item. As an alternative to using the context menu, we can select the Processors and then click the Start icon in the toolbar at the top of the screen. -Once started, the icon in the top-left corner of the Processors will change from a stopped icon to a Running icon. We can then -stop the Processors in the same manner that we started them but using the Stop icon in the toolbar or the Stop menu item -as opposed to the Start button. +Once started, the icon in the top-left corner of the Processors will change from a stopped icon to a running icon. We can then +stop the Processors by using the Stop icon in the toolbar or the `Stop` menu item. Once a Processor has started, we are not able to configure it anymore. Instead, when we right-click on the Processor, we are given the option to view its current configuration. In order to configure a Processor, we must first stop the Processor and @@ -482,7 +481,7 @@ process the data. In addition to having Processors that are able to extract particular pieces of information from FlowFile content into Attributes, it is also common for users to want to add their own user-defined Attributes to each FlowFile at a particular place in the flow. The UpdateAttribute Processor is designed specifically for this purpose. Users are able to add a new prope
nifi git commit: Update getting-started.adoc
Repository: nifi Updated Branches: refs/heads/master bd1f0a82a -> acfc01213 Update getting-started.adoc Corrected button, menu item and icon inconsistencies/errors. Fixed bulleted list formatting error in "Working with Templates" section. This closes #329. Signed-off-by: Andy LoPresto Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/acfc0121 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/acfc0121 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/acfc0121 Branch: refs/heads/master Commit: acfc01213cf81bb920d2917bc89dcb8f4cd14f63 Parents: bd1f0a8 Author: Andrew Lim Authored: Tue Apr 5 16:34:21 2016 -0400 Committer: Andy LoPresto Committed: Tue Apr 12 17:08:15 2016 -0700 -- .../src/main/asciidoc/getting-started.adoc | 48 ++-- 1 file changed, 23 insertions(+), 25 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/acfc0121/nifi-docs/src/main/asciidoc/getting-started.adoc -- diff --git a/nifi-docs/src/main/asciidoc/getting-started.adoc b/nifi-docs/src/main/asciidoc/getting-started.adoc index 82ba1d8..7c51888 100644 --- a/nifi-docs/src/main/asciidoc/getting-started.adoc +++ b/nifi-docs/src/main/asciidoc/getting-started.adoc @@ -132,7 +132,7 @@ image:nifi-toolbar-components.png["Toolbar Components"] === Adding a Processor We can now begin creating our dataflow by adding a Processor to our canvas. To do this, drag the Processor icon -image:iconProcessor.png["Processor"] from the top-left of the screen into the middle of the canvas (the graph paper-like +(image:iconProcessor.png["Processor"]) from the top-left of the screen into the middle of the canvas (the graph paper-like background) and drop it there. This will give us a dialog that allows us to choose which Processor we want to add: image:add-processor.png["Add Processor"] @@ -160,16 +160,16 @@ for each type. Properties that are in bold are required properties. The Processo properties have been configured. The most important property to configure for GetFile is the directory from which to pick up files. If we set the directory name to `./data-in`, this will cause the Processor to start picking up any data in the `data-in` subdirectory of the NiFi Home directory. We can choose to configure several different -Properties for this Processor. If unsure what a particular Property does, we can hover over the help icon ( +Properties for this Processor. If unsure what a particular Property does, we can hover over the Help icon ( image:iconInfo.png["Help"] ) next to the Property Name with the mouse in order to read a description of the property. Additionally, the -tooltip that is displayed when hovering over the help icon will provide the default value for that property, +tooltip that is displayed when hovering over the Help icon will provide the default value for that property, if one exists, information about whether or not the property supports the Expression Language (see the <> section below), and previously configured values for that property. In order for this property to be valid, create a directory named `data-in` in the NiFi home directory and then -click the `OK` button to close the dialog. +click the `Ok` button to close the dialog. === Connecting Processors @@ -239,9 +239,8 @@ Processor, and then hold the Shift key while selecting the other Processor in or right-click and choose the `Start` menu item. As an alternative to using the context menu, we can select the Processors and then click the Start icon in the toolbar at the top of the screen. -Once started, the icon in the top-left corner of the Processors will change from a stopped icon to a Running icon. We can then -stop the Processors in the same manner that we started them but using the Stop icon in the toolbar or the Stop menu item -as opposed to the Start button. +Once started, the icon in the top-left corner of the Processors will change from a stopped icon to a running icon. We can then +stop the Processors by using the Stop icon in the toolbar or the `Stop` menu item. Once a Processor has started, we are not able to configure it anymore. Instead, when we right-click on the Processor, we are given the option to view its current configuration. In order to configure a Processor, we must first stop the Processor and @@ -482,7 +481,7 @@ process the data. In addition to having Processors that are able to extract particular pieces of information from FlowFile content into Attributes, it is also common for users to want to add their own user-defined Attributes to each FlowFile at a particular place in the flow. The UpdateAttribute Processor is designed specifically for this purpose. Users are able to add a new
[jira] [Commented] (NIFI-1478) Audit SSLContextFactory and SSLSocketFactory usage throughout application
[ https://issues.apache.org/jira/browse/NIFI-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238270#comment-15238270 ] ASF GitHub Bot commented on NIFI-1478: -- GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/346 Nifi 1753 Removed legacy X.509 certificate implementation references Various logic throughout the application referenced `javax.security.cert.X509Certificate` which is a deprecated class and exists only for legacy compatibility with older JSSE implementations. As of Java SE 6, new development should use `java.security.cert.X509Certificate`. Most references to the legacy classes were in similar logic blocks to retrieve the distinguished name (DN) from the client certificate chain presented during TLS mutual authentication. I refactored this logic into a common utility method to deduplicate and provided utility methods for converting legacy `X509Certificate`s and the abstract `java.security.cert.Certificate` returned by the replacement method (`javax.net.ssl.SSLSession#getPeerCertificateChain()` is succeeded by `javax.net.ssl.SSLSession#getPeerCertificates()`) to the correct version of `X509Certificate`. The module `nifi-security-utils` was added as a dependency to `nifi-utils` but contains only two utility classes with static helper methods and four enums. This change may be reverted/expanded as part of the larger-scale work on NIFI-1478, NIFI-1480, etc., but that is 1.0.0 refactor work, while this was a surgical fix for both 0.7.0 and 1.0.0. This will be rebased & squashed before merging. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-1753 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/346.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #346 commit e2d1c359bb94148679a057b5521b3fb17f6a936a Author: Andy LoPresto Date: 2016-04-12T03:11:45Z NIFI-1753 Added convenience methods for converting legacy X.509 certificates and abstract certificates to correct X.509 format. Added unit tests for certificate manipulation. Replaced logic retrieving legacy X.509 certificates with new logic in NodeProtocolSenderImpl. Added bcpkix (Bouncy Castle PKI implementation) dependency to nifi-standard-processors pom. commit b9868ef297228ef7398a30fbddb915abef98eda6 Author: Andy LoPresto Date: 2016-04-12T03:36:38Z NIFI-1753 Added convenience method for extracting DN from peer certificate chain in SSL socket (canonical implementation to reduce code duplication and references to legacy certificate implementations). Refactored logic retrieving legacy X.509 certificates with reference to convenience method in NodeProtocolSenderImpl. Replaced logic retrieving legacy X.509 certificates with reference to convenience method in SocketProtocolListener. Cleaned up exception handling in SocketProtocolListener. Replaced legacy X.509 certificate declarations with new declarations in HandleHttpRequest (needs manual test). commit 64208973bf9046d447230c8f096dde09d47d01bd Author: Andy LoPresto Date: 2016-04-12T03:46:33Z NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in PostHTTP. commit cd35f9b36d1e373b99490a6f6df735cd16175383 Author: Andy LoPresto Date: 2016-04-12T04:16:57Z NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in SSLSocketChannel and EndpointConnectionPool. Temporary work-around of duplicate certificate conversion util method because nifi-utils cannot depend on nifi-security-utils. commit 965b766f2a0be9f703d10e231ca595464bc57f07 Author: Andy LoPresto Date: 2016-04-12T22:19:14Z NIFI-1753 Removed temporary work-around of duplicate certificate conversion util method and added nifi-security-utils as dependency of nifi-utils. commit 613fe0b62e56898ebe99fed50a25f3aa515d727a Author: Andy LoPresto Date: 2016-04-12T22:20:22Z NIFI-1753 Fixed nifi-utils pom.xml comment about additional dependencies. > Audit SSLContextFactory and SSLSocketFactory usage throughout application > - > > Key: NIFI-1478 > URL: https://issues.apache.org/jira/browse/NIFI-1478 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 0.5.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Critical > Labels: certificate, security, tls > Original Estimate: 336h > Remaining Estimate: 336h > > The internal use of {{SSLSocketFactory}} and {{SSLCont
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237999#comment-15237999 ] ASF GitHub Bot commented on MINIFI-13: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi/pull/6 > Create a Reporting Task to Send Provenance data > --- > > Key: MINIFI-13 > URL: https://issues.apache.org/jira/browse/MINIFI-13 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Data Format, Data Transmission >Reporter: Joseph Percivall >Assignee: Joseph Percivall > Fix For: 0.0.1 > > > With initial effort to re-use as much of NiFi as possible it is not possible > to easily create a ProvenanceReporter to add provenance events as attributes > to FlowFiles as it would require changing the ProvenancenReporter interface. > This will require utilizing a different extension point to transmit the > provenance data back to a core NiFi instance. > Probably the most efficient way to do this is to create a ReportingTask which > reports the provenance events using the S2S protocol. > In the future this will probably be retired as a reporting task as MiNiFi > grows to rely less on NiFi but this Reporting task could also be contributed > back to NiFi. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi-minifi git commit: MINIFI-13 created a provenance reporting task to send provenance information via S2S
Repository: nifi-minifi Updated Branches: refs/heads/master 59f2d4418 -> 61e3a925d MINIFI-13 created a provenance reporting task to send provenance information via S2S This closes #6 Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/61e3a925 Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/61e3a925 Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/61e3a925 Branch: refs/heads/master Commit: 61e3a925d3af52da04a2866a298bb88264b20c5b Parents: 59f2d44 Author: Joseph Percivall Authored: Fri Apr 8 13:17:33 2016 -0400 Committer: Joseph Percivall Committed: Tue Apr 12 17:03:59 2016 -0400 -- minifi-assembly/pom.xml | 6 + .../minifi-framework-bundle/pom.xml | 2 +- .../minifi-provenance-reporting-nar/pom.xml | 41 ++ .../src/main/resources/META-INF/NOTICE | 15 + .../minifi-provenance-reporting-task/pom.xml| 82 .../reporting/ProvenanceReportingTask.java | 457 +++ .../org.apache.nifi.reporting.ReportingTask | 16 + .../reporting/TestProvenanceReportingTask.java | 186 .../minifi-provenance-reporting-bundle/pom.xml | 41 ++ minifi-nar-bundles/pom.xml | 3 +- pom.xml | 13 +- 11 files changed, 854 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-assembly/pom.xml -- diff --git a/minifi-assembly/pom.xml b/minifi-assembly/pom.xml index b076459..8dd223a 100644 --- a/minifi-assembly/pom.xml +++ b/minifi-assembly/pom.xml @@ -105,6 +105,12 @@ limitations under the License. minifi-runtime 0.0.1-SNAPSHOT + +org.apache.nifi.minifi +minifi-provenance-reporting-nar +0.0.1-SNAPSHOT +nar + http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-nar-bundles/minifi-framework-bundle/pom.xml -- diff --git a/minifi-nar-bundles/minifi-framework-bundle/pom.xml b/minifi-nar-bundles/minifi-framework-bundle/pom.xml index 7fbd6b7..dca3a5c 100644 --- a/minifi-nar-bundles/minifi-framework-bundle/pom.xml +++ b/minifi-nar-bundles/minifi-framework-bundle/pom.xml @@ -20,7 +20,7 @@ limitations under the License. xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> minifi-nar-bundles -org.apache.nifi +org.apache.nifi.minifi 0.0.1-SNAPSHOT 4.0.0 http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml -- diff --git a/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml new file mode 100644 index 000..be74790 --- /dev/null +++ b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/pom.xml @@ -0,0 +1,41 @@ +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + +4.0.0 + +org.apache.nifi.minifi +minifi-provenance-reporting-bundle +0.0.1-SNAPSHOT + + +minifi-provenance-reporting-nar +nar + +true +true + + + +org.apache.nifi.minifi +minifi-provenance-reporting-task +0.0.1-SNAPSHOT + + +org.apache.nifi +nifi-standard-services-api-nar +nar + + + http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/61e3a925/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE -- diff --git a/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE new file mode 100644 index 000..be55e59 --- /dev/null +++ b/minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-nar/src/main/resources/META-INF/NOTICE @@ -0,0 +1,15 @@ +minifi-provenance-reporting-task-nar +Copyright 2015-2016 The Apache Software Foundation + +This
[jira] [Created] (NIFI-1758) Unable to scroll in the Comments field when configuring process group
Andrew Lim created NIFI-1758: Summary: Unable to scroll in the Comments field when configuring process group Key: NIFI-1758 URL: https://issues.apache.org/jira/browse/NIFI-1758 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 0.6.0 Reporter: Andrew Lim Priority: Minor When configuring a process group, the Comments field allows a large amount of text to be entered or pasted in. The size of the comments field displays about 6-7 rows (depending on what web browser is being used). However, scrolling is not allowed for that field, so the user is unable to easily see or edit the values in the field. A workaround is to use the keyboard arrow keys. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-2) File based implementation
[ https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237955#comment-15237955 ] ASF GitHub Bot commented on MINIFI-2: - Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi/pull/7 > File based implementation > - > > Key: MINIFI-2 > URL: https://issues.apache.org/jira/browse/MINIFI-2 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Processing Configuration >Reporter: Aldrin Piri > Fix For: 0.0.1 > > Attachments: ConfigFileExplanation.xml, Properties_Guide.md, > config.yml > > > A defined schema for processing configuration will need to be defined. XML > or YAML seem like possible options for achieving this. > Agent configuration will come from two places, a bootstrap properties file > and a flow configuration file. The bootstrap will make use of some of the > original nifi.properties in order to start up the agent. The flow > configuration file will essentially be a stripped down version of a flow.xml > (with potentially S2S properties from nifi.properties). > The format for the bootstrap.properties should likely be similar to > nifi.properties. > Further, a way to incorporate this to leverage the libraries from core NiFi > will aid in achieving a proof of concept implementation more expediently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi-minifi git commit: MINIFI-2 MINIFI-4 Establishing a base implementation of the configuration change listener and notification services as well as a File implementation.
Repository: nifi-minifi Updated Branches: refs/heads/master 5751e23d4 -> 59f2d4418 MINIFI-2 MINIFI-4 Establishing a base implementation of the configuration change listener and notification services as well as a File implementation. Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/59f2d441 Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/59f2d441 Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/59f2d441 Branch: refs/heads/master Commit: 59f2d441867eb349104795cbe98d490385eb415a Parents: 5751e23 Author: Aldrin Piri Authored: Fri Apr 8 14:59:58 2016 -0400 Committer: Aldrin Piri Committed: Tue Apr 12 16:49:37 2016 -0400 -- .../ConfigurationChangeListener.java| 34 +++ .../ConfigurationChangeNotifier.java| 57 + .../minifi-framework/minifi-runtime/pom.xml | 28 ++- .../configuration/FileChangeNotifier.java | 173 .../configuration/TestFileChangeNotifier.java | 206 +++ .../src/test/resources/config.yml | 0 pom.xml | 41 7 files changed, 533 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/59f2d441/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java -- diff --git a/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java new file mode 100644 index 000..0d09f72 --- /dev/null +++ b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeListener.java @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.minifi.configuration; + +import java.io.InputStream; + +/** + * Interface for handling events detected and driven by an associated {@link ConfigurationChangeNotifier} to which the listener + * has registered via {@link ConfigurationChangeNotifier#registerListener(ConfigurationChangeListener)}. + */ +public interface ConfigurationChangeListener { + +/** + * Provides a mechanism for the implementation to interpret the specified configuration change + * + * @param is stream of the detected content received from the change notifier + */ +void handleChange(InputStream is); + +} http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/59f2d441/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java -- diff --git a/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java new file mode 100644 index 000..30a98c2 --- /dev/null +++ b/minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java @@ -0,0 +1,57 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.minifi.configuration; + +import java.util.Properties; +import java.util.Set; + +public interfa
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237933#comment-15237933 ] ASF GitHub Bot commented on MINIFI-13: -- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59449408 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java --- @@ -251,6 +254,17 @@ public void onTrigger(final ReportingContext context) { if (state.containsKey(LAST_EVENT_ID_KEY)) { firstEventId = Long.parseLong(state.get(LAST_EVENT_ID_KEY)) + 1; } + +if(currMaxId < firstEventId){ +getLogger().debug("Current provenance max id is {} which is less than what was stored in state as the last queried event, which was {}. This means the provenance restarted it's " + --- End diff -- super minor: it's -> its > Create a Reporting Task to Send Provenance data > --- > > Key: MINIFI-13 > URL: https://issues.apache.org/jira/browse/MINIFI-13 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Data Format, Data Transmission >Reporter: Joseph Percivall >Assignee: Joseph Percivall > Fix For: 0.0.1 > > > With initial effort to re-use as much of NiFi as possible it is not possible > to easily create a ProvenanceReporter to add provenance events as attributes > to FlowFiles as it would require changing the ProvenancenReporter interface. > This will require utilizing a different extension point to transmit the > provenance data back to a core NiFi instance. > Probably the most efficient way to do this is to create a ReportingTask which > reports the provenance events using the S2S protocol. > In the future this will probably be retired as a reporting task as MiNiFi > grows to rely less on NiFi but this Reporting task could also be contributed > back to NiFi. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-2) File based implementation
[ https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237924#comment-15237924 ] ASF GitHub Bot commented on MINIFI-2: - Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/7#discussion_r59448506 --- Diff: minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/configuration/FileChangeNotifier.java --- @@ -42,33 +43,18 @@ */ public class FileChangeNotifier implements Runnable, ConfigurationChangeNotifier, Closeable { +private Path configFile; +private WatchService watchService; +private long pollingSeconds; + private final ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor(); -private final Path configFile; -private final WatchService watchService; private final Set configurationChangeListeners = new HashSet<>(); -private static final int DEFAULT_POLLING_PERIOD_INTERVAL = 15; -private static final TimeUnit DEFAULT_POLLING_PERIOD_UNIT = TimeUnit.SECONDS; - -/** - * @param configFile to monitor for changes - * @throws IOException if there are any issues with accessing the specified config file or generating the associated {@link WatchService}. - */ -public FileChangeNotifier(Path configFile) throws IOException { -this(configFile, initializeWatcher(configFile)); -} - - -public FileChangeNotifier(Path configFile, WatchService watchService) { -final File file = configFile.toFile(); -if (!file.exists() || !file.canRead() || !file.isFile()) { -throw new IllegalArgumentException(String.format("The specified path %s must be a readable file.", configFile)); -} - -this.configFile = configFile; -this.watchService = watchService; -} +protected static final String CONFIG_FILE_PATH_KEY = "nifi.notifier.file.config.path"; +protected static final String POLLING_PERIOD_INTERVAL_KEY = "nifi.notifier.file.polling.period.seconds"; --- End diff -- Should include "minifi" either by replacing "nifi" with it or by adding it subsequently. > File based implementation > - > > Key: MINIFI-2 > URL: https://issues.apache.org/jira/browse/MINIFI-2 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Processing Configuration >Reporter: Aldrin Piri > Fix For: 0.0.1 > > Attachments: ConfigFileExplanation.xml, Properties_Guide.md, > config.yml > > > A defined schema for processing configuration will need to be defined. XML > or YAML seem like possible options for achieving this. > Agent configuration will come from two places, a bootstrap properties file > and a flow configuration file. The bootstrap will make use of some of the > original nifi.properties in order to start up the agent. The flow > configuration file will essentially be a stripped down version of a flow.xml > (with potentially S2S properties from nifi.properties). > The format for the bootstrap.properties should likely be similar to > nifi.properties. > Further, a way to incorporate this to leverage the libraries from core NiFi > will aid in achieving a proof of concept implementation more expediently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237908#comment-15237908 ] Rob Moran commented on NIFI-1323: - Updated these two to show examples of components that a user may be unauthorized to access. *nifi-sample-flow* *nifi-component-samples* > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components, as well as those components when a user > is unauthorized to access. > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Description: (I will attach mockups and supporting files as they become available) I am starting to work on a design to modernize the look and feel of the NiFi UI. The initial focus of the design is to freshen the UI (flat design, SVG icons, etc.). Additionally, the new design will propose usability improvements such as exposing more flow-related actions into collapsible panes, improving hierarchy of information, etc. Going forward, the design plan is to help lay the foundation for other UI/UX related issues such as those documented in NIFI-951. --- *flowfont.zip* Contains icon font and supporting files *nifi-add-processor-dialog.png* Dialog sample. This sample shows the 'Add Processor' dialog. *nifi-component-samples.png* To show styling for all components, as well as those components when a user is unauthorized to access. *nifi-configure-processor-properties* *nifi-configure-processor-properties-set-value* *nifi-configure-processor-scheduling* *nifi-configure-processor-settings* Configure Processor dialog. See related Comments below (in Activity section). *nifi-dialog-sample-@800px* Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab of a provenance event. *nifi-drop.svg* NiFi logo without 'nifi *nifi-global-menu* To show global menu *nifi-interaction-and-menu-samples.png* To demonstrate user interactions - hover states, tooltips, menus, etc. *nifi-lineage-graph-@800px* To show lineage graph with explicit action to get back to data provenance event list. *nifi-sample-flow-@800px* Shows a very useable UI down to around 800px in width. The thinking here is that at anything lower than this, the NiFi user experience will change to more of a monitoring and/or administrative type workflow. Future mockups will be created to illustrate this. *nifi-logo.svg* NiFi logo complete *nifi-sample-flow.png* Mockup of sample flow. Updated to show revised tool and status bars. Management related actions will move to a menu via mouseover (see _nifi-global-menu_). Added benefits here include reducing clutter and more user-friendly menu with text labels to reduce time spent scanning only a large set of icons. This also helps gain valuable viewport width in a browser (see _nifi-sample-flow-@800px_) *nifi-shell-sample-@800px* Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' table. was: (I will attach mockups and supporting files as they become available) I am starting to work on a design to modernize the look and feel of the NiFi UI. The initial focus of the design is to freshen the UI (flat design, SVG icons, etc.). Additionally, the new design will propose usability improvements such as exposing more flow-related actions into collapsible panes, improving hierarchy of information, etc. Going forward, the design plan is to help lay the foundation for other UI/UX related issues such as those documented in NIFI-951. --- *flowfont.zip* Contains icon font and supporting files *nifi-add-processor-dialog.png* Dialog sample. This sample shows the 'Add Processor' dialog. *nifi-component-samples.png* To show styling for all components *nifi-configure-processor-properties* *nifi-configure-processor-properties-set-value* *nifi-configure-processor-scheduling* *nifi-configure-processor-settings* Configure Processor dialog. See related Comments below (in Activity section). *nifi-dialog-sample-@800px* Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab of a provenance event. *nifi-drop.svg* NiFi logo without 'nifi *nifi-global-menu* To show global menu *nifi-interaction-and-menu-samples.png* To demonstrate user interactions - hover states, tooltips, menus, etc. *nifi-lineage-graph-@800px* To show lineage graph with explicit action to get back to data provenance event list. *nifi-sample-flow-@800px* Shows a very useable UI down to around 800px in width. The thinking here is that at anything lower than this, the NiFi user experience will change to more of a monitoring and/or administrative type workflow. Future mockups will be created to illustrate this. *nifi-logo.svg* NiFi logo complete *nifi-sample-flow.png* Mockup of sample flow. Updated to show revised tool and status bars. Management related actions will move to a menu via mouseover (see _nifi-global-menu_). Added benefits here include reducing clutter and more user-friendly menu with text labels to reduce time spent scanning only a large set of icons. This also helps gain valuable viewport width in a browser (see _nifi-sample-flow-@800px_) *nifi-shell-sample-@800px* Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' table. > UI visual design enhancement > > > Key: NIFI-1323 >
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: (was: nifi-component-samples.png) > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: nifi-sample-flow.png nifi-component-samples.png > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: (was: nifi-sample-flow.png) > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MINIFI-14) Add Listener/Notifier Logic to RunMiNiFi
[ https://issues.apache.org/jira/browse/MINIFI-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri reassigned MINIFI-14: - Assignee: Aldrin Piri > Add Listener/Notifier Logic to RunMiNiFi > > > Key: MINIFI-14 > URL: https://issues.apache.org/jira/browse/MINIFI-14 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Processing Configuration >Reporter: Joseph Percivall >Assignee: Aldrin Piri > Fix For: 0.0.1 > > > The interface for the ConfigurationChangeNotifier and > ConfigurationChangeListener is done as part of MINIFI-4 but configuration and > implementation in RunMiNiFi needs to be added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237890#comment-15237890 ] ASF GitHub Bot commented on MINIFI-13: -- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59445052 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/test/java/org/apache/nifi/minifi/provenance/reporting/TestProvenanceReportingTask.java --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import static org.junit.Assert.assertEquals; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.UUID; + +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.provenance.ProvenanceEventBuilder; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.provenance.ProvenanceEventType; +import org.apache.nifi.provenance.StandardProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.EventAccess; +import org.apache.nifi.reporting.InitializationException; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.ReportingInitializationContext; +import org.apache.nifi.state.MockStateManager; +import org.apache.nifi.util.MockPropertyValue; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.Mockito; +import org.mockito.invocation.InvocationOnMock; +import org.mockito.stubbing.Answer; + +public class TestProvenanceReportingTask { + +@Test +public void testSerializedForm() throws IOException, InitializationException { +final String uuid = "1000----"; +final Map attributes = new HashMap<>(); +attributes.put("abc", "xyz"); +attributes.put("xyz", "abc"); +attributes.put("filename", "file-" + uuid); + +final Map prevAttrs = new HashMap<>(); +attributes.put("filename", "1234.xyz"); + +final Set lineageIdentifiers = new HashSet<>(); +lineageIdentifiers.add("123"); +lineageIdentifiers.add("321"); + +final ProvenanceEventBuilder builder = new StandardProvenanceEventRecord.Builder(); +builder.setEventTime(System.currentTimeMillis()); +builder.setEventType(ProvenanceEventType.RECEIVE); +builder.setTransitUri("nifi://unit-test"); +attributes.put("uuid", uuid); +builder.fromFlowFile(createFlowFile(3L, 3000L, attributes)); +builder.setAttributes(prevAttrs, attributes); +builder.setComponentId("1234"); +builder.setComponentType("dummy processor"); +builder.setLineageIdentifiers(lineageIdentifiers); +final ProvenanceEventRecord event = builder.build(); + +final List dataSent = new ArrayList<>(); +final ProvenanceReportingTask task = new ProvenanceReportingTask() { +@SuppressWarnings("unchecked") +@Override +protected SiteToSiteClient getClient() { +final SiteToSiteClient client = Mockito.mock(SiteToSiteClient.class); +final Transaction transaction = Mockito.mock(Transaction.class); + +try { +Mockito.doAnswer(new
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237823#comment-15237823 ] ASF GitHub Bot commented on MINIFI-13: -- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59438870 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java --- @@ -0,0 +1,443 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.text.DateFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TimeZone; +import java.util.UUID; +import java.util.concurrent.TimeUnit; + +import javax.json.Json; +import javax.json.JsonArray; +import javax.json.JsonArrayBuilder; +import javax.json.JsonBuilderFactory; +import javax.json.JsonObject; +import javax.json.JsonObjectBuilder; +import javax.net.ssl.SSLContext; + +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.Validator; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.controller.status.PortStatus; +import org.apache.nifi.controller.status.ProcessGroupStatus; +import org.apache.nifi.controller.status.ProcessorStatus; +import org.apache.nifi.controller.status.RemoteProcessGroupStatus; +import org.apache.nifi.events.EventReporter; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.Severity; +import org.apache.nifi.ssl.SSLContextService; +import org.apache.nifi.ssl.SSLContextService.ClientAuth; + +@Tags({"provenance", "lineage", "tracking", "site", "site to site"}) +@CapabilityDescription("Publishes Provenance events using the Site To Site protocol.") +@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's last event Id so that on restart of NiFi the task knows where it left off.") +public class ProvenanceReportingTask extends AbstractReportingTask { +private static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'"; +private static final String LAST_EVENT_ID_KEY = "last_event_id"; + +static final PropertyDescriptor DESTINATION_URL = new PropertyDescriptor.Builder() +.name("Destination URL") +.description("The URL to post the Provenance Events to.") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.URL_VALIDATOR) +.build(); +static
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237818#comment-15237818 ] ASF GitHub Bot commented on MINIFI-13: -- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59438406 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java --- @@ -0,0 +1,443 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.text.DateFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TimeZone; +import java.util.UUID; +import java.util.concurrent.TimeUnit; + +import javax.json.Json; +import javax.json.JsonArray; +import javax.json.JsonArrayBuilder; +import javax.json.JsonBuilderFactory; +import javax.json.JsonObject; +import javax.json.JsonObjectBuilder; +import javax.net.ssl.SSLContext; + +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.Validator; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.controller.status.PortStatus; +import org.apache.nifi.controller.status.ProcessGroupStatus; +import org.apache.nifi.controller.status.ProcessorStatus; +import org.apache.nifi.controller.status.RemoteProcessGroupStatus; +import org.apache.nifi.events.EventReporter; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.Severity; +import org.apache.nifi.ssl.SSLContextService; +import org.apache.nifi.ssl.SSLContextService.ClientAuth; + +@Tags({"provenance", "lineage", "tracking", "site", "site to site"}) +@CapabilityDescription("Publishes Provenance events using the Site To Site protocol.") +@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's last event Id so that on restart of NiFi the task knows where it left off.") +public class ProvenanceReportingTask extends AbstractReportingTask { +private static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'"; +private static final String LAST_EVENT_ID_KEY = "last_event_id"; + +static final PropertyDescriptor DESTINATION_URL = new PropertyDescriptor.Builder() +.name("Destination URL") +.description("The URL to post the Provenance Events to.") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.URL_VALIDATOR) +.build(); +static
[jira] [Commented] (MINIFI-2) File based implementation
[ https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237735#comment-15237735 ] ASF GitHub Bot commented on MINIFI-2: - Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/7#discussion_r59429946 --- Diff: minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java --- @@ -21,6 +21,11 @@ public interface ConfigurationChangeNotifier { /** + * Begins the associated notification service provided by the given implementation. In most implementations, no action will occur until this method is invoked. + */ +void initialize(); --- End diff -- That works for me > File based implementation > - > > Key: MINIFI-2 > URL: https://issues.apache.org/jira/browse/MINIFI-2 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Processing Configuration >Reporter: Aldrin Piri > Fix For: 0.0.1 > > Attachments: ConfigFileExplanation.xml, Properties_Guide.md, > config.yml > > > A defined schema for processing configuration will need to be defined. XML > or YAML seem like possible options for achieving this. > Agent configuration will come from two places, a bootstrap properties file > and a flow configuration file. The bootstrap will make use of some of the > original nifi.properties in order to start up the agent. The flow > configuration file will essentially be a stripped down version of a flow.xml > (with potentially S2S properties from nifi.properties). > The format for the bootstrap.properties should likely be similar to > nifi.properties. > Further, a way to incorporate this to leverage the libraries from core NiFi > will aid in achieving a proof of concept implementation more expediently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1757) If Bulletin Board is opened, the requests keep occurring in the background even after the Bulletin Board is closed
Mark Payne created NIFI-1757: Summary: If Bulletin Board is opened, the requests keep occurring in the background even after the Bulletin Board is closed Key: NIFI-1757 URL: https://issues.apache.org/jira/browse/NIFI-1757 Project: Apache NiFi Issue Type: Bug Components: Core UI Reporter: Mark Payne I clicked to open the Bulletin Board and then closed it. Opening Chrome's Developer Tools, I can see that the requests are still made to retrieve bulletins every few seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-2) File based implementation
[ https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237689#comment-15237689 ] ASF GitHub Bot commented on MINIFI-2: - Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/7#discussion_r59427453 --- Diff: minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java --- @@ -21,6 +21,11 @@ public interface ConfigurationChangeNotifier { /** + * Begins the associated notification service provided by the given implementation. In most implementations, no action will occur until this method is invoked. + */ +void initialize(); --- End diff -- Actually, given that this is a separate task, how would you feel about initialize in its current context being a #start method and then #initialize providing the properties as discussed? > File based implementation > - > > Key: MINIFI-2 > URL: https://issues.apache.org/jira/browse/MINIFI-2 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Processing Configuration >Reporter: Aldrin Piri > Fix For: 0.0.1 > > Attachments: ConfigFileExplanation.xml, Properties_Guide.md, > config.yml > > > A defined schema for processing configuration will need to be defined. XML > or YAML seem like possible options for achieving this. > Agent configuration will come from two places, a bootstrap properties file > and a flow configuration file. The bootstrap will make use of some of the > original nifi.properties in order to start up the agent. The flow > configuration file will essentially be a stripped down version of a flow.xml > (with potentially S2S properties from nifi.properties). > The format for the bootstrap.properties should likely be similar to > nifi.properties. > Further, a way to incorporate this to leverage the libraries from core NiFi > will aid in achieving a proof of concept implementation more expediently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MINIFI-14) Add Listener/Notifier Logic to RunMiNiFi
Joseph Percivall created MINIFI-14: -- Summary: Add Listener/Notifier Logic to RunMiNiFi Key: MINIFI-14 URL: https://issues.apache.org/jira/browse/MINIFI-14 Project: Apache NiFi MiNiFi Issue Type: Sub-task Reporter: Joseph Percivall The interface for the ConfigurationChangeNotifier and ConfigurationChangeListener is done as part of MINIFI-4 but configuration and implementation in RunMiNiFi needs to be added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-2) File based implementation
[ https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237679#comment-15237679 ] ASF GitHub Bot commented on MINIFI-2: - Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/7#discussion_r59426270 --- Diff: minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java --- @@ -21,6 +21,11 @@ public interface ConfigurationChangeNotifier { /** + * Begins the associated notification service provided by the given implementation. In most implementations, no action will occur until this method is invoked. + */ +void initialize(); --- End diff -- I can dig that. Will add. > File based implementation > - > > Key: MINIFI-2 > URL: https://issues.apache.org/jira/browse/MINIFI-2 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Processing Configuration >Reporter: Aldrin Piri > Fix For: 0.0.1 > > Attachments: ConfigFileExplanation.xml, Properties_Guide.md, > config.yml > > > A defined schema for processing configuration will need to be defined. XML > or YAML seem like possible options for achieving this. > Agent configuration will come from two places, a bootstrap properties file > and a flow configuration file. The bootstrap will make use of some of the > original nifi.properties in order to start up the agent. The flow > configuration file will essentially be a stripped down version of a flow.xml > (with potentially S2S properties from nifi.properties). > The format for the bootstrap.properties should likely be similar to > nifi.properties. > Further, a way to incorporate this to leverage the libraries from core NiFi > will aid in achieving a proof of concept implementation more expediently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-2) File based implementation
[ https://issues.apache.org/jira/browse/MINIFI-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237670#comment-15237670 ] ASF GitHub Bot commented on MINIFI-2: - Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/7#discussion_r59425771 --- Diff: minifi-api/src/main/java/org/apache/nifi/minifi/configuration/ConfigurationChangeNotifier.java --- @@ -21,6 +21,11 @@ public interface ConfigurationChangeNotifier { /** + * Begins the associated notification service provided by the given implementation. In most implementations, no action will occur until this method is invoked. + */ +void initialize(); --- End diff -- This should pass a Properties object so that change notifiers can be configured. The user would add properties to the bootstrap.conf (in the same way NiFi has notification services configured there) and RunMiNiFi would pass the bootstrap properties to each Change Notifier. > File based implementation > - > > Key: MINIFI-2 > URL: https://issues.apache.org/jira/browse/MINIFI-2 > Project: Apache NiFi MiNiFi > Issue Type: Sub-task > Components: Processing Configuration >Reporter: Aldrin Piri > Fix For: 0.0.1 > > Attachments: ConfigFileExplanation.xml, Properties_Guide.md, > config.yml > > > A defined schema for processing configuration will need to be defined. XML > or YAML seem like possible options for achieving this. > Agent configuration will come from two places, a bootstrap properties file > and a flow configuration file. The bootstrap will make use of some of the > original nifi.properties in order to start up the agent. The flow > configuration file will essentially be a stripped down version of a flow.xml > (with potentially S2S properties from nifi.properties). > The format for the bootstrap.properties should likely be similar to > nifi.properties. > Further, a way to incorporate this to leverage the libraries from core NiFi > will aid in achieving a proof of concept implementation more expediently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237593#comment-15237593 ] ASF GitHub Bot commented on MINIFI-13: -- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59417825 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java --- @@ -0,0 +1,443 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.text.DateFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TimeZone; +import java.util.UUID; +import java.util.concurrent.TimeUnit; + +import javax.json.Json; +import javax.json.JsonArray; +import javax.json.JsonArrayBuilder; +import javax.json.JsonBuilderFactory; +import javax.json.JsonObject; +import javax.json.JsonObjectBuilder; +import javax.net.ssl.SSLContext; + +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.Validator; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.controller.status.PortStatus; +import org.apache.nifi.controller.status.ProcessGroupStatus; +import org.apache.nifi.controller.status.ProcessorStatus; +import org.apache.nifi.controller.status.RemoteProcessGroupStatus; +import org.apache.nifi.events.EventReporter; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.Severity; +import org.apache.nifi.ssl.SSLContextService; +import org.apache.nifi.ssl.SSLContextService.ClientAuth; + +@Tags({"provenance", "lineage", "tracking", "site", "site to site"}) +@CapabilityDescription("Publishes Provenance events using the Site To Site protocol.") +@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's last event Id so that on restart of NiFi the task knows where it left off.") +public class ProvenanceReportingTask extends AbstractReportingTask { +private static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'"; +private static final String LAST_EVENT_ID_KEY = "last_event_id"; + +static final PropertyDescriptor DESTINATION_URL = new PropertyDescriptor.Builder() +.name("Destination URL") +.description("The URL to post the Provenance Events to.") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.URL_VALIDATOR) +.build(); +static final
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237592#comment-15237592 ] ASF GitHub Bot commented on MINIFI-13: -- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59417575 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java --- @@ -0,0 +1,443 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.text.DateFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TimeZone; +import java.util.UUID; +import java.util.concurrent.TimeUnit; + +import javax.json.Json; +import javax.json.JsonArray; +import javax.json.JsonArrayBuilder; +import javax.json.JsonBuilderFactory; +import javax.json.JsonObject; +import javax.json.JsonObjectBuilder; +import javax.net.ssl.SSLContext; + +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.Validator; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.controller.status.PortStatus; +import org.apache.nifi.controller.status.ProcessGroupStatus; +import org.apache.nifi.controller.status.ProcessorStatus; +import org.apache.nifi.controller.status.RemoteProcessGroupStatus; +import org.apache.nifi.events.EventReporter; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.Severity; +import org.apache.nifi.ssl.SSLContextService; +import org.apache.nifi.ssl.SSLContextService.ClientAuth; + +@Tags({"provenance", "lineage", "tracking", "site", "site to site"}) +@CapabilityDescription("Publishes Provenance events using the Site To Site protocol.") +@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's last event Id so that on restart of NiFi the task knows where it left off.") +public class ProvenanceReportingTask extends AbstractReportingTask { +private static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'"; +private static final String LAST_EVENT_ID_KEY = "last_event_id"; + +static final PropertyDescriptor DESTINATION_URL = new PropertyDescriptor.Builder() +.name("Destination URL") +.description("The URL to post the Provenance Events to.") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.URL_VALIDATOR) +.build(); +static final
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237588#comment-15237588 ] ASF GitHub Bot commented on MINIFI-13: -- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59417236 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/java/org/apache/nifi/minifi/provenance/reporting/ProvenanceReportingTask.java --- @@ -0,0 +1,443 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.text.DateFormat; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TimeZone; +import java.util.UUID; +import java.util.concurrent.TimeUnit; + +import javax.json.Json; +import javax.json.JsonArray; +import javax.json.JsonArrayBuilder; +import javax.json.JsonBuilderFactory; +import javax.json.JsonObject; +import javax.json.JsonObjectBuilder; +import javax.net.ssl.SSLContext; + +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.Validator; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.controller.status.PortStatus; +import org.apache.nifi.controller.status.ProcessGroupStatus; +import org.apache.nifi.controller.status.ProcessorStatus; +import org.apache.nifi.controller.status.RemoteProcessGroupStatus; +import org.apache.nifi.events.EventReporter; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.Severity; +import org.apache.nifi.ssl.SSLContextService; +import org.apache.nifi.ssl.SSLContextService.ClientAuth; + +@Tags({"provenance", "lineage", "tracking", "site", "site to site"}) +@CapabilityDescription("Publishes Provenance events using the Site To Site protocol.") +@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's last event Id so that on restart of NiFi the task knows where it left off.") +public class ProvenanceReportingTask extends AbstractReportingTask { +private static final String TIMESTAMP_FORMAT = "-MM-dd'T'HH:mm:ss.SSS'Z'"; +private static final String LAST_EVENT_ID_KEY = "last_event_id"; + +static final PropertyDescriptor DESTINATION_URL = new PropertyDescriptor.Builder() +.name("Destination URL") +.description("The URL to post the Provenance Events to.") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.URL_VALIDATOR) +.build(); +static final
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237563#comment-15237563 ] ASF GitHub Bot commented on MINIFI-13: -- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59414741 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/test/java/org/apache/nifi/minifi/provenance/reporting/TestProvenanceReportingTask.java --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import static org.junit.Assert.assertEquals; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.UUID; + +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.provenance.ProvenanceEventBuilder; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.provenance.ProvenanceEventType; +import org.apache.nifi.provenance.StandardProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.EventAccess; +import org.apache.nifi.reporting.InitializationException; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.ReportingInitializationContext; +import org.apache.nifi.state.MockStateManager; +import org.apache.nifi.util.MockPropertyValue; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.Mockito; +import org.mockito.invocation.InvocationOnMock; +import org.mockito.stubbing.Answer; + +public class TestProvenanceReportingTask { + +@Test +public void testSerializedForm() throws IOException, InitializationException { +final String uuid = "1000----"; +final Map attributes = new HashMap<>(); +attributes.put("abc", "xyz"); +attributes.put("xyz", "abc"); +attributes.put("filename", "file-" + uuid); + +final Map prevAttrs = new HashMap<>(); +attributes.put("filename", "1234.xyz"); + +final Set lineageIdentifiers = new HashSet<>(); +lineageIdentifiers.add("123"); +lineageIdentifiers.add("321"); + +final ProvenanceEventBuilder builder = new StandardProvenanceEventRecord.Builder(); +builder.setEventTime(System.currentTimeMillis()); +builder.setEventType(ProvenanceEventType.RECEIVE); +builder.setTransitUri("nifi://unit-test"); +attributes.put("uuid", uuid); +builder.fromFlowFile(createFlowFile(3L, 3000L, attributes)); +builder.setAttributes(prevAttrs, attributes); +builder.setComponentId("1234"); +builder.setComponentType("dummy processor"); +builder.setLineageIdentifiers(lineageIdentifiers); +final ProvenanceEventRecord event = builder.build(); + +final List dataSent = new ArrayList<>(); +final ProvenanceReportingTask task = new ProvenanceReportingTask() { +@SuppressWarnings("unchecked") +@Override +protected SiteToSiteClient getClient() { +final SiteToSiteClient client = Mockito.mock(SiteToSiteClient.class); +final Transaction transaction = Mockito.mock(Transaction.class); + +try { +Mockito.doAnswer(new Answ
[jira] [Commented] (MINIFI-13) Create a Reporting Task to Send Provenance data
[ https://issues.apache.org/jira/browse/MINIFI-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237555#comment-15237555 ] ASF GitHub Bot commented on MINIFI-13: -- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/6#discussion_r59413942 --- Diff: minifi-nar-bundles/minifi-provenance-reporting-bundle/minifi-provenance-reporting-task/src/main/test/java/org/apache/nifi/minifi/provenance/reporting/TestProvenanceReportingTask.java --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.minifi.provenance.reporting; + +import static org.junit.Assert.assertEquals; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.UUID; + +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.provenance.ProvenanceEventBuilder; +import org.apache.nifi.provenance.ProvenanceEventRecord; +import org.apache.nifi.provenance.ProvenanceEventType; +import org.apache.nifi.provenance.StandardProvenanceEventRecord; +import org.apache.nifi.remote.Transaction; +import org.apache.nifi.remote.TransferDirection; +import org.apache.nifi.remote.client.SiteToSiteClient; +import org.apache.nifi.reporting.EventAccess; +import org.apache.nifi.reporting.InitializationException; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.reporting.ReportingInitializationContext; +import org.apache.nifi.state.MockStateManager; +import org.apache.nifi.util.MockPropertyValue; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.Mockito; +import org.mockito.invocation.InvocationOnMock; +import org.mockito.stubbing.Answer; + +public class TestProvenanceReportingTask { + +@Test +public void testSerializedForm() throws IOException, InitializationException { +final String uuid = "1000----"; +final Map attributes = new HashMap<>(); +attributes.put("abc", "xyz"); +attributes.put("xyz", "abc"); +attributes.put("filename", "file-" + uuid); + +final Map prevAttrs = new HashMap<>(); +attributes.put("filename", "1234.xyz"); + +final Set lineageIdentifiers = new HashSet<>(); +lineageIdentifiers.add("123"); +lineageIdentifiers.add("321"); + +final ProvenanceEventBuilder builder = new StandardProvenanceEventRecord.Builder(); +builder.setEventTime(System.currentTimeMillis()); +builder.setEventType(ProvenanceEventType.RECEIVE); +builder.setTransitUri("nifi://unit-test"); +attributes.put("uuid", uuid); +builder.fromFlowFile(createFlowFile(3L, 3000L, attributes)); +builder.setAttributes(prevAttrs, attributes); +builder.setComponentId("1234"); +builder.setComponentType("dummy processor"); +builder.setLineageIdentifiers(lineageIdentifiers); +final ProvenanceEventRecord event = builder.build(); + +final List dataSent = new ArrayList<>(); +final ProvenanceReportingTask task = new ProvenanceReportingTask() { +@SuppressWarnings("unchecked") +@Override +protected SiteToSiteClient getClient() { +final SiteToSiteClient client = Mockito.mock(SiteToSiteClient.class); +final Transaction transaction = Mockito.mock(Transaction.class); + +try { +Mockito.doAnswer(new Answ
[jira] [Commented] (NIFI-1296) Add capabilities to Kafka NAR to use new Kafka API (0.9)
[ https://issues.apache.org/jira/browse/NIFI-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237463#comment-15237463 ] ASF GitHub Bot commented on NIFI-1296: -- GitHub user olegz opened a pull request: https://github.com/apache/nifi/pull/345 NIFI-1296[REVIEW_ONLY] initial commit of Kafka processors that leverage Kafka 0.9 API This PR is not ready to be merged and is only to initiate incremental review and initial testing of a new pair of Kafka processors that are build using Kafka 0.9 API You can merge this pull request into a Git repository by running: $ git pull https://github.com/olegz/nifi NIFI-1296 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/345.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #345 commit 465f1e049bbc37aaed1864121c6e8bc18d05e8ee Author: Oleg Zhurakousky Date: 2016-04-07T11:15:25Z NIFI-1296 initial commit of Kafka processors that leverage Kafka 0.9 API > Add capabilities to Kafka NAR to use new Kafka API (0.9) > > > Key: NIFI-1296 > URL: https://issues.apache.org/jira/browse/NIFI-1296 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 0.4.0 >Reporter: Oleg Zhurakousky >Assignee: Oleg Zhurakousky > Fix For: 0.7.0 > > > Not sure when can we address this, but the interesting comment in > https://github.com/apache/nifi/pull/143. The usage of new API may introduce > issues with running against older Kafka brokers (e.g., 0.8). Need to > investigate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1756) RuntimeDelegate.class confliction.
Brandon Zachary created NIFI-1756: - Summary: RuntimeDelegate.class confliction. Key: NIFI-1756 URL: https://issues.apache.org/jira/browse/NIFI-1756 Project: Apache NiFi Issue Type: Bug Reporter: Brandon Zachary Hello, I'm creating a controller service that uses a Client object from the javax.ws.rs-api-2.0.1.jar. Whenever, I go to enable my controller service I get a ClassCastException where there is a jsr311-api-1.1.1.jar in the nifi-framework-nar that has a javax.ws.rs.ext.RuntimeDelegate.class that is trying to cast to the RuntimeDelegate that's in my javax.jar. Clearly the two don't mesh as I feel the jsr311 version is out of date. I would prefer to have the client instantiated in the controller service but for now I'm creating it in the processor. Apparently the processor doesn't go through the same nifi-framework calls as controller services do. Can that jar be updated to a more recent implementation of javax.ws.rs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237225#comment-15237225 ] Rob Moran commented on NIFI-1323: - [~jskora], thank you! Reading your comment over again, I think I know what you're saying but it is a little unclear. Are concerned about the redesign now, or you were before but not anymore? Re: the sample sizes. I export these mockups at a higher (than 'screen') resolution, so they appear larger when viewing at 100%. This helps depending on where they are seen - through a projector, for example, which we've done at meetups. To answer you question though, I think you'll notice a slight increase in the overall scale of elements in the redesign, but it should not be anything drastic. The new design also adds in more padding, margin, etc. to kind of reduce clutter and set things up for added (future) functionality. > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1755) Remote Process Group Status
Matt Gilman created NIFI-1755: - Summary: Remote Process Group Status Key: NIFI-1755 URL: https://issues.apache.org/jira/browse/NIFI-1755 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Core UI Reporter: Matt Gilman Priority: Minor Fix For: 1.0.0 If a flow has multiple Remote Process Groups (RPG) pointed at the same target NiFi instance, each RPG will report the total Sent/Received to/from that system. Not the amount that each individual RPG Sent/Received alone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237253#comment-15237253 ] Joe Skora commented on NIFI-1323: - [~rmoran], my concerns are gone. I like zooming in on the samples, so I the sizing makes sense. And I use big screens mostly, so I don't mind a slight size increase, especially if it improves readability. > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1748) Remove potentially copyrighted text from KafkaPublisherTest
[ https://issues.apache.org/jira/browse/NIFI-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237238#comment-15237238 ] ASF subversion and git services commented on NIFI-1748: --- Commit 5ec79e341a03043a607a5f174df433e115b140b4 in nifi's branch refs/heads/support/nifi-0.6.x from [~ozhurakousky] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=5ec79e3 ] NIFI-1748 removed un-used test data This closes #344. Signed-off-by: Joe Skora > Remove potentially copyrighted text from KafkaPublisherTest > --- > > Key: NIFI-1748 > URL: https://issues.apache.org/jira/browse/NIFI-1748 > Project: Apache NiFi > Issue Type: Task >Reporter: Oleg Zhurakousky >Assignee: Oleg Zhurakousky >Priority: Trivial > Fix For: 1.0.0, 0.7.0, 0.6.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1748) Remove potentially copyrighted text from KafkaPublisherTest
[ https://issues.apache.org/jira/browse/NIFI-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-1748. --- Resolution: Fixed merged into 0.6.1 > Remove potentially copyrighted text from KafkaPublisherTest > --- > > Key: NIFI-1748 > URL: https://issues.apache.org/jira/browse/NIFI-1748 > Project: Apache NiFi > Issue Type: Task >Reporter: Oleg Zhurakousky >Assignee: Oleg Zhurakousky >Priority: Trivial > Fix For: 1.0.0, 0.7.0, 0.6.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-1748 removed un-used test data
Repository: nifi Updated Branches: refs/heads/support/nifi-0.6.x abbd21531 -> 5ec79e341 NIFI-1748 removed un-used test data This closes #344. Signed-off-by: Joe Skora Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/5ec79e34 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/5ec79e34 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/5ec79e34 Branch: refs/heads/support/nifi-0.6.x Commit: 5ec79e341a03043a607a5f174df433e115b140b4 Parents: abbd215 Author: Oleg Zhurakousky Authored: Sat Apr 9 17:24:37 2016 -0400 Committer: joewitt Committed: Tue Apr 12 10:07:31 2016 -0400 -- .../processors/kafka/KafkaPublisherTest.java| 44 1 file changed, 7 insertions(+), 37 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/5ec79e34/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/test/java/org/apache/nifi/processors/kafka/KafkaPublisherTest.java -- diff --git a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/test/java/org/apache/nifi/processors/kafka/KafkaPublisherTest.java b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/test/java/org/apache/nifi/processors/kafka/KafkaPublisherTest.java index 3e582a6..46507d2 100644 --- a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/test/java/org/apache/nifi/processors/kafka/KafkaPublisherTest.java +++ b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/test/java/org/apache/nifi/processors/kafka/KafkaPublisherTest.java @@ -43,13 +43,6 @@ import kafka.javaapi.consumer.ConsumerConnector; public class KafkaPublisherTest { -private static final String sampleData = "The true sign of intelligence is not knowledge but imagination.\n" -+ "It's not that I'm so smart, it's just that I stay with problems longer.\n" -+ "The only source of knowledge is experience.\n" -+ "Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.\n"; - -private static final String sampleData2 = "foo|bar|baz"; - private static EmbeddedKafka kafkaLocal; private static EmbeddedKafkaProducerHelper producerHelper; @@ -69,7 +62,7 @@ public class KafkaPublisherTest { @Test public void validateSuccessfulSendAsWhole() throws Exception { -InputStream fis = new ByteArrayInputStream(sampleData.getBytes(StandardCharsets.UTF_8)); +InputStream fis = new ByteArrayInputStream("Hello Kafka".getBytes(StandardCharsets.UTF_8)); String topicName = "validateSuccessfulSendAsWhole"; Properties kafkaProperties = this.buildProducerProperties(); @@ -93,7 +86,8 @@ public class KafkaPublisherTest { @Test public void validateSuccessfulSendAsDelimited() throws Exception { -InputStream fis = new ByteArrayInputStream(sampleData.getBytes(StandardCharsets.UTF_8)); +InputStream fis = new ByteArrayInputStream( +"Hello Kafka 1\nHello Kafka 2\nHello Kafka 3\nHello Kafka 4\n".getBytes(StandardCharsets.UTF_8)); String topicName = "validateSuccessfulSendAsDelimited"; Properties kafkaProperties = this.buildProducerProperties(); @@ -118,33 +112,9 @@ public class KafkaPublisherTest { } @Test -public void validateSuccessfulSendAsDelimited2() throws Exception { -InputStream fis = new ByteArrayInputStream(sampleData2.getBytes(StandardCharsets.UTF_8)); -String topicName = "validateSuccessfulSendAsDelimited2"; - -Properties kafkaProperties = this.buildProducerProperties(); -KafkaPublisher publisher = new KafkaPublisher(kafkaProperties); - -SplittableMessageContext messageContext = new SplittableMessageContext(topicName, null, "|".getBytes(StandardCharsets.UTF_8)); - -publisher.publish(messageContext, fis, null, 2000); -publisher.close(); - -ConsumerIterator iter = this.buildConsumer(topicName); -assertNotNull(iter.next()); -assertNotNull(iter.next()); -assertNotNull(iter.next()); -try { -iter.next(); -fail(); -} catch (ConsumerTimeoutException e) { -// that's OK since this is the Kafka mechanism to unblock -} -} - -@Test public void validateSuccessfulReSendOfFailedSegments() throws Exception { -InputStream fis = new ByteArrayInputStream(sampleData.getBytes(StandardCharsets.UTF_8)); +InputStream fis = new ByteArrayInputStream( +"Hello Kafka 1\nHello Kafka 2\nHello Kafka 3\nHello Kafka 4\n".getBytes(StandardCharsets.UTF_8)); String topicName = "validateSuccessfulReSendOfFailedSegments"; Properties kafkaProperties = this.buildProd
[jira] [Commented] (NIFI-1752) EvaluateXPath invalid when starting processor
[ https://issues.apache.org/jira/browse/NIFI-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236732#comment-15236732 ] Guillaume Pool commented on NIFI-1752: -- My environment is CentOS with OpenJDK openjdk version "1.8.0_71" OpenJDK Runtime Environment (build 1.8.0_71-b15) OpenJDK 64-Bit Server VM (build 25.71-b15, mixed mode) > EvaluateXPath invalid when starting processor > - > > Key: NIFI-1752 > URL: https://issues.apache.org/jira/browse/NIFI-1752 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.0 >Reporter: Joseph Witt > > Guillaume Pool: > Hi, > > If anyone could help I would appreciate it: > > After upgrading to 0.6.0 all my EvaluateXMLPath processors report as invalid > with nothing specific reported as the error. > > I use flowfile attribute as the destination, xpath query similar to > /message/address > > Return type auto-detect / string > > Error: > > 2016-04-11 17:37:12,465 INFO [NiFi Web Server-64629] > org.apache.nifi.web.filter.RequestLogger Attempting request for (anonymous) > PUT > http://sv-hdpdf1.hdp.supergrp.net:8443/nifi-api/controller/process-groups/d02a0bbc-c142-37ba-adaa-ee191b98790d/processors/e49acf40-76d1-3b6e-a67d-dc40153605ad > (source ip: 172.27.0.97) > 2016-04-11 17:37:12,474 INFO [NiFi Web Server-64629] > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: > EvaluateXPath[id=e49acf40-76d1-3b6e-a67d-dc40153605ad] is not in a valid > state. Returning Conflict response. > > Reports this even if you drop a new processor in. > James Wing: > I am able to see the same EvaluateXPath issue Guillame Pool reported on NiFi > 0.6.0. I created a template gist at > https://gist.github.com/jvwing/7e7948d8eb5ad3643f38966b4ba3ce2e that > illustrates this for me. You don't even need to run the whole thing, just > try to start the EvaluateXPath processor. I see a pop-up error trying to > start it "EvaluateXPath[id=b481c570-686a-4b6e-859c-9ebe27398fe1] is not in a > valid state". > Trying to troubleshoot this, I attached a debugger and found an > IllegalStateException thrown by StandardProcessorNode.verifyCanStart() on > line 1141. The error text was: > 'address' validated against '/message/address' is invalid because Unable to > initialize XPath engine due to > javax.xml.xpath.XPathFactoryConfigurationException: No XPathFactory > implementation found for the object model: http://saxon.sf.net/jaxp/xpath/om > I haven't figured out what change is the root cause, but the trail seems to > lead into validating the dynamic XPath attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)